Since we've built a model which is able to make predictions we've done some on the test data set.
We've exported them.
Now the people you share these predictions with or in fact yourself I know in my case I'm very very
interested might be curious of which parts of the data led to these predictions.
So this is where feature importance comes in feature importance.
In other words it seeks to figure out which different attributes of the data were most important when
it comes to predicting the target variable or in our case the sale.
Let's write that down a feature important seeks to figure out which different attributes of the data
were most important when it comes to predicting.
Let's put this in bold target variable.
So in our case it's the sale price.
All right.
So how might we do this.
We know we're using a random forest regress.
We've seen in a previous video in the classification project as well as the socket Line project some
psychic line models have the attribute feature responses.
But if we didn't know if we're using a model that we weren't familiar with maybe we could go something
like this so random forest regress are feature important
beautiful so maybe this will give us some results if we were to go and explore these.
We've got the psychic loan documentation that's always helpful so we're looking in here.
We might search for where it knew what we want feature importance is return the feature importance is
the higher the more important the feature.
Let's see what that actually does.
So let's take our model.
So find feature importance of our best model.
So we'll take our ideal model and we'll find a feature in IS.
Well now that returns a fairly large array whole bunch of different values here some of them are zero.
Some of them are pretty low.
Tend to the negative six there.
But what does this have in common.
Let's check out the length of this.
What does this have in common with how training dataset
hundred to one hundred and two.
Are we mining fur is that we've got 102 columns here now we've got 102 values here so we're getting
a value for every feature.
So sales I'd say would map to this and machine day would map to this.
Now we could make a dictionary by going ex trained up columns and intertwine these values but I don't
know about you.
I prefer to see things visually.
So let's make a what do we make a helper function which helps us do that.
So we might make a helper function help a function for plotting feature importance so we can see it
visually.
So go to here death plot features and we might take a list of columns and a list of importance is and
we're going to set an end to 20.
Now this will make sense in a second because we have 102 values here.
But realistically we want to look at a plot that we only want the top 20 values and I mean that's why
we set it so we can adjust it with this little function here helper functions are great.
I should have started using functions earlier writing functions saves a lot of time put it that way.
I used to just write the same line of code over and over and over again in different cells but I've
started to get into the habit of writing functions for different uses.
So what we might do is use a little panties trick here called chaining and what that means is just simply
putting a number of different Panda's functions in brackets and we'll see that in a second because we're
going to make a data frame called The F and it'll have two columns features can be columns and feature
important things can be important says right.
So we're just creating a data frame here and then we're going to call sort value so see here these are
still within the brackets needs to hear what this means is is going to do PDA or data frame and Dot
sort values in one hit and we want to sort it by feature in performances and we want ascending equal
to False so we want it to go from highest to lowest yet.
That makes sense and then we want to reset the index drop equals true right actually this bracket should
be down here.
So because you've got a bracket here and here and two dots that means it's just going to do these three
panda steps in one hit.
And now we need to plot so plot the diaphragm we've created.
So what we might do is go fig ax instantiate a plot peyote dot subplots and then we might do a horizontal
bar because in my experience plotting features it looks really good on a horizontal bar.
And now we only want up to n.
That's why we have n there.
So that's gonna be the first 20 examples.
And the same with feature important says we only want up to 20 so the first 20 then we're going to set
the Y label so acts dot set y label to features because it's a horizontal bar X and Y are rearranged
and we go ax dot set X label add some communication to it and put feature importance down the bottom.
Let's see what this looks like actually.
So we're gonna help a function we just run it there or we have to do to call this is go plot features
and then we'll pass it column.
So we just want the columns from X train because that's a variable there.
And then the feature importance is are just the ideal model dot feature importance is let's see what
this looks like okay.
So we got the features on the left here.
Their value here.
I want the top one the most valuable feature to be at the top.
So what we might do is ax darts.
How do we do it invert.
Can we invert.
I think it's X don't invert y axis.
Maybe that's not a function.
If in doubt run the code invert y axis and that didn't work.
Maybe it is a function bomb.
If in doubt run the code.
Trust your instincts right.
That's how you learn things you practice by running it without looking up.
If in doubt you can always look it up right.
You can always ask how to invert our horizontal bar plot map plot lib.
Google should be out to help.
All right.
So what can we infer from this.
Says a fair bit going on.
We've got about 20 different features here so it's saying that year nine.
Let's have a look at our X train go ahead.
Let's get that in here.
So saying that a year that the bulldozer was made is the most important feature based on the ideal model.
And in the product size what is that.
Let's look at our data dictionary product size product size.
Don't know what this is.
OK that's great.
The size grouping for a product group subsets with product group.
This is an example of where you maybe need to do some more research and figure out what features are
what if you've got something like this in your data dictionary might have to reach out to like a subject
matter expert or we could just go X train
product size
maybe we check the value counts
okay.
So there's one two three four five six different sizes.
Now zero is gonna be missing a lot of missing values there.
All right but maybe this doesn't really tell us much.
So maybe we could do it on our original data frame.
There we go.
All right.
So the product size or that kind of makes sense that the product size is influencing the sale price
sale year enclosure.
Also doing a fair bit of damage there enclosure value counts.
Okay.
So that's.
Mm hmm.
That doesn't really make much sense to me so then we go back to the data dictionary and we find and
closure machine configuration does machine have an enclosed cab or not.
All right.
So maybe we'd need a need to figure out what these different values might mean.
But this is a kind of exploration you'd probably do towards the end of after you've built a model ride
is bringing this to someone after you've made some predictions you might take this in a sort of a presentation.
So maybe you're meeting with a client or meeting with another team member or something you're going
Hey this is what the most important features our model has found.
Does this agree with your sort of intuition with what someone already knows about the data.
Does this make sense if a model is using these features to derive its predictions.
Does this make sense or this information here might influence how you go about collecting data in the
future on your bulldozer sales so you might put a bit more effort into the values here that are contributing
most to predicting the sale price of a bulldozer in the future.
The last probably question that I'll leave you with is we've kind of just glazed over it a little bit
but I want you to do a little bit of research is why my knowing the feature importance is of a train
model be helpful.
That's a finishing question.
So question to finish is going to involve some research.
Why might knowing the feature importance is of a trained machine learning model be helpful.
So that's gonna finish off this project to finish off with a little question here what you might want
to try and do is if you've followed along with all of these steps here you might want to see how far
you can you can go with hyper parameter tuning.
So something like this maybe you could leave this runway for a while on your computer see what it finds
out see if you can improve the the valid R and S L E score and see how far you'd get up on the leaderboard.
Now if you do reach a point where you're not really improving with a random forest model what a final
extension may also be is final challenge is what other machine learning models.
Could you try on our dataset and what that might involve is something like searching for the psychic
loan machine learning that so what we've done is we've gone through this and we've gone to regression
problem we found ensemble regresses and we've used a random forest maybe you want to try another one
of these regression models using the format of data that we've worked through in this notebook so I'll
leave a little hint here so hint
check out the regression section of this map or try to look at something like cat boost dot II or x
g boost dot II.
So these are two extra resources so extra curricular of course they're optional at this point because
once we've started to work through these sort of projects now we've gone end to end on a full machine
learning project your next steps are kind of trying to figure things out on your own taking what you've
learnt from here and then expanding on that.
That's the kind of information that's a kind of knowledge that that is really going to help out.
Right so rather than sort of always going through projects like this it's doing some research and trying
out new things remember step six of our little framework here.
Experimentation that's the challenge I leave to you.
But that being said if you have made it this far you have gone through the entire project.
Congratulations.
We've just gone end to end on a regression problem using machine learning how cool is that so have a
look in the notebook see what improvements you can make if you have any questions whatsoever.
Feel free to ask them.
Leave them in the discord hat or in the Udemy interface somewhere where you can leave a question and
we'll see what we can do from there.
But congratulations.
Working through a first end end regression project how exciting.
All the best.