All righty then.
So now we've covered some classification metrics and by the way I've just put this little paragraph
here so you can check it out of when to use different classification metrics just to kind of break it
down in five dot points for you.
But now we're going to cover some regression model evaluation metrics and put your hand up if you're
ready.
I'm holding my hand up too.
Don't worry.
All right.
So let's put in a little hitting for point to point to regression model evaluation metrics so as always
the documentation is ready to go.
So we've got this here.
I might put this in here.
Another link here model evaluation metrics documentation.
Now I don't want you to be scared like read in the documentation like I was when I first started.
If you read this and there might be a few times we go through it and you go wow you see something like
this you see a bunch of different words here that you don't really understand.
You look at a bunch of different examples you're reading this and it's like kind of confusing.
Don't worry.
That's exactly how I started but after a little bit of practice after implementing the code like we're
about to do.
That's why I'm such a big fan of implementing code.
I started to understand a little bit more started to understand about what I needed to use.
So speaking of what we need to use for regression we're going to look at three different ones.
Now these are three of the most common three of the most useful.
The first one is r squared pronounced r squared or coefficient of determination beautiful.
And the second one is main absolute area which is also known as M.I.T. and the third one is main squared
error which is also known as MSE.
Wonderful.
So now we've got that.
Let's bring back our regression model.
So from S.K. learn dot ensemble
import random forest regress.
Wonderful.
We'll set up a random seed.
So our results are reproducible.
Then we'll get our X which is the feature variables from our Boston data frame.
So we want to just drop the target and access equals 1.
Labels are the target column or is the target column.
Wonderful.
And now we'll split our data into train test
using and test split passing it X and Y
wonderful.
And then what we're going to do is instantiate our random forest regress.
Because we want to build a regression model so we can evaluate it.
Wonderful model dot fit X train y train.
Now this is going to give us a warning because we haven't set a number of estimates to be in 100 and
we're going to run this invalid syntax classic from now on.
Important.
Let get.
Oh Boston IDF is not defined because they've got typos.
There we go.
See this what happens right when you're writing code don't expect to get it right the first time you're
gonna get errors.
So always remember if in doubt run the code and then go back and fix the errors when you need to use
dot score we've seen this line test wonderful.
And so now ask Where can be calculated using.
Well actually when my getting asked Where from.
So this is remember regression metric number one and put it here involved so we know that we're looking
at ask grand so that is the default metric here.
Right.
And so the coefficient of determination.
R squared of the prediction.
Okay so if we wanted to figure out what exactly are squared was where we do that we go to Wikipedia
in statistics the coefficient of determination denoted r squared or square and pronounce r squared is
a proportion of the variance in the dependent variable that is predictable from independent variables.
There's a pretty complicated definitions.
Well after doing some research I created one of my own right.
This is what I'd like you to do right if you ever come across something and you see the formal definitions
of them.
When you look at them from first glance seem like this is kind of confusing is to go in and search and
find a way to explain things in your own words.
So in my own words and r squared value compares your models predictions here.
What squared does compares your model's predictions to the main of the target.
This is the values of our squared can range from negative infinity.
So that's the lowest possible.
That's a very poor model.
2 1.
Now this is where I love having example right for example.
If all your model does is predict the main of the targets it's a square value would be zero.
And if your model perfectly predicts a range of numbers it's a square value would be 1.
If ever you come across something you don't understand research you look at different sources go to
Wikipedia up here read the documentation right see an example but then most importantly implement it
yourself so I want you not to take my word for it here.
For example if all your model does is predict the mean of the targets it's ask what value would be zero
and if you model perfectly predicts a range of numbers it's ask when value would be 1.
So as I said don't take my word for it but let's see this in action.
So from SBA loan metrics import this is another way to calculate ask where it is.
You could just go to score from there.
So I get loan metrics.
Then we go here and we want to fill an array with why test main.
So the main values from the Y test dataset so we can do that pretty easily with number y.
So why test main equals NDP full so meaningful array of Len.
Why test.
So we want it to be the same length of Y test and then we're gonna fill it with Y.
Tests don't mean.
So does that make sense.
Beautiful.
So if we look at why tests don't mean
all this array is is an array full of those values well mobile will move on and so remember what my
example said.
If all your model does is predict the mean of the target it's r squared value would be zero.
Okay well let's test this out because that's what we do where we're engineers we test things out to
score.
Why test.
We're going to compare the true labels all our model did was predict the mean.
So very simple model 0 and gets an R2 score of 0.
Well okay.
And now the second part of that example was and if your model perfectly predicts a range of numbers
it's r squared value would be 1.
Okay.
So if our model got the exact same predictions as the test values it's r squared value would be 1.
So let's see then we can do this on pretty easily.
Why test.
Now I forgot the exact same predictions if it predicted the Y test labels perfectly it would end up
with a score of 1.
So what does this tell us.
Well this gives us a quick indication.
This score function which implements the coefficient of determination a.k.a. the r squared gives us
a quick insight into how closely our model's predictions are to perfect predictions.
So of course 1.0 is perfect and so we've got 1.0.
We saw that here and if it was predicting nothing but just the mean it would get zero.
And it says here that the value can range from negative infinity.
Well what this means is that if our model predicted completely off the radar this value can actually
go negative.
So the main is actually an okay prediction compared to something that was just predicting all zeros
right now.
Now we've seen r squared it kind of give us quick insight into how well our model may be doing but it
doesn't really tell us how far off each prediction is.
So to do that we're going to use mean absolute error.
So we'll look at that in the next video.