All right, so how do we get to making a prediction or a forecast? We have to provide 1
two things - 2
the estimated property price and the range that goes with the price. 3
Because this is what the pros do and will also provide an estimate of what the home is worth plus a 4
degree of uncertainty around that number. 5
The estimated price is easy enough, right? 6
Because once we have the theta parameters for the model, all we have to do is plug them in and then together 7
with the values for the individual features, we get an estimate, we get our y_hat. 8
But what about the range? 9
What does that come from? 10
The range will actually depend on the shape of the distribution that you're working with. 11
If we know the distribution we can estimate the range very accurately. 12
And here's the thing. 13
Our go to distribution is usually the normal distribution, because the very, very nice thing about the 14
normal distribution is that we know its shape. 15
We know that for a normal distribution, 68% of the observations are between these two points. 16
68% of all the values in this distribution are within this purple range right here. 17
And for normal distribution, we also know that around 95 percent of the values are between these two 18
points here. 19
95% of the observations fall within this pink range that I've highlighted right here. 20
The individual points that I've drawn on this histogram right here actually have a name. 21
They quantify the amount of variation around the mean. 22
The mean is right here in the middle of the distribution and the distance from the mean to that bright 23
purple point is called one standard deviation. For a normal distribution, 24
you'll usually see the Greek letter Sigma used to denote the standard deviation. 25
Now what about the other points that I had drawn on here earlier? 26
Well the left purple point is at minus one standard deviation - minus one standard deviation from the 27
mean and for our normal distribution, as we said before, around 68 percent of all the observations lie between 28
minus and plus one standard deviation. 29
OK. 30
What about the pink points? 31
Well the right pink point is at two standard deviations and the left pink point is at minus two standard 32
deviations. 33
And as we've said before, approximately 95% of observations lie between plus two and 34
minus two standard deviations for a normal distribution. 35
Now let me ask you this. 36
If this green point here right in the middle is our estimate for the property price from our model, our 37
y_hat, what is the distribution that we're gonna be looking at here? 38
Any guess? What's the distribution that tells us something about the variance in our price estimates? Well, 39
we're going to be coming full circle here. 40
It's actually the distribution of the residuals from our regression. 41
This is the reason why we cared so much about whether the distribution is a normal distribution or not. 42
Now the next question you might ask at this point is: Well, 43
ok, 44
so if the distribution is the distribution of the residuals, then what do the purple and pink dots represent? 45
How do we get our range? 46
Do you remember our mean squared error? 47
And no, the mean square is not the purple dot, but we can make one small modification to the mean squared 48
error and get something very, very handy for calculating the range and making predictions. 49
And that small modification is by taking the square root of this thing, by taking the square root of 50
the mean squared error, 51
we get another metric and this one is called, yes surprise, surprise, 52
the Root Mean Square Error or RMSE and it's this metric, the RMSE that has a really, really nice interpretation 53
because the Root Mean Squared Error represents one standard deviation of the differences between our 54
actual and our predicted values. 55
The Root Mean Squared Error is one standard deviation in the distribution of our residuals. 56
So let's look at the chart again. To create our range around our estimated price, our so-called prediction 57
interval, 58
the first thing we choose is how wide we want that interval to be, say we want to cover around 95 percent 59
of the observations. 60
Then we would use two standard deviations either side. 61
And this means taking our prediction and adding two times the Root Mean Squared Error to it for the 62
upper bound and subtracting two times the Root Mean Squared Error for our lower bound on our prediction 63
and that's how we get the range. In our Jupyter notebook, 64
this means simply taking the square root of our mean squared error that we've already calculated. Coming 65
to the cell where we've got our dataframe with our Mean Squared Error, 66
what I'm going to do is add another column, so I'll put a comma here, go to the next line, single quotes 67
and put RMSE here and then a colon, 68
and now I can take this entire list here, 69
copy it, 70
and then here, what I'm going to do is I'm going to to use numpy and call the square root function from numpy 71
and as an argument I'm going to pass in the list that I just copied and what this will do is it'll take 72
the square root of all the items in the list. 73
Let me refresh the cell. 74
Here we go. 75
Now that we've done that I want to give you a challenge. 76
Suppose we have an estimate from our model, 77
yeah, for a house price of 30000 dollars. 78
Can you calculate the upper and lower bound for a 95 percent prediction interval using the reduced log 79
model? 80
In other words, can you calculate the upper and the lower bound for the range around this estimate? 81
I'll give you a few seconds to pause the video before I show you the solution. 82
All right, 83
let's take it from the top. 84
So I'm going to add a print statement and I'm going to spell it out, 85
so I'm going to say "1 s.d. in log prices", because we've got units, is, and that will be the 86
square root of our log mean squared error, 87
so "np.sqrt(reduced_log_mse)". Agreed? What's that equal to? It's equal to this much. 88
Now what about two standard deviations? 89
I can calculate that simply by taking the first print statement, copying it and then multiplying this 90
whole thing by two. 91
Here we go. 92
This is two standard deviations - the upper bound for the prediction interval will be equal to our y_hat 93
plus two times the root mean squared error. 94
Now I've been pretty sneaky and I've given you the y_hat in dollar values. 95
So you actually have to use a log transformation "np.log(30)", since our model is given in thousands 96
and then you have to add "2*np.sqrt(reduced_log_mse)". So this 97
is our y_hat plus two times the root mean squared error. Let me print that out, so I'm going to say "print( 98
"'The upper bound for a 95% prediction interval is', upper_bound". That's 99
equal to 3.78 approximately. Now if we wanted to see this in dollar values, then we 100
can say, well "The upper bound for log prices for a 95% prediction interval is" this much and I 101
can copy this, paste it in and I can say "'The upper bound in normal prices is', np.e** 102
upper_bound." 103
So let's see what that is. 104
I can make that more explicit by multiplying this whole thing, times 1000, 105
putting a little dollar sign here and there I've got my upper bound. The lower bound is very, very similar. 106
You can even copy these three lines, paste them in and change this to "lower_bound = np.log( 107
30)" minus two times the root mean squared error and then I can change my print statements to read "'The lower 108
bound in log prices for a 95% prediction interval is', lower_bound" and the lower bound 109
in normal prices is "np.e**lower_bound * 1000". 110
Let's see what this reads. 111
The lower bound in this case is 20635 dollars. 112
Now the trick with this challenge is to do the addition of the root mean squared error and the transformation 113
in the right order, because otherwise you'll get a very, very different result. 114
The incorrect way of calculating the upper bound would have been to take two times the root mean squared 115
error and simply say "Well okay, we've got a estimate of thirty thousand and then we're going to add to 116
it the transformed value 'np.e**'", and then the root mean squared error 117
times 1000. 118
So this was the little trick in how I phrased this challenge. In summary, we often look towards the root 119
mean squared error when we're interested in the predictive power of our models, and to some extent we 120
can use the root mean squared error 121
to also compare the models. And the reason is is that the root mean squared is a very good measure of 122
how accurately the model predicts the target, because we can determine a range, the width of this range 123
is a very important criterion 124
if the main purpose of the model is prediction. And this is a big contrast to something like R-Squared 125
because R-Squared says absolutely nothing about the predictive power of the model or the prediction 126
error. 127
Okay. 128
So we're slowly coming towards the end of the section. In the next lesson, 129
we're gonna finish it up by building a valuation tool for our boss in our real estate office and that's 130
going to involve probably updating the prices to reflect today's dollar values a little bit. 131
I think the Boston upper price of fifty thousand dollars is not really that accurate anymore. 132
And we're also going to be looking at how we can create a Python function with optional arguments. 133
So arguments where there's already default values set similar to what we've seen with seaborn and 134
matplotlib and we're going to cover the syntax, the Python syntax how we can create functions like this 135
ourselves. I'll see you in the next lessons. Take care.