In the upcoming lessons we're going to talk about how to make predictions using our model. 1
And believe it or not, this gives us a chance to link together several concepts of the past lessons. In 2
our technical section on gradient descent 3
we've talked extensively about cost functions and the role of the mean squared error cost function 4
in machine learning. 5
We've also covered how an algorithm that finds the best fit line minimizes this particular equation. 6
And we've also talked a lot about metrics, like r-squared, and we touched on how to compare different 7
models. 8
And finally, we've talked a lot about how to analyze the residuals. 9
So how does this all tie together? 10
Let's go back to the equation of the mean squared error for a little bit, y_hat in this equation is the value 11
that is estimated by our model. Since we're using a multi variable regression with 11 features, our equation 12
for y_hat will actually break down into something like this. 13
We have all the thetas and then all the features. Taking it a step further, 14
we've also encountered the "y - y_hat" before. "y - y_ hat" were the residuals, the residuals are 15
the actual values minus the fitted values. 16
So what does that make the mean squared error? The mean squared error is just the residuals squared and 17
then summed up and then divided by the number of observations. And now do you see the squaring that's 18
happening here? 19
We're squaring the differences between the actual and the predicted values - we're squaring the residuals. 20
And this is nice because we're treating the positive and the negative residuals equally, which means 21
we can then add them up in the summation, but this mathematical operation of squaring the differences 22
also has another effect. 23
Think about this. 24
What if the difference between the predicted value and the actual value was large? 25
What if we have a large residual? What happens to the mean squared error? In this case, the squaring of 26
a big number will increase the mean squared error massively. 27
Right? 28
In other words, the mean squared error weighs the large differences more heavily and thus it punishes large 29
residuals. The effect of this is is that it makes the mean squared error very sensitive to outliers. 30
So that's one thing to be aware of. 31
Now let's have a think about how the main squared error compares to r-squared, and to do that we're 32
going to check out the differences between the two in Jupyter notebook. 33
In the past lessons, we've written some code where we plotted our residuals and calculated them and compared 34
the residual plots of three different models. 35
Now let's look at these models' r-squared and their mean squared errors. In some of the previous lessons, 36
the way we've worked with the mean squared error was by importing a function from scikit-learn. 37
So we said "from sklearn.metrics import mean_squared_error" 38
and then we used this function to calculate the MSE. 39
But in this case, we're using our statsmodel module here and it actually makes accessing the mean squared 40
error very, very easy because again we just have to use our results object to get hold of it. 41
Here's how. 42
Coming down here, I'm going to add a comment that reads "Mean Squared Error". 43
And I can access the mean squared error from our results objects by typing "results.mse_ 44
resid". 45
This will give us our mean squared error. 46
And what I'll do is I'll store this in a variable, I'm going to call this variable or "reduced_ 47
log_ 48
mse", and set that equal to the rounded value of this whole thing, 49
so 50
I'll add a round function and have the "results.mse_resid" inside the parentheses, put a comma 51
after it, round it to three decimal places and close it all off. 52
I'm going to do a same thing for the r-squared. 53
So in our comment I'll add "R-squared" and 54
I'm going to copy this line, paste it down here, 55
change the variable name to r-squared and change the attribute as well. 56
So "results.rsquared", rounded to three decimal places will be stored in this variable "reduced 57
_log_ 58
rsquared = round(results.rsquared, 3)". I'm going to 59
take this here, 60
these three lines of code and the comment, copy them and I'm going to come down here to model number 61
2 - 62
the original model using normal prices and all features. And what I'll do here is 63
I'm going to come down to the very bottom, paste it in and change the variable names, 64
the first one I'll call "full_normal_mse" 65
and the second one I'm going to call "full_normal_ 66
rsquared". 67
Now, I'm going to hit Shift+Enter on both of these cells since I've added new code, I think I didn't do it on 68
the first one, but I'm going to do it on this one, Shift+Enter and then coming up here, I'm going to hit Shift+Enter 69
as well. 70
This leaves us with our third model, the one where we omitted some variables and are still using log 71
prices. 72
And I'm going to paste in the three lines, the comment reading "Mean Squared Error and R-squared" 73
and then I'm going to change the variable names, 74
once again. I'm going to call this one "omitted_var_ 75
mse" and "omitted_var_rqsuared". I'm going to hit Shift+ 76
Enter, and now that we've added this code, calculated all these values, we can look at them side by side, 77
which is what I wanted to do in the first place. 78
So I put him in a data frame "pd.DataFrame({'R-Squared':" 79
and now I'm going to provide a list - a list of all the r-squared values. 80
So the first one was "reduced_log_rquared", the second one was "full_ 81
normal_rsquared", and the third one was "omitted_var_rsquared". 82
Those are all the r-squared values. 83
Now I'm going to put a comma here after the square brackets, go down to the next line and add our second key for 84
our dictionary, it's gonna be "mse", mean squared error, colon and again we're gonna be putting in a list of 85
values, so I can probably take this list here, 86
copy it, paste it in and then change that 87
last thing here in the name to 88
mse. This is how I've named all my mean squared error variables. So that's easy enough, 89
let's take a look at what we've got. I'm going to hit Shift+Enter and what we see is two columns, one reading "MSE" 90
and the other one reading "R-Squared". Now the rows are still indexed by 0, 1 and 2, but we can change that 91
by adding an additional argument here. 92
So I'm going to put a come up here and then I'm going to supply the index argument, and I'm going to make this 93
equal to, again, a list, 94
so square brackets, and I'm going put three values, the first one it's gonna be our "Reduced Log Model", 95
the second one is going to be our "Full Normal Price Model" and then the third one is going to be our 96
"Omitted 97
Var Model". Refreshing our output, 98
we see the names listed here very nicely. Okay, 99
so let's interpret what we're looking at. 100
First off the R-squared - 101
we know that R-squared is always between 0 and 1 for every single regression model out there. 102
But what does this mean? 103
R-squared is a relative measure of fit and R-Squared does not have any units, right? 104
This means that the R-Squared doesn't scale with the data, it's always between 0 and 1. 105
And in our table we can see that the simplified model with the log prices has a higher fit than our 106
full model with the normal prices or 107
of course, the model where we left out some variables. 108
This one in fact has the lowest fit of them all. 109
But what about the mean squared error? 110
In contrast to R-Squared, the mean squared error actually is an absolute measure of fit. 111
It's not a relative measure, it's an absolute measure and it has units, namely the units of the target, 112
our y variable, that 19.9 that you see here for the full normal price model, 113
yeah, 114
the second one, is in the units of the target which in this case is thousands of dollars. 115
So this mean squared error here is actually approximately twenty thousand dollars. 116
Is this third one better than the second one because it has a lower mean squared error? 117
And the answer is no, 118
right? 119
Because you can't compare the two because they're in different units. The model in the first row and 120
the model in the third row are using log prices, right? 121
Log prices in thousands. 122
In other words the scale of the MSE is different depending on the data that you're using. 123
But of course lower values of MSE indicate a better fit and an MSE of 0 indicates a perfect fit. 124
So you have to remember that when you're comparing models. Okay, so now that we've calculated our mean 125
squared error, let's talk about how we would go about making a prediction for a house price. 126
Now I think that before we go ahead and do that we should actually check out how the pros predict house 127
prices. Two websites that are very good at this are Zoopla in the UK and Zillow in the US, but of course 128
you can go and hunt around on the Internet to find a equivalent website 129
that's a bit closer to home depending on where you live, and you can either maybe like pause the video 130
and try out one of these websites to get an estimate for a home 131
or you can watch me go through the process. 132
One word of warning though, Zillow thinks they're being very cute or clever and calling their estimate 133
a Zestimate, 134
mm hmm, 135
yeah, 136
so, just something to note. So how did that go? 137
Did you give it a shot? 138
So yesterday I actually tried using Zoopla to get a price estimate for Buckingham Palace or the Big 139
Ben, but I didn't have much luck providing the postcodes for these landblocks, 140
but funny enough there were some properties listed in Windsor Castle of all places, including like the 141
Windsor Castle library. 142
So yeah I'm really not sure why, 143
but I tell you what, I did dig around and I want to show you how to get a property price estimate for 144
this particular home. 145
Let me show you the brochure. 146
So this is an old brochure and it's for this house in Kenwood. 147
And as you can see it's a it's a very, very nice house. 148
I think the agent Knight Frank was looking to sell it and the story with this house is that it used 149
to belong to John Lennon from the Beatles. 150
That's right. 151
This John Lennon. 152
Contrary to popular belief, he did not live on Abbey Road nor did he live in a yellow submarine. 153
OK. 154
So let me show you how to get an estimate for his previous crib. 155
Now to get a valuation for this home on Zoopla, what I have to do is I have to go to "Get your home 156
valued" and then I think it's like come down here and I think I have to click on "What's my home worth", 157
so I think the first place they send you is trying to get Agent valuations, but I'm actually interested 158
in the Zoopla estimate, so how much Zoopla thinks it's worth. 159
So I'm going to click on this get, a Zoopla estimate and then I'm going to punch in the postcode, 160
so this was KT13 0JU. I looked this up. 161
So John Lennon used to live on this postcode and from that property brochure we know that it's 162
Kenwood, Wood Lane. 163
So let's click on "Get estimate". Now we can supply some additional information about this property, the 164
"Property type" is it's a "Detached house"; the "Property style" is that it's a "Period" home and it's a "Freehold". 165
I don't know how many floors it has, but I did check the brochure and for bedrooms it was six bedrooms, 166
six bathrooms and six receptions. 167
I think they also had like a big garden and a swimming pool and for the internal area according to the 168
brochure it's a 1110 square metres, 169
so absolutely enormous. Now I can press "Continue" and then I'm going to tick "Swimming pool" here and I'm gonna leave 170
these other ones alone, 171
I can't be bothered. But what's very interesting is that Zoopla knows how much this home sold for previously. 172
This is actually public knowledge, right? 173
If you're wondering where they're getting this from, they're getting it from the UK Government's land 174
registry. 175
So the government basically tracks all the property transactions in the UK and you can search for house 176
prices of the previous transactions. 177
And if I put in the postcode here, 178
so KT13 0JU and then say "Show results", then I can see here that for Kenwood, Wood Lane in 2007 179
this home sold for 5.8 million pounds. 180
Yeah. 181
So this is where Zoopla is getting its price data from. 182
Yeah. 183
They're getting it directly from the government. 184
So that's one of the data sources. Now I'm going to confirm the information above is accurate and I agree 185
to the terms of use and I'm going to say "Get estimate" 186
and what we see is that Zoopla estimates John Lennon's former home to be worth 187
8.75 million pounds. 188
So it's sold for 5.8 189
back in 07 and today Zoopla reckons it's worth about 8.75 190
But here's the part I want to draw your attention to. 191
I want to draw your attention to this value range that they're providing here. 192
So we don't just get a price, 193
yeah. 194
Zoopla isn't telling us "Oh it's worth 8.75 million on the nose", 195
what we also get from them is a range around that price. 196
We see that Zoopla thinks the property is worth between 8.16 and 9.34 197
million pounds. 198
What Zoopla actually telling us is that they are estimating this home to be worth 8.75 199
plus or minus 590000 pounds. 200
They're giving us an upper and a lower bound to their estimate. 201
And this is a key component for making a good prediction. 202
Why? Well, 203
because it tells us how confident Zoopla is in their estimate. 204
I mean they're giving us this little bar here "Confidence level: high" but we can also infer how confident 205
they are by how wide this range is. In other words, 206
if Zoopla says "Oh it's 8.7 milion plus or minus 600000", that's very, 207
very different from saying "Oh it's worth 8.7 million plus or minus 2 million", 208
right? 209
The point I'm trying to make is that, when it comes to our predictions, our forecasting that we're going 210
to be doing, we're going to have to provide not just a house price but we're also going to have to provide 211
a range around that price. 212
And believe it or not, this range actually has something to do with the residuals and the mean squared 213
error. And 214
that's what we're going to be looking at in the next lesson. 215
But before you head over there let me, let me ask you a question. 216
Have you ever used a website like Zoopla or Zillow to get an estimate for a property? And, 217
if so, please post the link of the service that you used in the comments section for this video. 218
I'd actually be quite curious to know for which places on earth we can get estimates like this and 219
for which ones we can't. At the time of recording, 220
I tried finding the equivalent of Zoopla for Austria but no dice. 221
Yeah. 222
The property market in Austria is just not as transparent as in the UK. 223
And this property price transaction data that Zoopla is piggy backing off of to make their estimates, 224
you know, that they're getting from the land registry is just not publicly available. 225
Anyhow I'll see you in the next video. 226
Take care.