Now, 1
the thing is, all the prices that are in our target were collected in the 1970s. 2
Now, I wasn't around back then, but I am told the 70s was a glorious time to be alive. 3
The thing I know for sure, however, is that one dollar in 1970 was worth much, much more than one dollar 4
today. 5
In fact, inflation is a horrific thing for anybody with savings and to buy a house today you need many, 6
many more dollars than you do 7
in the 1970s. So given that we want to get a dollar price estimate that's a bit closer to today's values 8
than 1970s values, 9
let's make an adjustment to the estimates that our model is spitting out. 10
The only question is - how do we get a more realistic price out of our little valuation tool? 11
For starters, let's take a look at what the median price in our dataset is 12
as it stands currently. And we can see this if I pull up numpy's median function, so "np.median()" and 13
I supply the target values in the "boston_dataset". 14
So "boston_dataset.target" inside these parentheses for the median function will calculate 15
the median value of a property in our dataset. And what we see is that in 1970 the median value was 16
21200 dollars. 17
This is where we were at in 1970 in Boston. 18
But what about today? 19
If I Google "Zillow Boston home values", I am brought to this page here. 20
So this is on "zillow.com" which actually gives us a median estimate for home prices in Boston as 21
of today. 22
And what we see here is that today the median home value is approximately 583000 23
dollars, more than half a million. 24
And that's a combination of inflation as well as increases in the real price of homes. 25
Now, as a challenge, can you write the Python code that converts the estimated price from our model which 26
is in 1970s log prices to today's dollar values? 27
You're gonna have to convert the estimate itself, 28
the upper bound and the lower bound to today's prices. 29
And once you've converted it, round the values to the nearest thousand dollars. 30
I'll 31
give you a few seconds to pause the video and then I'll show you my approach to solving this problem. 32
Ready? 33
Here it goes. 34
So the first thing I'm going to do is I'm going to say "ZILLOW_MEDIAN_ 35
PRICE = 583.3". 36
This is the price in thousands that I saw on Zillow website for the median price in Boston at the time 37
of recording. 38
The next thing I'm going to do is I'm going to calculate some sort of scale factor, the number I'm going to have to 39
multiply my 21.2 by in order to get a realistic value for today. 40
So this is gonna be "SCALE_FACTOR" which is equal to, 41
well it's going to be equal to 583/21.2. 42
Right? 43
So "ZILLOW_MEDIAN_ 44
PRICE/np.median(boston_dataset.target)". 45
Let's take a look at what the scale factor is equal to, so "scale_factor", Shift+Enter, we now see 46
that the dollar prices should be multiplied by 27.5. 47
Okay. 48
So that was step one. 49
Now let me call my function. 50
So my function will give me a log estimate, 51
it will give me an upper price and a lower price 52
and it will give me some confidence interval. 53
Those of course are gonna get their values from our function, "get_log_estimate" 54
and then I'm going to calculate, I don't know, a price estimate for a mansion, 9 rooms and maybe "students 55
_per_classroom" is gonna be equal to, say 15, "next_to_river = 56
False" and "high_confidence = " is going gonna be equal to, 57
I don't know, False, let's vary it up a bit. 58
Okay. 59
So now I've got my scale factor calculated and I've got values that I'm going to calculate from my estimate, 60
my upper and lower bounds and my confidence interval, let me hit Shift+Enter, make sure this works and it 61
does, no errors. 62
Now what I'll do is I'll convert to today's dollars, so the "dollar_est" is gonna be equal to first 63
off, reversing the log transformation so "np.e**log_estimate", then times 64
1000, because our dollars are in thousands and then times, 65
well you guessed it, the "scale_factor" that we've calculated above. 66
This gives us a dollar estimate for today. 67
Let's take a look at what this one is actually, so "dollar_est", Shift+Enter, 68
here's our unrounded value for a mansion that is not next to the river and has a pretty small class 69
size, 15 students. Estimate in today's dollars is approximately 826000. 70
Now we just need to round it, right? Now I'm going to add another comment, say "Round the dollar values to nearest 71
thousand" and I'll create another variable that's going to hold onto the rounded estimate, 72
so I'll say "rounded_estimate" 73
is gonna be equal to the return value from some sort of rounding function. 74
I'm going to use numpy's around function, so "np.around(dollar_estimate, )" and 75
I have to say I want to round to the nearest thousand. 76
So I'm going to say -3. 77
Let's take a look at what this output actually reads. 78
So "rounded_est", Shift+Enter, 79
and here we see it's 826000. 80
So this is a rounded value. 81
Brilliant. 82
Now all that's left to do is do the same thing for the upper value and the lower value. 83
I can first off copy this line of code, paste it twice, change my variable name, so I'll say "hi" and then "low" 84
and we're transforming "upper", and we're transforming "lower" and then we're also going to round the upper 85
and the lower values. 86
So it's going to be "rounded_hi" and "rounded_low" for "dollar_hi" and "dollar_low". 87
Let me print all of this out with f-strings. So "print(f 88
'The estimated property value is {rounded_est}.')", Shift+ 89
Enter, 90
that works, except I've got a typo, "estimated property value". Let's add another print statement, 91
"print(f'At { 92
conf}% confidence the valuation range is)"; final print statement "f'USD 93
{rounded_low} at the lower end to USD {rounded_ 94
hi} at the high end.')". 95
Full stop. 96
Okay, so that was just a lot of typing, but now we can look at our output 97
nicely worded, using the variables that we've defined earlier inside some f strings. 98
Let's take another look at the output for this cell. 99
Brilliant. 100
That works beautifully. 101
Now, tell you what. 102
Let's add all of this to a function. 103
Let's create another function called "get_dollar_estimate" and make this function do all the conversion 104
as well as call this function here to actually get the estimate. 105
So I'm going to define a function, "def get_dollar_estimate 106
():" and then inside the body of this function I'm going to put all the code that we've 107
written above. 108
So all of this up to here, will go inside of the function. 109
So copy this, I'm going to paste it in here and then you'll see it's not indented, so I'm going to have to indent, 110
so it's actually part of this function, and I can do that by selecting all the lines and hitting Tab 111
on my keyboard. 112
Now what we want is we want this function here to actually take again like the 4 arguments that we're 113
passing through to our other function call. 114
We can call the first one "rm", 115
we can call the second one "ptratio", 116
we can call the third one "chas" and set that equal to False by default, 117
and then the last argument we can call it "large_range" and set that equal to True by default. 118
And this means that in our nested function call right here to get log estimate we can pass through these 119
parameters. 120
So this won't be number 9, 121
this will be "rm", this won't be "students_per_classroom = 15", 122
this will be "ptratio", 123
this won't be "next_to_river = False", but "next_to_river = chas", and this won't be "high_confidence 124
= False", but it'll be "high_confidence = large_range". 125
So I hope that you can see how using these different arguments works. For these two, 126
I've used an argument by keyword, so "next_to_river" is the keyword, "chas" is the value which we'll get from 127
this function call. 128
And for this argument "high_confidence" is the key word and "large_range" is the value, "rm" and "ptratio" 129
are just the values because I'm passing these arguments by position, not by keyword. 130
So this is argument number one, this is argument number two. 131
If I didn't want to pass the arguments by position, I could use a keyword as well, so I could say "students_ 132
per_classroom = ptratio" 133
and that's because this is the keyword that we've defined in our signature for "get_log_estimate". All right, 134
so we've got our function, 135
we've got our way of getting the log estimates, 136
we've got our way of converting the log estimates to today's dollars 137
and we've got the part where we're rounding those values and then where we're showing some sort of 138
output to whoever's calling the function. 139
Let's try all of this out for good measure. 140
So "get_dollar_estimate()" for a small apartment with, say "rm = 2", and 141
a terrible school with 200 kids, so "ptratio = 200". 142
But on the upside this apartment will be next to the river, 143
so "chas = True". Before I hit Shift+Enter on the cell I'm going to make sure that I have had Shift+ 144
Enter on this cell beforehand and now I know that my function will be recognized as I do the same here. 145
The estimated property value is 0 and at 95 percent confidence the valuation range is between 0 146
at the lower end and a 1000 dollars at the high end. 147
Why is that? 148
That's because the parameters that we've supplied here are completely unrealistic, 149
right? This number in particular, 150
you never have 200 kids per teacher in the local school. 151
A more realistic number, something like say 30, which will then give us a more realistic estimate. 152
The same problem we might face if we have, say zero rooms, in the apartment which of course is nonsense. 153
In this case, even for a property with zero rooms, which is impossible, 154
we get a price estimate. Now there's two ways that you can avoid this kind of situation, 155
right? 156
First off, you might have to inform people what get_dollar_estimate does and what these values actually 157
represent before they start just calling this function. 158
And the second thing that you can do is you can reject giving an estimate 159
if these values are unrealistic. 160
Let me show you how you could do that for example. 161
So coming up here and at the very top of the function, we could have an expression like " if rm < 162
164 1:", "print( 163
'That is 164
unrealistic. 165
Try again.')" 166
Yeah. 167
And then, still within this if block as you can tell by the indentation, we return. 168
So as soon as this return instruction is hit, none of these following lines of code will be executed. 169
I'm going to hit Shift+Enter, refresh the cell, and then I'm going to try to run this code again, see what we 170
get. Our function now does not provide an estimate when the inputs for the rooms are completely 171
unreasonable. 172
If you wanted to check two things, say we wanted to make sure that this value here is not smaller 173
than 1 as well, right, 174
then what we could do is we'd come up here to our if statement and then we could use a logical "or" to 175
check two separate conditions, so if "rm < 1 or ptratio < 1", then 176
print this line and return from the function. 177
In other words, one of the things that you can do to make sure that your functions aren't being completely 178
abused is to actually check the values that are being supplied when they're being called. 179
So if I hit Shift+Enter now and I change this line to something reasonable like, say 6 rooms, and I 180
try to change this line here from ptratio od 30 to ptratio of -60 181
message. 182
Okay, so strategy 1 was checking for inputs, but better yet, let's add some sort of description about what 183
this function does 184
so that the person using it can pull up say the quick documentation. If I press Shift+Enter on this now, 185
I can see the signature, I can see it says "rm, ptratio, chas=False,..", I see the default 186
values and hitting that plus sign, I don't see anything else. What I would like to see though is something 187
called a Docstring. The Docstring should give us a description of our function, what it does, what 188
its inputs are and more or less like how to use it. 189
Let me show you how we can add this to our "get_dollar_estimate" function. At the very, very top we can 190
put in three double quotes and as you can see there's already six double quotes here, because Jupyter 191
notebook inserted the closing three double quotes. 192
In other words, you have three double quotes at the beginning and you have three double quotes at the 193
end. The first three double quotes mark the beginning of the docstring, the last three quotes mark the 194
end of the docstring and whatever we put in between will appear in the quick documentation. Check it 195
out. "Estimate the price of a property in Boston.", Shift+Enter and then Shift+Tab will now show 196
this description in the quick documentation. One of the good things to do when you're writing function 197
that you want other people to use is to include a little bit of information about what the function 198
does and also what the keyword arguments are. 199
After all, if we do that on "np.around()" and press the little plus sign, we do indeed see some information 200
on the parameters that this function takes - a, decimals and out. 201
Yeah. 202
So we can see a little description here and we can use this function the way it was intended to. 203
Let's do the same thing for our function. 204
So if I come in here and say "Keyword arguments:", "rm -- number of rooms in the property" 205
and then "ptratio -- number of students per teacher in the classroom for the school in the 206
area", "chas -- True 207
if the property is next to the river, False 208
otherwise", "large_range 209
-- True for a 95% percent prediction interval, 210
False for a 68% interval." 211
Adding this description here for our keyword arguments and hitting Shift+Enter allows us to see what 212
these arguments are and get a little description in the quick documentation when we hit Shift+Enter 213
and take a look. 214
Brilliant. 215
All right. 216
We've almost made it to the end. 217
The last thing I want to show you is how we can package what is in this Jupyter notebook here 218
as a Python module. If we look at our MLProjects folder here, a Python module is a file that ends in 219
".py". To create one of these files, 220
what we can do is we can upload a ".py" file to this folder or alternatively, we can go to "New" < "Text 221
File" and create this py file in Jupyter notebook directly. 222
I personally quite like the Atom text editor for writing Python code and editing it. 223
You, of course, you can use other text editors as well to create these py files, but Atom is definitely 224
a good one. To save you the trouble of installing a text editor, I'll show you quickly how you can do 225
this in Jupyter notebook directly. 226
First off, let's rename this file. Let's rename it as "valuation. 227
py". 228
Or better yet let's name it as "boston_valuation. 229
py". Click "OK" 230
and now we're gonna copy some of our Python code and paste it into this empty ".py" file. 231
Let's copy the imports for example, 232
put those here, let's copy the cell where we're gathering the data, 233
put this here, 234
coming down, let's copy this cell here with our indices and our property_stats variable, 235
copy it and put it here. I don't really need this comment so I'll delete it, scroll down, add a few more lines, 236
go back to our notebook, 237
scroll down a little more, copy the part where 238
we're running our regression, put it in here, come back, copy the function where we're doing our log estimate 239
of course, add it to our ".py" file and then come down here where we've got our scale factor, 240
copy those two lines, go into our .py file, scroll to the top in this case and I'm going to to add it below 241
our CRIME and ZN indices. 242
So going to paste it in here. 243
And then finally I'm going to grab all the code in this cell here, copy it and paste it at the very end 244
of my .py file. 245
All that's left to do now is "File" > "Save" and now in my projects folder I have a "boston_valuation.py" 246
module that contains all my Python code, which I can now use in any of the Python notebooks 247
in this projects folder. 248
Check it out. In the valuation tool, I can come down and I can say "import boston_valuation 249
as val" and I can call a function from this module, namely our "get_dollar_estimate" function with "val. 250
get_ 251
dollar_estimate(6,12, True)". Hitting Shift+Enter, 252
I get a price estimate. All my arguments here are passed by position and this function call here comes 253
from the module, 254
right, 255
it comes from the module that I've just imported. This one here comes from the notebook, 256
this one here comes from our "boston_valuation" module. 257
Now of course this is a lot more clear if I open my Multivariable Regression notebook and try to do 258
the same thing. Under the notebook imports I can import "boston_valuation as val" and if I hit Shift+ 259
Enter here, scroll to the very bottom, and say "val.get_dollar_estimate(8,15,False)", 260
I will get an output from my get_dollar_estimate function from inside my .py file. 261
Fantastic. 262
So we've covered quite a few things again in this lesson. 263
It's been a review of a lot of the concepts that we've talked about so far, but we've also learned a 264
lot of new things. On the Python front, 265
we've learned how to create functions with optional arguments, how to call these functions with the arguments 266
where we supply the values by position and by keyword. 267
We've learned about how to make the quick documentation show up for functions that we ourselves have 268
written using docstrings. 269
We've covered how you might exit a function if you don't like the inputs that it's getting. 270
And we've covered how to use if-else blocks to check for condition or even two different conditions. 271
So again, congratulations for making it all the way through this. 272
I know the learning curve is getting steeper and I look forward to seeing you in the next modules. 273
Take care.