In a previous lesson we've already written this line of code "from sklearn.linear_model import 1
LinearRegression". 2
In other words we've already imported our linear regression functionality into this Python intro notebook. 3
For consistency, 4
let's follow the same pattern that we employed when we were estimating our movie revenue. 5
We're going to create a variable called "regr" and this variable is going to store our linear regression 6
object. 7
To run our regression, 8
all we have to do is call the good old fit method. 9
So we're gonna see regr.fit, 10
open the parentheses and then for our explanatory or independent variable we're going to use the amount 11
of drugs in the tissue. 12
So we're gonna type LSD and then put a comma after it, 13
and now we can add our dependent variable or the values that we're going to try to predict. 14
In this case this is the score. And that's it. 15
Now if you remember previously our explanatory variable was called capital X and our dependent variable 16
was called lowercase y. 17
Now remember our scikit-learn's fit method essentially computes the parameters of this equation. 18
We've got our theta zero which is our intercept and we've got our theta one which is our coefficient 19
in front of our explanatory variable. 20
Let's see what happens now when we try to run our regression. 21
We in fact get an error. 22
Looking down, we see that we get a value error: "Found input variables with inconsistent numbers of samples". 23
Now this is odd, right? 24
Why would there be an inconsistent number of samples? 25
We've got seven rows in each of our two columns. 26
We've got seven rows in LSD and we've got seven rows in the math score. 27
So why would there be an inconsistent number of samples? 28
Now, even though this error message seems a little bit like a red herring, 29
the reason that we're getting this problem actually has to do with the fact that we are working with 30
series and not data frames. We're getting this error because this notation here data[], 31
and then the column name actually extracts an object of type series instead of an object of type data 32
frame. 33
Now, if you recall, the way to get a data frame from another data frame is to double up on the square 34
brackets. 35
So if we add an additional pair of square brackets, this notation now will extract an object of type 36
data frame and store it in LSD and in score. 37
So if I press Shift+Enter now, we are now no longer working with series we are working with data frames. 38
Going back down to where we're fitting our regression I can press Shift+Enter again to rerun this line 39
of code and we can see that our regression runs without a problem. 40
Now you don't have to take my word for it 41
regarding the change in types - you can double check this yourself. So you can always add a cell below 42
and check the types, so if I say type(LSD), then I see it is a data frame and if I take away the square brackets, 43
press Shift+Enter and rerun this again, I can see that it is a series in this case and this is what we've 44
talked about when working with data frames in the previous lesson. 45
So now that we've successfully fitted our regression, let's take a look at the values of this theta one 46
and this theta zero parameter. 47
If you remember the way we did this in the past it was looking at the attribute of our regression object - 48
the attribute in question was called "coef" with an underscore. 49
So let's add that here "regr.coef_" and let's hit Shift+Enter, and see what happens. 50
Our Jupyter notebook will print out array, then parentheses, 51
then one set of square brackets, two set of square brackets and then the value of our theta one parameter. 52
So we can see that our coefficient is stored inside an array. 53
It's an array of one element. 54
Let's pick that element out. 55
So I'm going to say [0] to access the first element in the array. 56
Let's see what happens when I press Shift+Enter now. 57
Huh. 58
In this case one of the square brackets has disappeared, but we're still left with an array. 59
We're not yet able to access the number inside directly. 60
We're still getting a collection containing just one element. 61
So you might ask, did this just work? 62
We access the first element of our array and we still got an array. 63
It seems like a bug, right? 64
Well the answer is is that we have to go one level deeper to get the raw value. 65
You've probably noticed that there was one square bracket less, right? 66
So if I don't have this at the end, I have two square brackets but if I do access an element inside my 67
array I have one square bracket. 68
And the reason we get this is that we have to go two levels deep. 69
We actually have an array of arrays. 70
Mind blown, right? Our coefficient is buried inside an array that's inside another array. 71
So how do we access an array inside an array? 72
Well, we can access the first element which is the array and then we can access the first element of 73
that array again to get the raw value. 74
And this is how you would access a particular value of a nested array. 75
Let me add this to a print statement. 76
So I'm going to say "print", and just say "Theta 1 : "comma and then close our brackets here. 77
So I'm going to add this to a print statement like this. 78
Now let's take a look at our intercept - our intercept was the intercept_ attribute from 79
our regression object. So I can say "regr.intercept_" and press Shift+Enter and we see that 80
our intercept is also inside a collection that's also inside of an array. 81
But there's only one set of square brackets here, so we can access the raw value inside just having that 82
[0] following the name of the attribute. 83
I hit Shift+Enter we see the raw value printed out. And again I can wrap this inside a print statement 84
'Intercept: ', regr.intercept_intercept[0], and close the parentheses at the end. 85
There we go. 86
So here's our intercept. 87
Now what about the goodness of fit or our R squared. 88
To find out how much of the variation in our data is explained by the amount of drugs in the volunteers 89
tissue, 90
we call the score method on our regression. 91
So we type "regr.score" and then we have to provide two values. 92
One is our explanatory variable and the other one is our dependent variable - which was score. So 93
regr.score(LSD, score) 94
We're going to print this out, and we see that our R squared is approximately 0.88. Let's wrap this inside a print 95
statement as well - " 'R-Square: ', " - there we go. 96
So in this cell we fitted our regression, so we've run our machine learning model and we're printing 97
out a couple of stats about our regression. A couple of the statistics that describe 98
what went on with the calculation. One of them is the coefficient, another one is the intercept of our 99
line and another one is the R-squared or the goodness of fit. 100
So we've got some basic information about our regression and we see that the amount of drugs in the 101
contestants tissue explains close to 88% of the math test performance and we also see 102
that for every increase in LSD parts per million, our volunteers math performance was approximately 9 103
percent worse than the control - this is what the theta one is telling us. 104
Now even though this is all very well and good, it'd be really nice to represent this graphically because 105
we like pictures - pictures are very very important for making sense of data so let's create another plot. 106
I'm going to do this in the cell below. 107
Now one thing that you've already seen a little bit in the Python code is that when creating nice looking 108
graphs it's a two part process. In the first part, 109
we do all the styling and in the second part we plot the data and show it off. 110
So what I'm going to do is I'm going to do the second part first. 111
I'm going to plot the data and then I'm going to add my styling code later on. 112
So plotting the data as it is I can write plt.scatter and then provide the inputs to our scatter 113
plot and that's going to be the LSD parts per million, comma, and then the score. 114
These are the math scores. 115
So let's see what this plot looks like 116
by adding by adding plt.show() beneath. 117
Here we go. 118
This is what our plot looks like before we've done any styling on it. 119
Now I think this chart looks, 120
I think this looks super ugly actually so we're going to have to do something about this. For starters 121
let's add some arguments by keyword to this plot. 122
So in your scatter method you're going to put a comma at the end after score and then write 123
color = 'blue' 124
Let's hit Shift+Enter to see what it looks like. Now we've got our data points in blue. 125
So this is a slight improvement to the black and white version. 126
Now we don't have many dots on here. 127
We don't have many, many data points. 128
So let's increase the size of these individual dots on our chart. 129
And I want to leave this to you as a challenge. 130
So I've got the documentation of the scatter method up in front of you right now. 131
And what I would like you to do is I'd like you to look at this documentation and see if you can figure 132
out how to increase the size of these data points 133
and also maybe add some transparency - in other words, instead of having it a solid blue color make those 134
blue dots slightly transparent. 135
I'll give you a few seconds to pause the video. 136
The hint I'll give you is that it's going to be in the keyword arguments of the scatter function. And, 137
here's the solution. 138
So we wanted to increase the size of our dots. 139
So the way to do this is to look at these keyword arguments. 140
So, for example, "s" is the size in points of our dots and the transparency is this alpha value here. 141
The alpha value will be between zero which is transparent and one which is opaque. Coming back to our 142
Python code, 143
we can add these key word arguments to our scatter method. 144
So after 'blue', I'm going to add a comma and then I'm going to see "s=" and let's experiment here 145
a little bit. 146
So what happens if I say "s = 500" and hit Shift+Enter? 147
I get enormous blue dots. This actually doesn't look half bad but I think I'm going to go with something 148
like, maybe 100 is the right value here. So that's the size of our data points covered. 149
Let's change our transparency now. 150
This was in the alpha parameter. So I'm going to say 151
"alpha = ", I don't know, 152
0.7 - it's going to be a value between 0 and 1, remember? 153
So hitting Shift+Enter, I get a nice little bit of transparency here on my data points which are now 154
little bit larger so we can actually tell what's going on. 155
OK, so I'm going to leave it at that. 156
And now I'm going to add some labels to our chart and make it look a little nicer. Since we've done this 157
before, 158
I'm going to leave this to you as a challenge so you can return and remember the Python code that you 159
wrote. 160
Can you set the title of the plot as a whole as "Arithmetic vs LSD-25" and then add some labels on 161
the side - one for the x axis that reads "Tissue LSD ppm" and one for the y axis that reads 162
"Performance Score"? 163
And here's the solution. 164
So we take our plotting object, put a dot after it and write "title()", and then provide the string 165
"Arithmetic vs LSD 25" and then we do the same for the labels on the x axis and y axis, so plt.xlabel( 166
"Tissue LSD ppm") and plt.ylabel( 167
"Performance Score"). 168
There we go. 169
Let's hit Shift+Enter and take a look at what this looks like and we see that may be the thing to do is to increase 170
the font size a little bit on these three labels. 171
I think in our previous chart 17 for the title and 14 for the labels worked really well. So I'm going to say 172
"fontsize=17" for the title, and then I'm going to add another keyword argument to our X labels and Y labels 173
"fontsize = 14" and "fontsize = 14" again. 174
So let's take a look. 175
That's starting to look pretty good. 176
Now to round things off a little bit, 177
we can try again setting a limit on the range. 178
So ylim 179
is gonna be between 25 and 85, 25 and xlim is gonna be between maybe 1 and 6.5 180
Yeah. 181
Doesn't need to go all the way to seven, 182
I reckon. And for the style maybe "plt.style.use" 183
then we can choose our good old friend 184
'fivethirtyeight'. Let's hit Shift+Enter and to apply the changes. 185
I don't think that worked. 186
Let's try again. 187
Okay, so here we go. 188
This is how it would look like with our styling as it is currently. 189
We've got our range set. 190
We've got our colors set and we've got the font size set as well. 191
At the very top of the cell I'm going to add again this little percentage sign and write 'matplotlib 192
inline'. 193
And what this does is it tells Jupyter notebook to export this graph as it is when we say File > Download 194
as > Notebook. 195
So there's really only one thing left to do which is plotting our regression line on here. 196
Because at the moment we've got our data points we've got our chart nicely formatted and looking good, 197
all we have to do now is plot our predictions from our machine learning model on here. So our machine 198
learning model will have a prediction for every level of LSD tissue concentration in the data set. To 199
get hold of these predictions, 200
we use a method called predict so we would write "regr. 201
predict", 202
And as a parameter here, as an argument here, 203
we would supply the LSD tissue concentration. 204
So this predicts a math score based on the amount of drugs in the tissue. 205
Now we'll want to store that information somewhere. 206
So I'm going to create a variable called "predicted_score" and set it equal to the output 207
from this method right here. 208
Now remember, you've got a press Shift+Enter to actually run this cell. 209
Otherwise the cells below won't know about this code that we've just written. 210
So I'm going to hit Shift+Enter now. 211
So looking down at our chart, we see the actual scores indicated by the blue dots, 212
and now we just have to plot the predicted scores alongside these actual ones. And all these predicted 213
scores are gonna be connected by a line. 214
This is the line that we want to superimpose on our graph, so we can write "plt.plot" and then provide 215
the line that we want to draw. 216
It's gonna be the LSD tissue concentration on the x axis and then on the y axis it's gonna be 217
our predicted scores, right? 218
"predicted_score" 219
Now let's hit Shift+Enter. 220
And here we go. 221
We've got our predicted values connected by a line superimposed upon our scatter plot. 222
Of course we can style this line any way we want to. 223
So I'm going to say " color = 'red' " and " linewidth = 3 ". 224
Now we've got a chart with even more contrast between the blue data points and our red fitted regression 225
line. 226
So I think that concludes all the analysis that we're gonna do for our Python intro. 227
And what have we learned from all this? 228
Well, drugs are bad for you, 229
boys and girls, especially if you're studying math tests. 230
But the other thing that you'll notice is that if you look at the original paper and you look at the 231
equation that the researchers have estimated we can actually see what their estimate was for the intercept 232
and the coefficient that we've estimated as well. 233
So they've got 89.7 for the intercept and -9.44 for 234
the coefficient in their equation. In contrast, our coefficient is -9.0 and our intercept 235
is 89.1, not 89.7. 236
So we can see that we're not able to reproduce the researchers' output exactly. 237
Now I suspect that's because the researchers and we are not working off exactly the same numbers. You 238
see, they actually don't provide the information on parts per million 239
and the math scores in the PDF that we're looking at. 240
I actually had to hunt around the web to get these numbers and they might be slightly different from 241
what's in the original paper. 242
But, that said, I think that our results are so close that we can say that we've successfully reproduced 243
the research that's in the paper there. 244
Oh and, by the way, this is in no way relevant to the study at all. 245
But many of the calculations that we just did the original authors ran on something called an IBM 360 246
computer. 247
The reason you've probably never heard of the IBM 360 is because you don't have one of these monstrosities 248
sitting in your living room. 249
Now, 250
I find this so funny that the researchers actually mentioned this particular computer model in their 251
actual paper and I can't figure out if it's maybe some 1968 humble brag about how high tech they are 252
or if IBM actually paid them for this shout out. In any case, plugging the computer model that you've 253
done your research on in your scientific paper has probably gone a little bit out of fashion these 254
days. 255
But yeah, I I did find this interesting. 256
Now, if your reaction of me telling you about this just now was "Wait a minute, 257
IBM made computers?". Then, 258
I highly, highly recommend watching this documentary called Silicon Cowboys. Silicon Cowboys is a really, 259
really fascinating film about a little startup called Compaq that battled it out with big blue in days 260
gone by. 261
So yeah watch it and I'll see you in the next lessons. 262
Take care.