So now that we understand the intuition behind our regression algorithm, we can run and evaluate our 1
regression. 2
We will write the Python code to actually crunch the numbers. 3
So I hope that at this point you still have your Jupyter notebook open and your session is still connected. 4
While the big advantage of using the online version of Jupyter notebook is that you can get started 5
right away and you don't have to install anything, but if you're inactive for a while and you haven't 6
been using it then it's possible that you can get disconnected and lose your work. 7
So in that case you might see something like this. 8
And it's important to remember that you can always save your work by saying "Download as" and then "Notebook" 9
and you can always restore your work by going back to try Jupyter with Python and then simply uploading 10
the Jupyter notebook that you downloaded previously and your data file. 11
So if you upload those, then you can continue where you left off. In the next module, 12
I will walk you through how to install Jupyter locally on your machine. 13
You might only encounter this situation if you are trying it out using binder through the web portal. 14
Now without further ado, let's give our notebook the capability to run a regression. This capability, just 15
like the others, it's going to come from a module. In this case this module is going to be called scikit-learn. 16
Scikit-learn is one of the most popular machine learning modules in Python and we can get hold 17
of it in our Jupyter notebook simply by typing "import sklearn", but we're only looking for something 18
very specific out of scikit-learn, so instead of importing all of scikit-learn, what we're gonna do instead 19
is we're going to say from as sklearn.linear_model and I hit Tab on my keyboard here to bring 20
up this option, we're gonna import linear regression. 21
Once again, I typed the first few characters there and then I hit Enter to insert the rest of the code 22
and this avoids any sort of typos like, you know, having a lowercase r here for example. After we've 23
added this line of code, let's hit Shift+Enter on our keyboard or click "Run" to run the cell. 24
Alternatively, if you've opened this notebook from fresh, you might have to go to "Cell" and then "Run All". 25
Let me add a few more cells here at the bottom, and then we can add the code to run our linear regression. 26
The task of running a linear regression for calculating the slope of our line and the intercepts is 27
once more going to be done by an object; so we're gonna create this object and we're gonna give it a name. 28
So what I'm going to call it - regression, and I'm going to set it equal to LinearRegression with some 29
parentheses at the end. 30
This bit of code here will create our object and we're gonna be storing it inside here so we can always 31
refer to our linear regression by the name "regression". 32
So now that we've created our object we can actually tell it to do something; we can tell it to run our 33
regression. And the way we're gonna do that is simply by using the regression and then putting a dot 34
after it and then writing "fit". Inside the parentheses we have to tell it two things: namely the features - 35
our X, and our labels - our lowercase y. When we hit Shift+Enter on the cell it will crunch the numbers 36
and, just like that, we've actually run our regression. 37
Now, I know we don't see any output here but trust me, 38
the numbers have been crunched and to prove this we can pull up the slope coefficient and the intercept 39
that were calculated by our regression. We can get hold of the slope through our regression object. 40
So regression.coef and here I'm hitting Tab on my keyboard to bring up that menu there and then hit 41
Enter to insert the code. 42
Now you can, of course, type this out and you'll get the same result but just remember that there's an 43
underscore here at the end. 44
That's part of the name. 45
So let me run the cell and let's take a look at what the slope coefficient is. 46
So here we see that it's 3.11. 47
Now one of the great things about Jupyter notebook is that we can add some markdown cells so I can add 48
a cell here and I can change it to "Markdown" to add a little bit of explanation to what my code is doing. 49
That way if I come back to this notebook in the future and I'm wondering what regression.coef_ does, 50
I can look at the markdown cell above and I can leave myself a little note; for example, I can say slope 51
coefficient and Shift+Enter and then I'll get this text inserted here. 52
Now, if I wanted to leave myself little notes inside the actual cells where I've got my Python code, 53
then I have to use something called a comment and I can add a comment here with a hashtag or a pound 54
symbol and I can add this here and right, for example, theta _1 and you can see that the text 55
here is green and that means that this text is considered to be a comment and will be ignored. 56
It's not going to be treated as code and I can actually execute the cell and I'll run just fine, but 57
if I were delete this little symbol here - this pound symbol, and try to run it then I would get an error 58
and it would tell me "invalid syntax". 59
So that little pound symbol is very, very important. 60
That's what's going to differentiate code from comments. 61
OK, so we've got our slope coefficient. 62
What about our intercept? Our intercept 63
we can pull up similarly through this regression object. 64
So "regression.", and then put "intercept" with the underscore at the end. 65
Once again I hit Tab on my keyboard there - I'm not such a fast typer, and it inserted the code for 66
me. 67
So let's take a look at what the value of this intercept is. 68
Here we can see that it's about negative 7.2 million. 69
So, now we know the regression slope and the intercept. 70
And while it's not bad to have these two numbers to hand, we can actually do better than this. 71
Wouldn't it be nice if we could just plot a line on our chart? 72
Because we've painstakingly visualized our data. 73
Why not just plot the regression line on this chart as well? 74
So let's do just that. 75
What I'm going to do is I'm gonna take this entire cell and I'm going to copy it and then I'm going 76
to come down here and I'm going to paste it. 77
And the reason I'm doing this is simply to have a reference. 78
So here I'm going to have the chart without the line and here I'm going to modify my code here so I 79
get the chart with the line. And you can move these cells around as well. 80
So if I use this little up arrow here, then I can move it above this cell here. Wonderful! 81
So, how do we plot a line on here? 82
Well, we can use matplotlib once again. 83
So, matplotlib 84
has a functionality called plot. 85
So plt.plot will allow us to plot a line on this chart, but we have to supply some information; we 86
have to tell 87
matplotlib what exactly to plot. 88
And for that it will actually need two things. 89
It will need the Xs and the y's so I'll need some information for where it should plot things on this axis, 90
and where should plot things on this axis. Now, 91
production budgets are our feature. 92
So for this axis we can use our Xs, right? 93
So I'm going to put an X here, and then a comma and I have to supply the y value. And what should this 94
be? 95
We don't want to use the actual value that we have for the gross revenue. 96
Instead, what we would like to do is we'd want to use the predicted value from our regression and we 97
can get hold of those values by calculating a predicted value for each of the X values. 98
To do that we'll need the regression object from scikit-learn that we used earlier. 99
All we need to do is type regression.predict(x), 100
then it will calculate a prediction for each of the budget values in our 101
data and plot it on the graph. 102
So let me hit Shift+Enter on the cell and let's take a look at what we've got scrolling down. 103
I can see we've got a regression line right here. 104
Fantastic! 105
But, it'd be a bit nicer if it stood out a little more, right? 106
Let's give it a color. 107
Let's give it a width. 108
Let's make it a bit thicker. And we can do that by going into this line of code here. 109
And just before this ending parenthesis we can add a comma and then we're gonna add a color here. 110
So I'm gonna see color is equal to red in single quotes and that will make our line red, 111
as you can guess. And to make it thicker, we can specify the line width. 112
So linewidth, all in lowercase in one word is equal to, let's try 4; if I hit Shift+Enter to refresh 113
my cell, then I can see that my regression line now is a lot thicker and it's changed in color. 114
What we see now is, we see the relationship between our production budgets and our movie revenue as predicted 115
by our linear regression model. 116
And that means we can move on to the final part of our data science workflow, namely evaluating and analyzing 117
our algorithms performance. 118
How do we do? Did we do a good job or a bad job? 119
What can the movie's budgets tell us about the movie revenue? 120
This is the point where we have to think very, very hard about our model. 121
The question that we should ask ourselves at this point is: are these parameters actually plausible? 122
Let's take a look at that slope coefficient. 123
It's 3.11. 124
It means that there is a positive relationship between budget and revenue and not only that, it means 125
that for each dollar that we spend on producing the movie, we should get around 3.1 dollars 126
in revenue in return. 127
And I actually think this seems to make sense. 128
Bigger budget films tend to do better. 129
So that's good news for us, right? Because we've put our life savings on the line for this zombie movie, 130
right? 131
Now, what about the other parameter? The intercept. 132
This one is at -7.2 million. 133
How do we interpret that? 134
What this intercept is literally telling us is that a movie with a budget of zero would actually lose 135
over 7 million dollars. 136
So that's a bit problematic. 137
Right? 138
That seems quite unrealistic because if you and I went out to make a movie with a thousand dollars, it's 139
pretty unlikely that seven million would just disappear from our bank accounts. 140
So, this is a lot less realistic. 141
And this begs the question - what should we conclude about our model? 142
Well it means that we should actually take it with a grain of salt. 143
We just have to accept that our model is a dramatic simplification of the real world. 144
And as such we should be a little bit careful on how much we believe the predictions of our model, especially 145
at the extreme ends. 146
Just look at the distance here look at the gap between this data point and our line - the predictions 147
of our model seem to fit the data a lot worse at the extreme. 148
So how would we use this model to make a prediction, anyhow? Say we need to predict the revenue for a film with 149
a 50 million dollar budget. 150
We know what our intercepts are. 151
We know what the theta zero and the theta one is equal to. 152
And if we want to know what the revenue would be for a film with a budget of 50 million, all we would 153
have to do is to substitute the values of our parameters that we estimated into this equation. 154
So our X is going to be 50 million and if we do the math then we get our prediction, at least according 155
to our model. 156
Right? 157
On a chart it would look something like this. 158
We can draw a line up from 50 million and then predict how much, 159
what it is that we're actually going to make of the movie? 160
And it's gonna be a little bit less than three times the amount that we invested so about 148 million 161
dollars - which is not bad, right? 162
But, how do we know if it's accurate? 163
How can we measure how good our model is? 164
So even though it is very very simplistic we can still ask the question of how much of the real world 165
data it actually explains. 166
And for that we need some kind of measure. 167
We need some kind of statistic and the measure that we're going to look at is called R squared. 168
Also called the goodness of fit. To look at our R squared we'll simply take our regression and we write 169
regression.score. 170
And within the parentheses we supply our capital X and our lowercase y - our feature and our labels and 171
we hit Shift+Enter and here what we see is that the R squared is approximately 0.55. 172
This number here is the amount of variation in film revenue that is explained by the film's budget. 173
And I got to say that 55 percent is actually pretty good because think about it this way are very simplistic 174
model with a single feature, namely the production budget, can explain around 55 percent of the variation 175
that we see in worldwide movie earnings. 176
I'd say that's pretty good news for a first try but, of course, we should be a little bit cautious 177
reading into this model too much, because we've still got a lot to learn. 178
For example how would our model do if we added more features, like how long it took to make or if it's 179
a sequel? 180
Would we get more realism? 181
Would it make our model perform better and make our predictions more accurate? 182
And perhaps we should evaluate our model, not just on the data that we used for training it, but on new 183
data, data that it hasn't seen yet and also, what if the relationship that we have here is actually non-linear. 184
What if we somehow need to transform the data to get a better fit? 185
So in a way our analysis has left us with a lot more questions that we should investigate and we will 186
do just that in the upcoming modules. 187
Well, you got started on the first project and you went through the whole data science workflow; you've 188
gathered the data, you've cleaned the data, you've visualized it and then you ran a machine learning algorithm 189
and then you've evaluated the results and you've even made a prediction, but this is only the start. 190
We've got a whole lot more ground to cover and we'll dive deep into a lot of these concepts and the 191
techniques that we've introduced here. In the next module we're going to install Jupyter and we're also 192
going to learn a little bit more about regression. 193
But the real focus will be on learning more Python programming. 194
From there, we're going to learn about gradient descent and how optimization works for many machine learning 195
algorithms that we're going to encounter. 196
And after that we're going to use multivariable regression and predict some real estate prices in Boston. 197
From there we're going to build an actual spam filter from scratch using a Naive Bayes classifier and 198
then we're gonna take it up a notch, 199
and we're gonna dive into deep learning with neural networks and TensorFlow. 200
So for all of that and more, I'll see you on the next lessons.