Let's revisit our old friend the scatter plot. We saw in the previous lessons 1
how important it is to use plots in conjunction with descriptive statistics to spot patterns and outliers. 2
Using both of these tools together 3
we get a more complete picture of what's actually going on. 4
So far we've been visualizing our data and looked at the distribution of values of some individual features, 5
like RM for like the average number of rooms, or RAD, the index of accessibility to highways. 6
We can dig deeper into the relationships between these feature pairs as well as between the features 7
and our target value with some scatter plots. 8
The correlation matrix that we created already hinted at the fact that there are relationships amongst 9
the features that we can visualize. So on that note, I'd like to start this lesson off with a challenge. 10
First, I want you to picture what the relationship would look like between the NOX and DIS features. 11
If you recall, NOX was a measure of pollution and DIS was a measure of distance from employment centers. 12
Picture a graph in your head and then write the two lines of Python code to visualize the scatter plot. 13
I'll give you a few seconds to pause the video before I show you the solution. 14
Ready? 15
Here we go. 16
So we've created a scatter plot many times before with "plt", which is our matplotlib module, 17
".scatter()" and then we have to supply two things, the data for the x axis and the data for the y axis. 18
So on the x axis we're gonna have our "data[ 19
DIS']", 20
so this is going to be our distance which is going to be on the x axis, and on the y axis, 21
we're going to have our pollution measure, which is going to be "data['NOX']" and then 22
finally "PLT.show()". 23
So hitting Shift+Enter just gave me this error because I've come back to this notebook and I haven't 24
run the cells above it yet. 25
So I'm going to go to "Cell" > "Run All" and then I'm going to wait a little bit, scroll all the way down, 26
and here we go. 27
Was this the relationship that you imagined in your head? 28
A kind of downward sloping line? 29
Let me add some labels to this graph before I give you my interpretation. 30
So "plt.xlabel('DIS - Distance from employment', fontsize = 14)", and then for 31
the Y label, I'm going to copy this line, paste it in, 32
change it to the Y label, 33
change that to read "NOX - Nitric Oxide Pollution", and then my figure size I want to change as well. 34
So I'm going to make it a bit bigger, so I'm going to say "plt.figure(figsize = ())", 35
say maybe 9 and 6. 36
And then I'm also going to add a title here, I'm going to say "plt.title('DIS vs 37
NOX', fontsize = 14)". We're going to refresh this graph, see what we get. 38
Okay so this makes the relationship between distance from employment centers and NOX, our nitric oxide 39
pollution much, much more clear. What we can see here is that as distance increases, as we go more to the 40
right of this chart here, pollution goes down and this makes sense, right? 41
The city center of Boston is going to be an employment center but city centers would also have much 42
more air pollution than in the suburbs or on the outskirts of the city. 43
Now one thing that might be quite interesting to add to this graph is a little bit of transparency on 44
these data points as well as the, maybe putting down the correlation that we've calculated up here and 45
including that in our title. 46
So let's do that now. To calculate the correlation, 47
I'm going to add a nox_dis_corr variable, set that equal to "data['NOX']. 48
corr(data['DIS'])" and then when I'm going to do is in the title I'm going to 49
use this variable here and I'm going to include it in my string and I'm going to use Python's fstring notation 50
to accomplish this. 51
So I'm going to put f in front of the single quote and then I'm going to modify my string as follows, I'm 52
going to say "(Correlation )" and here's the key, 53
"{nox_dis_corr}". 54
So this is going to grab our variable from up here, 55
it's gonna grab our correlation between distance and pollution and it's going to insert it into our string. 56
And that's thanks to the fact that we have 57
the curly bracket notation outside of the variable name and this little f in front. 58
So let me hit Shift+Enter and see what this looks like. 59
Voila! 60
Now we've got a graphical representation of our data and the correlation, 61
all in one place. And the correlation is indeed negative and it's quite high actually with 0.77. 62
Now in terms of styling, 63
you might say to yourself: You know what this number is way too precise, 64
it's difficult to read because it's got too many values after the decimal point. 65
So why don't we round it? And we can do this with the Python round function. 66
So I'm going to do it up here where I've actually calculated the correlation and I'm just going to surround 67
my correlation calculation with this Python function, so "round", comma at the end and then a value for 68
how many decimal places I want to round it to. So I'm going to round it to three decimal places and close my 69
parentheses at the end. 70
If I press Shift+Enter now it should refresh and we should get something like this, we should get 71
-0.769. 72
The other thing I quite like doing with scatter plots is adding a little bit of transparency to the 73
data points so that we can get a better feel for how dense particular areas of the chart are. 74
So in my line of code where I'm creating my scatter plot, namely this one, I'm going to add some other 75
keyword arguments. 76
The transparency is set with the alpha keyword, and I'm going to set it to a value of 0.6. 77
Let me hit Shift+Enter and we can clearly see that there's a lot more data points here than over here. 78
I think this is a nice touch, but we can make this even more explicit by changing the size of our dots 79
and making them a little bit larger. 80
So if I choose something like 80, "s = 80" as a keyword argument, changing the size, then I've got slightly 81
larger dots for my data points. 82
Now of course we can continue adding keyword arguments here to style the graph as we see fit, famously 83
color and there's quite a few to choose from. 84
I'm going to go with indigo and give my scatter plot a purple make over. Okay, 85
so I think creating a scatterplot with matplotlib is pretty straightforward, 86
but now let's do the same thing with the seaborn module to mix it up a little bit, because remember 87
I said that seaborn builds upon matplotlib? 88
Well you're gonna see in a minute how seaborn really adds some nice little touches to these visualizations. 89
Check this out. 90
So I'm going to come down here, add few more cells and then I'm going to write the following code, 91
I'm going to say "sns.jointplot" so sns being the name for seaborn module and then jointplot 92
being the function to create our scatter plot. 93
So I'm going to say "jointplot(x=data['DIS'], y=data['NOX'])" and 94
then on the next line I'm going to say "plt.show()", hit Shift+Enter and what we get is something like this. 95
Now again I've only specified two parameters in my function call here, but you can already see that there 96
is some sort of histogram on the side and there's some additional data being provided here in this corner. 97
Now if you can't read this on your screen, this is actually the Pearson correlation coefficient down 98
to two decimal places, -0.77. 99
I can make the chart a little larger so that it's a bit more clear by going to the arguments and providing 100
the size argument. 101
So I'm going to say "size = 7", increase the size a little bit but not too much. 102
Now you should see the chart appear a little bit larger on your screen, but I think these histogram is 103
and the correlation coefficient and the fact that it adds some labels for the y axis and the x axis 104
automatically straight out of the box is a really, really nice touch. In terms of styling. 105
one thing that you might notice is that that the Jupyter notebook remembers how you've styled charts 106
previously. 107
So if you're working in a new cell and you want a new look for the chart you might have to reset the 108
styling. The way to reset the styling for seaborn is with a function called "set". 109
So "sns.set()" will reset the styling to the default styling. 110
So now if I press Shift+Enter I get the default parameters for the styling and we kind of get this look 111
right here. 112
This set function is a good function to remember if you've ever got like a little bit of a longer notebook 113
that we've got here and you might have written some code up above 114
that changes the styling of these charts and you want to do something different and your notebook 115
is behaving a little bit unexpectedly, so "sns.set()" resets the styling to default and 116
"sns.set_style()" allows us to choose kind of like a template style to use for the chart. 117
So there's a couple of templates to choose from, one of them is called white and then and this template 118
will make our chart look like so which is kind of what we had before. 119
But there's another template called white grid, which then have these grid lines to the chart like so. 120
Now of course there's also like dark red and dark and pressing Shift+Tab on this function will actually 121
show us what some of the options are - dark grid, dark, dark, white, ticks, 122
got a couple to choose from if we want. And you even got some examples on how you would use them. 123
So for example if you wanted to use ticks you can even provide the tick size as an additional argument. 124
All right. 125
So that's a little bit more detail on how you can control the aesthetics of your seaborn chart in your 126
notebook. 127
But the last thing I want to mention is that there is an additional template that you can mix and match 128
with say white grid or dark grid and these templates are called contexts, if you will. 129
So "sns.set_context()" will allow us to put in a template here for how this 130
chart is gonna be used. For example, a context might be "talk", and if I use that then you can see that the 131
font size is a lot larger and the dots are a little bit more clear. 132
So this is presumably because you want to present this chart somewhere. 133
Now there's a couple of other contexts as well. 134
You can use "notebook" which will make the chart look like this. 135
This is a template that's quite good if you're viewing this kind of stuff on a monitor and you're not 136
having to throw it up on a screen or like a presentation and pressing Shift+Enter on context shows 137
us that there's a couple of other options as well. 138
So there's "paper", there's "poster" and there's "talk" and "notebook" which we've already looked at. 139
I'm going to go with "talk" just to make it a little bit more readable on the video. The very last thing 140
I'm going to mention on the styling front is how you can get the similar sort of transparency and the color 141
that we have here on matplotlib. 142
So I'm going to show you how you can set that by supplying certain arguments. The color argument is pretty 143
straightforward, 144
so "color = 'indigo'" will give us a purple chart but when it comes to the transparency, you have to supply 145
the argument in a different way, because jointplot doesn't take an argument called alpha, that's only 146
for matplotlib. 147
Instead you have to go to the keyword arguments, so "joint_kws = " 148
and here you provide a dictionary, a Python dictionary, so that uses the curly braces notation and then 149
a key value pair - "alpha" for their key, colon, and then say 0.5 for the value. If I press Shift+ 150
Enter now we'll have the transparency applied to our data points. 151
So I hope you find it useful to see two different ways of generating this chart with different modules. 152
The first one matplotlib and the second one being seaborn. 153
Now there's a really cool thing I want to show you next. 154
And that's to do with the fact that this jointplot method is actually incredibly powerful. 155
So let me copy this cell and paste it below. 156
So I have two copies of it now and what I'm going to do is just for comparison I'm going to modify how these 157
data points are represented here. 158
So I'm going to go with a blue color to set them apart. 159
So I've got my blue one here, purple one here. 160
And then what I'm going to do is I'm going to show you this keyword argument that I'm going to change in the 161
quick documentation. So pressing Shift+Enter jointplot shows us that this keyword argument, "kind", 162
is set to scatter by default. 163
But there's other values that this can take, for example "kde", "reg", "resid" and "hex". 164
So we've actually got a choice between five different values. 165
Let me show you what one of them does. 166
I'm going to go ahead and delete this argument here where we've set our alpha value. 167
And if I press Shift+Enter you can see that we no longer have any alpha values on our chart. 168
But if I come in here and I change the kind to "hex", so writing "kind = 'hex'" 169
and then I hit Shift+Enter, we get the following. We get a chart that looks like this and what this chart is 170
doing is that it's aggregating the data points that all fall in a certain area and then it shades them 171
in depending on how many data points there are in that particular sector. So you're aggregating the data 172
points over like a little 2D area. 173
And again the shading gives us a very good idea of the density of the data points in that particular 174
part of the plot. 175
In other words we're aggregating the data in a hexagonal grid. 176
And I think this is a quite a beautiful visualization actually. 177
And it's one that you don't tend to see that often but it does remind me a little bit of a board game 178
called Settlers of Catan. 179
But maybe that's just me.