0
1
00:00:00,960 --> 00:00:03,960
Let me scroll back up to where we had our cost.
1

2
00:00:04,020 --> 00:00:06,070
We're going to add two loops here.
2

3
00:00:06,210 --> 00:00:09,730
The first one will be "for i in range()",
3

4
00:00:09,870 --> 00:00:11,250
and then how far do we want to go?
4

5
00:00:11,730 --> 00:00:15,220
We want to go across all the theta values.
5

6
00:00:15,240 --> 00:00:23,250
So it's gonna be "nr_thetas" between the parentheses and then colon and we're going to
6

7
00:00:23,250 --> 00:00:32,550
do the same thing for our second loop we're going to say "for j in range(nr_thetas
7

8
00:00:33,120 --> 00:00:34,580
):".
8

9
00:00:34,590 --> 00:00:37,080
Now what do we want to do inside the loop?
9

10
00:00:37,080 --> 00:00:39,360
Let's do something very unimaginative
10

11
00:00:39,360 --> 00:00:50,310
first. Let's print out all the values that we have inside our plot_t0 variable and we're
11

12
00:00:50,310 --> 00:00:52,050
going to print them out one by one.
12

13
00:00:52,050 --> 00:00:59,310
So we're gonna have "plot_t0[]" and now we have a choice - we can go row by
13

14
00:00:59,640 --> 00:01:01,500
column or column by row.
14

15
00:01:01,510 --> 00:01:13,890
So if we go by column then we'd have i in the first set of square brackets and then j in the second
15

16
00:01:14,220 --> 00:01:15,810
set of square brackets.
16

17
00:01:15,810 --> 00:01:18,320
Let's see what happens.
17

18
00:01:18,320 --> 00:01:19,550
There we go.
18

19
00:01:19,550 --> 00:01:21,860
This is us printing out 25
19

20
00:01:21,860 --> 00:01:30,370
values, the 25 values that are stored inside our theta matrix, our theta five by five
20

21
00:01:30,380 --> 00:01:35,460
array. The other way, say j and i,
21

22
00:01:35,480 --> 00:01:38,330
our output would look like this.
22

23
00:01:38,330 --> 00:01:44,990
In this case we're staying in the same column, namely the first one, and we're looking at all the first
23

24
00:01:45,020 --> 00:01:46,190
elements.
24

25
00:01:46,340 --> 00:01:51,820
If we have i and then j, then we're doing one row at a time.
25

26
00:01:51,830 --> 00:01:53,800
So this is the first element in the row.
26

27
00:01:53,810 --> 00:01:55,420
This the second element in the row.
27

28
00:01:55,430 --> 00:01:56,540
This is the third one.
28

29
00:01:56,570 --> 00:01:58,730
This is the fourth one. This is the fifth one.
29

30
00:01:58,730 --> 00:02:08,210
And then this is the first element in the second row. On the other hand, if we have j and then i, then
30

31
00:02:08,210 --> 00:02:14,270
we would be looking at the first element in the first row, the first element in the second row, the first
31

32
00:02:14,270 --> 00:02:17,120
element in the third row and so on.
32

33
00:02:17,120 --> 00:02:24,410
So the order in which we supply the indices here to our two dimensional array it matters in the sense
33

34
00:02:24,410 --> 00:02:30,870
that it determines in which order our loop will go through the array.
34

35
00:02:31,130 --> 00:02:36,190
But regardless in which order you go through you still hit up all the values.
35

36
00:02:36,230 --> 00:02:42,650
So I'm going to comment this out and now I'm going to write the code to calculate our estimated y values.
36

37
00:02:42,770 --> 00:02:49,840
So y_hat is equal to the first value in our theta_0 array.
37

38
00:02:49,900 --> 00:03:00,140
It's going to be "plot_t0[i]
38

39
00:03:00,140 --> 00:03:11,050
[j]". So this is gonna be our first t_0 value. I'm going to add "plot_t1
39

40
00:03:11,080 --> 00:03:13,830
[i]
40

41
00:03:13,940 --> 00:03:19,390
[j]*x_5".
41

42
00:03:19,730 --> 00:03:29,500
So this is gonna calculate our predicted y values for the first set of theta parameters. And then what
42

43
00:03:29,500 --> 00:03:36,660
we're gonna say is we're going to say that the plot_cost matrix is going to update and it's going
43

44
00:03:36,660 --> 00:03:39,130
to update in the same position.
44

45
00:03:39,200 --> 00:03:43,940
It's gonna be updating in [i][j].
45

46
00:03:43,950 --> 00:03:52,100
And it's gonna be equal to the output from our lovely mean squared error function,
46

47
00:03:52,160 --> 00:04:05,030
so "mse(y_5, y_hat)", so we're gonna calculate our y_hat
47

48
00:04:05,030 --> 00:04:15,440
value for these theta parameters and then based on that we're going to calculate our mean squared
48

49
00:04:15,470 --> 00:04:25,020
error and we're gonna update our two dimensional cost array. And we're gonna do that for all of the theta
49

50
00:04:25,020 --> 00:04:28,470
values so we can plot them later on.
50

51
00:04:28,470 --> 00:04:32,700
So there's a lot of calculations going on here but there's only four lines of code.
51

52
00:04:32,730 --> 00:04:37,130
This is very, very terse but very powerful at the same time.
52

53
00:04:37,140 --> 00:04:44,040
Let's take a look at what the plot_cost variable actually looks like after the loop has
53

54
00:04:44,040 --> 00:04:45,410
run.
54

55
00:04:45,420 --> 00:04:46,420
Let's take a look.
55

56
00:04:46,600 --> 00:04:54,830
I'm going to hit Shift+Enter and here we see that it is a five by five matrix.
56

57
00:04:54,900 --> 00:04:58,330
I can prove this even though it's hard to see because of the lines wrapping.
57

58
00:04:58,380 --> 00:05:05,880
If I say "plot_cost.shape", you can see here it's five by five and we can see that any for
58

59
00:05:05,880 --> 00:05:14,180
each element we've got a mean squared error for a particular combination of thetas. Let's print this
59

60
00:05:14,180 --> 00:05:18,680
stuff out so that we can verify what's going on when we make changes later on.
60

61
00:05:18,860 --> 00:05:28,340
So I'm going to say "print", and then the string 'Shape of plot_t0', comma, "plot_t0
61

62
00:05:28,340 --> 00:05:36,390
.shape", then I'm going to copy this print statement, paste it two more times and in the second one
62

63
00:05:36,400 --> 00:05:47,460
way to change it to t1, and then the third one I'm going to change it to "plot_cost" so this
63

64
00:05:47,460 --> 00:05:55,170
is printing out the dimensions of all the three arrays that we're using and at the moment they're all
64

65
00:05:55,290 --> 00:06:00,310
five by five. Good stuff.
65

66
00:06:00,370 --> 00:06:03,470
Now it's time to throw the stuff up onto a chart.
66

67
00:06:03,670 --> 00:06:04,510
A 3D chart,
67

68
00:06:04,510 --> 00:06:05,480
no less.
68

69
00:06:05,480 --> 00:06:13,510
Let's take a look. I'm going to add a comment in this new cell "Plotting MSE".
69

70
00:06:13,510 --> 00:06:23,020
And I'm going to do our old trick with the figure, I'm going to say "fig=plt.figure(
70

71
00:06:25,140 --> 00:06:34,750
figsize=[16,12]" and then you've got some axes that I've got
71

72
00:06:34,750 --> 00:06:35,240
to create.
72

73
00:06:35,320 --> 00:06:43,870
So it's going to be "fig.gca", get current axes, and it's gonna be for a 3D projection, so "projection="
73

74
00:06:44,470 --> 00:06:52,540
and then the string "3d", lowercase d. And our chart's gonna have three axes, it's gonna have
74

75
00:06:53,140 --> 00:06:56,800
a Y label on X label and a Z label.
75

76
00:06:56,830 --> 00:07:08,890
So I'm going to see "ax.set_xlabel('Theta 0')". So
76

77
00:07:08,890 --> 00:07:18,640
theta 0 is gonna be on our x axis and it's gonna be at font size say 20.
77

78
00:07:18,640 --> 00:07:23,890
Copy that whole line, paste it two more times and then I will make some changes.
78

79
00:07:23,890 --> 00:07:25,870
Second one is gonna set the Y label.
79

80
00:07:25,930 --> 00:07:32,050
The third one is gonna set the Z label and then the string is gonna be updated to theta 1 and then
80

81
00:07:32,050 --> 00:07:33,700
for the z axis,
81

82
00:07:33,700 --> 00:07:36,280
this is gonna be it's gonna be our cost, right?
82

83
00:07:36,280 --> 00:07:39,460
"Cost - MSE".
83

84
00:07:40,750 --> 00:07:49,310
So we've created our plot, set the size of our figure, then we've grabbed our axes object from our figure,
84

85
00:07:49,970 --> 00:07:56,600
we've set some of the attributes of the axes using the set_xlabel, set_ylabel
85

86
00:07:56,600 --> 00:08:07,370
and "set_zlabel" functions and now we do the best part - "ax.plot_surface()",
86

87
00:08:07,770 --> 00:08:16,700
and inside the parentheses, we have to supply three two dimensional arrays; the first one was gonna be
87

88
00:08:16,700 --> 00:08:17,900
our theta 0,
88

89
00:08:17,990 --> 00:08:20,060
so this is on our x axis.
89

90
00:08:20,060 --> 00:08:29,000
The second one it's gonna be plot_t1, and this is gonna be for our y axis. And the third
90

91
00:08:29,000 --> 00:08:37,310
one is gonna be the cost that we calculated in our nested loop so plot_cpst.
91

92
00:08:37,310 --> 00:08:44,140
And finally we're gonna say "plot.show()" with some parentheses at the end.
92

93
00:08:44,150 --> 00:08:48,910
Now we should be able to get a visualization of all that work we just did.
93

94
00:08:49,040 --> 00:08:54,710
And before I hit Shift+Enter, let me just quickly check the code that I've written,
94

95
00:08:54,710 --> 00:09:01,540
see if I've made any typos I'm spotting one of them here in setting the figure we've got to use our 
95

96
00:09:01,550 --> 00:09:06,200
matplotlib object which is "plt.figure" not "plot.figure".
96

97
00:09:06,830 --> 00:09:14,810
So plt was our matplotlib object and plt.figure grabs our figure object.
97

98
00:09:14,810 --> 00:09:15,730
I think we're good now.
98

99
00:09:15,740 --> 00:09:23,840
So let me hit Shift+Enter and see what this looks like. Voila! It looks like..
99

100
00:09:23,890 --> 00:09:32,390
so I realize this might be a little bit underwhelming and it looks like a video game from the 1990s.
100

101
00:09:32,390 --> 00:09:39,320
This is I think the kind of graphics they had but we can change this very, very easily. If I scroll
101

102
00:09:39,320 --> 00:09:49,850
back up and increase the number of data points we're using from 5 to say 200 and I update
102

103
00:09:50,030 --> 00:09:56,960
this cell then we're gonna generate a lot more theta zeros and a lot more theta ones with our 
103

104
00:09:56,960 --> 00:09:59,080
linspace function as such.
104

105
00:09:59,210 --> 00:10:07,710
I'm going to hit Shift+Enter on this cell. I can see that we're now dealing with a 200 by 200 array.
105

106
00:10:07,910 --> 00:10:11,680
So a lot more data points that we're plotting now.
106

107
00:10:11,690 --> 00:10:18,890
And when I replot our surface by hitting Shift+Enter to update this cell then it will look something
107

108
00:10:19,010 --> 00:10:20,780
like this.
108

109
00:10:20,780 --> 00:10:22,400
Brilliant.
109

110
00:10:22,400 --> 00:10:25,250
How are you liking the navy blue?
110

111
00:10:25,250 --> 00:10:31,580
I think we've got to add a splash of color to this to make it a bit more interesting.
111

112
00:10:31,580 --> 00:10:38,180
I'm going to go back up to my plot_surface function and I'm going to make use of this color map that
112

113
00:10:38,180 --> 00:10:39,900
we imported earlier.
113

114
00:10:39,940 --> 00:10:42,960
It's gonna be an extra argument I'm gonna supply here,
114

115
00:10:42,980 --> 00:10:48,890
"cmap" and it's going to be set equal to our cm module,
115

116
00:10:48,890 --> 00:10:55,240
put a dot after it and then I'm going to use "hot" and hit Shift+Enter.
116

117
00:10:55,650 --> 00:10:58,190
Now this is starting to look pretty good.
117

118
00:10:58,220 --> 00:11:02,450
I personally think this surface doesn't just look hot,
118

119
00:11:02,450 --> 00:11:05,860
you could say it almost looks super hot.
119

120
00:11:06,760 --> 00:11:13,160
But before I lose you to this video game, let's compare our function and our calculations to our regression
120

121
00:11:13,220 --> 00:11:20,220
estimates because we can actually pull out the lowest mean squared error from our plot variable,
121

122
00:11:20,240 --> 00:11:20,740
right?
122

123
00:11:20,780 --> 00:11:34,970
So if I come down here and I write print and then write "Min value of plot_cost", comma 
123

124
00:11:35,030 --> 00:11:35,440
"plot_
124

125
00:11:35,440 --> 00:11:37,810
cost.min",
125

126
00:11:37,820 --> 00:11:45,470
and then some parentheses, I can actually pull out the lowest mean squared error from our surface plot
126

127
00:11:45,590 --> 00:11:54,290
that we just used for graphing and that is 0.94838.
127

128
00:11:54,410 --> 00:12:03,140
But hold on, I hear you say, what are the theta 0 and theta 1 values associated with that cost that
128

129
00:12:03,140 --> 00:12:04,650
we just pulled out?
129

130
00:12:04,730 --> 00:12:08,180
I mean, after all, we care about our parameter values,
130

131
00:12:08,210 --> 00:12:08,850
right?
131

132
00:12:08,870 --> 00:12:17,950
So how can we grab the row and the column indices which correspond to this minimum cost?
132

133
00:12:18,500 --> 00:12:21,070
And for that numpy has you covered.
133

134
00:12:21,380 --> 00:12:24,830
It's got a function called "unravel_index".
134

135
00:12:24,830 --> 00:12:36,140
So I'm going to say "ij_min=np.unravel_index()" and it needs two
135

136
00:12:36,140 --> 00:12:37,110
inputs -
136

137
00:12:37,160 --> 00:12:44,920
it needs the indices and that's going to be equal to "plot_
137

138
00:12:44,960 --> 00:12:45,920
cost."
138

139
00:12:46,130 --> 00:12:47,770
and then "argmin",
139

140
00:12:47,840 --> 00:12:56,320
so this will output the indices from our 2D array where the mean squared
140

141
00:12:56,330 --> 00:12:58,580
error is at its lowest.
141

142
00:12:59,000 --> 00:13:00,500
And then we have a comma afterwards.
142

143
00:13:00,530 --> 00:13:06,910
And we supply that second input to the unravel_index function and that's the dimensions,
143

144
00:13:06,940 --> 00:13:17,020
"dims" of the 2D array that we are supplying, so we can say "dims = plot_cost.
144

145
00:13:17,230 --> 00:13:21,670
shape" and this will give us the row and the column index.
145

146
00:13:21,700 --> 00:13:23,050
Let's print this out.
146

147
00:13:23,140 --> 00:13:28,540
"print('Min occurs at (i,
147

148
00:13:28,630 --> 00:13:29,320
j)
148

149
00:13:29,470 --> 00:13:40,570
:', ij_min". So the minimum occurs at the 111th throw and 
149

150
00:13:40,570 --> 00:13:42,430
91st column.
150

151
00:13:42,430 --> 00:13:48,760
So this is where after running that nested for loop the mean squared error was the lowest. The theta 0
151

152
00:13:48,760 --> 00:13:55,450
value associated with row number 111 and column 91 is as follows:
152

153
00:13:55,450 --> 00:14:09,340
"print('Min MSE for Theta 0 at plot_t0[111][91]',
153

154
00:14:09,370 --> 00:14:20,300
plot_t0[111][91])". Hitting Shift+
154

155
00:14:20,330 --> 00:14:27,980
Enter we can see that this is fiendishly close to what the scikit learn package calculated. Now of course
155

156
00:14:28,550 --> 00:14:35,750
using a nested for loop is not better, it's very much a brute force method but I kind of wanted
156

157
00:14:35,750 --> 00:14:44,090
to show you how you can use these 2d arrays and how you can work with them and how you can access particular
157

158
00:14:44,120 --> 00:14:50,840
values using the row and the column indices. So let's do that one more time, but this time we'll grab the
158

159
00:14:50,840 --> 00:15:01,550
theta 1 value just for good measure so it'll be "print('Min MSE for Theta 1 at plot_
159

160
00:15:01,700 --> 00:15:11,570
t1[111][91]', plot_t1
160

161
00:15:11,660 --> 00:15:15,080
[111][91])".
161

162
00:15:15,250 --> 00:15:18,670
So this should be close to 1.2.
162

163
00:15:18,710 --> 00:15:23,540
Let's take a look. And it is, it's 1.23.
163

164
00:15:23,850 --> 00:15:24,960
Brilliant.
164

165
00:15:24,960 --> 00:15:32,850
Now all that's left to do is run our gradient descent algorithm on this cost function and then we've
165

166
00:15:32,850 --> 00:15:34,500
come full circle.
166

167
00:15:34,710 --> 00:15:36,550
I'll see you in the next lesson.
167

168
00:15:37,050 --> 00:15:43,050
Super hot is actually one of the most interesting games I've played recently by the way. So I just had
168

169
00:15:43,050 --> 00:15:50,370
to mention it. It's this first person shooter but with a twist, time only moves forward when you move
169

170
00:15:50,730 --> 00:15:56,690
and there's no music it's just the sound effect of glass shattering. That's right. That's it.
170

171
00:15:56,690 --> 00:15:57,250
So yeah.
171

172
00:15:57,300 --> 00:16:02,450
If you're curious, check it out on Steam if you want a break from programming.
172

173
00:16:02,460 --> 00:16:06,030
I do hope you give Super Hot a try and let me know what you think.
173

174
00:16:06,240 --> 00:16:12,060
Just the don't do what I did and play it really late at night because if you're anything like me you'll
174

175
00:16:12,060 --> 00:16:14,350
get too excited and you'll have trouble sleeping.
175

176
00:16:14,350 --> 00:16:16,800
So yeah take care.