﻿1
00:00:00,133 --> 00:00:01,766
Hello my friends, and welcome.

2
00:00:01,766 --> 00:00:04,633
To part two of.
This implementation, where we're going to.

3
00:00:04,633 --> 00:00:07,300
Build together
the. Convolutional neural network.

4
00:00:07,300 --> 00:00:11,466
And more specifically, the whole
architecture of this new artificial.

5
00:00:11,466 --> 00:00:12,700
Neural network.

6
00:00:12,700 --> 00:00:14,933
All right.
So it's actually. Going to start.

7
00:00:14,933 --> 00:00:17,733
The same as
with our artificial neural network.

8
00:00:17,733 --> 00:00:19,933
Because a convolutional neural network.

9
00:00:19,933 --> 00:00:22,233
Is still a. Sequence of layers.

10
00:00:22,233 --> 00:00:24,400
Therefore we're going
to. Initialize here our.

11
00:00:24,400 --> 00:00:25,266
CNN with the.

12
00:00:25,266 --> 00:00:26,433
Same class which.

13
00:00:26,433 --> 00:00:28,966
Remember. Is sequential class.

14
00:00:28,966 --> 00:00:29,400
And that's.

15
00:00:29,400 --> 00:00:31,800
A first step where. We're going to not.

16
00:00:31,800 --> 00:00:35,600
Only call that sequential class,
but mostly create that.

17
00:00:35,700 --> 00:00:38,266
CNN. Variable which.

18
00:00:38,266 --> 00:00:39,800
Will represent exactly.

19
00:00:39,800 --> 00:00:42,166
This. Convolutional neural network.

20
00:00:42,166 --> 00:00:44,600
And the CNN variable. Will be created.

21
00:00:44,600 --> 00:00:48,433
Once again
as an instance of that sequential class.

22
00:00:48,600 --> 00:00:50,200
Which allows us to create an.

23
00:00:50,200 --> 00:00:51,033
Artificial neural.

24
00:00:51,033 --> 00:00:54,033
Network as a. Sequence of layers.

25
00:00:54,100 --> 00:00:56,200
So now remember how we can get.

26
00:00:56,200 --> 00:00:57,166
Access to this class.

27
00:00:57,166 --> 00:00:57,800
Well, first.

28
00:00:57,800 --> 00:00:58,133
We need to.

29
00:00:58,133 --> 00:01:00,933
Call TensorFlow which has a shortcut TF.

30
00:01:00,933 --> 00:01:02,100
From which we're going to.

31
00:01:02,100 --> 00:01:04,266
Call the Keras library.

32
00:01:04,266 --> 00:01:05,700
From which we're going to get.

33
00:01:05,700 --> 00:01:08,666
Access to the. Models. Module.

34
00:01:08,666 --> 00:01:10,200
And from which we're going to call.

35
00:01:10,200 --> 00:01:12,533
That sequential class.

36
00:01:12,533 --> 00:01:14,700
So exactly the same as before.

37
00:01:14,700 --> 00:01:17,133
And this initializes. Indeed our.

38
00:01:17,133 --> 00:01:18,466
CNN as. A.

39
00:01:18,466 --> 00:01:19,166
Sequence.

40
00:01:19,166 --> 00:01:22,166
Of layers
as opposed to a computational graph.

41
00:01:22,666 --> 00:01:24,300
Great. And now.

42
00:01:24,300 --> 00:01:26,500
Step by step we're going to use this.

43
00:01:26,500 --> 00:01:28,266
You know, add method.

44
00:01:28,266 --> 00:01:29,866
To add to. Different.

45
00:01:29,866 --> 00:01:31,166
Layers, whether they are.

46
00:01:31,166 --> 00:01:33,000
Convolutional. Layers or.

47
00:01:33,000 --> 00:01:34,500
Fully connected layers.

48
00:01:34,500 --> 00:01:36,366
And in the end the output layer.

49
00:01:36,366 --> 00:01:37,766
So we're going to use successively.

50
00:01:37,766 --> 00:01:41,166
Now the add method
starting with step one convolution.

51
00:01:41,466 --> 00:01:43,366
And so now and then. You could sell.

52
00:01:43,366 --> 00:01:44,933
Well first we'll take our.

53
00:01:44,933 --> 00:01:49,266
CNN object our convolutional
neural network from which we're going to.

54
00:01:49,266 --> 00:01:52,666
Call of course the add method to add

55
00:01:52,666 --> 00:01:55,666
our very first convolutional layer.

56
00:01:55,800 --> 00:01:56,100
All right.

57
00:01:56,100 --> 00:01:59,300
So what we want to add right now is
of course a convolutional layer.

58
00:01:59,300 --> 00:02:01,800
And once again
this convolutional layer. Will be.

59
00:02:01,800 --> 00:02:06,666
An. Object of a certain class
which is the conv 2d class.

60
00:02:06,966 --> 00:02:08,333
In this class, just after.

61
00:02:08,333 --> 00:02:08,733
Dense.

62
00:02:08,733 --> 00:02:10,600
Class
which allows us to build a fully connected

63
00:02:10,600 --> 00:02:15,333
layer, belongs to the same module which is
a layers module from the Keras library.

64
00:02:15,500 --> 00:02:17,266
From this time. TensorFlow.

65
00:02:17,266 --> 00:02:18,700
It's TensorFlow 2.0.

66
00:02:18,700 --> 00:02:22,500
So first we're going to get TensorFlow,
from which we're going to get access

67
00:02:22,500 --> 00:02:27,000
to the Keras library, from which we're
going to get access to the layers module.

68
00:02:27,466 --> 00:02:32,100
And there we go from which
we're going to call that conv 2d.

69
00:02:32,100 --> 00:02:35,266
There we go. Class comes to the class.

70
00:02:35,300 --> 00:02:38,100
Then we add some parentheses of course
because that's a class.

71
00:02:38,100 --> 00:02:41,100
And inside we have to input three.

72
00:02:41,100 --> 00:02:41,666
Important.

73
00:02:41,666 --> 00:02:45,200
Parameters
which are the filters which basically.

74
00:02:45,200 --> 00:02:45,900
Is the number.

75
00:02:45,900 --> 00:02:48,433
Of feature detectors you want to apply to.

76
00:02:48,433 --> 00:02:50,633
Your images,
you know, to. Detect features.

77
00:02:50,633 --> 00:02:53,633
And these are indeed
also called filters or kernels.

78
00:02:53,666 --> 00:02:54,133
Right.

79
00:02:54,133 --> 00:02:55,566
If we scroll. Down here.

80
00:02:55,566 --> 00:02:58,800
Actually you have all the info
of these parameters, right.

81
00:02:58,800 --> 00:03:00,500
Arguments filters.

82
00:03:00,500 --> 00:03:03,000
The number of output. Filters.
In the convolution.

83
00:03:03,000 --> 00:03:05,700
Well basically
that's the feature detectors.

84
00:03:05,700 --> 00:03:07,700
So that's our first argument.

85
00:03:07,700 --> 00:03:10,500
Then we will. Also specify. A kernel size.

86
00:03:10,500 --> 00:03:11,266
And actually.

87
00:03:11,266 --> 00:03:13,866
I've prepared some. Slides
that I made to. Show you.

88
00:03:13,866 --> 00:03:15,933
So basically this is the filter.

89
00:03:15,933 --> 00:03:16,600
And this.

90
00:03:16,600 --> 00:03:20,033
Filters parameter will tell us
how many feature detectors.

91
00:03:20,033 --> 00:03:21,933
We want, how many filters we want.

92
00:03:21,933 --> 00:03:22,700
And the kernel.

93
00:03:22,700 --> 00:03:26,166
Size is exactly,
you know, the size of that feature.

94
00:03:26,166 --> 00:03:28,700
Detector, meaning the number of. Rows,
which is also.

95
00:03:28,700 --> 00:03:30,233
The number of columns, because.

96
00:03:30,233 --> 00:03:32,366
It's is usually a square array.

97
00:03:32,366 --> 00:03:32,900
You know.

98
00:03:32,900 --> 00:03:35,066
So for example,
if we choose a size of three,

99
00:03:35,066 --> 00:03:37,700
which is actually the size,
we're going to choose, well,

100
00:03:37,700 --> 00:03:41,033
that means that the size of our feature
detector will be three by three.

101
00:03:41,166 --> 00:03:42,900
Okay. As simple as that.

102
00:03:42,900 --> 00:03:45,366
And then we have. Some other. Arguments.

103
00:03:45,366 --> 00:03:46,400
But no worries.

104
00:03:46,400 --> 00:03:47,733
We'll just keep the default value

105
00:03:47,733 --> 00:03:50,366
for the rest of them
because that will be just fine.

106
00:03:50,366 --> 00:03:52,233
However, of course. We're not.

107
00:03:52,233 --> 00:03:55,566
Going to keep the default value
of that activation parameter,

108
00:03:55,566 --> 00:03:57,333
which corresponds of course,
with the activation.

109
00:03:57,333 --> 00:03:59,866
Function,
because indeed, you know, on a general.

110
00:03:59,866 --> 00:04:01,100
Rule, as long as we haven't

111
00:04:01,100 --> 00:04:05,133
reached the output layer,
we rather want to get a rectifier.

112
00:04:05,133 --> 00:04:06,333
Activation function.

113
00:04:06,333 --> 00:04:07,800
And therefore for this one.

114
00:04:07,800 --> 00:04:08,666
Activation.

115
00:04:08,666 --> 00:04:11,500
We're going to choose once again
the ReLU parameters name

116
00:04:11,500 --> 00:04:14,500
which corresponds
to the rectifier activation function.

117
00:04:14,533 --> 00:04:15,600
And finally. We.

118
00:04:15,600 --> 00:04:18,566
Have one less argument
we have to input in which is not.

119
00:04:18,566 --> 00:04:19,666
Displayed here because.

120
00:04:19,666 --> 00:04:21,600
That's hidden by this one, but which you.

121
00:04:21,600 --> 00:04:23,100
Can actually see here.

122
00:04:23,100 --> 00:04:24,700
That's the input shape.

123
00:04:24,700 --> 00:04:25,133
When you.

124
00:04:25,133 --> 00:04:29,233
Add your very first layer,
whether it is a convolutional layer.

125
00:04:29,233 --> 00:04:30,300
Or dense layer.

126
00:04:30,300 --> 00:04:33,766
Well, you have to specify
the input shape of your.

127
00:04:33,766 --> 00:04:35,333
Inputs. And here.

128
00:04:35,333 --> 00:04:37,700
Since we were working with. Colored
images.

129
00:04:37,700 --> 00:04:39,500
Therefore in three. Dimensions, you know.

130
00:04:39,500 --> 00:04:42,733
Corresponding
to the RGB code of colors, and.

131
00:04:42,733 --> 00:04:43,133
Since we.

132
00:04:43,133 --> 00:04:46,133
Actually resized in parts one, our images

133
00:04:46,166 --> 00:04:49,933
down to 64 by 64, well, the input shape.

134
00:04:50,100 --> 00:04:51,233
Of our images will.

135
00:04:51,233 --> 00:04:54,066
Be 64, 64. And three.

136
00:04:54,066 --> 00:04:56,400
And if we were working
with black and white images.

137
00:04:56,400 --> 00:04:58,266
Here,
we would have. One instead. Of three.

138
00:04:58,266 --> 00:04:59,600
But we're working with.

139
00:04:59,600 --> 00:05:01,300
Colored images and therefore.

140
00:05:01,300 --> 00:05:04,233
That's what will be our input shape 6464.

141
00:05:04,233 --> 00:05:05,133
And three.

142
00:05:05,133 --> 00:05:05,500
All right.

143
00:05:05,500 --> 00:05:08,600
And that's the essential parameters
that we have to enter.

144
00:05:08,600 --> 00:05:09,500
So there we go.

145
00:05:09,500 --> 00:05:10,500
Let's enter them.

146
00:05:10,500 --> 00:05:13,500
Starting with the filters. Parameter.

147
00:05:13,800 --> 00:05:14,833
There we go.

148
00:05:14,833 --> 00:05:16,533
And now the question is well how many.

149
00:05:16,533 --> 00:05:18,166
Feature detectors we want.

150
00:05:18,166 --> 00:05:21,133
Well we're just going to go
for a classic architecture.

151
00:05:21,133 --> 00:05:23,200
You can find many. Architectures of.

152
00:05:23,200 --> 00:05:25,100
Convolutional neural networks online.

153
00:05:25,100 --> 00:05:26,833
We're going to choose. A classic. One,
which.

154
00:05:26,833 --> 00:05:29,700
I of course tried. On our images
and which.

155
00:05:29,700 --> 00:05:31,233
Turns out to work very well.

156
00:05:31,233 --> 00:05:35,733
And that architecture consists
of having 32 filters in that first.

157
00:05:36,033 --> 00:05:38,833
Convolutional layer and then another 30.

158
00:05:38,833 --> 00:05:41,600
Two. Filters
in that second convolutional layer.

159
00:05:41,600 --> 00:05:41,933
All right.

160
00:05:41,933 --> 00:05:45,233
So let's just choose here
filters equals 32.

161
00:05:45,466 --> 00:05:48,366
And feel. Free
of course to. Choose another.

162
00:05:48,366 --> 00:05:49,633
Parameter value.

163
00:05:49,633 --> 00:05:50,600
I remind that.

164
00:05:50,600 --> 00:05:51,800
This is an artist job.

165
00:05:51,800 --> 00:05:52,633
You're free to.

166
00:05:52,633 --> 00:05:55,266
Experiment any. Architecture you'd like.

167
00:05:55,266 --> 00:05:57,833
And you.
Might probably get some better results.

168
00:05:57,833 --> 00:06:00,400
Than the ones
we're about to get in the end.

169
00:06:00,400 --> 00:06:02,033
All right, so filters equal 32.

170
00:06:02,033 --> 00:06:03,566
That's for the first parameter.

171
00:06:03,566 --> 00:06:05,166
And now for the second parameter.

172
00:06:05,166 --> 00:06:05,966
As we said.

173
00:06:05,966 --> 00:06:08,966
It is going to be the kernel underscore.

174
00:06:09,300 --> 00:06:10,600
Size. This one.

175
00:06:10,600 --> 00:06:11,833
And as we said we want a.

176
00:06:11,833 --> 00:06:13,700
Three by three. Dimensions.

177
00:06:13,700 --> 00:06:16,500
And we only have. To specify three.

178
00:06:16,500 --> 00:06:18,600
And then as. We said. We want to make.

179
00:06:18,600 --> 00:06:21,533
Sure that we have a rectifier
activation. Function.

180
00:06:21,533 --> 00:06:23,033
So here. For this new.

181
00:06:23,033 --> 00:06:26,166
Activation parameter
we're going to choose in quotes.

182
00:06:26,166 --> 00:06:28,200
Well the ReLU activation.

183
00:06:28,200 --> 00:06:29,433
Function exactly the same as.

184
00:06:29,433 --> 00:06:30,200
Before.

185
00:06:30,200 --> 00:06:31,866
And finally for that.

186
00:06:31,866 --> 00:06:33,200
Last parameter.

187
00:06:33,200 --> 00:06:35,666
We have to specify. The input.

188
00:06:35,666 --> 00:06:38,733
Shape of our images, which we.

189
00:06:38,733 --> 00:06:41,500
Have to enter in
a. Pair of square brackets.

190
00:06:41,500 --> 00:06:42,566
And there we go.

191
00:06:42,566 --> 00:06:44,300
We resized. Our images to.

192
00:06:44,300 --> 00:06:47,666
Have the 64 by 64.

193
00:06:47,766 --> 00:06:49,466
Dimensions. And since we're.

194
00:06:49,466 --> 00:06:52,766
Working with images,
we have to add here three.

195
00:06:52,766 --> 00:06:54,533
Instead of. One. All right.

196
00:06:54,533 --> 00:06:57,033
That's our input shape. And that's. It.

197
00:06:57,033 --> 00:06:58,433
This adds.

198
00:06:58,433 --> 00:07:00,866
The convolutional. Layer to your.

199
00:07:00,866 --> 00:07:04,033
CNN so far initialized
as a sequence of layers.

200
00:07:04,700 --> 00:07:05,333
Good.

201
00:07:05,333 --> 00:07:06,666
So now we can move on.

202
00:07:06,666 --> 00:07:09,000
To step two pooling.

203
00:07:09,000 --> 00:07:10,466
So let's create a new code cell.

204
00:07:10,466 --> 00:07:12,333
And this consists of. Course of.

205
00:07:12,333 --> 00:07:14,033
Applying pooling. And more.

206
00:07:14,033 --> 00:07:15,000
Specifically we're going to.

207
00:07:15,000 --> 00:07:17,666
Apply max. Pooling just as you saw in.

208
00:07:17,666 --> 00:07:19,433
The intuition lectures.

209
00:07:19,433 --> 00:07:19,800
All right.

210
00:07:19,800 --> 00:07:22,800
So once. Again we need. To take our. CNN.

211
00:07:23,066 --> 00:07:24,300
Object from.

212
00:07:24,300 --> 00:07:26,900
Which we're going to call now
a new method.

213
00:07:26,900 --> 00:07:28,333
But is that really new.

214
00:07:28,333 --> 00:07:30,066
According to you. Do we. Have to call the.

215
00:07:30,066 --> 00:07:31,233
Ad method again?

216
00:07:31,233 --> 00:07:35,633
Well, yes we do actually
we're adding the pooling layers to.

217
00:07:35,633 --> 00:07:39,266
Our convolutional layer, you know,
as the next step in the sequence.

218
00:07:39,266 --> 00:07:40,133
Of layers.

219
00:07:40,133 --> 00:07:40,800
So there we go.

220
00:07:40,800 --> 00:07:42,800
We call the. Add method once again.

221
00:07:42,800 --> 00:07:43,800
And then inside.

222
00:07:43,800 --> 00:07:45,833
Well that's. Max. Pooling layer.

223
00:07:45,833 --> 00:07:46,700
Will once again.

224
00:07:46,700 --> 00:07:47,733
Be created.

225
00:07:47,733 --> 00:07:50,833
As an object, you know,
as an instance of a certain class.

226
00:07:50,966 --> 00:07:52,266
Which. Is called the max.

227
00:07:52,266 --> 00:07:53,966
Pool 2D class.

228
00:07:53,966 --> 00:07:56,366
And this class belongs to the same module.

229
00:07:56,366 --> 00:07:58,333
As before, the layers module.

230
00:07:58,333 --> 00:07:59,700
So there we go. We're actually. Going to.

231
00:07:59,700 --> 00:08:01,366
Take this.

232
00:08:01,366 --> 00:08:02,800
You know I'm going to.

233
00:08:02,800 --> 00:08:05,800
Copy this paste that right here.

234
00:08:05,933 --> 00:08:08,033
And then instead of
taking the cons to the.

235
00:08:08,033 --> 00:08:10,766
Class we're going to take this. Time
the max.

236
00:08:10,766 --> 00:08:13,766
Pool 2D class. Perfect.

237
00:08:13,800 --> 00:08:14,466
And inside.

238
00:08:14,466 --> 00:08:16,366
We have to specify two essential.

239
00:08:16,366 --> 00:08:18,433
Arguments which are. The pool size.

240
00:08:18,433 --> 00:08:20,933
So let me show you
what this corresponds to.

241
00:08:20,933 --> 00:08:23,133
So this is a whole.
Convolutional. Process.

242
00:08:23,133 --> 00:08:24,966
You know with the feature detector
applied.

243
00:08:24,966 --> 00:08:27,966
To the input image
resulting in having that.

244
00:08:28,166 --> 00:08:29,866
Created feature map.

245
00:08:29,866 --> 00:08:30,533
All right.

246
00:08:30,533 --> 00:08:31,633
So that's. All this.

247
00:08:31,633 --> 00:08:33,900
And there we go.
Now we move on to step two.

248
00:08:33,900 --> 00:08:37,933
So we have our feature map which
is the result of our previous convolution.

249
00:08:38,333 --> 00:08:40,466
And we apply max pooling to obtain this.

250
00:08:40,466 --> 00:08:41,766
Pooled feature map.

251
00:08:41,766 --> 00:08:44,733
You know that same feature
map of the convolutional layer.

252
00:08:44,733 --> 00:08:46,966
After we apply. Max pooling.

253
00:08:46,966 --> 00:08:48,166
And when we do.

254
00:08:48,166 --> 00:08:50,866
This, you know,
we have this little frame. Here.

255
00:08:50,866 --> 00:08:51,233
That will.

256
00:08:51,233 --> 00:08:52,133
Get the maximum.

257
00:08:52,133 --> 00:08:54,333
Pixel of the four cells.

258
00:08:54,333 --> 00:08:56,666
Inside,
you know, the. Four pixels. Inside.

259
00:08:56,666 --> 00:08:57,266
And that.

260
00:08:57,266 --> 00:08:59,400
Pool size parameter, which we're about.

261
00:08:59,400 --> 00:09:00,433
To enter here.

262
00:09:00,433 --> 00:09:02,666
Is exactly the size of that.

263
00:09:02,666 --> 00:09:05,000
Frame, you know,
which is once again a square.

264
00:09:05,000 --> 00:09:07,633
So we only have to specify the width.

265
00:09:07,633 --> 00:09:08,700
Or you know, the. Height.

266
00:09:08,700 --> 00:09:09,800
So basically.

267
00:09:09,800 --> 00:09:12,933
Here in this example. Two okay. And well.

268
00:09:12,933 --> 00:09:15,866
You know speaking of two
that's exactly the pool size.

269
00:09:15,866 --> 00:09:17,333
Which we're going to choose and which. Is.

270
00:09:17,333 --> 00:09:19,366
The one I recommend to. Work.

271
00:09:19,366 --> 00:09:21,566
With when applying max pooling.

272
00:09:21,566 --> 00:09:22,200
Okay.

273
00:09:22,200 --> 00:09:23,100
Then let's see.

274
00:09:23,100 --> 00:09:25,600
The other parameter that is important is.

275
00:09:25,600 --> 00:09:27,800
The strides okay. So interesting.

276
00:09:27,800 --> 00:09:30,366
Let's have a look at our slides
once again.

277
00:09:30,366 --> 00:09:32,766
So I'm going to move to the next slide
here.

278
00:09:32,766 --> 00:09:34,566
And let's. See by which.

279
00:09:34,566 --> 00:09:35,066
Number.

280
00:09:35,066 --> 00:09:38,566
Of pixels
is this frame shifted to the right.

281
00:09:38,900 --> 00:09:39,700
Let's see.

282
00:09:39,700 --> 00:09:40,066
All right.

283
00:09:40,066 --> 00:09:42,833
So it is actually shifted by two pixels.

284
00:09:42,833 --> 00:09:44,466
You know instead of going.

285
00:09:44,466 --> 00:09:47,566
From this frame to that.

286
00:09:47,566 --> 00:09:49,800
Frame. Which would be a slide of one.

287
00:09:49,800 --> 00:09:51,566
Well we went directly from this frame.

288
00:09:51,566 --> 00:09:53,766
To that one. Okay.

289
00:09:53,766 --> 00:09:55,800
And this makes sense
when applying max pooling,

290
00:09:55,800 --> 00:09:57,666
because we only want
to get the maximum of.

291
00:09:57,666 --> 00:10:00,166
Each square here of pixels. Right.

292
00:10:00,166 --> 00:10:03,066
So the. Recommended. Slide. Once again.

293
00:10:03,066 --> 00:10:07,366
Is two that indeed
we're sliding two by two pixels.

294
00:10:07,533 --> 00:10:09,333
And when we reach, you know that.

295
00:10:09,333 --> 00:10:10,366
Border of the feature.

296
00:10:10,366 --> 00:10:13,000
Map here,
you know with extra empty cells. Here.

297
00:10:13,000 --> 00:10:15,133
Well you can actually.
Choose two different ways.

298
00:10:15,133 --> 00:10:16,800
And that corresponds to that.

299
00:10:16,800 --> 00:10:18,666
Parameter which is the padding.

300
00:10:18,666 --> 00:10:21,700
The default value is valid,
but the other value is same.

301
00:10:21,966 --> 00:10:24,733
And well the difference is,
is that you know with a valid padding,

302
00:10:24,733 --> 00:10:27,633
you will just ignore. The other
two cells here.

303
00:10:27,633 --> 00:10:29,666
And with the same padding you will. Just.

304
00:10:29,666 --> 00:10:31,166
Add an extra column.

305
00:10:31,166 --> 00:10:33,900
With just fake pixels
that are equal to zero.

306
00:10:33,900 --> 00:10:34,633
But don't.

307
00:10:34,633 --> 00:10:36,033
Worry too much. About this padding.

308
00:10:36,033 --> 00:10:37,800
I tried actually with the two values.

309
00:10:37,800 --> 00:10:42,100
This doesn't change anything in the
final result, and so I just recommend to.

310
00:10:42,133 --> 00:10:43,900
Keep the default value.

311
00:10:43,900 --> 00:10:45,500
But the slide is important.

312
00:10:45,500 --> 00:10:48,400
And now we're sliding. Two by two okay.

313
00:10:48,400 --> 00:10:49,600
So let's do this.

314
00:10:49,600 --> 00:10:52,333
Let's enter
our parameter is. For that. Max.

315
00:10:52,333 --> 00:10:53,566
Pool to the class.

316
00:10:53,566 --> 00:10:56,833
The first one is as we said the pool size.

317
00:10:56,833 --> 00:10:57,500
There we. Go.

318
00:10:57,500 --> 00:10:58,333
And we want.

319
00:10:58,333 --> 00:11:01,700
To have exactly as in this slide
a two by two.

320
00:11:01,700 --> 00:11:02,833
Frame which we only have.

321
00:11:02,833 --> 00:11:05,833
To specify with the parameter input two.

322
00:11:06,000 --> 00:11:07,533
And then the second argument.

323
00:11:07,533 --> 00:11:10,033
Is just tried,
you know, the. Essential one.

324
00:11:10,033 --> 00:11:12,000
And we want to shift that frame.

325
00:11:12,000 --> 00:11:12,966
Every two pixels.

326
00:11:12,966 --> 00:11:15,866
And therefore
we're going to choose a straight of two.

327
00:11:15,866 --> 00:11:16,300
All right.

328
00:11:16,300 --> 00:11:19,300
And that apply successfully max pooling.

329
00:11:19,333 --> 00:11:21,900
As simple as that.
And you can just copy paste this.

330
00:11:21,900 --> 00:11:24,333
Line whenever you want to apply
max pooling to.

331
00:11:24,333 --> 00:11:25,666
Your. CNN.

332
00:11:25,666 --> 00:11:27,166
And actually speaking of which.

333
00:11:27,166 --> 00:11:31,233
Well, you know, now we want to add
a second convolutional layer and admire

334
00:11:31,233 --> 00:11:33,600
how I'm going to do. This so efficiently.

335
00:11:33,600 --> 00:11:36,000
So first I'm going to create a new code
cell.

336
00:11:36,000 --> 00:11:37,533
Then I'm going to.

337
00:11:37,533 --> 00:11:42,133
Take that cell, copy it,
then paste it right here.

338
00:11:42,466 --> 00:11:43,500
Then for the pooling.

339
00:11:43,500 --> 00:11:48,966
Take that cell copy it,
then paste it right below.

340
00:11:49,166 --> 00:11:49,733
And now.

341
00:11:49,733 --> 00:11:50,800
According to you, do we have.

342
00:11:50,800 --> 00:11:52,666
To change something here or can we.

343
00:11:52,666 --> 00:11:53,800
Leave it this way?

344
00:11:53,800 --> 00:11:55,466
Well we can't leave it this way.

345
00:11:55,466 --> 00:11:56,633
Actually, we just need to.

346
00:11:56,633 --> 00:12:02,266
Remove that input shape parameter because
this one is enter only when you add.

347
00:12:02,266 --> 00:12:03,766
Your very first layer.

348
00:12:03,766 --> 00:12:05,733
You know, to automatically connect.

349
00:12:05,733 --> 00:12:06,966
That first layer to the.

350
00:12:06,966 --> 00:12:08,866
Input layer which automatically adds.

351
00:12:08,866 --> 00:12:11,833
The input layer.
But here we are already with our.

352
00:12:11,833 --> 00:12:13,466
Second convolutional. Layer.

353
00:12:13,466 --> 00:12:16,166
So that's all good. We can just remove it.

354
00:12:17,566 --> 00:12:18,066
Good.

355
00:12:18,066 --> 00:12:18,766
And now.

356
00:12:18,766 --> 00:12:19,500
Perfect.

357
00:12:19,500 --> 00:12:22,566
All this adds a. Second convolutional.

358
00:12:22,566 --> 00:12:24,833
Layer. With max pooling applied.

359
00:12:24,833 --> 00:12:26,933
And we can now move on. To step three.

360
00:12:26,933 --> 00:12:29,666
Flattening
which will consist. Of of course.

361
00:12:29,666 --> 00:12:30,466
Flattening.

362
00:12:30,466 --> 00:12:32,000
The result of all these.

363
00:12:32,000 --> 00:12:35,600
Convolutions
and pooling into a one dimensional.

364
00:12:35,600 --> 00:12:38,000
Vector which. Will become the input.

365
00:12:38,000 --> 00:12:39,566
Of a future fully.

366
00:12:39,566 --> 00:12:41,366
Connected neural network. Just as.

367
00:12:41,366 --> 00:12:43,366
We built. In the previous section.

368
00:12:43,366 --> 00:12:43,666
All right.

369
00:12:43,666 --> 00:12:44,300
So let's do this.

370
00:12:44,300 --> 00:12:47,000
Let's implement
that. Step three. Flatten thing.

371
00:12:47,000 --> 00:12:48,733
And well as usual.

372
00:12:48,733 --> 00:12:49,400
You know we need to.

373
00:12:49,400 --> 00:12:50,200
Take our.

374
00:12:50,200 --> 00:12:52,500
CNN. Object from.

375
00:12:52,500 --> 00:12:53,400
Which we're going to.

376
00:12:53,400 --> 00:12:54,633
Call the ad.

377
00:12:54,633 --> 00:12:56,100
Method once again because.

378
00:12:56,100 --> 00:12:57,733
The way we're going to create that.

379
00:12:57,733 --> 00:12:59,100
Flattening. Layer is.

380
00:12:59,100 --> 00:13:03,600
Once again, by creating an instance of a
certain class, and that certain class.

381
00:13:03,600 --> 00:13:05,400
Is the flatten class.

382
00:13:05,400 --> 00:13:08,400
And, you know,
Keras will automatically understand that.

383
00:13:08,433 --> 00:13:09,900
It is the result of all.

384
00:13:09,900 --> 00:13:11,500
These convolutions and pooling.

385
00:13:11,500 --> 00:13:12,333
That will be.

386
00:13:12,333 --> 00:13:15,000
Flattened into this one
dimensional. Vector.

387
00:13:15,000 --> 00:13:17,566
So the only thing we have. To do
is just to specify.

388
00:13:17,566 --> 00:13:19,200
That we want to apply flattening.

389
00:13:19,200 --> 00:13:22,300
And to do this
we need to call once again the layers.

390
00:13:22,300 --> 00:13:25,266
Module by the Keras library
by. TensorFlow.

391
00:13:25,266 --> 00:13:28,066
From which we're going to call this.

392
00:13:28,066 --> 00:13:29,400
Time that.

393
00:13:29,400 --> 00:13:32,233
Flatten class. And good news.

394
00:13:32,233 --> 00:13:34,233
This class actually doesn't need to.

395
00:13:34,233 --> 00:13:36,133
Take anything as. Parameters.

396
00:13:36,133 --> 00:13:36,466
All right.

397
00:13:36,466 --> 00:13:36,966
So this.

398
00:13:36,966 --> 00:13:40,066
Simply implements step. Three. Flattening.

399
00:13:40,333 --> 00:13:42,833
And we can directly move on to. Step four.

400
00:13:42,833 --> 00:13:44,466
For connection.

401
00:13:44,466 --> 00:13:44,866
All right.

402
00:13:44,866 --> 00:13:46,333
So now it's your turn.

403
00:13:46,333 --> 00:13:48,000
You can actually. Do it yourself.

404
00:13:48,000 --> 00:13:51,133
So I would like you to please
press pause on the video.

405
00:13:51,300 --> 00:13:54,100
Because we are in the. Exact
same situation as before.

406
00:13:54,100 --> 00:13:56,400
You know, building a fully connected
neural network.

407
00:13:56,400 --> 00:13:58,400
So you know exactly how to do this.

408
00:13:58,400 --> 00:14:00,400
I would like you to add a new.

409
00:14:00,400 --> 00:14:02,000
Fully connected layer.

410
00:14:02,000 --> 00:14:04,266
To that flatten layer.

411
00:14:04,266 --> 00:14:07,433
Which is now nothing else
than a one dimensional vector.

412
00:14:07,433 --> 00:14:08,833
That. Will become the input.

413
00:14:08,833 --> 00:14:11,300
Of a fully. Connected neural network.

414
00:14:11,300 --> 00:14:11,933
So do this.

415
00:14:11,933 --> 00:14:15,000
First and will implement this
solution in a few seconds.

416
00:14:16,500 --> 00:14:17,100
Good.

417
00:14:17,100 --> 00:14:18,133
All right. Let's do this.

418
00:14:18,133 --> 00:14:19,200
So you know exactly.

419
00:14:19,200 --> 00:14:21,966
How to do this. First
we create a new code cell.

420
00:14:21,966 --> 00:14:25,800
Then we take once again
our CNN neural network

421
00:14:25,966 --> 00:14:30,233
from which we're going to add method,
because now we're about to add a.

422
00:14:30,233 --> 00:14:32,733
New layer
which is a fully connected layer.

423
00:14:32,733 --> 00:14:33,666
And which belongs.

424
00:14:33,666 --> 00:14:37,500
Still to that Tf.keras layers,
which I've already copied.

425
00:14:37,733 --> 00:14:39,400
So I can just paste that here.

426
00:14:39,400 --> 00:14:42,333
And this time. Take that. Dense.

427
00:14:42,333 --> 00:14:43,800
Class. Perfect.

428
00:14:43,800 --> 00:14:46,133
And now into some new parenthesis.

429
00:14:46,133 --> 00:14:47,600
I'm sure you figured out.

430
00:14:47,600 --> 00:14:49,333
What to enter as parameters.

431
00:14:49,333 --> 00:14:52,866
First, remember we have the units,
which is the number of hidden neurons

432
00:14:52,866 --> 00:14:54,000
you want to have into.

433
00:14:54,000 --> 00:14:56,100
This new. Fully connected layer.

434
00:14:56,100 --> 00:14:59,566
And since now we're actually dealing
with a more complex problem,

435
00:14:59,566 --> 00:15:03,333
you know, computer vision
is way more complex than data mining.

436
00:15:03,333 --> 00:15:05,133
Classic data set as before.

437
00:15:05,133 --> 00:15:07,600
Well, we're going to choose a larger.
Number. Of.

438
00:15:07,600 --> 00:15:09,000
Hidden. Neurons. We're going to choose.

439
00:15:09,000 --> 00:15:11,400
128 hidden neurons.

440
00:15:11,400 --> 00:15:13,966
However,
if you picked the. Same number. As.

441
00:15:13,966 --> 00:15:15,933
Before,
you know, in the previous section with our

442
00:15:15,933 --> 00:15:18,600
and well,
I'm sure that's totally fine and I'm sure.

443
00:15:18,600 --> 00:15:20,566
You'll get great results afterwards.

444
00:15:20,566 --> 00:15:23,033
But we might. Get a better accuracy
in. The end.

445
00:15:23,033 --> 00:15:25,200
With a larger. Number of neurons,
and therefore.

446
00:15:25,200 --> 00:15:28,200
Let's choose here units equals.

447
00:15:28,333 --> 00:15:31,133
128. Great.

448
00:15:31,133 --> 00:15:32,733
And now second argument.

449
00:15:32,733 --> 00:15:34,966
That's of course the activation function.

450
00:15:34,966 --> 00:15:36,666
And once again my recommendation is.

451
00:15:36,666 --> 00:15:40,166
That as long as you haven't
reached the final output layer I.

452
00:15:40,166 --> 00:15:40,666
Recommend to.

453
00:15:40,666 --> 00:15:43,466
Use a rectifier activation. Function.

454
00:15:43,466 --> 00:15:44,333
And that's exactly.

455
00:15:44,333 --> 00:15:47,333
What we're going to specify here
with this activation.

456
00:15:47,366 --> 00:15:48,566
Parameter.

457
00:15:48,566 --> 00:15:53,266
And remember the code name for
the rectifier activation function is ReLU.

458
00:15:53,900 --> 00:15:54,533
All right.

459
00:15:54,533 --> 00:15:56,866
And perfect this at our.

460
00:15:56,866 --> 00:15:59,033
Fully connected. Layer.

461
00:15:59,033 --> 00:16:01,800
And finally the step five.

462
00:16:01,800 --> 00:16:03,500
You see how we're being so efficient.

463
00:16:03,500 --> 00:16:06,100
Here in that step five
we need to add the final.

464
00:16:06,100 --> 00:16:08,800
Output layer which will still be fully.

465
00:16:08,800 --> 00:16:11,066
Connected. To that. Previous hidden layer.

466
00:16:11,066 --> 00:16:12,066
Therefore we're going to.

467
00:16:12,066 --> 00:16:14,100
Call once again the dense class.

468
00:16:14,100 --> 00:16:16,433
And therefore in the new. Code cell.

469
00:16:16,433 --> 00:16:17,766
I'm going to. Actually.

470
00:16:17,766 --> 00:16:20,766
You know. Take all this. Then.

471
00:16:20,866 --> 00:16:22,100
Based off here.

472
00:16:22,100 --> 00:16:23,500
And inside will only have to.

473
00:16:23,500 --> 00:16:28,233
Replace two things which are the values
of these two parameters, because indeed.

474
00:16:28,400 --> 00:16:32,366
The number of units, you know,
the number of neurons in the final.

475
00:16:32,366 --> 00:16:33,400
Output layer is.

476
00:16:33,400 --> 00:16:35,466
Definitely not 120. Eight.

477
00:16:35,466 --> 00:16:37,466
But you tell me.

478
00:16:37,466 --> 00:16:38,800
Well, it's actually, of course.

479
00:16:38,800 --> 00:16:40,966
One, it's exactly the same as before.

480
00:16:40,966 --> 00:16:43,300
We're doing binary classification.

481
00:16:43,300 --> 00:16:47,000
Therefore we only need one neuron
to encode that binary.

482
00:16:47,000 --> 00:16:50,066
Class zero. Or one or,
you know. Cat or dog.

483
00:16:50,300 --> 00:16:54,733
And therefore we only need one neuron
and for the activation function remember.

484
00:16:54,733 --> 00:16:57,666
That for the output layer.
It is not recommended to.

485
00:16:57,666 --> 00:16:58,466
Have a rectifier.

486
00:16:58,466 --> 00:16:59,633
Activation function.

487
00:16:59,633 --> 00:17:02,633
But rather sigmoid.

488
00:17:02,666 --> 00:17:04,166
Activation function.

489
00:17:04,166 --> 00:17:05,366
And that's because. Of course we're.

490
00:17:05,366 --> 00:17:06,866
Doing binary classification.

491
00:17:06,866 --> 00:17:07,800
Otherwise if we.

492
00:17:07,800 --> 00:17:10,033
Were doing multi-class. Classification.

493
00:17:10,033 --> 00:17:13,300
We would have remember
a softmax activation function.

494
00:17:13,600 --> 00:17:14,400
But there we go.

495
00:17:14,400 --> 00:17:17,333
This will add a great output layer
which will.

496
00:17:17,333 --> 00:17:19,933
Optimize the result in the end.

497
00:17:19,933 --> 00:17:22,500
And now. I have to say. Once again a.

498
00:17:22,500 --> 00:17:25,800
Huge congratulations to you
because there you go.

499
00:17:25,800 --> 00:17:26,700
You just built.

500
00:17:26,700 --> 00:17:28,766
A convolutional neural network.

501
00:17:28,766 --> 00:17:30,433
We're done with part two.

502
00:17:30,433 --> 00:17:32,133
We can already move on to part.

503
00:17:32,133 --> 00:17:34,000
Three. Training the CNN.

504
00:17:34,000 --> 00:17:35,833
And this will consist. Of course.

505
00:17:35,833 --> 00:17:39,700
Making this brain, you know,
this artificial brain with some eyes.

506
00:17:39,933 --> 00:17:41,066
Pretty smart to.

507
00:17:41,066 --> 00:17:44,066
Recognize. Cats or. Dogs in images.

508
00:17:44,366 --> 00:17:46,633
So now we just have a good break.

509
00:17:46,633 --> 00:17:48,566
So make sure. To recharge.

510
00:17:48,566 --> 00:17:51,533
In good energy for the next. Tutorial.
And as soon as.

511
00:17:51,533 --> 00:17:53,366
You're ready, well, let's smash together.

512
00:17:53,366 --> 00:17:55,833
Part three training the CNN.
