1 00:00:00,133 --> 00:00:01,766 Hello my friends, and welcome. 2 00:00:01,766 --> 00:00:04,633 To part two of. This implementation, where we're going to. 3 00:00:04,633 --> 00:00:07,300 Build together the. Convolutional neural network. 4 00:00:07,300 --> 00:00:11,466 And more specifically, the whole architecture of this new artificial. 5 00:00:11,466 --> 00:00:12,700 Neural network. 6 00:00:12,700 --> 00:00:14,933 All right. So it's actually. Going to start. 7 00:00:14,933 --> 00:00:17,733 The same as with our artificial neural network. 8 00:00:17,733 --> 00:00:19,933 Because a convolutional neural network. 9 00:00:19,933 --> 00:00:22,233 Is still a. Sequence of layers. 10 00:00:22,233 --> 00:00:24,400 Therefore we're going to. Initialize here our. 11 00:00:24,400 --> 00:00:25,266 CNN with the. 12 00:00:25,266 --> 00:00:26,433 Same class which. 13 00:00:26,433 --> 00:00:28,966 Remember. Is sequential class. 14 00:00:28,966 --> 00:00:29,400 And that's. 15 00:00:29,400 --> 00:00:31,800 A first step where. We're going to not. 16 00:00:31,800 --> 00:00:35,600 Only call that sequential class, but mostly create that. 17 00:00:35,700 --> 00:00:38,266 CNN. Variable which. 18 00:00:38,266 --> 00:00:39,800 Will represent exactly. 19 00:00:39,800 --> 00:00:42,166 This. Convolutional neural network. 20 00:00:42,166 --> 00:00:44,600 And the CNN variable. Will be created. 21 00:00:44,600 --> 00:00:48,433 Once again as an instance of that sequential class. 22 00:00:48,600 --> 00:00:50,200 Which allows us to create an. 23 00:00:50,200 --> 00:00:51,033 Artificial neural. 24 00:00:51,033 --> 00:00:54,033 Network as a. Sequence of layers. 25 00:00:54,100 --> 00:00:56,200 So now remember how we can get. 26 00:00:56,200 --> 00:00:57,166 Access to this class. 27 00:00:57,166 --> 00:00:57,800 Well, first. 28 00:00:57,800 --> 00:00:58,133 We need to. 29 00:00:58,133 --> 00:01:00,933 Call TensorFlow which has a shortcut TF. 30 00:01:00,933 --> 00:01:02,100 From which we're going to. 31 00:01:02,100 --> 00:01:04,266 Call the Keras library. 32 00:01:04,266 --> 00:01:05,700 From which we're going to get. 33 00:01:05,700 --> 00:01:08,666 Access to the. Models. Module. 34 00:01:08,666 --> 00:01:10,200 And from which we're going to call. 35 00:01:10,200 --> 00:01:12,533 That sequential class. 36 00:01:12,533 --> 00:01:14,700 So exactly the same as before. 37 00:01:14,700 --> 00:01:17,133 And this initializes. Indeed our. 38 00:01:17,133 --> 00:01:18,466 CNN as. A. 39 00:01:18,466 --> 00:01:19,166 Sequence. 40 00:01:19,166 --> 00:01:22,166 Of layers as opposed to a computational graph. 41 00:01:22,666 --> 00:01:24,300 Great. And now. 42 00:01:24,300 --> 00:01:26,500 Step by step we're going to use this. 43 00:01:26,500 --> 00:01:28,266 You know, add method. 44 00:01:28,266 --> 00:01:29,866 To add to. Different. 45 00:01:29,866 --> 00:01:31,166 Layers, whether they are. 46 00:01:31,166 --> 00:01:33,000 Convolutional. Layers or. 47 00:01:33,000 --> 00:01:34,500 Fully connected layers. 48 00:01:34,500 --> 00:01:36,366 And in the end the output layer. 49 00:01:36,366 --> 00:01:37,766 So we're going to use successively. 50 00:01:37,766 --> 00:01:41,166 Now the add method starting with step one convolution. 51 00:01:41,466 --> 00:01:43,366 And so now and then. You could sell. 52 00:01:43,366 --> 00:01:44,933 Well first we'll take our. 53 00:01:44,933 --> 00:01:49,266 CNN object our convolutional neural network from which we're going to. 54 00:01:49,266 --> 00:01:52,666 Call of course the add method to add 55 00:01:52,666 --> 00:01:55,666 our very first convolutional layer. 56 00:01:55,800 --> 00:01:56,100 All right. 57 00:01:56,100 --> 00:01:59,300 So what we want to add right now is of course a convolutional layer. 58 00:01:59,300 --> 00:02:01,800 And once again this convolutional layer. Will be. 59 00:02:01,800 --> 00:02:06,666 An. Object of a certain class which is the conv 2d class. 60 00:02:06,966 --> 00:02:08,333 In this class, just after. 61 00:02:08,333 --> 00:02:08,733 Dense. 62 00:02:08,733 --> 00:02:10,600 Class which allows us to build a fully connected 63 00:02:10,600 --> 00:02:15,333 layer, belongs to the same module which is a layers module from the Keras library. 64 00:02:15,500 --> 00:02:17,266 From this time. TensorFlow. 65 00:02:17,266 --> 00:02:18,700 It's TensorFlow 2.0. 66 00:02:18,700 --> 00:02:22,500 So first we're going to get TensorFlow, from which we're going to get access 67 00:02:22,500 --> 00:02:27,000 to the Keras library, from which we're going to get access to the layers module. 68 00:02:27,466 --> 00:02:32,100 And there we go from which we're going to call that conv 2d. 69 00:02:32,100 --> 00:02:35,266 There we go. Class comes to the class. 70 00:02:35,300 --> 00:02:38,100 Then we add some parentheses of course because that's a class. 71 00:02:38,100 --> 00:02:41,100 And inside we have to input three. 72 00:02:41,100 --> 00:02:41,666 Important. 73 00:02:41,666 --> 00:02:45,200 Parameters which are the filters which basically. 74 00:02:45,200 --> 00:02:45,900 Is the number. 75 00:02:45,900 --> 00:02:48,433 Of feature detectors you want to apply to. 76 00:02:48,433 --> 00:02:50,633 Your images, you know, to. Detect features. 77 00:02:50,633 --> 00:02:53,633 And these are indeed also called filters or kernels. 78 00:02:53,666 --> 00:02:54,133 Right. 79 00:02:54,133 --> 00:02:55,566 If we scroll. Down here. 80 00:02:55,566 --> 00:02:58,800 Actually you have all the info of these parameters, right. 81 00:02:58,800 --> 00:03:00,500 Arguments filters. 82 00:03:00,500 --> 00:03:03,000 The number of output. Filters. In the convolution. 83 00:03:03,000 --> 00:03:05,700 Well basically that's the feature detectors. 84 00:03:05,700 --> 00:03:07,700 So that's our first argument. 85 00:03:07,700 --> 00:03:10,500 Then we will. Also specify. A kernel size. 86 00:03:10,500 --> 00:03:11,266 And actually. 87 00:03:11,266 --> 00:03:13,866 I've prepared some. Slides that I made to. Show you. 88 00:03:13,866 --> 00:03:15,933 So basically this is the filter. 89 00:03:15,933 --> 00:03:16,600 And this. 90 00:03:16,600 --> 00:03:20,033 Filters parameter will tell us how many feature detectors. 91 00:03:20,033 --> 00:03:21,933 We want, how many filters we want. 92 00:03:21,933 --> 00:03:22,700 And the kernel. 93 00:03:22,700 --> 00:03:26,166 Size is exactly, you know, the size of that feature. 94 00:03:26,166 --> 00:03:28,700 Detector, meaning the number of. Rows, which is also. 95 00:03:28,700 --> 00:03:30,233 The number of columns, because. 96 00:03:30,233 --> 00:03:32,366 It's is usually a square array. 97 00:03:32,366 --> 00:03:32,900 You know. 98 00:03:32,900 --> 00:03:35,066 So for example, if we choose a size of three, 99 00:03:35,066 --> 00:03:37,700 which is actually the size, we're going to choose, well, 100 00:03:37,700 --> 00:03:41,033 that means that the size of our feature detector will be three by three. 101 00:03:41,166 --> 00:03:42,900 Okay. As simple as that. 102 00:03:42,900 --> 00:03:45,366 And then we have. Some other. Arguments. 103 00:03:45,366 --> 00:03:46,400 But no worries. 104 00:03:46,400 --> 00:03:47,733 We'll just keep the default value 105 00:03:47,733 --> 00:03:50,366 for the rest of them because that will be just fine. 106 00:03:50,366 --> 00:03:52,233 However, of course. We're not. 107 00:03:52,233 --> 00:03:55,566 Going to keep the default value of that activation parameter, 108 00:03:55,566 --> 00:03:57,333 which corresponds of course, with the activation. 109 00:03:57,333 --> 00:03:59,866 Function, because indeed, you know, on a general. 110 00:03:59,866 --> 00:04:01,100 Rule, as long as we haven't 111 00:04:01,100 --> 00:04:05,133 reached the output layer, we rather want to get a rectifier. 112 00:04:05,133 --> 00:04:06,333 Activation function. 113 00:04:06,333 --> 00:04:07,800 And therefore for this one. 114 00:04:07,800 --> 00:04:08,666 Activation. 115 00:04:08,666 --> 00:04:11,500 We're going to choose once again the ReLU parameters name 116 00:04:11,500 --> 00:04:14,500 which corresponds to the rectifier activation function. 117 00:04:14,533 --> 00:04:15,600 And finally. We. 118 00:04:15,600 --> 00:04:18,566 Have one less argument we have to input in which is not. 119 00:04:18,566 --> 00:04:19,666 Displayed here because. 120 00:04:19,666 --> 00:04:21,600 That's hidden by this one, but which you. 121 00:04:21,600 --> 00:04:23,100 Can actually see here. 122 00:04:23,100 --> 00:04:24,700 That's the input shape. 123 00:04:24,700 --> 00:04:25,133 When you. 124 00:04:25,133 --> 00:04:29,233 Add your very first layer, whether it is a convolutional layer. 125 00:04:29,233 --> 00:04:30,300 Or dense layer. 126 00:04:30,300 --> 00:04:33,766 Well, you have to specify the input shape of your. 127 00:04:33,766 --> 00:04:35,333 Inputs. And here. 128 00:04:35,333 --> 00:04:37,700 Since we were working with. Colored images. 129 00:04:37,700 --> 00:04:39,500 Therefore in three. Dimensions, you know. 130 00:04:39,500 --> 00:04:42,733 Corresponding to the RGB code of colors, and. 131 00:04:42,733 --> 00:04:43,133 Since we. 132 00:04:43,133 --> 00:04:46,133 Actually resized in parts one, our images 133 00:04:46,166 --> 00:04:49,933 down to 64 by 64, well, the input shape. 134 00:04:50,100 --> 00:04:51,233 Of our images will. 135 00:04:51,233 --> 00:04:54,066 Be 64, 64. And three. 136 00:04:54,066 --> 00:04:56,400 And if we were working with black and white images. 137 00:04:56,400 --> 00:04:58,266 Here, we would have. One instead. Of three. 138 00:04:58,266 --> 00:04:59,600 But we're working with. 139 00:04:59,600 --> 00:05:01,300 Colored images and therefore. 140 00:05:01,300 --> 00:05:04,233 That's what will be our input shape 6464. 141 00:05:04,233 --> 00:05:05,133 And three. 142 00:05:05,133 --> 00:05:05,500 All right. 143 00:05:05,500 --> 00:05:08,600 And that's the essential parameters that we have to enter. 144 00:05:08,600 --> 00:05:09,500 So there we go. 145 00:05:09,500 --> 00:05:10,500 Let's enter them. 146 00:05:10,500 --> 00:05:13,500 Starting with the filters. Parameter. 147 00:05:13,800 --> 00:05:14,833 There we go. 148 00:05:14,833 --> 00:05:16,533 And now the question is well how many. 149 00:05:16,533 --> 00:05:18,166 Feature detectors we want. 150 00:05:18,166 --> 00:05:21,133 Well we're just going to go for a classic architecture. 151 00:05:21,133 --> 00:05:23,200 You can find many. Architectures of. 152 00:05:23,200 --> 00:05:25,100 Convolutional neural networks online. 153 00:05:25,100 --> 00:05:26,833 We're going to choose. A classic. One, which. 154 00:05:26,833 --> 00:05:29,700 I of course tried. On our images and which. 155 00:05:29,700 --> 00:05:31,233 Turns out to work very well. 156 00:05:31,233 --> 00:05:35,733 And that architecture consists of having 32 filters in that first. 157 00:05:36,033 --> 00:05:38,833 Convolutional layer and then another 30. 158 00:05:38,833 --> 00:05:41,600 Two. Filters in that second convolutional layer. 159 00:05:41,600 --> 00:05:41,933 All right. 160 00:05:41,933 --> 00:05:45,233 So let's just choose here filters equals 32. 161 00:05:45,466 --> 00:05:48,366 And feel. Free of course to. Choose another. 162 00:05:48,366 --> 00:05:49,633 Parameter value. 163 00:05:49,633 --> 00:05:50,600 I remind that. 164 00:05:50,600 --> 00:05:51,800 This is an artist job. 165 00:05:51,800 --> 00:05:52,633 You're free to. 166 00:05:52,633 --> 00:05:55,266 Experiment any. Architecture you'd like. 167 00:05:55,266 --> 00:05:57,833 And you. Might probably get some better results. 168 00:05:57,833 --> 00:06:00,400 Than the ones we're about to get in the end. 169 00:06:00,400 --> 00:06:02,033 All right, so filters equal 32. 170 00:06:02,033 --> 00:06:03,566 That's for the first parameter. 171 00:06:03,566 --> 00:06:05,166 And now for the second parameter. 172 00:06:05,166 --> 00:06:05,966 As we said. 173 00:06:05,966 --> 00:06:08,966 It is going to be the kernel underscore. 174 00:06:09,300 --> 00:06:10,600 Size. This one. 175 00:06:10,600 --> 00:06:11,833 And as we said we want a. 176 00:06:11,833 --> 00:06:13,700 Three by three. Dimensions. 177 00:06:13,700 --> 00:06:16,500 And we only have. To specify three. 178 00:06:16,500 --> 00:06:18,600 And then as. We said. We want to make. 179 00:06:18,600 --> 00:06:21,533 Sure that we have a rectifier activation. Function. 180 00:06:21,533 --> 00:06:23,033 So here. For this new. 181 00:06:23,033 --> 00:06:26,166 Activation parameter we're going to choose in quotes. 182 00:06:26,166 --> 00:06:28,200 Well the ReLU activation. 183 00:06:28,200 --> 00:06:29,433 Function exactly the same as. 184 00:06:29,433 --> 00:06:30,200 Before. 185 00:06:30,200 --> 00:06:31,866 And finally for that. 186 00:06:31,866 --> 00:06:33,200 Last parameter. 187 00:06:33,200 --> 00:06:35,666 We have to specify. The input. 188 00:06:35,666 --> 00:06:38,733 Shape of our images, which we. 189 00:06:38,733 --> 00:06:41,500 Have to enter in a. Pair of square brackets. 190 00:06:41,500 --> 00:06:42,566 And there we go. 191 00:06:42,566 --> 00:06:44,300 We resized. Our images to. 192 00:06:44,300 --> 00:06:47,666 Have the 64 by 64. 193 00:06:47,766 --> 00:06:49,466 Dimensions. And since we're. 194 00:06:49,466 --> 00:06:52,766 Working with images, we have to add here three. 195 00:06:52,766 --> 00:06:54,533 Instead of. One. All right. 196 00:06:54,533 --> 00:06:57,033 That's our input shape. And that's. It. 197 00:06:57,033 --> 00:06:58,433 This adds. 198 00:06:58,433 --> 00:07:00,866 The convolutional. Layer to your. 199 00:07:00,866 --> 00:07:04,033 CNN so far initialized as a sequence of layers. 200 00:07:04,700 --> 00:07:05,333 Good. 201 00:07:05,333 --> 00:07:06,666 So now we can move on. 202 00:07:06,666 --> 00:07:09,000 To step two pooling. 203 00:07:09,000 --> 00:07:10,466 So let's create a new code cell. 204 00:07:10,466 --> 00:07:12,333 And this consists of. Course of. 205 00:07:12,333 --> 00:07:14,033 Applying pooling. And more. 206 00:07:14,033 --> 00:07:15,000 Specifically we're going to. 207 00:07:15,000 --> 00:07:17,666 Apply max. Pooling just as you saw in. 208 00:07:17,666 --> 00:07:19,433 The intuition lectures. 209 00:07:19,433 --> 00:07:19,800 All right. 210 00:07:19,800 --> 00:07:22,800 So once. Again we need. To take our. CNN. 211 00:07:23,066 --> 00:07:24,300 Object from. 212 00:07:24,300 --> 00:07:26,900 Which we're going to call now a new method. 213 00:07:26,900 --> 00:07:28,333 But is that really new. 214 00:07:28,333 --> 00:07:30,066 According to you. Do we. Have to call the. 215 00:07:30,066 --> 00:07:31,233 Ad method again? 216 00:07:31,233 --> 00:07:35,633 Well, yes we do actually we're adding the pooling layers to. 217 00:07:35,633 --> 00:07:39,266 Our convolutional layer, you know, as the next step in the sequence. 218 00:07:39,266 --> 00:07:40,133 Of layers. 219 00:07:40,133 --> 00:07:40,800 So there we go. 220 00:07:40,800 --> 00:07:42,800 We call the. Add method once again. 221 00:07:42,800 --> 00:07:43,800 And then inside. 222 00:07:43,800 --> 00:07:45,833 Well that's. Max. Pooling layer. 223 00:07:45,833 --> 00:07:46,700 Will once again. 224 00:07:46,700 --> 00:07:47,733 Be created. 225 00:07:47,733 --> 00:07:50,833 As an object, you know, as an instance of a certain class. 226 00:07:50,966 --> 00:07:52,266 Which. Is called the max. 227 00:07:52,266 --> 00:07:53,966 Pool 2D class. 228 00:07:53,966 --> 00:07:56,366 And this class belongs to the same module. 229 00:07:56,366 --> 00:07:58,333 As before, the layers module. 230 00:07:58,333 --> 00:07:59,700 So there we go. We're actually. Going to. 231 00:07:59,700 --> 00:08:01,366 Take this. 232 00:08:01,366 --> 00:08:02,800 You know I'm going to. 233 00:08:02,800 --> 00:08:05,800 Copy this paste that right here. 234 00:08:05,933 --> 00:08:08,033 And then instead of taking the cons to the. 235 00:08:08,033 --> 00:08:10,766 Class we're going to take this. Time the max. 236 00:08:10,766 --> 00:08:13,766 Pool 2D class. Perfect. 237 00:08:13,800 --> 00:08:14,466 And inside. 238 00:08:14,466 --> 00:08:16,366 We have to specify two essential. 239 00:08:16,366 --> 00:08:18,433 Arguments which are. The pool size. 240 00:08:18,433 --> 00:08:20,933 So let me show you what this corresponds to. 241 00:08:20,933 --> 00:08:23,133 So this is a whole. Convolutional. Process. 242 00:08:23,133 --> 00:08:24,966 You know with the feature detector applied. 243 00:08:24,966 --> 00:08:27,966 To the input image resulting in having that. 244 00:08:28,166 --> 00:08:29,866 Created feature map. 245 00:08:29,866 --> 00:08:30,533 All right. 246 00:08:30,533 --> 00:08:31,633 So that's. All this. 247 00:08:31,633 --> 00:08:33,900 And there we go. Now we move on to step two. 248 00:08:33,900 --> 00:08:37,933 So we have our feature map which is the result of our previous convolution. 249 00:08:38,333 --> 00:08:40,466 And we apply max pooling to obtain this. 250 00:08:40,466 --> 00:08:41,766 Pooled feature map. 251 00:08:41,766 --> 00:08:44,733 You know that same feature map of the convolutional layer. 252 00:08:44,733 --> 00:08:46,966 After we apply. Max pooling. 253 00:08:46,966 --> 00:08:48,166 And when we do. 254 00:08:48,166 --> 00:08:50,866 This, you know, we have this little frame. Here. 255 00:08:50,866 --> 00:08:51,233 That will. 256 00:08:51,233 --> 00:08:52,133 Get the maximum. 257 00:08:52,133 --> 00:08:54,333 Pixel of the four cells. 258 00:08:54,333 --> 00:08:56,666 Inside, you know, the. Four pixels. Inside. 259 00:08:56,666 --> 00:08:57,266 And that. 260 00:08:57,266 --> 00:08:59,400 Pool size parameter, which we're about. 261 00:08:59,400 --> 00:09:00,433 To enter here. 262 00:09:00,433 --> 00:09:02,666 Is exactly the size of that. 263 00:09:02,666 --> 00:09:05,000 Frame, you know, which is once again a square. 264 00:09:05,000 --> 00:09:07,633 So we only have to specify the width. 265 00:09:07,633 --> 00:09:08,700 Or you know, the. Height. 266 00:09:08,700 --> 00:09:09,800 So basically. 267 00:09:09,800 --> 00:09:12,933 Here in this example. Two okay. And well. 268 00:09:12,933 --> 00:09:15,866 You know speaking of two that's exactly the pool size. 269 00:09:15,866 --> 00:09:17,333 Which we're going to choose and which. Is. 270 00:09:17,333 --> 00:09:19,366 The one I recommend to. Work. 271 00:09:19,366 --> 00:09:21,566 With when applying max pooling. 272 00:09:21,566 --> 00:09:22,200 Okay. 273 00:09:22,200 --> 00:09:23,100 Then let's see. 274 00:09:23,100 --> 00:09:25,600 The other parameter that is important is. 275 00:09:25,600 --> 00:09:27,800 The strides okay. So interesting. 276 00:09:27,800 --> 00:09:30,366 Let's have a look at our slides once again. 277 00:09:30,366 --> 00:09:32,766 So I'm going to move to the next slide here. 278 00:09:32,766 --> 00:09:34,566 And let's. See by which. 279 00:09:34,566 --> 00:09:35,066 Number. 280 00:09:35,066 --> 00:09:38,566 Of pixels is this frame shifted to the right. 281 00:09:38,900 --> 00:09:39,700 Let's see. 282 00:09:39,700 --> 00:09:40,066 All right. 283 00:09:40,066 --> 00:09:42,833 So it is actually shifted by two pixels. 284 00:09:42,833 --> 00:09:44,466 You know instead of going. 285 00:09:44,466 --> 00:09:47,566 From this frame to that. 286 00:09:47,566 --> 00:09:49,800 Frame. Which would be a slide of one. 287 00:09:49,800 --> 00:09:51,566 Well we went directly from this frame. 288 00:09:51,566 --> 00:09:53,766 To that one. Okay. 289 00:09:53,766 --> 00:09:55,800 And this makes sense when applying max pooling, 290 00:09:55,800 --> 00:09:57,666 because we only want to get the maximum of. 291 00:09:57,666 --> 00:10:00,166 Each square here of pixels. Right. 292 00:10:00,166 --> 00:10:03,066 So the. Recommended. Slide. Once again. 293 00:10:03,066 --> 00:10:07,366 Is two that indeed we're sliding two by two pixels. 294 00:10:07,533 --> 00:10:09,333 And when we reach, you know that. 295 00:10:09,333 --> 00:10:10,366 Border of the feature. 296 00:10:10,366 --> 00:10:13,000 Map here, you know with extra empty cells. Here. 297 00:10:13,000 --> 00:10:15,133 Well you can actually. Choose two different ways. 298 00:10:15,133 --> 00:10:16,800 And that corresponds to that. 299 00:10:16,800 --> 00:10:18,666 Parameter which is the padding. 300 00:10:18,666 --> 00:10:21,700 The default value is valid, but the other value is same. 301 00:10:21,966 --> 00:10:24,733 And well the difference is, is that you know with a valid padding, 302 00:10:24,733 --> 00:10:27,633 you will just ignore. The other two cells here. 303 00:10:27,633 --> 00:10:29,666 And with the same padding you will. Just. 304 00:10:29,666 --> 00:10:31,166 Add an extra column. 305 00:10:31,166 --> 00:10:33,900 With just fake pixels that are equal to zero. 306 00:10:33,900 --> 00:10:34,633 But don't. 307 00:10:34,633 --> 00:10:36,033 Worry too much. About this padding. 308 00:10:36,033 --> 00:10:37,800 I tried actually with the two values. 309 00:10:37,800 --> 00:10:42,100 This doesn't change anything in the final result, and so I just recommend to. 310 00:10:42,133 --> 00:10:43,900 Keep the default value. 311 00:10:43,900 --> 00:10:45,500 But the slide is important. 312 00:10:45,500 --> 00:10:48,400 And now we're sliding. Two by two okay. 313 00:10:48,400 --> 00:10:49,600 So let's do this. 314 00:10:49,600 --> 00:10:52,333 Let's enter our parameter is. For that. Max. 315 00:10:52,333 --> 00:10:53,566 Pool to the class. 316 00:10:53,566 --> 00:10:56,833 The first one is as we said the pool size. 317 00:10:56,833 --> 00:10:57,500 There we. Go. 318 00:10:57,500 --> 00:10:58,333 And we want. 319 00:10:58,333 --> 00:11:01,700 To have exactly as in this slide a two by two. 320 00:11:01,700 --> 00:11:02,833 Frame which we only have. 321 00:11:02,833 --> 00:11:05,833 To specify with the parameter input two. 322 00:11:06,000 --> 00:11:07,533 And then the second argument. 323 00:11:07,533 --> 00:11:10,033 Is just tried, you know, the. Essential one. 324 00:11:10,033 --> 00:11:12,000 And we want to shift that frame. 325 00:11:12,000 --> 00:11:12,966 Every two pixels. 326 00:11:12,966 --> 00:11:15,866 And therefore we're going to choose a straight of two. 327 00:11:15,866 --> 00:11:16,300 All right. 328 00:11:16,300 --> 00:11:19,300 And that apply successfully max pooling. 329 00:11:19,333 --> 00:11:21,900 As simple as that. And you can just copy paste this. 330 00:11:21,900 --> 00:11:24,333 Line whenever you want to apply max pooling to. 331 00:11:24,333 --> 00:11:25,666 Your. CNN. 332 00:11:25,666 --> 00:11:27,166 And actually speaking of which. 333 00:11:27,166 --> 00:11:31,233 Well, you know, now we want to add a second convolutional layer and admire 334 00:11:31,233 --> 00:11:33,600 how I'm going to do. This so efficiently. 335 00:11:33,600 --> 00:11:36,000 So first I'm going to create a new code cell. 336 00:11:36,000 --> 00:11:37,533 Then I'm going to. 337 00:11:37,533 --> 00:11:42,133 Take that cell, copy it, then paste it right here. 338 00:11:42,466 --> 00:11:43,500 Then for the pooling. 339 00:11:43,500 --> 00:11:48,966 Take that cell copy it, then paste it right below. 340 00:11:49,166 --> 00:11:49,733 And now. 341 00:11:49,733 --> 00:11:50,800 According to you, do we have. 342 00:11:50,800 --> 00:11:52,666 To change something here or can we. 343 00:11:52,666 --> 00:11:53,800 Leave it this way? 344 00:11:53,800 --> 00:11:55,466 Well we can't leave it this way. 345 00:11:55,466 --> 00:11:56,633 Actually, we just need to. 346 00:11:56,633 --> 00:12:02,266 Remove that input shape parameter because this one is enter only when you add. 347 00:12:02,266 --> 00:12:03,766 Your very first layer. 348 00:12:03,766 --> 00:12:05,733 You know, to automatically connect. 349 00:12:05,733 --> 00:12:06,966 That first layer to the. 350 00:12:06,966 --> 00:12:08,866 Input layer which automatically adds. 351 00:12:08,866 --> 00:12:11,833 The input layer. But here we are already with our. 352 00:12:11,833 --> 00:12:13,466 Second convolutional. Layer. 353 00:12:13,466 --> 00:12:16,166 So that's all good. We can just remove it. 354 00:12:17,566 --> 00:12:18,066 Good. 355 00:12:18,066 --> 00:12:18,766 And now. 356 00:12:18,766 --> 00:12:19,500 Perfect. 357 00:12:19,500 --> 00:12:22,566 All this adds a. Second convolutional. 358 00:12:22,566 --> 00:12:24,833 Layer. With max pooling applied. 359 00:12:24,833 --> 00:12:26,933 And we can now move on. To step three. 360 00:12:26,933 --> 00:12:29,666 Flattening which will consist. Of of course. 361 00:12:29,666 --> 00:12:30,466 Flattening. 362 00:12:30,466 --> 00:12:32,000 The result of all these. 363 00:12:32,000 --> 00:12:35,600 Convolutions and pooling into a one dimensional. 364 00:12:35,600 --> 00:12:38,000 Vector which. Will become the input. 365 00:12:38,000 --> 00:12:39,566 Of a future fully. 366 00:12:39,566 --> 00:12:41,366 Connected neural network. Just as. 367 00:12:41,366 --> 00:12:43,366 We built. In the previous section. 368 00:12:43,366 --> 00:12:43,666 All right. 369 00:12:43,666 --> 00:12:44,300 So let's do this. 370 00:12:44,300 --> 00:12:47,000 Let's implement that. Step three. Flatten thing. 371 00:12:47,000 --> 00:12:48,733 And well as usual. 372 00:12:48,733 --> 00:12:49,400 You know we need to. 373 00:12:49,400 --> 00:12:50,200 Take our. 374 00:12:50,200 --> 00:12:52,500 CNN. Object from. 375 00:12:52,500 --> 00:12:53,400 Which we're going to. 376 00:12:53,400 --> 00:12:54,633 Call the ad. 377 00:12:54,633 --> 00:12:56,100 Method once again because. 378 00:12:56,100 --> 00:12:57,733 The way we're going to create that. 379 00:12:57,733 --> 00:12:59,100 Flattening. Layer is. 380 00:12:59,100 --> 00:13:03,600 Once again, by creating an instance of a certain class, and that certain class. 381 00:13:03,600 --> 00:13:05,400 Is the flatten class. 382 00:13:05,400 --> 00:13:08,400 And, you know, Keras will automatically understand that. 383 00:13:08,433 --> 00:13:09,900 It is the result of all. 384 00:13:09,900 --> 00:13:11,500 These convolutions and pooling. 385 00:13:11,500 --> 00:13:12,333 That will be. 386 00:13:12,333 --> 00:13:15,000 Flattened into this one dimensional. Vector. 387 00:13:15,000 --> 00:13:17,566 So the only thing we have. To do is just to specify. 388 00:13:17,566 --> 00:13:19,200 That we want to apply flattening. 389 00:13:19,200 --> 00:13:22,300 And to do this we need to call once again the layers. 390 00:13:22,300 --> 00:13:25,266 Module by the Keras library by. TensorFlow. 391 00:13:25,266 --> 00:13:28,066 From which we're going to call this. 392 00:13:28,066 --> 00:13:29,400 Time that. 393 00:13:29,400 --> 00:13:32,233 Flatten class. And good news. 394 00:13:32,233 --> 00:13:34,233 This class actually doesn't need to. 395 00:13:34,233 --> 00:13:36,133 Take anything as. Parameters. 396 00:13:36,133 --> 00:13:36,466 All right. 397 00:13:36,466 --> 00:13:36,966 So this. 398 00:13:36,966 --> 00:13:40,066 Simply implements step. Three. Flattening. 399 00:13:40,333 --> 00:13:42,833 And we can directly move on to. Step four. 400 00:13:42,833 --> 00:13:44,466 For connection. 401 00:13:44,466 --> 00:13:44,866 All right. 402 00:13:44,866 --> 00:13:46,333 So now it's your turn. 403 00:13:46,333 --> 00:13:48,000 You can actually. Do it yourself. 404 00:13:48,000 --> 00:13:51,133 So I would like you to please press pause on the video. 405 00:13:51,300 --> 00:13:54,100 Because we are in the. Exact same situation as before. 406 00:13:54,100 --> 00:13:56,400 You know, building a fully connected neural network. 407 00:13:56,400 --> 00:13:58,400 So you know exactly how to do this. 408 00:13:58,400 --> 00:14:00,400 I would like you to add a new. 409 00:14:00,400 --> 00:14:02,000 Fully connected layer. 410 00:14:02,000 --> 00:14:04,266 To that flatten layer. 411 00:14:04,266 --> 00:14:07,433 Which is now nothing else than a one dimensional vector. 412 00:14:07,433 --> 00:14:08,833 That. Will become the input. 413 00:14:08,833 --> 00:14:11,300 Of a fully. Connected neural network. 414 00:14:11,300 --> 00:14:11,933 So do this. 415 00:14:11,933 --> 00:14:15,000 First and will implement this solution in a few seconds. 416 00:14:16,500 --> 00:14:17,100 Good. 417 00:14:17,100 --> 00:14:18,133 All right. Let's do this. 418 00:14:18,133 --> 00:14:19,200 So you know exactly. 419 00:14:19,200 --> 00:14:21,966 How to do this. First we create a new code cell. 420 00:14:21,966 --> 00:14:25,800 Then we take once again our CNN neural network 421 00:14:25,966 --> 00:14:30,233 from which we're going to add method, because now we're about to add a. 422 00:14:30,233 --> 00:14:32,733 New layer which is a fully connected layer. 423 00:14:32,733 --> 00:14:33,666 And which belongs. 424 00:14:33,666 --> 00:14:37,500 Still to that Tf.keras layers, which I've already copied. 425 00:14:37,733 --> 00:14:39,400 So I can just paste that here. 426 00:14:39,400 --> 00:14:42,333 And this time. Take that. Dense. 427 00:14:42,333 --> 00:14:43,800 Class. Perfect. 428 00:14:43,800 --> 00:14:46,133 And now into some new parenthesis. 429 00:14:46,133 --> 00:14:47,600 I'm sure you figured out. 430 00:14:47,600 --> 00:14:49,333 What to enter as parameters. 431 00:14:49,333 --> 00:14:52,866 First, remember we have the units, which is the number of hidden neurons 432 00:14:52,866 --> 00:14:54,000 you want to have into. 433 00:14:54,000 --> 00:14:56,100 This new. Fully connected layer. 434 00:14:56,100 --> 00:14:59,566 And since now we're actually dealing with a more complex problem, 435 00:14:59,566 --> 00:15:03,333 you know, computer vision is way more complex than data mining. 436 00:15:03,333 --> 00:15:05,133 Classic data set as before. 437 00:15:05,133 --> 00:15:07,600 Well, we're going to choose a larger. Number. Of. 438 00:15:07,600 --> 00:15:09,000 Hidden. Neurons. We're going to choose. 439 00:15:09,000 --> 00:15:11,400 128 hidden neurons. 440 00:15:11,400 --> 00:15:13,966 However, if you picked the. Same number. As. 441 00:15:13,966 --> 00:15:15,933 Before, you know, in the previous section with our 442 00:15:15,933 --> 00:15:18,600 and well, I'm sure that's totally fine and I'm sure. 443 00:15:18,600 --> 00:15:20,566 You'll get great results afterwards. 444 00:15:20,566 --> 00:15:23,033 But we might. Get a better accuracy in. The end. 445 00:15:23,033 --> 00:15:25,200 With a larger. Number of neurons, and therefore. 446 00:15:25,200 --> 00:15:28,200 Let's choose here units equals. 447 00:15:28,333 --> 00:15:31,133 128. Great. 448 00:15:31,133 --> 00:15:32,733 And now second argument. 449 00:15:32,733 --> 00:15:34,966 That's of course the activation function. 450 00:15:34,966 --> 00:15:36,666 And once again my recommendation is. 451 00:15:36,666 --> 00:15:40,166 That as long as you haven't reached the final output layer I. 452 00:15:40,166 --> 00:15:40,666 Recommend to. 453 00:15:40,666 --> 00:15:43,466 Use a rectifier activation. Function. 454 00:15:43,466 --> 00:15:44,333 And that's exactly. 455 00:15:44,333 --> 00:15:47,333 What we're going to specify here with this activation. 456 00:15:47,366 --> 00:15:48,566 Parameter. 457 00:15:48,566 --> 00:15:53,266 And remember the code name for the rectifier activation function is ReLU. 458 00:15:53,900 --> 00:15:54,533 All right. 459 00:15:54,533 --> 00:15:56,866 And perfect this at our. 460 00:15:56,866 --> 00:15:59,033 Fully connected. Layer. 461 00:15:59,033 --> 00:16:01,800 And finally the step five. 462 00:16:01,800 --> 00:16:03,500 You see how we're being so efficient. 463 00:16:03,500 --> 00:16:06,100 Here in that step five we need to add the final. 464 00:16:06,100 --> 00:16:08,800 Output layer which will still be fully. 465 00:16:08,800 --> 00:16:11,066 Connected. To that. Previous hidden layer. 466 00:16:11,066 --> 00:16:12,066 Therefore we're going to. 467 00:16:12,066 --> 00:16:14,100 Call once again the dense class. 468 00:16:14,100 --> 00:16:16,433 And therefore in the new. Code cell. 469 00:16:16,433 --> 00:16:17,766 I'm going to. Actually. 470 00:16:17,766 --> 00:16:20,766 You know. Take all this. Then. 471 00:16:20,866 --> 00:16:22,100 Based off here. 472 00:16:22,100 --> 00:16:23,500 And inside will only have to. 473 00:16:23,500 --> 00:16:28,233 Replace two things which are the values of these two parameters, because indeed. 474 00:16:28,400 --> 00:16:32,366 The number of units, you know, the number of neurons in the final. 475 00:16:32,366 --> 00:16:33,400 Output layer is. 476 00:16:33,400 --> 00:16:35,466 Definitely not 120. Eight. 477 00:16:35,466 --> 00:16:37,466 But you tell me. 478 00:16:37,466 --> 00:16:38,800 Well, it's actually, of course. 479 00:16:38,800 --> 00:16:40,966 One, it's exactly the same as before. 480 00:16:40,966 --> 00:16:43,300 We're doing binary classification. 481 00:16:43,300 --> 00:16:47,000 Therefore we only need one neuron to encode that binary. 482 00:16:47,000 --> 00:16:50,066 Class zero. Or one or, you know. Cat or dog. 483 00:16:50,300 --> 00:16:54,733 And therefore we only need one neuron and for the activation function remember. 484 00:16:54,733 --> 00:16:57,666 That for the output layer. It is not recommended to. 485 00:16:57,666 --> 00:16:58,466 Have a rectifier. 486 00:16:58,466 --> 00:16:59,633 Activation function. 487 00:16:59,633 --> 00:17:02,633 But rather sigmoid. 488 00:17:02,666 --> 00:17:04,166 Activation function. 489 00:17:04,166 --> 00:17:05,366 And that's because. Of course we're. 490 00:17:05,366 --> 00:17:06,866 Doing binary classification. 491 00:17:06,866 --> 00:17:07,800 Otherwise if we. 492 00:17:07,800 --> 00:17:10,033 Were doing multi-class. Classification. 493 00:17:10,033 --> 00:17:13,300 We would have remember a softmax activation function. 494 00:17:13,600 --> 00:17:14,400 But there we go. 495 00:17:14,400 --> 00:17:17,333 This will add a great output layer which will. 496 00:17:17,333 --> 00:17:19,933 Optimize the result in the end. 497 00:17:19,933 --> 00:17:22,500 And now. I have to say. Once again a. 498 00:17:22,500 --> 00:17:25,800 Huge congratulations to you because there you go. 499 00:17:25,800 --> 00:17:26,700 You just built. 500 00:17:26,700 --> 00:17:28,766 A convolutional neural network. 501 00:17:28,766 --> 00:17:30,433 We're done with part two. 502 00:17:30,433 --> 00:17:32,133 We can already move on to part. 503 00:17:32,133 --> 00:17:34,000 Three. Training the CNN. 504 00:17:34,000 --> 00:17:35,833 And this will consist. Of course. 505 00:17:35,833 --> 00:17:39,700 Making this brain, you know, this artificial brain with some eyes. 506 00:17:39,933 --> 00:17:41,066 Pretty smart to. 507 00:17:41,066 --> 00:17:44,066 Recognize. Cats or. Dogs in images. 508 00:17:44,366 --> 00:17:46,633 So now we just have a good break. 509 00:17:46,633 --> 00:17:48,566 So make sure. To recharge. 510 00:17:48,566 --> 00:17:51,533 In good energy for the next. Tutorial. And as soon as. 511 00:17:51,533 --> 00:17:53,366 You're ready, well, let's smash together. 512 00:17:53,366 --> 00:17:55,833 Part three training the CNN.