1
00:00:00,900 --> 00:00:09,790
In the last video we use data augmentation techniques to increase our validation accuracy to a 2 to

2
00:00:10,080 --> 00:00:13,840
3 percent in this video.

3
00:00:13,840 --> 00:00:22,630
We will use DG sixteen model architecture to further increase our validation accuracy about 90 percent

4
00:00:26,720 --> 00:00:36,170
vs 60 was the runner up of 2014 i.e. unless we are C competition.

5
00:00:36,280 --> 00:00:43,320
The problem is statement of that competition was to categorize millions of pictures and two thousand

6
00:00:43,330 --> 00:00:55,020
different categories the pictures but of animals humans etc. and the categories were of different animal

7
00:00:55,140 --> 00:01:01,890
species and many other so the problem we are trying to solve.

8
00:01:01,890 --> 00:01:10,230
You can consider it as a subset of or is null two thousand forty eight unless VRC completion data

9
00:01:13,210 --> 00:01:22,070
as we discuss in order to be lectured we can use conventional part of this three grain model architectures

10
00:01:22,580 --> 00:01:23,900
and other problems

11
00:01:27,110 --> 00:01:36,620
these models consist of two parts convolution base and then a fully connected neural network base the

12
00:01:36,620 --> 00:01:45,650
constitutional base is used to identify features from the images and then the fully connected neural

13
00:01:45,650 --> 00:01:48,590
base is used to classify those features

14
00:01:51,980 --> 00:02:00,920
so for any similar kind of problem we can easily use retrain on the delusional beast to extract features

15
00:02:00,920 --> 00:02:09,790
from our images and then we can all three layers of fully connected neural network to classify the desert

16
00:02:09,820 --> 00:02:14,480
of this can be says in this video.

17
00:02:14,510 --> 00:02:25,380
The idea is same we will use the corn base of VDI 16 model and then we will add one fully connected

18
00:02:25,650 --> 00:02:32,760
layer and one output layer to classify the features extracted from what we do this extreme on base

19
00:02:35,890 --> 00:02:37,330
so let the sun.

20
00:02:37,720 --> 00:02:42,660
For us we will be creating two object green gender data and validation units.

21
00:02:43,390 --> 00:02:50,560
We have already use the same generators in our previous cases also.

22
00:02:50,560 --> 00:02:58,060
So we are using the same setting we are using these scaling off one by 255 to convert our odyssey we

23
00:02:58,060 --> 00:03:05,170
will lose from 0 to 255 2 0 2 1.

24
00:03:05,430 --> 00:03:13,710
Then we have rotation rotation which chip a chair shearing zoom range and horizontal flip to create

25
00:03:13,770 --> 00:03:21,390
dummy augmented data and our target size of images is 150 by 150.

26
00:03:21,390 --> 00:03:24,760
And we are using a bed size of 20.

27
00:03:24,840 --> 00:03:30,120
We have already discussed this in detail in our previous videos.

28
00:03:30,210 --> 00:03:37,420
So we are not going to discuss this here just like in our previous guests.

29
00:03:37,490 --> 00:03:45,680
We have our own cool images for training and posing images for validation.

30
00:03:45,740 --> 00:03:54,990
Now the second step was to create architecture photo or mortal now our idea is to first use of con base

31
00:03:55,110 --> 00:03:59,340
of which uses 16 and then use to dense layer

32
00:04:03,630 --> 00:04:05,520
so to use on base of.

33
00:04:05,530 --> 00:04:10,830
We do this 16 you can vertically import that from us.

34
00:04:10,830 --> 00:04:15,770
There is no need to manually build all the cone layers in that base.

35
00:04:15,960 --> 00:04:18,940
So the import we do the 60.

36
00:04:19,020 --> 00:04:23,340
You can just write from 10 sort of flowed out get a third application import.

37
00:04:23,350 --> 00:04:28,230
We do these 16 and then give these three different parameters

38
00:04:32,360 --> 00:04:35,760
so we are creating our cone base object.

39
00:04:36,050 --> 00:04:42,070
And we are using we do this 60 and this out of the three parameters that we are passing.

40
00:04:42,200 --> 00:04:52,280
First we need to provide weights so in any new national neural network first we provide randomized weight

41
00:04:52,910 --> 00:05:03,210
and then our constitutional network tries to optimize those words since we do these 16 was used in that

42
00:05:03,210 --> 00:05:08,850
competition and we can use the final weights of that more than

43
00:05:12,290 --> 00:05:14,280
so to use those weights.

44
00:05:14,360 --> 00:05:17,640
We have to write weights equal to imagine it.

45
00:05:17,890 --> 00:05:20,300
Imagine that is the competition.

46
00:05:20,500 --> 00:05:22,440
Add as we see completion

47
00:05:25,520 --> 00:05:34,030
so to use train weights we just separate very quickly made to net and then there were two parts of that

48
00:05:34,120 --> 00:05:43,590
we do these 16 model first for one base and then the fully connected neural network base we only want

49
00:05:43,630 --> 00:05:53,260
the cone base from that model since cone bases are reusable those are mainly used to extract features

50
00:05:53,440 --> 00:06:02,050
and not to categorize the images so we will be only using the base and we only need to import based

51
00:06:02,820 --> 00:06:12,540
that's why we are using include underscore top equal to False if we want to import the whole model along

52
00:06:12,540 --> 00:06:21,180
with the fully connected dense layers then you have to change it to group but in our case since we are

53
00:06:21,220 --> 00:06:26,550
only importing the constitutional base we are waiting for this here

54
00:06:29,750 --> 00:06:38,880
then next parameter is to give the input shape the input chip of our images are 1 4 feet by 1 4 feet

55
00:06:38,910 --> 00:06:39,560
by 3.

56
00:06:39,570 --> 00:06:43,800
That's why we are providing this couple here.

57
00:06:43,950 --> 00:06:49,710
Let's turn this so we have imported our corn base from.

58
00:06:49,710 --> 00:06:50,970
We did this 16 model

59
00:06:54,160 --> 00:06:56,340
now to look at this.

60
00:06:56,400 --> 00:06:58,830
You can just say one based thought somebody

61
00:07:02,080 --> 00:07:11,660
oh if you're done this you will get details of all the layers of this which is this extreme corn based

62
00:07:15,110 --> 00:07:23,270
now as we discuss in our TV lecture we use these 16 have this convolution of blocks.

63
00:07:23,270 --> 00:07:26,700
So here you can see first convolution of block.

64
00:07:26,750 --> 00:07:30,420
Then second conventional block in each block.

65
00:07:30,470 --> 00:07:32,640
There are my people corn layers.

66
00:07:32,720 --> 00:07:34,510
So in a sense I can blocks.

67
00:07:34,520 --> 00:07:44,390
There are two layers and then a max pooling layer in third fourth and fifth look there are three corn

68
00:07:44,390 --> 00:07:47,060
layers and then a max pooling layer

69
00:07:52,340 --> 00:08:01,790
so in a way by importing BDD 16th we avoided creating these many layers and we have already imported

70
00:08:01,790 --> 00:08:04,600
the final weights of that model.

71
00:08:05,120 --> 00:08:10,950
So there is no need to randomly provide weights and optimize those weights.

72
00:08:11,000 --> 00:08:18,760
We already have the final more than weights with us in this part of now.

73
00:08:18,820 --> 00:08:27,710
The next step is to add fully connected dense layer and output layer in front of this one base.

74
00:08:27,820 --> 00:08:33,520
Now this is similar to creating any CNN model.

75
00:08:33,520 --> 00:08:35,940
We just have to create our model for us.

76
00:08:36,190 --> 00:08:44,440
We are using models that sequential and then just like you add any other layer you can add the corn

77
00:08:44,440 --> 00:08:50,720
base that we have imported so we will write model not AG.

78
00:08:50,850 --> 00:08:55,770
And here you can just write the variable in which we have is stored.

79
00:08:55,780 --> 00:08:57,510
This we did 60.

80
00:08:57,690 --> 00:09:00,370
So what variable name was on base.

81
00:09:00,600 --> 00:09:04,470
So plus we can add this one base.

82
00:09:04,470 --> 00:09:11,250
Next we have to use a flat and layer and then include a fully connected dense layer and then output

83
00:09:11,250 --> 00:09:11,550
layer.

84
00:09:11,990 --> 00:09:19,620
So plus we are shedding light on layer and our dense layer with 256 Newton and then an output layer

85
00:09:19,830 --> 00:09:22,040
with a single neuron.

86
00:09:22,410 --> 00:09:28,130
The application is that a low in the dense layer and that duration is sigmoid in the output layer

87
00:09:31,230 --> 00:09:40,460
you can run this and then you can look at the more than somebody as well so if you see this as a lot

88
00:09:40,460 --> 00:09:44,500
more than somebody our first layer is reduced to 60.

89
00:09:44,950 --> 00:09:51,880
We have around 14 millions trainable parameter in this we did this extreme layer.

90
00:09:52,060 --> 00:09:53,950
Then we have a flatten layer.

91
00:09:53,950 --> 00:10:00,460
Then we have a dense layer with around 2 million animal parameters and then finally an output layer

92
00:10:00,460 --> 00:10:08,950
with 257 trainable parameters the total cranial parameter in our model is around 16 million.

93
00:10:10,000 --> 00:10:15,310
Now as I told you earlier we were using the weights of the final.

94
00:10:15,310 --> 00:10:23,150
We did this scene model so the weights are already optimized in this we do this extremely it.

95
00:10:23,770 --> 00:10:31,460
Now if you don't want to screen those weights you can just freeze that layer to freeze that.

96
00:10:31,540 --> 00:10:36,160
You can use corn based not training but equally false.

97
00:10:36,460 --> 00:10:46,330
In that case the training parameter here will down to zero and our model will not try to optimize the

98
00:10:46,330 --> 00:10:49,210
weights of this layer in that way.

99
00:10:49,210 --> 00:10:56,740
We can significantly reduce the number of criminal parameters in our model and significantly improve

100
00:10:56,740 --> 00:10:58,060
our execution time.

101
00:11:00,400 --> 00:11:07,450
So if you run this one based or training equal to False our number of clinical parameters will reduce

102
00:11:07,450 --> 00:11:17,480
from 16 million to just two point one million but here we are not running this and we are creating all

103
00:11:17,480 --> 00:11:24,540
the 16 million parameters but in case you want to say thank you you can run this corn based or cleaning

104
00:11:24,660 --> 00:11:25,610
equity funds

105
00:11:31,450 --> 00:11:31,880
now.

106
00:11:31,910 --> 00:11:40,320
The next step is to come by a lower margin we will be using the last function of binary cross entropy

107
00:11:41,130 --> 00:11:50,220
since we have two classes then we are using Artemus prop as our optimizer and a learning rate of 2 and

108
00:11:50,240 --> 00:11:53,160
210 is what a minus 5.

109
00:11:53,460 --> 00:12:01,260
We are using somewhat smaller learning rate to get just because we want to fine tune our already train

110
00:12:01,260 --> 00:12:10,350
more than the weights of this constitutional limits are already optimized and we just want to optimize

111
00:12:10,490 --> 00:12:14,420
it and little steps leading to our problem.

112
00:12:14,490 --> 00:12:22,560
So since we are fine tuning it we are not draining it from randomly assigned weights we can use a smaller

113
00:12:22,560 --> 00:12:23,600
learning rate.

114
00:12:23,760 --> 00:12:34,230
That is why we are using to train then race minus five and the metrics we want to calculate is of accuracy.

115
00:12:34,230 --> 00:12:38,970
So training these models really takes somewhere between eight to 10 hours.

116
00:12:39,450 --> 00:12:50,330
So it is better to use callbacks to say what are more than offset each epoch we are creating our check

117
00:12:50,330 --> 00:12:51,170
point.

118
00:12:51,630 --> 00:12:57,840
And we are saving our model for itI Paul.

119
00:12:58,000 --> 00:13:02,220
You can also use save best only parameter here.

120
00:13:02,230 --> 00:13:10,960
If you don't want to save 30 different models and if you give saved best quality crew we will save the

121
00:13:10,960 --> 00:13:16,320
more than with the best value addition in score.

122
00:13:16,560 --> 00:13:19,470
Now the next step is to the training data.

123
00:13:21,760 --> 00:13:29,250
The step is similar to the last thing we will use for DNA rigor and then green generator with steps

124
00:13:29,260 --> 00:13:37,090
but epoch and we'll also give validation generator and validation in steps for our evaluation data.

125
00:13:37,090 --> 00:13:43,020
And yet we are also providing callback just to say well and more than that after each epoch.

126
00:13:43,960 --> 00:13:51,570
So I have already executed this and these are the results.

127
00:13:51,990 --> 00:13:59,420
So if you see the validation accuracies are in the range of 92 97.

128
00:13:59,430 --> 00:14:08,100
So at the end of the book we will getting our training accuracy of our own 98 percent and the validation

129
00:14:08,160 --> 00:14:13,990
accuracy of 98 percent as well.

130
00:14:14,010 --> 00:14:19,440
You can see each epoch is taking around 15 minutes to train.

131
00:14:19,740 --> 00:14:28,010
So just remember this may take up to eight to 10 hours to train our modern.

132
00:14:28,930 --> 00:14:35,070
Now let's look at how accuracies and losses are changing with e.g. Paul

133
00:14:38,800 --> 00:14:39,660
the orange line.

134
00:14:39,670 --> 00:14:42,070
Here is for training accuracy.

135
00:14:42,070 --> 00:14:44,930
The red line here is for validation accuracy.

136
00:14:45,010 --> 00:14:51,960
And similarly we have validation lost in green and greening loss in blue

137
00:14:55,330 --> 00:15:06,100
you can see that validation accuracy is oscillating between 97 to 98 and there is no further improvement

138
00:15:06,130 --> 00:15:13,320
in accuracy as we move from lower epoch value to higher value so we can see that we have achieved a

139
00:15:13,330 --> 00:15:21,070
convergence in our model and it is not possible to further improve this validation accuracy with increasing

140
00:15:21,100 --> 00:15:24,150
number of epochs.

141
00:15:24,220 --> 00:15:31,340
So if you compare the validation accuracy with our last CNN model in the last CNN model we were getting

142
00:15:31,340 --> 00:15:34,230
the maximum accuracy of 84 percent.

143
00:15:34,690 --> 00:15:44,830
But in this by using BDD 16 retrain more than we are achieving up to 97 98 percent of validation accuracy

144
00:15:46,780 --> 00:15:50,820
and it is very easy to train our model using this.

145
00:15:50,820 --> 00:15:52,360
We train models

146
00:15:55,480 --> 00:16:01,520
so there is no need to create our own corn business.

147
00:16:01,720 --> 00:16:06,640
You can just use any one on this pre train corn basis.

148
00:16:06,670 --> 00:16:16,780
If the problem is segment is somewhat similar to that image neck problem misstatements.

149
00:16:17,540 --> 00:16:21,840
Now I am also saving this history variable and glossy it to fight.

150
00:16:21,890 --> 00:16:28,200
There is no need to do this this year will not.

151
00:16:28,280 --> 00:16:39,250
We were on legal polluting the accuracies on our validation sets but now it's time to use our tests

152
00:16:39,830 --> 00:16:45,280
to see how this model performs on our test asset.

153
00:16:45,310 --> 00:16:51,700
Now we have to follow the same steps to evaluate our model for performance.

154
00:16:51,880 --> 00:16:55,780
Again we will be using best generated

155
00:16:58,970 --> 00:17:01,750
so we are creating another generator.

156
00:17:01,800 --> 00:17:03,560
We are calling the best generator

157
00:17:06,540 --> 00:17:09,250
we are using the test and that's called data.

158
00:17:09,420 --> 00:17:15,080
This is the same object we use for regulation as well so hidden in this object.

159
00:17:15,090 --> 00:17:23,610
We are just reshaping our data from 0 to 255 to 0 to 1 and then we are using flow from better tree and

160
00:17:23,610 --> 00:17:28,060
here we are providing past that tree instead of validation vertically.

161
00:17:28,140 --> 00:17:32,380
So what is generated is really no normally.

162
00:17:32,500 --> 00:17:43,810
If we have data in the form of a data frame we use evaluate it but since we have our data flowing from

163
00:17:43,810 --> 00:17:51,140
our directory that's why we have to use a really weird underscore generator.

164
00:17:51,640 --> 00:17:53,850
So there is a simple back and forth effect.

165
00:17:53,890 --> 00:17:55,510
We are using for a generator.

166
00:17:57,040 --> 00:18:00,560
Similarly for you earlier we are using it will your generator.

167
00:18:01,150 --> 00:18:08,800
And here also we have to provide our best generator object and the number of steps we have a bed size

168
00:18:08,800 --> 00:18:09,520
of 20.

169
00:18:10,030 --> 00:18:14,200
We have total data size of our home potent images.

170
00:18:14,230 --> 00:18:21,490
That's why we need 50 steps thousand divided by 20 equal to 50.

171
00:18:21,880 --> 00:18:26,500
So we will be able to give our all our estimate is in 50 steps.

172
00:18:26,500 --> 00:18:31,930
So if you run this just like evaluate method you will get two values.

173
00:18:31,950 --> 00:18:33,280
First is the lost value.

174
00:18:33,280 --> 00:18:35,860
And second is the accuracy value.

175
00:18:35,920 --> 00:18:41,080
And here you can see that the accuracy we are getting is that all 97 percent

176
00:18:44,090 --> 00:18:45,670
so just 50 ways.

177
00:18:45,700 --> 00:18:51,470
We started with a simple constitutional model.

178
00:18:51,820 --> 00:18:57,200
At that time we were getting accuracy of around 70 percent to 40 percent.

179
00:18:57,340 --> 00:19:03,950
Then we used data augmentation techniques to create dummy data and avoid or what.

180
00:19:04,600 --> 00:19:15,700
In that case we were getting accuracy of our own 83 84 percent and in this case we used up drain.

181
00:19:15,720 --> 00:19:26,280
We do these 16 more than caught our problem and in this case we are getting an accuracy of around 97

182
00:19:26,280 --> 00:19:26,700
percent.

183
00:19:27,810 --> 00:19:38,430
So we have increased our validation accuracy from 73 percent to 98 percent during this project.

184
00:19:38,490 --> 00:19:39,870
That's all for this project.

185
00:19:40,350 --> 00:19:40,800
Thank you.