1
00:00:00,590 --> 00:00:07,070
We've spent a bit of time making our models predictions more visual which is very very important for

2
00:00:07,070 --> 00:00:08,480
when you're communicating your work.

3
00:00:08,480 --> 00:00:14,030
I don't want you to think that just building a model and having an accuracy value like we've got up

4
00:00:14,030 --> 00:00:22,100
here is enough because at the end of the day you can actually run this to get something like this in

5
00:00:22,100 --> 00:00:23,560
about 10 lines of code.

6
00:00:23,690 --> 00:00:29,750
But what really matters is how you're going to integrate your models use and how you're going to communicate

7
00:00:29,750 --> 00:00:32,140
what it's actually doing to other people.

8
00:00:32,270 --> 00:00:36,130
Because Robert always bring yourself back to what the project goal is.

9
00:00:36,290 --> 00:00:43,260
What are we trying to do here with dog vision so let's take our little visualization functions and it'd

10
00:00:43,260 --> 00:00:45,990
be good if we could see more than just one at a time.

11
00:00:46,030 --> 00:00:51,790
So let's write some code to do that or leave a little bit of text here.

12
00:00:52,170 --> 00:00:52,890
Now we've got

13
00:00:55,730 --> 00:01:05,910
some functions to help us visualize our predictions and evaluate our model.

14
00:01:05,990 --> 00:01:09,440
Let's check out a few.

15
00:01:09,550 --> 00:01:18,050
So what we can do is probably create a few subplots and I want to get this image compared to its top

16
00:01:18,050 --> 00:01:20,540
10 pred confidences.

17
00:01:20,630 --> 00:01:21,740
So that's what I'm going to do.

18
00:01:23,390 --> 00:01:28,420
Let's check out a few predictions.

19
00:01:28,430 --> 00:01:33,030
And there are different values nice and simple.

20
00:01:33,170 --> 00:01:38,510
So we want to go I multiply this to make sense in a second so we'll keep it at zero to begin with.

21
00:01:38,510 --> 00:01:42,800
And again this is probably something we can also function eyes maybe that's a little challenge to you.

22
00:01:42,800 --> 00:01:47,780
So we've got a number of rows three num calls equals two.

23
00:01:47,780 --> 00:01:54,170
So we're going to plot out a subplot that's got three rows two columns and the number of images total

24
00:01:54,230 --> 00:02:03,630
is just going to be num rows times num calls and then we'll go to plot that figure we'll set the fig

25
00:02:03,630 --> 00:02:04,870
size.

26
00:02:04,950 --> 00:02:09,030
I've had a bit of playing around with this so this might have to change depending on what your notebook

27
00:02:09,030 --> 00:02:20,670
is or actually we could probably just go 10 times num calls and five times now it throws.

28
00:02:20,800 --> 00:02:21,400
All right.

29
00:02:21,430 --> 00:02:26,830
So now we want for I in range num images so five times two.

30
00:02:26,860 --> 00:02:35,740
We're going to plot three times two sorry we're gonna plot six images total now images we need a semicolon

31
00:02:35,740 --> 00:02:36,500
there.

32
00:02:36,820 --> 00:02:46,030
We want to create a subplot peyote that subplot and it's gonna have two rows but it's going to have

33
00:02:47,750 --> 00:02:51,110
two times number of columns and this will make sense once you pull it out.

34
00:02:51,200 --> 00:02:53,960
So two times I.

35
00:02:54,260 --> 00:02:55,850
This is the index we want to use.

36
00:02:56,210 --> 00:03:02,710
Now we want to go plot Fred prediction probabilities

37
00:03:05,090 --> 00:03:07,910
is equal to one that go to the next line.

38
00:03:07,910 --> 00:03:18,690
Come on Mark prediction probabilities equals predictions and then we want to go what else is after that

39
00:03:18,690 --> 00:03:30,750
we need to posit labels so labels is our Val labels and then images equals Val images and then the n

40
00:03:30,800 --> 00:03:36,740
so the number or the index of the imagery want is I plus I multiplier.

41
00:03:37,480 --> 00:03:40,880
And I'm starting to realize how janky this code is but that's a right.

42
00:03:40,880 --> 00:03:42,200
It's gonna work.

43
00:03:42,760 --> 00:03:44,930
We can probably function is this.

44
00:03:44,960 --> 00:03:50,210
Remember I said you a little challenge at the start of video function or is this that could be a cool

45
00:03:50,210 --> 00:03:51,180
series function.

46
00:03:51,230 --> 00:03:51,870
Is this.

47
00:03:51,950 --> 00:03:56,220
Someone just writes Paul code and then someone turns it into a function.

48
00:03:56,390 --> 00:04:01,430
We go here plot thread com prediction probabilities

49
00:04:03,870 --> 00:04:11,310
predictions and then we want to go labels equals valid labels because what we want to do is just compare

50
00:04:11,400 --> 00:04:16,320
our predictions to the troop levels and then the end is going to be again I.

51
00:04:16,350 --> 00:04:24,940
Plus I multiply and then let's say what it looks like plot don't show hopefully this works.

52
00:04:28,790 --> 00:04:33,120
A It's starting to come out member might take a little while.

53
00:04:33,170 --> 00:04:41,140
Oh that looks pretty terrible how can we fix this so it's kind of getting what we wanted.

54
00:04:41,370 --> 00:04:49,290
Like an image with a title and then the top 10 prediction probability so see here Border Terrier number

55
00:04:49,310 --> 00:04:51,490
one but it got it wrong.

56
00:04:51,540 --> 00:04:53,700
The true label was Khan.

57
00:04:53,860 --> 00:04:54,770
Or can I can't.

58
00:04:54,780 --> 00:04:56,190
I can't really pronounce that.

59
00:04:56,400 --> 00:05:00,060
We have a city in a shire called Cairns which is spelled very like that.

60
00:05:00,060 --> 00:05:01,880
It has an S on the end.

61
00:05:01,950 --> 00:05:03,000
Let's take this plot.

62
00:05:03,000 --> 00:05:10,260
We want plot type layout should help us out so tight layout is just basically like hey we're working

63
00:05:10,260 --> 00:05:16,690
with a subplot don't just plot them all over the place bring them together are you know why I think

64
00:05:16,690 --> 00:05:22,990
this is work broken because we are calling plot docs show Inside the for loops that's gonna show a subplot

65
00:05:23,050 --> 00:05:24,190
every time it plots

66
00:05:29,550 --> 00:05:36,080
a gallon ha there we go look at that and our labels are cut off.

67
00:05:36,080 --> 00:05:47,560
This is where plot type layout will help so one last adjustment to fix up our plot peyote type layout

68
00:05:48,280 --> 00:05:52,570
and we we're gonna do Hage pads so horizontal padding.

69
00:05:52,570 --> 00:05:58,900
So basically like take the padding along the horizontal axis and just give us a bit more space equals

70
00:05:59,020 --> 00:05:59,710
1.0

71
00:06:04,410 --> 00:06:08,370
and would you look at that fourth time lucky.

72
00:06:08,910 --> 00:06:09,940
That's what it's all about right.

73
00:06:09,960 --> 00:06:11,370
Trial and error trial and error.

74
00:06:11,400 --> 00:06:13,560
If in doubt run the code.

75
00:06:13,560 --> 00:06:14,900
So what's happening here.

76
00:06:14,940 --> 00:06:16,270
We've got our image here.

77
00:06:16,290 --> 00:06:18,690
We've got border terrier 22 percent calm.

78
00:06:18,930 --> 00:06:23,310
We can say we just looked at this one let's look at this Scotch Terrier so we can see it's got a few

79
00:06:23,310 --> 00:06:24,820
other predictions here.

80
00:06:24,930 --> 00:06:31,540
It's overwhelmingly predicting Scotch Terrier this one it got it wrong Lakeland terrier and it's actually

81
00:06:31,540 --> 00:06:33,040
a blue tick.

82
00:06:33,040 --> 00:06:40,670
So wow this is the fourth the fourth label so it's overwhelmingly predicting likely a..

83
00:06:40,670 --> 00:06:43,690
What's a like land terrier.

84
00:06:43,730 --> 00:06:48,980
Look I'm using underscores because that's what our labels have images.

85
00:06:52,450 --> 00:06:57,940
Okay you can kind of see that it's a fair bit going on in that image I think that now that even might

86
00:06:57,940 --> 00:06:59,290
be a cat over there.

87
00:07:00,240 --> 00:07:00,800
OK.

88
00:07:02,180 --> 00:07:03,520
Bell linked a..

89
00:07:03,750 --> 00:07:04,650
Yeah it's got that.

90
00:07:05,540 --> 00:07:07,360
It's got that one really right.

91
00:07:07,370 --> 00:07:08,030
That's amazing.

92
00:07:08,030 --> 00:07:12,650
Oh now this that is that is a base prediction.

93
00:07:12,740 --> 00:07:17,600
98 percent of our dish drops off that's what's up.

94
00:07:17,610 --> 00:07:23,200
Okay now let's change this to maybe see the eye multiplier is going to modify our end.

95
00:07:23,220 --> 00:07:24,060
So we want to try it.

96
00:07:24,060 --> 00:07:26,430
Let's just try the next 10 images

97
00:07:29,270 --> 00:07:32,960
all Irish Wolfhound 34 percent but it's actually a Yorkshire terrier.

98
00:07:32,960 --> 00:07:36,470
Again this is you can start to see how this image is pretty hard.

99
00:07:36,470 --> 00:07:40,130
Look how much is going on we're going to show sure what that is in the background.

100
00:07:40,130 --> 00:07:41,560
We've got two people here.

101
00:07:41,990 --> 00:07:46,190
But what's an Irish wolfhound look like Irish Wolfhound

102
00:07:48,900 --> 00:07:50,450
hahaha big dog

103
00:07:55,100 --> 00:07:55,850
OK.

104
00:07:55,890 --> 00:07:57,520
You can almost.

105
00:07:57,630 --> 00:08:02,860
It's really clutching at straws to try and say it's an Irish wolfhound because look it's even got its

106
00:08:02,880 --> 00:08:09,690
even got three more predictions toy poodle like land terrier Italian Greyhound before the actual true

107
00:08:09,690 --> 00:08:10,980
label.

108
00:08:10,980 --> 00:08:11,940
So this is really fun.

109
00:08:13,770 --> 00:08:16,260
This is predicted give us give us.

110
00:08:16,260 --> 00:08:20,700
But it's actually a border collie and now you can see how it's probably got confused because there's

111
00:08:20,700 --> 00:08:21,750
another dog there.

112
00:08:23,140 --> 00:08:26,330
Whippet miniature poodle standard.

113
00:08:26,330 --> 00:08:26,800
Look at that.

114
00:08:26,810 --> 00:08:27,800
How fancy is that.

115
00:08:27,810 --> 00:08:29,600
Ah OK.

116
00:08:29,900 --> 00:08:36,620
So see here the top three labels are all poodles but it's a standard poodle when it's predicted a miniature

117
00:08:36,620 --> 00:08:37,000
poodle.

118
00:08:36,990 --> 00:08:41,390
Now this is where a lot of evaluation machine learning models.

119
00:08:41,420 --> 00:08:43,510
This is what you got to remind yourself of.

120
00:08:43,850 --> 00:08:48,650
If a human classifier like if we were looking at this image and if you knew dogs I don't really know

121
00:08:48,650 --> 00:08:54,210
dogs except for like the three breeds that we've spoken about before I would say that that's just a

122
00:08:54,210 --> 00:08:54,790
poodle.

123
00:08:54,930 --> 00:08:58,560
But there's actually multiple different kinds of poodles.

124
00:08:58,590 --> 00:09:03,170
So you've got to give your model a little bit of leniency here and go okay.

125
00:09:03,260 --> 00:09:08,910
If a human is going to get confused on predicting a machine learning model is probably also going to

126
00:09:08,910 --> 00:09:10,620
get confused at some point.

127
00:09:10,620 --> 00:09:15,930
So let's do one more sample because this is just fun to look at dogs and look out dog vision application

128
00:09:15,930 --> 00:09:18,120
coming together.

129
00:09:18,150 --> 00:09:21,560
Now remember this is only trained on a thousand total images.

130
00:09:21,840 --> 00:09:26,930
So hopefully we see a little bit of improvement when we trying to full model looking at asset based

131
00:09:26,970 --> 00:09:29,920
prediction golden retriever.

132
00:09:29,960 --> 00:09:35,050
It's actually a flat coated retriever Khan Khan.

133
00:09:35,380 --> 00:09:38,550
Yes that's a great prediction there.

134
00:09:38,560 --> 00:09:40,300
So this is this is cool right.

135
00:09:40,300 --> 00:09:43,830
This is really cool visualizing what's going on if we were to show this to someone else.

136
00:09:43,840 --> 00:09:48,310
We could explain to them rather than just showing them a prediction probability array we could start

137
00:09:48,310 --> 00:09:52,580
to explain to them what's happening and if we were building our dog vision app.

138
00:09:52,660 --> 00:09:57,340
This is probably something we'd like it to display and even something like this may be looking a little

139
00:09:57,340 --> 00:09:58,480
bit cleaner.

140
00:09:58,510 --> 00:10:02,350
This is it's still a bit matte plot layered spec and in a jeep and a notebook.

141
00:10:02,620 --> 00:10:09,310
But this is a really important part of building a machine learning workflow.

142
00:10:09,310 --> 00:10:15,140
So here we are evaluate the model what we might move onto next we might jump all over the place a bit

143
00:10:15,490 --> 00:10:22,850
we might save and reload our train model because right now if our Google collab session decided to quit

144
00:10:23,380 --> 00:10:26,410
and we didn't save our model we'd lose all of our predictions.

145
00:10:26,420 --> 00:10:31,520
So we'd have to retrain a model so let's figure out the next video now that we visualize some of our

146
00:10:31,520 --> 00:10:32,240
predictions.

147
00:10:32,270 --> 00:10:37,520
I could play around with this all day actually let's figure out how to save our train model and load

148
00:10:37,520 --> 00:10:39,240
a trained model intensive low.