1
00:00:00,720 --> 00:00:01,750
All righty.

2
00:00:01,770 --> 00:00:09,390
So now we've got a way to get validation images validation labels so a.k.a. the truth labels of what

3
00:00:09,390 --> 00:00:13,350
an image is and prediction labels.

4
00:00:13,440 --> 00:00:19,290
So you might be thinking this is a fair few steps to visualize what's going on but I hope you realize

5
00:00:19,290 --> 00:00:24,180
where the importance is because we're always thinking about what's the end use case we're not just here

6
00:00:24,210 --> 00:00:29,070
to build models for the sake of building models we want to build dog vision to return we could just

7
00:00:29,070 --> 00:00:33,240
return an array of like this to a user and then they would go Wow.

8
00:00:33,270 --> 00:00:37,080
I'm not going to use this app anymore because these numbers are just a bunch of jargon to me.

9
00:00:37,500 --> 00:00:44,640
So that's why it's important that we can visualize our model's predictions in a way that people can

10
00:00:44,640 --> 00:00:50,730
understand someone who isn't as versed in machine learning and prediction probabilities as us can understand.

11
00:00:50,730 --> 00:00:54,180
That's our end goal and that's what I want you to start thinking about all of the projects that you're

12
00:00:54,180 --> 00:00:56,840
working on rather than just building models.

13
00:00:56,880 --> 00:01:02,610
What is your angle are you working towards something how can you communicate what you're doing to someone

14
00:01:02,610 --> 00:01:02,970
else.

15
00:01:03,830 --> 00:01:06,490
All right so let's write a little note to ourselves.

16
00:01:06,540 --> 00:01:10,440
Now we've got ways to get

17
00:01:13,140 --> 00:01:26,480
prediction labels validation labels which are the truth labels and validation images let's make some

18
00:01:26,480 --> 00:01:31,980
functions to make these all a bit more visual.

19
00:01:32,120 --> 00:01:34,880
Basically we need a bit of color in our lives

20
00:01:38,980 --> 00:01:40,000
OK.

21
00:01:40,030 --> 00:01:41,470
Now how might we do this.

22
00:01:41,620 --> 00:01:44,810
So we're probably going to make a function as we said before.

23
00:01:44,810 --> 00:01:49,090
That's going to show us an image and it'd be good if it could just plot an image that's what we can

24
00:01:49,090 --> 00:01:49,420
use.

25
00:01:49,440 --> 00:01:52,390
Map plotted something like that.

26
00:01:52,390 --> 00:02:00,780
Take an array of image values which we've got here maybe a good thing we work towards is It plots the

27
00:02:00,780 --> 00:02:04,080
predicted label it's predicted probability.

28
00:02:04,080 --> 00:02:09,900
So the max value up here so something like this.

29
00:02:09,900 --> 00:02:14,820
So this is 75 percent and then it's actual label.

30
00:02:14,970 --> 00:02:22,470
I think we can get them along to a single plot maybe we'll write that down so we are don't confuse ourselves

31
00:02:22,570 --> 00:02:24,660
with the rubber ducky technique from a boy Ron.

32
00:02:24,660 --> 00:02:26,010
Thanks Rob.

33
00:02:26,130 --> 00:02:29,040
We'll create a function which

34
00:02:31,740 --> 00:02:45,930
takes an array of prediction probabilities and array of truth labels and an array of images and integers.

35
00:02:45,930 --> 00:02:50,690
Now again I must stress I'll say it too I go horse but we are dealing with images.

36
00:02:50,700 --> 00:02:55,410
But the visualization steps that we're doing goes to almost any data that you're working with.

37
00:02:56,580 --> 00:03:02,940
So we're gonna go convert the prediction probabilities of course she may need to use some other visualization

38
00:03:02,940 --> 00:03:10,440
techniques that aren't map pot lib to plot or to visualize a data sample but you get the picture predicted

39
00:03:10,440 --> 00:03:15,560
label and plot the predicted label

40
00:03:18,450 --> 00:03:21,150
it's predicted probability

41
00:03:23,790 --> 00:03:33,320
the true label and the target image on a single plot like so how might we do that.

42
00:03:33,330 --> 00:03:41,090
Let's go here we'll call it def some simple plot spread it takes what does it take takes prediction

43
00:03:41,090 --> 00:03:52,070
probabilities probabilities it takes labels an array of true labels and an array of images and maybe

44
00:03:52,070 --> 00:03:53,290
only one integer.

45
00:03:53,390 --> 00:04:00,560
Let's make this a plot to a single image a single prediction label and a single truth label and an integer

46
00:04:02,400 --> 00:04:04,870
so maybe we could just say this has an equals 1

47
00:04:08,390 --> 00:04:13,490
that's going to error out because we've got nothing there that's all right get rid of that era we don't

48
00:04:13,490 --> 00:04:22,040
like seeing them like calls me a little bit of stress so we go here view the prediction Ground Truth

49
00:04:23,090 --> 00:04:32,430
and image for sample n and again getting trigger happy right it's like almost instinct for me to shift

50
00:04:32,430 --> 00:04:37,770
and into whenever I finish writing a line of code okay.

51
00:04:37,780 --> 00:04:43,360
So the first thing we'll need is the pred prob it's actually prediction probability but I'm not typing

52
00:04:43,360 --> 00:04:48,040
that multiple times through label and the image.

53
00:04:48,130 --> 00:04:54,580
So this is gonna equal member we want the end sample so we do have to actually type prediction probabilities

54
00:04:55,720 --> 00:05:05,050
and so the index of the end sample and then we want labels n and again there's probably a thousand ways

55
00:05:05,050 --> 00:05:06,070
to write this function.

56
00:05:06,070 --> 00:05:13,150
This is just one way that I figured works could be probably optimized in some way but it works for now

57
00:05:14,070 --> 00:05:16,360
equals get Fred label.

58
00:05:16,600 --> 00:05:23,620
Now that we had the prediction probability we can use our function get pred label to get the pred label

59
00:05:25,590 --> 00:05:28,600
brilliant nomenclature of our functions if I do say so myself.

60
00:05:28,770 --> 00:05:37,740
Plot image and remove ticks because we just want a square of images we don't want accesses dot M show

61
00:05:38,070 --> 00:05:38,430
image.

62
00:05:38,430 --> 00:05:40,560
So this is a map partly a function.

63
00:05:40,560 --> 00:05:41,850
You pass it an umpire.

64
00:05:41,880 --> 00:05:50,820
If it's an image display an image by a data on a 2D regular raster far out sometimes Doc strings are

65
00:05:50,820 --> 00:05:56,520
written in a complex way like it could adjust said puts an image

66
00:05:59,370 --> 00:06:01,460
and Y takes.

67
00:06:01,620 --> 00:06:02,850
So this is just empty list.

68
00:06:02,970 --> 00:06:09,390
So what is turning them off and all basically is just going to show nothing because there's nothing

69
00:06:09,390 --> 00:06:10,350
in that list.

70
00:06:10,350 --> 00:06:23,790
We might need the plot title Iran change plot title to be predicted and what predicted libel probability

71
00:06:23,880 --> 00:06:33,440
of prediction and truth labels peyote that title.

72
00:06:33,650 --> 00:06:38,180
Now we've got our pred prob we've got our true label we've got our pride label so we're just gonna have

73
00:06:38,180 --> 00:06:50,400
to create a string of three of these spread probability and true label now this one is going to have

74
00:06:50,400 --> 00:06:54,390
to be 2.0 what is this doing.

75
00:06:54,870 --> 00:07:01,680
This is just reducing the amount of decimal points in our prediction probability so that it's not something

76
00:07:01,680 --> 00:07:06,930
long like this better example is not something long like this.

77
00:07:06,960 --> 00:07:12,640
It's a percentage so rather than being that it's 75 percent that's what we're off.

78
00:07:14,590 --> 00:07:15,960
Come down here.

79
00:07:15,990 --> 00:07:26,570
We'll just do a little format and we'll go red label because that's the first one so prediction is there.

80
00:07:27,380 --> 00:07:37,330
And then we want NDP Max Fred Traub Time's 100 times a thousand Woo.

81
00:07:37,930 --> 00:07:43,440
So this is just going to be the prediction probability value.

82
00:07:43,650 --> 00:07:48,570
So the max of this of the pred problem Ray times 100.

83
00:07:48,570 --> 00:07:54,100
So when we saw a point seven five before it's just going to times that by 100 might not be.

84
00:07:54,100 --> 00:07:59,580
Point seven five because it'll be a different value for a different image.

85
00:07:59,600 --> 00:08:00,230
There we go.

86
00:08:00,230 --> 00:08:02,610
And we want to label is the last one.

87
00:08:03,540 --> 00:08:04,370
So does that make sense.

88
00:08:05,120 --> 00:08:15,740
So plot title we want to get rid of that pass plot title squiggly brackets for pred label squiggly brackets

89
00:08:15,770 --> 00:08:22,430
two decimal places percentage for the maximum prediction probability and squiggly brackets for the true

90
00:08:22,430 --> 00:08:24,350
label.

91
00:08:24,350 --> 00:08:26,690
Let's see what this bad boy looks like.

92
00:08:26,760 --> 00:08:31,030
Unexpected end of string.

93
00:08:31,120 --> 00:08:31,920
Oh we need another one

94
00:08:35,100 --> 00:08:36,240
plot Fred.

95
00:08:36,270 --> 00:08:36,760
Okay.

96
00:08:36,770 --> 00:08:40,820
And now let's go plot Fred.

97
00:08:40,820 --> 00:08:44,410
We want predictions.

98
00:08:45,680 --> 00:08:50,390
On a prediction probabilities equals predictions.

99
00:08:50,830 --> 00:08:58,030
And then we want labels equals Val labels and images equals Val images.

100
00:08:58,060 --> 00:09:02,880
So this is just from up before where we UNbacked define our validation dataset.

101
00:09:02,890 --> 00:09:06,150
We've got here and let's come down.

102
00:09:06,540 --> 00:09:07,920
Let's test add a little function

103
00:09:11,010 --> 00:09:16,810
O now labels are printing out incorrectly.

104
00:09:16,990 --> 00:09:20,100
What's happened here true labels.

105
00:09:20,200 --> 00:09:26,600
He calls pred label get Fred prob what is happening here.

106
00:09:27,900 --> 00:09:28,740
True label

107
00:09:32,070 --> 00:09:44,390
while we didn't convert I think I see what's going on what we have to do is convert our true label into

108
00:09:44,390 --> 00:09:46,420
an actual label.

109
00:09:46,660 --> 00:09:47,740
So right now.

110
00:09:47,740 --> 00:09:50,730
This is debugging your own code right.

111
00:09:50,740 --> 00:09:55,650
So what we have to do is change our own batch of five function to be unique breeds.

112
00:09:55,780 --> 00:10:00,160
And this is going to be any ARG Max label.

113
00:10:00,160 --> 00:10:01,060
Does that make sense.

114
00:10:01,570 --> 00:10:02,550
Yes.

115
00:10:02,740 --> 00:10:08,980
So just a very similar functionality what we could actually do is use this function here get pred label

116
00:10:09,000 --> 00:10:11,350
what we might save that just for predictions.

117
00:10:11,350 --> 00:10:15,430
So this is gonna get the unique braids NPR and Max label we need to rerun this

118
00:10:18,270 --> 00:10:24,300
so make sure you update your own batch of high function and then if we rerun this this should be correct

119
00:10:25,760 --> 00:10:29,170
and then if we rerun this Oh yeah.

120
00:10:29,210 --> 00:10:31,520
Scott Terry are 68 percent correct.

121
00:10:31,520 --> 00:10:32,810
This is not very intuitive.

122
00:10:32,810 --> 00:10:37,360
What if someone was looking at this and it was like Scotch Terrier 68 percent Scotch Terrier.

123
00:10:37,370 --> 00:10:40,940
Which one is the predicted and which one is the the non predicted.

124
00:10:41,030 --> 00:10:43,490
Let's add some color.

125
00:10:43,640 --> 00:10:45,190
That's how we could do it.

126
00:10:45,200 --> 00:10:51,430
So what we will do will create a little if else cause so we'll go change the color.

127
00:10:51,440 --> 00:10:53,520
That looks disgusting to me.

128
00:10:53,950 --> 00:10:57,230
Ah see I'm Australian and we spell color.

129
00:10:57,260 --> 00:10:58,380
Oh you are.

130
00:10:58,580 --> 00:11:03,680
So whenever I say oh ah even though that's the actual little parameter that we have to pass to format

131
00:11:04,400 --> 00:11:11,720
I don't like it the color of the title depending on if the prediction is right or wrong.

132
00:11:12,410 --> 00:11:17,270
So all we're going to do is change this title to be green.

133
00:11:17,270 --> 00:11:35,620
If it's right and red if it's wrong Red Label equals true label we want color equals green else color

134
00:11:35,920 --> 00:11:39,960
equals red and then we're gonna pass a little parameter here.

135
00:11:40,210 --> 00:11:44,110
So it's all about making out plots more visual

136
00:11:47,410 --> 00:11:56,940
Oh what what in that change if pred label equals true label.

137
00:11:56,970 --> 00:11:57,450
There we go.

138
00:11:57,540 --> 00:11:59,500
I found it.

139
00:11:59,700 --> 00:12:07,040
So that's part of the format we need to pass this outside of the format that makes sense.

140
00:12:07,730 --> 00:12:12,080
So we're passing this color parameter directly to title rather than to format

141
00:12:17,690 --> 00:12:20,780
Scotch Terrier 68 percent beautiful.

142
00:12:20,870 --> 00:12:23,760
Now remember this is predicting for index 1.

143
00:12:24,110 --> 00:12:29,890
So what we might do is just make sure it works on another image was our favorite number 42

144
00:12:33,600 --> 00:12:34,850
you got that one wrong.

145
00:12:34,850 --> 00:12:40,280
That's a good example actually it's pretty confident 76 percent predicted walk around but it's actually

146
00:12:40,280 --> 00:12:42,140
an English fox hand.

147
00:12:42,140 --> 00:12:45,560
So let's look up English fox hound.

148
00:12:45,800 --> 00:12:53,080
What does an English fox hound look like images okay.

149
00:12:53,210 --> 00:12:59,960
We go back to dog vision wow you can kind of see how they look pretty similar.

150
00:12:59,980 --> 00:13:07,410
Especially that part on the back those two they're see how that's black there and that's black they're

151
00:13:08,010 --> 00:13:12,540
seeing kind of say oh look at these puppies this is a beautiful thing about working on a project of

152
00:13:12,540 --> 00:13:16,510
vision you get to look at dogs so you can kind of see where that's going wrong.

153
00:13:16,520 --> 00:13:17,300
Let's look at one more.

154
00:13:17,300 --> 00:13:18,790
My other favorite number is seven.

155
00:13:18,830 --> 00:13:20,390
So why not seventy seven

156
00:13:23,750 --> 00:13:24,780
and I got that one right.

157
00:13:24,800 --> 00:13:26,090
That's really confident.

158
00:13:26,120 --> 00:13:27,000
That's awesome.

159
00:13:27,000 --> 00:13:29,740
A grow grow and grow in India.

160
00:13:30,370 --> 00:13:31,230
Mm hmm.

161
00:13:31,380 --> 00:13:38,150
Well this is very exciting because now we're actually seeing our model's predictions compared to actual

162
00:13:38,150 --> 00:13:41,240
images dog vision is officially coming to life.

163
00:13:41,240 --> 00:13:46,550
This is the kind of functionality we wanted from the very start to be able to pass our model and image

164
00:13:47,120 --> 00:13:48,460
and it make a prediction.

165
00:13:48,620 --> 00:13:52,280
And now we're seeing how confident it is on that prediction.

166
00:13:52,370 --> 00:13:57,830
So prediction probability or confidence interval and we're actually comparing it to the real label.

167
00:13:57,860 --> 00:14:03,130
So this is an image because it's from validation dataset and image our model has never seen before.

168
00:14:03,320 --> 00:14:07,910
So you better pat yourself on your back because you've just felt a deep learning model that's able to

169
00:14:08,570 --> 00:14:12,110
take photos of dogs and predict what kind of breed it is.

170
00:14:12,110 --> 00:14:13,760
That's pretty cool.

171
00:14:13,760 --> 00:14:20,870
Now one more thing we'll probably do to visualize a little bit more is see what other labels our model

172
00:14:20,870 --> 00:14:24,440
predicted because there's prediction probabilities for every label.

173
00:14:24,440 --> 00:14:30,740
It would be interesting to know what other dog breeds our model looked at an image and thought it was.

174
00:14:30,740 --> 00:14:36,440
So this one's 96 percent so it's pretty darn confident on it that it's a growing N Dale breed of dog.

175
00:14:36,440 --> 00:14:39,410
I can't even pronounce some of these breeds of dogs.

176
00:14:39,440 --> 00:14:41,120
Well let's do that in the next video.

177
00:14:41,120 --> 00:14:44,600
Hey we'll create a plot to show the top 10 predictions.