1 00:00:00,360 --> 00:00:02,060 There's a really exciting time. 2 00:00:02,130 --> 00:00:05,340 Dog vision is officially coming together. 3 00:00:05,490 --> 00:00:09,110 So now we've got one function to visualize what our model's predicting. 4 00:00:09,180 --> 00:00:13,920 We left off the last video saying it might be helpful figuring out what other predictions our model 5 00:00:13,920 --> 00:00:15,240 is making. 6 00:00:15,240 --> 00:00:17,970 So let's first go back up. 7 00:00:17,970 --> 00:00:22,740 We'll take off this tick off all of these because we've made a function that does these 8 00:00:29,460 --> 00:00:29,940 emoji 9 00:00:32,830 --> 00:00:34,090 so let's come back. 10 00:00:34,140 --> 00:00:37,490 All right a little instruction set for ourselves. 11 00:00:37,540 --> 00:00:48,460 Now we've got got one function to visualize our models predictions. 12 00:00:48,460 --> 00:00:53,560 Let's make another or our models top prediction. 13 00:00:53,980 --> 00:01:00,810 Let's make another to view our models top 10 predictions. 14 00:01:00,800 --> 00:01:03,430 Mm hmm wonder how we might do this. 15 00:01:03,460 --> 00:01:12,090 You start thinking I'm going to keep talking the function will we're probably going to need the input 16 00:01:12,150 --> 00:01:19,500 of the prediction probabilities array as well as the truth labels because remember all comparisons when 17 00:01:19,500 --> 00:01:21,240 we're evaluating our model. 18 00:01:21,390 --> 00:01:27,470 This is we're up to a lot of times evaluating a model is just comparing what the predictions are to 19 00:01:27,470 --> 00:01:31,200 the truth labels in supervised learning that is. 20 00:01:31,310 --> 00:01:41,710 So we go here we'll go take an input of prediction probabilities array and ground truth. 21 00:01:41,720 --> 00:01:42,160 All right. 22 00:01:42,470 --> 00:01:50,000 And an integrity similar setup to what we've got up here and then what's it going to do. 23 00:01:50,010 --> 00:01:55,560 It's gonna find the predicted label using get pred label 24 00:01:58,260 --> 00:02:00,710 it'll need to find the top 10 of a few things. 25 00:02:00,710 --> 00:02:04,710 So find the top 10 we'll go here. 26 00:02:04,740 --> 00:02:08,740 Prediction probabilities probability indexes. 27 00:02:08,760 --> 00:02:14,090 Sorry prediction probabilities values. 28 00:02:14,220 --> 00:02:16,500 And then finally the prediction labels 29 00:02:19,190 --> 00:02:21,890 and then what we're going to do is plot the top 10. 30 00:02:21,890 --> 00:02:23,990 So plot all of those. 31 00:02:23,990 --> 00:02:34,040 That's basically all we're gonna be the top 10 prediction probability values and labels coloring the 32 00:02:34,040 --> 00:02:42,490 true label Green another era because that's not marked down. 33 00:02:42,600 --> 00:02:47,400 Now what this is called is kind of top 10 accuracy or top 1 accuracy. 34 00:02:47,400 --> 00:02:52,110 So you'll see in classification models and whatnot there's top 5 accuracy. 35 00:02:52,170 --> 00:02:54,810 So it's the accuracy of your model. 36 00:02:55,020 --> 00:02:57,690 If you included the top five predictions. 37 00:02:57,690 --> 00:03:02,780 So it's usually higher than the top one prediction and if that doesn't make sense we'll put it up and 38 00:03:02,780 --> 00:03:03,230 I'll show you. 39 00:03:03,770 --> 00:03:08,420 So let's make this is gonna be a bit of a beast of a function because that's what we're all about. 40 00:03:08,510 --> 00:03:17,370 Big Dog functions on dog vision prediction probabilities labels and equals one. 41 00:03:17,570 --> 00:03:21,830 But we've got a lot of momentum from what we've been doing above we've trying to full blown deep learning 42 00:03:21,830 --> 00:03:26,600 model and the other thousand images but that's right we're gonna scale it up soon and we're starting 43 00:03:26,600 --> 00:03:32,930 to visualize what our model is capable of plus the top 10 highest prediction confidences. 44 00:03:32,930 --> 00:03:43,510 That's what we want to do along with the truth label for sample in that makes sense. 45 00:03:43,600 --> 00:03:57,310 So pred prob true reliable we need these two things equals prediction probabilities and labels. 46 00:03:57,790 --> 00:04:04,310 Wonderful get the predicted label spread label. 47 00:04:04,540 --> 00:04:07,110 Just gonna be using our function that we've used before. 48 00:04:07,120 --> 00:04:08,050 Get pred label 49 00:04:10,760 --> 00:04:20,270 prayed prob and now we're going to find the top 10 prediction confidence indexes. 50 00:04:20,330 --> 00:04:21,590 So this is what we're looking for. 51 00:04:21,620 --> 00:04:23,930 We need to find the top 10 because we've just done this. 52 00:04:23,960 --> 00:04:26,240 Let's take these off while we're here. 53 00:04:35,640 --> 00:04:37,020 Beautiful. 54 00:04:37,200 --> 00:04:40,820 So find the top 10 prediction. 55 00:04:40,920 --> 00:04:43,740 Confidence indexes. 56 00:04:43,800 --> 00:04:46,370 So before we do this in a function it's probably easier. 57 00:04:46,370 --> 00:04:50,130 I could just type it out but that's not a lot of fun. 58 00:04:50,130 --> 00:04:54,750 If you're not sure what's going on and what I mean by this let's just cut it out first. 59 00:04:54,750 --> 00:04:58,830 Come on Daniel stop talking predictions. 60 00:04:58,830 --> 00:05:01,970 Zero dot org sort. 61 00:05:02,250 --> 00:05:02,610 What's. 62 00:05:02,640 --> 00:05:06,010 OK so let's have a look at this okay. 63 00:05:06,070 --> 00:05:11,110 So what this is ARG sort is doing for us if we go predictions 0 64 00:05:13,950 --> 00:05:22,820 it's giving us the order of indexes that will sort this array so the highest value is here 1. 65 00:05:22,820 --> 00:05:24,320 So somewhere there. 66 00:05:24,320 --> 00:05:32,770 There we go so if we were to put all of these indexes in order this array would be ordered. 67 00:05:33,050 --> 00:05:36,080 Let's see that in action so negative 10. 68 00:05:36,080 --> 00:05:38,120 Because we want the top 10 what happens 69 00:05:40,870 --> 00:05:50,300 100 Now what if we go reverse that invalid Index. 70 00:05:50,330 --> 00:05:52,400 No scalar variable. 71 00:05:52,430 --> 00:05:53,250 Mm hmm. 72 00:05:53,660 --> 00:05:54,560 Well that's interesting. 73 00:05:54,920 --> 00:05:56,330 How would we fix this. 74 00:05:56,330 --> 00:05:57,970 I think what we might need to do. 75 00:05:58,160 --> 00:06:01,160 Grab the prediction probabilities. 76 00:06:01,260 --> 00:06:01,520 All right. 77 00:06:01,520 --> 00:06:03,100 I figured that. 78 00:06:03,110 --> 00:06:03,890 There we go. 79 00:06:03,920 --> 00:06:04,640 That's what we missed 80 00:06:07,430 --> 00:06:11,960 this little bad boy I say when you're slicing when you're slicing and dicing. 81 00:06:11,960 --> 00:06:13,520 Always be careful. 82 00:06:13,520 --> 00:06:14,870 Same rule when you're using a knife. 83 00:06:14,870 --> 00:06:18,470 Always be careful so that's a rule for slicing. 84 00:06:18,470 --> 00:06:19,100 What is this. 85 00:06:19,100 --> 00:06:27,400 This is the top 10 values so the top 10 indexes we want to reverse this. 86 00:06:27,400 --> 00:06:28,790 There we go. 87 00:06:28,790 --> 00:06:33,210 So here's the top 10 values of prediction 0. 88 00:06:33,270 --> 00:06:49,860 So what if we went predictions 0 then indexed it using that two point one six figure up here two point 89 00:06:49,860 --> 00:06:50,600 1 6. 90 00:06:50,610 --> 00:06:51,570 That's the max value. 91 00:06:51,600 --> 00:06:55,850 So let me just one last demonstration to know what's going on. 92 00:06:56,180 --> 00:06:56,970 Don't max 93 00:07:00,190 --> 00:07:00,970 two point 1 6. 94 00:07:00,970 --> 00:07:01,400 There we go. 95 00:07:01,390 --> 00:07:05,770 So there's the top 10 prediction indexes. 96 00:07:05,950 --> 00:07:08,310 So this is the prediction values as well. 97 00:07:08,590 --> 00:07:16,070 But if we remove this these are the top 10 indexes of prediction 0. 98 00:07:16,850 --> 00:07:17,690 So that's what we can do. 99 00:07:17,720 --> 00:07:21,250 We can take this and put it up here. 100 00:07:21,250 --> 00:07:34,660 So top 10 pred indexes equals pred prob the ARG sort negative 10 colon. 101 00:07:34,800 --> 00:07:43,020 If you're wondering what this double semicolon or double colon negative one does it reverses this that's 102 00:07:43,020 --> 00:07:43,850 all that's doing. 103 00:07:43,890 --> 00:07:54,940 If we took this here notice 17 is at this end and if we go here it's at this end but we want it in descending 104 00:07:54,940 --> 00:08:02,760 order so we do call and call a negative 1 beautiful so it come back up here. 105 00:08:02,780 --> 00:08:06,460 Now we want to find the top 10 confidence values. 106 00:08:06,470 --> 00:08:14,570 Now this is pretty easy now because we can just take this index here find the top 10 prediction. 107 00:08:14,720 --> 00:08:19,910 Confidence values and remember how I said prediction probability and confidence values kind of the same 108 00:08:19,910 --> 00:08:27,120 thing or at least colloquial Fred prob and we want to take top 10 109 00:08:29,670 --> 00:08:38,010 Fred indexes because we've already just found that so would be the equivalent of going predictions zero 110 00:08:39,120 --> 00:08:50,950 and then give me these indexes to the top 10 values of prediction 0 in descending order so we can take 111 00:08:50,950 --> 00:08:52,510 off two of these. 112 00:08:52,510 --> 00:09:00,030 Look at that look at us working through these functions like nothing. 113 00:09:00,210 --> 00:09:01,470 Boom. 114 00:09:01,510 --> 00:09:02,620 So what's next. 115 00:09:02,650 --> 00:09:03,810 What do we need to find. 116 00:09:03,940 --> 00:09:06,410 The prediction labels. 117 00:09:06,650 --> 00:09:07,230 All right. 118 00:09:07,260 --> 00:09:08,490 That's not too hard either. 119 00:09:08,490 --> 00:09:13,590 Find the top 10 prediction labels. 120 00:09:13,590 --> 00:09:14,820 Now one should have think about this. 121 00:09:14,850 --> 00:09:21,000 If we've got the top 10 prediction index values how could we use those to find the top 10 prediction 122 00:09:21,000 --> 00:09:22,230 labels. 123 00:09:22,230 --> 00:09:24,360 This is where unique breeds comes back into play. 124 00:09:24,390 --> 00:09:35,610 So top 10 red labels equals unique reads top 10 10 as if it's the start red indexes. 125 00:09:35,650 --> 00:09:38,000 Well look at what we've got. 126 00:09:38,100 --> 00:09:39,950 We've got all these top trends. 127 00:09:40,110 --> 00:09:45,610 Let's just copy and paste this copy if we want to see this in action. 128 00:09:45,880 --> 00:09:52,760 Let me just show you here so unique breeds top 10 credit indexes so we could take this. 129 00:09:52,760 --> 00:10:03,830 This is the top 10 indexes and unique grades plot that will index it on that amazing. 130 00:10:03,830 --> 00:10:10,120 So these are the top 10 labels that our model has predicted for prediction zero. 131 00:10:10,120 --> 00:10:12,010 So item 0. 132 00:10:12,180 --> 00:10:17,100 Now we need to set up a plot set up plot. 133 00:10:17,200 --> 00:10:18,750 We've got a top plot. 134 00:10:18,810 --> 00:10:31,940 Call it something simple we want to bomb plot so NDP arrange Len top 10 pred. 135 00:10:33,830 --> 00:10:35,450 So these are the x values. 136 00:10:35,480 --> 00:10:40,880 So I want my x values of my bomb plot to be the prediction labels. 137 00:10:40,910 --> 00:10:50,200 So you'll see what this looks like in a second and then after that we want the top 10 Fred values. 138 00:10:50,210 --> 00:10:55,620 So this is the prediction probabilities for each of our samples. 139 00:10:55,910 --> 00:11:00,840 And this is the color gray. 140 00:11:00,860 --> 00:11:07,270 Now we want to go plot x ticks is NPV a range. 141 00:11:07,320 --> 00:11:12,910 Len top 10 Fred labels. 142 00:11:13,040 --> 00:11:18,590 So this is just saying that we're creating NPA range land top 10 parade labels is because we want a 143 00:11:18,590 --> 00:11:19,650 range of 10. 144 00:11:19,690 --> 00:11:21,750 So we want 10 bars on our plot. 145 00:11:22,100 --> 00:11:29,650 And now we want the X ticks to be the names of our top 10 pred labels a.k.a. these. 146 00:11:29,690 --> 00:11:37,060 So along the bottom of our bar plot we should have the labels of our top 10 predictions here. 147 00:11:37,140 --> 00:11:45,200 Labels equals top 10 red labels. 148 00:11:45,200 --> 00:11:54,050 And then the rotation is vertical because otherwise if it's horizontal and the X ticks just strings 149 00:11:54,080 --> 00:11:58,000 that's going to get out of control and we don't want that. 150 00:11:58,100 --> 00:12:04,550 And one final thing we're going to add some color change the color of the true labels of the true label 151 00:12:04,550 --> 00:12:08,070 appears in the top 10 so NDP is in. 152 00:12:08,150 --> 00:12:16,100 Which is just if something is in something if one value this value is in this array color it Green 153 00:12:21,290 --> 00:12:22,370 will go top plot 154 00:12:25,260 --> 00:12:26,890 and then we'll index it. 155 00:12:26,910 --> 00:12:41,560 So NDP Max so the maximum value of top 10 at pred labels equals true label what is the index of top 156 00:12:41,560 --> 00:12:42,430 ten read labels. 157 00:12:42,430 --> 00:12:44,460 Where is that equal to true label. 158 00:12:44,470 --> 00:12:54,040 Get that and then set the color of that index to be green. 159 00:12:54,350 --> 00:12:55,430 Else pass. 160 00:12:55,430 --> 00:12:56,960 We don't want to worry about that far out. 161 00:12:56,960 --> 00:13:03,200 That is a lot going on but let's see it in action if in doubt run the code and again if you want to 162 00:13:03,200 --> 00:13:07,400 test out what's going on in each of these functionalities don't forget you can always break it out. 163 00:13:08,290 --> 00:13:17,860 So plot thread comp prediction probabilities equals predictions 164 00:13:20,600 --> 00:13:27,870 labels equals vowel labels and N What do you want might be number nine. 165 00:13:31,140 --> 00:13:31,960 What's happening here. 166 00:13:31,980 --> 00:13:32,660 Unique breed. 167 00:13:32,670 --> 00:13:34,940 We got a typo as usual. 168 00:13:35,370 --> 00:13:36,390 Unique breeds 169 00:13:40,480 --> 00:13:44,410 and again rotation vertical spelling vertical wrong. 170 00:13:45,940 --> 00:13:50,860 Vertices El plot trade cough. 171 00:13:50,960 --> 00:13:51,850 There we go. 172 00:13:52,160 --> 00:13:52,900 Check that out. 173 00:13:52,910 --> 00:13:55,720 So this is pretty cool. 174 00:13:56,800 --> 00:13:59,200 So along the bottom remember what we said we're gonna do. 175 00:13:59,230 --> 00:14:03,910 We're gonna go to the X techs are the top 10 prediction labels so these are the top 10 predictions that 176 00:14:03,910 --> 00:14:05,260 our model is made. 177 00:14:05,260 --> 00:14:08,080 And as you can see the first index one the. 178 00:14:08,080 --> 00:14:12,640 So this is the predicted label because it's got the actual predicted label because this is got the highest 179 00:14:12,640 --> 00:14:19,110 value prediction value and these are the other top 10 value so you can see how they go down after the 180 00:14:19,110 --> 00:14:25,200 top value and the true label is the one in grain so Border Collie. 181 00:14:25,200 --> 00:14:31,440 So when our model is done as predicted Collie but the correct prediction is actually border collie. 182 00:14:31,440 --> 00:14:33,530 Now to me that kind of sound like the same thing. 183 00:14:33,570 --> 00:14:35,190 Let's figure that out. 184 00:14:35,300 --> 00:14:35,860 Let's go. 185 00:14:35,870 --> 00:14:40,270 Collie dog images. 186 00:14:43,250 --> 00:14:43,820 Border Collie. 187 00:14:43,850 --> 00:14:45,560 Well Mm hmm. 188 00:14:45,650 --> 00:14:48,890 This doesn't really make sense to me Border Collie 189 00:14:51,690 --> 00:14:56,110 so even Google Images doesn't really show images of colleagues it just shows Border Collie so you can 190 00:14:56,110 --> 00:15:00,080 kind of see why our model I predict Collie instead of Border Collie. 191 00:15:00,100 --> 00:15:10,590 But it's also predicted malamute what's a malamute malamute o okay. 192 00:15:10,760 --> 00:15:14,390 Well you can kind of see a dance got black and white just like a border collie. 193 00:15:15,140 --> 00:15:18,050 So this is where we're starting to really understand what's going on with our model. 194 00:15:18,050 --> 00:15:22,040 Let's delete some of these cells in the next video. 195 00:15:22,040 --> 00:15:27,620 Now that we've got these two functions that it can plot some intuitive understanding of what our model's 196 00:15:27,620 --> 00:15:29,110 predictions are doing. 197 00:15:29,330 --> 00:15:31,080 Let's take this one off here. 198 00:15:33,360 --> 00:15:42,600 Let's write some code to combine these two plot spread and plot pred conf to have the image over this 199 00:15:42,600 --> 00:15:49,420 side and the top 10 prediction comes on the right so maybe we could do it for 10 images or something 200 00:15:49,420 --> 00:15:51,730 like that but I'll see in the next video.