So we've seen how we can evaluate our models performance or at least initial performance on something
like epoch accuracy an epoch loss and see how we can track different experiments.
But this is just on the models training and validation sets.
And it's only really on what the model has learned in the data set.
What are other really good way to evaluate what our model has learned is to use it for what we're building
it for.
If we're building dog vision and where you're always going to keep in mind what the end goal is with
whatever project we're working on we're building dog vision to see if we can build a machine learning
model that can identify a dog breed in a photo.
So it's going to make some predictions giving a photo so let's do that if we come back to have a look
at our workflow.
We fit the model to the data.
Now it's time to make a prediction.
So we're in we're still in Step 3 We've done a little bit we've touched on step four.
And remember these aren't really linear steps.
So these are just I've just put them here because it's easy to understand it like this.
But remember they aren't necessarily linear so you can do them out of order.
We come back we'll make a little heading making and evaluating predictions using a trained model command
ma'am for turning it into markdown shift and into Okay so making predictions with a trained model is
very similar to how we did it in socket loan.
So we can call the predict function and pass it data in the same form that the model was trained on
that's the important thing.
Let's see the code first and then we'll discuss what's happening so make predictions on the validation
data.
So remember we created a validation data batch.
Now this was not used to train on.
That's the important point.
If we come back to our three sets how we evaluate a model is because it trains in the training set.
We want to check it out.
Its initial results on the validation set and then both of these exams so the practice exam and the
final exam are data sets that the model hasn't seen before.
So we come back.
Let's go.
Model or actually we'll save it to predictions predictions.
Equal model don't predict Val data because let's just remind ourselves of what value data is the data
batch data set.
There we go.
We've got some images and some labels so we know what the true labels are of the validation.
So when we pass this batch data set to model not predict because of the nature of the predict function
it's only going to look at the images in our data and then make predictions based on those images.
So then what we can do is compare those predictions which are going to be in the form of labels while
not exactly we're going to see what they look like in a second.
We can compare the models predictions that it makes on the validation images to the actual labels from
the validation images.
Now that was a lot of talking.
It's much better to see this in encode.
So verbose is just going to say hey when you're making predictions.
Show me your progress
while that was really quick because we've got 200 images.
So that's the beauty of working on a GP you predictions are really fast as well.
And so that only took about 187 milliseconds so not even a whole second.
And if we didn't set verbose to equal one we wouldn't get this little progress bar at putting you might
be looking at this and going what is going on here.
Lots of just different numbers.
So let's look at the shape here prediction shape for remember.
Where does this line up.
So if we go Len y Val two hundred.
So there's the first shape.
So two hundred images.
Where do you think the second shape is coming from.
Hundred and twenty where have we seen that before.
Len unique breeds
120.
So this means that we've got an array of 200 by 120 so we have 200.
Let's see the first element so we have 200 go predictions zero two hundred arrays of one hundred and
twenty different numbers that are really small.
And we're wondering what is going on here.
Well let's find out the length of this.
Hundred and twenty.
Wonderful.
So what this actually is is an associated probability
for the likeliness.
So basically what our model thinks a certain image is so there's a probability value here for every
single label.
So the value here the highest value in this predictions array.
So the zero predictions array.
This is for one image.
The first image in the validation data so the highest value in here is going to correspond to the index
of the label that the model thinks is most likely.
And again this is a lot of talking it's gonna make a bit more sense once we start to put it together
with code but just bear with me for a second.
If we sum up all of these they're going to equal very close to 1 or maybe 1 Exactly and I say very close
because the way computer store numbers is not exactly.
So if you keep going decimal points right it's gonna get very close to one but it might not be exactly
one and you might be saying Daniel what are you telling me there's the way the computer store numbers
is not exact.
Well I don't even fully understand it but you just have to know that when a computer stores a number
that has lots of decimals because of the way it's most efficient to actually stored on a computer chip.
It's not going to be perfectly exact.
So that's why I say when you sum up a single prediction array it's going to be very close to one and
now I'm going to tie this back up into where we've come back into our model.
This is why we've used soft Max activation.
You might be saying Daniel far out.
This is a video as ago go.
You've already told us about soft Max it didn't really make sense and it's really not making sense now.
But if we go back to soft Max we just go Yeah how about we go what is soft Max.
If we come back to our friendly Wikipedia page it's going to tell us that but after applying soft Max
each component will be in the interval between 0 to 1 and the components will add up to 1 so that is
what we're getting here.
We're getting our components because we've used a soft Max activation in the last layer of our network.
It's output it an array a prediction array that is 120 in length and all of these values in prediction
0 and each other.
Prediction here.
So predictions 1 will be very close to 1 as well.
So predictions 1 add up to 1 or very close to 1 0.
Predictions.
There we go.
See this is just over 1 so very close to one of the more decimals there are the less exact a number
is with a computer.
And so let's put this all together because right now it's just been a lot of talking.
I want to show you a concrete example.
Let's take predictions zero.
So we're going to remove these cells
and just one little tidbit as well.
Remember our mobile Net V2 architecture.
All that that our soft Max layer is doing is taking these 1280 output.
This array of numbers and changing it into an array of numbers like this of length one hundred and twenty
using a soft Max activation.
Because remember for four multi class classification problems we want a soft Max activation but if we
were doing only single classification we'd use sigmoid.
So to come back here.
Let's check it out.
So the first prediction I'm going to type out a few little indexing tricks here and I just want you
to just watch on I'll talk through it a little bit as it goes but I want you to start thinking about
how these things all tied together so let's do it.
Because what our goal now is our model has made some predictions here.
But our goal is to understand these predictions furthers our computer understands these right because
they're old numeric.
So remember at the start how we spent a lot of time turning our data into numbers kind of when you get
to the output of a machine learning model right.
So this is what we're focused on we spent a lot of time preparing our inputs.
Now we're going to spend a lot of time preparing our outputs which is what we're doing now so let's
go print prediction zero and then what we're going to do is go find the max value probability so remember
the function inside get loan you might not call predict program which outputs a prediction probability.
This is what predict does by default in tensor flow when we go here.
Prediction and pay Max predictions zero.
So we're finding let's remind ourselves prediction 0 what it looks like.
We're finding the max value in here that's all that line does and then what are we going to do next.
So print we want to.
What else do we want.
We want to find the sum so NDP are actually we might set this up so we can do multiple so index equals
zero just so we can see it with multiple examples.
So the sum sum is going to be NDP some predictions index.
Nice and simple then we're gonna go wow that was a terrible typo.
Daniel come on we're gonna go to the max index so find the index in this array which has the max value
so which is equal to max value NDP ARG Max predictions index.
And of course what we're doing now is a little bit tedious.
We're going to function ize this later on.
So you could imagine if we just kept writing cells like this this is where we're functioning using all
of our code as much as we can so that we don't just have to continually write these type of print statements
to explore our data or explore our predictions predictions index wonderful predicted label now before
we run this cell we're going against our cardinal rule of if in doubt run the code once you do just
have a think about what this one is doing.
Let me remind you of what unique breeds is
so these are all our dog breeds so if we have predictions and we find the max index of it and then we
find where that index occurs in unique breeds What do you think that will return hint we've typed it
out here all right.
Let's delete this bad boy and then we'll shift into
wonderful.
So here's index 0.
This is that array just the exact same here.
The output of our model when we call predict.
So the max value the probability of prediction the maximum this can be as one because remember the total
of a soft max function is between 0 1 so the highest a single value can be is 1.
So this is saying we're predicting the highest label in here with a twenty one point six percent prediction
probability.
Now the sum of all of these values is very close to 1.
Again aligning with the soft Max rule between 0 and 1 and will add up to 1.
But because of how computers calculate numbers not exactly 1 and the maximum index are where this value
occurs is 17.
So if we counted three we find 17 somewhere maybe there.
That's it.
Now the predicted label is border area.
So for index 0 or for validation image 0 the predicted label is Border Terrier.
Now let's change this up.
What if we did.
What if we did INDEX What's our favorite number 42
so we got max value probability of prediction so this is a lot higher.
This is seventy five percent so it's pretty confident with this one.
So you could say the higher this value is to 1 the closer it is to 1 the more confident a model is about
a certain prediction the sum is very close to 1 the maximum index is 113 and the predicted label is
walking hand.
So if we go here unique breeds if we were to find index number one hundred and thirteen we would find
walk a hand.
There we go walk around
113 walk a hand.
So this is how we convert our prediction probabilities in numeric form something that our computer understands
to something that we understand a predicted label.
So now we've got a kind of way to turn our predictions into something that we can understand what we'll
start to do next is we'll create a bit of a function that's going to allow us to find the predicted
labels for every sample in the validation set So about 200 or so images and then we're going to UN batch
this dataset the validation data and compare the predictions in a visual way to the truth labels now
again a lot of talking but in the next video we'll start to build out that functionality and we'll see
what it looks like in reality.
And I'm so excited right because one of the funnest parts is comparing your model's predictions with
what's actually going on dog vision is coming to life.