In the last video we talked about how we can convert our model's prediction outputs okay.
This long list of numbers that are really hard to understand if you're just looking at them for the
first time into something that we can understand a predicted label in the form of a string or what we
want to do is to make it a little bit more visual because this is pretty tedious just comparing it like
this.
We'll create some functions that are gonna help us basically do this step as well as visualize the actual
image that a model is making a prediction on and then compare it to that image is actual labor or the
truth label and just a little tidbit.
These are also known as confidence intervals.
If you hear them this is a prediction probability.
There is also confidence interval.
So one methodology of evaluating a model is to go okay.
Only give me the predictions that have a confidence value over point seventy five.
Every prediction that's max value is under point seven five gets pushed to the wayside.
Those are things you might want to pass to a human classifier but everything over point seven five you
might want to let the model predict itself.
Let's write a little note to ourselves.
Having the above functionality
is great but we want to be able to do it at scale and it would be even better if we could see the image.
The prediction is being made on part of being a data scientist or machine learning engineer is not just
being able to write this sort of code.
A more important step than building a machine learning model is evaluating it.
So this is what we're kind of doing we're communicating to ourselves what's going on because it's one
thing to just show someone and accuracy score.
We could just go yep there you go.
Our model's accuracy is 100 percent on the training set and that might go well.
Okay.
Let me see what's going on.
So that's what we're doing here.
So I've yet to report it to someone else or if you had to show a colleague or maybe your partner with
a business who doesn't really know what you're talking about when you tell them the probability of prediction
is zero point seven five.
It's much easier when you can show them something visual.
So we go here.
Note prediction probabilities are also known as confidence levels markdown.
Wonderful.
All right.
So a little function.
Let's go turn prediction probabilities which is what we have up here.
These are all probabilities so if you hear me say prediction probabilities I'm referring to a long array
of numbers like this that are probabilities.
And we go here turn probabilities into their respective label.
Easier to understand.
So we'll go def get.
We might do it of a single one so maybe get pred label will pass it.
Prediction probabilities array which remember is just this prediction probabilities
because what we're doing here.
Remember our goal is to just function eyes this ugly Prince damn well it's not really that ugly it's
actually some pretty neat code but we want to make this available to us at scale.
Prediction probabilities yes.
And then we want to label our doctoring for ourselves turns an array of prediction probabilities into
a label we can just do a simple return statement here.
Unique braids or we want to do is index a unique breeds for the ARG max of prediction probabilities
again we're just replicating this one here all we're doing is we just want the predicted label from
this list of numbers let's see what that looks like if we want to let's give a little example get a
predicted label based on an array of prediction probabilities typos galore Fred label get pred label
predictions let's go on the first index so all we're doing we're just calling our function our beautiful
little function that we've just written and hopefully this returns us the parade label of 0
proletariat that's what we're after 9.
What's your favorite number mine used to be eighty one because nine times nine is eighty one one plus
eight is eighty one eight minus one is seven I have a dog called Seven and I had a small lacks level
eighty one once I traded it to a little American boy who has made a friends with at a resort his name
with Gareth and he left the resort without ever trading back my know that that was the end of that story.
All right wonderful.
We've got a list of different predictions that our model has made and we've got some functionality to
turn those prediction probabilities into a label.
So we're making our predictions more easier to evaluate for our eyeballs.
We've already got some metrics on them with tensor board.
Now what we'll probably do is sense our validation data is in a batch form it's a batch dataset.
We want to get a list.
We want to UNbacked it basically because we want to rip out those images.
So we want to get these images out of here and these labels because these are the truth labels.
That way we can make predictions on the validation images and then get our prediction labels using our
function and then compare those prediction labels to the actual validation labels so let's make a small
function which does that.
So what we'll do go here.
Now since our validation data is still in a batch dataset remember it's in a batch dataset because that's
very efficient tend to fly to make calculations on.
We'll have to on batch ify it and just made that word up to make predictions on the validation
images and then compare those predictions to the validation labels.
Truth labels.
This is gonna be really cool once it is up and running.
So let's right here.
Create a function to UN batch batch data set.
The people who make tens of flow have actually thought about this problem well not even a problem it's
just a little roadblock because if you batch the data set surely there's a way to unmatched and there
is if we go tensor flow 2.0 on batch
whereas in data set on batch
and batch here we go splits elements of a dataset into multiple elements.
For example if the dataset elements a shape b something something something where B may very fit I'll
let you read that if you want but basically you've got elements here have data set from generator data
set down on batch list data set as Num by iterator.
So once we have unmatched it we can call as Num pi iterator and it's going to turn it into an intolerable
list.
So again if this code doesn't make sense when you first rate it the best way to figure it out is to
write it.
So let's do that let's write it in a non function first and then we'll function is it.
So images because we want to get the validation images or we want to on batch these we want to get these
and these into separate entities not one labels will make an empty list oh maybe these I don't want
to override a variable I had before.
So like images will just underscore them.
There we go then we want to loop through unmatched data.
So we go image label right.
We're going to topple here of image label in Val data dot unmatched.
So all I'm doing calling on batch and I'm going to do these in one hit.
So I'm going to call on batch and dot as Num pi iterator in one hit so let's see what happens if we
go print image.
This might actually bus our computer but we'll see it anyway.
If in doubt run the code what's it going to return boom so there we go it's gonna go that's gonna go
for two hundred or so because that's how many samples are in our batch now validation batch but all
it's doing now is printing out the image and the label image and the label image and the label but the
image is remember in the form of a processed image a pre processed image which is that pre processed
image function we wrote right at the top.
We don't want them just like that we want to apprehend them to our list of images so append image and
labels append label.
So now what we should have if we go image is zero.
Image is zero.
So there we go so there's one image and its counterpart label should have just done that the same time
label zero
why in that comment.
That's why labels
see confused myself with the variables that we're saving.
This is why it's helpful to function I things because if you create different variables and you lose
track of of what variables they are you confuse yourself there we go so there's a true label.
So this is image validation zero and it's ground truth label so our goal is to visualize this image
and compare this ground truth label to the prediction label that it's made through here.
Go get pred.
Label labels 0
wonderful Khan with the prediction label was so we could go get Red Label predictions.
0
Border Terrier.
So that's a prediction that our model got wrong.
So this is what we're trying to visualize right now.
Let's do this.
That's not really that fun and we want to make things fun so let's function eyes this and then we'll
finish the video there.
Actually that's a challenge before I go off and start functioning using this little sequence I want
you to give it a try.
Call the function something cool like UNbacked Fi and it should take an input of a data set a batch
dataset and it should do this functionality and return these to list.
So that's your little challenge.
I'll give you a couple of seconds all right.
I'm back.
Didn't really leave I was here the whole time.
Hopefully you had a little turn of a little try of a function izing this beautiful and batch of five
function that we're going to write so we'll write some dark string takes a batch dataset of image label
tenses and returns separate rise of images and labels tick tick tick and we want to create images because
it's within a function we can use whatever variable name we want and then we're gonna go loop through.
So basically all I'm doing is just rewriting this up here instead of copy posturing I'm just running
it out again because I want to practice want to practice.
So for image label in all we only have to use data so this is because I've set up a data parameter so
I can just pass it any batch data set I want.
So data on batch
dot as Num pi iterator beautiful images dot append image labels dot append label and then we can return
images labels we test add a little functions on batch of fi let's see what your abilities are like on
batch if I do you stand up to the challenge of batch of filing now a validation dataset equals on batch
of Fi and we're gonna pass it allow data batch exactly the same as what we've done here we just function
I used it and we'll check out Val images zero and Val labels zero
boom suggests the exact same thing they're that's what we've got here.
One final check is we can get pride label what was it OK can maybe I'm saying that wrong figure here.
So get Red Label Val images 0 0 9 needs to be labels 0 did that okay.
That worked all night.
So that's gonna work because that's numbers as well.
So we actually want Val labels right card.
Wonderful.
Okay.
So now we have a function for unmatched filing our data we can get our validation images which is great.
So we can now work out some little functionality to make predictions on those images we can get the
ground truth label and then compare that to the prediction label Let's delete this code here and we'll
finish up the video and in the next one what should we do.
We should make some functions to plot an image so visualize an image of a beautiful little dog.
We'll also compare the truth label to the predicted label that sounds like a plan to me.
I'll see you in the next video.