So now we've got a fully trained model on the full training dataset a.k.a. ten thousand images.
Because if we look at X we got 10000 images and labels.
Now let's use our fully trained model to make some predictions on the test dataset which is also about
10000 images we'll have a look at that in a second.
But let's come up to Kaggle and see how they want our predictions to look.
So a submission file for each image in the test set you must predict the probability for each of the
different breeds.
The file should contain a header and have the following format.
So we need eye day dog breed one.
I'm not even going to pretend I'm going to try to pronounce that once again Afghan hound Yorkshire terrier.
And then the prediction probabilities for all of those.
All right.
Now it says multi class log loss.
That's just the prediction probabilities for each dog breed create another header section making predictions
on the test data set.
Now I want you to have a think before we go through the code.
What did we have to do to train our model on the training images again.
Pretend we're this is data agnostic whatever data are working with whether it's text or audio or images
or video or something like that.
What is our rule before we can use it with a machine learning model or in our case a deep learning no
network we've got to convert it into numbers.
So that's what we have to do the exact same data type that our model was trained on.
So the data batch up here the data batch of images we need to create one of those out of the test images
so that we can make predictions on those images so we'll write ourselves a little note.
Since our model has been trained on images in the form of tens of batches to make predictions on the
test data we'll have to get it into the same format.
So if we come back to our graphic and keynote remember how I always said we're focusing on the inputs.
Okay getting our data into the right shape for our machine learning algorithm and then making sure the
outputs are correct.
This is what we focus on in the beginning.
This is already being implemented for us in quite a large way as we get deeper and deeper as you get
into further projects.
Then you might want to start diving into this but for the time being and for approaching your first
projects like we're doing now you're gonna be focused on inputs and outputs now a little tidbit here
is again our functions coming in handy.
Luckily we created create data batches earlier which can take a list of file names as input.
If you don't remember go back up to where we create data batches and convert them into tensor batches
so what steps do we have to do to make predictions on the test data.
Let's write those down to make predictions on the test data.
Well now this is kind of what I do with a lot of my projects is just talk to myself talk myself through
it.
The rubber ducky technique so get the test image file names that'll involve us writing just a little
list comprehension like we've done before to access the test go through all the files in here and save
the file names to some sort of list Yep we can do that and then we've got to convert the file names
into test data batches using create data batches and setting the test data.
Remember how we created parameters and create data batches if you don't that's okay we're gonna see
in the second parameter to true because why what's different about our test data to our training data.
There's no labels with the test data.
So since the test data doesn't have labels and then finally we make a predictions predictions array
by passing the test data batches to the predict method called on our model.
We've got three steps there but we can do them we can tackle them one by one.
How about we start with the first one.
That's pretty logical.
Load test image file names
what could we do.
We could go test file names.
Nice and simple equals.
Actually it might set up the test path.
How about we do that test path equals and we'll just copy it here to save us.
Writing it out twice copy path.
That's a little handy tip you can do that and paste.
We don't need content we could just keep it there if we wanted to.
But I'm going to remove it so it's consistent with the other path that we have.
And I'm gonna put this one here so that's my path to the test folder.
So now we're gonna go create a little list comprehension equals test path plus f name file name for
F name in our stop list test path now we've seen this before when we're creating the original training
file names but let's just remind ourselves what it's doing.
So our stop Lister is just going through a list directory that's what it's short for.
It's just saying hey Python get me all of the file names in this folder and then we're creating a list
which appends the test path which is this all adds together.
The test path plus every single file name in the test folder let's have a look.
If in doubt run the code.
Wonderful.
So now we've got the file names towards our test data.
We can take that off get our green take emerging.
This notebook is gonna be filled with them by the end.
Okay.
Now we've got it.
What's our next step.
Convert the file names into test data batches using create data batches.
Oh I said date there.
That's wrong.
Now we go fix it up first of all let's just do a little a length check of our test file name see how
many images we have
ten thousand three hundred fifty seven.
Wonderful.
So now to create a data batch it's pretty simple.
We just bring in our function we go test data we write a comment here before we get started.
So create test data batch and we're going to create data batches on it knows what we're talking about
and then we're gonna pass it the test file names and we're also gonna see there's our pest data Lane.
So we go to test data equals true and we'll run that wonderful.
We get a little printout saying creating test data batches such a great function that we wrote.
Let's come up here and have a look at it again.
Turning our data into batches so we've got a function create data batch create data batches and what
does it do.
If it's test data which it is it prints out this little statement and then it goes hey turn our file
names into tenses and then use from 10 to slices to convert it into a tensor dataset and then create
a data batch by mapping the process image function.
Remember we don't pass get image label to the test dataset because there's no labels with the test data
set.
So we process our images just like we did on the training data and then we turn it into a batch.
In our case batches of size 32 so that it can be computed really fast on with our GPA.
So let's come back to where we were.
That's what we've got.
And if we go test data
it's a batch dataset of shapes.
So there's no labels here.
These are just images of two to four by two to four by three for the color channels of type tensor float
float.
So beautiful.
We've got our test data in the form of tensor batches.
That's what we're after.
And now the finale we can come up here see how handy writing our functions to begin with has helped
us out.
We've created a full model we've grabbed a data batch in one hit.
Now we're going to make a predictions array.
So how can we do that make predictions on test data match using the loaded full model.
All right.
So test predictions Eagles loaded full model don't predict test data.
We're going to set this to the Bose Eagles 1 Excellent.
Now I kind of have to warn you that this is going to take a fairly long time reason being is because
let's put a little note here before we run MSL.
Note calling even though we're running on a GPO.
This is just like training our four model so calling predict on our full model and passing it the test
data batch will take a long time to run and then it's gonna be something like about now.
We'll just go about one hour that's what I found anyway.
Reason being is because we have another 10000 images here and what we have to do is get our loaded for
models so all of the patterns that are loaded for model has learned and the training data set it has
to go through those process the 10000 images in our test file names or our test data batch and then
make predictions on those based on the patterns it finds in that test data.
So we've set the votes equal to one so that when we run this it's going to give us another little time
step to update us about what's going on.
So without any further ado let's run this and wait for it to load and then we'll do the same thing again.
All speed up the video so it'll be like instantaneously loaded for you.
In reality it'll take you a while to run this cell.
In reality it take me a while to run this cell but I'm just doing it to show you it loading up creating
an EPA message and then going through making some predictions on the test data so we'll just wait a
little while this is going to go up.
There we go.
So EDTA about 38 minutes.
It has to go through 324 batches and so that is because if you go ten thousand three and fifty seven
divided by 32 it's going to round that up to 324.
That's where that number comes from
okay.
So I'm going to wait for this to go through.
You might have to wait for it to go through as well you don't actually have to wait.
I'll be back in a few seconds so I'm going to pause my video here and I'll be back in three two one
and we're back.
Now I've gone and done something a little bit cheeky here as you see we've got about an hour left but
I figured rather me sit here and wait for this.
I've actually like any good chef has done I've prepared a dish earlier.
So I've actually gone through this process and waited for one hour to go through.
And now you could wait for yours to go through but if you'd like to download a premade CSB of the test
predictions all attach that in the resource section so you don't have to wait that full hour to go through.
And now I probably should have told you this before running this cell but it's a good thing to sort
of just get your feet wet and then figure out a better way of doing things.
If we wanted this to save test predictions to a CSB file after it runs through because if we waited
a full hour for it to go through and then we somehow ran time disconnected and it didn't save they've
been pretty disappointing.
So what we can do is use MP dot saved text this function here to save will this loading dock Schering
while this is running.
I don't think it will.
But what this is going to do.
Oh we forgot this parades array.
So let's just say that this finished if I ran this cell after it it's going to save test predictions.
So this num pi array which is a prediction probabilities array to this file as a CSB with the delimiter
comma for comma separated values.
And then once it was saved it's going to appear in our files as spreads array don't see us V because
that's the file name we gave it and then we can use MP dot load text to load it back in so what we'll
do rather than wait for this to finish I'll just show you that in action.
So these are some prediction probabilities that I've made in the past with a full model.
So I'm going to stop this had about an hour to go.
So again my estimation it's going to take about an hour is incorrect.
So keyboard interrupt.
There we go.
So we're not going to run that because I've already I've already saved this to press array but I am
going to run there so let's pretend I've run this after this cell has completed so this EDTA has reached
0 so I'm going to load in this array from a c v file
beautiful.
Now let's have a look at the first 10.
There we go.
So these are all prediction probabilities for the 10000 images so let's have a look at the shape of
test predictions
ten thousand three and fifty seven so that's how many test images we have.
And each one of them has 120 different prediction probabilities.
Now we've seen these values before and so remember I've just taken this.
Let it run to its extent save this HSV to a file that I've done earlier as if we've cooked something
earlier and then I've just reloaded it in here.
If my runtime disconnected I'd want this file to be saved somewhere that I can access it later and so
what's our next mission.
If we wanted to submit this predictions array to cargo what would it have to look like.
So the sample submission is here.
Something went wrong all letting your data.
That's not very fun.
But luckily I've got this file in dog vision and it's a V.
And so all I've done is I've just download it and opened it and Google Sheets.
So this is what it looks like.
This is the the sample submission we need to get our predictions array into something like this.
So an idea column and then a column for each of the different dog breeds and then their prediction probabilities.
So that's what we'll work towards in the next video we'll get our prediction probabilities array for
dog vision or we're actually going back to the Kaggle dog breed identification format.
So we'll get it set up like that so we can make a submission to Kaggle with our predictions.