We've got a predictions array but now we need to get it in the Kaggle submission format so that's what
we'll be working towards in this video.
Let's go make ourselves a little heading.
Preparing test data set predictions for Kaggle and if we look at the sample submission we kind of know
what we need and we know we need an I.D. column which is the file I.D. from our test images.
We know we need columns for each of the different dog breeds and we know we need the prediction probabilities
which is what we've got at the moment we've got an array with all of these numbers.
We basically need to just set up our predictions probability array into a data frame like this.
Export it as a CSB and then upload that to Kaggle.
So let's write that down.
Actually we'll put a little link in here of what Kaggle wants a valuation.
Copy that.
This is just what the sample submission has to look like.
Looking at the Kaggle sample submission we find that it wants our models prediction probability outputs
in a data frame with an idea column and a column for each different dog breed.
Well it's not really a data frame it's a CSA but we can be a little bit incorrect there in our wording
and how we're gonna do this.
Let's write ourselves some guides.
So to get the data in this format well this is a testament to our favorite little graphic here of preparing
inputs and outputs.
So now we've worked with the inputs we've saved and reloaded our train model.
Now we're making a bunch of predictions that we have made predictions.
We're preparing our outputs more specifically preparing our outputs for Kaggle.
We will create a panda's data frame with an I.D. idea column as well as a column for each dog breed
and then we're going to add data to the idea column by extracting the test image ideas from their file
parts and then we're going to add data.
The prediction probabilities to each of the dog breed columns and then we want to export the data frame
as a CSP to submit it to Kaggle.
This is what we're after.
And probabilities is spelled wrong standard Daniel type poking some nice little fool stops all right.
So the first step.
Let's go through and create a panel is data frame we've had experience of this way back in the panda
section with empty columns.
So we'll go to parades DLF equals PD data frame so just the standard nomenclature for prediction data
frame parades IDF parades data frame all predictions data frame columns is going to be equal to I D
plus we need a column for each different dog breed so we'll do one list unique breeds and I just want
to show you what list unique breeds looks like list unique breeds
so there we go.
So if we added I.D. to that let's give that about I date IPS trigger happy once again plus there we
go I.D. plus you list unique breeds
that is some column names for us we'll comment out that and we will check out the head of our spreads
IDF should be empty with just a bunch of column names that's what we're after I.D. and all of our unique
breeds in order we've done this.
So putting in some emojis.
All right add data to the idea come on by extracting the test image I.D. from their file paths I believe
we have already done that but what we might do is just do it again test file names but the way we can
get each I.D. is by we want to extract this out so we could do a little bit of fancy regex on this but
I'll show you another way append test image I.D. to predictions data frame.
So we want to go test path look did we already create test path we did.
Okay.
Wonderful.
So let's get test I.D..
So this is just the path to our test file and test I.D. is going to be equal to a list comprehension
of our stock path.
And then what is it split text.
I'll show you what this does in the second split text and for path in always don't list the test path.
That should be correct how can I show you an example.
Always step path dot split text.
We want to get test file names 0.
Well this work.
There we go.
OK.
So it splits off the last little bit of the path but we only want this.
So that's why we have to do it for just the test path so we want to just go through this test folder.
Now it's not going to show up here is it because there's a lot of images in there.
That'll take a long time to load and Colette you're just going to have to trust me with what this does
let's just see it in action anyway.
Test ideas.
Will this work if in doubt run the code.
All righty actually we don't need the JPEG bit.
So what we're going to have to do we only want the first element of each of these.
So we're going to have to put a little zero index there and let's see what this does.
That's what we're after.
Test days.
I don't want a large print out there.
So what I'm gonna do is just go Fred's DNF.
So I'm taking my data frame from up here.
I'm going to make an I.D. column with the test I.D..
So just taking that and putting that there little bit of overkill running this cell three times but
that is all right.
Let's check out the head of our spreads ADF as we always do when we manipulate our data okay.
So you've got some ideas in there.
Now we need to just append all of our prediction probabilities for each dog breed.
Now how might we do that.
Mm hmm.
I want you to have a think about this for a second.
What do we have.
We have test predictions which is an array of predictions probabilities and what do we have that's a
list of different dog breeds.
We have unique breeds cos we do so add the prediction probabilities this is getting exciting we're getting
so close to being out to submit our first full blown DB learning model to Kaggle.
We've gone from head to toe the whole process the whole shebang.
She can't tell I'm pumped.
So this is going to access the parades DLF of all of the unique breeds.
So all of the braids columns and then we want to just set that easy this is to test predictions.
Parades the F dot head
and we are done ladies and gentlemen have a go at that.
We would compare that to the sample submission.
Of course this goes on for a lot longer but ours is using the model we have built.
The numbers are going to be different but that's okay.
Now let's export it to a CSA so we can submit it to Kaggle.
How do we export Panda's data frame to CSP.
We use to see as vehicles we want to go drive my drive I'm going to go to dog vision and I'm just gonna
save it as maybe full model predictions submission one CSA or maybe we'll all tell ourselves my net
V2 don't see us.
And we don't want index.
So index equals false we'll go here.
We'll just write what we're doing save our predictions data frame to see us v for submission to Kaggle
is is gonna save.
Fingers crossed.
Yes of course it will.
Now let's have a look in here.
In our files oh maybe we've glitched out we've just saved a c v there may be lives lost the plot on
us.
Give me a second for my lab window to reload and I'll be back once I have access to this folder okay.
I'm back.
So sorry about that little interruption in broadcast.
Seems my collab window decided to go bonkers.
I see why.
Because I decided to open the test folder and I need to close that and I might have to pause it again
one second.
All right.
Third time's a charm.
We are back.
So be very careful with folders that you open in your lab window.
If they're full of data a.k.a. 10000 plus images.
It will freeze up your co lab notebook there we go for model predictions.
Well we can't really see the whole thing there.
Oh there we go.
And that's just our CSB file there.
So if we wanted to we can download that.
We do want to.
So we'll just wait for that to go through.
I'll speed up my video until this is fully downloaded Ronnie.
So this is fully downloaded.
All I did was just click download and then waited for it to go to about a minute or so not too long.
This is about a 30 megabyte CSB.
We come to my downloads folder.
If we come here.
Yeah about a 30 megabyte C as V so 10000 predictions.
Now we've got that.
Let's go to cargo and we can go late submission because this competition has already close you can see
my past one there five days ago.
So let's upload I want to show you that this works open.
I'm just selecting the CSB we just downloaded it's gonna go through a briefly describe your submission.
Let's go.
Transfer learning that's what we'll call it because that's what we used.
Transfer learning my ball net the to dog vision.
Baby and then we'll make submission and it's gonna score it.
Now there's something up with this.
Oh 20 so that would be our goal is to beat this one on the next one.
And if we go back to the leaderboard see where we appeared we'll probably find public leaderboard score
0.
I'd be very skeptical of any kind of competition where people have a score of zero.
So what you can actually find out is you can actually janky or actually hack the score here so don't
worry too much about submitting to here on a late submission.
Your next goal is to reduce your own score and if you want my example submission CSA I'll just like
this in the resource section you'll probably get the same score as me.
Our next step in this project if we were to keep extending it and train a new model would be to beat
our original score rather than paying attention to the leaderboard straight away is focus on when you
first begin beach your own score.
So now we've seen how to make full blown prediction for the cargo competition but that's not our original
focus.
Well it kind of was.
I lied right back up the top.
What's this notebook called Dog vision so our problem is identifying the breed of a dog given an image
of a dog.
When I'm sitting at the cafe and I take a photo of a dog I want to know what breed of dog it is.
So it's likely you want to be able to make predictions on your own custom images.
So rather than just images that Kaggle has given us in a test and training set.
What if you had some images of your own dog which I happen to have or a dog that you saw at the cafe
that looked really cute and pretty and you wanted to know what breed it was you took a photo and you
came back to a machine learning model in this notebook and you like I want to see what breed of dog
that was using my model.
Well let's have a look at how to do that.
The next video making predictions on custom images.