How exciting is this whole we've gone a long way from battling to get our images and stances our labels
into tenses preparing our data.
Choosing a model on tensor flow hub creating model callbacks we're finally at the stage where we're
ready to train our first deep learning model so let's create a heading training a model.
Now we're gonna put on a subset of data because if you remember back when we split our data up we're
only going to start training on 1000 images because why do we want to do that.
Let's write here.
Our first model is only going to train on 1000 images to make sure everything is working.
So that's what we're trying to do.
We're trying to minimize our time between experiments.
So we begin by starting on a subset of the data to make sure all of this code.
I mean it might break.
Right.
So we want to make sure that it's working before we spend a long time training on 10000 images because
training on a thousand images is going to go a lot faster than 10000 images and you could imagine the
same goes if you increase it to 100000 images or in the case of image net fourteen point two million
so there's one more variable that we have to define before we can get into training a model.
And that is number of epochs.
And so the number of epochs is how many passes of the data.
We'd like our model to do and a pass is you can imagine is equivalent to our model trying to find patterns
in each dog image and see which patterns relate to each label that's going to hear Naam epochs
equals one hundred and we're going to do something cool here we're gonna create a little slider like
we have before param and then we're going to create type slider and then the men will be 10 the max
can be hundred and then the step can be 10 looking at how cool is that.
So if we wanted to do 10 epochs we could do 10 but we're gonna stick it to 100 because we've got our
early stopping callback and we're gonna see all of this in action in a second but essentially what a
training pass is going to do if we go back to our Kino one epoch would be giving our model a single
chance to look at all the training data and then validating it from their 100 epochs so we're giving
our model up to 100 chances to go through the training data set and figure out the patterns so it's
going to what a pass is it's going to go through these layers we're going to pass our training images
to our model it's gonna go through these layers and then our model as it learns patterns is going to
make guesses on which label belongs to which image and the worse off the guess is the higher the loss
will be.
So that's where Adam comes in.
Remember that guy down the bottom of the hill at the International Hill descent championships is telling
our model how it can improve its guesses with each epoch and then we're gonna see how our model is doing
on an accuracy level because we've got that spectator judge at the bottom of the hill watching how well
our model is performing.
So again if all this doesn't make sense it's going to make more sense once we actually start to run
a model so let's do one final check the final check is one last time to make sure we're using a GP you
because if we're not using a GP you this training our model on images is going to take a very very long
time you saw right at the beginning a GP you can speed up our code by up to 30 times and sometimes more
let's write that code check to make sure we're still running on a GPO so print
GP you
available and then we're gonna go Yes If TAF got config not least physical devices and then we want
to check GPA else not available so this is just the same code we wrote above but we need a GP to be
out of run this in a considerable amount of time
yes we could also have checked here remember that's a little shortcut trick but I love saying this print
out here now boom we've got a GP running we've got a number of epochs set up let's create a simple function
which trains a model so let's write that here let's create a function which trains a model and the function
is going to create a model using create model and then we want to go set up a tensor board callback
using our tensor board function so create C this is where creating functions has really come into play
for us here.
Otherwise we'd have to be writing code ad hoc but writing functions to begin with takes a little longer
but in the long run saves us time so call the fit function on our model.
Passing it the training data validation data number of epochs to train for which is now a box we just
defined that without sick little slider and the callbacks which are our helper functions.
We'd like to use and then finally we want it to return the model.
We're gonna change this in a markdown that is looking phenomenal.
Okay let's go here.
Build a function to train and return a trained model.
So here we are.
Def train well I can't believe we're finally up to this.
This is amazing.
We're training our first deep learning and no network so trains a given model and returns the trained
version
nice and simple.
Remember even though we're getting excited doesn't mean that we're losing our communicative habits so
we'll create a model.
So we're going to model equals say this is where our create model function comes in.
Oh yeah that's so satisfying to write.
And then we go create new tensor board session every time we train a model because remember with our
tensor board with our tensor board callback we are actually we just call this tensor board.
We don't need that callback.
We set up our function.
So if we come back up here every time it's called it's going to create a new folder in logs with the
current day time.
So this is important because every time we run our train model function so every time we try to train
a new model a new experiment we're creating a logs folder which we can then track our model's performance.
So that's a real helpful thing later on when we're trying to evaluate which experiment did better than
another.
And then we go fit the model to the data passing it the callbacks we created.
Now again there's probably a better way to do this function but we're just trying to make it so that
it works and then we can go back after we've sort of got through the phase of fitting a model for the
first time and refine it.
That's got a model to fit and we're gonna pass it X equals the train data which is a data batch.
If you remember which contains the images and labels so epochs is going to be num epochs a.k.a..
How many times is our model allowed to look at the training data before it stops or how many chances
did our model have to pass over the entire training dataset to find patterns.
And then the validation data is going to be Val data which is a data bunch as well or data back.
Sorry.
And the validation frequency is how often we want to test the patterns that our model has found on our
validation set and we set it to 1 because we want it to test its patterns that it's found in the training
set on the validation data every epoch.
So once an epoch and then our callbacks are tense a board which two e's just created above with our
function and early stopping who.
And then we want to return model put a little
return the fitted model.
How cool is that.
And now we can fit our model to the data by as simple as going model equals train model.
You see the benefits in creating functions so we could run this function after doing a bunch of different
experiments later on and so are you ready now.
A little tidbit here.
When training a model for the first time especially on image data or any other kind of large scale data
the first epoch will usually take the longest compared to the rest.
Now this is because the functions that we've written is getting the data and it's been initialized so
a.k.a. loading the data into the memory of our GP you now using more data will generally take longer
which is why we've started with a thousand images and so after the first epoch right we've set it up
so it's capped at 100.
The first one might take a couple of minutes after the first epoch is subsequent epoch should only take
a couple of seconds so you're ready.
We're gonna do this together.
Fingers crossed all of our functions because this actually depends on a fair few lines of code up here.
Let's say it works 3 2 1 Oh look at that.
Building a model with my bone at V2
I've got my fingers crossed.
I hope you do too.
Oh yes.
It's starting up a posse.
There we go.
A pork one of 100 because a box is set to 100 so it's going to try and for twenty five steps which means
our batch size.
So if we go we have eight hundred images in our training set because remember we started with a thousand.
So if we divide that by thirty two we get twenty five so that's why training is twenty five steps because
there's batch size 32 and validation.
If we go 200 divided by 32 it's gonna rounded up so it's rounded it up to seven.
That's why we have validation so 200 images over batch sizes 32 and check this out.
There we go 88 about five minutes.
And so here's what I'm talking about with the loss.
So now all these things that we created right up here so the loss.
So our goal is remember to minimize the loss because we're at the top of the hill we're afraid of heights.
We want to get to the bottom of the hill you want to minimize the loss function.
The loss is the height of the hill.
We come down here so hopefully this number goes down but we want to maximize accuracy so that's how
we'll know if our model is training or not if it's learning patterns this loss should go down in this
accuracy should go up so it's gonna take a few minutes to get through the first epoch I'm going to wait
until it's gone through the first epoch and then so I'll speed this video up and then I'll see you once
it's past the first epoch because you're gonna see subsequent epochs after this so epoch 2 out of 100
is gonna be pretty quick because we use the GP you.
The only reason the first one takes a while is because it has to load all of those images into the GP
Q memory.
I'll see you pretty instantaneously but for me it'll be about three minutes Alrighty I'm back.
So we've got an ACA here which stands for estimated time of arrival is about 12 seconds left in this
first epoch.
As long as everything goes to plan and so you can see the loss has reduced slightly and the accuracy
has increased.
So this is a good thing.
Now what we should see at the end of this first epoch if we set up correctly we should say this is lost
on the training set and accuracy on the training set.
We should see the loss on the validation data and the accuracy on the validation set.
Pop up at any second now so we're just waiting.
So probably what it's doing now is it's been a little bit longer than 12 seconds but it's loading the
validation data into memory so it can calculate and evaluate how it's performing on that dataset.
I can't believe we're doing this we like training a model together in real time.
This is phenomenal.
I've never done this before.
That is as long as your is working here we go.
Look at that.
So now we have this one I'm talking about.
We have this metric is on the training data and this metric is on the validation.
So see how quickly our accuracy is improving.
Now I told you epochs are going to speed up.
Look at that.
The first one took about five minutes but these are taking five seconds.
We're already at one point zero accuracy that's 100 percent accuracy on the training data.
That is insane.
So what this is telling us actually is it's a good thing our model is over fitting because it's performing
way better on the training data than it is on the validation data but this is great.
Our model is working it's finding patterns it's using what mobile Net V2 has learned an image net and
it's applying it to our dog's dataset dog vision is coming to life.
So stoked.
Now I hope yours is running like this as well because we're training a model together in real time.
This is beautiful so what we should see is once the validation accuracy stops improving for a number
of epochs.
So three because of our early stopping callback it's going to stop training.
So we actually I don't think we'll reach the 100 epochs maybe not even close.
Cassie here we're getting one point zero accuracy with a tiny loss on our training data so here we go
it stopped.
My heart is racing.
That was phenomenal.
So see what I meant.
The first epoch takes a while because it's loading data into memory.
Once it's loaded that initial data and the memory the GPO is just going to go all right time to step
on the gas and we'll turn on the afterburners or gas no brakes maybe.
So we reached 18 out of 100 epochs and we can see that our model on our training data set is performing
at 100 per cent accuracy.
That's pretty crazy 1.0.
You're going to times this number by 100.
The loss is fairly low.
I mean a loss of zero is is perfect but we can tell that our model is over fitting because it's performing
way better on the training data than it is on the validation data.
So if we come back it's like our model has memorized the course materials rather than the problem solving
principles of those course materials and it's struggling to adapt to a data set it hasn't seen before
a.k.a. a practice exam or our validation set.
So right now our model has a poor ability to generalize.
So that was I think that's enough for one video.
That's a lot of excitement what I want you to do a little bit of homework is before we go and check
the tensor board box in the next video because we've used our tensor board callback before.
I want you to check out this is a question is this is a really important one.
After we've tried to model I want you to go question.
It looks like our model is over feeding because it's performing
far better on the training data set than the validation dataset What are some ways to prevent model
over feeding in deep learning neural networks so we'll turn that into markdown.
That's a question I want you to look up before we get into the next video.
What are some ways to prevent model over feeding and deep known networks so check that out even if you're
not sure of them.
I just want you to start getting curious about what's going on.
I want you to stop picking up on the clues as to when your model is over feeding such as performing
far better on the training data set than the validation data set.
And a little note here note over fitting to begin with is a good thing.
It means our model is learning so that I'm phenomenally excited that we have just trained our first
deep learning neural network together using transfer Learning dog vision is coming to life.
So we finish training a model at least a first subset of a model.
This is not on the full data.
This is only on 1000 images.
So our next step is to if we come into our keynote what are we up to so we fit the model to the data
we haven't made a prediction yet.
So what we might do is evaluate the model using our tensor board callback so you can see that and then
we'll figure out how to make a prediction with our trained model sound like a plan so check out this
question and I'll see you in the next video.