Hello everyone and welcome to Part 3 of the feminist multi-layer percept trons Lecture Series.
Let's go ahead and jump right back to Jupiter notebook to get started.
All right.
Here I am at the Jupiter notebook.
Last time we left off by basically creating our cost optimization functions which were built in tensor
flow and we also saw how to create the bias and weight dictionaries the x and y placeholders and then
we actually programmed out our entire multi-layer percept Tron which takes in a hidden layer a second
hidden layer in the output layer and a lot of these hidden layers takes that data multiplies that by
the weights and then adds the bias to it and runs that through this rectifier function.
Now before we actually begin training the model there is one more convenience function I want to explain
that's available in our AM just data objects so if we look at this train there's a method here called
next batch.
And what this does is you can pass in a number and I'll return a tuple of a batch of 10 samples.
And essentially if we just go ahead and assign this to t.
So I can explain what's actually going on here.
We can see that the length of T is 2.
However if we take a look at t of 0 you'll notice that it's length 10.
And if we look at t of 1 it's also length 10.
And basically what this is it's just a bunch of or it's 10 samples of the actual training data meaning
that 784 are 28 squared.
So 784 long array.
And then the second item is the actual sample.
Let me go ahead and show you just by grabbing one of these and then I going actually use tuple and packing
to show X Samp y Samp is equal to T.
And let's go ahead and see what X stamp looks like.
This is just essentially one array of the 784 values.
So I can just do as we did before reshape this 28 by 28 and then say peel t that show what this looks
like and we can see it this is the number one and we can actually set the c map to be grayscale if you
want it to by saying Grae's.
OK.
And then if you notice what the Y sample looks like it's the actual label for this.
So here.
Note that this is a 1 so at index 1 position so 0 1 it has 1 here meaning it's basically encoded.
So if a label is true here we put in a 1 and that's basically what next batch does.
It returns the sort of output of these samples and their label depending on how many numbers you put
in here.
And we're going to put in if you remember back to the very top when we created this we have our batch
size.
We're going to grab 100 of these data samples at a time.
So now that we understand how this next batch method works let's actually run our session now create
a little break here so run the session.
So we're going to go ahead and launch what's known as an interactive session so we don't have to open
it like we did last time previously we did something like this we said with T.F. thought session as
cesse and then we did a bunch of operations here instead of doing that so we don't have to essentially
open that session or we can do is say SPSS is equal to T.F. dot interactive session close parentheses
.
And now we've launched the session and we can do whatever we want in the remaining cells instead of
having to have everything underneath that with line.
So I'm going to go ahead and do a couple of things here.
I'm first going to initialize all the variables.
So if we scroll back up here remember that we created a bunch of variables intenser for this variable
object.
Now tensor flow is aware of them and we can actually initialize them all.
And I'm going to say in it is equal to T.F. dot initialize all variables close parentheses.
So it is that function call.
And then in that session I'm going to run that something I say as the estopped run in it and now I'm
going to going to go ahead and actually train the model.
So I'm going to say for each park in range and then passen training a pox room or so that's 15.
So that's going to be 15 loops I'm going to go ahead and start with my cost.
So my route my cost is what I'm trying to minimize and I want to say average underscore cost is equal
to zero point zero.
And make sure it's a floating point numbers that go along.
Next thing you want to do is to convert the total number of batches to an integer songless say total
underscore Bache is equal to the number of samples divided by the batch size and I'm going to cast that
to be an integer.
So I can actually grab the number of batches we're doing so I have the number of samples divided by
the number of batches cast an integer is the actual batch size.
And then I can say for I in range
total underscore batch.
So I'm going to have my samples divided by my batch size and that's how many total batches I have to
run.
So for every epoch I'm going to go and in every batch of data I will grab the next batch of training
data and data labels using a convenience function we just went over to say batch X that y is equal to
this train dot next batch and then just in that size which remember is 100.
So essentially I'm going to do 550 batches because we had 55000 samples and I'm going to divide it by
100.
So this is going to run for I in range to 550 and now we're going to go ahead and use the feed dictionary
for the optimization and lost value and I need to do something that's a little common in Python that
use an underscore when you don't need a value in a tuple and packing situation.
I will go in and say cesse thought run and I'm going to pass in my optimizer and my cost function.
So optimizer cost.
And then I'm going to pass in a feed dictionary so we can essentially just think of these as the operations
we did in the basics lecture.
So when I say feed is equal to x as bad checks and why is that underscore why.
And then we're going to compute the average loss.
So the reason I put an underscore here is because we're never actually going to use this.
This returns a loss and this is what I want this see value.
So I want the average cost and I'm going to basically go along and see divided by the total batch.
And then after that I'm going to go ahead imprints what comments are going to say eat and in and these
print for money to do this and and display the cost up to four decimal points.
Floating point and say that format e Pawk plus one average underscore cost.
And then once that whole thing is done outside of the for loop and prints model has completed
pocks of training and then it's going to pass and format wise the number of training epochs.
Okay let's go ahead and zoom out a little bit so we can see this whole thing.
And let's go ahead and review each line of code and make sure we have everything correct before we actually
run this.
I have for IPAC interchange of training ebox.
So basically I'm going to run for 15 POCs set my average cost to begin with as 0 and the total number
of batches are going to run through is end underscore and this should be actually samples and then say
divided by Bache underscores size then for I in range total batch.
I'm going to grab the next batch of training data and labels using that trained our next batch here
with that batch size and then you start run passen the optimizer and the cost and then give them the
feed dictionary that checks that.
Why get my average cost.
And then I'm going to run this essentially 15 times.
Let's go ahead and run the see how it works.
All right.
So the pending on your computer speed this may take a while for you.
They should keep in mind that real algorithms will not just do something like 15 pox.
They'll do more on the range of the thousands.
That really depends on what your data looks like and how large your data set is and how complicated
it is to train.
I'm going to go ahead and just jump to this training session finishing 15 loops.
It shouldn't take super long.
But keep in mind we are using CPQ sort of GPS.
You all go ahead and jump to the completion now.
OK here we're done that took around 30 seconds on my computer.
So just keep that in mind the floor of the computer.
The longer this will take.
But now it's time to actually evaluate this model and test it comes of some built in functions to help
evaluate our model including things such as T.F. equal to forecast and T.F. that reduce underscore mean
.
So I'm going to go ahead and say it's time to evaluate the model say model evaluations so let's go ahead
and start off with TFT equal which is essentially just a way of checking how many predictions are equal
to the test.
So on the same T.F. are good max of pred one and then say T.F. our max of wine to 1 and I'm going to
say TFK equal passen those two arguments and call these my correct predictions.
OK so let me break down what's happening here.
Run this mixture syntaxes OK.
Looks like it's good.
All right so if equal is essentially just an equal check so just does this how many returns a bull an
array of x is equal to y element wise.
So if we look at the two things we just passed in we passed in the pred oche remember after running
this we go ahead and look at what pred was scrolling back up.
That was our actual multi-layer percept Tron which was returning the outlier.
The actual predictions.
So here we have our predictions and we're saying OK fine.
RMX where it's equal to 1.
Remember what the actual output looked like.
We looked at why SAP the output looks like this so we're saying OK we'll just find we the index location
where it's equal to 1.
And that's what your prediction was.
And then we're going to say OK well give me the max of Y.
And those are the actual true values and say we're that is equal to 1.
And then tell me as a boolean array how many truths and falses we have.
And we'll call these correct predictions and then we can go ahead and do is just show you what this
looks like second say.
Prince correct predictions are just like the first one.
It's the type Boolean.
Is it tense or.
So in order to get the numerical value of four predictions we're going to need to use TFT cast to cast
the tensor booleans back into a tensor of floating point values in order to take the average of it.
So I want to take the average by converting all those trues into ones.
So and I'll say correct predictions is equal to cast and I going to cast this tensor object this correct
predictions which has booleans into a float or a tensor of float.
I should say so then if I say print's correct predictions the first instance there.
Now the type is float 32 who want.
Now we'll go ahead and use T.F. that reduce mean function in order to grab the mean of the elements
across the tents or.
And you can basically just think of this as Pied-Bot mean except since we're dealing with tensor objects
.
When you use tensor Flo's own methods here and it's called The reduce mean you can just do shift that
appear just computes the mean of the elements across the dimensions are at tenths or so we'll say pasand
correct predictions and we'll call this accuracy.
So we'll go in and say accuracy's to T.F. that reduce underscore mean correct predictions and it would
still took up the type of the accuracy we have a tensor output but it's working for us.
Now this may seem a little strange but again this accuracy is still a tensor object.
And remember that we still in the past in our actual test data.
So now we can do is call the feminist test labels and images and evaluate our accuracy.
So I'm going to go ahead and say M.A. test labels show what that looks like.
Here are the labels themselves.
We can take a look at just one of them and see an example label for the test data and then we can go
ahead and grab the test images and for an example of what a singular test image looks like it's just
this array of 784 numbers or elements.
Now the evil method will allow you to directly evaluate this tensor in a session without having to actually
call T.F. but SEST not run or something.
So let me show you what I mean by that.
We grab this accuracy object we just did.
And we're going to go ahead and call evil on it.
And this going this is going to evaluate this tensor and I'm going to evaluate it by saying X is going
to be the test images lips.
Let me fix that
and then y is going to be the actual labels.
And just test the labels.
And if we go ahead and run this we can see what our accuracy is and we're getting about a ninety four
point four percent accuracy which actually is pretty good.
But given the state of the art capabilities of deep learning models it's not anywhere near as good as
it actually could be.
Training this for many more POCs maybe training for around almost 10 or 10000 or 20000 cycles can produce
an accuracy up to 99 percent.
But we don't do that here because that can take a really long time to run.
About half an hour so we're fine are just about a minute of running it.
Let me go ahead and break down everything we just did here in this final lecture.
And keep in mind that this is kind of the harder way of running of tensor flowback creating a session
but it allows you more flexibility.
So ugly actually start all the way from the top and break down everything we did.
We started by importing intenser flow and then we grabbed the input data.
The sensor floor has available to us.
We checked that the data understood what it looks like.
Is just an array of numbers.
We set up some parameters.
The learning rate training a pox the batch size and then we had some neural network parameters such
as the number of classes we're going to deal with the number of samples the input and how many nodes
we're going to put in in the hidden layers recreated a function that actually takes in these objects
along with a placeholder for the input data because the input data is going to be put in in batches
and we taken also a weights and biases dictionary after we've just taken the data multiply that by the
way add the bias to it and if we look at weights and biases here this is in the form of T.F. variable
and it's essentially just a random normal distribution in the shape that we needed to be in and for
the bias's the same thing except here is one dimensional because we're just adding.
You're not doing a matrix multiplication.
And then we went ahead and set up placeholders for X and Y and we went ahead and said pred is equal
to this multi-layer percept Tron.
We went ahead and ran it and then we kind of used C for tensor Flo's own built in Cosson optimization
functions.
Then we went ahead and trained the model by learning about this next batch which returns essentially
the x and y samples we ran an interactive session initialized all the variables we ran that initialization
and then for each epoch in the training range of the box we start off of an average cost of zero.
And then we had this total batches which is how many batches were actually going to run.
The number of samples divided by the batch size we said this thuck trained that next batch for each
batch here.
He went ahead and calculated using just optimizer.
And this costs of these tensor operations give it the feed dictionary of the batches themselves grab
the Casa's the output and say the average cost then we just added the current average cost to see divided
by that total batch and then we went ahead and printed out our current costs and you can see how it
decreases which is exactly what we want to do.
And until we get 15 epoxied run that and then went ahead and evaluated.
Which is what we just did.
We said correct predictions.
Compare the predictions themselves and we had a cash flow in order to do the reduced mean essentially
just tensor flow's version of the mean and we passed in our test data and saw our accuracy using the
event method.
Okay that's it for this lecture series.
Now this should have felt probably pretty heavy.
If you don't have much experience of deep learning or sensor flow syntax because it can be quite tricky
to see this for the first time.
Up next we're going to go ahead and learn is country to learn which is basically going to give you a
psychic learn type interface and syntax to actually access deep neural networks with tensor flow.
And that syntax and formatting is going to be much easier to handle especially since you've worked up
the sikat learn library so much.
Let's go ahead and see how to quickly create deep neural network models either for classification or
regression.
In the next lecture.
Thanks everyone and I'll see you there.