Hello everyone and welcome to the feminist multi-layer percept Tron lecture and this lecture we're going
to be taking a look at the famous M this dataset which is a dataset of handwritten digits.
We'll go ahead and build a multi-layer precept Tron or a deep learning model based off of this data
.
Let's go ahead and jump to the Jupiter notebook to get started.
All right.
Here I am at the Jupiter notebook.
The first thing to do is go ahead an import sensor flow as T.F. and then we're going to go ahead and
grab the data.
The data we're going to be working with are just again like I mentioned handwritten images of numbers
.
And basically it's going to be in the form of an array.
Now the status that's actually built into tensor flow because it's such a common data center.
You can think of this as very similar to the iris data sets relationship of machine learning.
It's a very common data set as an example and you can grab it from pensar flow by saying from tensor
flow examples Daut tutorials and NYST imports input underscore data and then we'll go ahead and make
a variable called and just equal to input underscore data read data sets and we'll pass in a string
here to make a temporary file for this data and then we'll say one hot equals true and we'll see what
that means later on.
But here we go we've extracted that data we're ready to go.
If this is your first time running this line of code it may take a little while longer so just keep
that in mind.
Ok this data as I mentioned is essentially just handwritten digits and these digits are going to be
the numbers 0 through 9 to just single digits.
And we're going to try to correctly predict what numbers written down based solely on trying to read
the image data this image data is stored in the form of an array.
Let me go ahead and show you how it's stored.
So if we just check out and just as an object it's kind of this special tensor flow data object meaning
you can grab attributes of this data off of this amnesty object.
And one of them you can grab is this train images and if you take a look at this this is actually all
the arrays and all the images.
So if we just check this out it's an umpire.
You can check out that and to show you the shape of it.
It's essentially all five thousand images and each image is an array of seven hundred and eighty four
.
So I'm going to grab one of those arrays to show you what I mean by that.
Let's go ahead and just grab a random array there and here is a array.
And if I took out the shape of the array it's 784 and it's just a single vector.
Now if you want to actually see the image in order to visualize what our data looks like we can reshape
this to the original size.
In this case it was 28 by 28.
So if we just take a look at 28 times 28 that is equal to seven hundred eighty four.
So if I take a look at just reshaping that I can see now I have an array and it's mostly zeros but you
can see as you scroll down that there are some values here and they basically can go from zero to 1
and that represents the darkness of the pixels in that array.
Now that we have the array in this form we can actually visualize it using that plot lib.
Let me go ahead and call this sample.
So this is just a single sample image and run that and I'm going to go ahead and say import map plot
lib PI plot as PLCC.
And since I'm using Jupiter up book I'll say that plotted in line so I can see the image in line of
this notebook map.
Lim has a function called him show which can actually just show the two missional matrix based off some
color scale.
Now I'm going to go ahead and say See map is equal to Grays to actually see a more realistic representation
of what this data actually looked like.
The default color mapping is a jet sort of color mapping but this is actually what these pixels represent
.
Essentially we have a bunch of zeros which is a blank or a white pixel.
And then the values that can go from 0 to 1 is how dark that pixel was at that particular location.
And here you can see that you have the number four but you can go ahead and just set the sample to any
random image from the fifty five thousand images so we can just say one two three there and see if we
can read an image.
So if I just run this well that's clearly probably a one you'll notice if by playing around some of
these are kind of hard to tell.
I don't know if this one is going to be hard to tell.
Well that's another one.
We'll try one more so if we get lucky.
Kind of a weird one.
Nope just another one.
OK.
Last one OK.
But you can see that there's going to be a little bit of sloppiness this so you can see there's kind
of a bulge here and that ain't.
And we're going to try to make a deep learning or multi-layer preceptress network be able to just take
in the pixel information here and correctly classify it as to whatever number it represents.
All right.
Now that we have an understanding of what the data looks like.
Our next step is to define some parameters.
The first thing we're going to do is define three basic parameters that we're going to use later on
.
We're actually building the multi-layer percept Tron and those are learning rate the number of training
epochs and then the patch batch size.
So I'm going to go ahead and say learning underscore rate and that's going to be how quickly we adjust
the cost function and I'll talk a little bit about the cost function later on.
But we'll go ahead and set this to just zero point zero zero one and there'll be a tradeoff between
setting it to a very small number who is a very large value in all explained that as we actually implement
the learning rate then we have the training POCs parameter and that's going to be how many training
cycles we go through.
We'll just go through a 15 training cycles and then we have batch size as our final basic parameter
here.
And that's going to be the size of the batches of training data will go and it's in that equal to 100
.
Okay.
And then we're going to have some network parameters and these are parameters that will directly define
what our neural network will look like this multilayer percept Tron and some of these are pretty basic
.
For example the number of classes is one parameter we're going to need and we expect 10 possible output
for the number of classes.
Remember it goes from zero to 9.
Then the number of samples we have.
So the number of samples we have is fifty five thousand but you can actually just grabbed this from
this train.
No underscore examples.
And if you run that you can just check out the number of examples we have and it was 55000 as we saw
earlier of the shape of that array.
And then we'll go ahead and say numb underscore input and underscore input is equal to seven eighty
four.
And that's going to be what we expect the input to look like.
Remember we're dealing with 28 by 28 arrays for the images but the actual input is the flattened version
we had to reshape it to the actual input shape is 784 and that's going to go here for an underscore
input.
And then finally we're going to go ahead and choose how many neurons we want in the two hidden layers
are going to be using one common valid to use and you're working with image data is 256 and that's because
of the way a computer store image information.
And it's called 8 bit color storage in case you want to look that up.
But we can just go ahead and say hit and underscore one is equal to 256 and hit an underscore to 256
.
All right.
So now we have our parameters and let's go ahead and quickly review.
We're going to be doing in the next lecture first we're going to receive the input data array and send
it to the first hidden layer.
That's this layer right here of 256 neurons.
Then the data will begin to have a weights attached to it between the layers.
Remember that initially starts as a random value and that's essentially that number that went along
with that vector so you multiply that way by the actual value that's passing along between the layers
.
And we also add a bias along of that as we mentioned in the theory lecture then basically that continues
onto the next hidden layer and so on until that final output layer.
In our case we're just going to use to hit in layers.
But the more you use the longer we'll take the model to run.
However if you use a bunch of hidden layers has the opportunity to possibly be more accurate on the
training data.
That's kind of a tradeoff there between using more hidden layers and treating the model for a longer
time.
Once that transformed data that's been multiplied by these waits has reached the output layer.
We're going to need to evaluate it.
We need to evaluate it once it's reached the end how far off we are and that's where the sort of reinforcement
process comes in.
We're going to use what is known as a loss function and this is also called a cost function and we want
to use this function to evaluate how far off we are from the desired results.
And in this case we're going to basically just check how many classes did we get correct so how many
of these between 0 and 9 digits as far as the classes that we correctly classify all the way at the
end.
This is just the first run through and we apply that last function so we get some sort of loss or cost
.
And we want to do is apply an optimization function to try to minimize the cost or lower that error
.
And this is done by adjusting the weight values accordingly across the entire network.
In our example we're going to be using a very common optimizer known as the Atom optimizer and there's
a link in your notebook in case you want to check out the publication for the Adam optimizer.
But keep in mind relative to mathematical concepts actually pretty recently developed the atom optimizer
publication if you check out the link in the notebook was published in 2015.
So you can see we're dealing with pretty recent advances in programming and machine learning Burgon
going to see how quickly to apply this optimization by changing or earlier learning rate parameters
so we have this learning rate parameter here and we can adjust this learning rate parameter and this
learning rate parameter basically just says how quickly do you want to apply that optimization function
.
And the lower the rate the higher the possibility for accurate training results.
But that comes at the cost of having to wait as far as physical time for the results.
And of course after a certain point there's no benefit to continually lowering this learning rate.
There's going to be diminishing returns.
All right.
So that's basically our game plan for the next lecture.
We're actually going to define and create our multi-layer percept Tron model and run it through a tensor
flow session along with the weights and biases between the layers.
Thanks everyone and I will see you at the next lecture.
We're going to continue right where we left off and actually build that model.