All right so in this lesson the rubber is going to meet the road in this lesson we're going to build
our model out in Charisse.
Now the thing is anytime you're doing anything intensive flow and in this case carris is using tensor
flow as the back end.
It involves using a three step process to build and construct your model.
The first step is defining your model.
So you have to set out the structure of the model.
That's step one step two is compiling the model.
And that means telling tensor flow in advance how you'd want to measure the loss what kind of calculations
you'd want to run to adjust the weights.
And this is still part of the setup.
We need to tell tensor flow how it should optimize our model.
Once it comes to training it tensor flow needs to know about the calculations needs to run ahead of
time speaking of training the model that's actually step three training the model is what you can do
after you've done the setup and the word you'll see to describe this increase is the word fit during
this step tensor flow will actually crunch through the data and do all the calculations.
So let's talk through Step 1 defining the model because we're going to use carrots and I want to talk
you through the code before we're actually going to write it in Jupiter notebook.
Let's look at a simple example for image classification.
Suppose we are aiming to have this structure here that I'm showing here on the slide this very very
simple artificial neural network that we're going to build actually has a name.
It's called the multi layer perception and since that one is defining our model we have to set up the
architecture for this perception.
And that means creating the layers.
The very first layer that we will create in our code is actually not the input layer.
The very first layer that we need to create is the first hidden layer Charisse only needs a little bit
of help from us to work out what the input layer is.
So what does this help that we need to provide carrots.
Well we have to tell carrots how many inputs there are to this first layer.
If we're working with an image then the number of inputs is gonna be determined by the resolution of
that image and the color space of that image.
So if I've got this image right here and it's 32 pixels by 32 pixels and it's a color image then the
total number of inputs is gonna be 32 by 32 by 3 3 because we've got a red green and blue value for
every single pixel so 32 by 32 by three is gonna be equal to three thousand and seventy two.
That's the information we're gonna provide in our code when we create our first hidden layer the function
call from Caris to create this layer is called dense dense will create all the neurons for us in this
layer and the arguments that we have to feed to the dance function is the number of outputs that we
want.
So in this case the number of units or the number of neurons in the layer itself.
And because we are calling this for the very first time and we're creating our very first hidden layer
we have to specify the input dimensions.
And this is where that three thousand and seventy two comes in.
The final thing that we should specify is the activation function.
In other words how are the neurons going to behave.
And again we have a choice here.
And I'm going to talk about this in detail in a minute.
So now that we've set up the first hidden layer what would our code look like for a second hidden layer
this hidden layer has five outputs because there's five nodes so the number five has to be specified
as the units in the dense function.
And again we specified activation function but in this case carries a smart enough to know how many
inputs the second hidden layer is going to have.
It only needed to know the number of inputs explicitly for the very very first hidden layer for that
second layer it knows to look towards the first hidden layer that we've already created to work out
the number of inputs let me ask you this.
What's the code roughly going to look like for that final output layer unsurprisingly.
We're also going to create this layer using the dense function from Caris and for the number of units
we've got for.
Because we've got four neurons on the slide but the only difference here is for the output layer we've
got a different activation function instead of Rela.
We're using soft banks now.
I promise to you to talk a little bit more about activation of functions.
So let's do that now.
This is a really good opportunity to dive into this topic in a little bit more detail one of the things
that we really talked about with neurons is that they may activate and pass that signal down to all
the neurons that are connected downstream.
The very first models of these neurons imagined a sort of binary output for the neurons.
The imagine that the neuron would either activate or not activate and this can be modeled with an activation
function that takes the form of a stepwise function.
In other words if the activation function looks like this then you either get a value between 0 and
1 from the neuron.
Now these were sort of the early models from the 1940s.
Since then we found that well maybe a neuron can activate strongly or weakly or somewhere in the middle.
So we really want to get sort of a range of different signals from a neuron.
And if this activation function were shaped a little bit differently then we can accomplish exactly
that end to a different activation function.
In this case we've got a sigmoid activation function which will give us a signal between 0 and 1.
Now our neuron can send a strong or a weak signal depending on where on the curve it lies.
Looking at this chart if it's getting input value of six then it's going to send a strong signal of
one.
But if it's getting an input value of negative six then it's not going to activate it it's not going
to send anything if the inputs are somewhere in between.
It's going to activate by 25 percent or by 75 percent.
This is the idea behind an activation function that's a curve like the sigmoid function but a few minutes
earlier we've seen two other functions we've seen the real function and a soft backs function as the
activation function for the neurons.
So what are those and what do they look like.
Well the really function stands for rectified linear unit which is quite a mouthful and why everybody
appreciates this.
Reload and the function itself actually has a very simple shape it's a straight line for all the negative
values and it's a 45 degree line for all the positive values.
Now we've seen three different activation of functions in this case so which one should you choose.
Well there's actually many more.
If you go to the cross documentation we are presented with a menu.
So you've got available activations soft backs you see Lou soft plus soft sign reload which we've talked
about 10 H sigmoid hard sigmoid and so on.
So we've got quite a few different activations to choose from for our neurons.
Now at first this seems really intimidating but there's an easy solution.
What you often find is that a particular field a particular type of problem actually favors a particular
activation function.
So all you have to do is look what kind of activation functions are used for the problem that you're
trying to solve.
And stick with that one that's going to be a really good starting point in our case.
We're gonna go off the reload function.
This is going to be our activation function of choice for all the neurons in our hidden layers.
The big exception was the output layer.
This is where we saw that would soft Max.
We had real activation functions in our first hidden layer.
And our second hidden layer but we had the soft Max activation function in our output layer.
Why is that.
Well soft Max is a mathematical function that will transform our outputs into a probability.
And this means that we get a really nice interpretation for our output.
In other words the model can tell us something like there is a 20 percent chance of this image containing
a cat.
And the reason that we can interpret the output of a model like this is thanks to this soft Max activation
function in the output layer soft Max will give us a distribution for all our output numbers and all
these numbers are going to be between 0 and 1 and they're all going to sum to one soft Max is basically
going to reach scale our output.
And this is why you often see soft Max in the output layer for all of these multi class classification
problems.
But enough about the theory.
Let's head back into our Jupiter notebook and configure our multilayer perception.
I'll add a markdown cell here that's going to read define the neural network using carrots.
Next we're going to go to the very top to our input statements and input the caris functionality that
we need and that includes from Caris thought models import sequential also from Kerry's dot layers.
We're gonna import you guessed that dense and we're also going to import activation lets it shift enter
on the cell scroll all the way back down and add our code to create our perception.
So I'm going to call this one model underscore one because we're gonna have more than one model that
we're going to try out next.
I'm going to use sequential parentheses and then square brackets and to add on this line between the
two square brackets.
I'm going to call Dennis open parentheses and I'll say units as equal to.
And here I get to choose how many output units my first hidden layer will have.
I'm going to go with 1 128.
Put a comma after that and then I'm going to specify my input dimensions.
Input underscore DRM and a set that equal to.
We said well three thousand and seventy two right.
But we don't have to use this magic number here.
We've already added a constant here.
So let's use this one instead.
Total underscore inputs.
We'll put that right here and then we can specify something else namely the activation.
And here we're going to go with single quotes reload all lowercase.
That's our very first hidden layer in our neural network.
Now that we've got our first 10 layered on let's add a couple of more layers to this neural network.
So to do that we just have to call that dense function a couple of more times.
So when I had a comma here on this line go down to the next line and add our second hidden layer here.
And this is going to have 64 neurons in it.
And as the activation function we're also going to go with redo rectified linear unit but we don't have
to specify the input dimensions because Caris is smart enough to work this out from the previous layer.
That's our second hidden layer done.
Let's add a third hidden layer.
We're really really going deep.
Inception style.
And on this one we'll have sixteen units or 16 neurons will add the activation function again will specify
reload and add a comma at the end go to the next line.
And here we're going to specify our output layer.
In this case we're going to have 10 different outputs.
Right.
We're gonna have 10 different categories that we're differentiating in the model.
So we'll have 10 neurons or 10 units.
And for the activation we won't have really too but because this is our output layer we're gonna be
using soft Max soft Max is ideal for giving us a probability interpretation in a classification problem.
So this is what we'll use when creating the layers.
I've added the parameter name here for the first two and I've left it out for the second two.
This is how you'll often see it written in the documentation and on Stack Overflow.
Now let's hit shift enter on the cell and that's it.
We've created our Caris model.
If we check the type of model underscore one we see that this object is a sequential model.
Fantastic.
So we figured out quite a lot of stuff.
We've written some very terse code where a lot is happening behind the scenes and we've demystified
a lot of the vocabulary that we're seeing in this code here.
Having laid out the structure of our model with in this case three hidden lives and an output layer
it's time to compile our model.
It's time to specify what calculations are going to take place during the training step and how the
weights are adjust that and how the losses are measured.
All that and more in the next lesson.
I'll see you there.