All righty.
So now we've got a function to pre process our images a.k.a. turn them into tenses.
We can now start to figure out what we need to do next.
So we've got get data ready turn it into tenses pick a model.
Well we're not quite up to this.
So if we come back we're going to read a little bit more into learning images intensive low we'd figure
out that the flow likes to see things in the form of a data batch a.k.a. a tensor containing an image
example or a piece of data for example remember this is data agnostic.
We're just using images as our practical example.
So it likes it in the form of.
Okay.
Give me a piece of data and it's label as a combined tensor.
So I can process them really fast.
And if you're wondering what a batch is.
Well it's also called a mini batch which is a small portion of your data.
Now generally if you're wondering what a good batch size is or how much is a small portion 32 is generally
the default batch size with most deep learning models and I'll show you.
This is how I figure out a lot of things.
So we go John McCain that size and you might be wondering who the hell is John McCain.
What is batch size all these things you know to me.
But I want to show you a little trick when you find or when you hear of someone who is let's say a figure
in the field of machine learning or Deep Learning a trick that I use to figure out what they think of
a certain topic is I google their name and I Google the concept I'm looking for another one is Jeremy
Howard.
So John McCain and Jeremy Howard you could go Jeremy Howard batch size Google that I'm just telling
you these two people off the top of my head John McCain I'll let you research him.
But he is one of the founders of convolution on neural networks which is kind of what we're going to
use to process our images but getting sidetracked here if we Google John McCain batch size and then
we come to to Twitter to it is actually an amazing resource to figure things out well goodness training
with large mini batches is bad for your health.
More importantly it's bad for your test error.
Friends don't let friends use mini batches larger than 32 then we've got a paper here linked on different
batch sizes so I'll let you look into that.
I'll link this tweet in the resources section so you can check out that paper.
But this is not going to get worked towards we're going to transform our data into batches of size 32
so they process really efficiently intensive flow.
So let's make a little heading turning our data into batches and these little tidbits.
The one I just showed you like searching someone's name.
These kind of things I think are important right.
How would you know that if I didn't sort of tell you or you sort of didn't stumble across it.
The only way I found that was by stumbling across hey there's all this great information that people
are sharing.
How do I find that every so often and again my way is not perfect but it's helpful.
I google someone's name and a certain concept I want to find out about more.
This is why it's important to share your work.
If you find something valuable.
OK.
Getting sidetracked once again.
But why would we turn our something into batches.
Well let's say
we turn our data into batches and if you're still confused on batches we're just taking a small portion
32 images and calculating on those 32 images at one time and then the next loop the next training session
we calculate on the next 32 because if we tried to calculate patterns in all 10000 images.
So here let's say you're trying to process 10000 plus images in one go
they all might not fit into memory.
So what I mean by that is that a GP you has although it's really fast computing.
It still has a limited amount of memory.
And so my computer has 16 gigabytes of ran the GPO backing this might have eight gigabytes of RAM.
So if we have 10000 plus images and we're trying to process the patents on that in one go that may be
more than eight gigabytes which is going to slow down our training process it's going to break everything.
So that's why we do 32 at a time.
So that's why we do about 32 images at a time.
You can do two images batch size that's right here.
This is batch size you can manually adjust the batch size if need be.
All right.
So I spent enough time talking let's start writing some code.
What do we need.
So write it here in order to use tensor flow effectively.
We need our data in the form of tensor topples which look like this so image in the form of a tensor
and label so as pairs.
So then we'd pass thirty two of these at a time to our machine learning model and it would figure out
the patterns in this image that relate to this label and then it would end up doing that in a sort of
loop fashion until it gets through all ten thousand plus images proof.
All right let's build some functionality but first we need a way to get topples.
So we need before we can turn them into batches of 32 we need to get a top all that looks like this
so let's write a little function.
So create a simple function to return a topple a couple of tenses that rolls off the tongue nicely.
So we want to go def get image label
cause we've got a function here that's going to return our image.
So all we want to do is just basically get the label associated to whatever path we pass this function
so we get here.
Image path and now really you're going to laugh at me for how simple this function is because all we're
doing is passing it the image path the actual label and the image pre processing function we just write
so process image image path and then we want to go return image label.
Look at that how nice is that so takes an image file path name and the ASO she added liable process
is the image and returns a topple of image label.
The doc string is longer than the function
beautiful.
We didn't even need that line we could have just done it I'll return here.
Now let's just give a a quick example there.
Let's just see exactly what this would look like process image we want file names let's get forty to
our favorite number and what is it.
What's it called.
Boolean labels.
We could just use X and Y couldn't we.
That's probably a bit better ex
y 22 demo of the above
beautiful so that's in a topple out image is in a tensor because we pass it to process image.
But our label is not yet in a tensor but that's a right we could turn it into a tensor by just going
to f constant.
There we go.
So now we've got a couple of an image and label pair in the form of tenses.
Now we're making some good progress here and getting our data ready.
I remember how I said a lot of machine learning is just pre processing your data into a format that
fits into a machine learning model.
That's what we're doing here and now.
Again this process we're sort of taking getting our data into the format of data label in a table of
tenses is agnostic to whatever data you're using.
So if you're not using images you're using something like text.
This would be a very similar process except the way you pre process the text will be different to the
image you still have to get it into tenses.
All right so we've got a little function to get image and label pairs in the form of topples.
Write a function in the next video to turn our image and label pairs into batches and there's a few
little tidbits that we need to pay attention to depending on if our images are in a training batch of
validation batch and a test batch.
But we'll see that in the next video.