Now that we've set up our notebook and downloaded our data let's start exploring it a little bit and
talk a little bit more in detail about what kind of data we're actually working with we're going to
be working with some image data.
And this is image data of some handwritten digits all of the data is made of grayscale images of these
handwritten digits.
Lucky for us the hardest part of the work namely cleaning the data and getting this data ready for us
to use has already been done by the National Institute of Standards and Technology the NIST.
That's the nest behind amnesty.
They've collected all this data and formatted it for processing part of the original data is actually
available on their Web site and you can download it if you like.
The only problem is is that the form that they've got chosen is a little bit strange.
It's it's very efficient plant.
It does require some pre processing what I've done instead is giving you the CSP because in the CSB
we've got values for all the images in an array format and this is very similar to what we've encountered
before.
So these images are 20 pixels wide and 28 pixels tall.
And they're also in grayscale meaning there's only a single color channel.
And what this means is that each pixel in this image has a single value associated with it.
That shows how light or how dark this pixel is.
This value is basically an integer between 0 and 255 as such the total number of inputs for our perception
is 28 times 28 times 1 1 being the number of color channels.
And that's equal to seven hundred and eighty four.
How are we getting these seven hundred and eighty four data points in a C as V file.
Well this is how I structure them.
I essentially put them all into a giant array.
Let's take a look at this array in our Jupiter notebook here.
I'll add a markdown cell that it's going to read explore and let's take a look at these x on this quatrain
on the score all an X on the score test and Y underscore test number high raise and a bit more detail
so X on this quatrain on a scroll the shape is going to show us that we've got sixty thousand examples.
And each example has seven hundred and eighty four values associated with it here's how the very first
one in our training data set looks like I'll pull this up with X on the squat train on a score all square
brackets zero and what we see is an array as promised.
It's got values between zero and two hundred and fifty five a zero value means that the pixel is completely
white and a value of 255 means that the pixel is completely black.
Everything else in between is a shade of gray and there are more than 50 shades of gray here as we can
tell.
Let me minimize this output now and quickly take a look at why underscore a train on a scroll and we
see that it has 60000 labels to go along with the feature data for 60000 examples X on a school test.
On the other hand only has ten thousand examples.
So our testing dataset is a tad smaller than our training dataset which also explains the difference
in file size and loading time down.
What does r y on a squat train on a square all actually looked like.
Let's take a look at the very first entry in there.
We've got a five.
What about the first five entries.
So semicolon and then five shows us we've got 5 0 4 1 9.
These here correspond to the categories or the classes four digits.
Now one thing that you'll actually notice is that I've flattened our arrays for us because what I've
actually done is I've given you all the pixels laid out in a single row for each image.
If I had not done that then this shape would read 60000 by 28 by 28 by one right for the color channel.
I've already combine all of these into a single row now that makes it a little bit easier to work with
but it's not necessarily a good thing because we actually lose a little bit of positional information
on each pixel like we don't know what other pixels surround that pixel by flapping them like this.
So that's something to bear in mind for your own projects and for any future projects and tutorials
that you're working through.
Now that we explored our data and we've got a feeling for what it looks like we're gonna do a little
bit more pre processing so that we can feed it into our neural network.
So I hope you're looking forward to that as much as I am.
And that's coming right up in the next lesson.
I'll see you there.