Welcome back.
The last video we learned a few little arithmetic tricks and we learned that arithmetic is a fancy word
for mathematical operations that we can do on num pi arrays.
And so now we're going to learn a bit more about manipulating numbers higher res through aggregation
our aggregation is another one of those fancy words.
Let's Google and define aggregation the formation of a number of things into a cluster.
Yeah that's kind of what we're gonna do.
So aggregation performing the same operation on a number of things.
And in our case the number of things is our number.
Hi Ray.
So let's have a look.
You might be familiar with the some method.
So dot some or Python some.
So let's make a list list.
List equals one two three.
Nice and simple.
And we'll go type list a list listed list is not defined of course we're making typos.
There we go.
Type python.
List.
So if we wanted to get the total of the listed list we could type in this some listed list.
Now that's a python function right there.
Some but none Pi has its own version of these.
So let's see.
Let's go back to our A1 array which is a lot more trusty.
Well not a lot more but it's just a different type.
To list list even though it contains the same information it's in a type num pi NDA array.
So let's have a look.
What can we do with a one we can call some on it and we'll get six.
We can also call MP dot sum on a one we get six as well.
Okay.
Now why would there be different functions here.
And what is the main difference between these two.
It can get confusing having two different ways to perform the same aggregation on a set of data and
remember aggregation just performing the same operation on a number of things.
Some is just an aggregation to add up all the elements of a certain a real list.
Now the little tidbit you can use here is of which some you should use whether it be Python some or
n p dot some is use python's methods on Python data types and use now implies methods on num pi arrays
so this is some actually reminded this one in Mark down so that's a bit easier to read we don't want
heading one.
We'll put this in code so this is our tidbit.
NDP dot some.
There we go.
Now let's say this in action.
So we're gonna create a massive num pi array massive array.
This will really demonstrate the power of how fast num pi can be.
Remember how we talked about right at the start.
Create a massive array with 100 thousand different elements.
This won't be too uncommon for machine learning problems.
So there we go.
Size one hundred thousand.
Let's just view just so we can see what it looks like.
Massive array.
Let's view the first 100 elements.
There we go.
All random numbers right.
We might reduce that to 10.
So we have a bit more space.
Wonderful.
Now we're going to use one of Jupiter notebooks or Python's magic functions with remember a magic function
has a little percentage sign at the start of it.
We got time it and this is going to time how long a particular line of code takes to run.
We'll try Python some.
So some across massive array.
We'll just put a little comment here.
So Python some.
And then time it.
We want now implies some massive array and we'll put a little comment here now and some.
So let's run this and see what happens.
It might take a little while because it's adding up 100000 different numbers.
So what have we got here.
Well the first line is going to be dedicated towards this line here.
And the second line is gonna be dedicated towards non pi some.
So the first line to some over using Python some across 100000 different numbers took seventeen point
nine merely seconds per loop plus or minus two point nine for milliseconds ten loops.
But now implies some end paid out some input that paid out some took 34 micro.
This little symbol here is Mew for microseconds.
So let's convert that we want microseconds to milliseconds
equals one microsecond equals zero point zero zero one milliseconds.
So let's convert this is merely seconds we want to convert that to microseconds seven pain point nine
milliseconds actually two microseconds.
There we go.
So seventeen point nine milliseconds equals seventeen thousand nine hundred microseconds.
My goodness.
So seventeen thousand nine hundred divided by thirty four equals five hundred and twenty six.
So that means now implies MP dot sum is five hundred and twenty six times faster than Python some so
going forward.
Remember that tidbit here.
If you're working with NUM pi data use num pi methods but if you're working with Python data types use
python methods because going back to the start num Pi has been optimized to perform numerical calculations.
So whenever you want to perform numerical calculations.
Always remember to use the NUM pi version of an aggregation function.
And speaking of aggregation functions let's have a look at a few more we'll go a two who will have a
look at our second array that we created as our trusty two dimensional array from before.
Let's find the main of this.
So your MP dot mean a two three point six three three beautiful.
Now the main would be if we added these up one plus two plus three point three plus four plus five for
six point five then divided by the total number of items.
So six we won't do that but that's how you find the main if we wanted the maximum you can use MP dot
Max if we wanted the minimum you can do MP dot mean if we wanted the standard deviation we can do STV
for standard deviation remember we can always press shift tab compute the standard deviation along the
specified axis if we pass no access it's gonna do it over the entire array which is what we want.
Wonderful and if we want the variance we can do in p dot var a2 wonderful shift tab compute the variance
along the specified axis and now you might be saying Daniel what the hell is a standard deviation and
what the hell is the variance.
Well variance let's put a little cone here.
What you need to remember is the variance equals the measure of the average degree to which each number
is different to the main so we want higher variance equals wider range of numbers lower variance equals
lower range of numbers beautiful and now the standard deviation the formal definition is standard deviation
equals standard deviation is a measure of how spread out a group of numbers is from the main the standard
deviation is actually just the square root of the variance so let's try this out.
So we've got MP There's another aggregation function square root of MP dot var a A2 should be equal
to the standard deviation so let's write this down standard deviation equals square root of variance
shift and enter there we go these two numbers are the same.
Now if you'd like to look into a bit more of the math behind this standard deviation the variance all
leave for some reasons in the resources section for you otherwise in the next video we'll see a bit
more of an example of these two in practice they're an important concept to know going forward in any
type of data science or machine learning or statistical work so let's take a quick break.
Play around with some aggregation functions that you've seen on some of the arrays we've made or create
your own and I'll see you in a second.