Welcome back.
In the last video we looked at some aggregation functions with using num pi such as getting the sum
of all the elements in array.
We saw how much faster num PIs code is to pure Python code when working on num pi data types.
So always use num pi aggregation functions on no higher res.
And then we left to with a brief introduction to standard deviation and variance.
Now the main thing to remember about these two metrics is in a word.
They're just the measures of this spread of data.
But let's not talk about it.
Let's always focus on running code and seeing practical examples so that's what we're gonna do in this
video.
So go demo of and var standard deviation FTD a short for standard deviation var show 4 variance.
So we'll create a high bar array equals empty array 1 100 200 300 4000 5000 wonderful low var array
equals empty array maybe we'll make this one nice and simple 2 4 6 8 10.
There we go.
Now before we even run any code how about look at the numbers in these arrays.
Imagine these are actual data.
What would you infer from just looking at these numbers.
So this one goes starts from 1 goes to 100 200 300 then jumps to 4000 finishes on 5000 and this one
kind of just goes to 4 6 8 10 so first of all you might look at the difference between the two maximums.
That's what I kind of looked at.
So this one goes from one to 5000 and this one goes from two to 10.
So if we looked at this we might say that these numbers to me these are more spread out whereas these
are pretty close together.
And the maximum distance is only eight and they have the same step in between each one.
Whereas this kind of varies it goes up by 100 at a time and then up by a few thousand and then up by
a thousand to finish off.
But let's put it into practice let's run the code let's see what's going on.
So we shift into let's find the violence of both of them.
High bar then we can compress tab for autocomplete MP bar we want to have auto complete.
We need to actually type the start of a variable that would be helpful.
Beautiful.
We'll hit shift and enter.
Huh.
Now this is an absolutely massive number here and this is not too big.
Mm hmm.
Well we're not going to look at the mathematics behind these two.
What's more important is the concept here and the variance.
If we go up here remember variance equals a measure of the average degree to which each number is different
to the mean higher variance equals wider range of numbers.
Yep lower variance equals lower range of numbers.
And that makes sense with these two arrays this array here.
High var array has very high variance whereas lower Ray because the numbers are close together it has
a lower variance.
So let's have a look.
Let's do standard deviation FTD.
We want pi var array and we also want the low VAR array.
Standard deviation before we do this.
What do you think the standard deviation is here.
Is it going to come out similar to what this looks like.
Well let's have a look.
There we go.
And that makes sense.
The standard deviation is higher for the high variance array then for the low variance array and what
the standard deviation a measure of how spread out a group of numbers is from the main.
Okay.
They've both got mean in their definition so let's have a look.
Let's find the main dot high var array e and we want N.P. main low VAR array however is not defined.
This is why we should use to have auto complete.
So what this is saying is a standard deviation this number here is the average distance a number is
away from the main.
So if the main is one thousand six hundred.
That means on average any other number in this array in high var array is two thousand seventy two.
Away from the next number.
The same goes for the low var array any number in the low bar is on average 2.8.
Away from the main so that means 10.
This number is although it's 4 away from 6 because the mean is 6 8 is closer to 6.
So that's what brings the average down.
All right now let's check out this spread.
Remember that's the main crux of standard deviation of variance is just the spread of numbers how they
appear.
Let's plot them so we can do that by importing that plot lib.
We're gonna have a look at map plot lib in a future section so don't worry too much if you're not sure
what's going on here.
Map plot lib dot pi law as peyote as what we want.
Yeah plot dot hist.
So this is a histogram.
Hi.
We just want that tab order complete.
If in doubt tab order complaint see what comes up.
Shift into no module Name at plot.
We got that wrong map plotted.
There we go.
Okay so there we go.
The high var array.
Now let's do the same for the low VAR right thought haste low bar array and we won't plot.
Don't show wonderful.
So now we can see visually if you couldn't see it from these numbers in the number line here.
Now visually we can kind of see the spread of the numbers of high var array is a lot bigger so there's
a lot more whitespace here.
A lot of the numbers appear towards zero and only to the samples appear up here whereas in our low VAR
array the spread is kind of similar across the whole thing because there's gaps of 2 between each data
point and the maximum distance here is only 8.
All right so that's standard deviation and variance in a conceptual level.
Now remember if you do want the mathematics behind this how these two values here the standard deviation
and the variance are calculated.
I'll leave some extra resources but just remember the main takeaway from this is that standard deviation
and variance are measures of spread of data.
Now with that being said we finished up some of the most common aggregation functions you'll see a num
pi.
Let's get into the next section where we can check out some more ways of manipulating arrays.