Now for our next comparison what we might do is try to combine a couple of independent variables such
as age thatll act which is maximum heart rate and then compare them to our target variable heart disease.
So if we have a look back at our data from so what we might compare is age fell AK and target all in
one.
And now if we go to check our different values for fail act now the reason why I know this is max heart
rate because we come back out in a second.
So we can see there's a lot of different values here.
So the length 91 if you call value counts on any column and the length comes up is something that means
that there's that many different values in that column.
So what that might be telling us is because there's so many different values looking at it on a bar
graph may not be the best type of graph because if we look at this plot there's only four columns here
and that's pretty interpretable.
But when there's 91 that might be a bit harder to look at.
So that's where we might bring in something like a scatter graph.
Let's go back to our data dictionary and have a look at what fell AK is come up here.
How AK maximum heart rate achieved and again we're really just diving in reading our data dictionary
here.
You might just go through different different columns read them which ones stand out to you and try
to compare fewer of them with the target variable and remember we come back to our data frame and these
are often referred to as features or independent variables.
And this is often referred to as a target or dependent variable.
So that's what we're doing here we're comparing independent variables to our target variable.
This might seem a little as if we're jumping all over the place I know I've said this before it's because
we we are we are literally just picking columns here that spark our interest.
This one sparked my interest and we're trying to figure out how they relate to the target.
That's what we're doing.
You may choose a couple of other columns.
I'm just going through the ones that look most interesting to me.
So let's make a little heading age the max heart rate for heart disease and then we'll turn that into
markdown.
Beautiful.
So to compare three such as aged slack and heart disease what we might have to do is create a plot with
two plots on it so let's start off first by creating another figure BLT figure and we'll posit a fig
size it goes 10 6.
Then we might go scatter might do a scatter with the positive examples first.
So just bear with me for a second and we'll talk through what's going on while we're while we're going
through it.
So BLT don't scatter.
We want to see on this scatter the positive examples so we can go DFT on age or days of age in brackets.
They really mean the same thing.
But I like to do it this way and if they have to an age and then in here we're going to do def dot target
equals.
Of course you couldn't do this if your column names had spaces in them.
But in our case none of our column names have spaces.
What this is doing is a subset of the dataset.
We're taking the age column from our data frame where this condition equals true.
So where the target equals 1.
So if we did that in a cell below here the after age day after Target equals one.
So this is going to show us all the age columns where Target is equal to 1.
So that's what we want.
Beautiful.
And we also want DFT foul act which is maximum heart rate def dot target equals one.
Beautiful.
And we're gonna give it a color of salmon.
What would this look like if we called that wonderful.
So along here we've got maximum heart rate and we got age.
So what could you infer from this pretty quickly by just looking at it here.
Well you might infer that there's kind of a downward trend.
So if you were to draw a line through there just a straight line it's all over the place right.
But it's kind of you can kind of see the pattern going down here.
The youngest someone is the higher their heart rate.
That's the kind of trend that you'd be seeing there.
That makes a little bit of sense right now.
These are positive examples so these are patients because we've got target equals one.
These are patients with heart disease.
And so now we want the negative examples on the same plot.
So scanner with negative examples BLT dots scatter the IDF dot age all we're going to do is just copy
what we've got above but accept target equals zero because they're negative example.
This is without heart disease like DFT at Target equals zero wonderful C equals when I go light blue
it's a color scheme we're set up for now beautiful.
So now we've got positive and negative examples on there and again they're kind of all over the place.
But if you were to look at the trend the line kind of goes down from both.
But again you can't really split them.
This is where our machine learning model have to take over from our ability to have a look at this.
So if you were if you were to look at this now I just put a semicolon there to stop that little map
properly about.
If you were to look at this and you try to decipher what's going on here what was the maximum heart
rate of of someone who had or didn't have heart disease the salmon versus blue dots blue dots are without
heart disease salmon are with heart disease.
If you had to decipher this it'd be pretty hard right because these are all so mixed up together maybe
you might infer that there's some sort of pattern.
There's a fair bit cluster here there's a big cluster here but really I can't really tell a pattern.
Maybe if I had some more time to look at it I could find something but this is where machine learning
is going to come into play.
The patterns that you can't necessarily see straight away.
Machine learning is going to dive into the data.
It's going to form some calculations and figure out these patterns that we can't really see.
Again if we had enough time maybe we'd find something.
But when machine learning engineers were data scientists we prefer that the algorithm does the job for
us.
So what we might do is add some helpful info just to make this plot a little bit more complete.
So BLT dot title heart disease in the function of age and max heart rate then penalty x label down the
bottom is the age and they NPL T dot y label.
This is gonna be max heart rate and then we'll add a legend so people can understand which dot is which
and they're not just thinking about well what's going on here with this salmon and like blue dots amazing
color scheme but not sure what's what's happening.
And we'll put the semicolon there instead of there.
Wonderful.
So that's looking at a little bit better but again you let me know if you can see some sort of pattern.
But to me all I can really see is is a bit of a downward trend so as someone gets older their maximum
heart rate decreases which kind of makes sense again when you're exploring a dataset you won't necessarily
always find patterns to begin with.
We're just familiarizing ourselves with what's going on and so because we have just an age when we might
check out is what's the distribution of the age.
So to do so go check the distribution of the age column with a histogram
now the distribution is another word for spread of the data.
So we kind can't see here that there's almost an even spread.
So you may recall what kind of distribution and even spread of data has.
And if you don't that's perfectly fine.
It took me a while to understand this as well to me a long time actually.
And I still have to research.
So this kind of distribution here is otherwise known as a normal distribution and if we were to really
say what can normal distribution is we'd look at this perfect normal distribution.
Let's have a look.
Let's go normal distribution is gonna look like a bell curve to see something like that.
So if we come to ours ours is very close to that shape but it's kind of swaying towards the right.
So we can see that most of our population or most of our samples their age is around this big mid gap
here.
And we don't have that many around the 30 year old age or past.
This might be 80 up or something like oh we don't have many past there.
The majority of our dataset are within the 50 to 60 range.
And so this is what you might want to do for a bunch of different columns is check the distributions
check the spreads if we did have someone out here that was like 150 that would maybe be some type of
data that we'd have to clean up because I'm not sure if anyone's ever lived to 150 or if we had someone
down here that was maybe five or something like that.
We'd also have to think about okay is that something we want to include in our dataset.
But this is going to be different Column 2 Column so we're just checking the age here is the normal
distribution we might do the same for other columns but the way you sort of think about samples out
here is they're referred to as outliers.
And now how will you tell that if there's any outliers.
Well distribution plot this histogram is one of the best ways to do it.
So if you're getting some weird results like some weird results in terms of your machine learning models
later on it may be because there's outliers in the data and that's where you'll have to check different
distributions of each columns.
So that's one way to do it.
Now what we might do is compare another two columns so if we come up here to our data dictionary I saw
this one before the chest pain type and I thought that would be interesting to see so if we got here
for different types of chest pain chest pain related decreased blood supply to the heart chest pain
not related to heart typically esophageal spasms.
So I guess that's like your esophagus which is I think that tube from your mouth to your stomach or
something like that asymptomatic chest pain not showing signs of disease.
Okay so this will be pretty interesting right.
If we compare this to the target column.
So chest pain versus target so does chest pain relate to whether or not someone has heart disease.
So let's come down here guy make a little heading.
Heart disease frequency.
Chest pain type and what we might do actually is just remind ourselves copy our data dictionary for
chest pain so we can remember what each different value is.
So if we come in here and we'll copy this shift and enter will bring it down and we'll paste it here.
Wonderful so this is what we're going to compare we can do that with a fairly quickly with a PD cross
tab def not def not target Okay so if we look at this what would we devise in here.
Mm hmm.
It seems as chest pain goes up so does whether they have heart disease but if we get zero chest pain
there's a lot more that don't have heart disease than do.
But if we get to there's only 18 with zero so don't have heart disease but 69 so nearly over three times
the amount that do have heart disease.
So this is non a genial pain typically or non heart related Well that's that's interesting that it's
non heart related and so these are the type of patterns you'll start to find out new data some make
sense.
That doesn't really make sense to me non heart related pain yet more people have heart disease with
two than not.
And so these are the type of things you might be wanting to discuss with a subject matter expert if
you're looking through a dataset such as us looking through this heart disease dataset and I'm not a
doctor I'm not trained medically I've researched some things about health but I've known nothing no
idea about chest pain.
So if I got this data set off someone from a medical profession I'm trying to use machine learning to
figure out whether someone has heart disease or not.
And I'm coming across these patterns that I don't really understand that's where I probably reach out
and go hey to a medical professional or I do my own research such as looking up what these actually
mean we did that before defining angina going and hey I'm seeing this in the data but can you shed some
light it's kind of confusing to me.
Is this correct is this not that's the kind of insights that we're looking for is both patterns that
make sense to us such as the normal distribution of age such as the declining max heart rate as someone
gets older this one maybe doesn't make as much sense is heart disease more prevalent in women than in
these males.
So this is the type of questions that we're trying to get we're trying to formulate an idea in the data
by not finding answers but finding questions that we can ask.
And so what we might do is make this a little bit more visual as we've done before.
Make the cross more visual paid a cross tab def not he def target we're just writing what we've got
above their dot plot same plot again we might do a bar because there's four different values here versus
two different values there.
So we should be at a two look at it fine on a bar graph.
Think size equals ten and six.
Wonderful.
I'm going to use our favorite colors.
Color equals what are we using light blue Salmon Actually I think the order was the other way round.
Mary did salmon first then light blue.
Doesn't really matter beautiful.
Now we're gonna add some add some communication penalty dot title
heart disease frequency her chest pain type
peyote dot ex label and do chest pain type men on the why is penalty don't y label amount beautiful
BLT dot legend.
We're gonna put on here no disease
disease and then we'll finish it off with.
Again we're going to make sure that the X ticks are vertical so it's a bit easier to read shifting into
what do we got pi plot has nothing y label we've got a little typo beautiful.
There we go.
So that's a bit more visual.
So then we're just turning our cross tab into a graph here.
Now this is something that we could take pretty quickly to someone and go Hey what's going on here.
Chest pain type has way more accounts with the disease than chest pain too that is and it's supposed
to be known and general pain so typically esophageal.
That word is very hard for me to pronounce esophageal spasms non heart related.
So this is again raising questions from the data.
That's what we're trying to do in this exploratory data analysis.
All right.
So we got a pretty visualization there what we might do next is check out the correlation between independent
variables and our dependent variable.
So again if we have a look at our data frame this is what you want to be jumping in out of right just
using of head to quickly have a snapshot of your data frame.
These are our independent variables remind ourselves and we're trying to use them to predict target.
So we'll see how we can compare those and we're gonna use a correlation matrix but we'll see that in
the next video.