Hello everyone and welcome to the principal component analysis with Python lecture.
This lecture is going to be a bit of a continuation on the theory lecture we'll start putting in Python
and sikat learn commands into play.
We'll go ahead and jump to the Jupiter note book in order to see how to use Python and sikat learned
to actually perform a principle component analysis and try to give a visualization to explain what each
component actually stands for.
Let's go to jump to Jupiter notebook to get started.
All right.
I am at the lecture notebook.
This is under the machine learning section principal component analysis folder.
There's a notebook called principal component analysis.
And since this isn't exactly a full machine learning algorithm but instead an unsupervised learning
algorithm used for component reduction we're going to just have a lecture on this topic not a full machine
learning portfolio project although we will walk through a cancer cell with PCa is a real data set we're
going to be working with.
Again make sure to watch the previous video lecture 3 presentation for a full overview of PCa and if
you want the mathematical background behind PCA you can go ahead and do the reading from an introduction
to mystical learning.
But remember that overall PCA is just a transformation of your data and it attempts to find out what
features explain the most variance in your data.
So for example if you have an original data of two components.
Remember we try to get rid of the components that don't explain that much the variance in your data
.
So you can transform your data and then transform it back with either the second component dropped and
then only have the first component here.
So basically we just drop the second component and we eventually transform this.
So you just have this information along that features axes.
All right.
We'll go ahead and continue on.
I'm going to start a new book using my environment to get started.
OK.
The first thing to do is import some libraries here to say import map plot lived high plots as see import
pandas as PD import non-players and we import seaborne as SNS you fix the order.
And then finally map plot lib in line.
Let's go ahead and walk through the data we're going to be working with.
We're going to be working for a cancer data set because it has a ton of features and is great use case
for PCa.
We can actually just get this data set directly from sikat learn.
So you're going to go in and say from SK learn dot data sets imports load underscore breast cancer and
you can actually just tab out a complete that and it should work.
And then what you do is you set a variable and you actually just call load breast cancer and basically
sikat learn has this data sets and you can say import load.
And if I just go in and click tab here you'll notice that there's a bunch of options.
Load Boston load diabetes digits etc..
These are just built in data sets that sikat learn has in order to show some examples of machine learning
algorithms in the documentation of sikat learn and what you do in order to use these data sets.
Is you actually just call the load function set it as a variable and then that variable you'll notice
is if we just go in and check out the type of this thing.
It's kind of this object type called S-K learned that data sets that base up bunch.
Not a typical object that we're used to working with but it essentially just acts like a dictionary
.
So you can call cancer keys off of this to get the keys of the dictionary and then you'll see that there
is a description data feature names target and target name keys and then you can go ahead and prints
cancer and call the description key the D E A C R D E S C R.
If you don't call print what will happen just to show you because it's kind of a common mistake with
beginners.
You'll get some weird string that looks like this much a line breaks in it.
So make sure to actually call print cancer description in order to see this format it correctly.
So we have 569 instances.
So the date is roughly small but it's a pretty good size.
However the number of attributes in comparison to a number of bits is actually quite large we have 30
attributes.
They're all numeric and there's a lot of information for them.
So you can check out the summary system here if you want all this information.
But basically this is just information on some breast cancer tumors.
And what we're going to try to predict is whether they are malignant or benign.
Now that would be if you're actually doing a classification on this dataset what we're going to be trying
to do with principle component analysis is find out what components are the most important ones that
explain the most variance of this data set.
We're not actually going to do a whole machine learning process on this as far as predicting what class
the cancer tumor belongs in let me go ahead and continue to set this up using Pandurs so I'm going to
set this up to what we usually see for data by saying DSF is equal to PD data frame and then going to
grab the data key.
So I have the right data.
Those are all just numerical values.
And then bigamous the columns equal to answer and then the feature names and then if we checked the
head of that data frame we get something that looks like this is exactly what we want.
Notice we have a ton of.
And in fact there's actually a dot dot dot to signify panderers they're running out of room to show
them all but we have a ton of features we don't have the actual target.
If you look at the target.
So if I say cancer target Let me make sure Whoops that's spelled wrong answer target.
We have ones and zeros and if you check out the target names it's malignant or benign.
So just keep that in mind.
We're not actually going to be working any machine learning algorithms off of this for prediction of
classification.
We're going to stick with principle component analysis.
The reason being if you were to use this dataset with another classification algorithm you would probably
want to do PCA first to actually get an idea of what's important in trying to see if a tumor is malignant
or benign.
OK.
As we've noticed before it's pretty difficult to visualize high emotional data in this case we have
30 dimensions.
And what we can do is use PCA to find the first two principal components and visualize the data in this
new to them missional space.
Right now we're dealing with 30 dimensions which we can't make a third dimensional plot here.
The max you can really do is 3D.
Maybe for you of some color but what we can do is visualize this using the two principal components
into the space.
Let me go ahead and now show you the real point of this lecture which is PCa with sikat learn.
So you're going to say from S-K learn pre-processing in poor standard scaler.
Now we're going to go ahead and do is scale our data so that each feature has a single unit variance
before we actually use PCA on this.
So we've seen standard scaler before we just call it as if it was any normal estimator and sikat learn
and then we go ahead and fit it to our data frame.
In this case data frame is just the features themselves and then we go ahead and transform it.
So we say scaled data is scaler transform día.
Okay.
And PCA sikat learn uses a very similar process to the other pre-processing functions that come with
sikat learn such a standard scaler we're going to do is instantiate a PC object.
Find the principal components using the fit method and then apply the rotation in dimensionality reduction
like calling transform and we can also specify how many components we want to keep.
When creating the PCA object.
OK.
So it's time to perform the actual PCA.
We've already set everything to have standard scale.
We're going to go ahead and do a save from S-K learn and the family PCA is under is decomposition which
kind of makes sense because you're essentially decomposing this into its principal components.
And there we're going to say import PCa and then make a variable called PCA is equal to.
And you actually call PCa and you can do shift tab to explore the principle component analysis here
but what are you going to do is specify the number of components you want to keep and what we're going
to go ahead and do is say number of components we want to keep.
It's just two and I'm going to try to visualize this entire three dimensional data set just by the two
principal components and I'm going to say and component's equals two that is all I want to keep two
components.
And then once we've done that we go ahead and say PCa that fit to my scale the data
and then what we go ahead and do is transform the state to its first principle components.
So I can say X underscore PCA is equal to PCA.
That transform the scaled data and then I can check out my scaled data shape
notice my skill data is five hundred sixty nine by 30 and if I check out my X PCA shape what I just
made after the transformation.
Now it's five six nine by two.
So I've reduced it to the first two principal components and now that we've reduced 30 dimensions to
just two we can actually just plot out these I mentioned using that plot live and we can say something
like this fealty figure six sizes equal to let's say 8 by 6 so it's a little larger and then I'm going
to make a scatterplot of these two shape or at least two columns in our SPCA.
So I say X PCA these are numb piracies.
I want to grab all the rows from column 0 and plot those against all the rows from column 1
and let's go ahead.
In essence t x label is the first whoops first principal components
and peeled t y label.
That's the second principle components.
Now if I just plot this mixture it runs.
I now see the data plotted by the first principal component and the second principle component in as
mentioned in the lecture.
Interpreting these components is actually not that easy to understand but what I want to show you is
just how powerful this technique is I'm going to go ahead and add one more argument into it to peel
teeth that scatter.
And I'm going to say C remember C is color and we can actually pass in a column for color.
I'm going to pasand the target column for color meaning I want to color these points by malignant and
benign so I can say sequel's cancer target.
I run this again and now it's in black and white.
So to make this a little easier to see on your screen we're just going to pasan another color mapping
scheme I can go ahead and say plasma and here we have a plot.
And basically this really shows the power of principal component analysis based off of just the first
principal component and the second principle component we can see we have a very clear separation of
what the malignant tumors look like versus the benign tumors.
And this is just based off two principle components versus the 30 dimensions of data.
I mean you can actually almost use this as a compression algorithm of sorts.
You keep a lot of the information and work explaining a lot of the variants still.
But now we're only doing it with two columns of data versus 30 columns of data.
Now let's go ahead and discuss what we actually mean when we say first principal component and second
principle component.
Unfortunately like I mentioned with this great power of dimensionality reduction it comes at the cost
of being able to easily understand what each of these components represent.
They don't relate one to one to a specific feature in your data.
The components really correspond to combinations of the original features the components themselves
are stored as an attribute of the PCA object.
Let me go on and show you how you can see that you can say PCA thought components underscore and you'll
see an array of the actual components.
And in this numpad matrix array each row represents a principal component and each column actually relates
back to the original features.
And we can visualize this relationship with a heat map.
So I'm going to try to show you visually what these principal components represent.
I'm going to say DFA underscore comp is equal to PD data frame PCA components underscore and set the
columns equal to cancer feature unders or names.
So here I've created a data frame with it go ahead and take a look at it.
It's a principle component zero in principle component 1.
So the first principal component and the second principle component in their relationship for each of
the 30 features.
Now we're going to go ahead and do is put this into a heat map so say s n s Heat Map past that N we're
going to see mapped to plasma so we can go ahead and take a look at it.
And then finally I'm going to actually make this figure a little bigger before we show it.
When I say big size is equal to 12 6 and now we have a heat map that shows the relationship between
the correlation of the various features and the principal component themselves.
Basically each principal component is shown here as a row and the higher the number are the hotter looking
the colors or it's the yellow it's more correlated to a specific feature in the columns.
And that's really the best explanation for what these principle components represent their combinations
of all these features.
And you can see what features were specifically important to one of the principal components versus
the other.
All right I hope this information is useful to you when you're dealing with high dimensional data now
that you have these principal components where you can go ahead and do is feed in your reduced version
of these principal components so you can fit in here X underscore PCA into a classification algorithm
.
So you could do something like a logistic regression on X underscore PCA instead of doing a logistic
regression on the entire data frame of features.
And you can see here that this is easily separable by just almost a straight line and support vector
machines that we'll learn about later may actually be a good choice for this.
OK.
Feel free to post the Q&A forums.
I really recommend that you do a little EXOR reading for principal component analysis such as the book
Introduction to Cisco learning topics on principle comput analysis.
Or even just the Wikipedia page for principal Capone analysis.
It's a really powerful tool and it's easy just to kind of get lost in it by just memorizing a few lines
of code.
Sikat learn makes it really easy to just transform data sets into its principal components.
You should really try your best to understand what's going on behind the scenes.
And I think the best way to do that is through kind of creating this heat map on your own.
So what I would recommend is play around with this data see if you can see what the top three principal
components are.
Maybe make a 3D analysis of that and just really explore and play with this data so you can understand
principal component analysis fully.
OK.
Hope you enjoy this lecture.
Thanks everyone and I'll see you at the next one.