Hello everyone and welcome to the operations lecture this lecture we're just going to review some of
the most important operations for pandas as well as learn some useful operations that we haven't gone
over yet.
Let's go ahead and jump to the Jupiter notebook to get started.
All right.
Here I am at the Jupiter notebook.
I went ahead and just copied the first cell of the operation is the book that goes with this lecture
in order to create this data frame.
It just has three columns.
Call 1 call to call 3 and it has an index of just numbers 0 1 2 3 and call 1 we have 4 digits and called
2.
We have four numbers here.
But notice that 4 4 4 is repeated and then call 3 we just have four sets of strings.
Let's talk about finding unique values in a data frame.
There's three main useful methods concerned with finding unique values in a data frame.
Imagine you wanted to find all the unique values in column two.
The method we can use for that is just the unique method and this will return a higher rate of all the
unique values in column two instead of actually wanting the array of unique values.
Let's say you just actually want the number of values itself.
Well there's two methods for this.
You could just check the length of the array that gets returned and the sort of response to 3 or 3 unique
values in column 2.
But Halo's actually has a built in method for this and it's just and unique.
And if I run the cell I get the same answer.
All right.
So we have unique.
For actual returning an array of unique values and unique.
If I want the number of unique values the less useful function I want to show you is value counts.
And this basically means if you want a table of the unique values and how many times they show up you
can just use value underscore counts as a method.
Run this and this returns.
How many times each unique value occurred in that column.
In this case 4 4 4 occur twice and the other numbers occurred one time each.
All right those are three very useful methods and you'll find yourself using them often in the exercises
.
Let's go ahead and talk about selecting data.
We've already gone over conditional selection but I want to reemphasize just how important it is and
how powerful it is as a tool.
We have our data frame DMF and we can use conditional selection by passing in a conditional selection
statement such as saying return the T.F. where column 1 is greater than 2.
And this will return all the rows of the data frame where column 1 happens to be greater than 2.
Remember that this command inside is essentially just a boolean series of False False True true.
Now as a quick reminder if you want to combine conditions you can wrap these conditions in parentheses
and then use either an ampersand or a pipe operator.
Penny if you want and or or.
And then you can say in other conditions such as where called to the you send has a string has a quality
with 4 4 4 and the return just that last row 3 and there it is.
OK.
So just a quick reminder of how to select data using conditional selection.
Now let's talk about the applied method the applied method is going to be one of the most powerful tools
in your tool belt when using panderers.
So let's go out and show you how it works.
Magin we have a function DPF times 2 that takes in a value X and then returns let's say X times 2.
All right a very simple function.
Now you know that you can already grab for instance a column and call a built in function off of it
such as the sum and that return the sum of the column.
But what if you want to apply your own custom function such as times too well.
Has the ability to do that.
You can just say the column name and then apply and then you just pass in whatever function you want
to apply this case we're going to go ahead and apply times to and this will broadcast that function
to each element in that column 2 4 6 8.
You can also apply built in functions such as length.
For instance if we take a look at column three we had a bunch of strings.
Let's say we wanted a column there were present at the length of each string.
Well then we could just say apply in person Elyon and that returns the length of the stream which in
this case was 3 for all those strings apply.
It's going to be especially powerful when you combine it with land expressions.
That way you don't need to take the whole time to define an entire function.
If you're just going to apply it once for instance let's go ahead and make a lamp expression out of
times 2.
I'm going to go out and say Lamda X colon and then just x times 2 I'm going to apply this to call him
to.
Let's go ahead and see how this works.
And there you have it.
We were able to multiply each value in that column by two and we were able to do this without having
to write an entire function for it.
We just did this as a lamb the expression and this is the sort of thing you're going to be using all
the time as you get more and more comfortable with pandas.
This probably one of the most powerful features of pantless the ability to apply your own custom lambda
expressions or functions.
All right let's talk about removing Combs again.
If we have if we want to let's say remove.
Column one.
We can just say ZF drop passen the column we want to draw c o l 1.
And then remember since it's on the columns we have to specify access is equal to 1.
And this will drop that column if we want this to occur in place.
We have to specify in place equals true.
Let's go ahead and not run that.
So we retain our original data frame if you want to get the column and index names off a data frame
.
Those are actually just attributes of the data frame itself.
Meaning you can say DF columns will return this kind of index object essentially just a list of column
names.
This is going to be especially useful when you're trying to index a column and maybe the spacing is
weird or you can't quite figure out what the string is.
You can just call DFA columns or whatever your data frame name is.
And Ill return this list of column names and you can actually do the same thing for an index.
You can say DFA index it'll give you information of the index.
This case since it's a range index it actually just reports back the start stop and step size of the
index.
All right let's talk about sorting and ordering a data frame.
Here we have deep breath and imagine you want to sort it by column 2.
Essentially we want 4 4 4 and then 4 for 4 again before we see 5 5 5 and 6 6.
We can use sort values in order to do this.
You called DMF saur underscore values.
And then as an argument you say by now we've checked shift tab here we can say bye and you could order
this off of either the rows or the columns.
But essentially you just Pessin as an argument the column you want to sort by.
So I want to sort this by column 2 and you can say just column to as such shift enter.
And now I have sort of state of frame by column to know how the index stays attached to the row.
So you don't ever lose that information there.
And that's the same thing as just saying by column two same thing because it expects by to be the first
argument.
All right.
Let's talk about just a couple of more useful operations.
A really useful method to find no values in your data frame is the DFT is null and this will return
a data frame of booleans indicating whether or not the value was null or not.
In this case we get a boolean of all falses because we have known nulls in our data frame.
All right.
Now let's finally talk about one last function or method in pantless called the pivot table.
And this methodology will be really familiar for Advanced Excel users.
I'm going to go ahead and copy and paste some code from the note book to create a new data frame.
We go out and check out the state of frame we have something that looks like this.
We have a b c and d columns.
And notice here that we have repeating values in both the A column B column and the C column a column
has Foo's and bars B has one twos.
C has X's and wise and then D has some integer values you can use DFT pivot table.
Create a pivot table and if you're not familiar with Pivot Tables I wouldn't worry about this too much
but if you are maybe an Excel user who is familiar with Pivot Tables Hopefully you'll find this functionality
useful.
Basically where we're going to do for not excel users is just create a multi index out of this table
or a data frame.
The method to do this is give it underscore table and pivot underscore table takes in three main arguments
the values the index and the columns.
In this case let's go ahead and indicate that the values we want in our table are these values.
This is what we actually want.
The data frame data points to be made up of.
So I'm going to say values is equal to the D column.
And then I will set my index to be equal to the A column and the B column and that's basically going
to make a and b into a multilevel index.
So going to passenger list then since it's more than one column index and B.
And then finally I want my actual columns to be defined by the C column.
This case just see that I pass that in as a list here and note that I'm saying See it's going to be
the column.
So I should end up with a column for X and a column for y.
Let's go ahead and see how this works.
I'm going to run this one more time here so we can compare the original data frame versus the pivot
table that I'm about to create.
All right so I ran this pivot table command.
And let's go in and compare the two.
Looks like a created index out of those and B columns so that makes sense lexically a bar and foo and
then one and two repeated then the actual columns I want it where the values in this case just X and
Y and the actual data point values were the D columns.
No I do have some no values here because there were no values that matched up for that specific point
.
Meaning if I looked back at the original data frame there is no value that goes with bar to x.
So if you look here you won't find barrer to X. you'll only find part 2 y.
And similarly for food to y.
There is no food to why there's only food to X and that's why we had to fill in no values here.
OK.
Feel free to post the Q&A if you have more questions on Pivot Tables or want some more resources for
examples on Pivot Tables.
We won't really use these that often in the course.
In fact there's only really one exercise question later on where we're working with heat maps as far
as how to use the pivot tables.
But for now only concern yourself with them if you're already familiar pivot tables from something like
Excel.
OK.
I hope some of these operations were useful to you.
They see just a grab bag of useful functions and methods that are available to you in panas.
You can go ahead and reference the pandas operations lecture notebook that's underneath the panderers
folder.
Thanks everyone and I'll see at the next lecture