Hello everyone.
Welcome to the data frame's Part 2 lecture.
Last time we left off on how to create a basic data frame and grab information from that data frame
.
Now we're going to look at conditional selection as well as some multa index parts of the data frame
.
Let's go ahead and jump to Jupiter notebook to get started.
All right.
Here we are back at the Jupiter notebook.
I'm going to be using the same data frame as last time.
Shurmur we just had to say numpad S&P pins PD import Rand and set the seat to 101 and then say that
data frame was rent and 5 come for.
And then here we have our data frame.
Okay let's go in and talk about conditional selection.
A very important feature of pantless is the ability to perform conditional selection using bracket notation
and this is going to be very similar to PI for instance for a comparison operator against the data frame
such as asking where is the data frame greater than zero.
I will get a data from back of boolean values where it returns true if the data frame value at that
point was greater than zero and false if it was not greater than zero.
And this is very similar to the same conditional selection applied to an entire hiree except in this
case we're doing of pand the state of frames.
And just like this we can go ahead and say bullwhips bulled the f is equal to DFA greater than zero
.
And since say Boulia.
Go ahead and check that out.
And here we have our boolean data frame and if we pass that in to our original data frame with bracket
notation then we will get out as a result values where it was true and Knoll's or any n where it happened
to be false.
And that's the way you can use conditional selection along with a data frame.
Now usually you probably wouldn't be separating this out into two steps.
You would just say.
And then in brackets something like DF greater than zero and you would get this result.
This sort of operation with a data frame and against a comparison operator isn't that common.
Well most likely you'll end up doing is instead of just passing in the entire data frame you'll pass
in a row or a column value and instead of returning Knolles what that's going to do is return only the
rows or columns of a subset of the data frame where your conditions are true.
Now let me go to show an example of what I mean by that couldn't take a look at our original data frame
.
Here we have rows A-3 and columns w through Z.
What I'm going to go ahead and do is specify.
Of W greater than zero.
And notice I get a series back.
True true false true true and this is linked to the actual index.
And if I take a look at the W column I see that this corresponds with the W column values and the value
at index c it was actually less than zero.
So returns False for being greater than zero.
I can now use this series of boolean values corresponding to rows to filter out rows based off of a
column value and that means if I pass in the series into a data frame using bracket notation to say
something like D-F DFW greater than zero I will only get back the rows where this happens to be true
meaning I only get back the rows a b d and e it won't return row C since it was false here.
Go ahead and run that and see what this looks like.
Great.
And now since I'm passing in a series I don't get those no values anymore.
You only get those no values when you're doing this sort of condition on an entire data frame when you're
actually passing in conditions based off of columns.
You'll just get series values meaning you'll get only the rows were happens to be true.
And this is the sort of conditional selection we're going to be using most often throughout this course
.
Let's go in and show just a few more examples to really drive this point home.
Let's say I want to grab all the rows in my data frame where z is less than zero.
So I say DMF Where the f of column Z is less than zero.
Now let's take a look before we actually run this what should happen over Notice column Z.
Where is it.
Less than zero.
Well that's only occurring in row C so column Z as in Zelda and then row C is in cat.
This is the only row that should be returned.
But remember asking for the entire data from back.
So it should return just this row C for us.
And there it is.
OK great.
Now something else to note here is that when you're doing these sort of commands go ahead and go back
to R W greater than zero.
You get a data frame in response and that means you can go ahead and call commands off of this data
frame.
Then you can do this either in one step or two steps.
I'm going to go in and say result DSF is equal to this and then take a look at results.
DMF which is this subset of the data frame without that row see in it and that means I can go ahead
and grab results off of this such as X and here I have four values of X A B D and E based off of this
resulting data frame that didn't have that row seen it hasn't Charlie.
But instead of actually separating this out into two steps such as grabbing the X column I can do this
all in one step.
And let me show you what I mean by that I can go ahead and say return the data frame where the column
value of W happens to be greater than zero.
So return those rows to me.
That means all the rows minus the C row.
And since this is a data frame I can actually go ahead and stack commands such as bracket notation on
top of that such as return the X column where that's true.
And now I get the same results as I did up here except it's all done in one step and this is something
that can be intimidating to beginners at first.
So there's a lot of brackets and a lot of stuff going on here.
But essentially this is just instead of splitting everything up into multiple steps and putting it all
into one step.
And likewise since this is a data frame I can bracket for multiple columns by passing in the lists such
as why comma X and we can see we get a result here.
Let me go ahead and break this line down one more time to show the steps that we're taking.
First off we're going to go in and say Boudia is equal to.
Well it's going to say bhool series and since that's a little more accurate is we're at DFW is greater
than zero.
So if I say bhool series I get this boolean series back then what I'm going to go ahead and do is say
result is equal to DSF of that bool series.
Let's go ahead and check out the results.
And it should just be wherever that series was true which is a b d and e and there is my result.
That's the entire data frame without the row seen it.
And then I can go ahead and offer that result call a list of columns so we can go ahead and say why
comma X and if we wanted to we could actually even separate the separate this out and so one more step
by saying Mike calls for columns passing that list and then say a result of Mike calls and we get the
exact same thing we have here done in one line as in multiple lines.
That means if you're ever confused on something that looks like this of a ton of brackets and Pandurs
see if you can break it down into multiple steps.
If that makes more sense to you throughout the course we're going to be doing this type of notation
.
Kind of one liners.
And as you get more and more comfortable of pandas you'll find yourself using these one liners more
and more often but if you're ever confused always feel free to break it down into multiple steps.
The only disadvantage as far as bringing it down multiple steps is that you are taking up memory which
with the variable that you define.
So as you get better and better of pandas you want to move towards a simple one line abstraction of
all these commands.
All right go ahead and pause the video now and really make sure you understand how line in 58.
Here is the exact same thing as separate out into multiple lines.
All right.
Let's go ahead and explore real quick.
The concept of using multiple conditions such as two or more conditions.
Right now we feel comfortable with doing something like this which is returning a subset of the data
frame based off of one condition.
But what if you wanted multiple conditions such as where w is greater than zero and Y was greater than
1.
Well what you could do is try something like this is in parentheses pass in one condition and then say
and pass in another condition.
And this is going to return an error.
And I'm going to do this on purpose to explain why there's an error happening here and this is a common
mistake for beginners to put this and here and you'll get a mistake that says something about the series
being ambiguous.
We scroll down here we see this the truth value of a series is ambiguous.
And what that is trying to say is that Python's normal and operator can't actually take into account
a series of boolean values compared to another series of boolean values and operator can only take into
account single booleans at a time.
So true and false returns False True and true returns true etc..
That means when you pass an entire series of boolean values such as this the and operator begins to
get confused.
It's can only deal with these single instances of boolean values which is why when you're working with
pandas and you want multiple conditions you actually have to use in ampersands which is this.
And so that's going to be shift 7 on your keyboard for most standard keyboards.
And notice here that I'm passing these conditions in parentheses to show clear separation between each
of these conditions.
And then if I run this I get the Row E where this happens to be true.
OK let's go to break that down.
One quick last time when you're trying to use multiple conditions you can't use Python's normal and
operator.
You have to use the ampersand and then you pass in the multiple conditions in parentheses and then finally
put it inside of brackets for your data frame.
If you want to do an OR operation you can just replace this with the pipe operator which is right above
your.
Usually for most keyboards on your Enter key go it go right above it and then do shift on that backslash
.
And this should give you the pipe operator and that's going to be a replacement for or because if you
try using or you'll get the same error.
So that's with or if I try it with Python's normal or Again I'll get the truth value of a series is
ambiguous.
And that's because you get a series of those values instead of just one or two boolean values.
OK you can go ahead and reference a notes if you want more examples of this.
But basically if you want multiple conditions pass in each condition in parentheses and then use an
ampersand or pipe operator for the ends or let's go ahead now and shift gears to talk about index and
a little more details about it we're going to discuss now is resetting the index or setting it to something
else.
Let me go ahead and take a look at our original data frame.
I want to go in and clear these cells here by just running them.
And let's take a look at our data frame.
Again we have Rose aither e in columns w through Z in order to reset the index back to the default which
is just a range index or a 1 2 3 4 all the way up to the nth row.
You can do is call the method reset underscore index and remember to close parentheses that we actually
run that method.
And if you call that notice you will get the index reset to a column and you'll get the actual index
then to be numerical.
And that's how you can reset the index.
Something to note here is just like a lot of other hand those data from operations.
This doesn't occur in place unless you specify it for it to occur in place.
So if I call back T.F. I still have that original index of labels if you want this to actually occur
in place what you would do is say in place equals true and pass into the reset underscore index.
For now we'll go ahead and not do that.
That way we still retain our original data frame.
Now go ahead and clear this out again if you just want to reset the index to a numerical value.
You can just use that Reesa index and your old index will become a column of your data frame.
OK.
Let's go ahead and show one more new index example such as setting the index and in order to do this
I'm going to go ahead and create a new index called New i n d.
And I'm going to use a little trick here that I'll explain in just a second.
This is going to be a bunch of state abbreviations and I'm going to say that split on this string.
And remember that split off a string should say common method for splitting off a blank space.
But this is a nice quick way to create a list instead of having to type out every comma.
So I want to go ahead and set that list equal to new I indeed.
So I have my new index values here and what I'm going to do is put this as a column in my data frame
.
So I'm going to go ahead and say DFA States is equal to new i n d.
And since they mentions match so there's five items here.
It will match from 0 to 4 here on the state of frame.
So if I check my data frame I have this new column called states CA NY w y o RCO.
Now we know how to reset the index which is just doing a reset underscore index method.
But if you have a column in your data frame that you want to be the index which you can go ahead and
do is say set index instead of reset index.
For example I can say DF dot set underscore index and then pass in the column name you want to be the
new index so states here and if I do shift enter now the states column has become the index.
Keep in mind unless you retain this information of the index so overwrite your old index and you won't
actually be able to retain this information as a new column.
Unlike resets index that allows you to have that new column.
So that's set index versus reset index.
Again if I check out my data frame here this wasn't in place.
I would need to add the argument.
If I do shift enter here in place.
Equals true.
Let me go ahead and just do a quick review of what we just covered before we head up to part 3.
Ok we learned about conditional selection which is very similar to PI operations.
You basically get a data frame of boolean values and if you pass in a whole data frame of boolean values
you'll get the values where it's true and you'll get any n or no values where it was false.
If you pass in a series of boolean values such as a column with a comparison operator you'll end up
only getting the rows of the data frame where serious happened to be true and you could break this down
into multiple steps if you wanted to.
Again these two cells are probably the most important to understand.
Out of this lecture if you understand this then you understand how to do a conditional selection in
pantless.
Then scrolling down if you wanted to do multiple conditional selection you could do an ampersand or
a pipe operator cannot use Python's builtin and or or operators.
Because of the ambiguous values in a Truth series and remember to use parentheses to separate each conditional
selection statement.
Then finally we have reset.
Index reset index is just going to reset your index to a numerical index and take your old index and
set it as a new column called index and then you can set a new index instead of resetting any index
based off a current column just by saying Set underscore index.
And both these methods are not in place by default.
So make sure to specify in place ECOs true.
If you want the change to be permanent.
All right that's it for this lecture.
Thanks everyone.
I'll see at the next lecture or we'll discuss index higher.
Again all these notes are just in a notebook called Data frames.
Thanks and I'll see if the next lecture