So we left off the last video calculating our model's false positive rate.
True positive rate and at what thresholds using the rock curve function.
Now we kind of got back the false positive rate but it's in an array of numbers.
So looking at this again doesn't really make much sense but plotting it and seeing the rock curve the
actual rock curve will make a bit more sense.
What we're gonna do is we're going to create a function for plotting RISC curves.
Now in your practice when you need to make multiple plots as you saw in the map plot lib section you
might want to create a function that does plots for you if you want to make more of the same plot.
So that's what we're going to do here.
Just see an example of how we might create rock curve plotting function.
So first of all or import that plot lib pipeline.
We're going to go as P L T beautiful.
I'm gonna go call it nice and simple plot rock curve.
It's gonna take the false positive rate and the true positive right now we're going to come here.
We'll leave a nice dock string so it makes makes a bit of communicative sense.
Plots are rock curve given the false positive rate NPR and true positive rate TPR of the model.
Wonderful nice and simple.
Then go plot rub curve BLT dot plot we're gonna pass it the false positive rate in the true positive
rate and we'll give it a color score.
Remember these are these libraries take Americans smelling your here orange and then label equals rosy
for curve and then we're gonna go plot line with no predictive power.
This is gonna be a baseline you'll see it when the plot comes out it'll make a bit more sense that way
we can compare our model to some some other arbitrary model which just predicts nothing.
0 1.
Wonderful and the color is gonna be dark blue line style we'll keep it dotted.
Actually this is only 2 so it looks different to our rock curve and the label is guessing.
So this is this line you'll see when we plot is basically like our model was just guessing then we're
gonna go customize the plot.
Peyote don't ex label.
This is going to be a false positive rate if PR and then we're getting a penalty.
Why lie.
Well you might be hard to guess what this is.
This is the true positive right.
This is because we got FBR as the ex label and TPR as the y data true positive right.
And then we're gonna go receive our operating characteristic ROIC curve.
Now this is the type of plot you might give someone when they're asking for the rock.
Right.
So this is why I'm showing you this show and you'll probably also come along.
This sort of plot when you have a look at different classification problems online.
I might use the rock curve as an example of how the model's doing.
We want FBR first TPR.
Beautiful.
So this should work.
Wonderful.
What is going on here we finally get to see what a rock curve looks like.
If we go to here is the false positive rate is zero point six the true positive rate is gonna be 1.0
is the maximum score it can get is 1.0 up here and so this model here going from corner to corner is
guessing.
Can you guess where the most ideal rock curve might end up.
If this is guessing and our model is doing far better than guessing by getting about 80 percent 85 percent
82 percent something like that.
Can you guess where the most ideal curve will go.
That's all right.
If you can't we're going to have a look at it.
So before we do actually is let's have a look at the AUC score.
We can do this from S.K. loan import rock a U.S. score.
And so this is where the AUC comes into play from rock.
So when you hear area on the curve or receive operating characteristic what is a UC school.
And so we can do that.
Let's just see it in action.
This takes the test labels as well as the probabilities
so we can do Y probs.
We need
positives.
There we go.
So how did I know this again doctoring.
Why.
True why school.
Wonderful.
And so what is this measuring you might be asking.
Well this is.
Remember AUC stands for area under curve.
And so if we were to calculate take our curve ignore this one for a second let's just get rid of that
so if we were to take out curve this year it's a bit of a jagged curve but it is a curve.
And if we were to measure the area so shading everything under here shady and all of this and figure
it out then we go one across here well missing some space up here.
So our area areas definitely not going to be one our score is gonna get point eight four so point eight
for nine.
Now the maximum score you can get can you guess with a AUC score.
So area under the curve the maximum score you can get is one point zero.
And so that means that the curve goes straight up here to this corner and straight across here to that
corner.
Now let's see it in action so if we're gonna go plot perfect rock curve and a U C score.
So when want go to FBR TPR thresholds equals rock curve y test.
Now we're gonna go plot where you're gonna use our function here.
Rock curve.
See where it comes in handy because you want to plot multiple rock curves TPR FBR TPR.
Let's do that.
So this is a perfect rock curve here.
The area under the curve here.
Can you guess what it would be.
We just discussed it.
So perfect.
Are you say score is a U C score why test why test now.
That's one point zero.
And so in reality a perfect rock curve is very unlikely that means you've got a perfect model it's got
no false positives or everything's a true positive.
If you saw a 1.0 here's a rock IESE score.
Is that going to sort of maybe raise a flag to see if your model is predicting the right thing.
The main details here that a rock curve is predicting is a true positive rate versus a false positive
rate.
And the main metric you can use to boil it down rather than just being a curve.
You can use the AUC score.
So that's what you do to calculate a rock curve.
If someone asked you to calculate a rock curve for your model and it's going to compare the false positive
rate versus the true positive rate.
So now we've seen rock curves an area under the curve.
Let's have a look and the next classification metric on the next way to evaluate classification model
and that is with a confusion matrix.