OK so we've got a confusion my drinks and we've got a rock curve as well as an A you see Metrolink.
Let's write this down actually lets out communicate with ourselves.
So now we've got a rock curve and you see metric and a confusion matrix so we've got a rock curve an
asymmetric and a confusion matrix.
Let's get a classification report as well as cross validated.
This is a key one here cross validated precision recall and if one school
beautiful song if you want a classification report we've already imported the function and so we can
do it and go print classification report in our classification report is going to give us the precision
recall in F one school but let's see it first.
Classification report we can find it by passing it wide test.
And why parades boom this is given us everything that we need right.
Because validated precision recall that school.
Well not really.
The reason being is because this classification report it's been on why spreads.
It's only being done on one set of test data so we've got a classification report.
It's been found by this and we can tell that by coming up here back to where we made predictions.
So we only made predictions on the test dataset.
So if we come back we only make predictions on here.
So what we need to do is a cross validation score of precision recall and F1 which is the values that
we've got here.
Precision recall in F1.
Now you might be looking at this going.
Daniel's timing we need to cross validate precision recall in F1 still not exactly sure.
I kind of forgot what's actually going on here.
But that's okay.
Let's get a refresh of what's happening.
We've got precision so precision indicates the proportion of positive identification so a.k.a. the model
predicts Class 1 which is has heart disease which were actually correct.
So a model which produces no false positives right.
So false positive up here is false positive predicted one when it's supposed to be zero.
A model which predicts none of these has a precision of one point zero.
All right.
So that's precision now recall indicates the proportion of actual positives which were correctly classified
so a model which produces no false negatives has a recall of one.
So model which produces no false negatives so remember false negatives where the model predicts zero
in a binary classification predicts zero where it should have predicted one.
So model with zero in this box will have a recall of one and a model with a precision of 1 will have
a zero in this box.
So if our model was perfect these are both big zeros and our precision and recall would both be one
point zero and now F1 is a combination of precision and recall.
So if our model was perfect and had precision and recall scores of one point zero F one would also be
one point zero now support.
Support is the number of samples each metric was calculated on in our test dataset.
There are 29 samples with a value is 0 and 32 samples of the value of 1.
So has heart disease and now accuracy we've seen that before and macro average.
This is show for average as the average precision recall an f1 score between classes.
So between these two to the macro average doesn't take class imbalance into account.
So if you have class imbalances in our case we don't because we have relatively similar numbers for
values here.
So say for example this was 2 and this was fifty nine then you would have class imbalances.
If you do have class imbalances.
Pay attention to the macro average but for the weighted average is the weighted average precision recall
an f1 score so weighted it a key here between classes so weighted means each metric is calculated with
respect to how many samples there are in each class so a.k.a. this metric will favour the majority class
e.g. this will be a high value.
When one class outperforms another due to having more samples because that class has more samples and
it will be given a higher White who was a bit of a mouthful but you can look up this information in
the classification report documentation.
The classification report SBA loan.
You can also look up each of those metrics that we just covered individually right.
So if you wanted to find out more actually on want to use a guide Derek is going to tell you some information
about the classification report.
If you want to search for more go precision recall F1 and you could look these up macro average weighted
average in the socket loan documentation but remember we just said before that these metrics here are
only calculated if we look at our cross validation diagram are only calculated on one test split.
So what we want to do is recalculate these metrics but with using cross validation which makes him a
little bit more robust.
So how can we do that.
Mm hmm.
Well we might do is wait till the next video so spoiler alert in the next video we're gonna calculate
these metrics here using cross validation.