Hoo hoo hoo hoo hoo.
I'm rubbing my hands if you can't hear them.
You know why.
Because I'm pumped.
I'm ready to to evaluate our machine learning models or see how they go see how each of these models
using the function we've just created in the last video see how logistic regression goes.
Caner its name is classifier and random for its classifier goes at finding patterns in our training
data and then how those patterns get evaluated in our test data.
So without any further ado let's see how each of these models performs so we can call our function fit
and score and hopefully it works and we got here.
We see that it takes models or actually if we press shift tab this is where our doctoring comes in handy
because it tells us what happens fits and evaluates given machine learning models.
This is beautiful.
This is just like what we've seen with other functions but except we've created this one ourself.
That's the helpfulness of a doctrine right.
So we're gonna go here.
Models equals models which is our dictionary of machine learning models we could just go X train x test
but for completeness we're gonna go X train equals x train and then we're going to go X test equals
x test and then we're going to go y train equals Y train and then finally y test equals Y test.
And if we've written that function correctly what do you think it's going to return.
I'll give you a few seconds okay.
That's enough because we want to see it.
If in doubt run the code let's see what they our function returns
okay.
So we get some warnings here.
What's it say stop.
Total number of iterations reached would require us to dive in.
Increase the number of iterations so this is where it's helpful.
Increase the number of iterations Max ITER or scale it down.
So this is saying that potentially our logistic regression model could be improved and to figure out
how you could do that would require going to the documentation for alternative solver options.
But what we're gonna do is just work with what we've got so far.
So if we have a look at this this is the score how each of our models without tuning.
Oh that's a spoiler for what's coming out.
This is how each of our models as the baseline class has performed on finding patterns in our test data.
So it's getting the score here.
This is what we're getting back.
So if we look at this which ones are highest look at logistic regression coming in as the dark horse.
Not even on the machine learning map.
Right.
And it's getting the highest score.
Mm hmm.
Well this might take a little bit of investigation because remember where we up to we're two experiments
and that's what we're doing here.
An experiment is comparing different models trying different models and comparing them to each other.
Now this is a comparison in a dictionary.
But if we wanted to show this to someone we might compare it visually.
So let's do that nice and quickly we can make a model.
Actually we might make a little heading here.
So we go here model comparison beautiful model compare equals might turn our dictionary into a data
frame because that's nice and simple to model scores.
This is just taking this dictionary here.
Model scores and we're gonna set the index to be what does it have to be and has to be a list called
accuracy because that's what this score function here.
Because our models are all classifiers their default score metric is accuracy.
We saw that in their socket Loan Section.
So if we go here model of compare we need to transpose it plot dot bar boom.
Now I'll just show you why we need to transpose it because if we did that it would look we'd like that
and we actually want it to look good like that.
So this is this is quickly showing how accurate each of our different models are.
And you can see logistic regression just tips out our random forest and K and N Well because that's
nowhere near the accuracy of our logistic regression model.
We're going to say goodbye to cane in so once we've got this graph this might be something that we can
take to the boss or one of our colleagues and say hey look at this we've built a machine learning model
a beautiful logistic regression model and it's performed the best.
So we're going to use the logistic regression model in practice.
And so you might go.
I found it.
And your boss is like.
Nice one.
What did you find.
And then you're like well the best algorithm for predicting heart disease is logistic regression and
then she might say something like.
Excellent.
I'm surprised the hyper Reverend attorney isn't finished by now.
And then you might wonder what is hyper parameter tuning.
Yeah me too.
I went pretty quick.
So you sort of covering yourself here and then she might say Well I'm very proud.
How about you put together a classification report to show the team and be sure to include a confusion
matrix and the class validated precision recall on F1 scores.
I'd also be curious to see what features are most important.
Oh and don't forget to include an ROIC curve and then you might be thinking or asking yourself okay.
There's some person a lot of complex words there.
Right.
But then you actually say of course I'll have it to you by tomorrow.
We're going to take care of all these things.
We're going to take care of all these things in the next video because we're still at a point where
even though we found a great machine learning model and we've probably shown that to the boss of one
of our colleagues and they've gone wow logistic regression performing at 88 percent accuracy we're still
not near our valuation metric which is if we come back up the top which is we said we kind of want at
least 95 percent accuracy to continue with this experiment.
So that's what we're going to be doing as well as fulfilling all those requests that the boss asked
us just to make sure our model was a little bit more robust than just getting the default score metric
so without any further ado take a little break.
Have a review of what we've done so far and we're going to review some of the things that we need to
make sure we do to make sure our classification models are evaluated correctly and are improved as well
as possible.