Welcome back.
Last video we saw how to tune one of our machine learning models specifically the K newest name is classifier
by hand.
Now this is hyper parameter tuning what I've done is I've just adjusted this little title here to say
hyper parameter tuning by hand.
But as you could imagine if you had more than one parameter or hyper parameter in our case we only tuned
in neighbors would go up to the K neighbors classifier.
We only tuned this one parameter.
We wanted to chain all of these I get a bit tedious.
We were writing for loops for days.
So what we're going to do here is.
And remember we've also written off our K nearest neighbors square in the search of better results.
So because our logistic regression in random forest classifier performing better we're going to cut
the cane to its neighbors model out of our experimentation.
So now we're going to go hyper parameter tuning with randomized search CV.
And so this is what we saw in the socket line section.
And if we search it out randomized search save a S.K. line we'd find it here beautiful so there's the
documentation there you can read through that on your own time.
But we're going to see it implemented.
And again if you're wondering how you could find the hyper parameters to tune a set machine learning
model.
Well here's what I would do for logistic regression.
So how to tune a logistic regression machine learning model in Python something like that.
If we switch that up logistic regression model training with psychic line beautiful logistic regression
using Python psychic line high parameter optimization of machine learning models chaining parameters
for logistic regression.
If we were to go through these and research and do our own experiments and figure out things and ask
more questions and probably find some pretty good answers.
But for now we're going to pretend we've gone through those steps and we figured out that we can use
randomize search CV and that we can tuna a bunch of different parameters.
Let's do that.
We're going to have to Choon
logistic regression and logistic regression model which is that and random forest classifier
using randomized search CEV and again if you need information about which parameters you can use for
logistic regression I've even spent that wrong.
Difficult Daniel logistic regression and random forest classifier can go up here as canine logistic
regression so if we look in here this is going to give us a list of different parameters and you might
read the documentation and trust me.
I did this the first time I did this I read the documentation and I'm like this means all of these terms
like elastic net L2 penalties bool actually I know what it all means true or false right.
All these scenes didn't mean much to me.
It was only once I started to do some research and go through and figure out what all these different
parameters actually meant was that I started to understand it and the same goes for the random forest
classifier.
You could do the same thing put that into the SBA loan documentation or change this to be how to tune
a random forest
machine learning model in Python.
These are steps that people like machine learning engineers and data scientists take every single day
to figure out how to improve their models.
No one in the beginning knows how to do this off by heart.
Even after doing it many times I still have to look these things up.
So we go here let's see how we would do it.
So the first thing to do with randomized search CV is and if you're wondering what CV stands for it
stands for cross validation we saw that in psychic loan and more specifically cross validation what
it does is instead of doing a normal training test split like we've done before by creating one training
split and a test blend a.k.a. 80 percent in the training split and 20 percent in the test split it creates.
This should be really be okay.
But I've done five because it looks nice here.
This can be k fold cross validation.
So 5 is the default in get line in the latest version but you can really adjust this and what it's going
to do is go on it create five different versions of training data and five different versions of the
test data and then evaluate different parameters Rama because we're doing a hyper parameter search evaluate
different hyper parameters on all of these different sets of all of these versions of the training and
test data and work out which set of parameters or hyper parameters is best across these five different
splits rather than just being one single split.
So if we go here let's do it.
So what we need to do is create a hyper parameter grid for logistic
regression and reading the logistic regression documentation as well as searching up here for logistic
regression.
We figure out that there's a few high parameters that we can tune such as the value for C which if we
look in here we go down here see.
So inverse of regularization strength must be a positive flight.
Again you could read this the first on the documentation go what any of these words mean.
That's why it requires research to check it out.
And then there's another one called solver.
So we'll go here we'll just have a look at what a parameter grid looks like.
See.
And again there are more where the using to here.
So we're only using C and solver you might find in your research that you could adjust the penalty you
could adjust the fit intercept you could adjust a whole bunch of height parameters but the overall concept
of adjusting hot parameters is what we're focused on.
So we're gonna set this after our research we know that a good value is NDP log space negative four
and if you're wondering what the long space is I'm kind of throwing these things out relatively quickly
log space does if we go here.
Returns numbers spaced evenly along a log scale.
So the start stop number.
So this is gonna be between negative four between four and 50 of them.
And if you're wondering what a log space is these are the ways you can do it.
Let's go.
What is a log.
Space
log space reduction and complexity.
Can you explain in simple words what is log space reduction so we could read that again for a few complex
words there but after a little bit of effort you'll be able to figure it out.
So we go here we can only really use one solver here because really after the research and finding what
parameters we should adjust we find that c is probably the most valuable one for logistic regression.
So we're only going to use one value for solver.
So really what we're testing here we're creating a grid of numbers on a log space between negative 4
and 4 so there we go.
Oh sorry.
Because it's a long space.
It's 1 times 10 to the power of negative for up to 1 times tend to the powerful.
So a pretty big space there.
These numbers are pretty well separated.
So let's get rid of that and now we'll create type of parameter
read for random forest classifier wonderful and through our research we find that some of the best parameters
for a random forest number of estimates which is if we're using a random forest and estimate us is how
many trees that we have in our forest.
So N.P. a range and you might be wondering why I'm using ranges here when I'm going through this.
Well the reason is because if we look up the documentation I know in a previous video we have an explicit
the use range.
We've used a list just for a refresher on what a range does.
Let's check out this so it's basically just going to create a range of numbers space 50 apart because
at 50 here between 10 and 1000.
So if we look up randomized I think we already have it maybe here.
There we go there somewhere here.
In contrast
it is highly recommended to use continuous distributions of all continuous parameters.
So that's why we create a range of different values.
This is where we read that from rather than just being an explicit list we're creating a range of continuous
distribution for our hyper parameters for randomized search CVA.
But that's just in the documentation.
So we've got here you could still just use a list but the documentation of people have written ask a
loan lobby recommend us to use a range of values.
So I'm gonna go Max depth here which is another one that we found through our own research of a hyper
parameter that we can tune remember for any machine learning model.
A quick search of going hey I'm using this model I've figured it out I followed the psychic line.
I don't have the map do I.
Yeah we do.
I find the map and I've decided I'm going to use a random forest classifier.
How can I do high parameter tuning Oh.
Well that's where it comes to searching something like this.
Right.
So we go here we find another one is the mean samples split and we go here and we're going to go with
the range.
Actually I've said that we're gonna use a distribution but here Max depth we haven't used a distribution
we've tissues an explicit list.
We'll use another range here and then we'll go here and we'll also use mean samples leaf and p a range
one twenty two.
Beautiful.
So now we have to hyper parameter grids for the two models that we're going to try and have a parameter
tune using randomized search CV.
So what we'll probably do is we'll end this video here and then we'll come back and we'll we'll use
these two grids along with randomized search CV to try and improve our results of our logistic regression
model and our random forest classifier beyond what we initially got.
So beyond these initial values that we got.
Remember these values were obtained without using cross validation because we only used a single train
and test split whereas if we go into keynote and cross validation.
So this is what we used to get our original scores a single train and test split whereas in cross validation
we're going to be using multiple train in test plate remember in the psychic loan section we discuss
that if you're going to provide a metric in terms of how a model is performing it's probably best to
use a cross validation metric especially with classification problems.
So let's revisit that in the next video of tuning our random forest and logistic regression with randomize
search CV.