Welcome back.
And in the last video we checked out how to tune a model's hyper parameters by hand.
And we figured out by the end of it once he'd written some code ones we've split our data into train
validation and test sets manually.
We trade loan valuation function Yeah.
At the end we figured out cheering him on by hand.
That sounds like a lot of work and you're right.
It would be if we went through and tried a whole bunch of different parameters.
But luckily the developers socket loan have also found to come across this problem and have created
randomized search CV which stands for randomize search cross validation.
Let's see how we'll tune hub parameters so a.k.a. adjust the settings on our models to make better predictions
or hopefully make better predictions.
Using randomized search CV.
So let's go here and create another heading.
This can be 5.2.
So hyper parameter churning with randomized search.
See the beautiful first thing we'll do is Will important.
So from S.K. learn dot model selection import randomize name.
We might be had a press tab here at randomize search CV.
Beautiful.
Let's see an example of how we use it.
Before we check out the dog string.
So what we'll do is we'll create a grid of hybrid parameters and by grid I mean dictionary and I've
typed Rod here a dictionary of hyper parameters we'd like to adjust so if we come back up here these
are the parameters we're going to adjust.
So what we're going to do is create a dictionary with the hyper parameters we'd like to adjust as the
keys and then the values we'd like to try as the values of the dictionary.
So let's do that.
So an estimate is that's a key we're going to get a positive list.
So one hundred two hundred five hundred one thousand and 12 hundred beautiful.
Now if you're wondering Daniel like where are you getting these values.
Well remember as always with any model or estimating socket loan there's extensive documentation on
it which tells you some of the settings that you can change on the model.
So what I've done is I've read through here read through the examples done some research and found different
values for the high parameters we can try to get here.
None 5 10 so it looks like I'm just throwing our values here.
Trust me they're not just sort of throwing out values.
They are based upon some research and some experience and don't worry you won't begin with that.
No one begins with that.
It takes a little bit of practice to go through it.
That's why we're having a look hands on at all these different functions that we can use to improve
our models
wonderful.
And then we'll do two more.
Which is mean samples split
two four six Come here man samples leaf
and let's see how we'd implement randomized search saving.
So we'll set up a random seed
we'll split into x and y
so we want heart disease shuffled actually because that's what we've used before.
Don't drop.
Oh actually we need to make X here don't drop target access equals one and we'll make Y here say heart
disease shuffled
target beautiful.
We're gonna go split into train and test sets huh.
I've been wondering why we only using train and test here.
We've just created a validation set now.
We'll get to that.
Why train.
Why test.
Train test split.
X Y and we'll use a normal split he s a test size equals zero point two wonderful and then we're going
to instantiate
random forest classifier CSF equals random forest classifier and jobs equals.
Now we could do negative one here but at the moment negative one is broken for me.
Now that'll make sense in a in a second but in jobs stands for how much of your computer processor are
you going to dedicate towards this machine learning model and negative one means all of it.
So by default and jobs is one actually what is the default in jobs.
Let's go here and jobs equals none.
Okay so we'll said listen I'm actually I'll just settle as one so different values you pass to and jobs
will dictate how much of your computer processor that you want to dedicate towards machine learning
model.
And now once we've got a classifier instantiated we'll set up randomized search saving.
So we're gonna pass it we're gonna call it R.S. CNN.
So what that's going to do is it just pending arrest to it for randomize search and now randomized search
CV the first parameter it takes is estimated equals CSF.
So we're passing it this this random fast classifier we've we've instantiated param distribution is
the next one we're going to pass it out grid.
So this is great up here.
The next thing is we're going to define in it to say equals 10 and this is the number of models to try
would be five fold cross validation and then we'll set the bow city for both equals to while this is
some code we haven't seen before.
So what is happening when we run randomized search CV we look at the doctoring press shift tab in here
that's because we have an imported.
That's right.
Well press shift tab here now.
So what's the doctoring here.
Randomize search on hyper parameters randomize search CV implements a fit and a score method.
It also implements predict predict probe decision function transform in inverse transform.
If they implemented in the estimate we used as a randomized search on hyper parameters that can sound
a little bit confusing what does it mean by randomize searched so that's in the name randomize search
CV CV is for cross validation which is where you might recognize this parameter see vehicles five.
So that means we're using five fold cross validation.
So five fold cross validation.
This is why we don't necessarily have to create a validation set here.
So what randomize search CV will do is it will take our classifier and it'll take a pram distributions
grid which is this and it's going to search over this grid for 10 different times different combinations
of these parameters at random so for example on its first iteration.
So because it's doing any the Eagles 10 it's gonna do this 10 times on its first iteration.
It might try and model with 10 estimated as a max depth of nine Max features set to auto mean samples
split set to two mean samples leaf set to 1 and then on the next iteration so iteration 2 out of 10
it might try a thousand estimate as 30 as the max depth auto is the max features four as the main samples
split and then four as a means samples leaf then it's gonna keep going that up to 10.
Now we could set this to 100 to a thousand.
I don't think there's even a thousand combinations in here if you went through you could go six times
five that's 30 times to that 60 times three that's 180 times three that's four hundred and sixty on
the math may be a little bit off there but you get the point right if you went through all of these
and tried to combine them and every single combination that's gonna be a lot of different models.
So let's see this in action.
So once we've set it up instantiated randomize search CV we'll go here fit the randomized
such savy version of CLSA.
On R.S. CSF not fit X train that y train.
Now the reason we only have defeated X train and Y train is because the CV in randomize search CV stands
for cross validation.
So that is what it's going to do it's going to automatically make our validation sets for us which is
a beautiful thing.
So if we come back to our diagram it's going to take our data and then it's gonna try different hyper
parameters cross validated on different hyper parameters settings.
So that's what it's going to return.
It's gonna figure out which combination of these high parameters is the best up to 10 different models.
And again we can increase this if we wanted to try more combinations.
So let's see it in action.
So this is where we've set for verbose to 2 it's going to output and tell us what's going on.
So fitting 5 folds for each of 10 candidates totaling 50 fits a.k.a. it's trying 10 iterations of different
combinations of parameters in this grid and splitting each combination five times because save equals
five totalling 50 fits on the data.
So it's going to run this fit function 50 different times using different hyper parameters on different
splits of the data so we can see here the first bullet it's trying is an estimate as equals twelve hundred
because this is one of our hyper parameters up here.
Remember each split each iteration it's going to pick combinations of these at random.
It's also set min samples split to six here Min samples leave to two max features the square root and
Max depth to five.
Okay.
And then if we were to scroll through here we can see all of the different combinations it's tried so
let's pick one at random.
This one here.
So this combination is trying an estimate as equals 200 means samples of split equals six Min samples
leaf equals two max features equals square root and Max depth equals none.
And then it's gonna keep going until it's finished it's gonna give us a warning here so let's see once
it's finished.
What we'll be able to do is call best programs on it.
So if we go r s CnF not best parameter and this is going to show us the combination of hyper parameters
which combination of these got the best results.
Wonderful.
So where an estimate is equals 200 Min samples split equals 6 mean samples leaf equals 2 max features
equals square root and Max depth equals none.
They were the best cross validated results across 10 different models and now when we call predate on
our randomize search classifier by default it's going to use these parameters.
So instead of finding these by hand randomize search CV has found them for us.
So let's do it.
Let's make predictions with the best hybrid parameters
what we might do are as y parades eagles are ACL left up predict we use we could use the validation
said here but in our case we're gonna use the test set and then we'll go evaluate the predictions
IRS metrics Eagles evaluate
why test are X Y parades
well now function is supposed to be evaluate parades there we go wonderful.
So if we come back up here did we see an improvement here.
No we didn't.
And so this is the sort of where the experimentation comes in hyper parameter tuning you won't always
find an improvement after running something like this.
Maybe we could run it for longer and try 50 different combinations.
And after it's tried 50 different combinations it will find some parameters which end up being better
than our manually tuned result.
But what I hope you're saying to see is that using randomized search CV rather than running through
all these different settings by hand it gives us a way to codify or function eyes to the tuning of hyper
parameters.
And so now there's one more way we can use to improve our models hyper parameters and it's with grid
search TV.
So it's kind of similar to randomize search but it's got one.
One key difference.
We'll have a look at that in the next video.