WEBVTT - This file was automatically generated by VIMEO

0
00:00:07.810 --> 00:00:12.150
Hi. In this video we will learn more about cross validation.

1
00:00:13.030 --> 00:00:18.020
Cross validation is a method of evaluating a model on multiple folds of data.

2
00:00:19.130 --> 00:00:22.340
This also a resampling procedure used.

3
00:00:22.400 --> 00:00:27.350
If we have limited data available. Uh, it's best illustrated on a diagram.

4
00:00:33.000 --> 00:00:37.700
We have already split our data, uh, to train and, uh, test it,

5
00:00:38.560 --> 00:00:41.700
and we trained our model on the whole training set.

6
00:00:42.290 --> 00:00:46.770
What we do with cross validation is to, uh, further,

7
00:00:47.150 --> 00:00:51.360
uh, split the training data to multiple faults.

8
00:00:51.660 --> 00:00:56.120
In this case, there are five faults and we train our model, uh,

9
00:00:56.150 --> 00:01:00.320
five times, uh, with the same parameters each time on four faults,

10
00:01:00.580 --> 00:01:05.310
and ed on the fifth fault. This way, uh,

11
00:01:05.410 --> 00:01:10.190
we will have a better view, uh, how our model generalizes on unseen data.

12
00:01:12.680 --> 00:01:16.020
The test data is used, uh, same as before,

13
00:01:16.160 --> 00:01:19.380
as last and final evaluation. When you get, uh,

14
00:01:19.450 --> 00:01:21.780
satisfiable results using cross validation,

15
00:01:21.880 --> 00:01:25.460
and you have found a good hyper parameters of the model.

16
00:01:29.740 --> 00:01:34.120
We can use a cross validate method for this, uh, where we input our model,

17
00:01:35.040 --> 00:01:39.820
uh, training data. Uh, we want just, uh, three folds. Uh,

18
00:01:39.880 --> 00:01:41.500
we want on also cut.

19
00:01:43.590 --> 00:01:45.610
We can use cross validate method for this,

20
00:01:45.980 --> 00:01:50.610
where we input our model training data. We want just,

21
00:01:50.790 --> 00:01:51.770
uh, three fold.

22
00:01:52.770 --> 00:01:56.830
We also want to see a performance of the model on training data,

23
00:01:58.070 --> 00:02:01.570
and we also specify the scoring metrics, which we would like to see

24
00:02:03.550 --> 00:02:06.560
when we run it. Uh, we see the results on these three folds,

25
00:02:06.560 --> 00:02:09.360
and we see that our model is a bit over fitting.

26
00:02:10.190 --> 00:02:14.880
That means it predicts excellent on training data, but worse on data.

27
00:02:15.750 --> 00:02:20.000
E D L is approximately the same performance on both. Uh,

28
00:02:20.060 --> 00:02:21.520
we see it has, uh,

29
00:02:21.900 --> 00:02:25.800
almost a perfect performance on training data in all of the metrics,

30
00:02:26.700 --> 00:02:29.730
uh, but worse, uh, on the test data,

31
00:02:32.310 --> 00:02:36.040
what we could do is to change few parameters of the model. For example,

32
00:02:36.060 --> 00:02:40.240
number of trees or depth of the trees, uh, to lower values.

33
00:02:41.290 --> 00:02:44.150
Uh, but uh, in our case it is not so bad. Uh,

34
00:02:44.210 --> 00:02:49.110
but if it would be more than 10%, we would have to do that. In

35
00:02:51.770 --> 00:02:54.690
our next video, we will start to make pipelines.
