WEBVTT - This file was automatically generated by VIMEO

0
00:00:07.960 --> 00:00:12.740
Hi. In this video we will learn how to make machine learning pipelines in sticky

1
00:00:12.740 --> 00:00:16.540
learn. First, we need to answer a question.

2
00:00:16.650 --> 00:00:18.220
What is a machine learning pipeline?

3
00:00:19.050 --> 00:00:23.940
Machine learning pipeline is a series of tips which needs to be

4
00:00:24.010 --> 00:00:28.060
performed with the data, starting from data reading from various sources,

5
00:00:29.040 --> 00:00:32.280
uh, to making final predictions. Uh,

6
00:00:32.310 --> 00:00:36.600
what is in between highly depends on your data and your choices of algorithms.

7
00:00:36.620 --> 00:00:38.480
And a goal, what you are trying to achieve.

8
00:00:39.610 --> 00:00:43.610
A typical machine learning pipeline may contain these steps, uh,

9
00:00:43.670 --> 00:00:47.670
but be aware that, uh, not all are necessary. Uh,

10
00:00:47.670 --> 00:00:49.750
first you load the data from some source,

11
00:00:49.820 --> 00:00:53.510
like a database I p I or c s V files and so on.

12
00:00:55.070 --> 00:00:59.730
Uh, data pre-processing, uh, this step, uh, includes all necessary steps,

13
00:00:59.990 --> 00:01:04.930
uh, to process the data to its right form and shape correcting errors, uh,

14
00:01:05.000 --> 00:01:08.570
data cleaning, handling outliers, inputting missing values,

15
00:01:09.150 --> 00:01:11.250
or encoding categorical variables.

16
00:01:13.010 --> 00:01:16.900
Then your pipeline may include steps where you are creating new, uh,

17
00:01:17.260 --> 00:01:18.460
variables from existing ones.

18
00:01:18.610 --> 00:01:22.990
This is the feature engineering step feature selection.

19
00:01:24.130 --> 00:01:27.470
Uh, after this, uh, if your dataset has many variables,

20
00:01:27.530 --> 00:01:32.190
you may consider a feature selection where you drop unnecessary variables,

21
00:01:32.330 --> 00:01:36.350
for example, uh, which have very low variance or a lot of missing values.

22
00:01:37.710 --> 00:01:42.040
Sometimes you also need to normalize your dataset. This is the feature scaling

23
00:01:43.870 --> 00:01:45.600
dimensional reduction. Uh,

24
00:01:45.620 --> 00:01:50.160
you may also use some algorithm for dimensional reduction like, uh,

25
00:01:50.190 --> 00:01:52.840
principle component analysis. For example,

26
00:01:54.650 --> 00:01:59.580
feature dimens reduction methods are trying to preserve most of the

27
00:01:59.580 --> 00:02:01.260
variants in the dataset, uh,

28
00:02:01.280 --> 00:02:04.780
but they are trying to transform the data set that way. Uh,

29
00:02:04.800 --> 00:02:08.940
so its output will have much less dimensions like the inputs.

30
00:02:10.290 --> 00:02:14.630
And then you perform hyper parameter optimization where you search for the best

31
00:02:14.630 --> 00:02:19.070
hyper parameters of your model or even the right parameters in other steps of

32
00:02:19.070 --> 00:02:20.350
the pipeline. For example,

33
00:02:20.490 --> 00:02:23.950
you could optimize depths of the trees in XG boost algorithm,

34
00:02:24.850 --> 00:02:29.750
but also you may want to search for the optimal threshold for qual range used in

35
00:02:29.750 --> 00:02:30.590
robust scaler.

36
00:02:31.250 --> 00:02:36.110
The model evaluation step is present in a previous step, uh, but uh,

37
00:02:36.110 --> 00:02:40.490
you may want to explore the model, uh, into more detail. And finally,

38
00:02:40.830 --> 00:02:45.280
you make the predictions after you created your pipeline,

39
00:02:45.660 --> 00:02:49.440
uh, and your best model, uh, you have to deploy it and set actions.

40
00:02:49.550 --> 00:02:51.680
What should be taken based on these predictions.

41
00:02:52.300 --> 00:02:55.640
And this is sometimes even more difficult than the model training, uh,

42
00:02:55.640 --> 00:02:57.960
to set it properly. Today,

43
00:02:58.390 --> 00:03:03.200
most of the machine learning is basically about data engineering and

44
00:03:03.370 --> 00:03:07.400
about, uh, driving as many reasonable variables to the pipeline.

45
00:03:08.250 --> 00:03:12.430
And to have enough resources like, uh, CPUs and ra, uh,

46
00:03:12.450 --> 00:03:16.390
to perform hyper parameter tuning to have, uh, satisfiable results.

47
00:03:17.310 --> 00:03:21.360
It's common that even for simple business problems like propensity models,

48
00:03:22.220 --> 00:03:26.400
you throw thousands or even tens of thousands of variables into the pipeline.

49
00:03:26.900 --> 00:03:28.720
So everything should be automated.

50
00:03:29.660 --> 00:03:34.400
You can usually train your models on a good sample of your data in terms

51
00:03:34.500 --> 00:03:38.240
of millions of observations, or even a few hundred thousand,

52
00:03:38.940 --> 00:03:43.760
but sometimes we may need also a lot of resources to make predictions for all of

53
00:03:43.760 --> 00:03:47.200
the data, which can be in billions of observations.

54
00:03:48.050 --> 00:03:49.910
So it may be exactly the opposite,

55
00:03:49.940 --> 00:03:53.590
that making predictions will require much more resources than the training.

56
00:03:55.230 --> 00:03:55.810
In short,

57
00:03:55.810 --> 00:03:59.890
machine learning is mostly about available resources and data engineering,

58
00:03:59.890 --> 00:04:03.540
which takes a significant amount of time. Now,

59
00:04:03.540 --> 00:04:05.140
we can go into our notebook.

60
00:04:11.050 --> 00:04:15.910
LEA provides us a very effective framework how to perform most of the

61
00:04:15.910 --> 00:04:19.870
steps I mentioned. Uh, it has a pipeline class,

62
00:04:21.480 --> 00:04:24.190
which can be used to create a machine learning pipelines.

63
00:04:25.150 --> 00:04:28.100
In a list of two pole, uh,

64
00:04:28.100 --> 00:04:30.420
we specify name of the pipeline step

65
00:04:32.140 --> 00:04:36.940
and the estimator or transformer. Uh, we want to use. Uh,

66
00:04:36.940 --> 00:04:39.460
the difference between, uh, ticket learners, uh,

67
00:04:39.460 --> 00:04:43.980
estimators and transformers is that transformers only

68
00:04:44.460 --> 00:04:48.950
transform the data and they must have the transform method. For example,

69
00:04:49.010 --> 00:04:50.990
the minimax scaler is a transformer.

70
00:04:52.430 --> 00:04:56.500
Estimators are basically the models used, and they make the predictions.

71
00:04:56.920 --> 00:05:01.860
So they must have the predict method. For example, the E G B classifier,

72
00:05:03.170 --> 00:05:07.270
you can even create very easily your own transformers or estimators.

73
00:05:09.980 --> 00:05:12.430
Okay, now let's create our first pipeline.

74
00:05:14.720 --> 00:05:18.580
The first step is, uh, feature selection, where I used, uh,

75
00:05:18.940 --> 00:05:20.380
Univa eight feature selection method,

76
00:05:20.750 --> 00:05:25.670
which in combination with the select K best returns,

77
00:05:25.860 --> 00:05:30.230
only the best, uh, 20 variables according their ANOVA F value.

78
00:05:32.900 --> 00:05:37.080
As a second step, I choose the robust scaler with default parameters.

79
00:05:37.420 --> 00:05:40.880
And the last step is our XG boost classifier

80
00:05:44.350 --> 00:05:48.710
pipeline will have the same methods as the last transformer or

81
00:05:48.820 --> 00:05:51.890
estimator in the pipeline. In our case,

82
00:05:52.230 --> 00:05:56.570
the last step is the X GB classifier, which has the feed method,

83
00:05:56.830 --> 00:05:57.680
predict method,

84
00:05:57.980 --> 00:06:02.830
and also predict probus method to predict the probabilities and score method.

85
00:06:03.530 --> 00:06:06.430
So we can call these methods on the pipeline as well.

86
00:06:10.390 --> 00:06:15.070
When we call the feed method behind the scenes,

87
00:06:15.330 --> 00:06:19.030
the pipeline will move the input data into the first step.

88
00:06:19.780 --> 00:06:22.550
Call the fit transform method on the first transformer,

89
00:06:23.400 --> 00:06:26.330
move the output data of that step into the next one.

90
00:06:27.080 --> 00:06:29.330
Call the fit transform method on that step,

91
00:06:29.470 --> 00:06:31.890
and move the output data to last step,

92
00:06:32.260 --> 00:06:37.010
where in our case it'll call the fit method. And after fitting our pipeline,

93
00:06:37.150 --> 00:06:38.530
we call the score method,

94
00:06:39.530 --> 00:06:42.450
like here all steps in the pipeline,

95
00:06:42.510 --> 00:06:47.090
except the last step must be transformers. In case of the last step,

96
00:06:47.710 --> 00:06:50.090
it may be transformer or an estimator.

97
00:06:55.340 --> 00:06:57.270
When we score, uh, the pipeline,

98
00:06:57.530 --> 00:07:01.390
we see that it has the accuracy on the test set, uh,

99
00:07:01.620 --> 00:07:05.770
only 93%. Now,

100
00:07:05.860 --> 00:07:08.170
let's go ahead and cut.

101
00:07:09.370 --> 00:07:12.470
Now let's go ahead and create our second pipeline.

102
00:07:15.730 --> 00:07:20.500
Here is an example of normalization with the standard scaler

103
00:07:20.640 --> 00:07:22.900
in combination with, uh, P C A.

104
00:07:23.250 --> 00:07:28.010
This is the principle component analysis, which is, uh,

105
00:07:28.230 --> 00:07:32.530
dimensionality reduction technique. Uh, the parameter and,

106
00:07:32.750 --> 00:07:37.430
and components tells us how much variance I would like to

107
00:07:37.670 --> 00:07:42.350
preserve. I put here 90%, but, uh, this means that,

108
00:07:42.530 --> 00:07:47.310
uh, the P C A could return any number of columns. It can be 10, 20,

109
00:07:47.650 --> 00:07:49.630
or 30, or any other number.

110
00:07:50.290 --> 00:07:55.290
You can specify here on integer visual ensure that the P C A

111
00:07:55.290 --> 00:07:59.010
will return only specific number of columns in P C A,

112
00:07:59.010 --> 00:08:00.250
there are called components.

113
00:08:01.290 --> 00:08:04.890
Regardless of how much variance is preserved in the transformer data.

114
00:08:06.110 --> 00:08:09.630
I highly encourage you, uh, to learn more about this technique.

115
00:08:13.390 --> 00:08:16.200
When we feed the pipeline and score it, uh,

116
00:08:16.300 --> 00:08:20.000
we see the accuracy of 9 3, 3, uh, percent as well.

117
00:08:22.920 --> 00:08:26.780
Now a little bit, uh, on how to access and the steps of the pipeline.

118
00:08:30.750 --> 00:08:35.490
We can access the transformers or estimators in the pipeline very easily as

119
00:08:35.490 --> 00:08:39.370
well. Uh, for example, uh, here, uh, in this cell,

120
00:08:41.910 --> 00:08:43.240
this one, uh,

121
00:08:43.400 --> 00:08:47.520
I access the p c a step where I printed out the explained, uh,

122
00:08:47.600 --> 00:08:52.140
variance here of all the resulting components.

123
00:08:53.710 --> 00:08:55.760
We also see, uh, that, uh,

124
00:08:55.760 --> 00:09:00.600
with retaining of 90% of variance in the data, it,

125
00:09:00.600 --> 00:09:05.560
uh, reduced the number of dimensions from 57 to 43.

126
00:09:09.800 --> 00:09:11.370
Here in the next cell, uh,

127
00:09:11.370 --> 00:09:16.130
we see that we can call the transform method on this already fitted, uh,

128
00:09:16.370 --> 00:09:21.010
P C A, and we see, uh, what is the output of it, uh,

129
00:09:21.010 --> 00:09:24.650
which is then, uh, uh, fed into the XG boost algorithm.

130
00:09:28.480 --> 00:09:32.060
And of course, uh, we can make the cross validation of the pipeline,

131
00:09:32.210 --> 00:09:33.940
same as with the single model,

132
00:09:37.730 --> 00:09:39.070
and we see is the results here.

133
00:09:44.760 --> 00:09:45.700
In our next video,

134
00:09:46.120 --> 00:09:50.820
we will learn more about hyper parameter tuning focused on grid search.
