WEBVTT - This file was automatically generated by VIMEO

0
00:00:12.490 --> 00:00:15.430
Hi everyone, and welcome to our ninth module,

1
00:00:15.760 --> 00:00:20.190
which will be focused on machine learning in Jupiter and Lea.

2
00:00:22.150 --> 00:00:23.570
To introduce our module,

3
00:00:24.190 --> 00:00:28.970
we will learn the basics about gaming clustering, and we will create, uh,

4
00:00:28.970 --> 00:00:30.050
and evaluate, uh,

5
00:00:30.220 --> 00:00:35.170
cames models and find its optimal number of clusters using, uh, various methods.

6
00:00:36.410 --> 00:00:40.500
Then we'll learn the basics about, uh, XG boost algorithm,

7
00:00:41.160 --> 00:00:43.140
and we'll make a binary model with it.

8
00:00:44.620 --> 00:00:49.280
We will learn also the basics of live G B M and we'll tackle a

9
00:00:49.770 --> 00:00:51.280
multi classification problem.

10
00:00:52.630 --> 00:00:56.540
We'll learn how to effectively evaluate our models, uh,

11
00:00:56.540 --> 00:00:57.980
using cross validation.

12
00:00:59.560 --> 00:01:04.390
We'll also create a whole pipeline consisting of multiple pre

13
00:01:04.390 --> 00:01:06.070
protesting steps and the model.

14
00:01:06.930 --> 00:01:10.790
And since the algorithms and the pre-processing steps can have, uh,

15
00:01:10.790 --> 00:01:12.030
many hyper parameters,

16
00:01:12.800 --> 00:01:17.430
we'll learn how to find the best ones using few methods of hyper parameter

17
00:01:17.430 --> 00:01:22.220
tuning. In this video, we will learn more about,

18
00:01:22.360 --> 00:01:23.820
uh, k use clustering.

19
00:01:26.240 --> 00:01:30.330
It's objective is to partition a set of observations into

20
00:01:31.170 --> 00:01:36.050
a specific number of clusters where each observations belongs to a cluster with

21
00:01:36.050 --> 00:01:40.050
the nearest mean serving as a prototype for this cluster,

22
00:01:41.440 --> 00:01:43.930
this 10 trade based, uh,

23
00:01:43.930 --> 00:01:48.130
that means it works by assigning data points to the next, uh,

24
00:01:48.490 --> 00:01:52.850
centrate and then recalculating centrate based on the points assigned to it.

25
00:01:53.430 --> 00:01:58.290
And this process is repeated until convergence, it's distance based.

26
00:01:58.960 --> 00:02:03.860
That means this calculating the and distance between points, uh, scaling,

27
00:02:04.240 --> 00:02:08.500
uh, is very important. Uh, since it is, uh, distance based,

28
00:02:08.880 --> 00:02:13.260
it is a crucial factor. All variables need to be on a same scale.

29
00:02:14.620 --> 00:02:18.750
Number of clusters need to be specified in advance. Uh,

30
00:02:18.820 --> 00:02:23.470
this might be a disadvantage of this clustering algorithm, uh,

31
00:02:23.470 --> 00:02:27.630
but ca is clustering is used usually, uh, where you have, uh,

32
00:02:27.890 --> 00:02:32.880
domain knowledge to know approximately the number of clusters in advance. But,

33
00:02:33.060 --> 00:02:36.440
uh, there are methods to find suitable number of clusters,

34
00:02:37.070 --> 00:02:41.760
even if you do not know the exact number. It works with numeric data,

35
00:02:42.650 --> 00:02:45.790
uh, because it is distance based. It works, uh,

36
00:02:45.940 --> 00:02:50.550
best with only numeric data like continuous or ordinal variables.

37
00:02:51.730 --> 00:02:52.563
You can, of course,

38
00:02:53.070 --> 00:02:57.310
transform categorical variables into numeric using one hot encoding,

39
00:02:57.310 --> 00:03:02.100
for example. But, but, uh, in case of Cains,

40
00:03:02.160 --> 00:03:06.700
it is not recommended. Uh, so if you can use only numeric variables,

41
00:03:08.190 --> 00:03:12.360
it's, uh, a very scalable algorithm in terms of, uh,

42
00:03:12.360 --> 00:03:14.720
number of rows and also number of variables.

43
00:03:15.460 --> 00:03:17.720
But in the relation to number of variables,

44
00:03:18.020 --> 00:03:22.880
it is highly influenced by the curse of dimensionality. That means,

45
00:03:23.060 --> 00:03:25.280
uh, that with higher number of variables,

46
00:03:25.540 --> 00:03:29.560
the distances between observations tend to become more uniform.

47
00:03:30.700 --> 00:03:33.160
Uh, it's, uh, basically means that, uh,

48
00:03:33.260 --> 00:03:37.400
all points seem are roughly equity distance from one or other.

49
00:03:38.510 --> 00:03:42.580
Uh, in short, the less variables used, the better, uh,

50
00:03:42.750 --> 00:03:47.140
Cains algorithm assumes, uh, clusters to be similar size,

51
00:03:48.380 --> 00:03:50.640
uh, density and spherical.

52
00:03:51.700 --> 00:03:56.080
It may happen that clusters of different sizes will not be identified as well as

53
00:03:56.240 --> 00:04:00.240
clusters, uh, with various densities or with complicated shapes.

54
00:04:02.180 --> 00:04:05.650
Cains algorithm is sensitive to initial conditions.

55
00:04:06.750 --> 00:04:10.440
Different initializations may lead to different results,

56
00:04:10.620 --> 00:04:15.020
and Cains may be stuck in local mini. But, uh,

57
00:04:15.020 --> 00:04:19.780
there are few initialization techniques which help, uh, to tackle this problem.

58
00:04:21.070 --> 00:04:25.930
Uh, it is also very sensitive to outliers. Uh, so, uh,

59
00:04:26.470 --> 00:04:30.010
proper, uh, normalization method is also very important.

60
00:04:32.070 --> 00:04:36.730
It means clustering, uh, it's an, uh, unsupervised national learning method.

61
00:04:37.030 --> 00:04:37.250
Uh,

62
00:04:37.250 --> 00:04:42.210
that means we do not know the group where the observations are belonging in

63
00:04:42.210 --> 00:04:46.650
advance. We assigning the group, uh, or the cluster, uh,

64
00:04:46.650 --> 00:04:50.970
based on similarity of the inputs. In our next video,

65
00:04:51.390 --> 00:04:54.450
we will create a clustering model in Jupiter.
