WEBVTT - This file was automatically generated by VIMEO

0
00:00:07.660 --> 00:00:08.380
Hi.

1
00:00:08.380 --> 00:00:12.960
In this video I would like to explain a little bit the notebook I created for

2
00:00:12.960 --> 00:00:16.080
this module. So in addition to Jupiter lab,

3
00:00:16.080 --> 00:00:19.660
you'll learn a little bit more about national learning, uh,

4
00:00:19.940 --> 00:00:22.100
specifically about the binary classification.

5
00:00:23.780 --> 00:00:26.140
I use the same data like in the fourth module,

6
00:00:26.680 --> 00:00:30.940
but here we will create a model to predict if the email is spam or not.

7
00:00:31.990 --> 00:00:36.520
This is called a binary classification problem because our target variable can

8
00:00:36.520 --> 00:00:41.130
have only two classes, zero or one as inputs.

9
00:00:41.290 --> 00:00:44.810
To this model, we will use all other remaining variables.

10
00:00:46.300 --> 00:00:50.640
Models that we will try is the logistic regression and, uh,

11
00:00:50.700 --> 00:00:52.320
random forest classifier.

12
00:00:54.340 --> 00:00:59.270
I'll not go into details about the theory or inner workings and assumptions of

13
00:00:59.270 --> 00:01:00.150
these algorithms.

14
00:01:00.660 --> 00:01:05.630
Important for us is that these models can be used for classification

15
00:01:05.630 --> 00:01:09.800
problems like ours. Now let's go into our notebook.

16
00:01:13.840 --> 00:01:18.820
You'll see that typical machine learning workflow will consist of these steps.

17
00:01:19.400 --> 00:01:21.820
You load the data from some data source,

18
00:01:22.530 --> 00:01:26.770
then you make our introductory and exploratory analysis.

19
00:01:27.260 --> 00:01:31.570
After that, you will pre the data, so that is suitable for modeling.

20
00:01:32.660 --> 00:01:37.240
You may also do some pre-processing steps during or before the initial analysis.

21
00:01:39.700 --> 00:01:43.590
Then you fit the data, you evaluate the model with suitable metrics.

22
00:01:44.790 --> 00:01:45.930
You score the data.

23
00:01:46.480 --> 00:01:50.970
That means you assign the predicted values or probabilities to each of the

24
00:01:50.980 --> 00:01:51.813
entry,

25
00:01:52.200 --> 00:01:56.260
and then you save this back to some storage where other users may access it.

26
00:01:58.950 --> 00:02:03.770
In the first sections, we prepare the data and product view charts,

27
00:02:04.350 --> 00:02:09.290
uh, just to check if the protein is working fine. And, uh,

28
00:02:09.610 --> 00:02:14.510
considered this as a very, very basic, uh, explanatory analysis.

29
00:02:20.570 --> 00:02:23.740
Then we splitted the data to train and test it.

30
00:02:25.690 --> 00:02:30.520
After that, we have fitted the logistic regression model and evaluated with, uh,

31
00:02:31.080 --> 00:02:33.520
accuracy metric on train and test set.

32
00:02:35.540 --> 00:02:40.130
Then we protested the confusion metrics and cumulative gain charts.

33
00:02:41.250 --> 00:02:44.310
We did the same with the random forest classifier.

34
00:02:50.650 --> 00:02:55.360
These evaluation methods and metrics depend on what problem you are dealing with

35
00:02:55.700 --> 00:03:00.280
if the data set is, uh, balanced and on the model used as well.

36
00:03:01.540 --> 00:03:03.840
Uh, one word to accuracy metric. Uh,

37
00:03:03.870 --> 00:03:08.080
this is basically a percentage of correctly classified emails.

38
00:03:08.670 --> 00:03:11.560
Doesn't matter if it is zero or one class.

39
00:03:18.470 --> 00:03:22.130
For example, accuracy of 90% might look excellent,

40
00:03:22.430 --> 00:03:23.930
but consider in our case,

41
00:03:24.390 --> 00:03:27.570
if we would predict for all the emails zero,

42
00:03:28.590 --> 00:03:32.440
then we would be correct in, uh, 60% of the cases.

43
00:03:40.140 --> 00:03:41.800
Now let's compare our two models.

44
00:03:46.590 --> 00:03:49.930
We see that the accuracy of the logistic regression is, uh,

45
00:03:49.930 --> 00:03:54.610
92% on train and test set. And, uh,

46
00:03:54.870 --> 00:03:56.610
for the random forest classifier,

47
00:03:57.060 --> 00:04:00.330
there is 74 and 70% accuracy.

48
00:04:00.870 --> 00:04:04.530
We see that the random forest classifier is definitely a worse model,

49
00:04:05.840 --> 00:04:09.460
but, uh, let's go a little bit deeper and explore the confusion metrics.

50
00:04:10.160 --> 00:04:14.730
The confusion metrics tells us how many emails were

51
00:04:14.730 --> 00:04:17.930
correctly or incorrectly classified depending on the class.

52
00:04:19.520 --> 00:04:23.500
We see that emails, uh, which are spam. This is the class one.

53
00:04:25.060 --> 00:04:29.440
We correctly classified them in 570 cases and we

54
00:04:29.790 --> 00:04:33.600
wrongly classified it, uh, in, uh, 63 cases.

55
00:04:37.400 --> 00:04:41.860
Uh, the same is for emails which are not spam. This is the Class Zero.

56
00:04:43.540 --> 00:04:48.160
We currently classified, uh, these emails in 832 cases,

57
00:04:48.900 --> 00:04:50.920
and we were broke in 54 cases.

58
00:04:56.520 --> 00:05:00.630
When we compare it to the random forest classifier, uh,

59
00:05:00.730 --> 00:05:03.350
the logistic regression is a little bit, uh,

60
00:05:03.400 --> 00:05:06.150
worse in classifying the zero class,

61
00:05:07.160 --> 00:05:11.580
but it's basically comparable, almost the same, uh,

62
00:05:11.640 --> 00:05:16.420
but the random first classifier is much worse in classifying the positive class.

63
00:05:17.080 --> 00:05:17.300
Uh,

64
00:05:17.300 --> 00:05:22.260
basically it is wrong in predicting the positive class in around 60% of

65
00:05:22.260 --> 00:05:23.093
the cases.

66
00:05:26.420 --> 00:05:27.950
With cumulative gain charts,

67
00:05:28.050 --> 00:05:32.750
we can go even deeper to have another view on how our models are performing.

68
00:05:33.740 --> 00:05:38.150
They are used to evaluate effectiveness of a binary classifier like ours.

69
00:05:40.160 --> 00:05:44.740
Uh, it can be used also for comparing two or more binary classifiers.

70
00:05:45.750 --> 00:05:48.000
Intuition behind it is very simple.

71
00:05:48.940 --> 00:05:53.200
You order the entries by the predicted probability that's sending,

72
00:05:54.220 --> 00:05:58.160
and you check the percentage of correctly classified emails. We have,

73
00:05:58.350 --> 00:06:01.920
depending on selection of top and percent of records,

74
00:06:03.850 --> 00:06:07.190
the dotted line, uh, this is a random classification.

75
00:06:09.130 --> 00:06:12.700
We see that, for example, logistic regression.

76
00:06:13.670 --> 00:06:18.280
When we look at the top 30% based on our

77
00:06:18.280 --> 00:06:19.320
predicted probabilities,

78
00:06:21.610 --> 00:06:26.580
we have targeted around 70% of,

79
00:06:26.760 --> 00:06:27.660
uh, spam emails.

80
00:06:29.650 --> 00:06:32.180
When we compare it to the random first classifier,

81
00:06:33.160 --> 00:06:37.540
we see that in the top three details. Uh, they perform, uh, the same basically,

82
00:06:38.460 --> 00:06:42.600
but, uh, the random classifier is, uh, much worse in, uh,

83
00:06:42.870 --> 00:06:43.960
ongoing deciles.

84
00:06:48.850 --> 00:06:51.100
Okay, that was it for model evaluation,

85
00:06:52.480 --> 00:06:56.620
and our last step is to save our scores to disc.

86
00:06:59.790 --> 00:07:04.170
In our next video, we will explore how to manage Jupiter Labs extensions,

87
00:07:04.390 --> 00:07:06.650
and we will try to use some of them.
