WEBVTT - This file was automatically generated by VIMEO

0
00:00:07.700 --> 00:00:12.480
Hi. In this video we will learn how to manage Jupiter lab extensions

1
00:00:13.140 --> 00:00:17.480
and we will try two of them, variable inspector and a link extension.

2
00:00:18.150 --> 00:00:18.983
When

3
00:00:22.820 --> 00:00:26.580
we go to the extension step, we can click enable.

4
00:00:27.920 --> 00:00:32.700
We will see what extensions we have installed and uh, order, uh, which,

5
00:00:32.800 --> 00:00:35.950
uh, we can, uh, install and discover.

6
00:00:38.360 --> 00:00:41.340
For example, I have already installed plot Extension,

7
00:00:42.340 --> 00:00:43.980
extension and others.

8
00:00:46.500 --> 00:00:49.560
And let's look for the variable inspector.

9
00:00:59.370 --> 00:01:01.700
Feel free to explore other extensions,

10
00:01:02.320 --> 00:01:06.860
but be aware that these are third party extensions and you might encounter

11
00:01:06.860 --> 00:01:11.780
issues while using them. Uh, some extensions might not even be listed here,

12
00:01:11.880 --> 00:01:14.420
but had to be installed like a regular Python package.

13
00:01:16.300 --> 00:01:19.920
Now let's install this variable inspector. The first one.

14
00:01:27.080 --> 00:01:30.260
Now we see that a build is needed to include the latest changes.

15
00:01:45.920 --> 00:01:50.790
Click, rebuild and wait for a few minutes, 10 minutes or so. Uh,

16
00:01:50.790 --> 00:01:54.270
this will take some time and the Jupiter lab will seem like, uh,

17
00:01:54.270 --> 00:01:58.320
nothing is happening, but be patient. Uh,

18
00:01:58.320 --> 00:02:00.560
we can monitor the process, what is happening

19
00:02:03.400 --> 00:02:04.690
with the top command.

20
00:03:00.380 --> 00:03:04.430
Okay, rebuilding Jupyter Lab, uh, has been done. Uh,

21
00:03:04.430 --> 00:03:05.590
click save and remote.

22
00:03:15.270 --> 00:03:16.660
Let's go to our notebook.

23
00:03:22.460 --> 00:03:26.770
Right click on the notebook. And, uh, here is Open Variable Inspector.

24
00:03:33.600 --> 00:03:37.290
Let's arrange it this way. And you see that, uh,

25
00:03:37.440 --> 00:03:39.250
when you around the notebook,

26
00:03:40.050 --> 00:03:44.890
we'll able to see all the variables created in the variable inspector.

27
00:03:49.680 --> 00:03:53.980
We see their name, type tape and also content.

28
00:03:55.070 --> 00:03:59.210
And you can also easily delete the variables by clicking on the cross.

29
00:04:10.300 --> 00:04:14.800
Now, uh, let's try the other extension, a link from Mackin Rocks.

30
00:04:15.890 --> 00:04:20.330
I found this extension just recently and I was waiting for this kind of

31
00:04:20.330 --> 00:04:23.010
extension for a very long time. Uh,

32
00:04:23.070 --> 00:04:26.450
so I have to share it with you because it adds a lot of benefits.

33
00:04:27.350 --> 00:04:31.940
Maybe you have already noticed that the notebooks can be sometimes very messy.

34
00:04:32.930 --> 00:04:36.120
Table of contents help us to structure the notebook,

35
00:04:36.900 --> 00:04:41.120
but when you are thinking about the codependency with this,

36
00:04:41.280 --> 00:04:45.440
I mean the dependencies between code cells in the notebook, it doesn't help

37
00:04:47.200 --> 00:04:48.800
frequently when working with the notebook,

38
00:04:49.260 --> 00:04:53.030
you do an extensive analysis and after that,

39
00:04:53.210 --> 00:04:57.110
you build the model and try also here different approaches.

40
00:04:58.200 --> 00:05:03.180
Then you continue fitting the model on the winning approach. And,

41
00:05:03.280 --> 00:05:06.940
uh, when designing the notebook this way you have to know, uh,

42
00:05:06.940 --> 00:05:11.180
which parts of the notebook must be executed sequentially and other,

43
00:05:11.470 --> 00:05:16.060
which can be executed in parallel and do not depend on other cells.

44
00:05:17.370 --> 00:05:19.790
In this help, exactly this extension,

45
00:05:20.900 --> 00:05:25.600
it allows to create code pipelines where a pipeline consists of

46
00:05:26.000 --> 00:05:27.760
multiple cells with different branches,

47
00:05:28.780 --> 00:05:32.750
different branches of the pipeline can be executed in parallel.

48
00:05:34.200 --> 00:05:37.510
Steps of the pipeline can be cached, saved and shared.

49
00:05:38.990 --> 00:05:43.870
A whole pipeline or parts of the pipeline can be executed on remote servers.

50
00:05:44.850 --> 00:05:48.110
And also you can set up parameters and optimize them.

51
00:05:48.110 --> 00:05:52.850
Using inbuilt optimizer, you can do everything,

52
00:05:52.850 --> 00:05:56.890
which I mentioned and nicely, visually or automatically create a diagram.

53
00:05:58.450 --> 00:06:02.670
And also you can access, modify, and create these pipelines,

54
00:06:02.770 --> 00:06:06.150
not only directly in the notebook, but from Python code as well.

55
00:06:07.930 --> 00:06:11.740
I'll show you the basics, how it works in our notebook. Uh,

56
00:06:11.740 --> 00:06:13.820
first let's install it. We

57
00:06:17.410 --> 00:06:19.930
have the instructions here, how to install it,

58
00:06:20.190 --> 00:06:24.690
and we see it has to be installed as a regular Python package via peep.

59
00:06:27.630 --> 00:06:31.060
Let's go and install it into our base environment.

60
00:06:48.020 --> 00:06:52.390
Okay, the installation was successful, but we see here, uh,

61
00:06:52.390 --> 00:06:57.030
that we have some dependency conflicts. Uh, but let's try now our extension.

62
00:06:57.150 --> 00:07:01.130
Anyway, our base environment is really big. Uh,

63
00:07:01.310 --> 00:07:05.450
so if you experience issues in other features, uh, like, uh,

64
00:07:05.560 --> 00:07:09.250
when using widgets, exporting the notebook, using voila orders,

65
00:07:10.410 --> 00:07:14.290
I recommend you to use other Jupyter image where the base environment is more

66
00:07:14.290 --> 00:07:17.650
lightweight or create your own Docker image by yourself.

67
00:07:17.790 --> 00:07:22.410
You will have the skills or install Con and Python directly into our machine and

68
00:07:22.410 --> 00:07:25.450
install packages, which you want and need without Docker.

69
00:07:27.810 --> 00:07:29.510
Now let's restart the container.

70
00:07:40.230 --> 00:07:42.390
I have restarted it on another screen,

71
00:07:44.160 --> 00:07:46.460
but now it should, uh, run.

72
00:08:02.410 --> 00:08:06.940
Okay, now it's recommended to rebuild the Jupyter app again. Uh,

73
00:08:07.080 --> 00:08:08.620
so let's click build

74
00:08:11.820 --> 00:08:12.653
and let's wait.

75
00:08:44.790 --> 00:08:49.360
Okay, this being a little bit shorter now. Click seven, reload.

76
00:09:00.050 --> 00:09:04.730
I have already approved key, uh, so I will just input it here.

77
00:09:22.970 --> 00:09:27.590
Now let's open our notebook in the module five folder and click on the lean

78
00:09:27.590 --> 00:09:28.423
extension here.

79
00:09:31.720 --> 00:09:35.220
And now let's create our pipeline. Uh, it's uh, very easy.

80
00:09:35.560 --> 00:09:39.220
You see that each of the cell has a plus button here,

81
00:09:40.890 --> 00:09:42.470
and when we click on the plus button,

82
00:09:42.930 --> 00:09:46.950
we can specify the component name and its parents last name.

83
00:09:46.980 --> 00:09:48.070
This one packages,

84
00:09:53.070 --> 00:09:53.903
it has no parents.

85
00:10:00.150 --> 00:10:02.320
This one might be debugger test death

86
00:10:08.020 --> 00:10:09.070
also without any parents.

87
00:10:15.690 --> 00:10:19.660
This one has, uh, the parent, uh, the debugger test, test

88
00:10:28.650 --> 00:10:31.940
Pre-process data has the parent, uh, packages,

89
00:10:47.510 --> 00:10:49.210
target class distribution

90
00:10:55.440 --> 00:10:57.580
has the parent, uh, pre-processing data.

91
00:11:02.870 --> 00:11:03.800
This one as well,

92
00:11:16.620 --> 00:11:17.960
the Plotly bar chart.

93
00:11:27.200 --> 00:11:30.500
It has parent the target class distribution cell

94
00:11:36.690 --> 00:11:37.830
and test split.

95
00:11:44.300 --> 00:11:46.520
It has parent, uh, reprocessing data,

96
00:11:53.080 --> 00:11:57.520
logistic regression as parents. Um,

97
00:11:59.460 --> 00:12:00.450
trend test split

98
00:12:06.190 --> 00:12:07.080
confusion matrix,

99
00:12:17.320 --> 00:12:18.630
cumulative gain plot.

100
00:12:28.660 --> 00:12:31.450
We'll do the same for the random forest classifier.

101
00:13:16.540 --> 00:13:17.540
I have to edit this one.

102
00:13:26.270 --> 00:13:29.310
Fusion matrix, uh, cumulative gain plot.

103
00:13:39.910 --> 00:13:42.250
Its parents are only R F D.

104
00:13:46.140 --> 00:13:50.660
And finally, savings scores to disk. It has parents,

105
00:13:51.040 --> 00:13:53.340
uh, all these, uh, parcel cells.

106
00:14:18.770 --> 00:14:22.260
Okay, now our pipeline is ready. Uh, let's try it.

107
00:14:23.570 --> 00:14:25.110
We can run all components.

108
00:14:38.370 --> 00:14:40.740
Perfect. Everything is working correctly.

109
00:14:43.150 --> 00:14:47.190
We can now run only part of the pipeline. For example,

110
00:14:47.340 --> 00:14:48.510
this branch to

111
00:14:53.070 --> 00:14:55.370
we see that only this branch has been executed.

112
00:15:01.080 --> 00:15:03.740
We can set up the execution in parallel.

113
00:15:12.150 --> 00:15:16.570
In most of the cases, uh, the execution of the components will run faster.

114
00:15:23.380 --> 00:15:27.740
You'll see that they are executed, uh, in parallel. Now,

115
00:15:34.080 --> 00:15:37.140
we can also cache specific components.

116
00:15:37.890 --> 00:15:42.420
This is very useful because, uh, some pre-processing steps or, um,

117
00:15:42.740 --> 00:15:43.573
training of the model.

118
00:15:44.500 --> 00:15:49.270
This might be very useful because some pre-processing steps or fitting

119
00:15:49.330 --> 00:15:51.470
the model may take, uh,

120
00:15:51.490 --> 00:15:56.260
too much time when you right click on the logistical regression,

121
00:15:57.000 --> 00:15:58.380
enable component caching

122
00:16:00.180 --> 00:16:02.320
and also the random first classifier.

123
00:16:13.030 --> 00:16:14.650
Now the cache is being filled,

124
00:16:17.540 --> 00:16:19.120
and when we run it next time,

125
00:16:19.330 --> 00:16:22.800
it'll be much faster because only this cells will be executed.

126
00:16:31.070 --> 00:16:31.903
Yes.

127
00:16:36.250 --> 00:16:39.100
Okay, this is it. I think you have the idea.

128
00:16:39.570 --> 00:16:41.860
What is this extensions capable of.

129
00:16:42.540 --> 00:16:46.870
I highly encourage to you to explore it more. In our next video,

130
00:16:47.250 --> 00:16:50.750
we will see how Jupyter Lab Collaborative editing works.
