WEBVTT - This file was automatically generated by VIMEO

0
00:00:07.720 --> 00:00:08.240
Hi.

1
00:00:08.240 --> 00:00:13.020
In this video we will learn how to plot our data using data shader.

2
00:00:17.160 --> 00:00:21.450
Data Shader is an excellent library, which allows us to plot, uh,

3
00:00:22.080 --> 00:00:26.810
even, uh, multiple billion of data points very effectively into one chart.

4
00:00:27.500 --> 00:00:30.840
We have seen that, uh, when we want to make a scatter plot,

5
00:00:31.290 --> 00:00:35.360
where are too many points plotted to gain meaningful insights might be a little

6
00:00:35.380 --> 00:00:39.760
bit tricky. Uh, now let's make a scatter plot of, uh,

7
00:00:39.760 --> 00:00:43.560
source port and, uh, length column, uh, like in this cell.

8
00:00:45.440 --> 00:00:49.920
We can do it this way. Uh, first we have to make a canvas.

9
00:00:50.340 --> 00:00:54.030
Uh, this is the place where all the plots will be plotted. Uh,

10
00:00:54.030 --> 00:00:57.070
then we specify that we want to plot the points.

11
00:00:58.890 --> 00:01:03.430
Our source is the data frame. We specify the X and the Y variable,

12
00:01:04.090 --> 00:01:08.850
and we want that. Uh, when some points overlap,

13
00:01:09.030 --> 00:01:11.010
uh, we want to calculate their counts.

14
00:01:13.150 --> 00:01:17.810
The most time consuming operation is calculating the canvas dot point object.

15
00:01:22.120 --> 00:01:23.210
Here in the next cell,

16
00:01:23.470 --> 00:01:27.930
we color the points using default color map and with equalizing histogram.

17
00:01:27.980 --> 00:01:29.210
Color mapping, uh,

18
00:01:29.320 --> 00:01:33.530
this type of color mapping attempts to use the full range of colors

19
00:01:33.840 --> 00:01:38.090
equally often to highlight both large and small features in the data.

20
00:01:40.050 --> 00:01:41.510
And, uh, here is the result.

21
00:01:46.990 --> 00:01:51.730
Now we can do the same, uh, but now let's do it, uh, on the length and, uh,

22
00:01:53.320 --> 00:01:54.700
uh, T c P length column

23
00:01:58.800 --> 00:02:02.650
here. Uh, I also used a different color map, uh, called fire.

24
00:02:03.820 --> 00:02:07.720
And here we see the relationship between these two variables. And we also see,

25
00:02:08.140 --> 00:02:13.000
uh, this area is, uh, more orange and yellow. Like that means it has, uh,

26
00:02:13.310 --> 00:02:15.520
much more higher density than the rest.

27
00:02:18.810 --> 00:02:20.780
Unfortunately, in data shader,

28
00:02:20.960 --> 00:02:24.900
we cannot plot access information or legends. Uh, but, uh,

29
00:02:24.920 --> 00:02:29.900
it is anyway very useful. In particular in plotting maps. Uh, for our use case,

30
00:02:30.000 --> 00:02:34.060
uh, the access info and legend might be added, uh, via another package,

31
00:02:35.100 --> 00:02:36.600
uh, called to use, uh,

32
00:02:36.600 --> 00:02:40.000
which has the advantage of setting plotting backend like bouquet, mud,

33
00:02:40.000 --> 00:02:42.000
plot leap or plotly or others.

34
00:02:47.580 --> 00:02:51.230
Here we create first the point element, uh,

35
00:02:51.290 --> 00:02:53.990
so hollow use novels. We want to plot a scatter plot.

36
00:02:54.990 --> 00:02:57.680
Then we specify the options for the hollow use plot.

37
00:03:00.140 --> 00:03:03.400
We want the color bar in the bottom position. Uh,

38
00:03:03.540 --> 00:03:07.400
we want the fire color map, uh, color mapping, uh,

39
00:03:07.400 --> 00:03:12.250
width and head of the plot. And then we color ize,

40
00:03:12.250 --> 00:03:16.370
which is essentially, uh, conversion the of the data points to pixels.

41
00:03:17.070 --> 00:03:20.400
And that's it, uh, on the whole data frame.

42
00:03:20.490 --> 00:03:24.200
It'll take some time to calculate, uh, usually around, uh,

43
00:03:24.250 --> 00:03:28.440
three to seven minutes maybe. Uh,

44
00:03:28.460 --> 00:03:30.280
so the charts, uh, is, uh,

45
00:03:30.280 --> 00:03:35.020
plotted via boke and is also interactive. Uh,

46
00:03:35.160 --> 00:03:39.580
you can basically zoom, can, um, hover over the chart to see,

47
00:03:40.320 --> 00:03:42.980
uh, how many points, uh, there are and so on.

48
00:03:49.730 --> 00:03:54.700
Okay, uh, this is it. Uh, for this module. Uh, we can close the dust cluster,

49
00:03:55.000 --> 00:03:57.020
uh, with the command client close.

50
00:04:01.530 --> 00:04:04.870
Now, uh, let's summarize this module. We have learned, uh,

51
00:04:04.870 --> 00:04:08.750
briefly what are the strategies on how to deal with data sets,

52
00:04:08.750 --> 00:04:11.270
which are two big, uh, to fit into our ram.

53
00:04:12.080 --> 00:04:16.020
We have learned also what are the differences and common features between, uh,

54
00:04:16.090 --> 00:04:17.340
dusk and pandas data frame.

55
00:04:18.430 --> 00:04:23.090
We learned how to start local dusk cluster on our computer and how to

56
00:04:23.090 --> 00:04:24.730
perform our calculations on it.

57
00:04:25.310 --> 00:04:28.720
We have learned also what is, uh,

58
00:04:28.740 --> 00:04:30.920
the graph and how to read it.

59
00:04:31.980 --> 00:04:36.330
We have learned how to use the D dashboard. And lastly,

60
00:04:36.470 --> 00:04:37.610
we have learned, uh,

61
00:04:37.710 --> 00:04:42.130
how to plot big data in our dust data frame using data shader and all of use.

62
00:04:44.100 --> 00:04:47.550
Next, we'll start our next module, uh, focused on machine learning.
