WEBVTT - This file was automatically generated by VIMEO

0
00:00:07.970 --> 00:00:12.230
Hi. In this video we will learn how to do B variate analysis,

1
00:00:12.480 --> 00:00:16.110
which is the analysis of a relationship between two variables.

2
00:00:18.310 --> 00:00:20.250
We have basically three options.

3
00:00:20.600 --> 00:00:23.450
Analyze relationship of two categorical variables,

4
00:00:23.670 --> 00:00:28.610
two numeric variables or relationship between one categorical and one numeric

5
00:00:28.850 --> 00:00:32.200
variable. For all of these options, there are multiple methods.

6
00:00:35.340 --> 00:00:38.650
First, let's analyze a relationship between two categorical variables.

7
00:00:38.870 --> 00:00:41.010
We can make a group bar chart for this, uh,

8
00:00:41.010 --> 00:00:45.450
where we first calculate the counts for each category combination with the PO

9
00:00:45.450 --> 00:00:47.600
table like here, uh,

10
00:00:47.650 --> 00:00:52.200
where I analyzed the relationship between attack variable and protocol type.

11
00:00:53.040 --> 00:00:57.580
Uh, do not be confused here that I use the duration variable as values.

12
00:00:58.410 --> 00:01:01.710
It basically doesn't matter which variable you use here because we are

13
00:01:01.710 --> 00:01:02.990
calculating only counts.

14
00:01:06.320 --> 00:01:08.860
We see here that the Smurf attack dominates, of course,

15
00:01:08.860 --> 00:01:11.780
because it has the highest number of observations,

16
00:01:12.120 --> 00:01:14.700
but they are only via I C M P protocol.

17
00:01:18.120 --> 00:01:23.080
We see that the Neptune attack uses only T C P protocol and normal connection

18
00:01:23.380 --> 00:01:28.000
use sees, uh, mostly T C P protocol, the u d DP and I C M P,

19
00:01:29.320 --> 00:01:32.020
uh, to have a better view of other attacks. Uh,

20
00:01:32.020 --> 00:01:34.580
we could filter out these three attack categories,

21
00:01:34.800 --> 00:01:39.500
or we can do stacked relative bar chart where each bar sums to one,

22
00:01:39.720 --> 00:01:44.190
uh, same as we did before. This is the calculation,

23
00:01:45.980 --> 00:01:47.200
and here is the bar chart.

24
00:01:52.730 --> 00:01:53.020
Here.

25
00:01:53.020 --> 00:01:57.860
In this chart is an example of a normal tuck bar chart where we can see a

26
00:01:57.860 --> 00:02:01.450
relationship between flag and, uh, protocol variable

27
00:02:10.720 --> 00:02:11.070
Here.

28
00:02:11.070 --> 00:02:15.570
In the next example is a chart describing relationship of categorical variable

29
00:02:15.570 --> 00:02:18.290
attack with numeric variable count. Uh,

30
00:02:18.430 --> 00:02:21.130
but we bend the numeric variable to beans.

31
00:02:21.990 --> 00:02:26.330
Now we have basically a categorical variable, uh, to be more precise. Uh,

32
00:02:26.330 --> 00:02:29.730
we have an ordinal variable because there is an order in this beans

33
00:02:33.240 --> 00:02:37.730
here. We basically plotted the distribution of count variable per attack. Now,

34
00:02:37.730 --> 00:02:42.070
we are able to see very interesting patterns here, uh,

35
00:02:42.180 --> 00:02:47.110
that for example, the normal connections, uh, they dominate in the lower values,

36
00:02:47.490 --> 00:02:51.490
uh, and with higher values, uh, of the count variable,

37
00:02:51.670 --> 00:02:56.250
we see higher proportions of the teardrop attack. Here's this one

38
00:03:00.360 --> 00:03:02.810
also here. The Neptune attack, uh,

39
00:03:02.810 --> 00:03:07.010
starts to dominate until almost 290

40
00:03:08.100 --> 00:03:09.450
value of the count variable

41
00:03:12.760 --> 00:03:15.380
values of count variable about, uh,

42
00:03:16.950 --> 00:03:21.160
420, uh, is completely covered by the smurf attack,

43
00:03:21.490 --> 00:03:25.560
which has also very high proportions of count values between uh,

44
00:03:26.090 --> 00:03:28.520
300 and 400.

45
00:03:31.070 --> 00:03:34.720
Another way how to interpret this chart is that, uh,

46
00:03:34.750 --> 00:03:39.200
when you randomly sample or when you have data about new unseen

47
00:03:39.200 --> 00:03:43.560
connection and it has value of count variable bigger than 430,

48
00:03:44.300 --> 00:03:46.960
you can be a hundred percent sure it is a smurf attack.

49
00:03:48.190 --> 00:03:52.920
Another interpretation example might be that if you see a new connection

50
00:03:53.070 --> 00:03:57.400
with value of count variable of, uh, 80, for example, here,

51
00:03:58.750 --> 00:04:03.740
there is about a 70% chance it is a Neptune attack,

52
00:04:04.110 --> 00:04:05.900
maybe around 10%. Ss shes,

53
00:04:05.920 --> 00:04:10.050
it is a normal connection and around 15% s shes,

54
00:04:10.230 --> 00:04:11.810
it is a teardrop attack.

55
00:04:18.370 --> 00:04:23.140
Here is another example of plotting the distribution of numeric variable, uh,

56
00:04:23.160 --> 00:04:27.140
per categorical variable. We can use the box plot for this.

57
00:04:27.840 --> 00:04:32.820
We see here that most of the distribution, uh, of count variable in uh,

58
00:04:33.620 --> 00:04:37.380
I C M P protocol is squeezed around the value of 500.

59
00:04:38.270 --> 00:04:41.090
We have learned already that it is basically the smurf attack.

60
00:04:42.020 --> 00:04:46.280
Do not be confused with this, uh, think blank line. Uh,

61
00:04:46.280 --> 00:04:50.040
these are basically the outliers plotted or on top of each other.

62
00:04:54.080 --> 00:04:58.570
Another example of group to box plot is here where we see the

63
00:04:58.570 --> 00:05:02.490
relationship between attack and count variable from another perspective,

64
00:05:03.610 --> 00:05:06.970
although we cannot conclude from this, the probabilities, uh,

65
00:05:06.970 --> 00:05:11.610
we can see interesting patterns as well. Uh, for example

66
00:05:14.370 --> 00:05:18.500
here the Saturn attack has distribution of count variable squeeze around, uh,

67
00:05:18.500 --> 00:05:21.220
500, uh, same as the smurf attack.

68
00:05:26.450 --> 00:05:30.630
We can check the relationship between attack and service count variable. Uh,

69
00:05:30.770 --> 00:05:35.750
we see that the Smurf attack has also the distribution squeeze around

70
00:05:35.780 --> 00:05:39.720
500, but not the, uh, Satan attack.

71
00:05:43.640 --> 00:05:48.570
Another very useful way to plot categorical variable versus numeric is, uh,

72
00:05:48.760 --> 00:05:51.810
faceted the histogram where the x axis,

73
00:05:51.930 --> 00:05:55.490
y axis or both are shared. Uh, here,

74
00:05:55.670 --> 00:05:59.200
the service count variable on the axis is shared. Uh,

75
00:05:59.200 --> 00:06:02.640
that means all charts have, uh, x axis on the same scale,

76
00:06:02.980 --> 00:06:05.960
and we can compare the distribution in form of a histogram.

77
00:06:11.270 --> 00:06:13.640
Here I wanted to see the distribution of, uh,

78
00:06:13.880 --> 00:06:17.080
duration variable per attack to see if there are any interesting patterns.

79
00:06:19.270 --> 00:06:23.970
It is better that we zoom in. Uh, we can use the island parameter for this.

80
00:06:32.060 --> 00:06:36.920
We see that the pore sweep attack has, uh, very interesting patterns. Uh,

81
00:06:37.860 --> 00:06:42.110
you see that, uh, the attacks, uh, are equally spaced.

82
00:06:53.340 --> 00:06:57.400
Now let's check how to analyze, uh, relationship between two numeric variables.

83
00:06:58.220 --> 00:07:00.800
The most common chart to do this is the scatter plot,

84
00:07:01.020 --> 00:07:05.680
but when you have many points, uh, there is a problem that they are overlapping,

85
00:07:06.340 --> 00:07:10.640
we can solve it partially with the alpha parameter. Uh, but anyway, in our case,

86
00:07:10.650 --> 00:07:13.000
there are half a million points plotted on this chart,

87
00:07:13.420 --> 00:07:16.240
and it is very hard to identify the dense areas.

88
00:07:17.190 --> 00:07:20.490
We do not know if this is the most dense area or this one,

89
00:07:20.790 --> 00:07:23.170
or this one or this one. For example,

90
00:07:26.880 --> 00:07:31.700
we have seen that, uh, we can bin numeric variable into bins. Uh, so now, uh,

91
00:07:31.720 --> 00:07:34.860
we see more information. What is their relationship? We can

92
00:07:37.840 --> 00:07:42.740
plot also a line chart with the median and the mean, uh,

93
00:07:42.740 --> 00:07:47.520
to better see some patterns. Uh, we see here a big jump in around,

94
00:07:47.740 --> 00:07:52.640
uh, value of 300 of the count variable, but as we already know,

95
00:07:52.770 --> 00:07:56.160
there are multiple populations depending on the attack. Uh,

96
00:07:56.160 --> 00:07:59.640
so discharge provides you some information, but not all of it.

97
00:08:03.430 --> 00:08:07.370
In our next video, we will learn how to do multivariate analysis, uh,

98
00:08:07.370 --> 00:08:10.970
which is the analysis of relationship between more than two variables.
