WEBVTT - This file was automatically generated by VIMEO

0
00:00:08.020 --> 00:00:11.320
Hi. In this video we will introduce our dataset

1
00:00:13.110 --> 00:00:16.610
for this module. I chose dataset, which I created using Shar,

2
00:00:16.740 --> 00:00:20.170
where I recorded all the packets on my wifi interface.

3
00:00:24.810 --> 00:00:28.220
It's total of, uh, 56 million of packet,

4
00:00:28.470 --> 00:00:33.180
which I recorded over a 26 hour period. It's total size. Uh,

5
00:00:33.180 --> 00:00:37.420
when you put into memory is, uh, about 13 gigabytes, uh,

6
00:00:37.420 --> 00:00:40.260
but compressed on disk. It takes around one gigabyte.

7
00:00:42.240 --> 00:00:43.420
Uh, please, uh,

8
00:00:43.780 --> 00:00:47.980
download it from provided link and extract it into the data folder.

9
00:00:50.840 --> 00:00:53.060
Uh, you will see 500 packet files.

10
00:00:53.310 --> 00:00:57.500
Originally I had only 142 C S V files, uh,

11
00:00:57.600 --> 00:00:58.780
output from Wireshark,

12
00:00:59.240 --> 00:01:03.540
but I had to process it a little bit like changing the IP address. And, uh,

13
00:01:03.940 --> 00:01:06.980
I also added the ID of the captured file as well.

14
00:01:08.420 --> 00:01:10.980
I also them to have 500 smaller files.

15
00:01:13.840 --> 00:01:18.230
I did this, uh, processing in, uh, this file, uh,

16
00:01:18.230 --> 00:01:23.090
module zero eight prep notebook. You can explore it, uh, if you want,

17
00:01:24.350 --> 00:01:29.010
uh, after I finished this lecture. Uh, but, uh, the main notebook,

18
00:01:29.010 --> 00:01:31.290
which we will use is, uh, this one

19
00:01:33.680 --> 00:01:34.513
we can open it.

20
00:01:40.090 --> 00:01:42.990
The dataset contains, uh, 14 columns, uh,

21
00:01:42.990 --> 00:01:47.270
which have the basic information about the packet, like source and destination,

22
00:01:47.450 --> 00:01:51.110
IP address, and port, uh, logged in, buy timestamp, and so on.

23
00:01:52.840 --> 00:01:55.540
Uh, one thought, the parking file format, uh,

24
00:01:55.540 --> 00:01:59.900
this format is much more efficient in terms of storage and in, uh,

25
00:02:00.030 --> 00:02:02.300
input and output operations than C S V.

26
00:02:02.760 --> 00:02:05.140
So the reading into dusk will be much faster.

27
00:02:08.070 --> 00:02:12.490
In our next video, we'll start to work with dusk and explain the dusk graph.
