WEBVTT

00:00:01.840 --> 00:00:04.560
All right, now that we have Logstash configured,

00:00:05.040 --> 00:00:08.460
we need to finish up and create the data source in Arkime,

00:00:09.040 --> 00:00:13.740
then we'll validate it and check out some of our data. Now is the

00:00:13.740 --> 00:00:16.960
time to jump into the lab and get configuring.

00:00:16.960 --> 00:00:23.430
And so here we are in our Malcolm dashboards, and so I have two tabs pulled up.

00:00:23.440 --> 00:00:26.510
This tab that we're on right now is our Kibana dashboard.

00:00:26.510 --> 00:00:31.950
It's our welcome dashboard when we first log into the Kibana piece of Malcolm.

00:00:32.340 --> 00:00:37.480
So we have this nice dashboard that gives us the log counts that we have,

00:00:37.490 --> 00:00:41.250
it gives us the different types, the protocols that we see in the environment,

00:00:41.740 --> 00:00:43.860
service by destination country,

00:00:45.040 --> 00:00:47.680
and then we can go through and see all of the logs,

00:00:47.680 --> 00:00:49.080
for example, down here.

00:00:49.080 --> 00:00:52.300
We can see it in either table format or JSON format.

00:00:52.310 --> 00:00:55.370
If you're familiar with how Kibana works then you're familiar with the

00:00:55.370 --> 00:00:57.950
visualizations and dashboards that we can get with it.

00:00:59.340 --> 00:01:03.600
We also have all of these different tabs from the Zeek side of things in here.

00:01:03.610 --> 00:01:05.620
We can look at the common protocols, click them,

00:01:05.620 --> 00:01:08.880
we can go look at specific logs, so like the weird logs,

00:01:08.890 --> 00:01:09.960
for example.

00:01:10.540 --> 00:01:13.460
I know, I always default to the weird logs for some reason.

00:01:13.460 --> 00:01:16.710
But we can see a lot of the information about this stuff.

00:01:16.710 --> 00:01:21.170
And we can also go to the Discover piece and actually

00:01:21.170 --> 00:01:26.410
look at the logs that are coming in, so we have the one index pattern here,

00:01:26.420 --> 00:01:30.770
and then we see all the different logs that we have coming in from Zeek.

00:01:30.770 --> 00:01:32.070
And as you can see,

00:01:32.070 --> 00:01:34.960
it's using Filebeat as the tool that's sending the logs for us.

00:01:36.240 --> 00:01:39.450
Now I'm going to hop over to the CLI forward again,

00:01:40.540 --> 00:01:43.990
and I'm going to go ahead and navigate to our Moloch directory,

00:01:43.990 --> 00:01:48.030
or our Arkime directory, and, specifically,

00:01:48.030 --> 00:01:49.400
I'm going into the WISE directory.

00:01:49.400 --> 00:01:53.240
Now this is where we can go to actually view the views that we

00:01:53.240 --> 00:01:56.860
create that show the mappings for Zeek to Arkime.

00:01:57.740 --> 00:02:01.850
What I want to look at here is this source.zeeklogs.js file.

00:02:02.640 --> 00:02:06.970
And as you can see, it's the WISE data source definitions for the Zeek logs.

00:02:06.980 --> 00:02:10.610
Now if you're doing this integration on your own using your own

00:02:10.610 --> 00:02:13.680
instance of Arkime and Zeek and the ELK Stack,

00:02:13.690 --> 00:02:18.200
then you're going to need to add this WISE data source to be able to

00:02:18.210 --> 00:02:21.450
view the Zeek logs in Arkime's dashboard properly.

00:02:22.540 --> 00:02:26.230
In here we are doing similar things to what we did in

00:02:26.240 --> 00:02:28.870
the Logstash pipeline configurations.

00:02:28.870 --> 00:02:30.390
We're defining different fields,

00:02:30.390 --> 00:02:34.530
but in this case we're calling different areas so Arkime

00:02:34.530 --> 00:02:36.020
can map the fields to different things.

00:02:36.020 --> 00:02:40.810
So we have this file right here, the source.zeeklogs.js file,

00:02:40.810 --> 00:02:44.760
Arkime's config.ini file has the field definitions there,

00:02:44.840 --> 00:02:47.390
and then also Kibana's Zeek template has them.

00:02:47.390 --> 00:02:49.560
And so scrolling through here,

00:02:49.570 --> 00:02:51.980
there's all of these different field definitions in here.

00:02:51.990 --> 00:02:57.270
And so these all are defined in this WISE data source so that Arkime

00:02:57.270 --> 00:03:00.070
can actually use them and show them in the dashboard.

00:03:00.080 --> 00:03:03.380
So that's essentially what we mean by creating the WISE

00:03:03.390 --> 00:03:05.850
view, or the WISE data source for us,

00:03:05.860 --> 00:03:11.040
is defining all of the fields that Arkime will pull and then mapping them to

00:03:11.040 --> 00:03:14.480
the different tools and stuff like that like we saw above.

00:03:14.480 --> 00:03:15.960
And also,

00:03:15.970 --> 00:03:19.400
what's really cool about Arkime is that we can add right‑click pivoting to the

00:03:19.400 --> 00:03:24.820
dashboard in Arkime to be able to pivot to things like searching IANA for

00:03:24.820 --> 00:03:28.650
services and searching VirusTotal and so on and so forth.

00:03:29.640 --> 00:03:33.860
So we'll demonstrate this capability when we get up into the GUI to view it.

00:03:35.040 --> 00:03:38.160
We're also using this data source to define the different

00:03:38.160 --> 00:03:40.400
arrays and connection information as well.

00:03:40.400 --> 00:03:44.700
And as you can see here, we have a lot of the Zeek field values here,

00:03:44.700 --> 00:03:47.410
and you see it kind of maps them to the other data

00:03:47.410 --> 00:03:49.950
sources or the other field definitions.

00:03:49.950 --> 00:03:51.640
And that's it.

00:03:51.650 --> 00:03:56.000
So if you're familiar with Arkime and how to create these WISE data sources,

00:03:56.010 --> 00:03:59.160
then you should be good to go with configuring this yourself.

00:04:00.540 --> 00:04:04.760
But essentially, you can pull templates for WISE data sources from Arkime.

00:04:04.760 --> 00:04:08.490
You can go in and download Malcolm and use this prebuilt template

00:04:08.490 --> 00:04:11.040
and just make modifications if you need to.

00:04:11.040 --> 00:04:14.670
And so now let's go up to the web and hop over to

00:04:14.670 --> 00:04:17.320
Arkime and view the information.

00:04:17.320 --> 00:04:21.420
Before we actually go into these sessions and start diving into them,

00:04:21.430 --> 00:04:24.180
if we go over to the Settings and then Views column,

00:04:24.180 --> 00:04:27.550
if you look here, we have these views that are saved,

00:04:27.550 --> 00:04:29.260
different logs within Arkime.

00:04:30.740 --> 00:04:35.100
Now the most common Zeek log that we usually view in Arkime and/or

00:04:35.100 --> 00:04:38.160
map the data to is going to be the connection log.

00:04:38.160 --> 00:04:39.670
Because if you think about it,

00:04:39.670 --> 00:04:42.890
connection log is tracking all of the connections that we see on the network,

00:04:42.900 --> 00:04:45.620
Arkime's capturing the packets on the network,

00:04:45.620 --> 00:04:50.330
so it makes sense that Zeek connection logs would fill in the blanks for

00:04:50.340 --> 00:04:53.100
full packet capture if it's not available in that location.

00:04:53.110 --> 00:04:54.950
So now we're going to go over to the sessions,

00:04:55.440 --> 00:04:59.200
and if you're familiar with Arkime you can experiment with this on your own,

00:04:59.200 --> 00:05:03.440
but I'm just going to filter on protocols == http to

00:05:03.440 --> 00:05:05.390
pull up any information that I can.

00:05:05.390 --> 00:05:07.460
As you can see, nothing's there.

00:05:07.460 --> 00:05:09.860
So let's go ahead and try DNS.

00:05:11.440 --> 00:05:16.520
How about protocols == icmp? Let's go check out some ICMP packets.

00:05:16.520 --> 00:05:19.360
And my spelling's failing me on this one.

00:05:19.370 --> 00:05:20.120
There we go.

00:05:20.130 --> 00:05:22.520
All right, so we have some ICMP packets.

00:05:22.520 --> 00:05:24.110
And if you look at this,

00:05:24.120 --> 00:05:28.050
what's really cool about this is we can see the data from the Zeek logs,

00:05:28.060 --> 00:05:30.960
and we can see the data coming in from Arkime.

00:05:31.340 --> 00:05:34.250
So right here we have a pcap from Arkime,

00:05:34.250 --> 00:05:37.140
but also right here we have the Zeek connection log.

00:05:37.140 --> 00:05:38.660
They're showing similar information.

00:05:39.640 --> 00:05:42.530
We can search by the community IDs here as well,

00:05:42.540 --> 00:05:47.030
if we wanted to, and tie them to Arkime's root ID.

00:05:47.030 --> 00:05:50.900
Let's see if that pulls up anything for us, and as you can see it did.

00:05:50.900 --> 00:05:54.860
And so here what we just did was mapped Arkime's

00:05:54.860 --> 00:05:57.750
session to Zeek's connection log.

00:05:57.750 --> 00:06:00.890
And we can download the entire pcap if we wanted to,

00:06:00.890 --> 00:06:04.600
we can download the source or destination versions of the pcap,

00:06:04.600 --> 00:06:08.830
we can go look at all of the sessions, so there's a lot of power here,

00:06:08.840 --> 00:06:12.300
and we were able to view the data in multiple ways.

00:06:12.300 --> 00:06:16.210
A. And in Arkime what's cool is it has a lot of the information about a lot of

00:06:16.210 --> 00:06:19.500
the fields and information that we would normally see in Zeek. We see the

00:06:19.500 --> 00:06:22.350
source and destination and the number of packets,

00:06:22.350 --> 00:06:24.550
the ports, and the IP addresses.

00:06:24.610 --> 00:06:28.460
Obviously, ICMP's not really a good example for ports and stuff like

00:06:28.460 --> 00:06:32.590
that. And we also have some of that enriched information in here so we

00:06:32.590 --> 00:06:38.280
have the OUI information from our enrichment pipelines showing that

00:06:38.290 --> 00:06:43.750
it's VMware. Now what we can do is take this information and pivot

00:06:43.750 --> 00:06:45.250
over to Kibana using it.

00:06:45.640 --> 00:06:50.880
So let's take this IP address, for example. If we click on this menu, and

00:06:50.880 --> 00:06:55.050
remember how we saw some of those right‑click pivots for VirusTotal and

00:06:55.050 --> 00:06:59.620
Kibana and reverse DNS lookups and stuff? Well, we can look at this host

00:06:59.630 --> 00:07:04.410
in Kibana just by clicking the Kibana source IP, and it's going to send

00:07:04.410 --> 00:07:08.770
us straight there and pull up a log search for that source type. And as

00:07:08.770 --> 00:07:12.270
you can see, we have six hits. So we can look at the information in

00:07:12.280 --> 00:07:13.010
Arkime,

00:07:13.010 --> 00:07:17.830
correlate between the Zeek logs and the Arkime data, and then pivot

00:07:17.840 --> 00:07:20.950
over to Kibana to view the full logs if we wanted to.

00:07:21.840 --> 00:07:25.960
And so that's the power of integrating Zeek with Arkime, and

00:07:25.960 --> 00:07:28.640
that gives us a lot more new capabilities.

00:07:28.640 --> 00:07:32.810
So we could create automatic correlation with the community IDs and root

00:07:32.810 --> 00:07:37.300
IDs between Zeek and Arkime. We could use some of that enrichment

00:07:37.300 --> 00:07:43.070
information like the OUIs to help us alert on specific communication

00:07:43.070 --> 00:07:45.550
mechanisms that we may or may not want to see.

00:07:47.040 --> 00:07:47.630
All right,

00:07:47.640 --> 00:07:50.680
this was a great module. We talked about a lot, and we

00:07:50.680 --> 00:07:53.760
configured a lot more throughout the module. We were able

00:07:53.760 --> 00:07:56.160
to ingest Zeek logs into ELK,

00:07:56.540 --> 00:08:01.160
parse them and enrich them with Logstash in Malcolm, and

00:08:01.160 --> 00:08:05.450
map them to Arkime's schema for correlation inside of the

00:08:05.450 --> 00:08:07.460
tool using Malcolm as well.

00:08:08.340 --> 00:08:11.810
I hope you learned a ton in this module and so far in this

00:08:11.810 --> 00:08:15.150
course. We have a few more modules to go,

00:08:15.350 --> 00:08:20.510
which means we have more integrations to learn about for Zeek. In the next

00:08:20.510 --> 00:08:26.270
module, we're going to integrate Zeek with another tool called RockNSM, or, more

00:08:26.270 --> 00:08:32.000
specifically, we're going to show Zeek's integration in RockNSM and also learn

00:08:32.000 --> 00:08:37.380
about the file carving capabilities within Zeek. Take the time to practice and

00:08:37.380 --> 00:08:42.039
explore what you've learned so far in this course, and when you're ready, join

00:08:42.039 --> 00:08:43.549
me in the next module.
