WEBVTT 00:00:01.930 --> 00:00:03.930 Before we finish up the course, 00:00:03.930 --> 00:00:07.530 let's talk a little bit about the options that we have to work with 00:00:07.530 --> 00:00:10.930 within Zeek with regards to the notifications. 00:00:10.930 --> 00:00:12.819 There are two different frameworks that we'll be working 00:00:12.819 --> 00:00:15.055 with here in this module to learn about, 00:00:15.055 --> 00:00:20.930 and they're the logging framework and the notice framework. 00:00:20.930 --> 00:00:23.748 The logging framework gives us the ability to output log 00:00:23.748 --> 00:00:27.530 files based on the events that Zeek sees and the scripts that 00:00:27.530 --> 00:00:29.930 are used to process those events. 00:00:29.930 --> 00:00:32.930 The default writing option for Zeek is to use ASCII logs, 00:00:32.930 --> 00:00:35.930 and we saw those earlier in the course. 00:00:35.930 --> 00:00:37.021 These can be great, 00:00:37.021 --> 00:00:40.930 especially if you can customize your SIM to view them in certain ways, 00:00:40.930 --> 00:00:44.930 but some want to be able to change the outputs to other formats within Zeek. 00:00:44.930 --> 00:00:48.930 And don't worry, that can happen. 00:00:48.930 --> 00:00:51.930 We can convert the ASCII logs to JSON format for export, 00:00:51.930 --> 00:00:55.930 and we also have the ability to log to SQLite databases 00:00:55.930 --> 00:00:58.930 as we discussed in an earlier module. 00:00:58.930 --> 00:01:00.930 Using the plugins that Zeek provides, 00:01:00.930 --> 00:01:04.180 we can also add additional writers so that we can fully 00:01:04.180 --> 00:01:07.930 customize the logging that's available to us. 00:01:07.930 --> 00:01:10.930 The notice framework is all about alerts. 00:01:10.930 --> 00:01:13.930 This framework helps us define, based on notice policies, 00:01:13.930 --> 00:01:16.930 what our important events are. 00:01:16.930 --> 00:01:21.486 We can write scripts to define these parameters as needed within 00:01:21.486 --> 00:01:25.930 the system and have several options to define. 00:01:25.930 --> 00:01:27.130 Depending on our setup, 00:01:27.130 --> 00:01:30.021 we can have the notice framework simply add a notice 00:01:30.021 --> 00:01:31.930 to the logging stream as a log, 00:01:31.930 --> 00:01:35.930 we can have it generate an alarm and send an email to someone, 00:01:35.930 --> 00:01:40.930 and we can even set it up to send an email to something like a paging system. 00:01:40.930 --> 00:01:45.730 What's awesome about this is that we can write our own policy hooks to expand 00:01:45.730 --> 00:01:48.930 the email notifications and add to the context of them, 00:01:48.930 --> 00:01:53.930 taking specific sections from the messages to include in the email. 00:01:53.930 --> 00:01:57.375 The ability to decide which events generate alerts and 00:01:57.375 --> 00:02:00.930 which ones don't is a great feature here. 00:02:00.930 --> 00:02:04.597 We're going to hop into the lab one last time now to see how we can 00:02:04.597 --> 00:02:09.385 convert the default ASCII logs into JSON format and show how they 00:02:09.385 --> 00:02:11.930 look with the new format within Splunk. 00:02:11.930 --> 00:02:14.657 We'll start out by looking at the log messages in 00:02:14.657 --> 00:02:18.930 Splunk within the ASCII format, like we've seen before. 00:02:18.930 --> 00:02:19.230 Then, 00:02:19.230 --> 00:02:23.430 we'll go through and configure both Splunk and Zeek to send and 00:02:23.430 --> 00:02:26.930 receive the JSON format instead of ASCII. 00:02:26.930 --> 00:02:27.216 Finally, 00:02:27.216 --> 00:02:30.730 we'll take a look at it and see how that expanded our 00:02:30.730 --> 00:02:33.294 capabilities within our SIM to view, search, 00:02:33.294 --> 00:02:36.930 and sort the data coming from the monitor. 00:02:36.930 --> 00:02:40.930 Let's hop in and check it out. 00:02:40.930 --> 00:02:45.930 I'm in my Splunk instance and I have the search pulled up from my Zeek host. 00:02:45.930 --> 00:02:46.930 In the beginning of the course, 00:02:46.930 --> 00:02:50.930 we took a look at the ASCII logs that were being sent via the 00:02:50.930 --> 00:02:53.930 universal forwarder and within the CLI. 00:02:53.930 --> 00:02:56.930 Let's look at this first event for example. 00:02:56.930 --> 00:02:59.930 To be able to really identify this information, 00:02:59.930 --> 00:03:01.930 we would need to normalize the data, 00:03:01.930 --> 00:03:05.930 as well as probably do some field extractions with it. 00:03:05.930 --> 00:03:09.430 We can tell some of the fields like the IP address that the 00:03:09.430 --> 00:03:14.930 event's for and likely the timestamps, but it's really difficult to read. 00:03:14.930 --> 00:03:18.930 I prefer the JSON format, even for just reading the logs, 00:03:18.930 --> 00:03:24.930 let alone it gives you the ability to ingest it into other apps and tools. 00:03:24.930 --> 00:03:26.430 To start the configuration, 00:03:26.430 --> 00:03:29.597 you should probably create an index within Splunk to 00:03:29.597 --> 00:03:32.930 handle just the messages from Zeek. 00:03:32.930 --> 00:03:35.930 This gives you some separation of your data. 00:03:35.930 --> 00:03:37.930 You can do this in almost any SIM out there. 00:03:37.930 --> 00:03:40.930 Splunk is just my personal favorite. 00:03:40.930 --> 00:03:42.930 If you need a refresher on how to install, 00:03:42.930 --> 00:03:45.930 configure, and use Splunk, 00:03:45.930 --> 00:03:47.708 Pluralsight has a great learning path within the 00:03:47.708 --> 00:03:53.930 platform called Splunk Fundamentals, and I have a few courses within it. 00:03:53.930 --> 00:03:56.930 This should help you out if you want to learn and use Splunk. 00:03:56.930 --> 00:04:00.748 So I'm creating an index called zeek here for it to 00:04:00.748 --> 00:04:02.930 handle my messages that I get from Zeek. 00:04:02.930 --> 00:04:06.180 I'll input this within the forwarder configuration on the Zeek 00:04:06.180 --> 00:04:09.930 virtual machine as well later on in this demo. 00:04:09.930 --> 00:04:15.930 Now I need to hop into the CLI so I can configure that now that Splunk is ready. 00:04:15.930 --> 00:04:19.530 I didn't install any apps within Splunk to ingest the JSON 00:04:19.530 --> 00:04:22.430 format since I just wanted to show you at the very least how 00:04:22.430 --> 00:04:25.930 nice it is from a readability perspective. 00:04:25.930 --> 00:04:31.930 The first thing that we should do is stop Zeek using that zeekctl stop command. 00:04:31.930 --> 00:04:36.930 Now that that's done, we can hop in and configure the parameters. 00:04:36.930 --> 00:04:39.930 I need to edit the local.zeek file that I was 00:04:39.930 --> 00:04:44.930 showing you in the previous module, so we'll use VI again to get into that. 00:04:44.930 --> 00:04:45.180 Remember, 00:04:45.180 --> 00:04:49.041 this is where all of our script loading takes place for the 00:04:49.041 --> 00:04:52.930 local site or local instance of Zeek. 00:04:52.930 --> 00:04:55.930 Since almost everything Zeek does is based on scripts, 00:04:55.930 --> 00:04:59.264 we need to add a script to tell it to output to JSON 00:04:59.264 --> 00:05:02.930 format instead of ASCII format, which it does by default. 00:05:02.930 --> 00:05:06.787 I'm going to delete some of this extra information here that I added 00:05:06.787 --> 00:05:09.930 earlier since we're not using that right now anyway. 00:05:09.930 --> 00:05:14.930 It's recommended to always use comments before you tell it which script to load, 00:05:14.930 --> 00:05:18.430 just like they do here so it's easier to keep track of what you have 00:05:18.430 --> 00:05:22.930 turned on and turned off and what each one does. 00:05:22.930 --> 00:05:24.264 So for this one, 00:05:24.264 --> 00:05:28.264 I'm going to call it Output to JSON and I'm going to use the 00:05:28.264 --> 00:05:33.930 app load command to tell it to load this json- logs.zeek file 00:05:33.930 --> 00:05:36.930 in the policy/tuning folder. 00:05:36.930 --> 00:05:38.930 This comes with the standard install of Zeek, 00:05:38.930 --> 00:05:42.930 but isn't turned on unless you tell it to do so. 00:05:42.930 --> 00:05:44.930 So I'm writing it to the file, 00:05:44.930 --> 00:05:47.930 then I'm going to start up Zeek again so that we can have it start sending 00:05:47.930 --> 00:05:51.930 some of the log messages over to Splunk in JSON format. 00:05:51.930 --> 00:05:54.680 I'm using the deploy command so I know it's going to 00:05:54.680 --> 00:05:57.930 install the new policy that we invoked. 00:05:57.930 --> 00:06:02.930 Let me hop over to the logs folder really quick to show you how it's changed. 00:06:02.930 --> 00:06:05.930 As you can see just by pulling up the dns.log file, 00:06:05.930 --> 00:06:10.930 the format is all different and it's tough to read what's going on. 00:06:10.930 --> 00:06:14.930 This format automatically tags the different fields and values to 00:06:14.930 --> 00:06:18.930 make it easier for SIMs to ingest and interpret. 00:06:18.930 --> 00:06:19.930 Hopping over to Splunk, 00:06:19.930 --> 00:06:23.330 as you can see we already have the new format coming over 00:06:23.330 --> 00:06:26.930 from Zeek and we can see the difference. 00:06:26.930 --> 00:06:30.930 Splunk took the liberty of sorting and color coding this formatted information, 00:06:30.930 --> 00:06:34.748 and it was able to automatically create fields for the messages 00:06:34.748 --> 00:06:36.930 based on the information that Zeek sent it. 00:06:36.930 --> 00:06:38.930 We'll talk about that in a minute. 00:06:38.930 --> 00:06:45.930 First, I'm going to go back over to the CLI and configure my Splunk forwarder. 00:06:45.930 --> 00:06:49.180 We need to find the right directory where it is so we can know what 00:06:49.180 --> 00:06:51.930 the configuration files we need to change are. 00:06:51.930 --> 00:06:54.930 First, we have to stop the Splunk forwarder. 00:06:54.930 --> 00:06:59.930 In a default installation, it needs to be done from the path/bin directory, 00:06:59.930 --> 00:07:03.930 and it can be done via this Splunk stop command. 00:07:03.930 --> 00:07:05.430 After it shuts down, 00:07:05.430 --> 00:07:09.930 we're free to edit the input and output files for the forwarder. 00:07:09.930 --> 00:07:14.597 The inputs.conf file for the Splunk forwarder is what 00:07:14.597 --> 00:07:16.930 tells us what to send to Splunk. 00:07:16.930 --> 00:07:17.546 For now, 00:07:17.546 --> 00:07:21.430 I'm just going to tag the host name as Zeek Sensor so we 00:07:21.430 --> 00:07:24.708 can search via that within Splunk instead of using the 00:07:24.708 --> 00:07:26.930 actual fully qualified domain name. 00:07:26.930 --> 00:07:31.930 The outputs.conf file is where we're sending the information to, 00:07:31.930 --> 00:07:34.430 and we don't really need to change any of it since 00:07:34.430 --> 00:07:36.930 we know it's working properly. 00:07:36.930 --> 00:07:41.264 I'm going to head over to the inputs.conf file one more time and define two of 00:07:41.264 --> 00:07:45.930 our Zeek logs to monitor and send to the Splunk instance. 00:07:45.930 --> 00:07:50.930 We're going to go ahead and use the conn.log and the dns.log files, 00:07:50.930 --> 00:07:53.264 and here's where we're going to tag the index that 00:07:53.264 --> 00:07:55.930 we want it to go into in Splunk. 00:07:55.930 --> 00:07:59.930 I'm also making the source type Zeek here. 00:07:59.930 --> 00:08:02.930 You can make these parameters whatever you want, 00:08:02.930 --> 00:08:06.930 and you can have it send all of the logs over to your SIM if you want to. 00:08:06.930 --> 00:08:10.930 I'm just doing these two as a quick example. 00:08:10.930 --> 00:08:14.597 The fields that you're able to modify here will highly depend on what 00:08:14.597 --> 00:08:17.930 sim and what type of forwarder you're going to use. 00:08:17.930 --> 00:08:21.930 Since I'm using the Splunk universal forwarder sending to a Splunk instance, 00:08:21.930 --> 00:08:25.930 this is what I'm choosing to modify and send. 00:08:25.930 --> 00:08:31.930 Now that that's finished, let's save and start that Splunk forwarder again. 00:08:31.930 --> 00:08:32.830 After that finishes, 00:08:32.830 --> 00:08:36.359 I'm hopping over to my Splunk instance and I'm going to change 00:08:36.359 --> 00:08:39.930 the search just for that particular index. 00:08:39.930 --> 00:08:42.930 And it looks like we already have six events that got sent over. 00:08:42.930 --> 00:08:48.130 What I love about this is that Splunk automagically knew what fields 00:08:48.130 --> 00:08:52.530 were interesting based on the formatting that Zeek provided it so we 00:08:52.530 --> 00:08:54.930 can search via the originating host and port, 00:08:54.930 --> 00:08:59.930 as well as the responding host and several other fields. 00:08:59.930 --> 00:09:03.748 I can now filter on something reaching out to Google's 00:09:03.748 --> 00:09:06.930 DNS server on port 53 very easily. 00:09:06.930 --> 00:09:09.699 Let me refresh the search so you can see one of these 00:09:09.699 --> 00:09:13.930 events in the nicely sorted format as well. 00:09:13.930 --> 00:09:16.930 And, here it is. 00:09:16.930 --> 00:09:24.430 My host 30.4 was reaching out to this 91.189 IP address over port 123, 00:09:24.430 --> 00:09:26.930 which is NTP. 00:09:26.930 --> 00:09:31.930 Normalizing this data within Splunk or any SIM can take some finesse, 00:09:31.930 --> 00:09:35.930 but this is the gist of how you can get your logs into JSON format. 00:09:35.930 --> 00:09:40.041 Adding apps to ingest it and give you nice visualizations 00:09:40.041 --> 00:09:43.930 and analysis can really help as well. 00:09:43.930 --> 00:09:44.930 This was fun. 00:09:44.930 --> 00:09:51.930 Thanks for checking this out with me.