Before we finish up the course,
let's talk a little bit about the options that we have to work with
within Zeek with regards to the notifications.
There are two different frameworks that we'll be working
with here in this module to learn about,
and they're the logging framework and the notice framework.
The logging framework gives us the ability to output log
files based on the events that Zeek sees and the scripts that
are used to process those events.
The default writing option for Zeek is to use ASCII logs,
and we saw those earlier in the course.
These can be great,
especially if you can customize your SIM to view them in certain ways,
but some want to be able to change the outputs to other formats within Zeek.
And don't worry, that can happen.
We can convert the ASCII logs to JSON format for export,
and we also have the ability to log to SQLite databases
as we discussed in an earlier module.
Using the plugins that Zeek provides,
we can also add additional writers so that we can fully
customize the logging that's available to us.
The notice framework is all about alerts.
This framework helps us define, based on notice policies,
what our important events are.
We can write scripts to define these parameters as needed within
the system and have several options to define.
Depending on our setup,
we can have the notice framework simply add a notice
to the logging stream as a log,
we can have it generate an alarm and send an email to someone,
and we can even set it up to send an email to something like a paging system.
What's awesome about this is that we can write our own policy hooks to expand
the email notifications and add to the context of them,
taking specific sections from the messages to include in the email.
The ability to decide which events generate alerts and
which ones don't is a great feature here.
We're going to hop into the lab one last time now to see how we can
convert the default ASCII logs into JSON format and show how they
look with the new format within Splunk.
We'll start out by looking at the log messages in
Splunk within the ASCII format, like we've seen before.
Then,
we'll go through and configure both Splunk and Zeek to send and
receive the JSON format instead of ASCII.
Finally,
we'll take a look at it and see how that expanded our
capabilities within our SIM to view, search,
and sort the data coming from the monitor.
Let's hop in and check it out.
I'm in my Splunk instance and I have the search pulled up from my Zeek host.
In the beginning of the course,
we took a look at the ASCII logs that were being sent via the
universal forwarder and within the CLI.
Let's look at this first event for example.
To be able to really identify this information,
we would need to normalize the data,
as well as probably do some field extractions with it.
We can tell some of the fields like the IP address that the
event's for and likely the timestamps, but it's really difficult to read.
I prefer the JSON format, even for just reading the logs,
let alone it gives you the ability to ingest it into other apps and tools.
To start the configuration,
you should probably create an index within Splunk to
handle just the messages from Zeek.
This gives you some separation of your data.
You can do this in almost any SIM out there.
Splunk is just my personal favorite.
If you need a refresher on how to install,
configure, and use Splunk,
Pluralsight has a great learning path within the
platform called Splunk Fundamentals, and I have a few courses within it.
This should help you out if you want to learn and use Splunk.
So I'm creating an index called zeek here for it to
handle my messages that I get from Zeek.
I'll input this within the forwarder configuration on the Zeek
virtual machine as well later on in this demo.
Now I need to hop into the CLI so I can configure that now that Splunk is ready.
I didn't install any apps within Splunk to ingest the JSON
format since I just wanted to show you at the very least how
nice it is from a readability perspective.
The first thing that we should do is stop Zeek using that zeekctl stop command.
Now that that's done, we can hop in and configure the parameters.
I need to edit the local.zeek file that I was
showing you in the previous module, so we'll use VI again to get into that.
Remember,
this is where all of our script loading takes place for the
local site or local instance of Zeek.
Since almost everything Zeek does is based on scripts,
we need to add a script to tell it to output to JSON
format instead of ASCII format, which it does by default.
I'm going to delete some of this extra information here that I added
earlier since we're not using that right now anyway.
It's recommended to always use comments before you tell it which script to load,
just like they do here so it's easier to keep track of what you have
turned on and turned off and what each one does.
So for this one,
I'm going to call it Output to JSON and I'm going to use the
app load command to tell it to load this json- logs.zeek file
in the policy/tuning folder.
This comes with the standard install of Zeek,
but isn't turned on unless you tell it to do so.
So I'm writing it to the file,
then I'm going to start up Zeek again so that we can have it start sending
some of the log messages over to Splunk in JSON format.
I'm using the deploy command so I know it's going to
install the new policy that we invoked.
Let me hop over to the logs folder really quick to show you how it's changed.
As you can see just by pulling up the dns.log file,
the format is all different and it's tough to read what's going on.
This format automatically tags the different fields and values to
make it easier for SIMs to ingest and interpret.
Hopping over to Splunk,
as you can see we already have the new format coming over
from Zeek and we can see the difference.
Splunk took the liberty of sorting and color coding this formatted information,
and it was able to automatically create fields for the messages
based on the information that Zeek sent it.
We'll talk about that in a minute.
First, I'm going to go back over to the CLI and configure my Splunk forwarder.
We need to find the right directory where it is so we can know what
the configuration files we need to change are.
First, we have to stop the Splunk forwarder.
In a default installation, it needs to be done from the path/bin directory,
and it can be done via this Splunk stop command.
After it shuts down,
we're free to edit the input and output files for the forwarder.
The inputs.conf file for the Splunk forwarder is what
tells us what to send to Splunk.
For now,
I'm just going to tag the host name as Zeek Sensor so we
can search via that within Splunk instead of using the
actual fully qualified domain name.
The outputs.conf file is where we're sending the information to,
and we don't really need to change any of it since
we know it's working properly.
I'm going to head over to the inputs.conf file one more time and define two of
our Zeek logs to monitor and send to the Splunk instance.
We're going to go ahead and use the conn.log and the dns.log files,
and here's where we're going to tag the index that
we want it to go into in Splunk.
I'm also making the source type Zeek here.
You can make these parameters whatever you want,
and you can have it send all of the logs over to your SIM if you want to.
I'm just doing these two as a quick example.
The fields that you're able to modify here will highly depend on what
sim and what type of forwarder you're going to use.
Since I'm using the Splunk universal forwarder sending to a Splunk instance,
this is what I'm choosing to modify and send.
Now that that's finished, let's save and start that Splunk forwarder again.
After that finishes,
I'm hopping over to my Splunk instance and I'm going to change
the search just for that particular index.
And it looks like we already have six events that got sent over.
What I love about this is that Splunk automagically knew what fields
were interesting based on the formatting that Zeek provided it so we
can search via the originating host and port,
as well as the responding host and several other fields.
I can now filter on something reaching out to Google's
DNS server on port 53 very easily.
Let me refresh the search so you can see one of these
events in the nicely sorted format as well.
And, here it is.
My host 30.4 was reaching out to this 91.189 IP address over port 123,
which is NTP.
Normalizing this data within Splunk or any SIM can take some finesse,
but this is the gist of how you can get your logs into JSON format.
Adding apps to ingest it and give you nice visualizations
and analysis can really help as well.
This was fun.
Thanks for checking this out with me.