In this lesson,
we're going to talk about NetFlow data
and how it's used to conduct traffic flow
analysis within our networks.
In order to best monitor traffic in our network,
we can either use full packet capture or NetFlow data.
Now, as you might have guessed,
packet captures can take up a lot of storage space
and they can grow quickly in size.
For example, if I'm conducting a full packet capture
on my home network each day, I would need several gigabytes
of storage just for my small family,
because every single packet that goes in
or out of my house would be captured and logged.
Every video game my son is playing online,
every YouTube video he watches,
every Netflix show my wife is binging.
All of that will be captured bit by bit
inside of that full packet capture.
Now a full packet capture or FPC
is going to capture the entire packet.
This includes the header and the payload for all the traffic
that's entering or leaving your network.
As I said, this would be a ton of information
and quickly eat up all of our storage.
Now because full packet capture takes up so much space,
we often don't collect it in a lot of organizations.
Most businesses and organizations instead will use NetFlow.
Now, NetFlow data and other similar protocols like that
are used to conduct something known as flow analysis.
Flow analysis will rely on a flow collector as a means
of recording metadata and statistics about network traffic
instead of recording each and every frame
or every single packet
that's going in or out of our network.
This allows us to use flow analysis tools
that provide network traffic statistics
sampled by the collector.
Now, by doing this, we can capture information about
the traffic flow instead of the data contained within
that data flow, and this saves us a lot of storage space.
Now with NetFlow and flow analysis,
we're not going to have the contents
of what's going over the network like we would
with a full packet capture.
But we can still gather a lot of metadata
and information about the network traffic
that's helpful to us in our monitoring.
This information is stored inside a database
and can be queried later by different tools
to produce different reports and graphs.
Now, the great thing about flow analysis
is it's going to allow us to highlight trends
and patterns in the traffic being generated by our network,
and this becomes really useful
in our network performance monitoring.
Flow analysis will allow us
to get alerts based on different anomalies we might see
and different patterns or triggers that are outside
of our expected baselines.
These tools also have a visualization component
that allow us to quickly create a map
of different network connections
and the associated flow patterns over those connections.
By identifying different traffic patterns
that might reveal bad behavior, malware in transit,
tunneling, or other bad things out there,
we're going to be able to quickly respond
to these potential problems or incidents.
Now, there are a few different tools we can use
when dealing with traffic flow analysis.
This includes things like NetFlow, Zeek,
and the multi router traffic grapher.
Let's take a look at each of these for a moment.
First, we have NetFlow.
NetFlow is a Cisco developed means
of reporting network flow information
to a structured database.
NetFlow is actually one of the first data flow analyzers
that was created out there,
and eventually it became basically the standard
that everyone started to use under the term IPFIX,
or IP flow information export.
Now, NetFlow allows us
to define a particular traffic flow based on different
packets that share the same characteristics.
For example, if we want to identify packets
with the same source and destination IP,
this could signify there's a session between those two hosts
and it should be considered one data flow
that we can collect information on.
Now, when you look at NetFlow data,
you can capture information about the packets
that are going over these devices,
like the network protocol interface that's being used,
the version and type of IP being used,
the source and destination IP address,
the source and destination port, or the IP type of service.
All this information can be gathered using NetFlow
and then analyzed and displayed visually
using our different tools.
For example, here you can see that I'm using SolarWinds
as a tool to show the NetFlow data of a network,
but you can also review this data in a text-based
environment using NetFlow exports themself.
In this graphical environment though, it becomes really easy
to see that there are 15 different traffic flows.
And if I expand the 15th data flow, we can see the source
and destination IP, the source port, the destination port,
some basic information about that data flow,
but we're not seeing the content of any of those packets
that were part of this data flow.
For us to be able to do that,
we would have to have a full packet capture,
but here we only captured the metadata
or the information about those traffic flows.
Now, if you want to be able to have the best of both worlds,
you can use something like Zeek.
Now, Zeek is a hybrid tool
that passively monitors your network like a sniffer,
but it's only going to log full packet captures
based on data of potential interest.
Essentially, Zeek is going to sample the data going across
the network just like NetFlow does.
But when Zeek finds something
that it deems interesting based on the parameters
and rules you've configured,
it's going to log the entire packet for that part
and then send it over to our cybersecurity analyst
for further investigation.
This method helps us reduce our storage
and processing requirements,
and it gives us the ability
to have all this data in a single database.
Now, one of the great things about Zeek is
that it performs normalization of this data as well,
and then stores it as either a tab delimited
or JavaScript object notation, or JSON formatted text file.
This allows you to use it with lots
of other different cybersecurity tools
and different network monitoring tools as well.
For example, now that I have this normalized data,
I can import that data into another tool
for visualization, searching and analysis.
Here, I've imported my Zeek logs into Splunk,
and then I can have my cybersecurity analyst search
for specific information during a potential incident.
Now, the third tool we have to talk about is MRTG
or the multi router traffic grapher.
The multi router traffic grapher is a tool that's used
to create graphs to show network traffic flows
going through our network interfaces on different routers
and switches, and it does this by pulling these appliances,
using SNMP, the simple network management protocol.
So what is useful about a visualization like this?
Well, you're going to be able to start seeing patterns emerge
that may be outside of your baseline.
For example, here in the top graph,
you could see a big spike in traffic
between 2:00 AM and 4:00 AM.
Is that normal?
Well, maybe and maybe not,
but it's something we should further investigate and analyze
because we're seeing this big spike occur
between 2:00 AM and 4:00 AM.
And that might be something normal
like doing offsite backups,
or it could be something malicious.
If it was the case of something that was normal,
like an offsite backup,
you're going to see this big spike in traffic
because we're sending a backup copy of all
of our data offsite to a cloud provider facility.
That might be a reasonable explanation,
and in that case, I wouldn't need to worry
because I would see that every single night
and I'd be used to seeing it.
Now, on the other hand,
maybe that server's been infected with malware.
Every day at 2:00 to 4:00 AM it's going to send
all of the data back to the bad actors
while all my administrators are at home sleeping.
This is considered data exfiltration
as part of an attack campaign.
That's something you want to be on the lookout for.
Now, just looking at this graphic,
I don't know which of these two cases it is.
Is this something normal like a backup
or is this something malicious?
But if you know your organization
and you know your baselines, now you can look at this graph
and identify what should be investigated based on seeing
that spike between 2:00 AM and 4:00 AM
and then figure out where that additional traffic flow
is going and why.
If we suspected something was malicious here,
like somebody exfiltrating our data,
then we might set up a network sniffer
in front of our file server
and see what traffic is leaving the network
and where it's going.
Then based on that,
we can have an instant response on our hands
and do our cleanup.
Now at this point, we just don't know if this is malicious
or not, but we do know it's something different
and something that is outside of the normal baseline
as indicated by that big spike.
So it's important for us to investigate it
for the health of our network.