WEBVTT 00:07.190 --> 00:11.030 In this episode, we're going to talk about one of my favorite subjects, thread hunting. 00:11.030 --> 00:15.620 How do we go in and actually find those different pesky little threats that are invading our network, 00:15.620 --> 00:19.310 and how have they corresponded to the mitigation factors that we've already talked about? 00:19.340 --> 00:22.310 Threat hunting really is one of those things that you love it or you hate it. 00:22.310 --> 00:26.630 But as a CSA, as far as the exam is concerned, we're going to cover the high levels. 00:26.630 --> 00:27.140 Don't worry. 00:27.140 --> 00:29.960 I'm going to input my own little spin on everything we talk about. 00:29.990 --> 00:34.880 Threat hunting really starts when you get an alarm, or you get a problem within your Siem system that 00:34.880 --> 00:37.100 points to a direction of malicious activity. 00:37.100 --> 00:43.160 This could be anything from a virus going off or a piece of machinery within your system suddenly crashing 00:43.190 --> 00:44.480 for no apparent reason. 00:44.510 --> 00:48.740 Threat hunting comes into play where I've got a problem within my network or within my systems, and 00:48.740 --> 00:51.950 I need to go through and find out the root cause of that problem. 00:51.950 --> 00:57.380 We start off with a simple hypothesis, meaning I need to take into account what the alert tells me, 00:57.380 --> 01:02.780 and that may be the minimal information that I get and come up with a hypothesis as to why that happened. 01:02.810 --> 01:08.000 For instance, let's say that I've got a piece of malware that suddenly went off and hit my system saying, 01:08.000 --> 01:09.500 hey, there's malware on the system. 01:09.530 --> 01:14.720 My hypothesis would be that I've got a Trojan horse or other virus that has infected this machine. 01:14.720 --> 01:17.300 That is a very simple hypothesis. 01:17.330 --> 01:20.120 We're going to go through and we're going to start to investigate the problem. 01:20.120 --> 01:25.190 We're going to dig in and find out what did that virus or that piece of malware do to our system. 01:25.220 --> 01:26.780 We're going to look for different patterns. 01:26.780 --> 01:29.420 Did it infect just this system or multiple systems? 01:29.450 --> 01:34.340 Is it just on the core operating system or has it expanded across the entire network? 01:34.370 --> 01:39.500 Those patterns really point us in the right direction and formulate what we need to do for threat hunting. 01:39.530 --> 01:41.150 Then finally we need to respond. 01:41.180 --> 01:46.280 That could mean eradicating the virus, removing the the piece of equipment from our network. 01:46.280 --> 01:47.600 We could isolate it. 01:47.630 --> 01:50.480 We could do all those different things with our incident response plan. 01:50.510 --> 01:55.010 Threat hunting really is at the core of what you do as a cybersecurity analyst, because it comes into 01:55.010 --> 01:59.650 play with saying, I've got this problem that my Siem pointed me to and I need to dive in and find out 01:59.650 --> 02:00.310 what happened. 02:00.340 --> 02:05.170 That's really all threat hunting is when you talk about threat hunting. 02:05.170 --> 02:06.910 We talked about the overall process. 02:06.910 --> 02:08.470 But why do I threat hunt? 02:08.500 --> 02:13.060 Our first question we need to ask ourselves before we start threat hunting for any legitimate problem 02:13.090 --> 02:15.880 on our network is what is the purpose of my hunt? 02:15.910 --> 02:20.380 The purpose could be that I have a bug in the machinery, and so I need to start researching that to 02:20.410 --> 02:21.100 find out. 02:21.130 --> 02:22.900 Hey, where did the bug start from? 02:22.900 --> 02:24.160 Was it from a piece of malware? 02:24.190 --> 02:26.770 Was it two pieces of software that didn't like each other? 02:26.770 --> 02:28.420 Maybe there was a hardware fault. 02:28.420 --> 02:33.520 Any of those could be the purpose of the hunt, but I need to find out why I'm actually going after 02:33.520 --> 02:34.570 that piece of problem. 02:34.570 --> 02:39.730 Sometimes it doesn't make sense to go after a hunt, meaning I may have a virus that went off on our 02:39.760 --> 02:42.640 antivirus program and it scrubbed it and it was one machine. 02:42.670 --> 02:43.750 We're not seeing patterns. 02:43.750 --> 02:45.220 We're not seeing any other problems. 02:45.220 --> 02:47.440 It may just be time to stop. 02:47.440 --> 02:48.910 Where will it be conducted? 02:48.910 --> 02:50.380 Is it across the entire network? 02:50.380 --> 02:54.460 Maybe it's just one Vlan, maybe it's one subnet, maybe it's just one machine. 02:54.460 --> 02:55.180 It doesn't matter. 02:55.180 --> 02:59.680 But we need to ask ourselves to provide a scope of what we're trying to do within our threads. 02:59.710 --> 03:02.920 What resources do I need to conduct the hunt? 03:02.950 --> 03:04.690 Do I just need my own machine? 03:04.720 --> 03:07.300 Do I need access to the antivirus program that's on there? 03:07.330 --> 03:09.070 Do I need access to the IPS? 03:09.100 --> 03:10.600 Is it a network holds a hold? 03:10.600 --> 03:12.100 Do I need access to the server? 03:12.130 --> 03:14.770 All of these come into play with again our scope. 03:14.770 --> 03:16.780 We need to identify our key stakeholders. 03:16.780 --> 03:22.180 And this is important because if I start going after a virus on a certain machine, and yet that machine 03:22.180 --> 03:25.390 is supposed to be used for something else that could cause some major problems. 03:25.390 --> 03:29.500 Usually one machine isn't going to stop us from moving on with our threat hunt, but let's say that 03:29.500 --> 03:30.370 it's a server. 03:30.370 --> 03:34.840 If I had to take an entire server offline, we may have some problems within our business scope. 03:34.870 --> 03:38.530 What if our servers are all at 80% capacity and I only have three of them? 03:38.560 --> 03:43.630 By taking one server off the line, our company could be losing money on a drastic rate, and those 03:43.630 --> 03:45.460 stakeholders may not like that too much. 03:45.460 --> 03:48.010 And then finally, what is the desired outcome of our hunt? 03:48.040 --> 03:49.690 Is it just to find the problem? 03:49.690 --> 03:50.200 Usually? 03:50.200 --> 03:51.160 Not usually. 03:51.160 --> 03:56.460 The overall outcome is supposed to be eradication or isolation, meaning I'm removing that threat from 03:56.490 --> 03:57.690 our actual systems. 03:57.690 --> 04:00.750 If we don't have a desired outcome, then we don't have an endpoint. 04:00.750 --> 04:02.940 And if we don't have an endpoint, we could go down rabbit holes. 04:02.940 --> 04:04.140 We don't want to go down. 04:04.140 --> 04:09.270 Remember, your job as an analyst is to find the problem, find out how the problem occurred, why it 04:09.270 --> 04:12.480 occurred, where it took place, and route into the system. 04:12.480 --> 04:13.530 Did it expand? 04:13.560 --> 04:18.330 Identify the stakeholders that are associated with that problem, and then let properly communicate 04:18.330 --> 04:23.820 to them with the desired outcome that we've already formulated, i.e. isolation, eradication, threat 04:23.850 --> 04:27.360 hunting really comes into play with those basic questions that you need to be asking yourself before 04:27.360 --> 04:28.740 we jump into the fray. 04:29.670 --> 04:35.190 The Threat Hunting Maturity Model, or HRM, is a maturity model that we use not as a person but as 04:35.190 --> 04:36.270 an organization. 04:36.270 --> 04:41.070 This means that we as an organization, need to understand where we come into play as a threat. 04:41.100 --> 04:46.290 Hunting infrastructure, meaning how well does our organization actually handle those different threats? 04:46.290 --> 04:51.000 If we're very at the point of emphasis in our threat hunting maturity model, it means that we don't 04:51.000 --> 04:52.530 usually have a lot of data collected. 04:52.530 --> 04:54.820 This means that our network usually doesn't have a Siem. 04:54.820 --> 04:57.220 It doesn't normally have a lot of logs that were corresponding. 04:57.220 --> 05:00.580 Maybe it's a brand new organization that's being stirred up for the first time. 05:00.580 --> 05:04.420 And all we really have on there are some machines with a basic network, so we really don't have a lot 05:04.420 --> 05:06.070 of information that's coming into play. 05:06.070 --> 05:12.490 This would be an Hmm zero infrastructure, meaning that we don't have the resources or the personnel 05:12.520 --> 05:15.010 to probably start searching for threat huts. 05:15.010 --> 05:16.510 Then there's MX one. 05:16.510 --> 05:21.100 Now this is where we have some data collected, maybe some minimal information is gathered, maybe we 05:21.100 --> 05:23.290 have a scene, but it just got kicked off yesterday. 05:23.290 --> 05:27.640 And so we're getting alerts, but we really don't have those alerts properly tracked or maintained. 05:27.670 --> 05:29.530 We're not really sure where everything came from. 05:29.530 --> 05:35.710 But we can develop a new hypothesis and start very, very low level threat hunting activities. 05:35.710 --> 05:39.130 This usually comes into play with organizations that are brand new to the scope. 05:39.130 --> 05:40.600 They just started their Siem. 05:40.600 --> 05:45.190 They just have a cyber security program, but they're very, very low maturity level when it comes into 05:45.190 --> 05:45.880 play. 05:45.880 --> 05:51.040 With MX two, we have high collection of some data types, meaning maybe we're collecting data from 05:51.040 --> 05:55.810 our routers or our Or firewalls or IPS, but maybe we're missing it from the servers. 05:55.810 --> 06:00.310 Or maybe we have it from the servers, but we're not collecting logs from our from our firewalls or 06:00.310 --> 06:01.030 our routers. 06:01.030 --> 06:05.830 This is where we have a lot of good information from some items and no information from others. 06:05.860 --> 06:11.020 Not the best practice comes into it, but we're developing our threat hunting or our maturity model. 06:11.020 --> 06:16.780 In this case, we can generate general Intel reports, we can use basic tools, and we can automate 06:16.780 --> 06:20.620 some of those threat handling procedures that we would in a normal SOC environment. 06:20.620 --> 06:25.990 But again, still standing up our systems and a layer three or excuse me, a level three. 06:26.230 --> 06:26.740 Hmm. 06:26.740 --> 06:31.390 We have a high collection of many data types, meaning maybe we're getting data from different servers. 06:31.420 --> 06:34.630 We're getting it from switches, we're getting from routers, we're getting it from firewalls, IPS, 06:34.660 --> 06:37.180 IDs, a whole gambit of different information. 06:37.180 --> 06:42.700 We may be missing a log file here and there, but overall we have a very high collection methodology. 06:42.730 --> 06:48.040 We also have described Intel reports meaning outsider intelligence reports are coming into us and we're 06:48.040 --> 06:49.330 getting those different feeds. 06:49.330 --> 06:53.860 We talked about that in an earlier episode, but if you remember, we saw different feeds from all types 06:53.860 --> 06:54.730 of blogs. 06:54.730 --> 06:59.500 We saw them from different websites, from the government, even from different other organizations. 06:59.500 --> 07:04.060 Those Intel reports provide us different trends and availability of information that we can visualize 07:04.060 --> 07:08.080 and better prepare ourselves for threat detection and mitigation. 07:08.110 --> 07:13.360 Finally, in level four, we have continuous mapping of possible threats to our organization. 07:13.360 --> 07:16.240 This is the highest level of maturity model available here. 07:16.240 --> 07:21.610 And we can not only search, but visualize and analyze different data sets to those corresponding information 07:21.610 --> 07:22.270 that we're getting. 07:22.270 --> 07:25.150 Meaning we're taking those Intel reports that we've received. 07:25.150 --> 07:27.520 We're also taking our data from our infrastructure. 07:27.520 --> 07:32.710 We're kind of combining them two together, and we're providing our own analysis of the different facts 07:32.710 --> 07:33.610 associated with it. 07:33.640 --> 07:38.410 That's usually reserved for high level organizations that can do both threat hunting and have a very 07:38.410 --> 07:41.110 robust security network with lots of manpower. 07:41.110 --> 07:47.230 Think about those high intrinsic SoCs that utilize various different departments, usually with something 07:47.230 --> 07:52.760 associated with maybe with AT&T or Verizon that have those very, very advanced socks. 07:52.760 --> 07:57.350 There's different techniques that you as a threat analyst can utilize in order to find different aspects 07:57.350 --> 07:58.910 of your different networks. 07:58.910 --> 08:02.990 That means that if I come into play, I can look at the different alerts and corresponding intelligence 08:02.990 --> 08:07.970 reports that I'm getting and set that data in a way that I can actually read it and go through the process. 08:08.000 --> 08:09.650 Volume is probably the easiest. 08:09.650 --> 08:13.940 This is pure simple, just the aspect of different alerts that are coming into play. 08:13.940 --> 08:17.720 If I get over 1200 different alerts for the same machine, that's volume. 08:17.720 --> 08:22.730 I'm getting a bunch of information from one specific machine or one type of alert that I need to be 08:22.730 --> 08:23.270 aware of. 08:23.300 --> 08:24.110 Pure and simple. 08:24.110 --> 08:25.520 Volume is just numbers. 08:25.520 --> 08:26.660 Then there's frequency. 08:26.690 --> 08:30.860 How frequent is the specific alarm or specific problem coming into play? 08:30.890 --> 08:35.450 Now, this is honestly sometimes confused with volume where we're looking at the full number, where 08:35.480 --> 08:40.610 frequency is more of the pattern that comes aligned with it, which means I could get a lot of volume 08:40.610 --> 08:41.720 for the same alert. 08:41.720 --> 08:43.820 But how often is it occurring? 08:43.820 --> 08:47.480 Let's say that I have one alarm coming into one system every day. 08:47.480 --> 08:52.160 It doesn't sound like much because it's only happening one day, but then it happens again, and then 08:52.160 --> 08:53.540 again and then again. 08:53.540 --> 08:57.050 And for seven straight days, we've got the same alarm at the same time. 08:57.050 --> 09:01.970 That only provides us a pattern, but it also provides this frequency, and that's where frequency analysis 09:01.970 --> 09:02.930 comes into play. 09:02.960 --> 09:04.550 Finally we have clustering. 09:04.550 --> 09:09.140 Now clustering is finding similarities in data by comparing the different characteristics. 09:09.140 --> 09:11.270 This is often confused with grouping. 09:11.690 --> 09:16.610 Clustering comes into the fact that I'm not actually looking for similarities, but I kind of am. 09:16.610 --> 09:21.710 I take this giant glob of data and I start sifting through it, and I notice different similarities 09:21.710 --> 09:22.790 as I'm going through. 09:22.820 --> 09:24.560 This is where costing comes into play. 09:24.560 --> 09:29.240 Let's say that I've got a bunch of data, and I start sifting through the data, and it all points out 09:29.240 --> 09:30.440 to the color blue. 09:30.440 --> 09:32.990 So I start to see blue pop up a lot more often. 09:32.990 --> 09:36.080 That's finding a similarity in the data I found blue. 09:36.110 --> 09:37.850 This is different from grouping. 09:37.850 --> 09:42.020 Grouping is where I have a bunch of data, and I specifically look for the color blue. 09:42.050 --> 09:45.260 That's where grouping is comes into difference with clustering. 09:45.260 --> 09:49.010 Clustering I'm not looking for the data or specific attributes grouping. 09:49.010 --> 09:51.320 I am looking for specific attributes. 09:51.320 --> 09:52.670 And then there's stacking. 09:52.670 --> 09:55.610 Stacking is sorting data and finding the extremes. 09:55.610 --> 09:58.010 Let's say that I have an average of five. 09:58.010 --> 10:01.760 I've got 500 different numbers and they're all different all over the place. 10:01.790 --> 10:02.030 Right. 10:02.060 --> 10:04.340 I take the median or the average of those numbers. 10:04.340 --> 10:05.270 And I say, you know what? 10:05.300 --> 10:08.360 My average is between 200 to 300. 10:08.390 --> 10:10.940 That's the average of those numbers from 1 to 500. 10:10.970 --> 10:12.740 That would be my main grouping. 10:12.740 --> 10:14.870 However, stacking takes the extremes. 10:14.870 --> 10:17.750 Maybe I have a number at 102. 10:17.780 --> 10:20.570 And on the flip end, maybe I have a number at 498. 10:20.600 --> 10:21.860 Those are the extremes. 10:21.860 --> 10:23.540 And that's where stacking comes into play. 10:23.540 --> 10:27.170 We're not looking at the core foundation or the medium or the average. 10:27.170 --> 10:31.550 We're looking at the extremes, the two ends where the data just doesn't support what's going on within 10:31.550 --> 10:32.180 our system. 10:32.180 --> 10:33.500 We talked about analysis. 10:33.530 --> 10:37.580 We talked about hypothesis, the process, the procedures and how you found everything. 10:37.580 --> 10:39.050 But what do we do after that? 10:39.050 --> 10:40.820 That's where reporting comes into play. 10:40.820 --> 10:45.440 Reporting is taking everything that we did and putting it into a well-documented aspect. 10:45.440 --> 10:47.610 Now you heard me pound on documenting. 10:47.610 --> 10:50.250 And we beat that dead horse into a million different pieces. 10:50.250 --> 10:51.420 But I'm going to say it again. 10:51.450 --> 10:52.650 You got a document. 10:52.650 --> 10:55.080 We need to document what our hypothesis came from. 10:55.080 --> 10:58.680 If I'm going threat hunting, I need to start my documentation right at the get go. 10:58.710 --> 10:59.760 Why am I threatening? 10:59.760 --> 11:02.730 What was the original alert or problem that I saw? 11:02.760 --> 11:06.210 And I need to document that along with my original hypothesis. 11:06.240 --> 11:11.460 Now, our hypothesis can change as we're going forward, and we can be proven wrong on a hypothesis. 11:11.460 --> 11:12.720 That's perfectly okay. 11:12.720 --> 11:13.710 It doesn't matter. 11:13.710 --> 11:16.710 We just need to document the differences from what we went. 11:16.710 --> 11:22.560 If we go from one hypothesis, i.e. I believe the sky is orange and now I believe it's green, then 11:22.560 --> 11:24.300 I need to document that hypothesis. 11:24.330 --> 11:27.420 If I'm suddenly wrong and the sky isn't green, it's now blue. 11:27.450 --> 11:28.410 Well, that's a finding. 11:28.410 --> 11:30.960 That's the overall finding that we we identified. 11:30.960 --> 11:31.650 That's okay. 11:31.680 --> 11:32.850 Our hypothesis is wrong. 11:32.850 --> 11:34.620 There's nothing wrong with being wrong. 11:34.620 --> 11:38.160 We just need to document why we found the color blue for the sky. 11:38.160 --> 11:40.320 We need to identify our different processes. 11:40.320 --> 11:46.590 How did we go through the actions of going in and finding what we found was a process to simply go through 11:46.590 --> 11:51.600 the antivirus program and figure out, hey, yeah, it stopped the virus and it isolated it. 11:51.600 --> 11:52.560 And so I'm done. 11:52.590 --> 11:56.730 Or was it something more complex where we went through the different procedures in order to actually 11:56.730 --> 12:01.980 identify that a piece of malware came in from this IP address because someone posted on a phishing link. 12:01.980 --> 12:04.920 We need to identify that process and how we went about that. 12:04.950 --> 12:07.860 We need to identify the different procedures that we utilize. 12:07.890 --> 12:12.150 Did we utilize a tool such as Pfsense for our firewall? 12:12.180 --> 12:15.330 Did we use a different router and trace the logs through there? 12:15.360 --> 12:16.860 Did we use our Siem system? 12:16.860 --> 12:22.020 All those tools need to be identified within our process and procedures and annotated correctly on our 12:22.020 --> 12:23.010 documentation. 12:23.040 --> 12:24.990 Finally, we need to identify our findings. 12:24.990 --> 12:28.170 If I finally discovered that the sky is blue, that's fine. 12:28.170 --> 12:29.700 I found out the sky is blue. 12:29.730 --> 12:31.080 Let's mark down our findings. 12:31.080 --> 12:35.730 We've identified how we got there, and then finally we need to identify our recommended actions. 12:35.730 --> 12:36.780 Well, the sky is blue. 12:36.780 --> 12:37.740 My recommended action. 12:37.740 --> 12:38.310 Nothing. 12:38.340 --> 12:42.390 However, what if it was a piece of malware that was on our systems and we discovered not only did we 12:42.390 --> 12:46.890 alert, but that malware has expanded into our servers and even some of our clients. 12:46.890 --> 12:51.120 Well, maybe our recommended action would be during the maintenance window to scrub those systems and 12:51.120 --> 12:55.320 get rid of and isolate those networks so that we can scrub those systems. 12:55.320 --> 12:59.970 When we go through and we go through the reporting process, we need to be very clear, very concise 12:59.970 --> 13:01.410 and have a proper workflow. 13:01.440 --> 13:05.730 Usually a company will have a template for you to follow, and usually just following that template 13:05.730 --> 13:10.260 just comes into play and we just go A, B, C, D and E and boom, we're done. 13:10.260 --> 13:11.610 But we want to annotate everything. 13:11.610 --> 13:15.870 We want to go through that documentation to make sure that it's very easy to read, and that anybody 13:15.870 --> 13:18.270 can follow our line of thought as we're going forward. 13:18.300 --> 13:23.160 Who knows, there may be another action two weeks from now that a brand new analyst is trying to identify, 13:23.160 --> 13:25.770 and your report will help them get through that process. 13:25.770 --> 13:28.500 We discussed today different threat hunting methodologies. 13:28.500 --> 13:30.840 We discussed the threat hunting maturity model. 13:30.840 --> 13:35.100 We went through the different process reporting of documentation and annotation. 13:35.100 --> 13:40.230 When you go through the Cisa exam as an analyst, you need to understand again that this is a very high 13:40.230 --> 13:41.280 level overview. 13:41.280 --> 13:44.870 You're not expected to be able to go through and say, oh, I found a piece of malware. 13:44.870 --> 13:47.270 And from that malware, I need to use this tool. 13:47.270 --> 13:51.200 And from that tool, depending on what it's not like that it's very high and very high level. 13:51.230 --> 13:55.880 Scenario based questions, much like we've discussed in the past, you should expect to see questions 13:55.880 --> 13:59.180 such as what's the threat maturity model would you use? 13:59.180 --> 14:04.460 Or what phase of that threat maturity model would you be in for your organization if you had access 14:04.460 --> 14:10.340 to some reports, but not all, some inaccuracies within your intelligence gathering and very low, 14:10.340 --> 14:15.200 limited tools at your discretion, then you would be able to answer that question between 0 and 4 of 14:15.200 --> 14:16.970 what maturity model level you were at. 14:17.000 --> 14:21.500 You should also expect to see different questions on different threat hunting, aspects of where the 14:21.500 --> 14:24.680 process from hypothesis all the way down to reporting. 14:24.680 --> 14:28.190 You would lie within this specific, uh, scenario. 14:28.220 --> 14:31.490 As you're going through these different questions, realize they're all scenario based. 14:31.490 --> 14:32.870 You shouldn't have any problems with it. 14:32.870 --> 14:37.040 Just understand and memorize the threat hunting maturity model as well as threat hunting graph that 14:37.040 --> 14:37.850 we propose. 14:37.880 --> 14:40.130 Remember, everything starts with a hypothesis.