WEBVTT 00:00:00.000 --> 00:00:01.700 [Autogenerated] so that's a pen test methodology and 00:00:01.710 --> 00:00:03.230 the sort of people should use for it. 00:00:03.240 --> 00:00:06.360 We got two sets of pent is where she three sets of pen test to do. 00:00:06.360 --> 00:00:10.410 Now, we've got first one on external ___________ test annually. 00:00:10.420 --> 00:00:13.180 And after that, significant changes. 00:00:13.180 --> 00:00:13.560 Yeah, 00:00:15.240 --> 00:00:20.500 on then an internal ___________ test and then a segmentation 00:00:20.500 --> 00:00:25.190 ___________ test if you're using segmentation to have your 00:00:25.200 --> 00:00:28.390 your scope production effectively, Although, 00:00:28.400 --> 00:00:29.970 although it's the same methodology, 00:00:29.970 --> 00:00:32.130 so they will be done by the same person at the same time. 00:00:32.140 --> 00:00:34.460 Eso no end. 00:00:34.470 --> 00:00:35.640 Sure, right. 00:00:35.650 --> 00:00:37.850 In the sense that no, they don't always have the same methodology. 00:00:37.850 --> 00:00:40.780 But maybe they can be done by the same person of the same firm, 00:00:40.790 --> 00:00:43.650 you know, in the same sort, of course of activity. 00:00:43.660 --> 00:00:44.130 Right. 00:00:44.140 --> 00:00:45.280 That could be the case. 00:00:45.290 --> 00:00:49.590 Uh, so So, yeah, the external, The internal ___________ tests are are, 00:00:49.600 --> 00:00:51.570 you know, annual or after significant change. 00:00:51.540 --> 00:00:56.300 I find a lot of organizations really sort of skate on that significant change 00:00:56.310 --> 00:00:59.010 because the pen test is often a fairly expensive endeavor. 00:00:59.020 --> 00:00:59.400 Right? 00:00:59.410 --> 00:01:02.620 And I don't wanna look cheaper than being compromised by criminal. 00:01:02.630 --> 00:01:06.170 Yeah, yeah, that's true. 00:01:06.180 --> 00:01:07.070 That's true. 00:01:07.080 --> 00:01:12.370 Um, if only that argument worked as well as it seems like it should, Right. 00:01:12.380 --> 00:01:16.060 Um but but a lot of organizations say, Well, 00:01:16.060 --> 00:01:17.720 you know, that wasn't really a significant change, 00:01:17.720 --> 00:01:19.840 even though we just like tor down from this one data center 00:01:19.840 --> 00:01:21.480 and move everything over to this other day. 00:01:21.490 --> 00:01:23.850 But do they know I have assurance functions that say, 00:01:23.860 --> 00:01:28.330 you know, the massive I mean, ignoring PC IDE s s any sort of change like that. 00:01:28.340 --> 00:01:30.500 I'd want to do my own pan test before the criminal do that. 00:01:30.510 --> 00:01:33.060 You would think you would think. 00:01:33.070 --> 00:01:35.580 Um, but yeah, the goal. 00:01:35.580 --> 00:01:35.760 Right. 00:01:35.760 --> 00:01:36.660 The external pen test. 00:01:36.660 --> 00:01:40.490 What can the attacker get from an internet perspective? 00:01:40.500 --> 00:01:40.710 Right. 00:01:40.710 --> 00:01:41.490 What can they do? 00:01:41.500 --> 00:01:41.820 Right? 00:01:41.820 --> 00:01:45.380 And that should that should cover sort of network layer or 00:01:45.390 --> 00:01:48.320 infrastructure layer and application layer right. 00:01:48.330 --> 00:01:52.280 The language and standards as network layer and application layer right 00:01:52.280 --> 00:01:55.450 and and read network layer as infrastructure layer. 00:01:55.460 --> 00:01:58.390 Right, um, and application layer. 00:01:58.400 --> 00:01:58.780 Right. 00:01:58.790 --> 00:02:00.000 And And that that matters, right? 00:02:00.000 --> 00:02:01.220 And the internal test. 00:02:01.230 --> 00:02:04.590 Let's take on the external because I wanna ask a really naughty 00:02:04.590 --> 00:02:07.810 question that people always ask about external pan test. 00:02:07.820 --> 00:02:11.910 Should I white list the pan test from my ideas. 00:02:11.920 --> 00:02:12.370 Right? 00:02:12.340 --> 00:02:13.090 But what's the point? 00:02:13.090 --> 00:02:16.360 Because, you know, I'm gonna run the argument that I hear. 00:02:16.550 --> 00:02:18.420 But look, that's what I've got on the network. 00:02:18.420 --> 00:02:20.190 So that's what a real criminal would see. 00:02:20.250 --> 00:02:24.150 And therefore the pipes or the wife and everything will just stop the activity. 00:02:24.640 --> 00:02:26.590 Well, okay. 00:02:26.590 --> 00:02:33.360 My counter argument to that is the The pen test is a sort of focused, 00:02:33.840 --> 00:02:35.810 limited time and effort kind of activity, 00:02:35.820 --> 00:02:36.120 right? 00:02:36.130 --> 00:02:39.450 You want to You want to pay the pen tester? 00:02:40.140 --> 00:02:40.820 You know, 00:02:40.820 --> 00:02:43.460 the reasonable sum for them to find what they're gonna 00:02:43.460 --> 00:02:45.110 find in the reasonable timeframe, Right? 00:02:45.120 --> 00:02:47.100 A to real attack. 00:02:47.110 --> 00:02:50.810 Right on attacker who is motivated by financial 00:02:50.810 --> 00:02:53.370 gain by harming your organization, uh, 00:02:54.440 --> 00:02:59.710 can use techniques to operate more slowly or to operate more surreptitiously, 00:02:59.720 --> 00:03:02.330 maybe take more time and effort, uh, 00:03:02.340 --> 00:03:05.940 but can evade defenses like an intrusion detection 00:03:05.940 --> 00:03:08.300 system or web application firewall. 00:03:08.310 --> 00:03:08.640 Right. 00:03:08.650 --> 00:03:12.080 So if he was a Q s, A c a pen test, 00:03:12.080 --> 00:03:15.340 where they weren't white listed where the external pen test was just blocked, 00:03:15.350 --> 00:03:16.760 by the way for by the ideas. 00:03:16.760 --> 00:03:18.720 And you see a report that says that you're not accepting it? 00:03:18.730 --> 00:03:20.170 No, I wouldn't accept that now. 00:03:20.180 --> 00:03:23.540 I think it's a reasonable goal first to white, 00:03:23.540 --> 00:03:27.050 list the tester and have the test to do their testing. 00:03:27.060 --> 00:03:27.580 And then, 00:03:27.580 --> 00:03:32.460 as maybe they're sort of ultimate step before they wrap testing to then unlike, 00:03:32.460 --> 00:03:36.790 list them and make sure that your detective controls error, 00:03:36.790 --> 00:03:38.560 actually working as advertised, right? 00:03:38.570 --> 00:03:38.880 Okay. 00:03:38.880 --> 00:03:42.730 Now go ahead and do some other testing just to see if my ideas picks it up. 00:03:42.740 --> 00:03:44.970 That's a reasonable testing goal, right? 00:03:44.940 --> 00:03:46.670 And maybe you should stick that in your pen. 00:03:46.670 --> 00:03:47.580 Testing methodology. 00:03:47.580 --> 00:03:49.570 Okay, that's a That's a really good TIFF, actually. 00:03:49.580 --> 00:03:51.650 Now, let's talk about the internal pen test. 00:03:51.650 --> 00:03:52.100 Sure. 00:03:52.110 --> 00:03:55.270 Again, How does that happen? 00:03:55.270 --> 00:03:55.480 What? 00:03:55.480 --> 00:03:55.790 What? 00:03:55.790 --> 00:03:59.000 Walk me through what you're looking for because you're coming 00:03:59.000 --> 00:04:00.560 outs and center at the end of the year, right? 00:04:00.570 --> 00:04:02.110 You hope they've had an internal pen test? 00:04:02.110 --> 00:04:03.520 What are you looking for in that? 00:04:03.530 --> 00:04:06.900 I want to I want to make sure that the internal tester, 00:04:06.910 --> 00:04:08.660 I mean, likewise with the extra faster. 00:04:08.660 --> 00:04:11.180 I wanna make sure they understood that scope, right, right? 00:04:11.190 --> 00:04:14.440 And that they tested all of the relevant portions of the 00:04:14.440 --> 00:04:17.600 environment from an interior vantage right. 00:04:17.610 --> 00:04:19.740 Similarly to the Vulnerability scan thing right? 00:04:19.740 --> 00:04:22.480 What can you see from the inside now? 00:04:22.490 --> 00:04:25.480 The sort of tactical way of going about it might 00:04:25.480 --> 00:04:26.760 have a few different approaches, Right? 00:04:26.760 --> 00:04:29.890 You might send your internal pen tester on site and they plug into some 00:04:29.890 --> 00:04:32.800 spot and do the testing in person while they're there. 00:04:32.810 --> 00:04:33.170 Right? 00:04:33.140 --> 00:04:35.010 And and sometimes that makes the most sense. 00:04:35.010 --> 00:04:38.060 And sometimes you know it's it's more travel costs than is really necessary. 00:04:38.060 --> 00:04:40.040 But that's one way to do it. 00:04:40.050 --> 00:04:40.620 Um, 00:04:40.630 --> 00:04:44.400 you could conceivably have them connect over like a VPN from 00:04:44.400 --> 00:04:46.820 wherever they happen to be into your environment. 00:04:46.830 --> 00:04:48.280 Um uh, 00:04:48.290 --> 00:04:55.730 tunneling pen testing traffic over VPN is often sort of problematic in terms 00:04:55.730 --> 00:04:59.150 of performance or or other sort of connectivity reasons. 00:04:59.150 --> 00:05:04.570 So most pen testers that I'm aware of really don't prefer that approach the 00:05:04.580 --> 00:05:08.770 sort of best approach I've seen for for a sort of limiting, 00:05:09.040 --> 00:05:11.270 uh, those kinds of problems and then not, 00:05:11.280 --> 00:05:13.730 you know, sending your pen testers all all over the country. 00:05:13.740 --> 00:05:17.060 Ises UI have the client, you know, 00:05:17.070 --> 00:05:21.460 set up a physical or a virtual host right apprentice box that 00:05:21.470 --> 00:05:24.920 phones back home to the tester and then allows the tester to 00:05:24.920 --> 00:05:27.120 connect into the environments right? 00:05:27.120 --> 00:05:28.390 And and run their utilities. 00:05:28.390 --> 00:05:28.620 Right? 00:05:28.620 --> 00:05:31.360 So they're they're testing again, the network sort of layer. 00:05:31.360 --> 00:05:33.990 And then and then conceivably the application layer for 00:05:33.990 --> 00:05:36.020 whatever applications are are visible there. 00:05:36.030 --> 00:05:36.140 Yeah. 00:05:36.140 --> 00:05:37.740 I mean, we I mean, you know, typically, 00:05:37.740 --> 00:05:41.250 we spend API UNIX box and give the tests and remote access to like, 00:05:41.250 --> 00:05:41.790 there's your box. 00:05:41.790 --> 00:05:44.810 Do are practice has a stable of physical boxes of 00:05:44.810 --> 00:05:49.380 somebody insists on physical box, and then has a standard, 00:05:49.380 --> 00:05:52.970 like virtual build that they can put in a sftp site or whatever. 00:05:52.970 --> 00:05:54.060 And have somebody download. 00:05:54.740 --> 00:05:59.630 Okay, Um, so you look when you get the pen test, 00:05:59.630 --> 00:06:02.190 you looking back at the methodology and checking that the 00:06:02.190 --> 00:06:03.730 actual pen test to follow their methodology. 00:06:03.730 --> 00:06:04.960 They said we're going to, right? 00:06:05.040 --> 00:06:06.840 Where do you get failures in here? 00:06:06.840 --> 00:06:10.540 So both internal and external pen test where the people fail, 00:06:10.540 --> 00:06:12.570 whether people become non compliance. 00:06:12.570 --> 00:06:17.920 So I find a lot of lousy pen tests where the tester either just wasn't 00:06:17.920 --> 00:06:21.700 very good or didn't put a lot of effort into it. 00:06:21.710 --> 00:06:25.140 You know, I found I found one of those sort of empty reports where, 00:06:25.150 --> 00:06:25.970 uh, 00:06:27.040 --> 00:06:29.510 like we couldn't find anything we couldn't get access to anything like 00:06:29.520 --> 00:06:31.900 What do you mean you couldn't get access to anything? 00:06:31.910 --> 00:06:36.630 Do you think that's because the climb said I'm gonna I'm gonna pay. 00:06:36.640 --> 00:06:38.720 I'm gonna pay this right. 00:06:38.730 --> 00:06:40.560 This only buys you a lousy pens. 00:06:40.560 --> 00:06:41.970 Us That. 00:06:41.980 --> 00:06:46.080 I think I think that is that is the maybe not expressly stated, 00:06:46.080 --> 00:06:48.410 but fundamental reason for for a lot of things. 00:06:48.410 --> 00:06:53.010 I think a lot of clients don't necessarily realize that a $7000 test is garbage. 00:06:53.020 --> 00:06:56.050 You're better off giving that $7000 to a monastery and 00:06:56.050 --> 00:06:57.760 having them pray for your network. 00:06:57.770 --> 00:07:01.330 Um A And I think Ok, well, 00:07:01.330 --> 00:07:03.740 I'm getting I'm getting something that's a lot less than of, 00:07:03.750 --> 00:07:07.810 you know, $40,000 test by, you know, a more reputable firm. 00:07:07.820 --> 00:07:08.840 Uh, right. 00:07:08.840 --> 00:07:13.880 But the tester maybe doesn't have the time to do that much 00:07:13.880 --> 00:07:16.420 effort because they have this tiny budget on. 00:07:16.420 --> 00:07:18.440 So I think you need to be sort of careful about that. 00:07:18.450 --> 00:07:22.360 Um, but if you're if you're sitting there is an assessor. 00:07:22.360 --> 00:07:23.440 It's the end of the year. 00:07:23.450 --> 00:07:25.750 You're doing your annual DSS assessment. 00:07:25.760 --> 00:07:26.390 You see, 00:07:26.390 --> 00:07:31.920 a lousy pen test with a lousy methodology is that failure e had a pan test. 00:07:31.920 --> 00:07:33.460 It doesn't say I have a decent fantastic. 00:07:33.460 --> 00:07:34.760 Just have a pan test. 00:07:34.760 --> 00:07:35.950 He could say have a lousy pants. 00:07:35.960 --> 00:07:36.510 Right? 00:07:36.510 --> 00:07:40.900 Well, but does it find Doesn't find the vulnerabilities that, 00:07:40.910 --> 00:07:44.210 frankly, I'm finding right by looking at it from the other perspective. 00:07:44.220 --> 00:07:49.750 Um, and you know, the methodology sort of requirement helps me in this, right. 00:07:50.060 --> 00:07:52.280 Did it test your entire scope? 00:07:52.290 --> 00:07:52.670 Right. 00:07:52.680 --> 00:07:55.390 I find pen test that, you know, 00:07:55.400 --> 00:07:59.890 have have some meaningless boilerplate front matter kind of text and then 00:07:59.890 --> 00:08:03.580 immediately launch into some grid of mostly useless findings that looked like 00:08:03.580 --> 00:08:05.660 they came straight out of a vulnerability scanner. 00:08:05.670 --> 00:08:09.040 Because because Because that's precisely what happened, right? 00:08:09.040 --> 00:08:12.610 Like, you know, barely any text formatting or anything, right? 00:08:12.620 --> 00:08:15.550 And there's no description of scope in the report. 00:08:15.560 --> 00:08:17.710 And there's no description of methodology, right? 00:08:17.720 --> 00:08:18.740 I mean, it's pretty implicit. 00:08:18.750 --> 00:08:21.310 It's, you know, hit the start button on the tool. 00:08:21.320 --> 00:08:25.320 Uh, go get tacos and then and then send the report to the client. 00:08:25.330 --> 00:08:25.750 Right. 00:08:25.760 --> 00:08:28.790 But want to to APIs that you going to? 00:08:28.800 --> 00:08:29.830 Yes. 00:08:29.840 --> 00:08:32.940 Do going to to quite well what I have done, right? 00:08:32.950 --> 00:08:34.659 What I've done in cases where I found this, 00:08:34.659 --> 00:08:37.169 particularly the last handful of years, uh, 00:08:37.240 --> 00:08:40.740 is that I've been a little bit fortunate in that my assessment work. 00:08:40.750 --> 00:08:45.230 Our assessment work comes in relatively shortly after the test has concluded, 00:08:45.230 --> 00:08:46.900 as the months and months right. 00:08:46.910 --> 00:08:49.580 And I look at the pen test report and say, There's no substance here. 00:08:49.590 --> 00:08:52.050 There's no methodology, There's no scope. 00:08:52.140 --> 00:08:54.430 I have no assurance that they've tested any of the 00:08:54.430 --> 00:08:55.620 stuff that they're supposed to. 00:08:55.630 --> 00:08:57.550 There's no description of any of the segmentation 00:08:57.550 --> 00:09:00.760 testing like this isn't gonna work right, 00:09:00.770 --> 00:09:02.970 And and one time to my clients credit. 00:09:02.970 --> 00:09:04.680 And I feel sort of bad for the tester. 00:09:04.690 --> 00:09:06.350 You know, we were on the phone. 00:09:06.360 --> 00:09:11.310 Uh, they basically put us on hold, called the tester and, 00:09:11.310 --> 00:09:11.470 like, 00:09:11.470 --> 00:09:14.930 joined this poor guy to the conference call where I I then 00:09:14.930 --> 00:09:16.830 proceeded to sort of roast him alive. 00:09:16.840 --> 00:09:20.430 Uh, and you know, then, in that case, 00:09:20.430 --> 00:09:23.140 that firm then went back and actually did some more testing, 00:09:23.150 --> 00:09:24.080 Uh, 00:09:24.090 --> 00:09:27.160 presume presumably brought us for the client and then 00:09:27.240 --> 00:09:29.460 yielded some more useful results. 00:09:29.470 --> 00:09:30.270 Right, Right. 00:09:30.440 --> 00:09:34.710 But I think what What I'm hearing is the best advice is to get 00:09:34.710 --> 00:09:39.340 your QoS a early to help you by your pen test, 00:09:39.350 --> 00:09:43.740 I'd say Engage your que se for helping you select a pen test fender, 00:09:43.750 --> 00:09:44.190 right? 00:09:44.200 --> 00:09:46.070 You've seen good ones of balance. 00:09:46.070 --> 00:09:47.560 You have a good view of the market, right? 00:09:47.570 --> 00:09:48.940 Right on. 00:09:48.940 --> 00:09:50.410 And I would say, You know, 00:09:50.410 --> 00:09:52.950 it's not even so much like I know this firm always does good work, 00:09:52.950 --> 00:09:54.640 and I know this from always does lousy work. 00:09:54.650 --> 00:09:57.550 But let's talk to the tester, right? 00:09:57.550 --> 00:10:00.070 Who's going to be staffed on this particular test? 00:10:00.540 --> 00:10:03.050 Get a sense of that person's expertise, 00:10:03.310 --> 00:10:09.590 Get a sense of this person sort of grasp of the client environment and in, 00:10:09.600 --> 00:10:11.250 you know, does that make sense, right? 00:10:11.260 --> 00:10:14.900 You know, does the does the scope of that engagement cover right? 00:10:14.910 --> 00:10:15.690 And really, 00:10:15.690 --> 00:10:18.030 there's There's some onus on the client to make sure that 00:10:18.030 --> 00:10:21.370 what they're agreeing to is appropriate, 00:10:21.380 --> 00:10:21.760 right? 00:10:21.770 --> 00:10:25.440 I mean, even our Prentice practice says, you know, we're a pen test practice. 00:10:25.450 --> 00:10:26.440 We're not a PC. 00:10:26.440 --> 00:10:29.690 I pen test practice, and we're not the que se right. 00:10:29.700 --> 00:10:32.160 The client says, I want you to pen test this. 00:10:32.170 --> 00:10:34.150 I want you to do these things in the pen test. 00:10:34.160 --> 00:10:35.650 We're going to do that. 00:10:35.640 --> 00:10:38.790 And if that's insufficient for their PC IDE is that's compliance purposes. 00:10:38.800 --> 00:10:42.290 Even if it's my colleagues over here in my same company that are 00:10:42.290 --> 00:10:44.880 the que se is like this was in our agreement. 00:10:44.880 --> 00:10:46.470 This is what we said we were gonna do. 00:10:46.480 --> 00:10:47.760 So because I mean, 00:10:47.770 --> 00:10:50.950 my reflection is that people don't know how to buy pan test very well. 00:10:50.950 --> 00:10:54.100 Which is why you should actually hire someone to help you speck out, 00:10:54.100 --> 00:10:57.290 too by pants, right, e frankly, again, 00:10:57.300 --> 00:11:00.390 I would say if I'm doing your assessment in a couple of months, 00:11:00.400 --> 00:11:01.280 call me. 00:11:01.290 --> 00:11:06.150 I would just assume not charge you to help you get the right pen test vendor 00:11:06.160 --> 00:11:11.290 right then have to have all of the extra time and grief and budget overrun in 00:11:11.290 --> 00:11:13.840 the assessment to go deal with the lousy pen test. 00:11:13.850 --> 00:11:14.850 Okay, now, 00:11:14.940 --> 00:11:17.750 getting the test is just one bit because there's a requirement that you don't 00:11:17.750 --> 00:11:20.180 have to fix things that are brought up in the pen test. 00:11:20.190 --> 00:11:21.370 So there's a queue. 00:11:21.370 --> 00:11:24.810 Say, when you do an assessment, you're looking for that fixing, right? 00:11:24.820 --> 00:11:25.380 Absolutely. 00:11:25.650 --> 00:11:27.860 And that's you're actually gonna go through. 00:11:27.860 --> 00:11:30.040 And so show me a retest. 00:11:30.050 --> 00:11:31.280 Show me a retest. 00:11:31.280 --> 00:11:32.410 Yeah, right. 00:11:32.420 --> 00:11:35.700 Uh, in some cases, right now, 00:11:35.710 --> 00:11:39.470 pen test vendors that are that are sort of moderately clever about this. 00:11:39.480 --> 00:11:43.300 Have some time limit on how long they will sit around waiting for you to 00:11:43.300 --> 00:11:46.680 request your retest as as the sort of tested entity. 00:11:46.690 --> 00:11:47.150 Right? 00:11:47.160 --> 00:11:48.930 And if you come back six or eight months later and say, 00:11:48.930 --> 00:11:51.280 Hey, I'm ready for a retest, they might say, 00:11:51.290 --> 00:11:52.560 That's nice, right? 00:11:52.570 --> 00:11:57.470 And that's fair because in many cases where I've been in the 00:11:57.470 --> 00:11:59.980 situation where the _________ vendor or will bend, 00:11:59.980 --> 00:12:03.480 the pen test vendor hasn't been sort of clever enough to cap that off. 00:12:03.490 --> 00:12:08.350 Right now, six or eight months have passed and the environment has changed.