WEBVTT

00:01.880 --> 00:09.260
Welcome to the video discovering hidden files and folders in this video we will use a command line tool

00:09.260 --> 00:14.540
built into Kalli called Derbe to probe Web sites for hidden files and folders.

00:15.690 --> 00:22.280
We'll use Derbez GUI equivalent called Derbe buster to further refine our scans and we'll also look

00:22.280 --> 00:29.120
at another open source tool called Go buster which can not only hunt for content within Web site directories

00:29.450 --> 00:35.240
but within domain names as well as we're finishing up on these final few videos of the course.

00:35.240 --> 00:39.980
I hope you can see how far we've progressed in a short amount of time.

00:40.010 --> 00:46.040
We really kind of started with nothing and went through building up a Kalli box from scratch installing

00:46.040 --> 00:51.260
all the updates and then we spent a lot of time learning how to figure out what's on our network and

00:51.260 --> 00:54.710
what services are running on various endpoints.

00:54.790 --> 01:00.760
From there we tightened our scope even more and started to zero in on enumerating the fine details of

01:00.760 --> 01:07.420
individual ports and tried to figure out what vulnerabilities might lie within what I'm hoping you have

01:07.420 --> 01:13.820
felt is a gradual arc throughout this course where we focused mainly on networks scanning to start.

01:14.170 --> 01:21.040
And now I'm trying what I'm the role of an ethical hacker someone who can think creatively and has the

01:21.040 --> 01:27.490
skills and patience to deeply explore an endpoint and learn as much as possible in order to discover

01:27.490 --> 01:34.800
vulnerabilities in the last video we looked at a tool called nikto to find low hanging vulnerabilities.

01:34.800 --> 01:40.710
And in this video I want to start with a tool called Derb which can help you find files and folders

01:40.920 --> 01:46.510
that are not otherwise directly visible or linked to from a site.

01:46.730 --> 01:52.340
Before we launched Derbe I'm going to open up a site that will use a lot in this exercise and it's called

01:52.490 --> 01:53.440
web scan.

01:53.440 --> 01:56.200
Test dot com.

01:56.210 --> 02:01.110
Now throughout this course I've said several times that you should not scan or pen test any site.

02:01.120 --> 02:06.410
You don't have explicit permission for the nice thing about web scan test.

02:06.530 --> 02:09.050
Is that it invites you to scan it.

02:09.080 --> 02:15.740
The site actually exists to test vulnerability scanning tools so we can use it for our content discovery

02:15.740 --> 02:17.730
searches as well.

02:17.790 --> 02:18.800
Back at the command line.

02:18.810 --> 02:22.510
Let's just type Derbe and see what we can learn about the tool.

02:23.760 --> 02:28.430
As with many tools the output doesn't often fit to just one screen.

02:28.530 --> 02:30.110
So let's do a Derbe.

02:30.510 --> 02:35.570
And then the pipe space more to make it a little more digestible.

02:36.880 --> 02:38.810
The syntax looks pretty basic.

02:38.920 --> 02:46.900
Just use Derbe Plus a site and a wordlist file and then any other additional options we want to add.

02:49.270 --> 02:54.830
Now as far as what you'll provide in the wordlist flag know that while there are tons of sample word

02:54.870 --> 03:01.400
lists out on the Internet Cauley comes pre-installed with a large number as well you can sift through

03:01.400 --> 03:12.250
these at slash us our slash share slash wordlist the rack you directx the list is an extremely popular

03:12.250 --> 03:14.870
one with pen testers.

03:14.950 --> 03:21.200
However for the sake of this demo I'm going to use a relatively short list so my scans go a bit faster.

03:22.880 --> 03:24.680
So I'm going to call up Derbe.

03:25.100 --> 03:34.740
And then the URL of my pluck V.M. followed by the full path to a short wordlist called small directx

03:34.740 --> 03:43.600
TE and that lives in the slash us our share slash word lists slash Derbe directory

03:46.310 --> 03:46.590
OK.

03:46.580 --> 03:53.130
Let's take a look at Derbe help again and see a few other options that might help us be more precise

03:53.130 --> 03:55.130
in our scanning.

03:55.170 --> 04:02.310
We've got a task Z flag which can add a delay between requests which might be nice to do so we don't

04:02.370 --> 04:09.790
overwhelm our host with traffic and then it looks like we can hit the key during our scan to see scan

04:09.790 --> 04:17.560
statistics 0 and we'll probably want to output our scan to a file so we should try out the task o option

04:17.680 --> 04:18.380
as well.

04:19.210 --> 04:22.340
So let's throw this all together in one big command.

04:22.690 --> 04:31.840
I will call up my last scan and add the taxi and then I'm going to do 500 to add a 500 millisecond delay

04:31.840 --> 04:40.610
between requests and then I'm also going to add the tech 0 and create a log file just called log log

04:42.760 --> 04:50.730
in as we can see Derbe is enforcing 500 millisecond delay and we're using our small data file.

04:51.180 --> 04:58.960
And if I hit our While the scan runs I can get an idea of how many words are left to check from my wordlist.

04:59.060 --> 04:59.910
That's cool.

05:00.290 --> 05:06.770
And then I can also see that inside my current directory there is a log log file which has all the details

05:06.770 --> 05:12.440
of my scan which is really important if I'm on a customer engagement and need to call up this data while

05:12.440 --> 05:13.610
I'm writing the report.

05:14.730 --> 05:20.700
Now for those of you who don't love tinkering around in the command line you can use Derbez goofy friend

05:20.940 --> 05:23.210
called Derbe buster.

05:23.290 --> 05:29.610
You can actually just type Derbe buster here at the command line and it'll pop right up.

05:29.840 --> 05:34.970
And actually this is one instance where I personally like the gooey version because it gets me thinking

05:34.970 --> 05:39.900
about things I should probably research before actually investing time in a scam.

05:41.490 --> 05:47.650
For instance look over here in the file extension box THP is the default.

05:47.710 --> 05:53.500
And essentially what that means is that as Der. Buster goes looking for files it's just going to look

05:53.500 --> 06:01.540
for files with a ph extension not HVM mail or dot J.S. or dot ASAP or anything else.

06:01.540 --> 06:08.530
If I bring up the web scan test site in my browser and then reference the handy web of lies or utility

06:08.540 --> 06:11.820
we installed a few videos back we can see that.

06:11.840 --> 06:18.030
Yeah this site is developed with PH P And so it's probably a safe bet that we can cross this site for

06:18.040 --> 06:19.390
PH P files.

06:20.510 --> 06:24.030
Another thing I really like about Buster is that it's multithreaded.

06:24.140 --> 06:28.440
So we should see a performance boost in using it over Derbe.

06:29.840 --> 06:36.070
Just to show a simple Derbe Buster example I'm going to have it start crawling for PH P files.

06:36.560 --> 06:46.890
So in the target field here all do HTP colon slash slash web scan tests come in then I'm going to load

06:46.890 --> 06:49.170
up my small about t s t list.

06:50.810 --> 07:02.470
And then I'm going to choose to not check for directories only files and then I'll click Start.

07:02.480 --> 07:07.080
Now if I click over to this result's tab I can watch the details come flooding in.

07:07.270 --> 07:11.780
And as you can see there are all sorts of interesting looking file names that you'd want to explore

07:12.440 --> 07:17.790
like config DEP DHP and log in duppy HP.

07:17.810 --> 07:18.670
Now don't get me wrong.

07:18.680 --> 07:21.690
Derbe and Derbe Buster are great tools.

07:21.710 --> 07:26.930
I used them heavily when I was studying for some of my certifications but they're getting a bit long

07:26.930 --> 07:27.840
in the tooth.

07:27.980 --> 07:34.640
As a matter of fact if we click the help menu here in Derbe buster and click home page we're brought

07:34.640 --> 07:40.640
to a site telling us that door buster is an interactive project and we should look for alternatives.

07:41.030 --> 07:42.870
So what alternatives are out there.

07:43.160 --> 07:46.010
Well the good news is there are tons of them.

07:46.010 --> 07:50.010
The bad news is there are tons of them but I want to show you one.

07:50.030 --> 07:53.660
I've been playing with lately called Go buster.

07:53.720 --> 07:55.280
It's available on get hub.

07:55.580 --> 08:01.990
It's actively maintained and it's a tool designed to brute force Web sites like we've been doing already.

08:02.210 --> 08:05.470
Plus DNS subdomains as well.

08:05.610 --> 08:09.380
And if you're not familiar with DNS subdomains We'll talk about them in just a few minutes.

08:10.160 --> 08:11.120
Now one thing to know about.

08:11.120 --> 08:11.810
Go buster.

08:11.810 --> 08:13.720
Is that at the time of this recording.

08:13.730 --> 08:20.330
It's a manual install so there are a few steps to get it running but it's not too tricky.

08:20.380 --> 08:27.040
First I'm going to click the releases link here and see that go Buster recently released install packages

08:27.820 --> 08:30.280
which makes our install process much easier.

08:31.500 --> 08:35.370
Now you could just click the link and save this file somewhere in your machine.

08:35.370 --> 08:39.410
But we're learning to be command line ninjas So let's do it the fun way.

08:39.410 --> 08:40.650
Right.

08:40.760 --> 08:47.470
I'm going to copy this link then dropped to a terminal window and make a new folder called slash Opt.

08:47.710 --> 08:48.290
slash.

08:48.320 --> 08:51.400
Go buster.

08:51.540 --> 08:57.060
And then I'm going to issue the W get command plus the you are a we just copied.

08:57.100 --> 09:04.090
Now I won't spend time in this course talking in depth about W get but if you do a W get space tac tac

09:04.090 --> 09:06.770
help you can read all about it.

09:06.790 --> 09:12.040
Essentially you could think of it as a tool to go out and get web resources for you from the command

09:12.040 --> 09:12.570
line.

09:12.880 --> 09:20.120
OK so the go Buster package is pulled down but it's got a weird dot 7 Z extension.

09:20.160 --> 09:25.710
Some of you might recognize that as the seven zip extension and fortunately for us Kelly has a tool

09:25.740 --> 09:28.280
to decompress these files built right into it.

09:29.400 --> 09:40.440
All you need to do is type 7 z a space the space the name of the file you want to decompress and now

09:40.440 --> 09:43.790
that it's decompressed we will see several files and folders.

09:44.070 --> 09:47.140
The main one we want is just called Go buster.

09:47.160 --> 09:51.640
So let's try to run this file and will see that we don't have permission.

09:52.400 --> 09:57.150
That errors are a little misleading though as we do have permission.

09:57.200 --> 09:59.800
The issue is that the file is not executable.

09:59.990 --> 10:10.030
Or use a command called S.H. maade Plus the xx flag Plass dot slash go buster to make it executable.

10:10.870 --> 10:17.020
I won't pause to get into too much detail on the S.H. mod command but again like many commands you can

10:17.020 --> 10:22.170
type S.H. maade space tech tech help for lots more info.

10:22.180 --> 10:24.610
Now let's try running go Buster again and.

10:24.660 --> 10:27.280
Ah it looks much more promising.

10:28.480 --> 10:29.710
So let's follow the basic.

10:29.710 --> 10:34.000
Go Buster syntax here and run it against our test site.

10:34.090 --> 10:36.670
So I'm going to do dot slash go buster.

10:36.850 --> 10:37.720
Space.

10:37.750 --> 10:41.320
You space the URL of web scan.

10:41.330 --> 10:45.550
Test dot com space tech w.

10:45.840 --> 10:54.380
And then the long path to our small Latics t wordlist file watching go Buster plug along.

10:54.380 --> 10:59.990
I really like the speed of the scan as well as the simple interface that lets us know the percentage

10:59.990 --> 11:02.740
of completion for each scan operation.

11:03.910 --> 11:09.290
And just like many of the tools we've used in this course go Buster has a help menu that you can access

11:09.290 --> 11:12.570
with TAC H.

11:12.660 --> 11:18.480
We've got options here to save our output to a file display verbose output and so on.

11:18.600 --> 11:24.630
But one feature I wanted to focus on is go Buster's ability to enumerate subdomains.

11:24.720 --> 11:27.740
If you're not familiar Take a look at this.

11:27.760 --> 11:29.740
Let's say we're going to do a google search.

11:29.800 --> 11:34.220
So we load up w w w dot Google dot com.

11:34.400 --> 11:42.230
In this example Google dot com is the domain and W W W is the subdomain.

11:42.370 --> 11:48.160
And let's say we suddenly remember that we're expecting a super important email from the boss so we

11:48.310 --> 11:55.650
get ready to log into our email and now notice that we're no longer at the W W W subdomain we were at

11:55.650 --> 11:57.870
the accounts subdomain.

11:57.870 --> 12:03.150
Now what you will sometimes find in your network scanning and penetration testing engagement's is that

12:03.150 --> 12:09.700
companies can have subdomains for various services that they don't directly linked to on their web site

12:10.470 --> 12:15.130
maybe because they don't want anybody but their I.T. security staff to know about them.

12:16.160 --> 12:23.600
For instance I've found on several engagements a subdomain called phone company dot com that points

12:23.600 --> 12:27.090
to a web interface of a company's VOIP phone system.

12:28.310 --> 12:34.610
Or maybe someone in the I.T. or security team set up a temporary subdomain to point to a test or development

12:34.610 --> 12:40.460
system somewhere in the cloud and that system is still out there and possibly full of vulnerabilities

12:40.880 --> 12:42.470
but has been long forgotten.

12:43.520 --> 12:48.950
Regardless of the reason being able to scan for these subdomains is really helpful for us as testers

12:49.440 --> 12:54.340
and fortunately for us go Buster has that search functionality built right into it.

12:55.070 --> 13:03.680
So from the command line again we'll run go buster and this time we'll add a TAC am space DNS This puts

13:03.680 --> 13:10.470
go buster in DNS mode so it knows to look for DNS subdomains instead of files and folders off a Web

13:10.470 --> 13:11.880
site like it usually does.

13:13.140 --> 13:21.360
Then I'll add a tack you and what the heck let's use 7 M-S us as our target which is the domain name

13:21.360 --> 13:23.750
for the seven minute security podcast.

13:25.280 --> 13:31.100
Then we'll add attack w and point to a small word list and then we'll let it rip.

13:32.460 --> 13:38.520
Again we get a nice indicator of how Go Buster scan is progressing and when it's done we've got some

13:38.580 --> 13:39.900
interesting results.

13:40.930 --> 13:49.150
It looks like seven M-S is likely running a blog some sort of development server a mail server something

13:49.150 --> 13:54.650
called test a VPN and a main GWW site.

13:54.840 --> 13:59.640
Since we're talking about subdomain reconnaissance Let me show you another tool that is handy for this

13:59.640 --> 14:02.770
purpose DNS dumpster.

14:02.770 --> 14:08.550
Dot com is a free site you can use to do subdomain enumeration.

14:08.570 --> 14:16.460
So let's just type in seven M-S us again and we'll see what DNS dumpster comes up with in our results

14:16.470 --> 14:20.580
here we get some information on the DNS servers for this domain.

14:21.370 --> 14:25.540
The domains Amex anti-X ti records.

14:25.750 --> 14:29.440
And then finally towards the bottom here we get the list of subdomains.

14:29.440 --> 14:34.840
Note that none of the subdomains we saw with go Buster were found here but that's because the site only

14:34.840 --> 14:36.710
collects information periodically.

14:37.870 --> 14:41.870
For an example of a more thorough enumeration of domains.

14:41.920 --> 14:46.800
Let's go back and search Microsoft dot com.

14:46.990 --> 14:51.930
And as we can see from this result the list is really extensive.

14:52.370 --> 14:57.860
All this information we've collected is a great jumping off point for further reconnaissance such as

14:57.860 --> 15:01.280
kicking off and map scans on these hosts.

15:01.290 --> 15:08.280
However keep in mind that these subdomains could point to third party host systems like Amazon AWOS

15:08.370 --> 15:11.960
which require you to gain permission before scanning.

15:11.970 --> 15:17.690
We talked about this in a past video this again stresses the importance of something we've talked about

15:17.690 --> 15:23.960
several times and that is you need to make sure the scope of systems you'll be scanning and attacking

15:24.410 --> 15:29.920
are agreed upon ahead of time both with your client and any necessary third parties.

15:32.340 --> 15:37.260
In our next video we'll manually scan for web site vulnerabilities using burp.
