Welcome to the video discovering hidden files and folders in this video we will use a command line tool
built into Kalli called Derbe to probe Web sites for hidden files and folders.
We'll use Derbez GUI equivalent called Derbe buster to further refine our scans and we'll also look
at another open source tool called Go buster which can not only hunt for content within Web site directories
but within domain names as well as we're finishing up on these final few videos of the course.
I hope you can see how far we've progressed in a short amount of time.
We really kind of started with nothing and went through building up a Kalli box from scratch installing
all the updates and then we spent a lot of time learning how to figure out what's on our network and
what services are running on various endpoints.
From there we tightened our scope even more and started to zero in on enumerating the fine details of
individual ports and tried to figure out what vulnerabilities might lie within what I'm hoping you have
felt is a gradual arc throughout this course where we focused mainly on networks scanning to start.
And now I'm trying what I'm the role of an ethical hacker someone who can think creatively and has the
skills and patience to deeply explore an endpoint and learn as much as possible in order to discover
vulnerabilities in the last video we looked at a tool called nikto to find low hanging vulnerabilities.
And in this video I want to start with a tool called Derb which can help you find files and folders
that are not otherwise directly visible or linked to from a site.
Before we launched Derbe I'm going to open up a site that will use a lot in this exercise and it's called
web scan.
Test dot com.
Now throughout this course I've said several times that you should not scan or pen test any site.
You don't have explicit permission for the nice thing about web scan test.
Is that it invites you to scan it.
The site actually exists to test vulnerability scanning tools so we can use it for our content discovery
searches as well.
Back at the command line.
Let's just type Derbe and see what we can learn about the tool.
As with many tools the output doesn't often fit to just one screen.
So let's do a Derbe.
And then the pipe space more to make it a little more digestible.
The syntax looks pretty basic.
Just use Derbe Plus a site and a wordlist file and then any other additional options we want to add.
Now as far as what you'll provide in the wordlist flag know that while there are tons of sample word
lists out on the Internet Cauley comes pre-installed with a large number as well you can sift through
these at slash us our slash share slash wordlist the rack you directx the list is an extremely popular
one with pen testers.
However for the sake of this demo I'm going to use a relatively short list so my scans go a bit faster.
So I'm going to call up Derbe.
And then the URL of my pluck V.M. followed by the full path to a short wordlist called small directx
TE and that lives in the slash us our share slash word lists slash Derbe directory
OK.
Let's take a look at Derbe help again and see a few other options that might help us be more precise
in our scanning.
We've got a task Z flag which can add a delay between requests which might be nice to do so we don't
overwhelm our host with traffic and then it looks like we can hit the key during our scan to see scan
statistics 0 and we'll probably want to output our scan to a file so we should try out the task o option
as well.
So let's throw this all together in one big command.
I will call up my last scan and add the taxi and then I'm going to do 500 to add a 500 millisecond delay
between requests and then I'm also going to add the tech 0 and create a log file just called log log
in as we can see Derbe is enforcing 500 millisecond delay and we're using our small data file.
And if I hit our While the scan runs I can get an idea of how many words are left to check from my wordlist.
That's cool.
And then I can also see that inside my current directory there is a log log file which has all the details
of my scan which is really important if I'm on a customer engagement and need to call up this data while
I'm writing the report.
Now for those of you who don't love tinkering around in the command line you can use Derbez goofy friend
called Derbe buster.
You can actually just type Derbe buster here at the command line and it'll pop right up.
And actually this is one instance where I personally like the gooey version because it gets me thinking
about things I should probably research before actually investing time in a scam.
For instance look over here in the file extension box THP is the default.
And essentially what that means is that as Der. Buster goes looking for files it's just going to look
for files with a ph extension not HVM mail or dot J.S. or dot ASAP or anything else.
If I bring up the web scan test site in my browser and then reference the handy web of lies or utility
we installed a few videos back we can see that.
Yeah this site is developed with PH P And so it's probably a safe bet that we can cross this site for
PH P files.
Another thing I really like about Buster is that it's multithreaded.
So we should see a performance boost in using it over Derbe.
Just to show a simple Derbe Buster example I'm going to have it start crawling for PH P files.
So in the target field here all do HTP colon slash slash web scan tests come in then I'm going to load
up my small about t s t list.
And then I'm going to choose to not check for directories only files and then I'll click Start.
Now if I click over to this result's tab I can watch the details come flooding in.
And as you can see there are all sorts of interesting looking file names that you'd want to explore
like config DEP DHP and log in duppy HP.
Now don't get me wrong.
Derbe and Derbe Buster are great tools.
I used them heavily when I was studying for some of my certifications but they're getting a bit long
in the tooth.
As a matter of fact if we click the help menu here in Derbe buster and click home page we're brought
to a site telling us that door buster is an interactive project and we should look for alternatives.
So what alternatives are out there.
Well the good news is there are tons of them.
The bad news is there are tons of them but I want to show you one.
I've been playing with lately called Go buster.
It's available on get hub.
It's actively maintained and it's a tool designed to brute force Web sites like we've been doing already.
Plus DNS subdomains as well.
And if you're not familiar with DNS subdomains We'll talk about them in just a few minutes.
Now one thing to know about.
Go buster.
Is that at the time of this recording.
It's a manual install so there are a few steps to get it running but it's not too tricky.
First I'm going to click the releases link here and see that go Buster recently released install packages
which makes our install process much easier.
Now you could just click the link and save this file somewhere in your machine.
But we're learning to be command line ninjas So let's do it the fun way.
Right.
I'm going to copy this link then dropped to a terminal window and make a new folder called slash Opt.
slash.
Go buster.
And then I'm going to issue the W get command plus the you are a we just copied.
Now I won't spend time in this course talking in depth about W get but if you do a W get space tac tac
help you can read all about it.
Essentially you could think of it as a tool to go out and get web resources for you from the command
line.
OK so the go Buster package is pulled down but it's got a weird dot 7 Z extension.
Some of you might recognize that as the seven zip extension and fortunately for us Kelly has a tool
to decompress these files built right into it.
All you need to do is type 7 z a space the space the name of the file you want to decompress and now
that it's decompressed we will see several files and folders.
The main one we want is just called Go buster.
So let's try to run this file and will see that we don't have permission.
That errors are a little misleading though as we do have permission.
The issue is that the file is not executable.
Or use a command called S.H. maade Plus the xx flag Plass dot slash go buster to make it executable.
I won't pause to get into too much detail on the S.H. mod command but again like many commands you can
type S.H. maade space tech tech help for lots more info.
Now let's try running go Buster again and.
Ah it looks much more promising.
So let's follow the basic.
Go Buster syntax here and run it against our test site.
So I'm going to do dot slash go buster.
Space.
You space the URL of web scan.
Test dot com space tech w.
And then the long path to our small Latics t wordlist file watching go Buster plug along.
I really like the speed of the scan as well as the simple interface that lets us know the percentage
of completion for each scan operation.
And just like many of the tools we've used in this course go Buster has a help menu that you can access
with TAC H.
We've got options here to save our output to a file display verbose output and so on.
But one feature I wanted to focus on is go Buster's ability to enumerate subdomains.
If you're not familiar Take a look at this.
Let's say we're going to do a google search.
So we load up w w w dot Google dot com.
In this example Google dot com is the domain and W W W is the subdomain.
And let's say we suddenly remember that we're expecting a super important email from the boss so we
get ready to log into our email and now notice that we're no longer at the W W W subdomain we were at
the accounts subdomain.
Now what you will sometimes find in your network scanning and penetration testing engagement's is that
companies can have subdomains for various services that they don't directly linked to on their web site
maybe because they don't want anybody but their I.T. security staff to know about them.
For instance I've found on several engagements a subdomain called phone company dot com that points
to a web interface of a company's VOIP phone system.
Or maybe someone in the I.T. or security team set up a temporary subdomain to point to a test or development
system somewhere in the cloud and that system is still out there and possibly full of vulnerabilities
but has been long forgotten.
Regardless of the reason being able to scan for these subdomains is really helpful for us as testers
and fortunately for us go Buster has that search functionality built right into it.
So from the command line again we'll run go buster and this time we'll add a TAC am space DNS This puts
go buster in DNS mode so it knows to look for DNS subdomains instead of files and folders off a Web
site like it usually does.
Then I'll add a tack you and what the heck let's use 7 M-S us as our target which is the domain name
for the seven minute security podcast.
Then we'll add attack w and point to a small word list and then we'll let it rip.
Again we get a nice indicator of how Go Buster scan is progressing and when it's done we've got some
interesting results.
It looks like seven M-S is likely running a blog some sort of development server a mail server something
called test a VPN and a main GWW site.
Since we're talking about subdomain reconnaissance Let me show you another tool that is handy for this
purpose DNS dumpster.
Dot com is a free site you can use to do subdomain enumeration.
So let's just type in seven M-S us again and we'll see what DNS dumpster comes up with in our results
here we get some information on the DNS servers for this domain.
The domains Amex anti-X ti records.
And then finally towards the bottom here we get the list of subdomains.
Note that none of the subdomains we saw with go Buster were found here but that's because the site only
collects information periodically.
For an example of a more thorough enumeration of domains.
Let's go back and search Microsoft dot com.
And as we can see from this result the list is really extensive.
All this information we've collected is a great jumping off point for further reconnaissance such as
kicking off and map scans on these hosts.
However keep in mind that these subdomains could point to third party host systems like Amazon AWOS
which require you to gain permission before scanning.
We talked about this in a past video this again stresses the importance of something we've talked about
several times and that is you need to make sure the scope of systems you'll be scanning and attacking
are agreed upon ahead of time both with your client and any necessary third parties.
In our next video we'll manually scan for web site vulnerabilities using burp.