Welcome to part two of this module this video is part two of the series dealing with burps sweet.
We're going to be covering the theory behind spider ring.
The purpose of this tutorial is to help you understand the spider ring process and how to go about configuring
it in burp suite.
There will be an extensive theory explanation here with a more detailed demonstration to follow towards
the end of the video.
This may not be a very exciting entry in the series and more advanced students were already familiar
with this material may wish to skip to the next module if they find the explanation to be too tedious
so before we get started with that I need to reiterate that the target we are going to be attacking
will be mutilate a web application which is pre installed on the Met splitter able machine that I've
been using for demonstration in most of these modules.
They met a split able to operating system is a very useful tool and if you are following along with
this tutorial I highly recommend using it for instructions on how to set it up as a virtual machine.
Please go back to Module 1 AND SEE THE APPROPRIATE VIDEO.
If you decide to follow along using some other tool Please make certain that it is located on a system
that you personally own or have written permission to test.
So we're going to be starting off with Matilda day.
As I said it comes pre installed with metal foible to which I have running as a virtual machine alongside
Kelly Linux which is also running as virtual machine.
So I'll just pull it up here please forgive the aspect ratio it's not possible to achieve a full screen
mode with met a split able and virtual box at least not as far as I know.
All we need to do here is get the IP address of the virtual machine using AI f config and we can see
that as in prior modules it is running on 10 0 0 8.
Yours may vary remember also that it is important to have virtual box configured to use the bridge adapter
if you have it set to Nat the two different instances of virtual box will both have the same IP address
and they won't be able to talk to each other.
It is very important to configure that correctly if you wish to follow along using this method.
All of this is of course assuming that you have chosen to follow along with this class using a virtual
box installation of Cally if you chose to install it either to a dedicated partition or a pen drive.
You may need to run virtual box within Cowley or on another computer if you wish to test against a split
able to under no circumstances do I recommend actually installing met a split able to to a real computer
on your network unless you're hosting hacker war games at Def Con or something.
This would be an exceedingly bad idea for what I hope are obvious reasons.
As an aside you could also get your meds Floyd label IP address in Cali using the Net discover command
as shown in previous modules provided that it's on your network.
Okay so we're going to open up our browser of choice and type in the IP address into the URL bar the
page will actually come up yet because of how we've configured our proxy settings in the prior module
which is to say if you remember we've gone into advanced connection settings and Firefox is currently
relying on burps sweet as a proxy so we're going to cancel out of this and we don't actually need this
up yet.
We're gonna load burp sweet
now while this is starting.
Let me explain what spider thing is.
The purpose of aspiring is to identify and map out what we want to scan.
This is not exactly scanning in the traditional sense though we'll be looking at that separately.
But spider ring is the process of mapping out our Web application and it is very useful for finding
links and web forms that are associated.
This is important for attacking web forms pages manipulating headers and so on.
When I talk about automatic spider ing with burp sweet What I'm essentially talking about is how when
burps spider ring it follows links to identify folders and forms within the web application itself.
It will record all of these requests and responses Well it is performing the whole spider ring process
so it looks like there's an update.
I'll do that later will once again select temporary project and we're going to go with brb default.
So now that we have burps sweet open let's have a look at it.
We'll go to the spider ring section and right away we see a simple menu with two tabs available.
We have a control tab and we have the options tab.
And this is the control tab.
Use these settings to monitor and control burp suite.
It allows you to start and stop the burp sweet spider thing and you can also clear the spider and Qs
the crawler setting allows us to specify the way in which the spider is going to crawl for the web content
on the web application itself.
We'll be looking at the maximum length depth and what that means passive spider ing allows us to continue
spider ing.
Well we are continuing to go through the web application and performing tests and getting responses
form submission is something we'll be looking at towards the end of the video once we start doing this
more practically the same with the application log in request headers are used to manipulate the H TTP
headers.
This is a more advanced subject.
We may be looking at that a bit more in a future video.
So back to the control tab.
Looking at spider scope can see that it will use the burps sweet default scope which is defined in the
target tab.
You can also use a custom scope or you can click the use advanced scope for right now we're just going
to stick with default
now here under the crawler settings I said I would speak about maximum length depth and this is probably
a good moment to do so I recommend that when you first start using burps sweet you leave the maximum
linked depth field alone.
This field specifies how many links you want the program to follow and by default the setting is on
five.
You might even consider dropping it down to three anything higher than five has a tendency to overload
the web application and causes it to lag significantly it will respond very very slowly.
That may not seem important at this stage the game but during a complex penetration test with a lot
of targets this can really drag down the whole process and give you diminishing returns in terms of
how much value you're really getting for the time spent using this tool.
It is up to you and if you feel this would make for a less thorough test than by all means increase
the maximum link depth.
But remember you were warned very high numbers are very likely to crash.
Older versions of burps sweet although this is less likely when run on a more modern system.
Now we'll talk a little bit about what passive spider ing is passive spider ing allows you to continue
scanning and performing your requests even as the spider ing process is continuing passive spider ring
monitors traffic through the proxy to update the site map without making any new requests.
You can toggle this on and off according to your needs.
I generally leave it on.
But some may find this muddies the waters a bit length depth to associate with proxy requests is also
a consideration.
It is zero by default and I do recommend keeping it between 0 and 3.
Again that is because the deeper you go the slower things get.
And if you are already set using a setting of five or higher for the maximum length depth of the spider
ing process itself you're likely to run into some major slowdowns.
Modern versions of Burke don't crash as readily as they did in the old days but overload the tool at
your own risk.
All right.
So with regards to form submission as I said we'll be looking at this a little more as we go along.
So we'll skip over all of that for now
as for the spider engine these are the settings to control the engine used for making H TTP requests
when spider ring.
We can see that by default the number of threads is set to 10.
I still recommend keeping this section between 3 to 5.
Otherwise you may find that the application slows down considerably.
You can adjust the number of retrials and pauses if you wish but this is really only relevant in situations
where the web application you are dealing with has some specific timing considerations built into it.
Now I did say that request headers are an advance subject but I should add a little more about what
they actually are.
You could use them to edit and change the request headers to manipulate the responses you get back.
For instance you could make the headers look as if they're coming from a mobile device instead of from
a personal computer.
Some applications behave quite differently depending on who or what they think is talking to them or
where from.
It's a complex subject.
As I said and rather advanced however it's worth pointing this out that this is where you would do it.
Alrighty.
That is the overall gist of the theory.
Now we'll be looking at how to actually go about the spider ing process with a direct demonstration.
Thank you for your patience.
I realize this was not the most interesting presentation to sit through but it is very important to
understand the basics of what is going on so the first thing we're going to do is go to the proxy tab
and we're going to turn intercept off.
This is because we aren't intercepting any requests or responses at the moment.
We'll pull up our web browser again
and we're going to refresh this page.
There we go.
Since burp sweet is listening we should be able to see what is going on on the device site map under
the target tab within the tool
having now pulled up the multiday site which is our target
we can see that something very interesting has just happened.
Our site map within the target tab has now been populated with folders and links associated with the
Web site that we're looking at in case you aren't clear at this point.
A site map is basically the format and structure of the web page in question and tells you a lot about
how the web page was constructed.
Now after all that talking we can finally make the thing do the thing.
Sorry bit a pen tester humor.
Our primary target is multiday so we write click on rather left click excuse me on our target and here
we see them until the day folder.
We also see other folders.
These are alternative targets that we could use for example the -- vulnerable web app.
It's a pretty good target but we're going to use motility so going to click on it right click and we're
going to select ad to scope
this allows us to conduct our spider ring in a focused manner with our target so selected it effectively
isolates all of the results that we're going to get.
So that we only see what we need to see without tons of pointless extraneous junk cluttering up the
report we're getting a box saying that we have added the target to the scope and it's asking us if we
want the proxy to stop sending out of scope items to the history or other burp suite tools we're going
to say yes here we want to make sure that we clear out all of the detritus that we don't need.
So now that we've done that we right click on motility again and we're going to select Spider this branch
finally something interesting is going to happen.
Burp sweet will begin gathering all of the information that it can further populating our site map.
We're going to get prompted with a submit form at some point.
And you can just ignore this.
There will likely be quite a few essentially what these are default log informs where it is asking you
to enter a username and password for links it has found that would require them in order to go deeper
with the spider ring process and build a deeper site map.
If you're conducting a white hat test you probably already have these credentials and you could enter
them one the dialog pops up if these were not provided you may have already have some idea what they
might be from the use of other tools such as for example multi go or other reconnaissance programs.
Here we go.
Better late than never.
This is an example of what I was talking about.
I'm just going to click Ignore form
and as I said this will come up a couple of times
so I'm going to come back over here to the spider tab.
Under control once the process is done you'll see that the requests made and bytes transferred will
stop adding up.
You can also click the button spider is running to pause the spider so now that the process is running
it's a good time to talk about some of these things I put off under the form submission tab.
You'll notice that it has all of the usual default fields that one would expect for various prompting
such as male first name last names surname etc. You can in fact change these values or add to them in
the case of unique targets if you wish.
The default values are mainly there as place holders and to verify the security of any form on the most
basic level
while we're waiting I'll also point out that if you wanted to add credentials automatically for all
of those instances where they're needed.
Those little pop up boxes rather than having to do it every single time you're prompted for it during
the process.
You can go down to application log in you can select the radial button to automatically submit whatever
credentials you enter here.
Changing the user name and password to whatever you think they might be.
All right.
So that should be enough for now.
I'm going to go back to control and I'm going to pause the spider ing process
and go back to our site map now at this point.
You might be asking.
I've seen a few reference sites and that's not particularly helpful.
Twitter hacks for charity backtrack.
Who cares I hear you ask.
We don't really need this information even though it gives us a basic idea of what sites are linked
to the web application.
It's still not very important information but if we click on the motility folder
I'll look very interesting.
It gives us the structure of the application and this is vitally important to know to go about penetrating
it.
You can inspect the styles read through the documents and understand what the person who is developing
the website was thinking and having this information really is the first step to actively exploiting
a system.
Of course the really juicy stuff the hidden files such as admin pages concealed log and pages and so
on are not supported features with the burp suite community additions.
If you see the potential here and you do a lot of penetration testing of Web applications this is one
more reason to consider getting the licensed version.
OK we have Spider the application we have the structure of what it looks like.
The web application itself.
Let me just wrap up this tutorial by showing you how to get rid of all of the extraneous dross that
you don't need to show items in the scope only just click on the filter bar
and click show only in scope items.
Click on the filter bar again
and while law.
There we go.
The filters just got rid of all the junk and we can look only at the requests and responses that were
defined to our scope filters just make life easier especially when you are first starting out because
it gets rid of all the confusing unnecessary information that this tool presents that you just don't
need.
Last but not least remember that you can click the re enable button at the top of the screen anytime
you want to re enable logging of out of scope traffic through the proxy so I hope you have found this
tutorial helpful and that you have a foundational knowledge with which to conduct base expiring.
As always when you're familiarity with the process has grown the small particulars should start to make
more sense.
I admit this was a lot of theory for what in the end was a very simple procedure but it is the first
step on web application penetration testing and I did not wish to skimp on it.
Thank you very much for your patience.