1
00:00:00,240 --> 00:00:09,230
Welcome to part two of this module this video is part two of the series dealing with burps sweet.

2
00:00:09,410 --> 00:00:12,830
We're going to be covering the theory behind spider ring.

3
00:00:12,920 --> 00:00:19,580
The purpose of this tutorial is to help you understand the spider ring process and how to go about configuring

4
00:00:19,580 --> 00:00:21,160
it in burp suite.

5
00:00:21,170 --> 00:00:28,310
There will be an extensive theory explanation here with a more detailed demonstration to follow towards

6
00:00:28,310 --> 00:00:29,880
the end of the video.

7
00:00:29,900 --> 00:00:36,380
This may not be a very exciting entry in the series and more advanced students were already familiar

8
00:00:36,380 --> 00:00:42,380
with this material may wish to skip to the next module if they find the explanation to be too tedious

9
00:00:43,650 --> 00:00:49,470
so before we get started with that I need to reiterate that the target we are going to be attacking

10
00:00:49,470 --> 00:00:57,100
will be mutilate a web application which is pre installed on the Met splitter able machine that I've

11
00:00:57,150 --> 00:01:01,310
been using for demonstration in most of these modules.

12
00:01:01,420 --> 00:01:06,720
They met a split able to operating system is a very useful tool and if you are following along with

13
00:01:06,720 --> 00:01:14,020
this tutorial I highly recommend using it for instructions on how to set it up as a virtual machine.

14
00:01:14,040 --> 00:01:18,160
Please go back to Module 1 AND SEE THE APPROPRIATE VIDEO.

15
00:01:18,510 --> 00:01:25,320
If you decide to follow along using some other tool Please make certain that it is located on a system

16
00:01:25,320 --> 00:01:28,890
that you personally own or have written permission to test.

17
00:01:28,920 --> 00:01:31,740
So we're going to be starting off with Matilda day.

18
00:01:31,830 --> 00:01:39,090
As I said it comes pre installed with metal foible to which I have running as a virtual machine alongside

19
00:01:39,090 --> 00:01:42,510
Kelly Linux which is also running as virtual machine.

20
00:01:42,990 --> 00:01:50,460
So I'll just pull it up here please forgive the aspect ratio it's not possible to achieve a full screen

21
00:01:50,460 --> 00:01:54,590
mode with met a split able and virtual box at least not as far as I know.

22
00:01:55,410 --> 00:02:03,400
All we need to do here is get the IP address of the virtual machine using AI f config and we can see

23
00:02:03,400 --> 00:02:08,980
that as in prior modules it is running on 10 0 0 8.

24
00:02:08,980 --> 00:02:16,600
Yours may vary remember also that it is important to have virtual box configured to use the bridge adapter

25
00:02:17,200 --> 00:02:24,010
if you have it set to Nat the two different instances of virtual box will both have the same IP address

26
00:02:24,520 --> 00:02:27,320
and they won't be able to talk to each other.

27
00:02:27,460 --> 00:02:32,590
It is very important to configure that correctly if you wish to follow along using this method.

28
00:02:32,620 --> 00:02:38,680
All of this is of course assuming that you have chosen to follow along with this class using a virtual

29
00:02:38,680 --> 00:02:46,480
box installation of Cally if you chose to install it either to a dedicated partition or a pen drive.

30
00:02:46,480 --> 00:02:52,420
You may need to run virtual box within Cowley or on another computer if you wish to test against a split

31
00:02:52,420 --> 00:02:59,770
able to under no circumstances do I recommend actually installing met a split able to to a real computer

32
00:02:59,770 --> 00:03:05,370
on your network unless you're hosting hacker war games at Def Con or something.

33
00:03:05,410 --> 00:03:10,450
This would be an exceedingly bad idea for what I hope are obvious reasons.

34
00:03:10,450 --> 00:03:17,470
As an aside you could also get your meds Floyd label IP address in Cali using the Net discover command

35
00:03:17,980 --> 00:03:22,720
as shown in previous modules provided that it's on your network.

36
00:03:22,750 --> 00:03:31,730
Okay so we're going to open up our browser of choice and type in the IP address into the URL bar the

37
00:03:31,730 --> 00:03:37,280
page will actually come up yet because of how we've configured our proxy settings in the prior module

38
00:03:38,400 --> 00:03:47,160
which is to say if you remember we've gone into advanced connection settings and Firefox is currently

39
00:03:47,160 --> 00:03:56,040
relying on burps sweet as a proxy so we're going to cancel out of this and we don't actually need this

40
00:03:56,100 --> 00:03:56,600
up yet.

41
00:03:56,610 --> 00:03:57,900
We're gonna load burp sweet

42
00:04:06,490 --> 00:04:07,660
now while this is starting.

43
00:04:07,660 --> 00:04:10,150
Let me explain what spider thing is.

44
00:04:10,300 --> 00:04:15,510
The purpose of aspiring is to identify and map out what we want to scan.

45
00:04:15,520 --> 00:04:21,180
This is not exactly scanning in the traditional sense though we'll be looking at that separately.

46
00:04:21,280 --> 00:04:28,720
But spider ring is the process of mapping out our Web application and it is very useful for finding

47
00:04:28,720 --> 00:04:32,710
links and web forms that are associated.

48
00:04:32,710 --> 00:04:38,990
This is important for attacking web forms pages manipulating headers and so on.

49
00:04:39,370 --> 00:04:46,540
When I talk about automatic spider ing with burp sweet What I'm essentially talking about is how when

50
00:04:46,840 --> 00:04:54,770
burps spider ring it follows links to identify folders and forms within the web application itself.

51
00:04:54,940 --> 00:05:01,120
It will record all of these requests and responses Well it is performing the whole spider ring process

52
00:05:02,830 --> 00:05:04,930
so it looks like there's an update.

53
00:05:05,080 --> 00:05:19,680
I'll do that later will once again select temporary project and we're going to go with brb default.

54
00:05:19,840 --> 00:05:23,830
So now that we have burps sweet open let's have a look at it.

55
00:05:25,160 --> 00:05:34,400
We'll go to the spider ring section and right away we see a simple menu with two tabs available.

56
00:05:34,440 --> 00:05:38,740
We have a control tab and we have the options tab.

57
00:05:39,000 --> 00:05:41,150
And this is the control tab.

58
00:05:41,160 --> 00:05:44,740
Use these settings to monitor and control burp suite.

59
00:05:44,760 --> 00:05:51,420
It allows you to start and stop the burp sweet spider thing and you can also clear the spider and Qs

60
00:05:54,690 --> 00:06:02,850
the crawler setting allows us to specify the way in which the spider is going to crawl for the web content

61
00:06:02,880 --> 00:06:05,410
on the web application itself.

62
00:06:05,430 --> 00:06:14,370
We'll be looking at the maximum length depth and what that means passive spider ing allows us to continue

63
00:06:14,370 --> 00:06:15,110
spider ing.

64
00:06:15,120 --> 00:06:21,300
Well we are continuing to go through the web application and performing tests and getting responses

65
00:06:24,820 --> 00:06:30,490
form submission is something we'll be looking at towards the end of the video once we start doing this

66
00:06:30,500 --> 00:06:40,900
more practically the same with the application log in request headers are used to manipulate the H TTP

67
00:06:40,900 --> 00:06:42,340
headers.

68
00:06:42,400 --> 00:06:45,020
This is a more advanced subject.

69
00:06:45,400 --> 00:06:48,970
We may be looking at that a bit more in a future video.

70
00:06:49,810 --> 00:06:51,790
So back to the control tab.

71
00:06:51,790 --> 00:06:58,510
Looking at spider scope can see that it will use the burps sweet default scope which is defined in the

72
00:06:58,510 --> 00:07:00,160
target tab.

73
00:07:00,160 --> 00:07:08,700
You can also use a custom scope or you can click the use advanced scope for right now we're just going

74
00:07:08,700 --> 00:07:09,810
to stick with default

75
00:07:20,840 --> 00:07:27,900
now here under the crawler settings I said I would speak about maximum length depth and this is probably

76
00:07:27,900 --> 00:07:34,950
a good moment to do so I recommend that when you first start using burps sweet you leave the maximum

77
00:07:34,950 --> 00:07:37,800
linked depth field alone.

78
00:07:37,800 --> 00:07:44,190
This field specifies how many links you want the program to follow and by default the setting is on

79
00:07:44,190 --> 00:07:44,810
five.

80
00:07:44,820 --> 00:07:52,250
You might even consider dropping it down to three anything higher than five has a tendency to overload

81
00:07:52,280 --> 00:07:59,930
the web application and causes it to lag significantly it will respond very very slowly.

82
00:08:00,350 --> 00:08:06,560
That may not seem important at this stage the game but during a complex penetration test with a lot

83
00:08:06,560 --> 00:08:13,250
of targets this can really drag down the whole process and give you diminishing returns in terms of

84
00:08:13,250 --> 00:08:18,800
how much value you're really getting for the time spent using this tool.

85
00:08:18,800 --> 00:08:25,640
It is up to you and if you feel this would make for a less thorough test than by all means increase

86
00:08:25,640 --> 00:08:26,980
the maximum link depth.

87
00:08:27,560 --> 00:08:34,210
But remember you were warned very high numbers are very likely to crash.

88
00:08:34,250 --> 00:08:41,330
Older versions of burps sweet although this is less likely when run on a more modern system.

89
00:08:41,360 --> 00:08:47,450
Now we'll talk a little bit about what passive spider ing is passive spider ing allows you to continue

90
00:08:47,450 --> 00:08:54,140
scanning and performing your requests even as the spider ing process is continuing passive spider ring

91
00:08:54,170 --> 00:09:00,800
monitors traffic through the proxy to update the site map without making any new requests.

92
00:09:00,920 --> 00:09:04,860
You can toggle this on and off according to your needs.

93
00:09:04,880 --> 00:09:06,700
I generally leave it on.

94
00:09:06,710 --> 00:09:15,110
But some may find this muddies the waters a bit length depth to associate with proxy requests is also

95
00:09:15,110 --> 00:09:16,980
a consideration.

96
00:09:17,120 --> 00:09:22,410
It is zero by default and I do recommend keeping it between 0 and 3.

97
00:09:22,610 --> 00:09:26,360
Again that is because the deeper you go the slower things get.

98
00:09:26,360 --> 00:09:33,580
And if you are already set using a setting of five or higher for the maximum length depth of the spider

99
00:09:33,580 --> 00:09:38,760
ing process itself you're likely to run into some major slowdowns.

100
00:09:39,170 --> 00:09:44,630
Modern versions of Burke don't crash as readily as they did in the old days but overload the tool at

101
00:09:44,630 --> 00:09:45,980
your own risk.

102
00:09:47,200 --> 00:09:47,620
All right.

103
00:09:47,650 --> 00:09:54,430
So with regards to form submission as I said we'll be looking at this a little more as we go along.

104
00:09:55,720 --> 00:09:58,320
So we'll skip over all of that for now

105
00:10:03,420 --> 00:10:11,220
as for the spider engine these are the settings to control the engine used for making H TTP requests

106
00:10:11,220 --> 00:10:13,020
when spider ring.

107
00:10:13,170 --> 00:10:17,800
We can see that by default the number of threads is set to 10.

108
00:10:17,820 --> 00:10:21,820
I still recommend keeping this section between 3 to 5.

109
00:10:22,050 --> 00:10:26,810
Otherwise you may find that the application slows down considerably.

110
00:10:26,940 --> 00:10:33,990
You can adjust the number of retrials and pauses if you wish but this is really only relevant in situations

111
00:10:33,990 --> 00:10:41,810
where the web application you are dealing with has some specific timing considerations built into it.

112
00:10:41,820 --> 00:10:48,000
Now I did say that request headers are an advance subject but I should add a little more about what

113
00:10:48,000 --> 00:10:49,950
they actually are.

114
00:10:50,250 --> 00:10:57,810
You could use them to edit and change the request headers to manipulate the responses you get back.

115
00:10:57,810 --> 00:11:05,190
For instance you could make the headers look as if they're coming from a mobile device instead of from

116
00:11:05,190 --> 00:11:07,190
a personal computer.

117
00:11:07,200 --> 00:11:13,770
Some applications behave quite differently depending on who or what they think is talking to them or

118
00:11:13,770 --> 00:11:15,640
where from.

119
00:11:15,660 --> 00:11:17,290
It's a complex subject.

120
00:11:17,340 --> 00:11:24,630
As I said and rather advanced however it's worth pointing this out that this is where you would do it.

121
00:11:25,320 --> 00:11:26,070
Alrighty.

122
00:11:26,180 --> 00:11:28,910
That is the overall gist of the theory.

123
00:11:28,910 --> 00:11:35,000
Now we'll be looking at how to actually go about the spider ing process with a direct demonstration.

124
00:11:35,000 --> 00:11:36,260
Thank you for your patience.

125
00:11:36,260 --> 00:11:43,640
I realize this was not the most interesting presentation to sit through but it is very important to

126
00:11:43,640 --> 00:11:51,850
understand the basics of what is going on so the first thing we're going to do is go to the proxy tab

127
00:11:52,360 --> 00:11:56,270
and we're going to turn intercept off.

128
00:11:56,280 --> 00:12:00,620
This is because we aren't intercepting any requests or responses at the moment.

129
00:12:00,900 --> 00:12:02,700
We'll pull up our web browser again

130
00:12:06,030 --> 00:12:11,750
and we're going to refresh this page.

131
00:12:11,760 --> 00:12:12,390
There we go.

132
00:12:14,450 --> 00:12:20,060
Since burp sweet is listening we should be able to see what is going on on the device site map under

133
00:12:20,060 --> 00:12:22,060
the target tab within the tool

134
00:12:30,640 --> 00:12:34,210
having now pulled up the multiday site which is our target

135
00:12:37,030 --> 00:12:40,300
we can see that something very interesting has just happened.

136
00:12:40,300 --> 00:12:47,950
Our site map within the target tab has now been populated with folders and links associated with the

137
00:12:47,950 --> 00:12:51,910
Web site that we're looking at in case you aren't clear at this point.

138
00:12:51,910 --> 00:12:58,660
A site map is basically the format and structure of the web page in question and tells you a lot about

139
00:12:58,660 --> 00:13:01,320
how the web page was constructed.

140
00:13:01,330 --> 00:13:05,800
Now after all that talking we can finally make the thing do the thing.

141
00:13:06,100 --> 00:13:08,180
Sorry bit a pen tester humor.

142
00:13:08,260 --> 00:13:17,780
Our primary target is multiday so we write click on rather left click excuse me on our target and here

143
00:13:17,780 --> 00:13:19,850
we see them until the day folder.

144
00:13:19,880 --> 00:13:21,230
We also see other folders.

145
00:13:21,230 --> 00:13:26,870
These are alternative targets that we could use for example the -- vulnerable web app.

146
00:13:26,900 --> 00:13:36,860
It's a pretty good target but we're going to use motility so going to click on it right click and we're

147
00:13:36,860 --> 00:13:38,990
going to select ad to scope

148
00:13:41,760 --> 00:13:49,320
this allows us to conduct our spider ring in a focused manner with our target so selected it effectively

149
00:13:49,410 --> 00:13:52,210
isolates all of the results that we're going to get.

150
00:13:52,560 --> 00:13:57,780
So that we only see what we need to see without tons of pointless extraneous junk cluttering up the

151
00:13:57,780 --> 00:14:06,600
report we're getting a box saying that we have added the target to the scope and it's asking us if we

152
00:14:06,600 --> 00:14:14,580
want the proxy to stop sending out of scope items to the history or other burp suite tools we're going

153
00:14:14,580 --> 00:14:22,170
to say yes here we want to make sure that we clear out all of the detritus that we don't need.

154
00:14:22,260 --> 00:14:29,170
So now that we've done that we right click on motility again and we're going to select Spider this branch

155
00:14:38,000 --> 00:14:40,580
finally something interesting is going to happen.

156
00:14:40,760 --> 00:14:48,140
Burp sweet will begin gathering all of the information that it can further populating our site map.

157
00:14:48,230 --> 00:14:53,350
We're going to get prompted with a submit form at some point.

158
00:14:55,680 --> 00:14:57,530
And you can just ignore this.

159
00:14:57,550 --> 00:15:06,850
There will likely be quite a few essentially what these are default log informs where it is asking you

160
00:15:06,850 --> 00:15:13,690
to enter a username and password for links it has found that would require them in order to go deeper

161
00:15:13,690 --> 00:15:18,370
with the spider ring process and build a deeper site map.

162
00:15:18,370 --> 00:15:23,980
If you're conducting a white hat test you probably already have these credentials and you could enter

163
00:15:23,980 --> 00:15:31,210
them one the dialog pops up if these were not provided you may have already have some idea what they

164
00:15:31,210 --> 00:15:39,680
might be from the use of other tools such as for example multi go or other reconnaissance programs.

165
00:15:39,700 --> 00:15:40,210
Here we go.

166
00:15:40,210 --> 00:15:41,410
Better late than never.

167
00:15:41,410 --> 00:15:44,160
This is an example of what I was talking about.

168
00:15:44,200 --> 00:15:46,300
I'm just going to click Ignore form

169
00:15:49,020 --> 00:15:51,780
and as I said this will come up a couple of times

170
00:15:56,880 --> 00:15:59,240
so I'm going to come back over here to the spider tab.

171
00:16:07,760 --> 00:16:15,830
Under control once the process is done you'll see that the requests made and bytes transferred will

172
00:16:15,830 --> 00:16:18,330
stop adding up.

173
00:16:18,470 --> 00:16:28,600
You can also click the button spider is running to pause the spider so now that the process is running

174
00:16:30,330 --> 00:16:35,910
it's a good time to talk about some of these things I put off under the form submission tab.

175
00:16:35,910 --> 00:16:41,940
You'll notice that it has all of the usual default fields that one would expect for various prompting

176
00:16:42,330 --> 00:16:54,120
such as male first name last names surname etc. You can in fact change these values or add to them in

177
00:16:54,120 --> 00:16:57,720
the case of unique targets if you wish.

178
00:16:57,720 --> 00:17:05,760
The default values are mainly there as place holders and to verify the security of any form on the most

179
00:17:05,760 --> 00:17:06,750
basic level

180
00:17:09,950 --> 00:17:15,920
while we're waiting I'll also point out that if you wanted to add credentials automatically for all

181
00:17:15,920 --> 00:17:17,930
of those instances where they're needed.

182
00:17:17,930 --> 00:17:24,290
Those little pop up boxes rather than having to do it every single time you're prompted for it during

183
00:17:24,290 --> 00:17:25,580
the process.

184
00:17:25,580 --> 00:17:34,670
You can go down to application log in you can select the radial button to automatically submit whatever

185
00:17:34,670 --> 00:17:36,890
credentials you enter here.

186
00:17:37,010 --> 00:17:40,880
Changing the user name and password to whatever you think they might be.

187
00:17:41,010 --> 00:17:41,460
All right.

188
00:17:41,480 --> 00:17:42,800
So that should be enough for now.

189
00:17:42,800 --> 00:17:46,310
I'm going to go back to control and I'm going to pause the spider ing process

190
00:17:50,490 --> 00:17:54,190
and go back to our site map now at this point.

191
00:17:54,200 --> 00:17:55,810
You might be asking.

192
00:17:56,810 --> 00:18:01,430
I've seen a few reference sites and that's not particularly helpful.

193
00:18:01,430 --> 00:18:05,600
Twitter hacks for charity backtrack.

194
00:18:05,600 --> 00:18:07,720
Who cares I hear you ask.

195
00:18:07,760 --> 00:18:13,790
We don't really need this information even though it gives us a basic idea of what sites are linked

196
00:18:14,210 --> 00:18:15,860
to the web application.

197
00:18:15,890 --> 00:18:22,280
It's still not very important information but if we click on the motility folder

198
00:18:27,080 --> 00:18:29,860
I'll look very interesting.

199
00:18:29,930 --> 00:18:37,580
It gives us the structure of the application and this is vitally important to know to go about penetrating

200
00:18:37,580 --> 00:18:38,570
it.

201
00:18:38,630 --> 00:18:45,170
You can inspect the styles read through the documents and understand what the person who is developing

202
00:18:45,170 --> 00:18:51,560
the website was thinking and having this information really is the first step to actively exploiting

203
00:18:51,560 --> 00:18:52,240
a system.

204
00:18:53,370 --> 00:19:01,740
Of course the really juicy stuff the hidden files such as admin pages concealed log and pages and so

205
00:19:01,740 --> 00:19:07,100
on are not supported features with the burp suite community additions.

206
00:19:07,350 --> 00:19:13,650
If you see the potential here and you do a lot of penetration testing of Web applications this is one

207
00:19:13,650 --> 00:19:17,540
more reason to consider getting the licensed version.

208
00:19:17,530 --> 00:19:22,460
OK we have Spider the application we have the structure of what it looks like.

209
00:19:22,460 --> 00:19:24,470
The web application itself.

210
00:19:24,660 --> 00:19:31,590
Let me just wrap up this tutorial by showing you how to get rid of all of the extraneous dross that

211
00:19:31,590 --> 00:19:37,800
you don't need to show items in the scope only just click on the filter bar

212
00:19:40,940 --> 00:19:45,530
and click show only in scope items.

213
00:19:45,530 --> 00:19:47,150
Click on the filter bar again

214
00:19:50,480 --> 00:19:51,650
and while law.

215
00:19:51,800 --> 00:19:53,300
There we go.

216
00:19:53,300 --> 00:19:59,090
The filters just got rid of all the junk and we can look only at the requests and responses that were

217
00:19:59,090 --> 00:20:06,500
defined to our scope filters just make life easier especially when you are first starting out because

218
00:20:06,500 --> 00:20:12,860
it gets rid of all the confusing unnecessary information that this tool presents that you just don't

219
00:20:12,860 --> 00:20:14,320
need.

220
00:20:14,330 --> 00:20:21,920
Last but not least remember that you can click the re enable button at the top of the screen anytime

221
00:20:21,950 --> 00:20:32,280
you want to re enable logging of out of scope traffic through the proxy so I hope you have found this

222
00:20:32,280 --> 00:20:38,890
tutorial helpful and that you have a foundational knowledge with which to conduct base expiring.

223
00:20:39,210 --> 00:20:45,420
As always when you're familiarity with the process has grown the small particulars should start to make

224
00:20:45,420 --> 00:20:46,840
more sense.

225
00:20:46,860 --> 00:20:52,560
I admit this was a lot of theory for what in the end was a very simple procedure but it is the first

226
00:20:52,560 --> 00:20:58,290
step on web application penetration testing and I did not wish to skimp on it.

227
00:20:58,290 --> 00:21:00,170
Thank you very much for your patience.