1 00:00:00,270 --> 00:00:05,400 In this video you're going to learn how to work with requests and beautiful soap modules and Python 2 00:00:05,460 --> 00:00:07,560 to conduct web attacks. 3 00:00:07,560 --> 00:00:09,570 We're going to be using the lab on a tack. 4 00:00:09,570 --> 00:00:16,370 Defense labs which has a platform to exercise and grow your penetration testing and cybersecurity skills. 5 00:00:16,360 --> 00:00:22,920 Now as you can see here we're dealing with a web portal and we need to do some initial recon and then 6 00:00:22,920 --> 00:00:28,380 actually brute force authentication all using Python. 7 00:00:28,590 --> 00:00:33,330 There is everything installed for us in the lab and we will be using Jupiter notebooks. 8 00:00:33,330 --> 00:00:40,320 So I'm going to open up the lab so clicking the lab Link is gonna take me to this location and we can 9 00:00:40,320 --> 00:00:41,430 see three files. 10 00:00:41,430 --> 00:00:47,380 The lab the lab with solutions and a dictionary file containing passwords. 11 00:00:47,430 --> 00:00:50,770 OK so we'll go into the lab. 12 00:00:50,960 --> 00:00:57,650 The simple lab file again we're going to use requests and beautiful suit. 13 00:00:58,010 --> 00:01:03,830 They are already installed so I only have to import them if you will be working with them. 14 00:01:03,840 --> 00:01:11,820 You can simply install them by a bit in a terminal or a command prompt so you can simply do that or 15 00:01:11,830 --> 00:01:14,730 that three install requests 16 00:01:17,960 --> 00:01:26,790 and that or 3 install beautiful soup for and that's it. 17 00:01:26,790 --> 00:01:32,460 Now to get a very similar scenario to the one that I have here you could have you could install a virtual 18 00:01:32,460 --> 00:01:40,740 machine in VM or virtual box and then install a vulnerable CNS on the machine like the one we're attacking 19 00:01:40,740 --> 00:01:41,830 in this scenario. 20 00:01:42,210 --> 00:01:43,430 And that's all for free. 21 00:01:43,590 --> 00:01:50,610 Or like I'm doing in this case you could get a subscription on attack defense whichever you find the 22 00:01:50,670 --> 00:01:52,710 best suitable for you. 23 00:01:52,740 --> 00:02:00,030 Now I'm using a tag defense labs platform for convenience here for no setup and for your better understanding. 24 00:02:00,050 --> 00:02:01,500 All right. 25 00:02:01,500 --> 00:02:04,620 We have a popular CRM and some import 80. 26 00:02:04,620 --> 00:02:08,280 We have the libraries installed and we also have the dictionary file. 27 00:02:08,280 --> 00:02:09,270 All good. 28 00:02:09,300 --> 00:02:16,680 First we need to check if the Web portal is up using a get request and that's as simple as import requests 29 00:02:17,150 --> 00:02:32,340 and then requests that get EDP 1 2 7 0 0 0 1 and I execute this code or this cell using shift and enter 30 00:02:35,600 --> 00:02:40,410 together response two hundred which is the HDTV code or status for. 31 00:02:40,440 --> 00:02:42,740 OK all right let's move on. 32 00:02:42,770 --> 00:02:45,870 So server software being used. 33 00:02:46,310 --> 00:02:49,620 We're going to do something similar and we're going to instantiate the. 34 00:02:49,730 --> 00:02:56,390 In this case we're going to instantiate the request in a variable so let's say rect underscore requests 35 00:02:56,760 --> 00:03:02,860 that get a GDP 1 2 7 0 0 0 0 1. 36 00:03:02,870 --> 00:03:08,360 Now what is great about our request module is that once it's made a request it saved the response in 37 00:03:08,360 --> 00:03:11,300 a very accessible and manner. 38 00:03:11,300 --> 00:03:15,430 In this case we have the response of the request in the rack underscore variable. 39 00:03:15,440 --> 00:03:20,590 And to answer the question we usually find a lot of information in the response fathers. 40 00:03:20,930 --> 00:03:26,720 So which server software is being used. 41 00:03:26,720 --> 00:03:34,940 We can simply say rack underscore dot headers and then shift enter to execute the cell and we can see 42 00:03:34,940 --> 00:03:36,470 engine x. 43 00:03:36,780 --> 00:03:38,760 Okay moving on number three. 44 00:03:38,960 --> 00:03:45,300 We are being asked for the headers and we already did that with the request on their score. 45 00:03:45,320 --> 00:03:55,640 Heather now for the task for we need to get the content or the text content of local host home page. 46 00:03:55,760 --> 00:03:59,170 Also can you tell which CNS is running on local hosts. 47 00:03:59,240 --> 00:04:01,280 Why do we need here is the content. 48 00:04:01,400 --> 00:04:05,670 So we'll say rec underscore dot content. 49 00:04:05,840 --> 00:04:10,610 So as you can see I've hit dot and then hit the tab. 50 00:04:10,610 --> 00:04:19,100 Now in this case we can simply access all the methods available at our disposal like for example we 51 00:04:19,100 --> 00:04:29,270 can get the cookies the encoding the history Jason so there regressed in Jason format next we can see 52 00:04:29,270 --> 00:04:31,640 that raw request and all that stuff. 53 00:04:31,640 --> 00:04:38,100 So in this case we need the text content of local host home page and that's as easy as rack underscore 54 00:04:38,100 --> 00:04:39,500 dot content. 55 00:04:42,290 --> 00:04:47,990 Shift enter and we can see this content. 56 00:04:48,410 --> 00:04:52,580 This also can be done with rack underscore text. 57 00:04:52,580 --> 00:04:59,080 And if I do this as a corner comment we basically get a very similar response. 58 00:04:59,900 --> 00:05:02,930 But anyway this doesn't look pretty nice. 59 00:05:03,290 --> 00:05:08,690 So it says also can you tell which CNS is running on local host. 60 00:05:08,720 --> 00:05:16,610 So if we look into this code we can see title target wordpress so we can safely assume that this is 61 00:05:16,610 --> 00:05:23,540 actually WordPress running as a CNS on the local host. 62 00:05:23,750 --> 00:05:36,080 Now moving on to number five we need to print this ugly response in a pretty form using beautiful soup. 63 00:05:36,140 --> 00:05:46,030 So we start with a variable actually first we start with importing from the as for import. 64 00:05:47,090 --> 00:05:50,130 Beautiful suit. 65 00:05:50,480 --> 00:05:52,510 This is all you import beautiful soup. 66 00:05:52,580 --> 00:06:01,760 Then we need a variable let's call it soup equals beautiful soup and then rack underscore dot text and 67 00:06:01,790 --> 00:06:07,070 we need to specify an HDMI all the answer which is going to be our parser. 68 00:06:07,070 --> 00:06:12,350 So we actually need to tell a beautiful soup to use by phones is built in a standard parser to pass 69 00:06:12,350 --> 00:06:18,470 our request and then we'll use the predefined method to print the content in a more readable way. 70 00:06:18,470 --> 00:06:30,440 So we'll save print soup that pretty fi awesome as you can see this looks much much better. 71 00:06:30,530 --> 00:06:30,940 All right. 72 00:06:30,950 --> 00:06:36,140 Now number six print the title of web portal hosted on local host. 73 00:06:36,140 --> 00:06:42,620 Now similar to requests a beautiful soup has a lot of methods to access the content such as HDMI elements 74 00:06:42,620 --> 00:06:43,690 in this case. 75 00:06:43,820 --> 00:06:52,160 Here we would simply need to actually print the soup that title 76 00:06:54,890 --> 00:07:02,030 now the great thing about Python is that there is at least a library for everything you need for whatever 77 00:07:02,030 --> 00:07:06,560 purpose you may have and you don't have to reinvent the wheel by writing code from scratch. 78 00:07:06,560 --> 00:07:11,050 Now be my guest to do that if you want but it in penetration testing. 79 00:07:11,060 --> 00:07:12,090 I want to automate. 80 00:07:12,140 --> 00:07:15,650 I want to get things done fast and very efficiently. 81 00:07:15,650 --> 00:07:22,740 All right now seven and eight are very similar so I'll just jump right into a scrape. 82 00:07:22,760 --> 00:07:30,440 All your URLs from the home page of local host and print unique URLs so we'll start by creating a variable 83 00:07:30,470 --> 00:07:38,330 let's say home their score and then we'll do requests that get a GDP local host. 84 00:07:39,630 --> 00:07:40,090 OK. 85 00:07:40,100 --> 00:07:53,420 And then we'll create a soup equals Beautiful Soup home page or home underscore that content 86 00:07:56,080 --> 00:08:00,970 and we also need to specify the HDMI parser of course. 87 00:08:01,830 --> 00:08:09,780 Now we need to look for all your URLs and in HD among these are denoted by D A element. 88 00:08:09,780 --> 00:08:16,470 So we'll say images because soon dot find all 89 00:08:19,720 --> 00:08:25,560 a day trip equals truth. 90 00:08:25,710 --> 00:08:27,510 Then we'll create an empty list. 91 00:08:27,630 --> 00:08:31,500 So we'll see images underscore each raft. 92 00:08:31,680 --> 00:08:43,050 We have an empty list and populated with the URLs using a for loop so we'll save for image and images. 93 00:08:43,290 --> 00:08:47,910 We'll say images underscore H ref pan. 94 00:08:48,000 --> 00:08:51,190 So we need to attend to this list. 95 00:08:51,840 --> 00:09:03,450 Image H ref Alright so now we have a list which may contain duplicates and we're required as we can 96 00:09:03,450 --> 00:09:07,700 see to print unique you or else. 97 00:09:07,800 --> 00:09:14,490 So for that we can simply make a set of our lists which will contain no duplicates as this is an inherent 98 00:09:14,490 --> 00:09:16,820 property in sets. 99 00:09:16,830 --> 00:09:22,820 You can check my Python basic source in which I go into more details about sets and lists. 100 00:09:23,030 --> 00:09:28,390 All right so this is as simple as images underscore a draft. 101 00:09:31,650 --> 00:09:40,470 Actually another variable image said it's gonna be images there's gonna be a set of images that each 102 00:09:40,470 --> 00:09:47,730 ref what we're left to do is to print the elements of the sand so again using a for loop we can simply 103 00:09:47,730 --> 00:09:52,880 save for image and images set print. 104 00:09:53,240 --> 00:10:08,090 I AMG All right shift and run that and as we can see we get a nice list of unique elements all right. 105 00:10:08,100 --> 00:10:12,540 Now for the fun part let's do some web attacking so task 9. 106 00:10:12,540 --> 00:10:16,780 Can you access the admin section of the CNS. 107 00:10:17,220 --> 00:10:18,670 We already know how to do that. 108 00:10:18,690 --> 00:10:21,920 So let's say word on their score. 109 00:10:21,930 --> 00:10:31,640 P equals requests that Cath HDD DP local host WP admin. 110 00:10:31,680 --> 00:10:41,710 This is what we're requested to try and access and then let's actually do a soup that word underscore 111 00:10:41,730 --> 00:10:51,000 word underscore P is gonna be a beautiful soup word underscore P text and of course we need to specify 112 00:10:51,000 --> 00:10:53,620 the H and not parser. 113 00:10:53,640 --> 00:11:04,130 Alright and then let's also print the soup underscore word P that pretty fi OK. 114 00:11:04,260 --> 00:11:06,100 Shift ends are to run this. 115 00:11:06,330 --> 00:11:13,740 So if we look a little bit into it we could infer it's a log in form and we'll need this information 116 00:11:13,740 --> 00:11:15,910 in desk number 10. 117 00:11:15,960 --> 00:11:22,680 So over here which says that we need to brute force the WordPress log in for user admin use the given 118 00:11:22,680 --> 00:11:33,480 dictionary so remember that we have a dictionary called Password underscored dictionary a couple of 119 00:11:33,480 --> 00:11:39,330 pointers since we're dealing with a log in form we'll be using a post request instead of a get request. 120 00:11:39,720 --> 00:11:49,350 So if we get the code again we can see input glass IP user log in name log input type text. 121 00:11:49,350 --> 00:11:56,700 We could also see input glass user pass named P.W. WD type password. 122 00:11:56,700 --> 00:12:02,970 We need this information when we create or post requests because we're going to send a post request 123 00:12:03,090 --> 00:12:05,260 to this forum again. 124 00:12:05,280 --> 00:12:11,940 So what we need to do is a post request and send the user name to the field log and a password to the 125 00:12:11,940 --> 00:12:18,820 field BW D This is what our browsers basically do when we authenticate. 126 00:12:18,830 --> 00:12:25,190 Now in this case we'll be brute forcing the password for user name admin as requested and we'll try 127 00:12:25,200 --> 00:12:32,190 all the passwords in the dictionary file by sending them one by one to the P W D field. 128 00:12:32,190 --> 00:12:33,780 This is what we're going to go it here. 129 00:12:33,780 --> 00:12:38,970 So for that first we need to instantiate a variable for our password file. 130 00:12:38,980 --> 00:12:43,920 So let's say fast file equals password underscore 131 00:12:50,430 --> 00:12:51,010 dictionary. 132 00:12:51,010 --> 00:12:51,950 But to extend 133 00:12:57,250 --> 00:13:06,880 and then we'll encapsulate everything in a with construct in Python so we'll say with open fast file 134 00:13:09,290 --> 00:13:15,870 in read mode as f for word in F. 135 00:13:16,090 --> 00:13:21,520 First we need to make sure that each line in the file is clean and only contains the word so we'll strip 136 00:13:21,520 --> 00:13:25,570 the line for trailing spaces otherwise our attack might fails. 137 00:13:25,630 --> 00:13:30,660 So we'll say word equals word that strip. 138 00:13:30,790 --> 00:13:31,930 New Line. 139 00:13:33,020 --> 00:13:34,480 And here's the most important part. 140 00:13:34,480 --> 00:13:37,570 The actual post request. 141 00:13:38,050 --> 00:13:48,970 So let's say trying underscore equals requests that post we're going to make a post request wage TTP 142 00:13:49,330 --> 00:13:54,810 local host WP admin. 143 00:13:54,910 --> 00:14:10,600 Actually we need to make a request not a WP admin but to WP log in that BHP so WP log in but BHP and 144 00:14:10,600 --> 00:14:22,100 then we need to send the data in a dictionary so data is going to we need to send log the user named 145 00:14:22,120 --> 00:14:32,350 Adnan so we need to send to log ad men to log we need to send the username admin because this is what 146 00:14:32,350 --> 00:14:36,300 we've been requested and then for password. 147 00:14:36,580 --> 00:14:44,310 So for the P W D field we need to send our word okay. 148 00:14:45,430 --> 00:14:50,590 Every time you try to log in with a wrong password the response of the request will contain the word 149 00:14:50,710 --> 00:14:51,440 error. 150 00:14:51,550 --> 00:14:58,240 Hence we can infer that if this word error is not in the response or brute force succeeded and we found 151 00:14:58,240 --> 00:14:58,930 the password. 152 00:14:59,230 --> 00:15:02,240 So how do we actually coded this. 153 00:15:02,400 --> 00:15:21,070 We'll say if error not in trying underscore dot text we'll simply say print success the password is 154 00:15:22,620 --> 00:15:24,750 and then the last word. 155 00:15:25,830 --> 00:15:38,170 And then we exit the loop we'll say break or else we'll say print incorrect password. 156 00:15:38,390 --> 00:15:39,840 Last word of course. 157 00:15:39,840 --> 00:15:40,320 All right. 158 00:15:40,360 --> 00:15:41,420 That's just about it. 159 00:15:41,420 --> 00:15:46,810 Now let's run the subways shift enter and it's probably gonna take a while to finish so let's keep forward. 160 00:15:47,200 --> 00:15:51,070 So shift enter you can see all the incorrect passwords 161 00:15:55,260 --> 00:15:56,230 and there you have it. 162 00:15:56,310 --> 00:15:58,440 The password is best 1. 163 00:15:58,490 --> 00:16:04,530 Now this is all you can use requests and beautiful soup and both on to conduct recon enumerate and attack 164 00:16:04,530 --> 00:16:12,420 web servers always only attack on assets you have permission I need to specify this at all times. 165 00:16:12,480 --> 00:16:18,590 OK now we won't be doing desks 11 and 12 because they are not in the scope of this video. 166 00:16:18,810 --> 00:16:23,880 As always there is much more to request and beautiful soup than what I've shown here. 167 00:16:23,880 --> 00:16:29,460 Therefore if you want to get deeper understanding of them start by studying their official documentation 168 00:16:29,460 --> 00:16:32,760 which you can easily find via a browser search.