WEBVTT 00:00:01.040 --> 00:00:04.960 To be effective, red teams must have key information about their targets. 00:00:05.440 --> 00:00:08.510 They need to know data about network services they can explore, 00:00:08.510 --> 00:00:10.910 technical details of systems they could attack, 00:00:10.910 --> 00:00:13.990 as well as organizational resources to which they 00:00:13.990 --> 00:00:15.460 could apply social engineering. 00:00:16.440 --> 00:00:18.970 This information feeds the planning and execution 00:00:18.970 --> 00:00:20.550 activities during the engagement. 00:00:21.540 --> 00:00:24.680 Having the right tools to recon the target leads to more 00:00:24.680 --> 00:00:27.150 efficient and effective red team testing. 00:00:28.640 --> 00:00:31.090 Hello and welcome to Reconnaissance was Shodan. 00:00:31.090 --> 00:00:34.950 My name is Keith Watson, and I'm an information security professional. 00:00:35.440 --> 00:00:38.040 I hope you've enjoyed my other courses on information 00:00:38.040 --> 00:00:41.260 security and penetration testing at Pluralsight. 00:00:41.260 --> 00:00:46.550 Here, we'll turn our focus to a key tool for red teams, Shodan. 00:00:46.550 --> 00:00:50.430 Reconnaissance is the first step in building a solid plan 00:00:50.430 --> 00:00:52.460 for red team testing against a target. 00:00:52.840 --> 00:00:56.210 It consists of both information‑gathering activities and an 00:00:56.210 --> 00:00:58.150 analysis of the information collected. 00:00:58.540 --> 00:01:02.320 This process is often the cycle as more specific information may be 00:01:02.320 --> 00:01:05.690 needed as the analysis of the gathered data progresses. 00:01:05.690 --> 00:01:09.280 The team may take advantage of open‑source intelligence resources, 00:01:09.280 --> 00:01:12.810 as well as more active technical approaches that probe a 00:01:12.810 --> 00:01:15.850 target for data directly or passive methods, 00:01:15.850 --> 00:01:19.260 which utilize information maintained or gathered by others. 00:01:20.140 --> 00:01:21.620 As information is collected, 00:01:21.620 --> 00:01:25.540 we can categorize it based on its type as either technical 00:01:25.540 --> 00:01:27.960 information about the target's digital assets, 00:01:28.440 --> 00:01:31.980 administrative information about the target's network domains, 00:01:31.980 --> 00:01:35.980 assigned IP address ranges, autonomous system numbering, 00:01:35.980 --> 00:01:38.180 and outsourced services, 00:01:38.180 --> 00:01:42.580 physical information about the location and characteristics of its offices, 00:01:42.580 --> 00:01:45.340 data centers, ancillary facilities, and operations, 00:01:45.340 --> 00:01:50.720 and collocated digital assets, and human information regarding employees, 00:01:50.830 --> 00:01:53.460 leaders, and the organizational structure. 00:01:54.340 --> 00:01:55.900 With that information collected, 00:01:55.900 --> 00:02:00.620 an analysis is needed to determine which data elements would be most valuable 00:02:00.620 --> 00:02:03.550 in the planning and execution phases of the engagement. 00:02:04.040 --> 00:02:07.730 The first consideration should be the goals of the red team exercise. 00:02:07.730 --> 00:02:11.050 Identifying the successful outcomes of the engagement will 00:02:11.050 --> 00:02:13.760 narrow the focus to relevant data elements. 00:02:14.740 --> 00:02:15.850 During the analysis, 00:02:15.860 --> 00:02:19.150 the rules of engagement must be considered as not all data elements 00:02:19.150 --> 00:02:21.640 might be allowed in execution of the engagement. 00:02:21.640 --> 00:02:25.270 From the analysis will come actionable targets on which 00:02:25.270 --> 00:02:26.930 the team can direct their efforts. 00:02:27.940 --> 00:02:31.870 The red team may plan to target specific business assets or try 00:02:31.870 --> 00:02:34.360 to manipulate a set of business processes. 00:02:35.340 --> 00:02:38.200 The outcome of the gathering and analysis activities then 00:02:38.200 --> 00:02:40.260 feeds into a threat modeling phase. 00:02:42.340 --> 00:02:45.250 Shodan brings a new dimension to reconnaissance for red team 00:02:45.250 --> 00:02:48.910 operations by exposing detailed information on internet‑connected 00:02:48.910 --> 00:02:51.340 systems through an easy‑to‑use interface. 00:02:51.340 --> 00:02:55.850 John Matherly, a programmer with a background in data mining and web development, 00:02:55.910 --> 00:02:59.060 created Shodan and launched it in 2009. 00:02:59.440 --> 00:03:02.750 The project started as a curiosity about what could be learned about 00:03:02.750 --> 00:03:06.110 devices connected to the internet and continues to be a leading source 00:03:06.110 --> 00:03:08.390 of information about the Internet of Things, 00:03:08.390 --> 00:03:12.180 including building control systems, industrial control systems, 00:03:12.400 --> 00:03:16.310 desktops, printers, voice over IP, surveillance cameras, 00:03:16.310 --> 00:03:19.010 and many other types of network‑enabled devices. 00:03:19.440 --> 00:03:20.450 Prior to Shodan, 00:03:20.460 --> 00:03:24.660 we had web‑focused search engines that gathered human consumable content. 00:03:25.040 --> 00:03:27.860 Shodan crawls the internet and reveals details about the 00:03:27.860 --> 00:03:30.370 various devices with public IP addresses. 00:03:30.370 --> 00:03:32.460 Similar to a web search engine, 00:03:32.900 --> 00:03:36.260 the Shodan data is easily searched and is very detailed. 00:03:38.240 --> 00:03:41.810 Shodan crawls the internet on a 24/7 basis and 00:03:41.810 --> 00:03:43.850 updates its database in real time. 00:03:44.340 --> 00:03:47.570 It also has crawlers in at least eight countries distributed 00:03:47.570 --> 00:03:50.720 around the world for a different view of systems and to avoid 00:03:50.720 --> 00:03:52.820 countrywide blocking that may be in place. 00:03:53.240 --> 00:03:54.420 As it crawls systems, 00:03:54.420 --> 00:03:58.720 Shodan gathers banner data from specific services it understands. 00:03:59.140 --> 00:04:01.540 It also records various bits of metadata. 00:04:01.540 --> 00:04:03.980 With that information in its database, 00:04:03.990 --> 00:04:07.120 users of Shodan can search that banner text and apply 00:04:07.120 --> 00:04:09.650 various filters to refine the search results. 00:04:10.040 --> 00:04:12.870 All the search capabilities are available to anyone 00:04:12.870 --> 00:04:16.160 through the Shodan website at shodan.io. 00:04:16.540 --> 00:04:19.040 Shodan also has a command line tool. 00:04:19.040 --> 00:04:24.150 Unlike Google or Bing, Shodan searches for systems behind the World Wide Web. 00:04:24.540 --> 00:04:26.720 It knows how to interact with specific types of 00:04:26.720 --> 00:04:28.910 network services to capture banner data, 00:04:28.990 --> 00:04:32.030 as well as metadata about the device, such as its operating system, 00:04:32.030 --> 00:04:35.960 software versions, configuration data, host names and IP addresses, 00:04:35.960 --> 00:04:38.060 and even its geographic location. 00:04:38.990 --> 00:04:42.160 There's an API for integration with other red team tools, 00:04:42.160 --> 00:04:46.560 and the data available in Shodan can be used in planning red team operations. 00:04:48.540 --> 00:04:51.460 Shodan is a reconnaissance tool and sits squarely in the 00:04:51.460 --> 00:04:53.460 recon space in the cyber kill chain. 00:04:53.840 --> 00:04:57.260 The information gathered and analyzed during the reconnaissance phase 00:04:57.270 --> 00:05:00.260 aids in progressing through other phases of the chain. 00:05:02.240 --> 00:05:05.120 The MITRE ATT&CK framework is a knowledge base of 00:05:05.120 --> 00:05:07.060 attacker tactics and techniques. 00:05:07.440 --> 00:05:09.860 You can see the high‑level tactics listed here. 00:05:10.240 --> 00:05:13.820 Red teams can use MITRE ATT&CK to replicate known attacker 00:05:13.820 --> 00:05:16.060 methods when conducting an engagement. 00:05:18.040 --> 00:05:21.120 Shodan can be a useful tool for red teams at the start of an 00:05:21.120 --> 00:05:24.850 operation. It specifically provides information for the first 00:05:24.850 --> 00:05:27.220 attack framework tactic of reconnaissance. 00:05:27.740 --> 00:05:28.530 Specifically, 00:05:28.840 --> 00:05:35.560 there are three relevant techniques, T1590, gather victim network information, 00:05:36.040 --> 00:05:43.800 T1592, gather victim host information, and T1596, search open technical 00:05:43.800 --> 00:05:51.840 databases. For T1590, the metadata collected and shown by Shodan corresponds 00:05:51.840 --> 00:06:04.940 to the following subtechniques, .002 DNS and .005 IP addresses. For T1592, the 00:06:04.940 --> 00:06:08.840 banner data collected and shown by Shodan corresponds to the following 00:06:08.840 --> 00:06:12.560 subtechniques, .001 hardware, 00:06:12.940 --> 00:06:22.260 .002 software, .003 firmware, and .004 client configurations. 00:06:24.140 --> 00:06:29.590 For T1596, Shodan an is a database of sorts providing a plethora of 00:06:29.590 --> 00:06:32.610 technical details about systems it has discovered. 00:06:33.040 --> 00:06:38.600 These include the following subtechniques, .003 digital 00:06:38.600 --> 00:06:42.960 certificates and .005 scan databases. 00:06:44.940 --> 00:06:48.360 Let's take a quick look at the architecture used by Shodan. 00:06:48.740 --> 00:06:51.330 We start with the users that want to understand 00:06:51.410 --> 00:06:53.150 systems that exist on the internet. 00:06:53.640 --> 00:06:54.420 Shodan, 00:06:54.430 --> 00:06:58.030 realizing this need, created a system of search engine crawlers 00:06:58.030 --> 00:07:01.820 that focus on the Internet of Things and collect banner data and 00:07:01.820 --> 00:07:06.500 metadata about the systems found, which is recorded into a database. 00:07:06.940 --> 00:07:12.070 Shodan created a web interface and API to make that data accessible and makes 00:07:12.070 --> 00:07:18.240 it available to users and for integration with tools. I mentioned that Shodan 00:07:18.240 --> 00:07:20.560 collects banner text from network services. 00:07:20.940 --> 00:07:26.190 Here's an example of an HTTP response from a web server it might collect. In 00:07:26.190 --> 00:07:31.880 the blue text, we see the HTTP response code. In the orange text are the 00:07:31.880 --> 00:07:37.290 HTTP headers and, in black, their values. Here, at the bottom, is the most 00:07:37.300 --> 00:07:39.060 interesting part of this information. 00:07:39.440 --> 00:07:41.450 We can see the server header field. 00:07:42.440 --> 00:07:47.870 This web server is powered by version 1.18.0 of the Nginx 00:07:47.870 --> 00:07:52.910 server software coupled with version 6.0.8 of the Phusion 00:07:52.910 --> 00:07:55.160 Passenger application server software. 00:07:56.640 --> 00:07:59.390 As you use Shodan, you'll find some very interesting 00:07:59.390 --> 00:08:01.460 information about the devices on the internet. 00:08:01.840 --> 00:08:04.910 Just remember that Shodan is gathering this information in 00:08:04.910 --> 00:08:06.960 order to protect systems on the internet. 00:08:07.440 --> 00:08:11.290 Our use of this information is to further that goal. Viewing this 00:08:11.290 --> 00:08:14.800 information is not illegal. Acting on the information without 00:08:14.800 --> 00:08:17.920 permission can be. As with any engagement, 00:08:18.010 --> 00:08:20.650 be sure to have permission of the system owners.