1 00:00:00,320 --> 00:00:02,029 Okay, so let's learn about CloudTrail. 2 00:00:02,029 --> 00:00:04,210 So let's open the CloudTrail Service 3 00:00:04,210 --> 00:00:06,570 and we're going to look at API activity 4 00:00:06,570 --> 00:00:09,460 within their accounts so let me close these things. 5 00:00:09,460 --> 00:00:11,760 And let's go on the left hand side 6 00:00:11,760 --> 00:00:12,810 and let's go to the dashboard. 7 00:00:12,810 --> 00:00:14,500 So currently the dashboard shows that 8 00:00:14,500 --> 00:00:16,000 we have no trails created 9 00:00:16,000 --> 00:00:18,760 and that we haven't enabled the CloudTrail insights. 10 00:00:18,760 --> 00:00:20,100 And we can look at the event history 11 00:00:20,100 --> 00:00:23,110 of everything that happened within my accounts recently. 12 00:00:23,110 --> 00:00:26,300 So if I go to event history here I get all the information 13 00:00:26,300 --> 00:00:29,060 of all the API calls made by some services 14 00:00:29,060 --> 00:00:31,520 and when it happens and what was the username 15 00:00:31,520 --> 00:00:33,210 and the event sources and stuff like this. 16 00:00:33,210 --> 00:00:35,860 So it's quite interesting because we can look at 17 00:00:35,860 --> 00:00:38,820 read only events or we can look at write events. 18 00:00:38,820 --> 00:00:40,410 So like this is read only false 19 00:00:40,410 --> 00:00:41,840 so that means these are all the write events 20 00:00:41,840 --> 00:00:44,860 such as delete role, delete policy, these kind of things. 21 00:00:44,860 --> 00:00:48,860 If we look at terminate instances for example, 22 00:00:48,860 --> 00:00:53,860 so I'm going to look at events name is terminates instances. 23 00:00:56,630 --> 00:00:58,040 Okay so this is a filter. 24 00:00:58,040 --> 00:01:02,280 We can see when instances were being deleted in EC2 25 00:01:02,280 --> 00:01:05,019 and so you can see these two instances right here 26 00:01:05,019 --> 00:01:07,010 were deleted by roots so this is me. 27 00:01:07,010 --> 00:01:09,010 One was deleted by the cloud nine service 28 00:01:09,010 --> 00:01:12,410 so when I was using cloud nine and one by auto-scaling. 29 00:01:12,410 --> 00:01:14,470 And if we want to drill down into an event for example 30 00:01:14,470 --> 00:01:16,590 we can look at this event right here. 31 00:01:16,590 --> 00:01:19,850 Okay so an instance was terminated on this date. 32 00:01:19,850 --> 00:01:22,780 And if I scroll down I get an information 33 00:01:22,780 --> 00:01:25,780 around the event records so all the information around 34 00:01:25,780 --> 00:01:28,587 when it happened, the region it happened from, the IP, 35 00:01:28,587 --> 00:01:29,980 the request parameters, 36 00:01:29,980 --> 00:01:31,710 all these kinds of things that allow me 37 00:01:31,710 --> 00:01:34,950 if I go back to it to really understand who initiated it 38 00:01:34,950 --> 00:01:37,540 and what, and when and how okay? 39 00:01:37,540 --> 00:01:39,350 So this is some really good information. 40 00:01:39,350 --> 00:01:41,785 And then it also links to the resource being referenced 41 00:01:41,785 --> 00:01:45,060 which it says this EC2 instance was trying to be deleted. 42 00:01:45,060 --> 00:01:47,010 Obviously, if I click on this resource name 43 00:01:47,010 --> 00:01:48,370 it's not going to take me anywhere 44 00:01:48,370 --> 00:01:50,210 because this instance was terminated 45 00:01:50,210 --> 00:01:52,660 and so it's not going to find any kind of instances. 46 00:01:52,660 --> 00:01:56,410 But you can see the usefulness of this event right here. 47 00:01:56,410 --> 00:01:57,290 Okay? 48 00:01:57,290 --> 00:02:01,270 So these are for, you can filter by events as we can see 49 00:02:01,270 --> 00:02:05,740 if there was a read only, and you just say true 50 00:02:05,740 --> 00:02:08,060 then you're going to get all the type of events 51 00:02:08,060 --> 00:02:08,900 that just read. 52 00:02:08,900 --> 00:02:11,350 For example describe instances is an event 53 00:02:11,350 --> 00:02:12,720 that doesn't changed anything. 54 00:02:12,720 --> 00:02:15,370 And so you get the idea around how CloudTrail works. 55 00:02:15,370 --> 00:02:17,230 As we can see event history shows you 56 00:02:17,230 --> 00:02:19,850 the last 90 days of event managements. 57 00:02:19,850 --> 00:02:23,110 And there is a problem if we want to have more than that 58 00:02:23,110 --> 00:02:25,410 but we'll see how we can fix this okay? 59 00:02:25,410 --> 00:02:28,080 So insights is not enabled. 60 00:02:28,080 --> 00:02:30,870 And if you create a trail with insights in it 61 00:02:30,870 --> 00:02:32,630 you gonna have to pay for it so I'm not going to do this 62 00:02:32,630 --> 00:02:35,240 but I will show you how we can enable it. 63 00:02:35,240 --> 00:02:37,106 And trails is how you would create a trail 64 00:02:37,106 --> 00:02:38,790 to capture more events. 65 00:02:38,790 --> 00:02:41,960 So let's click on create a trail and see how things work. 66 00:02:41,960 --> 00:02:45,140 So I call it demo trail. 67 00:02:45,140 --> 00:02:46,920 And as we can see, we could enable this 68 00:02:46,920 --> 00:02:49,410 for multiple accounts if we had an organization 69 00:02:49,410 --> 00:02:51,020 so that you can manage everything directly 70 00:02:51,020 --> 00:02:52,930 from the collateral service. 71 00:02:52,930 --> 00:02:55,530 And where do we want to send these logs? 72 00:02:55,530 --> 00:02:58,100 So do we want to send everything in a new bucket 73 00:02:58,100 --> 00:02:58,933 or an existing bucket? 74 00:02:58,933 --> 00:03:01,060 So yes, I can create a new buckets 75 00:03:01,060 --> 00:03:04,340 that will be created automatically to have this trail. 76 00:03:04,340 --> 00:03:06,860 Do we want this bucket to be encrypted? So sure why not? 77 00:03:06,860 --> 00:03:09,773 No. I'll disable. This is going to be easier. 78 00:03:09,773 --> 00:03:12,260 Log file validation sure I'll enable it. 79 00:03:12,260 --> 00:03:14,760 And SMS notification delivery I don't need it. 80 00:03:14,760 --> 00:03:17,750 Next we can also send these logs into CloudWatch logs. 81 00:03:17,750 --> 00:03:20,340 So I can enable this and create a new log group 82 00:03:20,340 --> 00:03:22,180 which is going to have this name. 83 00:03:22,180 --> 00:03:25,300 So that means that in CloudWatch logs, as well as an S3 84 00:03:25,300 --> 00:03:26,991 I'm going to have all the information 85 00:03:26,991 --> 00:03:30,043 around all these events happening within CloudTrail. 86 00:03:30,910 --> 00:03:32,550 Do we want to create a new role for this? 87 00:03:32,550 --> 00:03:33,780 Yes, let's create a new role 88 00:03:33,780 --> 00:03:35,500 and it will be created automatically. 89 00:03:35,500 --> 00:03:40,500 So I'll call it a CloudTrail role for demo. Okay. 90 00:03:43,310 --> 00:03:45,060 Next we can tag it and I don't need it 91 00:03:45,060 --> 00:03:47,520 so I don't need to tag this trail. 92 00:03:47,520 --> 00:03:49,960 Next with team we need to choose a type of log events 93 00:03:49,960 --> 00:03:54,390 that we want to log into CloudWatch and S3 94 00:03:54,390 --> 00:03:55,260 and this kind of things. 95 00:03:55,260 --> 00:03:58,030 So management events as we've seen is all the events 96 00:03:58,030 --> 00:04:01,810 that are happening on your AWS resources. 97 00:04:01,810 --> 00:04:03,430 So whenever we terminate an instance, 98 00:04:03,430 --> 00:04:06,770 whenever we create a new IM role these kind of things. 99 00:04:06,770 --> 00:04:08,492 But data events is going to be 100 00:04:08,492 --> 00:04:10,840 for events happening on your S3 buckets 101 00:04:10,840 --> 00:04:11,960 and render functions. 102 00:04:11,960 --> 00:04:14,730 And insights events if you want to enable cloud insights 103 00:04:14,730 --> 00:04:16,390 to detect unusual activity errors 104 00:04:16,390 --> 00:04:18,950 or user behavior in our accounts and you have to pay 105 00:04:18,950 --> 00:04:21,170 for both of these things if you enable it. 106 00:04:21,170 --> 00:04:24,210 I just wanna show you the options right now in the console. 107 00:04:24,210 --> 00:04:27,720 So if for management events, what do we want to have? 108 00:04:27,720 --> 00:04:29,790 Do we want to have read events, write events 109 00:04:29,790 --> 00:04:31,870 and do you want to exclude KMS events 110 00:04:31,870 --> 00:04:33,190 because encryption happens a lot 111 00:04:33,190 --> 00:04:34,910 and sometimes you just don't wanna see it. 112 00:04:34,910 --> 00:04:36,060 So we can click on this 113 00:04:37,190 --> 00:04:38,740 and you don't get any additional charges 114 00:04:38,740 --> 00:04:41,670 because management events are free. 115 00:04:41,670 --> 00:04:43,810 Next for data events. 116 00:04:43,810 --> 00:04:45,700 This is the data sources you need to choose 117 00:04:45,700 --> 00:04:49,120 so we have S3 or Lambda right now in terms of data events. 118 00:04:49,120 --> 00:04:50,590 So if you choose S3 119 00:04:50,590 --> 00:04:52,600 what do we want to log in terms of data events? 120 00:04:52,600 --> 00:04:56,150 So do we want to log all current and future S3 buckets 121 00:04:56,150 --> 00:04:57,120 for read and write actions? 122 00:04:57,120 --> 00:04:59,490 So put objects, read objects, get object 123 00:04:59,490 --> 00:05:00,590 this kind of things. 124 00:05:00,590 --> 00:05:03,930 Or do we want to have individual bucket selection 125 00:05:03,930 --> 00:05:06,640 and what do we want to have for logging for read and writes? 126 00:05:06,640 --> 00:05:09,750 So it's up to you to define as many types of S3 buckets 127 00:05:09,750 --> 00:05:12,940 as you want or just all of them okay? 128 00:05:12,940 --> 00:05:15,440 And then if you want to have Lambda and S3 129 00:05:15,440 --> 00:05:17,900 you can just add a new data events type 130 00:05:17,900 --> 00:05:20,310 and have Lambda and then choose all functions 131 00:05:20,310 --> 00:05:21,990 and this is going to log all data events 132 00:05:21,990 --> 00:05:23,220 around all these functions 133 00:05:23,220 --> 00:05:26,440 or you can just input the function name if you want it to. 134 00:05:26,440 --> 00:05:28,040 So pretty handy. 135 00:05:28,040 --> 00:05:29,870 And then for insights events 136 00:05:29,870 --> 00:05:32,670 it's just one nub here to enable. 137 00:05:32,670 --> 00:05:35,070 And then it says, okay, insights is enabled. 138 00:05:35,070 --> 00:05:37,360 And so usage anomalies are going to be logged 139 00:05:37,360 --> 00:05:38,678 and you can view them. 140 00:05:38,678 --> 00:05:41,630 Because I don't want to make you go 141 00:05:41,630 --> 00:05:42,480 out of the free trial here 142 00:05:42,480 --> 00:05:45,180 I'm just going to unselect data events and insight events 143 00:05:45,180 --> 00:05:46,340 but at least you've seen the option 144 00:05:46,340 --> 00:05:47,970 and you've seen the use cases. 145 00:05:47,970 --> 00:05:51,633 So I click on next and it will create this trail. 146 00:05:55,700 --> 00:05:57,360 Okay so my trail has now been created. 147 00:05:57,360 --> 00:06:00,840 If I click on it I'm able to see where it's going to go to 148 00:06:00,840 --> 00:06:05,840 so this is, I believe my S3 buckets that it is logging to. 149 00:06:05,990 --> 00:06:09,010 And this also goes into a CloudWatch logs group right here 150 00:06:09,010 --> 00:06:13,177 so I need to go into services and go to CloudWatch 151 00:06:14,340 --> 00:06:17,190 and I'm going to find CloudWatch logs so let's open this. 152 00:06:19,610 --> 00:06:22,630 And here is logs and I go to log groups 153 00:06:22,630 --> 00:06:26,370 and here I can see my CloudTrail logs right here 154 00:06:26,370 --> 00:06:27,410 that I have. 155 00:06:27,410 --> 00:06:29,580 So there's a log stream that was created as well 156 00:06:29,580 --> 00:06:31,750 and this is where the events would happen. 157 00:06:31,750 --> 00:06:35,280 So my S3 buckets contains the CloudTrail 158 00:06:35,280 --> 00:06:36,113 and CloudTrail digest. 159 00:06:36,113 --> 00:06:37,550 So if I click on CloudTrail 160 00:06:37,550 --> 00:06:39,960 this is where the objects would appear as well. 161 00:06:39,960 --> 00:06:41,690 So let me close this screen. 162 00:06:41,690 --> 00:06:44,922 So what I'm going to do is just wait a little bit 163 00:06:44,922 --> 00:06:48,390 for CloudTrail to start sending some data into S3 164 00:06:48,390 --> 00:06:49,223 and CloudWatch. 165 00:06:49,223 --> 00:06:50,390 That could take five minutes 166 00:06:50,390 --> 00:06:52,990 and we can do to have fun as you can, for example 167 00:06:52,990 --> 00:06:55,550 open a service in EC2. 168 00:06:55,550 --> 00:06:58,930 And for example I'm going to create a key pair just for fun 169 00:06:58,930 --> 00:07:00,020 so we can have a look at it. 170 00:07:00,020 --> 00:07:02,850 So I'm going to do funky pair 171 00:07:04,210 --> 00:07:06,610 and we'll try to find that events 172 00:07:06,610 --> 00:07:10,170 in at least CloudWatch or at least CloudWatch logs. 173 00:07:10,170 --> 00:07:12,030 Okay so fun keeper has been created. 174 00:07:12,030 --> 00:07:15,750 Now, let me wait a little bit and I will get back to you. 175 00:07:15,750 --> 00:07:17,880 Okay so I'm going to refresh my log events 176 00:07:17,880 --> 00:07:20,190 and as we can see, a lot of events have been logged 177 00:07:20,190 --> 00:07:23,360 by CloudTrail already into my CloudWatch logs. 178 00:07:23,360 --> 00:07:25,940 If I want to filter it for the API called name 179 00:07:25,940 --> 00:07:29,410 creates key pair and press enter. 180 00:07:29,410 --> 00:07:30,610 It's not being found. 181 00:07:30,610 --> 00:07:32,930 Let's go try to find it first into CloudTrail 182 00:07:32,930 --> 00:07:34,040 to see if it appears there. 183 00:07:34,040 --> 00:07:38,460 So let's go to the event history, and I'm going to look 184 00:07:38,460 --> 00:07:42,230 by event name and it's going to be creates key pair 185 00:07:42,230 --> 00:07:43,380 which was logged right here. 186 00:07:43,380 --> 00:07:46,160 So we can see this create key pair happening right here 187 00:07:46,160 --> 00:07:47,395 and it was done 188 00:07:47,395 --> 00:07:49,680 and we can even see the name of the key pair. 189 00:07:49,680 --> 00:07:52,890 Oh, this was my old key pair so this was my dummy keeper 190 00:07:52,890 --> 00:07:53,910 so we need to wait a little bit 191 00:07:53,910 --> 00:07:55,950 to get my fun key pair to appear here. 192 00:07:55,950 --> 00:07:57,650 So let's wait because CloudTrail can take 193 00:07:57,650 --> 00:07:59,960 up to five minutes, up to 15 minutes sometimes 194 00:07:59,960 --> 00:08:02,820 to make events appear so let's wait a little bit. 195 00:08:02,820 --> 00:08:04,690 So let's refresh our event history. 196 00:08:04,690 --> 00:08:07,180 So now we see two key pairs so perfect. 197 00:08:07,180 --> 00:08:09,480 The event was being delivered into CloudTrail. 198 00:08:09,480 --> 00:08:10,835 So that means that if I go to CloudWatch 199 00:08:10,835 --> 00:08:15,710 and search again for my events, yes I can see four events 200 00:08:15,710 --> 00:08:17,440 related to my creates key pair. 201 00:08:17,440 --> 00:08:18,880 So if we look at the events 202 00:08:18,880 --> 00:08:21,220 this is when it was created and so on. 203 00:08:21,220 --> 00:08:22,200 So this is quite cool. 204 00:08:22,200 --> 00:08:23,520 So we can look at all these events 205 00:08:23,520 --> 00:08:24,930 related to my creates key pair. 206 00:08:24,930 --> 00:08:28,630 And if I go into my S3 buckets and refresh my objects 207 00:08:28,630 --> 00:08:33,350 as we can see, we get a cloud trail directory in history 208 00:08:33,350 --> 00:08:34,940 for each region that we're in. 209 00:08:34,940 --> 00:08:38,610 So we're in EUS1 and then we can look by dates 210 00:08:38,610 --> 00:08:39,559 and here we go. 211 00:08:39,559 --> 00:08:42,659 We have some files that here I can download 212 00:08:42,659 --> 00:08:44,960 and open up and it will give me adjacent files okay 213 00:08:44,960 --> 00:08:45,793 that I can, 214 00:08:45,793 --> 00:08:48,280 that will look exactly the same as what's in here 215 00:08:48,280 --> 00:08:50,870 and what's in CloudTrail if I click on one of these events 216 00:08:50,870 --> 00:08:53,110 it will look at the same event record okay? 217 00:08:53,110 --> 00:08:55,980 But the cool thing is that because it is an Amazon S3 218 00:08:55,980 --> 00:08:59,210 then we can use Athena to query these records. 219 00:08:59,210 --> 00:09:04,210 So if I go into CloudTrail and look into the event history 220 00:09:04,320 --> 00:09:06,150 I can create an Athena table 221 00:09:06,150 --> 00:09:08,810 and choose my CloudTrail logs here. 222 00:09:08,810 --> 00:09:10,680 And this is going to create a table 223 00:09:10,680 --> 00:09:15,060 in Athena and this table in Athena that I can open up 224 00:09:15,060 --> 00:09:18,280 I will show you right here is going to allow me 225 00:09:18,280 --> 00:09:21,390 to query for historical events in CloudTrail. 226 00:09:21,390 --> 00:09:24,830 So if I look at this table and then I will look on, 227 00:09:24,830 --> 00:09:27,653 click on the three dots and say preview table. 228 00:09:29,610 --> 00:09:31,210 So let's run this query 229 00:09:32,160 --> 00:09:34,860 and there's no output location defined. 230 00:09:34,860 --> 00:09:36,930 So let me just scroll up, 231 00:09:36,930 --> 00:09:39,300 click on set up a queries at location 232 00:09:39,300 --> 00:09:42,110 and I need to just select a bucket for this. 233 00:09:42,110 --> 00:09:45,580 So I can select this one. 234 00:09:45,580 --> 00:09:50,580 This looks right and press select, click on save. Perfect. 235 00:09:51,720 --> 00:09:53,920 So now if I run this query again, this should work. 236 00:09:53,920 --> 00:09:55,000 Here we go. 237 00:09:55,000 --> 00:09:57,750 And we can see the results where we have the event version 238 00:09:57,750 --> 00:10:01,120 the user identity and with a bunch of columns into Athena 239 00:10:01,120 --> 00:10:03,700 so the event time, the event source, event name, 240 00:10:03,700 --> 00:10:06,300 region, source IP address, user agent, and so on. 241 00:10:06,300 --> 00:10:07,700 And we can start running some queries 242 00:10:07,700 --> 00:10:10,830 around these events and analyze our data historically 243 00:10:10,830 --> 00:10:12,710 which I think is really, really cool. 244 00:10:12,710 --> 00:10:14,600 So that's it for this lecture. 245 00:10:14,600 --> 00:10:16,150 I hope you liked it and if you wanted to 246 00:10:16,150 --> 00:10:17,340 just clean up after yourself 247 00:10:17,340 --> 00:10:19,520 you could delete these extra buckets. 248 00:10:19,520 --> 00:10:22,050 You could delete this CloudTrail log group 249 00:10:22,050 --> 00:10:26,040 and I'm going to do this right now in the trail itself. 250 00:10:26,040 --> 00:10:28,770 I can click on it and delete this trail 251 00:10:28,770 --> 00:10:30,860 so it stops logging stuff all around. 252 00:10:30,860 --> 00:10:31,693 So that's it. 253 00:10:31,693 --> 00:10:34,263 I hope you liked it and I will see you in the next lecture.