0 1 00:00:00,450 --> 00:00:04,109 Please note that this content is targeted for SysOps 1 2 00:00:04,109 --> 00:00:09,269 administrators. If you're a Solutions Architect or a developer you may want to 2 3 00:00:09,269 --> 00:00:15,389 skip over this one. Welcome back to backspace Academy in this practical lab 3 4 00:00:15,389 --> 00:00:20,789 on implementing CloudWatch monitoring scripts on ec2 we're going to start off 4 5 00:00:20,789 --> 00:00:26,010 by creating an ec2 server and then associating an IAM role to give it 5 6 00:00:26,010 --> 00:00:30,689 permission to access the CloudWatch service and then we're going to use that 6 7 00:00:30,689 --> 00:00:37,530 ec2 server to push metrics or custom metrics over to the cloud service and 7 8 00:00:37,530 --> 00:00:42,989 then we'll be able to view those metrics in CloudWatch as well. So the first 8 9 00:00:42,989 --> 00:00:49,019 thing we need to do is to create a role for our ec2 instance to access a cloud 9 10 00:00:49,019 --> 00:00:57,359 watch service so we go to roles create role and we're selecting an AWS server 10 11 00:00:57,359 --> 00:01:04,799 so we select ec2 and then use case ec2 to allow ec2 instances to call AWS 11 12 00:01:04,799 --> 00:01:10,680 services which is what we want so we'll select that and next permissions so what 12 13 00:01:10,680 --> 00:01:20,000 we need to do now is we need to create a custom policy so create policy and 13 14 00:01:20,000 --> 00:01:26,329 create your own policy we'll give it a name 14 15 00:01:35,270 --> 00:01:40,530 and I'll just paste in the policy document that I've created that will 15 16 00:01:40,530 --> 00:01:55,140 give the ec2 instance access to that cloud what services have we need okay so 16 17 00:01:55,140 --> 00:02:00,649 we just have a look at this policy document here so we've got an allow to 17 18 00:02:00,649 --> 00:02:06,530 allay our ec2 instance to get metrics statistics to list the metrics and to 18 19 00:02:06,530 --> 00:02:12,019 most importantly to put metric data there and we've just got that for all 19 20 00:02:12,019 --> 00:02:17,129 resources and we've also got another one that it's required for ec2 describeTags 20 21 00:02:17,129 --> 00:02:21,930 as well which is required for that that's script. So we'll create that 21 22 00:02:21,930 --> 00:02:34,650 policy now, now what we need to do is that we need to go back to our role, so 22 23 00:02:34,650 --> 00:02:39,540 now that we've created that policy we need to associate that policy with this 23 24 00:02:39,540 --> 00:02:45,569 role, so we'll just search that policy now and there it is the one that we 24 25 00:02:45,569 --> 00:02:51,900 created, so we select that one and we click on next review will give the 25 26 00:02:51,900 --> 00:02:58,709 roller name we won't worry about a description there and we'll create that 26 27 00:02:58,709 --> 00:03:06,419 role and then we can see there is our role that we just created we just have a 27 28 00:03:06,419 --> 00:03:13,260 look and make sure it's okay and there's a policy that's attached to it and we 28 29 00:03:13,260 --> 00:03:16,380 can see that it's got CloudWatch permissions that seems to be fine there 29 30 00:03:16,380 --> 00:03:21,720 so now we can go into the ec2 management console and start creating this ec2 30 31 00:03:21,720 --> 00:03:25,799 instance they can collect these CloudWatch metrics, so we'll go into the ec2 31 32 00:03:25,799 --> 00:03:35,280 dashboard click on launch instance, we will select an Amazon Linux AMI and a t2 32 33 00:03:35,280 --> 00:03:44,340 micro now what we want to do is that we want to assign that I am 33 34 00:03:44,340 --> 00:03:49,530 role to this ec2 instance so we select on oh I am role and will select that ec2 34 35 00:03:49,530 --> 00:03:54,300 monitoring role that we created the next thing that we want to do is we want to 35 36 00:03:54,300 --> 00:03:59,790 expand advanced details because we want to put in a BASH script in here and what 36 37 00:03:59,790 --> 00:04:05,160 that is going to do it's going to pull in the required files to run the perl 37 38 00:04:05,160 --> 00:04:10,320 script so it will have some some the Perl language environment which will be 38 39 00:04:10,320 --> 00:04:15,990 downloaded and installed and then it will also in download and unzip the 39 40 00:04:15,990 --> 00:04:23,610 actual Perl scripts that are going to be run so just jumping back into the lab 40 41 00:04:23,610 --> 00:04:27,350 notes and I'll just grab that script 41 42 00:04:32,120 --> 00:04:36,240 okay so just put that in there and I'll just expand it out a bit and show you 42 43 00:04:36,240 --> 00:04:44,370 what it looks like okay so we can see they were doing a yum update so just 43 44 00:04:44,370 --> 00:04:49,110 making sure that the the Linux operating system is up to date and then we're 44 45 00:04:49,110 --> 00:04:54,330 installing our Perl requirements that we need then we're doing a curl to download 45 46 00:04:54,330 --> 00:04:59,730 that zip file and then we're just done unzipping it into that CloudWatch 46 47 00:04:59,730 --> 00:05:05,070 monitoring scripts directory that we'll have so that looks fine we'll just 47 48 00:05:05,070 --> 00:05:11,280 review and launch and what I would like to do now is just to give it a name just 48 49 00:05:11,280 --> 00:05:18,120 so we can find it easy enough so just enter tags add a tag and put a name in 49 50 00:05:18,120 --> 00:05:30,420 there for it and just call it monitoring that'll be fine so we review and launch 50 51 00:05:30,420 --> 00:05:37,460 and launch select the key pair and we'll launch that instance 51 52 00:05:41,350 --> 00:05:45,050 so of course after a certain amount of time that will be up and running and 52 53 00:05:45,050 --> 00:05:54,920 then we can start to use our CloudWatch monitoring scripts okay so our ec2 53 54 00:05:54,920 --> 00:05:58,460 instance is up and running it's passed its status checks it's ready to go so 54 55 00:05:58,460 --> 00:06:01,880 what we can do now is connect to it so I just click on connect grab that 55 56 00:06:01,880 --> 00:06:09,500 connection string that I need and jump into cmnder 56 57 00:06:09,500 --> 00:06:15,170 so I just navigated to where my key pair for this instance is and just connect 57 58 00:06:15,170 --> 00:06:29,950 in and now I'll navigate to where the monitoring scripts are 58 59 00:06:37,650 --> 00:06:43,050 so there we can see we've got our mon-get instance stats and Mon put instant 59 60 00:06:43,050 --> 00:06:48,030 data, so we can test that out now without actually publishing anything to 60 61 00:06:48,030 --> 00:06:53,100 CloudWatch just with a simple command so there we've got Mon put instant 61 62 00:06:53,100 --> 00:06:59,280 data and we're going to get the memory utilization metric so let's run 62 63 00:06:59,280 --> 00:07:09,289 that so then we can see its memory utilization is 8.1 3% it's it's tried to 63 64 00:07:09,289 --> 00:07:13,530 look for credentials it hasn't found it so it's trying the default IAM role 64 65 00:07:13,530 --> 00:07:17,490 and it's found our ec2 monitoring om role which is fine and then it's 65 66 00:07:17,490 --> 00:07:26,370 delivered as JSON our payload there and so there we can see the value is 8.13 66 67 00:07:26,370 --> 00:07:33,539 the unit is percent and the metric is memory utilization, so what we'll do 67 68 00:07:33,539 --> 00:07:39,539 now is it will try and publish or push this out to the CloudWatch service and 68 69 00:07:39,539 --> 00:07:48,720 see if we can pick it up in CloudWatch in the CloudWatch console. Ok so 69 70 00:07:48,720 --> 00:07:52,889 this command now will actually push it out to CloudWatch, so we'll see if that 70 71 00:07:52,889 --> 00:07:57,150 works and then we can see it's successfully 71 72 00:07:57,150 --> 00:08:01,620 reported metrics to CloudWatch so we can now jump into CloudWatch and have a 72 73 00:08:01,620 --> 00:08:08,610 look and see if it's actually picked it up ok so let's just go to services and 73 74 00:08:08,610 --> 00:08:13,970 CloudWatch and we'll see if we can find that metric that's been pushed out to 74 75 00:08:13,970 --> 00:08:21,539 Claire watch so we're going to browse metrics and we're looking for Linux 75 76 00:08:21,539 --> 00:08:29,479 system metrics and we select instance ID 76 77 00:08:29,720 --> 00:08:38,159 and then we can see we've got some for some instances and this is the one we're 77 78 00:08:38,159 --> 00:08:42,240 looking as for as our instance name is monitoring and so that we're looking for 78 79 00:08:42,240 --> 00:08:46,290 so we did actually publish memory utilization so I'm just going to select 79 80 00:08:46,290 --> 00:08:51,270 that one and we can see there we've got one point there of 27.6 80 81 00:08:51,270 --> 00:08:59,370 for memory utilization so let's just jump back into the command line 81 82 00:08:59,370 --> 00:09:02,010 interface and see if we can send another data point and see where that gets 82 83 00:09:02,010 --> 00:09:08,730 picked up as well okay so yeah jumping back into the command line interface 83 84 00:09:08,730 --> 00:09:15,210 we've seen how we can publish or push a metric information out to CloudWatch 84 85 00:09:15,210 --> 00:09:21,960 manually so what we don't want to do now is to set up a CRON job that can do that 85 86 00:09:21,960 --> 00:09:26,040 on a regular basis so we've set it up so it does it on every minute we'll give 86 87 00:09:26,040 --> 00:09:29,850 the instance you know 10 minutes or so and we'll just see what happens and 87 88 00:09:29,850 --> 00:09:34,770 whether it actually gets picked up so the first thing we need to do is to go 88 89 00:09:34,770 --> 00:09:45,200 into crontab or a cron table and edit that so I'll just go and grab the 89 90 00:09:45,200 --> 00:09:54,930 command that we need okay so I just press the insert button 90 91 00:09:54,930 --> 00:09:59,390 and they'll bring me into insert I'll paste that in and we can see there that 91 92 00:09:59,390 --> 00:10:03,990 we've got the cron schedule which is just all stars which means every minute 92 93 00:10:03,990 --> 00:10:09,840 it will be running and it's going to run that mine put instant starter and going 93 94 00:10:09,840 --> 00:10:14,100 to publish some metrics out to cloud watch for us so just get out of that by 94 95 00:10:14,100 --> 00:10:18,890 pressing escape and then the colon (:) and WQ 95 96 00:10:18,890 --> 00:10:25,170 exclamation mark and that will quit and save that for us and it's probably good 96 97 00:10:25,170 --> 00:10:28,320 idea just to check that it worked okay so just did crontab -e and we can see 97 98 00:10:28,320 --> 00:10:38,670 that's fine, so I can just get out of that again, okay so we leave that for a 98 99 00:10:38,670 --> 00:10:41,940 while now I'm just going to go away and have a cup of coffee I'll come back and 99 100 00:10:41,940 --> 00:10:46,230 then we'll have a look at CloudWatch through the console and we see if for 100 101 00:10:46,230 --> 00:10:53,100 any of these metrics are being published every minute as they should be okay so 101 102 00:10:53,100 --> 00:10:57,510 it's been about 20 minutes or so and we should have a good amount of data that 102 103 00:10:57,510 --> 00:11:02,460 is coming in from our our cron job that we created so just back here in the in 103 104 00:11:02,460 --> 00:11:08,149 the CloudWatch console I'll select the monitoring instance and memory 104 105 00:11:08,149 --> 00:11:12,019 utilization we'll see what we've got and there we can see they've got quite a bit 105 106 00:11:12,019 --> 00:11:19,370 of data from 11:45 up to 12:05 it's gone from 27.6 up to 27.85 106 107 00:11:19,370 --> 00:11:24,699 down and up again. So that's working great so what we've done is that we've 107 108 00:11:24,699 --> 00:11:30,589 created means ec2 instance with an installed a perl script now that Perl 108 109 00:11:30,589 --> 00:11:35,690 script was created by AWS and it's collecting system information and 109 110 00:11:35,690 --> 00:11:41,149 publishing that to CloudWatch for us and we're now here able to see it so 110 111 00:11:41,149 --> 00:11:45,860 it's something that not only do you need to use AWS scripts you can actually do 111 112 00:11:45,860 --> 00:11:49,240 this yourself and I'll give you a good example is that you may have for example 112 113 00:11:49,240 --> 00:11:57,319 an Internet of Things device that is collecting data and you want to push 113 114 00:11:57,319 --> 00:12:00,769 that data out to somewhere to be stored so a great way to do it is is to really 114 115 00:12:00,769 --> 00:12:05,209 push it out as a metric if you're if you're recording a metric push it out to 115 116 00:12:05,209 --> 00:12:10,189 CloudWatch and you can view it in cloud watch and you can also use CloudWatch 116 117 00:12:10,189 --> 00:12:14,600 logs to archive that information as well so there's some great opportunities to 117 118 00:12:14,600 --> 00:12:18,829 do that and all you need to do is you know if you're using an SDK be using the 118 119 00:12:18,829 --> 00:12:26,180 nodejs SDK or the PHP or Java or whatever SDK it is so we'll have a 119 120 00:12:26,180 --> 00:12:33,670 feature there in CloudWatch to to put metrics out so just as an example I can 120 121 00:12:33,670 --> 00:12:42,759 jump into the AWS SDK for JavaScript documentation go into the API reference 121 122 00:12:43,660 --> 00:12:53,060 select CloudWatch and if I scroll down here we can see that we've got put 122 123 00:12:53,060 --> 00:12:58,130 metric data and so that publishes metric data points to Amazon CloudWatch and 123 124 00:12:58,130 --> 00:13:02,180 it'll do that as a custom metric just the same as our perl script has done so 124 125 00:13:02,180 --> 00:13:08,149 if you have got any use cases for that that's great and you don't need to you 125 126 00:13:08,149 --> 00:13:11,810 know you don't need to use the AWS scripts if you've got your own scripts 126 127 00:13:11,810 --> 00:13:15,949 that you want to use to collect whatever data you want to use by all means go 127 128 00:13:15,949 --> 00:13:18,710 ahead and do that so I think we've learned quite a lot 128 129 00:13:18,710 --> 00:13:23,110 and I'll see you in the next lesson.