welcome back to Backspace Academy In this lecture we'll talk about AWS x-ray 2
which is an AWS backend service that could provide insights into how your 3
applications are running on AWS 4
we'll talk about how it works how do we modify 5
our application code to integrate with AWS x-ray 6
how do we collect information 7
about our applications performance and how do we pass that information on to 8
the x-ray service so that we can see some visualizations and graphs of this 9
performance 10
we will talk about the x-ray daemon and what that is and how we can 11
use that to collect information from the performance of our application and for 12
that onto the x-ray service 13
we'll talk about the permissions that our application 14
and the x-ray daemon will need 15
we'll talk about the permissions that any 16
users that want to access this data will need 17
and we'll also talk about the x-ray 18
console which allows us to view very detailed fine grade information in the 19
format of lists and graphs and other visualizations of our data 20
and finally 21
we'll have a look at the x-ray development tools that are available to 22
use with our application 23
the AWS x-ray service it will collect data about requests that are served by 24
your application and you can look at that data to provide very fine-grain 25
detail around the performance of your application it consists of a software 26
development kit a demon or daemon and the actual x-ray console or x-ray 27
service now the x-ray software development kit you need to include that 28
with your application and that will send trace information to an x-ray daemon 29
that will receive that information so the x-ray daemon that will listen for 30
UDP traffic from your application it will buffer that data and then in 31
batches it will upload it to the AWS x-ray service and then the AWS x-ray 32
service will provide data collation and analysis and visualization tools that 33
you can access through the AWS management console now the x-ray 34
software development kit it provides interceptors that can trace HTTP 35
requests to your application it will have client handlers and that will allow 36
you to monitor calls to AWS services for example calls to Amazon s3 by your 37
application and there also be a HTTP client that can monitor calls to other 38
internal or even external HTTP web services so the combination of that will 39
provide a very fine-grained and detailed analysis of the performance of your 40
application now the first thing we need to do is that we need to include the 41
x-ray software development kit in our application code once we've done that 42
the x-ray SDK can then send information or trace information to the x-ray daemon 43
about what your application needs monitoring now the x-ray daemon it will 44
be running on its own compute resources that you need to provide so that could 45
be on ec2 or it could be on a Lambda function but 46
we'll talk more about that later and the x-ray demon will get those 47
instructions from the x-ray software development kit and it will start to 48
collect data about what is to be collected and once it's got that data it 49
will collate that data and it will forward it on to or using the x-ray API 50
it will forward that on to the x-ray service once that information has been 51
forwarded on to the x-ray service then end-users can go into the AWS management 52
console go into the x-ray console and then you'll have a range of 53
visualization and analysis tools available to view the performance of 54
your application if you want to analyze that data yourself outside of the x-ray 55
console there are scripts and tools that are available you can use the AWS 56
command-line interface to download that data for example and use it elsewhere or 57
you can use one of the many AWS software development kits to create your own 58
application to further analyze and visualize this data yourself after our 59
application has run we can go into the x-ray console and we can see here we've 60
got a service graph and that details segments of our application and sub 61
segments of our application and we can click on those and get more details 62
about those specific parts of our application and how they are performing 63
so here we have some more information about the individual traces that are 64
being monitored by the x-ray daemon so we can see here we've got a score keep 65
segment of our application and that is divided up into a number of different 66
calls to AWS services here so we can see there that we've got game model a save 67
game that's taken quite a bit of time so we can see there is that there is an 68
opportunity possibly to increase the performance of our application when you 69
use x-ray you break your application code down into segments and your 70
application code using the software development kit will forward information 71
about those segments over to the x-ray demon now 72
those segments can also be broken down into further sub segments that can 73
provide even greater granularity the segment's information provided to the 74
x-ray demon will include among other things a resources name details about 75
the requests and details about the work that is being done now the segment that 76
is passed from the x-ray demon which will contain that application data that 77
is passed over to the x-ray service will be a JSON representation of a request 78
that your application serves and a segment document will be information 79
that is provided through your applications through the x-ray software 80
development kit and that segment document will be passed on to the x-ray 81
demon and that will convey information about that segment and what to monitor 82
with the x-ray demon now the x-ray software development kit it will gather 83
information from request and response headers it will gather information from 84
the code in your application because the SDK can actually send information about 85
your application directly to the demon and it can also monitor metadata and 86
collect that data about the AWS resources which are running your 87
application now the type of data that can be defined or collected by the x-ray 88
demon can be defined in the applications code through the software development 89
kit or in the x-ray configuration through sampling rules and we'll talk 90
more about those coming up data sent from the x-ray demon to the x-ray 91
service is used to create a service graph and that will show all of the 92
resources that are sending data and they will appear as services or round circles 93
in that graph and they will be connected with edges and those edges connect those 94
services that interact with each other now this service graph is actually a 95
JSON document that can be retrieved from the x-ray service and you 96
can do that without even using the AWS console and you can do that using either 97
direct HTTP calls to the API the x-ray API or by using the command-line 98
interface or the software development kit if you want to analyze that service 99
graph yourself offline for example the x-ray daemon will collect in batches 100
trace information to the x-ray service for analysis now a trace is a collection 101
of all of the segments and sub segments that are generated by a request and the 102
x-ray daemon will use a trace ID which we used to track the path of a request 103
through all of those segments and sub segments now the service graph data that 104
will be retained for up to 30 days so if you want to keep it for longer than that 105
you're going to have to download that yourself and it can be retrieved using 106
the command-line interface with the get service graph command or through the API 107
the x-ray API or one of the number of software development kits by default the 108
x-ray SDK will define to record the first request each second and 5% of any 109
additional requests so it doesn't record every single request but it takes a 110
sample of those requests now those samples are defined in the sampling 111
rules and they can be created to modify that default behavior so you can control 112
the amount of data that you record and modify that sampling behavior on the fly 113
so you don't need to shut anything down you can do it on the fly without 114
modifying or redeploying your application code 115
so those sampling rules they can be changed through the x-ray console or 116
through the command-line interface using the update sampling rule command or the 117
x-ray API or one of the many software development kits 118
the x-ray demon it will listen for UDP traffic from your application through 119
the AWS x-ray software development kit you must provide a compute resource to 120
run the x-ray daemon AWS doesn't do that automatically for you you need to make 121
that happen and once you have done that the daemon 122
endpoint must be supplied to the x-ray software development kit on your 123
application so that it knows where to send the segment documents to now the 124
daemon compute resource must also have I am permission to communicate with the 125
x-ray service so appropriate compute resources for running the daemon 126
you can obviously run it on an ec2 instance you can run it as a container 127
on an instance using ECS you can also use the elastic Beanstalk service to 128
create a process on your elastic beanstalk ec2 instances that will run 129
the daemon automatically for you you can run it on AWS lambda as a lander 130
function or using the compute resources of your application that is running on AWS Lambda 131
and finally can use it locally or you can deploy it locally on 132
a Linux Mac Windows or Docker container environment 133
now if you're going to be 134
using the x-ray daemon on a lambda function when you trace your lambda 135
function the x-ray daemon will be automatically running in the lambda 136
environment but the x-ray daemon you need to consider that it consumes the 137
maximum of 16 MB or 3% of your functions memory allocation so if that 138
becomes a problem for you you need to look at doing something different so 139
lambda if you exceed your functions memory limit will try to terminate the 140
x-ray daemon so to avoid that if you want to make sure that the x-ray daemon 141
is running all the time you to increase your functions memory limit 142
or run the x-ray daemon as a separate lambda function in order to use the 143
x-ray service your users your application the x-ray daemon that you're 144
running on a computer environment will all need to have permission set up in 145
order to use that service so if a user wants to use a console to have a look at 146
the data that has been visualized they will need to have permission to get that 147
information so there is a managed I am policy called AWS x-ray read-only access 148
and if that manager policy is applied to a user they will be able to get access 149
to that information through the console the SDK and the x-ray daemon that you 150
are running will need to have write access to the AWS x-ray service and so 151
there is again a managed I am policy to do that and it's called AWS x-ray daemon 152
write access put trace segments is a permission that the AWS x-ray software 153
development kit that will be running on your application it will need to pass 154
segment documents to the x-ray daemon and put telemetry records will need 155
their permission applied to the x-ray daemon for it to pass its 156
telemetry records over to the AWS x-ray service and if you want to configure 157
encryption key settings or sampling rules you'll need to have AWS x-ray full 158
access and that simply has action in your policy x-ray and then the wild-card 159
symbol to give full access if you want to do a local development and testing by 160
running the x-ray daemon on your local environment on your local computer then 161
you'll need to have I am user credentials that will have a tell us 162
x-ray full access to enable you to access those resources that are running 163
on AWS 164
the x-ray service map is a visual representation of that trace data that 165
is sent from the x-ray daemon now it consists of round service notes and 166
edges between those connected nodes so a service note you can click on it to view 167
requests generated for that node and you can also click on the edges between 168
nodes to view requests that are travel between those two service no connections 169
you can also view a histogram of traces by duration and that will help you to 170
track down the problem areas within your code by looking at those long duration 171
traces the health of each node will be color-coded 172
green for successful calls red if you've got any server faults and yellow for 173
client errors for example if you if your client made a request for a page in a 174
web application that didn't exist then you'll get a 404 error and net will come 175
up as a yellow in the node and finally per book for a throttling errors in your 176
application so here we can see we've got a service map with most of it in green 177
but we have something in red which which defines that we have a server fault 178
going on there there's a problem with our applications server we can see there 179
we've got those service nodes and we've got the edges that connect between those 180
related service nodes here we can see we've got another service node with a 181
problem but it doesn't exist in all of the traces only part of those and so 182
part of those have got a yellow error which means it's going to have a 400 a 183
client-side error for example if your client that is accessing your 184
application makes a request for a web page that doesn't exist and here we see 185
we've got a histogram of a trace so it helps us to identify where the majority 186
of these problems are occurring with that trace 187
we can also add colors to that so we can have it so that it will show different 188
types of traces for example if they're ok or if they got errors and they will 189
come up as a color code on that histogram the trace list in the 190
x-ray console can list traces by URL response code and any other data from 191
that trace summary the trace idea itself can be clicked to view a timeline for 192
all of the requests that will be displayed as a hierarchy of these 193
segments and the sub segments of that segment clicking on a segment or sub 194
segment in that timeline will display a form that will have a number of 195
different tabs there's an overview tab that will give you the segment details 196
the time the errors and faults the request and response information there's 197
a resources tab which of course will detail the resources involved in that 198
segment and there will be an annotations metadata and exceptions tab as well you 199
can filter the data using filter expressions and that allows you to 200
narrow those results down for example if you just wanted to get the traces that 201
had a duration of greater than or equal to 5 milliseconds and lesson eight 202
milliseconds that is how you would enter a filter expression for that and you can 203
also download what you can access the raw trace data itself in JSON format 204
there you can see we've got a trace list and it has a number of fields epping ID 205
age method response time the URL and you can simply by clicking on those caret 206
see you can filter that or organize that based on any of those any of those 207
features air being age your response time or URL or whatever and if you click 208
on any of those segments or sub segments within that trace list that will open up 209
a hierarchy or a graph of the hierarchy of the segment and its sub segments 210
there as well and it will also show the different times and which is very good 211
to visualize exactly where the performances being lost or gained and 212
clicking on a segment within that graph you will get this form which will have 213
the overview resources annotations metadata and X 214
actions tab there which can again give you more information that you may need 215
and if the amount of results that you get is too much and as you can see here 216
comes a warning saying that the scanned results is is more than 1,000 matching 217
traces so then you can go in and you can adjust your filter expression to narrow 218
those results and finally you can always have a look at the raw data in JSON 219
format there and you can always copy and paste that into a JSON editor and have a 220
good look at that if you wish there are a number of development tools that are 221
available for the x-ray service and those are provided through the x-ray API 222
and it provides access to all x-ray functionality through either direct HTTP 223
calls to the API endpoint or by using one of the many AWS software development 224
kits or using the AWS command line interface if you are going to be using 225
the x-ray service to monitor performance data from your application there are a 226
couple of put functions in the SDK that you need to be aware of or in the API or 227
CLI the first one they've been put trace segments and that allows the SDK or the 228
API or the CLI to upload segment documents to your x-ray daemon and that 229
will describe exactly what you want to monitor in your application and finally 230
put telemetry records that allows the daemon to upload telemetry information 231
or data to the x-ray service so that brings us to the end of a pretty 232
complicated lecture but the best way to learn this is to get your hands on it 233
and coming up next we're going to have some labs very detailed labs on the 234
x-ray service I look forward to seeing you in those