1

00:00:10,500  -->  00:00:16,510
welcome back to Backspace Academy In
this lecture we'll talk about AWS x-ray

2

00:00:16,510  -->  00:00:22,779
which is an AWS backend service that
could provide insights into how your

3

00:00:22,779  -->  00:00:25,159
applications are running on AWS

4

00:00:25,159  -->  00:00:28,690
we'll talk about how it works how do we modify

5

00:00:28,690  -->  00:00:32,424
our application code to integrate with
AWS x-ray

6

00:00:32,424  -->  00:00:34,680
how do we collect information

7

00:00:34,680  -->  00:00:39,640
about our applications performance and
how do we pass that information on to

8

00:00:39,640  -->  00:00:45,640
the x-ray service so that we can see
some visualizations and graphs of this

9

00:00:45,640  -->  00:00:46,140
performance

10

00:00:46,140  -->  00:00:52,000
we will talk about the x-ray
daemon and what that is and how we can

11

00:00:52,000  -->  00:00:57,940
use that to collect information from the
performance of our application and for

12

00:00:57,940  -->  00:00:59,539
that onto the x-ray service

13

00:00:59,539  -->  00:01:02,559
we'll talk
about the permissions that our application

14

00:01:02,559  -->  00:01:04,209
and the x-ray daemon will need

15

00:01:04,209  -->  00:01:06,520
we'll
talk about the permissions that any

16

00:01:06,520  -->  00:01:09,423
users that want to access this data will
need

17

00:01:09,423  -->  00:01:11,680
and we'll also talk about the x-ray

18

00:01:11,680  -->  00:01:17,170
console which allows us to view very
detailed fine grade information in the

19

00:01:17,170  -->  00:01:21,410
format of lists and graphs and other
visualizations of our data

20

00:01:21,410  -->  00:01:23,530
and finally

21

00:01:23,530  -->  00:01:28,509
we'll have a look at the x-ray
development tools that are available to

22

00:01:28,509  -->  00:01:31,770
use with our application

23

00:01:33,040  -->  00:01:39,950
the AWS x-ray service it will collect
data about requests that are served by

24

00:01:39,950  -->  00:01:45,140
your application and you can look at
that data to provide very fine-grain

25

00:01:45,140  -->  00:01:50,900
detail around the performance of your
application it consists of a software

26

00:01:50,900  -->  00:01:57,440
development kit a demon or daemon and
the actual x-ray console or x-ray

27

00:01:57,440  -->  00:02:01,910
service now the x-ray software
development kit you need to include that

28

00:02:01,910  -->  00:02:07,760
with your application and that will send
trace information to an x-ray daemon

29

00:02:07,760  -->  00:02:13,160
that will receive that information so
the x-ray daemon that will listen for

30

00:02:13,160  -->  00:02:19,040
UDP traffic from your application it
will buffer that data and then in

31

00:02:19,040  -->  00:02:25,820
batches it will upload it to the AWS
x-ray service and then the AWS x-ray

32

00:02:25,820  -->  00:02:31,490
service will provide data collation and
analysis and visualization tools that

33

00:02:31,490  -->  00:02:36,560
you can access through the AWS
management console now the x-ray

34

00:02:36,560  -->  00:02:41,900
software development kit it provides
interceptors that can trace HTTP

35

00:02:41,900  -->  00:02:47,810
requests to your application it will
have client handlers and that will allow

36

00:02:47,810  -->  00:02:55,010
you to monitor calls to AWS services for
example calls to Amazon s3 by your

37

00:02:55,010  -->  00:03:01,900
application and there also be a HTTP
client that can monitor calls to other

38

00:03:01,900  -->  00:03:09,260
internal or even external HTTP web
services so the combination of that will

39

00:03:09,260  -->  00:03:15,620
provide a very fine-grained and detailed
analysis of the performance of your

40

00:03:15,620  -->  00:03:21,470
application now the first thing we need
to do is that we need to include the

41

00:03:21,470  -->  00:03:26,989
x-ray software development kit in our
application code once we've done that

42

00:03:26,989  -->  00:03:34,090
the x-ray SDK can then send information
or trace information to the x-ray daemon

43

00:03:34,090  -->  00:03:40,580
about what your application needs
monitoring now the x-ray daemon it will

44

00:03:40,580  -->  00:03:44,450
be running on its own compute resources
that you need to provide so that could

45

00:03:44,450  -->  00:03:48,810
be on ec2
or it could be on a Lambda function but

46

00:03:48,810  -->  00:03:53,249
we'll talk more about that later
and the x-ray demon will get those

47

00:03:53,249  -->  00:03:57,569
instructions from the x-ray software
development kit and it will start to

48

00:03:57,569  -->  00:04:03,810
collect data about what is to be
collected and once it's got that data it

49

00:04:03,810  -->  00:04:09,930
will collate that data and it will
forward it on to or using the x-ray API

50

00:04:09,930  -->  00:04:15,419
it will forward that on to the x-ray
service once that information has been

51

00:04:15,419  -->  00:04:20,459
forwarded on to the x-ray service then
end-users can go into the AWS management

52

00:04:20,459  -->  00:04:25,889
console go into the x-ray console and
then you'll have a range of

53

00:04:25,889  -->  00:04:30,120
visualization and analysis tools
available to view the performance of

54

00:04:30,120  -->  00:04:35,820
your application if you want to analyze
that data yourself outside of the x-ray

55

00:04:35,820  -->  00:04:40,410
console there are scripts and tools that
are available you can use the AWS

56

00:04:40,410  -->  00:04:45,539
command-line interface to download that
data for example and use it elsewhere or

57

00:04:45,539  -->  00:04:50,310
you can use one of the many AWS software
development kits to create your own

58

00:04:50,310  -->  00:04:57,120
application to further analyze and
visualize this data yourself after our

59

00:04:57,120  -->  00:05:01,620
application has run we can go into the
x-ray console and we can see here we've

60

00:05:01,620  -->  00:05:06,659
got a service graph and that details
segments of our application and sub

61

00:05:06,659  -->  00:05:11,099
segments of our application and we can
click on those and get more details

62

00:05:11,099  -->  00:05:15,680
about those specific parts of our
application and how they are performing

63

00:05:15,680  -->  00:05:20,400
so here we have some more information
about the individual traces that are

64

00:05:20,400  -->  00:05:26,060
being monitored by the x-ray daemon so
we can see here we've got a score keep

65

00:05:26,060  -->  00:05:30,449
segment of our application and that is
divided up into a number of different

66

00:05:30,449  -->  00:05:36,060
calls to AWS services here so we can see
there that we've got game model a save

67

00:05:36,060  -->  00:05:40,169
game that's taken quite a bit of time so
we can see there is that there is an

68

00:05:40,169  -->  00:05:46,560
opportunity possibly to increase the
performance of our application when you

69

00:05:46,560  -->  00:05:53,969
use x-ray you break your application
code down into segments and your

70

00:05:53,969  -->  00:05:59,250
application code using the software
development kit will forward information

71

00:05:59,250  -->  00:06:03,390
about those
segments over to the x-ray demon now

72

00:06:03,390  -->  00:06:07,830
those segments can also be broken down
into further sub segments that can

73

00:06:07,830  -->  00:06:13,980
provide even greater granularity the
segment's information provided to the

74

00:06:13,980  -->  00:06:18,750
x-ray demon will include among other
things a resources name details about

75

00:06:18,750  -->  00:06:23,820
the requests and details about the work
that is being done now the segment that

76

00:06:23,820  -->  00:06:29,730
is passed from the x-ray demon which
will contain that application data that

77

00:06:29,730  -->  00:06:35,580
is passed over to the x-ray service will
be a JSON representation of a request

78

00:06:35,580  -->  00:06:41,370
that your application serves and a
segment document will be information

79

00:06:41,370  -->  00:06:46,920
that is provided through your
applications through the x-ray software

80

00:06:46,920  -->  00:06:50,370
development kit and that segment
document will be passed on to the x-ray

81

00:06:50,370  -->  00:06:56,670
demon and that will convey information
about that segment and what to monitor

82

00:06:56,670  -->  00:07:01,170
with the x-ray demon now the x-ray
software development kit it will gather

83

00:07:01,170  -->  00:07:06,330
information from request and response
headers it will gather information from

84

00:07:06,330  -->  00:07:11,130
the code in your application because the
SDK can actually send information about

85

00:07:11,130  -->  00:07:18,620
your application directly to the demon
and it can also monitor metadata and

86

00:07:18,620  -->  00:07:23,490
collect that data about the AWS
resources which are running your

87

00:07:23,490  -->  00:07:30,120
application now the type of data that
can be defined or collected by the x-ray

88

00:07:30,120  -->  00:07:34,710
demon can be defined in the applications
code through the software development

89

00:07:34,710  -->  00:07:40,470
kit or in the x-ray configuration
through sampling rules and we'll talk

90

00:07:40,470  -->  00:07:46,440
more about those coming up data sent
from the x-ray demon to the x-ray

91

00:07:46,440  -->  00:07:52,800
service is used to create a service
graph and that will show all of the

92

00:07:52,800  -->  00:07:58,350
resources that are sending data and they
will appear as services or round circles

93

00:07:58,350  -->  00:08:05,190
in that graph and they will be connected
with edges and those edges connect those

94

00:08:05,190  -->  00:08:11,460
services that interact with each other
now this service graph is actually a

95

00:08:11,460  -->  00:08:17,580
JSON document that can be
retrieved from the x-ray service and you

96

00:08:17,580  -->  00:08:22,949
can do that without even using the AWS
console and you can do that using either

97

00:08:22,949  -->  00:08:28,949
direct HTTP calls to the API the x-ray
API or by using the command-line

98

00:08:28,949  -->  00:08:33,419
interface or the software development
kit if you want to analyze that service

99

00:08:33,419  -->  00:08:39,510
graph yourself offline for example the
x-ray daemon will collect in batches

100

00:08:39,510  -->  00:08:45,930
trace information to the x-ray service
for analysis now a trace is a collection

101

00:08:45,930  -->  00:08:51,810
of all of the segments and sub segments
that are generated by a request and the

102

00:08:51,810  -->  00:08:57,960
x-ray daemon will use a trace ID which
we used to track the path of a request

103

00:08:57,960  -->  00:09:03,660
through all of those segments and sub
segments now the service graph data that

104

00:09:03,660  -->  00:09:08,310
will be retained for up to 30 days so if
you want to keep it for longer than that

105

00:09:08,310  -->  00:09:13,170
you're going to have to download that
yourself and it can be retrieved using

106

00:09:13,170  -->  00:09:18,660
the command-line interface with the get
service graph command or through the API

107

00:09:18,660  -->  00:09:26,490
the x-ray API or one of the number of
software development kits by default the

108

00:09:26,490  -->  00:09:35,040
x-ray SDK will define to record the
first request each second and 5% of any

109

00:09:35,040  -->  00:09:39,720
additional requests so it doesn't record
every single request but it takes a

110

00:09:39,720  -->  00:09:45,180
sample of those requests now those
samples are defined in the sampling

111

00:09:45,180  -->  00:09:52,440
rules and they can be created to modify
that default behavior so you can control

112

00:09:52,440  -->  00:09:58,980
the amount of data that you record and
modify that sampling behavior on the fly

113

00:09:58,980  -->  00:10:03,420
so you don't need to shut anything down
you can do it on the fly without

114

00:10:03,420  -->  00:10:07,290
modifying or redeploying your
application code

115

00:10:07,290  -->  00:10:12,080
so those sampling rules they can be
changed through the x-ray console or

116

00:10:12,080  -->  00:10:17,940
through the command-line interface using
the update sampling rule command or the

117

00:10:17,940  -->  00:10:23,490
x-ray API or one of the many software
development kits

118

00:10:23,490  -->  00:10:29,249
the x-ray demon it will listen for UDP
traffic from your application through

119

00:10:29,249  -->  00:10:36,059
the AWS x-ray software development kit
you must provide a compute resource to

120

00:10:36,059  -->  00:10:41,579
run the x-ray daemon AWS doesn't do that
automatically for you you need to make

121

00:10:41,579  -->  00:10:44,879
that happen
and once you have done that the daemon

122

00:10:44,879  -->  00:10:49,559
endpoint must be supplied to the x-ray
software development kit on your

123

00:10:49,559  -->  00:10:55,139
application so that it knows where to
send the segment documents to now the

124

00:10:55,139  -->  00:11:02,160
daemon compute resource must also have I
am permission to communicate with the

125

00:11:02,160  -->  00:11:08,069
x-ray service so appropriate compute
resources for running the daemon

126

00:11:08,069  -->  00:11:12,959
you can obviously run it on an ec2
instance you can run it as a container

127

00:11:12,959  -->  00:11:19,679
on an instance using ECS you can also
use the elastic Beanstalk service to

128

00:11:19,679  -->  00:11:25,679
create a process on your elastic
beanstalk ec2 instances that will run

129

00:11:25,679  -->  00:11:31,889
the daemon automatically for you
you can run it on AWS lambda as a lander

130

00:11:31,889  -->  00:11:37,740
function or using the compute resources
of your application that is running on AWS Lambda

131

00:11:37,740  -->  00:11:43,139
and finally can use it
locally or you can deploy it locally on

132

00:11:43,139  -->  00:11:48,615
a Linux Mac Windows or Docker container
environment

133

00:11:48,615  -->  00:11:51,299
now if you're going to be

134

00:11:51,299  -->  00:11:57,660
using the x-ray daemon on a lambda
function when you trace your lambda

135

00:11:57,660  -->  00:12:02,579
function the x-ray daemon will be
automatically running in the lambda

136

00:12:02,579  -->  00:12:08,369
environment but the x-ray daemon you
need to consider that it consumes the

137

00:12:08,369  -->  00:12:16,410
maximum of 16 MB or 3% of your
functions memory allocation so if that

138

00:12:16,410  -->  00:12:21,269
becomes a problem for you you need to
look at doing something different so

139

00:12:21,269  -->  00:12:28,799
lambda if you exceed your functions
memory limit will try to terminate the

140

00:12:28,799  -->  00:12:34,259
x-ray daemon so to avoid that if you
want to make sure that the x-ray daemon

141

00:12:34,259  -->  00:12:39,230
is running all the time you
to increase your functions memory limit

142

00:12:39,230  -->  00:12:47,839
or run the x-ray daemon as a separate
lambda function in order to use the

143

00:12:47,839  -->  00:12:53,600
x-ray service your users your
application the x-ray daemon that you're

144

00:12:53,600  -->  00:12:57,199
running on a computer environment will
all need to have permission set up in

145

00:12:57,199  -->  00:13:03,019
order to use that service so if a user
wants to use a console to have a look at

146

00:13:03,019  -->  00:13:08,959
the data that has been visualized they
will need to have permission to get that

147

00:13:08,959  -->  00:13:15,319
information so there is a managed I am
policy called AWS x-ray read-only access

148

00:13:15,319  -->  00:13:20,869
and if that manager policy is applied to
a user they will be able to get access

149

00:13:20,869  -->  00:13:26,809
to that information through the console
the SDK and the x-ray daemon that you

150

00:13:26,809  -->  00:13:32,540
are running will need to have write
access to the AWS x-ray service and so

151

00:13:32,540  -->  00:13:39,079
there is again a managed I am policy to
do that and it's called AWS x-ray daemon

152

00:13:39,079  -->  00:13:47,269
write access put trace segments is a
permission that the AWS x-ray software

153

00:13:47,269  -->  00:13:50,769
development kit that will be running on
your application it will need to pass

154

00:13:50,769  -->  00:13:58,220
segment documents to the x-ray daemon
and put telemetry records will need

155

00:13:58,220  -->  00:14:03,740
their permission applied to the x-ray
daemon for it to pass its

156

00:14:03,740  -->  00:14:09,949
telemetry records over to the AWS x-ray
service and if you want to configure

157

00:14:09,949  -->  00:14:16,519
encryption key settings or sampling
rules you'll need to have AWS x-ray full

158

00:14:16,519  -->  00:14:22,699
access and that simply has action in
your policy x-ray and then the wild-card

159

00:14:22,699  -->  00:14:28,429
symbol to give full access if you want
to do a local development and testing by

160

00:14:28,429  -->  00:14:35,389
running the x-ray daemon on your local
environment on your local computer then

161

00:14:35,389  -->  00:14:39,709
you'll need to have I am user
credentials that will have a tell us

162

00:14:39,709  -->  00:14:44,660
x-ray full access to enable you to
access those resources that are running

163

00:14:44,660  -->  00:14:47,679
on AWS

164

00:14:47,710  -->  00:14:54,320
the x-ray service map is a visual
representation of that trace data that

165

00:14:54,320  -->  00:15:00,520
is sent from the x-ray daemon now it
consists of round service notes and

166

00:15:00,520  -->  00:15:06,470
edges between those connected nodes so a
service note you can click on it to view

167

00:15:06,470  -->  00:15:12,200
requests generated for that node and you
can also click on the edges between

168

00:15:12,200  -->  00:15:18,260
nodes to view requests that are travel
between those two service no connections

169

00:15:18,260  -->  00:15:25,280
you can also view a histogram of traces
by duration and that will help you to

170

00:15:25,280  -->  00:15:31,490
track down the problem areas within your
code by looking at those long duration

171

00:15:31,490  -->  00:15:34,760
traces the health of each node will be
color-coded

172

00:15:34,760  -->  00:15:40,610
green for successful calls red if you've
got any server faults and yellow for

173

00:15:40,610  -->  00:15:46,130
client errors for example if you if your
client made a request for a page in a

174

00:15:46,130  -->  00:15:50,450
web application that didn't exist then
you'll get a 404 error and net will come

175

00:15:50,450  -->  00:15:56,120
up as a yellow in the node and finally
per book for a throttling errors in your

176

00:15:56,120  -->  00:16:02,180
application so here we can see we've got
a service map with most of it in green

177

00:16:02,180  -->  00:16:07,580
but we have something in red which which
defines that we have a server fault

178

00:16:07,580  -->  00:16:12,320
going on there there's a problem with
our applications server we can see there

179

00:16:12,320  -->  00:16:16,610
we've got those service nodes and we've
got the edges that connect between those

180

00:16:16,610  -->  00:16:22,880
related service nodes here we can see
we've got another service node with a

181

00:16:22,880  -->  00:16:28,070
problem but it doesn't exist in all of
the traces only part of those and so

182

00:16:28,070  -->  00:16:32,540
part of those have got a yellow error
which means it's going to have a 400 a

183

00:16:32,540  -->  00:16:37,070
client-side error for example if your
client that is accessing your

184

00:16:37,070  -->  00:16:43,040
application makes a request for a web
page that doesn't exist and here we see

185

00:16:43,040  -->  00:16:48,170
we've got a histogram of a trace so it
helps us to identify where the majority

186

00:16:48,170  -->  00:16:51,050
of these problems are occurring with
that trace

187

00:16:51,050  -->  00:16:55,640
we can also add colors to that so we can
have it so that it will show different

188

00:16:55,640  -->  00:16:59,720
types of traces for example if they're
ok or if they got errors and they will

189

00:16:59,720  -->  00:17:06,350
come up as a color code
on that histogram the trace list in the

190

00:17:06,350  -->  00:17:13,640
x-ray console can list traces by URL
response code and any other data from

191

00:17:13,640  -->  00:17:20,360
that trace summary the trace idea itself
can be clicked to view a timeline for

192

00:17:20,360  -->  00:17:24,079
all of the requests that will be
displayed as a hierarchy of these

193

00:17:24,079  -->  00:17:29,870
segments and the sub segments of that
segment clicking on a segment or sub

194

00:17:29,870  -->  00:17:34,400
segment in that timeline will display a
form that will have a number of

195

00:17:34,400  -->  00:17:38,570
different tabs there's an overview tab
that will give you the segment details

196

00:17:38,570  -->  00:17:43,520
the time the errors and faults the
request and response information there's

197

00:17:43,520  -->  00:17:48,320
a resources tab which of course will
detail the resources involved in that

198

00:17:48,320  -->  00:17:53,750
segment and there will be an annotations
metadata and exceptions tab as well you

199

00:17:53,750  -->  00:17:58,700
can filter the data using filter
expressions and that allows you to

200

00:17:58,700  -->  00:18:03,080
narrow those results down for example if
you just wanted to get the traces that

201

00:18:03,080  -->  00:18:08,840
had a duration of greater than or equal
to 5 milliseconds and lesson eight

202

00:18:08,840  -->  00:18:13,700
milliseconds that is how you would enter
a filter expression for that and you can

203

00:18:13,700  -->  00:18:20,980
also download what you can access the
raw trace data itself in JSON format

204

00:18:20,980  -->  00:18:25,909
there you can see we've got a trace list
and it has a number of fields epping ID

205

00:18:25,909  -->  00:18:33,080
age method response time the URL and you
can simply by clicking on those caret

206

00:18:33,080  -->  00:18:38,090
see you can filter that or organize that
based on any of those any of those

207

00:18:38,090  -->  00:18:44,990
features air being age your response
time or URL or whatever and if you click

208

00:18:44,990  -->  00:18:50,150
on any of those segments or sub segments
within that trace list that will open up

209

00:18:50,150  -->  00:18:56,210
a hierarchy or a graph of the hierarchy
of the segment and its sub segments

210

00:18:56,210  -->  00:18:59,900
there as well and it will also show the
different times and which is very good

211

00:18:59,900  -->  00:19:05,120
to visualize exactly where the
performances being lost or gained and

212

00:19:05,120  -->  00:19:10,549
clicking on a segment within that graph
you will get this form which will have

213

00:19:10,549  -->  00:19:14,570
the overview resources annotations
metadata and X

214

00:19:14,570  -->  00:19:18,669
actions tab there which can again give
you more information that you may need

215

00:19:18,669  -->  00:19:23,840
and if the amount of results that you
get is too much and as you can see here

216

00:19:23,840  -->  00:19:28,429
comes a warning saying that the scanned
results is is more than 1,000 matching

217

00:19:28,429  -->  00:19:33,139
traces so then you can go in and you can
adjust your filter expression to narrow

218

00:19:33,139  -->  00:19:38,840
those results and finally you can always
have a look at the raw data in JSON

219

00:19:38,840  -->  00:19:44,029
format there and you can always copy and
paste that into a JSON editor and have a

220

00:19:44,029  -->  00:19:49,129
good look at that if you wish there are
a number of development tools that are

221

00:19:49,129  -->  00:19:54,500
available for the x-ray service and
those are provided through the x-ray API

222

00:19:54,500  -->  00:20:01,820
and it provides access to all x-ray
functionality through either direct HTTP

223

00:20:01,820  -->  00:20:08,330
calls to the API endpoint or by using
one of the many AWS software development

224

00:20:08,330  -->  00:20:13,429
kits or using the AWS command line
interface if you are going to be using

225

00:20:13,429  -->  00:20:19,100
the x-ray service to monitor performance
data from your application there are a

226

00:20:19,100  -->  00:20:24,500
couple of put functions in the SDK that
you need to be aware of or in the API or

227

00:20:24,500  -->  00:20:30,889
CLI the first one they've been put trace
segments and that allows the SDK or the

228

00:20:30,889  -->  00:20:38,809
API or the CLI to upload segment
documents to your x-ray daemon and that

229

00:20:38,809  -->  00:20:44,720
will describe exactly what you want to
monitor in your application and finally

230

00:20:44,720  -->  00:20:52,700
put telemetry records that allows the
daemon to upload telemetry information

231

00:20:52,700  -->  00:20:58,970
or data to the x-ray service so that
brings us to the end of a pretty

232

00:20:58,970  -->  00:21:03,259
complicated lecture but the best way to
learn this is to get your hands on it

233

00:21:03,259  -->  00:21:07,429
and coming up next we're going to have
some labs very detailed labs on the

234

00:21:07,429  -->  00:21:12,279
x-ray service I look forward to seeing
you in those