0
1
00:00:03,390 --> 00:00:08,889
Welcome back to BackSpace Academy.
In this lecture we're going to extend
1

2
00:00:08,889 --> 00:00:13,450
our knowledge of API gateway and have a
look at ways that we can improve the
2

3
00:00:13,450 --> 00:00:19,570
performance and reliability of our API's
we're going to look at throttling, so
3

4
00:00:19,570 --> 00:00:24,789
what happens when our API service gets
too many requests that it cannot handle
4

5
00:00:24,789 --> 00:00:29,949
and what we can do about making that
better, We'll look at API caching and how
5

6
00:00:29,949 --> 00:00:35,320
we can get our regularly used content
and case set so that we get very low
6

7
00:00:35,320 --> 00:00:40,900
latency response to our requests and
also so that we can reduce the costs
7

8
00:00:40,900 --> 00:00:45,640
associated with our API gateway service,
we will also look at usage plans that we
8

9
00:00:45,640 --> 00:00:51,040
can set up for individual users of our
service and we'll also look at some of
9

10
00:00:51,040 --> 00:01:01,059
the known issues around API gateway. When
your API service receives too many
10

11
00:01:01,059 --> 00:01:06,219
requests that it cannot handle, that's
when throttling occurs and so any
11

12
00:01:06,219 --> 00:01:13,659
further requests will receive a 429
too many requests error, the throttling
12

13
00:01:13,659 --> 00:01:19,060
and what actually gets throttled is
based upon a "token bucket algorithm" and
13

14
00:01:19,060 --> 00:01:24,850
that has two components, so first off
there is a steady state rate of requests
14

15
00:01:24,850 --> 00:01:31,329
per second limit, so what that means is a
over a one-second period you're only
15

16
00:01:31,329 --> 00:01:38,859
allowed to submit or receive so many
requests over that period if you exceed
16

17
00:01:38,859 --> 00:01:44,170
that number of requests over that one
second period any additional requests
17

18
00:01:44,170 --> 00:01:49,359
will be throttled and they will be sent
back with a 429 error. The other
18

19
00:01:49,359 --> 00:01:54,909
component is a burst total number of
concurrent requests limit, What that
19

20
00:01:54,909 --> 00:02:02,530
means is that at a specific time if your
API gets hit by a large amount of
20

21
00:02:02,530 --> 00:02:09,369
requests, concurrent requests, and it
exceeds a certain limit any addition 
21

22
00:02:09,369 --> 00:02:12,970
requests over that limit will again be
throttled and
22

23
00:02:12,970 --> 00:02:22,180
receive a 429 error message back. The
default account level of throttling
23

24
00:02:22,180 --> 00:02:27,610
limit is ten thousand for steady-state
so over a period of one second, you can
24

25
00:02:27,610 --> 00:02:34,870
receive ten thousand requests or at any
one point in time your limit will be
25

26
00:02:34,870 --> 00:02:39,790
five thousand requests or concurrent
requests and that is your burst limit,
26

27
00:02:39,790 --> 00:02:45,070
now this can be increased by applying to
AWS if you find that that is not enough
27

28
00:02:45,070 --> 00:02:50,950
for you. Now you can also define a custom
default throttling limit and that can be
28

29
00:02:50,950 --> 00:02:57,580
applied to a specific stage or API
method now those default throttling
29

30
00:02:57,580 --> 00:03:02,740
limits cannot exceed the account level
limits so if you've got ten thousand for
30

31
00:03:02,740 --> 00:03:07,959
your steady-state you can only go up to
ten thousand you can't go to eleven
31

32
00:03:07,959 --> 00:03:11,500
thousand you can put in 8,000 and the
same with your burst, you can put in a
32

33
00:03:11,500 --> 00:03:17,019
four thousand burst for an API but you
can't put in a six thousand unless your
33

34
00:03:17,019 --> 00:03:25,030
account level throttling has been
changed from the default. So let's have a
34

35
00:03:25,030 --> 00:03:30,580
look at how that works. On the vertical
axis here we've got the number of
35

36
00:03:30,580 --> 00:03:37,600
requests and our dotted red line there
it defines our 5000 requests burst limit
36

37
00:03:37,600 --> 00:03:44,739
and our blue line there shows the number
of outstanding requests that have been
37

38
00:03:44,739 --> 00:03:53,160
received by our API so initially we get
a burst of 5,000 requests come in
38

39
00:03:53,160 --> 00:03:59,019
anything above that 5,000 will be
limited and so at that point there we
39

40
00:03:59,019 --> 00:04:03,940
are burst limited we can only take in
5000 and then over a certain amount of
40

41
00:04:03,940 --> 00:04:09,340
time there that number of requests will
be reduced as our API gets through those
41

42
00:04:09,340 --> 00:04:18,100
requests and here we see it at point 0.5
seconds we have got through all of our
42

43
00:04:18,100 --> 00:04:24,250
requests and then we receive another
burst of requests and again we can go up
43

44
00:04:24,250 --> 00:04:29,170
to 5000 but we can't go above
that, So we can only receive 5,000 and
44

45
00:04:29,170 --> 00:04:34,990
then that again goes down when we get through
it, so it is burst limited but it's not
45

46
00:04:34,990 --> 00:04:39,520
steady-state limited and that is because
we received 5,000 here and then we've
46

47
00:04:39,520 --> 00:04:44,920
received another 5,000 requests in
one specific time there so the total
47

48
00:04:44,920 --> 00:04:50,320
number of requests that we received over
that one second period is 10,000 and
48

49
00:04:50,320 --> 00:04:57,820
that is below our steady state rate of
10,000 requests per second and we
49

50
00:04:57,820 --> 00:05:05,290
haven't exceeded our burst limit of 5000
there either. Okay so let's have a look
50

51
00:05:05,290 --> 00:05:13,090
at another example here so at one second
here we've got another burst of requests
51

52
00:05:13,090 --> 00:05:17,500
that have come in of concurrent requests
so we can only take 5,000 of those and
52

53
00:05:17,500 --> 00:05:24,220
anything above 5,000 will be burst
Limited now over a period of time we're
53

54
00:05:24,220 --> 00:05:28,930
getting through our API is getting
through those requests and we receive
54

55
00:05:28,930 --> 00:05:34,650
another lot of requests that come in
concurrently, now we can only go up to
55

56
00:05:34,650 --> 00:05:39,400
5,000 here so we can only receive
another two-and-a-half thousand to bring
56

57
00:05:39,400 --> 00:05:45,070
us up to here, so we've already received
5,000 plus two and a half thousand which
57

58
00:05:45,070 --> 00:05:49,180
is seven and a half thousand now we're
going to get through that five thousand
58

59
00:05:49,180 --> 00:05:54,160
over this period of time here, and then
at this point here we receive another
59

60
00:05:54,160 --> 00:06:00,700
lot of requests, now we're not going to
be burst limited here we're going to be
60

61
00:06:00,700 --> 00:06:06,870
steady state rate limited because over a
one-second period we cannot exceed
61

62
00:06:06,870 --> 00:06:12,040
10,000 requests, so the average number of
requests over that one second period
62

63
00:06:12,040 --> 00:06:18,400
cannot exceed 10,000, so we've got here
we received 5,000 then we received
63

64
00:06:18,400 --> 00:06:21,820
another two and a half thousand which is
seven and half thousand which leaves us
64

65
00:06:21,820 --> 00:06:25,960
only two and a half thousand left, so we
can only receive another two and a
65

66
00:06:25,960 --> 00:06:34,540
thousand requests above that it will be
steady state right limited. In order to
66

67
00:06:34,540 --> 00:06:39,730
reduce the possibility of throttling
occurring on your API service you can
67

68
00:06:39,730 --> 00:06:44,470
implement API
by caching what that does is it caches
68

69
00:06:44,470 --> 00:06:50,590
your popular requests to your end point
and that will reduce a number of calls
69

70
00:06:50,590 --> 00:06:55,840
to that end point because it will be
delivering the requests from the API
70

71
00:06:55,840 --> 00:07:00,520
cache that will reduce a latency of
those responses and it will obviously
71

72
00:07:00,520 --> 00:07:07,030
reduce a costs you'll be charged per
hour for using the API cache although
72

73
00:07:07,030 --> 00:07:12,699
this is not on the AWS Free tier but
it will reduce those number of calls to
73

74
00:07:12,699 --> 00:07:19,660
your endpoint that are already in your
API cache the default time to live for
74

75
00:07:19,660 --> 00:07:25,630
that cache is 300 seconds or five minutes
and the maximum is 3600 seconds or one
75

76
00:07:25,630 --> 00:07:35,560
hour. Usage plans can allow a deployed API
to be used as a product offering the way
76

77
00:07:35,560 --> 00:07:41,080
it does that it uses an API key that
identifies the individual client that is
77

78
00:07:41,080 --> 00:07:47,380
connected to the API and it will have
throttling and quota limits that will be
78

79
00:07:47,380 --> 00:07:53,320
enforced on that client and specific to
that client or group of clients they'll
79

80
00:07:53,320 --> 00:08:01,690
be generated in the console or imported
as a file in an API key file format the
80

81
00:08:01,690 --> 00:08:07,840
request will require that API key
whenever an a usage plan is enacted so
81

82
00:08:07,840 --> 00:08:14,020
you will not be able to access the API
unless you are using an API key when a
82

83
00:08:14,020 --> 00:08:19,780
usage plan is in place, a good feature of
this is that not only can it be used as
83

84
00:08:19,780 --> 00:08:27,030
a product offering it can also be sold
as a service on the AWS marketplace
84

85
00:08:27,240 --> 00:08:33,419
There is a significant list of known
issues with the AWS API Gateway service
85

86
00:08:33,419 --> 00:08:39,640
one of those being that Nginx may not
fully support SSL client authentication
86

87
00:08:39,640 --> 00:08:45,220
I think that's a significant one. Not all
of the swagger specification is
87

88
00:08:45,220 --> 00:08:49,570
implemented although that's not such a
big deal because pretty well everything
88

89
00:08:49,570 --> 00:08:54,339
you need has been implemented but there
is certainly a significant list of
89

90
00:08:54,339 --> 00:08:57,680
issues and I'm not going
to list them all here because that list
90

91
00:08:57,680 --> 00:09:02,720
is changing quite regularly as AWS
improves this service but for the for
91

92
00:09:02,720 --> 00:09:09,200
the latest information on that please go
to known issues in the AWS API gateway
92

93
00:09:09,200 --> 00:09:14,150
developer guide. So that brings us to the
end of this lecture I hope you've
93

94
00:09:14,150 --> 00:09:18,190
enjoyed it and I'll see you in the next
one