Welcome back to BackSpace Academy. In this lecture we're going to extend 1
our knowledge of API gateway and have a look at ways that we can improve the 2
performance and reliability of our API's we're going to look at throttling, so 3
what happens when our API service gets too many requests that it cannot handle 4
and what we can do about making that better, We'll look at API caching and how 5
we can get our regularly used content and case set so that we get very low 6
latency response to our requests and also so that we can reduce the costs 7
associated with our API gateway service, we will also look at usage plans that we 8
can set up for individual users of our service and we'll also look at some of 9
the known issues around API gateway. When your API service receives too many 10
requests that it cannot handle, that's when throttling occurs and so any 11
further requests will receive a 429 too many requests error, the throttling 12
and what actually gets throttled is based upon a "token bucket algorithm" and 13
that has two components, so first off there is a steady state rate of requests 14
per second limit, so what that means is a over a one-second period you're only 15
allowed to submit or receive so many requests over that period if you exceed 16
that number of requests over that one second period any additional requests 17
will be throttled and they will be sent back with a 429 error. The other 18
component is a burst total number of concurrent requests limit, What that 19
means is that at a specific time if your API gets hit by a large amount of 20
requests, concurrent requests, and it exceeds a certain limit any addition 21
requests over that limit will again be throttled and 22
receive a 429 error message back. The default account level of throttling 23
limit is ten thousand for steady-state so over a period of one second, you can 24
receive ten thousand requests or at any one point in time your limit will be 25
five thousand requests or concurrent requests and that is your burst limit, 26
now this can be increased by applying to AWS if you find that that is not enough 27
for you. Now you can also define a custom default throttling limit and that can be 28
applied to a specific stage or API method now those default throttling 29
limits cannot exceed the account level limits so if you've got ten thousand for 30
your steady-state you can only go up to ten thousand you can't go to eleven 31
thousand you can put in 8,000 and the same with your burst, you can put in a 32
four thousand burst for an API but you can't put in a six thousand unless your 33
account level throttling has been changed from the default. So let's have a 34
look at how that works. On the vertical axis here we've got the number of 35
requests and our dotted red line there it defines our 5000 requests burst limit 36
and our blue line there shows the number of outstanding requests that have been 37
received by our API so initially we get a burst of 5,000 requests come in 38
anything above that 5,000 will be limited and so at that point there we 39
are burst limited we can only take in 5000 and then over a certain amount of 40
time there that number of requests will be reduced as our API gets through those 41
requests and here we see it at point 0.5 seconds we have got through all of our 42
requests and then we receive another burst of requests and again we can go up 43
to 5000 but we can't go above that, So we can only receive 5,000 and 44
then that again goes down when we get through it, so it is burst limited but it's not 45
steady-state limited and that is because we received 5,000 here and then we've 46
received another 5,000 requests in one specific time there so the total 47
number of requests that we received over that one second period is 10,000 and 48
that is below our steady state rate of 10,000 requests per second and we 49
haven't exceeded our burst limit of 5000 there either. Okay so let's have a look 50
at another example here so at one second here we've got another burst of requests 51
that have come in of concurrent requests so we can only take 5,000 of those and 52
anything above 5,000 will be burst Limited now over a period of time we're 53
getting through our API is getting through those requests and we receive 54
another lot of requests that come in concurrently, now we can only go up to 55
5,000 here so we can only receive another two-and-a-half thousand to bring 56
us up to here, so we've already received 5,000 plus two and a half thousand which 57
is seven and a half thousand now we're going to get through that five thousand 58
over this period of time here, and then at this point here we receive another 59
lot of requests, now we're not going to be burst limited here we're going to be 60
steady state rate limited because over a one-second period we cannot exceed 61
10,000 requests, so the average number of requests over that one second period 62
cannot exceed 10,000, so we've got here we received 5,000 then we received 63
another two and a half thousand which is seven and half thousand which leaves us 64
only two and a half thousand left, so we can only receive another two and a 65
thousand requests above that it will be steady state right limited. In order to 66
reduce the possibility of throttling occurring on your API service you can 67
implement API by caching what that does is it caches 68
your popular requests to your end point and that will reduce a number of calls 69
to that end point because it will be delivering the requests from the API 70
cache that will reduce a latency of those responses and it will obviously 71
reduce a costs you'll be charged per hour for using the API cache although 72
this is not on the AWS Free tier but it will reduce those number of calls to 73
your endpoint that are already in your API cache the default time to live for 74
that cache is 300 seconds or five minutes and the maximum is 3600 seconds or one 75
hour. Usage plans can allow a deployed API to be used as a product offering the way 76
it does that it uses an API key that identifies the individual client that is 77
connected to the API and it will have throttling and quota limits that will be 78
enforced on that client and specific to that client or group of clients they'll 79
be generated in the console or imported as a file in an API key file format the 80
request will require that API key whenever an a usage plan is enacted so 81
you will not be able to access the API unless you are using an API key when a 82
usage plan is in place, a good feature of this is that not only can it be used as 83
a product offering it can also be sold as a service on the AWS marketplace 84
There is a significant list of known issues with the AWS API Gateway service 85
one of those being that Nginx may not fully support SSL client authentication 86
I think that's a significant one. Not all of the swagger specification is 87
implemented although that's not such a big deal because pretty well everything 88
you need has been implemented but there is certainly a significant list of 89
issues and I'm not going to list them all here because that list 90
is changing quite regularly as AWS improves this service but for the for 91
the latest information on that please go to known issues in the AWS API gateway 92
developer guide. So that brings us to the end of this lecture I hope you've 93
enjoyed it and I'll see you in the next one