So let's about SQS,
and at the core of SQS is a queue
because SQS is a simple queuing service.
So we have an SQS queue,
and it is going to contain messages.
And to contain messages,
well, something needs to send messages into our SQS queue
and whatever sends a message into our SQS queue
is called a producer.
So it's possible for us to have one producer,
but also have more.
You can have multiple producers
sending many messages into an SQS queue.
And the message could be whatever he wants.
For example, it could be process this order,
or process this video.
Whatever message you create goes into the queue.
Then something needs to process the messages from the queue
and receive them, and it's called consumers.
So consumers will poll the messages from the queue,
so that means that they will ask the queue,
do you have any message for me?
And the queue says, yes, here it is.
And the consumer will poll these messages
and get some information.
And then with that message,
it will process it and then delete it back from the queue.
And you may have multiple consumers
consuming messages from an SQS queue.
So a queuing service is here to be a buffer
to decouple between your producers and your consumers.
Now, SQS is a complicated service
and we'll see that in-depth,
but the very first offering is called Amazon SQS
for standard queues.
And SQS, historically on AWS, it is the oldest offering.
It was one of the first services on AWS.
It's over 10 years old so it's quite,
quite set in stone about how it works.
It's a fully managed service,
and it will be used to decouple applications.
So anytime you see application decoupling in your exam,
think about Amazon SQS.
Now, why is SQS so special?
Well, we get an unlimited throughputs.
That means you can send
as many messages per second as you want,
and the queue can have as many messages as you want as well.
So there's no limit on throughputs
and no limits in the number of messages in the queue.
Now, each message is short-lived.
What does that mean?
That means that by default,
the message will stay in the queue for four days
and the maximum amount of time
that a message can be in a queue is 14 days.
That means that as soon as you send a message to a queue,
it has to be read by consumer and deleted from the queue
after being processed within that retention period,
otherwise, it will be lost.
Then we have low-latency.
So SQS means that, for them,
is that whenever you send a message
or read a message from SQS,
you will get a response very quickly,
less than 10 milliseconds on publish and receive.
And the messages in SQS have to be small.
They have to be less than 256 kilobytes per messages sent.
Now SQS is a queuing service,
so you can see high throughput, high volume, and so on,
and so, it is possible to have duplicates messages.
That means that, for example,
sometimes a message will be delivered twice,
and so this is why it's called at least once delivery.
And if you go ahead with writing an application,
you need to take that into account.
It can also have out of the order messages,
which means it's best effort ordering,
and we'll see there is another type of offering from SQS
that can deal with that limitation,
but we'll see this later on in this section.
So, let's go back to our message producers.
So, messages that are up to 256 kilobytes
are sent into SQS by producers.
And how does it happen?
Well, the producers will send the messages to SQS
using an SDK, software development kits.
And the API to send a message to SQS is called SendMessage.
Very simple.
Now the message, it will be written,
it will be persisted into the SQS queue
until a consumer reads that message and deletes it,
which signifies that the message has been processed.
We know the retention.
And so, what's a use case for producing messages?
For example, you want to process an order,
for example, packets, and then ship it to the center.
And so you wanna do this in your own time
so you will send a message into the SQS queue
with maybe some information,
such as the order ID, the customer ID,
and any attributes you may want.
For example, the address, and so on.
And then your consumer, that is in application rights,
will have to deal with that message itself.
So, again, to confirm,
SQS standard has unlimited throughputs.
So we've seen about producers. Very simple.
Now let's look about consumers.
So consumers, they are applications
that we have to write with some code
and these applications can be running on EC2 instances,
so your virtual servers on AWS,
but also they can be running on your own on-premises servers
if you wanted to,
or, we haven't seen it yet,
but they can also be running
on Lambda functions on AWS Lambda.
We'll see in this course,
it is a serverless compute type of service.
What this means is that you can just read messages
directly from them as well.
We'll see this later on this course, do not worry.
So, going back to our simple use case about EC2 instances,
our queue has a consumer
and the consumer polls for SQS for messages.
And that means that the consumer will ask the queue,
do you have messages for me?
And the consumer may receive up to 10 messages at a time.
So if there are messages in the SQS queue,
it will receive a valid response saying
here are the messages that are waiting for you.
Then the consumer, it's your code,
has a responsibility to process these messages.
For example, insert some orders into an RDS database.
So you will go ahead and for each order,
you will insert it into your RDS database,
something that you have to write, obviously with your code,
and then because these messages have been processed
because they have been received
and inserted into an Amazon RDS database,
your consumer will go ahead
and delete these messages from the queue
using the DeleteMessage API.
And this will guarantee that no other consumer
will be able to see these messages
and therefore, the message processing is complete.
So now we can scale this up.
We can have multiple consumers at a time.
So our SQS queue can have multiple consumers
that will receive and process these messages in parallel.
So here we have three on EC2 instances,
and so each consumer will receive
a different set of messages by calling the poll function.
And so, if somehow a message
is not processed fast enough by a consumer,
it will be received by other consumers,
and so this is why we have at least once delivery.
And this is also why we have best-effort message ordering.
Now, as I said, when the consumers
are done with the messages, they will have to delete them,
otherwise, other consumers will see these messages.
And so what this means is that with SQS queues,
if we need to increase the throughputs
because we have more messages,
then we can add consumers and do horizontal scaling
to improve the throughput of processing.
And so, if you remember what we said,
this is a perfect use case for using SQS
with your Auto Scaling groups, or ASG.
So, what does that mean?
Well, that means that your consumers
will be running on EC2 instances
inside of an Auto Scaling group
and they will be polling for messages from the SQS queue.
But now your Auto Scaling group has to be scaling
on some kind of metric,
and a metric that is available to us is the Queue Length.
It's called ApproximateNumberOfMessages.
It is a CloudWatch Metric that's available in any SQS queue.
And we could set up an alarm,
such as whenever the queue length go over a certain level,
then please set up a CloudWatch Alarm,
and this alarm should increase the capacity
of my Auto Scaling group by X amount.
And this will guarantee that
the more messages you have in your SQS queue,
maybe because there's a surge of orders on your websites,
the more EC2 instances will be provided
by your Auto Scaling group,
and you will accordingly process these messages
at a higher throughputs.
So this is a very common integration
that you will see in the exam.
Now SQS, again, the use case
is to decouple between applications, so application tiers.
So, for example, let's take an example of an application
that processes videos.
We could have just one big application
that's called a front-end that will take the request
and whenever a video needs to be processed,
it will do the processing
and then insert that into an S3 bucket.
But the problem is that processing
may be very, very long to do
and it may just slow down your websites
if you do this in the front-end here.
So instead, you can decouple your application here and say,
wait a minute, the request of processing a file
and the actual processing of a file
can happen in two different applications.
And therefore, whenever you take a request
to process a file,
you will send a message into an SQS queue.
Now, when you do the request to process,
that file will be in the SQS queue
and you can create a second processing tier
called the back-end processing application
that will be in its own Auto-Scaling group
to receive these messages, process these videos,
and insert them into an S3 bucket.
So as we can see here with this architecture,
we can scale the front-end accordingly,
and we can scale the back-end accordingly as well,
but independently.
And because the SQS queue has unlimited throughputs
and it has unlimited number of messages
in terms of the queue, then you are really safe,
and this is a robust and scalable type of architecture.
And also, for your front-end,
you can use the optimal type of EC2 instances
or architecture for your front-end.
And for the back-end,
maybe if you're doing some video processing,
you can use some EC2 instances
that have a GPU, a graphical processing unit,
because you know that these type of instances
will be optimal for doing this kind of workload.
So this is the kind of architecture
that will come up in the exam
and that you're expected to know,
and this is an amazing and tremendous use case
for SQS queues.
Finally, SQS security.
So we have encryption in-flight
by sending and producing messages using the HTTPS API,
we get at-rest encryption using KMS keys,
and if we wanted to, we can do client-side encryption,
but that means that the client has to perform
the encryption and the decryption itself.
It's not something that's supported by SQS out of the box.
For access controls, IAM policies are going to be able to
regulate access to the SQS API,
but we also have SQS access policies,
which are similar to S3 bucket policies,
and they're very helpful when you want to do
cross-account access to SQS queue,
or when you want you to allow other services,
such as SNS that we'll see very soon, or Amazon S3,
to write to an SQS queue,
for example, with S3 events.
So that's it for SQS for an overview.
I hope you liked it,
and I will see you in the next lecture for some practice.