Hi, everyone, and welcome back.
Now in today's video, we are going to discuss about Dynamo DB streams.
Now at a high level overview, Dynamo, which streams captures the time ordered sequence of item level
modification in any Dynamo DB table, and it stores the information in a log for up to 24 hours.
For example, in this diagram you see client is performing multiple operations related to addition,
related to deletion and so on on the Dynamo DB.
So Dynamo DB stores all of this information in a specific log file through which the user can identify
the exact record.
Even the record need and other details will be logged as part of the Dynamo DB streams.
And this is what the second point states where application can access this log file to view the items
as they appeared before and after they were modified in real time.
And this is a great benefit not only in terms of auditing security, but also in terms of reconstructing
the entire table.
Now, in this slide, I have added a screenshot related to how a log might really look like.
Now, if you look into this specific log file in Cloud Watch, you have a remove and the keys in the
case you have a user ID and you have zero one and you also have something called this old image and
in the course name you have the E certification course.
So it will give you great detail related to what is the old record that was removed, what is the new
record that was added and so on.
So let's do one thing.
Let's go ahead and try it out practically.
So I'll search for Dynamo DB And within here let's go ahead and quickly create a new table.
I'll just call it as user underscore data, the partition key, let's just put it as user ID and we'll
go ahead and create a table.
So the status is creating.
If I quickly do a refresh, the status is now active.
Now let's quickly go to a table here and within here you have exports and streams.
Inside this tab you have Dynamo DB streams.
Let's go ahead and enable it.
Now, when you enable it in the view type, you have various things like new image, old image, new
and old image.
So basically, depending upon what you want to record, whether you only want to record the new things
that are being added to Dynamo.
DB you want to record both the new and old images and so on.
So I'll select the new and old images which states both the new and the old image of the change item.
So this is interesting.
Let's go ahead and use that and I'll also click on Enable Stream.
So if we go a bit down, you'll see our Dynamo TV stream is enabled and it is also giving us the URL
associated with it.
Now what we'll be doing will be integrating this dynamo DB with the lambda function, and that lambda
function will push the logs associated with the Dynamo DB stream to the Cloud Watch Log Service.
So before we do that, let's quickly have a sample use case based architecture, which gives you an
overview on how Lambda can be useful here.
So let's say that you have a Dynamo DB table over here and in this step one a new item have been added
to the Dynamo DB table.
Now, since the Dynamo DB streams is enabled, the data related to this new item would be added as part
of the Dynamo DB stream.
Now, as soon as a new record is added in the Dynamo DB stream, this will trigger a lambda function
that have been created, and this lambda function is integrated with the SNS topic and this SNS topic
will go ahead and send an email or maybe an SMS stating that there is a new item that have been added
in the Dynamo DB table.
So this is one of the use cases where lambda function proves to be useful.
Lambda can be used for other use.
Cases like it can retrieve the data from Dynamo DB streams and it can add it to cloud watch logs and
you can directly view it from cloud watch logs.
All right.
So with this base set, let's go ahead and quickly create a lambda function.
Now, from here, let's quickly open up the lambda.
All right.
Now, within the Lambda console, let's quickly create a function and we'll be using a blueprint over
here and let's search for Dynamo.
DB And you have Dynamo DB Process Stream Python.
So I'll be selecting this.
Let's click on configure, let's give a function name, I'll say Dynamo Stream to Cloud Watch and this
specific lambda function that we are creating, it needs access to this Dynamo DB stream and it also
needs access to the cloud watch log service.
So what we'll do will quickly create a new IAM role that allows our Lambda function to perform these
operations.
Now from the IEM console, let's quickly go to rules.
We'll create a new rule.
This rule will be for Lambda.
Let's do a next the permission just for simplicity.
Let's say Dynamo DB Full access.
Let's search for a cloud which will give our cloud watch full access just for simplicity.
Let's do a next the roll name.
You can just say lambda role for Dynamo DB
and currently there are two policies that are associated.
We'll go ahead and create a role.
All right.
So our role is created.
Let's go back to lambda.
Within the lambda, I'll go ahead and select an existing role here.
And within you you should see our lambda role for Dynamo.
DB Let's go a bit down in the Dynamo DB trigger.
You can go ahead and select the Dynamo DB table, which in this case is user data starting position.
You can just change it to dream horizon.
We'll go a bit down.
We'll create a function.
All right.
So when a trigger is created, just also verify whether it is in the enabled state or not.
So as of now, it is giving us this note stating that configured trigger of user underscore data is
in the disabled state.
So we do not really want it in the disabled state.
So what we'll do will go ahead and will enable it.
Great.
So once this is enabled, we are set.
All we need to do is we have to go ahead and test things out.
So let's go ahead and add some items to our Dynamo DB table.
So I'll go to actions.
I'll click on Create Item, let's say zero one, I'll add a new attribute of string, let's say name
of Ellis, and I'll add one more attribute.
Let's say interest and interest.
I'll say traveling, all right.
And I'll create an item.
So if you explore the table items over here, you see our item is ready.
Now let's go ahead and also modify this.
I'll go ahead and edit the item.
Let's say Alice has an interest from traveling to astronomy, so I change it.
I'll click on Save Changes and based on this, our lambda should have been triggered.
Now if you quickly open up the table once more, let's go to export sign streams.
In the trigger section you'll see our lambda function is appearing, the state is enabled and the last
process result is okay.
So the next part is to go to the cloud, watch log service to verify if Lambda has added the details
or not.
So I'll search for cloud watch and within here will quickly go to log groups and within here you should
see a log group related to lambda and the name is Dynamo.
Live stream to Cloud Watch.
Let's open this up.
You have a log stream.
Let's open this up and you have all the required details.
Let's look into it from scratch.
In fact, let me zoom in a little so it's easier to read.
Now in the first stage you have an insert record, the key user ID and you have zero one and the new
image with the interest is traveling.
This is something that we already know and the name was Alice, so this was one part.
Then you have the end request over here and then you have a new set of requests.
This time it was a modification request and what was modified here, the interest was modified and it
was astronomy.
And it is also showing you the old image.
And what was the old image?
The old image was traveling over here and then the request has ended.
Great.
So I hope that this you understood the basics of what Dynamo Stream allows us to do.
So before we conclude, let's quickly look into some of the use cases where Dynamo Live Stream can prove
to be useful.
First one is that it allows setting up the relationship across multiple tables in which, based on a
value of an item from one table, you can update the item in second table.
Again, this is something that you can achieve using lambda.
Second is you can trigger an event based on a particular item change.
This is very similar to the use case that we were discussing using the Dynamo DB, Lambda and maybe
Psns or any other service can be used.
Third is you can audit or even archive the data and it can also be used for replicating the data across
multiple tables.
So that's the high level overview and with this will conclude today's video.