Welcome back to BackSpace Academy. In this lecture we're going to talk about 2
encryption on AWS which is a very important subject. 3
Now I deal with a lot of senior IT managers for quite large companies and a 4
lot of these people just don't really understand encryption, and it's not their fault , 5
the problem is the way that encryption is taught can be confusing. 6
What I do is that I will break it down to very simple building blocks and from 7
there you will build and further understand how that applies to AWS and I 8
promise you at the end of this lecture you will well and truly get encryption 9
on AWS. We'll start off by discussing the two main types of encryption being 10
symmetric and asymmetric encryption. We'll talk about the key management 11
service that integrates with many AWS services. We'll talk about the CloudHSM 12
service that can allow us to securely store and manage our encryption keys. 13
We'll talk about the secrets manager which can be used for storing our 14
secrets that are in text files for example our username and password. 15
We'll talk about the AWS encryption software development kit specifically for 16
cryptographic operations, and finally we'll talk about how we can encrypt our 17
data on AWS both at rest and in transit. Symmetric encryption, now that uses the 18
same encryption key to both encrypt our data and to decrypt our data. So that 19
requires our sender who is going to be encrypting our data and sending it and 20
the receiver who is going to be decrypting our data, they need to agree 21
on the same key and keep it safe. They need to keep it secret and secure 22
because if that key is intercepted in transit or it's stolen or 23
some other way it has been compromised then that data can be compromised as 24
well. 25
Now to understand asymmetric encryption we need to understand the concept of a 26
two-way lock. So the easiest way to understand this is to think about, 27
we've got a padlock here and that padlock has two keys. Now one of those keys turns the 28
lock in a clockwise direction and it locks that lock, and if you try to turn 29
the key in the anti-clockwise direction to unlock that lock it won't let you do that, 30
but if you use your other key now that key only turns in the 31
anti-clockwise direction, and so you can put the other key into that lock turn to 32
anti-clockwise and you will unlock that two-way lock . So one key can only lock it, 33
can only go in one direction, the other key can only unlock, and so what we have 34
is that we have a public key and that can is use to lock that lock, it cannot 35
be used to unlock that lock, and then we have a private key now that private key 36
can be used to unlock that lock, but the private key cannot be used to lock the 37
lock at all. 38
So asymmetric encryption, which is also sometimes referred to as public key 39
encryption uses a pair of keys. A public keys that is only used to encrypt your 40
data, and a private key which is only ever used to decrypt your data. Now the 41
public and private key are mathematically linked but, if you have 42
the public key it's not possible to mathematically work out the private key. 43
You just wouldn't have a computer with enough resources to carry out that 44
massive algorithm to do that. So it's not possible to actually create one from the 45
other. So you may be wondering why do we have a public key and why do we have a 46
private key and what is the purpose of that. So anyone who has this public key 47
can encrypt that data for the receiver. So if you want people to send you some 48
data and encrypt it before they send it to you, what you can do is that you can 49
send them out your public key and they can use that public kree to encrypt that 50
data before it is transmitted over to you. They never receive the private key. 51
That private key is kept with you the receiver of that data and you use that 52
private key to decrypt the data and that is great because that private key never 53
leaves you and so if that public key, which is sent out to the transmitter, 54
if that is ever intercepted, it is of no use to anyone because a public key can never 55
be used to decrypt data. Only a private key can and that is a magic of 56
asymmetric encryption and why it is so important. Ok, so let's have a look at a 57
real-life example. Let's just say that we've launched an ec2 instance and we 58
want to connect to that instance using secure shell or SSH. 59
So the first thing that we need to do is obviously we'll need to make sure 60
that our our security group rules allow a connection on port 22 for SSH. 61
The other thing that we need to do is that we need to make sure that we download 62
the PEM file for connection to that ec2 server. Once we've got that, then we can 63
go through the process of connecting to that ec2 server. The first step is that 64
we need to make sure that we're running our ssh client on our computer that 65
we're connecting to, from our office or home or wherever it is, and once we've 66
got that ssh client and we've got the endpoint for our server, we can initiate 67
a session with that ec2 server. Once we've done that, the ec2 server will 68
provide us with a host key and that host key will be our public encryption key 69
and that will occur when we attempt to connect with that ec2 server. So once 70
we've got that public key and we've got our PEM private key, we can use 71
asymmetric cryptography to verify the identity of that ec2 server that we're 72
connecting to. Once that's done and once we're satisfied and the ec2 server is 73
satisfied with our identity, then this server will generate a symmetric key and 74
we use that symmetric key for bi-directional encryption from that 75
point onwards or for that session, and that way all of the traffic being sent 76
to and from this ec2 server will be encrypted while it is in transit. 77
One very important thing to remember is that, what happens if you lose your PEM file? 78
If that occurs, you will not be able to connect to your ec2 instance and AWS 79
will not be able to provide it for you because there is your private key and 80
only you have a copy of that. AWS does not keep a copy of your private key. 81
So in that situation, what you are going to 82
have to do is, that you will have to stop that instance. You will have to create 83
another private key or another key pair and then you're going to have to 84
associate that key pair by going into the user data, quite a complex process, 85
and associate that new key pair with that ec2 instance and then you'll be 86
able to use your new key pair to connect in. 87
The AWS key management service or KMS for short, that integrates with a lot of 88
different AWS services and it creates and manages the encryption keys that I 89
use with those services. It uses a single master key that is secured by AWS. So it 90
is symmetric encryption. We don't have a public and private key, we just have a 91
single key. Now that key always remains on AWS and you cannot download that at 92
all. The key is rotated automatically by the 93
KMS service every year by default. If automatic key rotation is enabled. If you 94
want to use the KMS service to encrypt any data then you must have IAM 95
permissions to use those keys, and so those permissions can also be granted to 96
other other AWS accounts as well if they would like to decrypt that data. Now the 97
list of services that integrate with the LMS is growing but if you go to the 98
Amazon website and go to slash KMS slash features you'll get a good list of all 99
of these services that integrate with the KMS. Because the KMS service uses 100
symmetric encryption, you may be questioning the security of that, but 101
there's no need to be alarmed because we have another process involved with the 102
KMS and that is called envelope encryption. So the way that works is that 103
we have a data key that is used to encrypt that data and that is our single 104
encryption key. Now to make sure that that data key is secure, so if someone 105
gets hold that key, what the KMS does is it actually encrypts that encryption 106
key. It encrypts the data key using a KMS master key, and so those master keys are 107
stored by AWS and they're called your customer master keys or CMKs 108
for short. Now although symmetric encryption with a single data key is 109
being used to encrypt that data, the KMS service can actually support symmetric 110
and asymmetric customer master keys that are going to be used to encrypt that 111
data key, and those CMK's are stored in a FIPS 140 - 2 compliant hardware 112
security module. So they are very safely kept. If for whatever reason the data key 113
is compromised, it's not a problem because it's encrypted using that master 114
key. 115
The AWS cloud HSM service is a cloud-based hardware security module or 116
HSM for short and it is used to secure and store your encryption keys, and it 117
conforms with the security requirements for cryptographic modules 118
federal information processing standard or FIPS for short 140 - 2 level 3. 119
What that means is that your encryption keys that are stored in this HSM module 120
are extremely secure. You control and manage your keys. AWS does not have 121
access to your keys, and it's only accessible, these keys are only 122
accessible, by users that you specify. The Cloud HSM will run inside a VPC inside 123
of your account. The encryption keys are installed inside of that Cloud HSM will 124
be accessible by a HSM client application that is running on an ec2 125
server that you launch, and there are HSM commands within the AWS software 126
development kit. It's also multiple availability zone as well. 127
Some use cases for CloudHSM. You may want to use a CloudHSM cluster as a 128
custom KMS key store instead of using the default KMS key store. You may 129
want to do that if you want to keep all of your keys in the one location using 130
Cloud HSM. Another use case you can use it to offload the SSL processing from 131
your web servers over to CloudHSM. By doing that, cloudHSM will take all of 132
that processing for the HTTPS requests and take that load off your web servers 133
and make sure that your web services are more available as a result of that and 134
reduce the latency as well. You can also use it to protect the 135
private keys that you use with a third-party certificate authority. So if 136
you're using another service other than AWS to issue your SSL 137
certificates, then you can protect those private keys using a CloudHSM as well. 138
The AWS secrets manager is a cloud-based service to store our secrets and that 139
could be database credentials, such as a username and password for logging into a 140
database. It could be our passwords, third party API keys. It could also be 141
arbitrary text that we can put in there as well that we would like to manage as 142
a secret. The big advantage of doing that and using the secrets manager is that we 143
can replace hard coded credentials that we've put in our code. For example, we may 144
have a application that is a web application that's running on an ec2 145
server and we may have put hard-coded credentials inside of that code to 146
connect to something else, and so that's a problem if someone manages to get hold 147
of our code. They've also got hold of the credentials as well. So instead of doing 148
that we can replace that with an API call 149
from that web server over to the AWS secrets manager, and then the secrets 150
manager can authenticate that server and then handover that secret to that ec2 151
server. It uses the AWS KMS service to encrypt the secret. So they are encrypted 152
and secure. The secrets can be created and retrieved using the AWS console, the 153
AWS CLI, or one of the many software development kits. To create or retrieve a 154
secret from the secrets manager you will need to have IAM permissions that will 155
allow that to occur. Ok, here is how we would create a secret 156
using the command line interface. The commander is secrets manager and then 157
the command there is create secret, and then we need to provide a name for this 158
secret, a description for it, and then the actual secret itself. So we can see here 159
we've got a secret - string and then we define what that secret is and it's a 160
file so the details of that secret will be in a file called my creds JSON. If 161
that was successful the command line interface will then output the ARN or 162
Amazon number for that secret and the name 163
that you provided for that secret and the version idea of that secret as wel.l 164
The IAM permissions that are required will be secrets manager create secret 165
and if you're using a customer manage cmk then you'll also need to have KMS 166
generate data key permissions as well. Now if we want to retrieve that secret 167
value we'll need to have the ARN or the name that we provided for that secret 168
and we of course will need to have IAM permission so we'll need to have secrets 169
manager get secret value, and again if we're using a customer manage cmk with 170
the KMS then we'll need to have KMS decrypted permissions as well. So the 171
command there will be AWS secrets manager and then get secret value and 172
then secret ID which will be either the ARN or the name that we provided for 173
that secret. 174
The AWS encryption software development kit and its associated command-line 175
interface is a client-side encryption library. So what that means is that you 176
will install the encryption command-line interface on a computer. Could be an ec2 177
server. Could be a remote computer whatever it is, or you could install an 178
application that is running the encryption software development kit as well. 179
Now this is separate from the AWS command-line interface and the AWS 180
software development kits. It stands on its own as a separate application. It 181
supports only symmetric CMKs but it uses envelope encryption to make sure 182
that that encryption key is secured. Okay let's have a look at an example using 183
the command-line interface. So the command there is AWS - encryption 184
- CLI and the parameters that we need to provide first off is encrypted, saying 185
that we're going to be encrypting this data. We then need to define the input 186
data and there it is a file hello dot txt. We then need to define the ARN or 187
the amazon resource number for our master key. We then define a folder where 188
we can store our metadata output. Now the next one here is encryption context and 189
that is optional but it is recommended by AWS what that is is a name value pair 190
or you could put multiple name value pairs in here you just need to separate 191
them with a space and they further define this encryption job. So when we go 192
to decrypt we can also provide this encryption context in our decrypt 193
command and that will make sure that we're getting the right file to decrypt. 194
So here we've got a name value pair of Department and IT and finally we just 195
need to define where this is going to be output. So we just got there output and 196
a dot which will output that to the the actual folder where this command is 197
being issued from, and so that will create 198
and encrypted file and it will put that into the folder where that command was 199
actually issued from, and that file will be called hello text dot encrypted. Okay 200
when we want to decrypt that file the command there again is a AWS - 201
encryption - CLI and the parameters there, 202
first off decrypt to decrypt it, we provide what the input is and there will 203
be hello dot text dot encrypted and that is the file that is encrypted that we 204
need to decrypt. We will supply the encryption context, if we supplied an 205
encryption context previously when we encrypted it. If we don't or if we didn't 206
then we don't need to worry about that. We need again to provide an output 207
folder for the metadata and then finally where we're going to output that to. 208
When we run that command it's going to create a decrypted file and that will be called 209
hello dot text dot encrypted. So the same name as before but we've a dot decrypted 210
on the end of that file name. 211
There are three main options for managing your encryption keys. The first 212
option is where you control everything. You control the method of encryption for 213
example AES whatever and you control the key management infrastructure or the KMI 214
and that will consist of the key storage where that key is going to be stored and 215
the key management, so who is going to be authorized to access it access that key 216
and you are the one that controls all of that. 217
AWS will not be involved in the management of that key. A good example of 218
that is using client-side encryption where you would encrypt your files 219
before you upload those to AWS. So you have your computer here and you run an 220
encryption client software it encrypts that data and then you can upload that 221
data to s3 and that will be uploaded as encrypted data. 222
AWS doesn't look after any of that encryption whatsoever. All it does is it 223
stores that encrypted file for you in S3. The next option is where you can control 224
the method of encryption and the key management who is authorized to access 225
that key and AWS will look after securely storing that key for you. A good 226
example of that is to use the CloudHSM service to securely store your keys 227
inside of a VPC inside of your account and then your applications that are 228
running on ec2 can access the CloudHSM and then access that key. 229
The last option there is where AWS controls everything. So they control the 230
method of encryption and the entire key management infrastructure, such as the 231
storage of those keys and the authorization of who can access those 232
keys. A good example that is to encrypt an s3 bucket using server-side 233
encryption and using a KMS managed key. That way 234
AWS will be looking after all of the encryption and they will also be looking 235
after who can access that key and also looking after the storage and management 236
of that key as well using the KMS service. We can use encryption with a lot 237
of the AWS storage services. First up there we can use the AWS backup service 238
to manage all of our encrypted storage. We can also use encryption with the s3 239
Glacier, EBS volumes, EFS volumes, CloudWatch logs, Snowballs, Storage Gateway, 240
Cloudtrail and also with CodeCommit, code deploy and code pipeline but with CodeCommit 241
we can only use it with KMS managed keys only. We can also use the AWS KMS 242
service to encrypt our databases as well including RDS Aurora, documentDB are 243
those MongoDB databases, Neptune the graph databases, ElastiCache, the 244
database migration service jobs and also DynamoDB. We can use it with server-side 245
encryption for encryption at rest of that data but we also have another 246
option with DynamoDB and that is to use a DynamoDB encryption client and that 247
allows us to encrypt that data both in transit and at rest that is residing in 248
DynamoDB. 249
Other AWS services that can be encrypted using the KMS. Kinesis streams and also 250
Kinesis firehose, SNS, SQS, SES, elastic MapReduce and sagemaker jobs as well. 251
Now this is not an exhaustive list of all of the services that are available 252
to be used with the KMS but if you want to get the latest and complete list of 253
all of the services just go to the Amazon website or the AWS website slash 254
kms slash features. So that brings us to the end of the encryption lecture and 255
I'm sure you fully understand it by now and I look forward to seeing you in the 256
next one.