WEBVTT 0:00:02.840000 --> 0:00:08.460000 Now I want to take a look at a feature of actually SQL Server, not even 0:00:08.460000 --> 0:00:12.900000 just Azure SQL Server, but a feature of SQL Server that I think is really 0:00:12.900000 --> 0:00:17.900000 frankly critical to securing any kind of sensitive information. 0:00:17.900000 --> 0:00:20.300000 That's what we call Always Encrypted. 0:00:20.300000 --> 0:00:24.480000 With Always Encrypted, what we're going to talk about is, well, we'll 0:00:24.480000 --> 0:00:26.540000 talk about what Always Encrypted is. 0:00:26.540000 --> 0:00:32.020000 Then I want to show you a demonstration of how you can use Always Encrypted. 0:00:32.020000 --> 0:00:35.960000 I want to go to the whiteboard because I want to talk about what I've 0:00:35.960000 --> 0:00:41.580000 always thought is a bit of a whole with built-in encryption capabilities. 0:00:41.580000 --> 0:00:45.960000 I'm going to say that I'm glad that Marchov has finally caught up with 0:00:45.960000 --> 0:00:48.600000 me, which is terrible. 0:00:48.600000 --> 0:00:51.940000 Just in terms of this is actually a practice I used long before there 0:00:51.940000 --> 0:00:55.220000 was always encrypted because it's just show you. 0:00:55.220000 --> 0:00:56.420000 It's something I believe in. 0:00:56.420000 --> 0:00:58.680000 Let's take a scenario. 0:00:58.680000 --> 0:01:06.840000 In this scenario, I've got an Azure SQL Server, really any SQL Server. 0:01:06.840000 --> 0:01:12.740000 Behind that SQL Server, I've got a database. 0:01:12.740000 --> 0:01:17.620000 We'll call it Azure SQL Server, but this is generic. 0:01:17.620000 --> 0:01:19.940000 I say we'll call it Azure SQL Server. 0:01:19.940000 --> 0:01:24.340000 I've got an Azure SQL Server that's an Azure class that makes sense. 0:01:24.340000 --> 0:01:26.800000 Then I've got an Azure, well, we'll stay with that. 0:01:26.800000 --> 0:01:30.120000 I've got an Azure SQL database behind that. 0:01:30.120000 --> 0:01:40.360000 Between my Azure SQL Server and my Azure SQL database, I've got some protection. 0:01:40.360000 --> 0:01:46.120000 For example, the Azure SQL database has transparent data encryption, TDE. 0:01:46.120000 --> 0:01:48.260000 It's encrypted. Great. 0:01:48.260000 --> 0:01:57.360000 The Azure SQL Server is going to require an encrypted connection. 0:01:57.360000 --> 0:02:03.160000 So I am protected from somebody taking my physical media or taking back 0:02:03.160000 --> 0:02:06.460000 up some of my media and being able to use it. 0:02:06.460000 --> 0:02:11.280000 I'm protected from somebody sniffing the communication between my client 0:02:11.280000 --> 0:02:14.380000 and my database. 0:02:14.380000 --> 0:02:25.980000 But what I'm not protected from is the bad actor. 0:02:25.980000 --> 0:02:31.540000 You always tell they're not smiling when they're capped to the back. 0:02:31.540000 --> 0:02:38.500000 This person has used, let's say, social engineering and they have stolen 0:02:38.500000 --> 0:02:45.020000 the credentials of our system administrator. 0:02:45.020000 --> 0:02:53.080000 We're very upset about that. 0:02:53.080000 --> 0:02:54.340000 We get an exclamation mark. 0:02:54.340000 --> 0:02:58.440000 But the problem is now, I still have this Achilles heel. 0:02:58.440000 --> 0:03:03.100000 Because if they get the credentials, everything else is in one place. 0:03:03.100000 --> 0:03:05.440000 They're going to be able to log in, they're going to be able to go through 0:03:05.440000 --> 0:03:08.600000 an encrypted connection, they're going to be able to access that data. 0:03:08.600000 --> 0:03:15.000000 At this point, they've got that one step clear access, they can return 0:03:15.000000 --> 0:03:16.960000 any data they want. 0:03:16.960000 --> 0:03:22.580000 When you've got sensitive data, one of the things that I've always done 0:03:22.580000 --> 0:03:28.020000 when I've developed applications, I've said, okay, here's some data, some 0:03:28.020000 --> 0:03:31.980000 sensitive data, maybe even some PII, right? 0:03:31.980000 --> 0:03:34.860000 Personally identifiable information. 0:03:34.860000 --> 0:03:44.480000 What I want to do is actually want to have somewhere a separate key, completely 0:03:44.480000 --> 0:03:50.460000 external to the storage and connectivity. 0:03:50.460000 --> 0:03:55.280000 I want to use that key to encrypt that PII. 0:03:55.280000 --> 0:04:00.640000 To me, this is analogous to having multi-factor authentication. 0:04:00.640000 --> 0:04:03.860000 So if you were to steal somebody's credentials, that might get you all 0:04:03.860000 --> 0:04:05.440000 the way to the database. 0:04:05.440000 --> 0:04:08.920000 But I'm going to take this key, I'm going to put this key somewhere else, 0:04:08.920000 --> 0:04:12.900000 I'm going to encrypt the data with that key and guess what? 0:04:12.900000 --> 0:04:18.720000 Unless you have access to the key, you're not going to get with any realistic 0:04:18.720000 --> 0:04:21.980000 level of effort, you're not going to get that personally identifiable 0:04:21.980000 --> 0:04:26.780000 information. I say reasonable level of effort because there's always the 0:04:26.780000 --> 0:04:30.700000 possibility that somebody could brute force it or whatever, but highly, 0:04:30.700000 --> 0:04:32.460000 highly unlikely. 0:04:32.460000 --> 0:04:41.880000 And so this separation of the key from that traditional or normal flow 0:04:41.880000 --> 0:04:43.940000 is really critical, right? 0:04:43.940000 --> 0:04:50.420000 And by the way, if I've got a good actor, if I have somebody that is not 0:04:50.420000 --> 0:04:57.500000 square, there we go, I've got somebody here, they're happy, right? 0:04:57.500000 --> 0:05:04.300000 I want that person to have access to the key, right? 0:05:04.300000 --> 0:05:06.060000 And I want that to be transparent. 0:05:06.060000 --> 0:05:10.460000 So when that person accesses the PII, they're good to go because they 0:05:10.460000 --> 0:05:11.980000 have the key, right? 0:05:11.980000 --> 0:05:14.300000 But they have it separate from that process. 0:05:14.300000 --> 0:05:20.440000 And that's really what Azure or SQL Server always encrypted does. 0:05:20.440000 --> 0:05:23.880000 And I'm going to just kind of draw out the components of that. 0:05:23.880000 --> 0:05:28.640000 And you may also hear it referred to as column encryption because that 0:05:28.640000 --> 0:05:33.880000 is the level that always encrypted is applied to, right? 0:05:33.880000 --> 0:05:37.620000 And the way this works, there's a few different components when we're 0:05:37.620000 --> 0:05:40.780000 talking about SQL Server always encrypted. 0:05:40.780000 --> 0:05:43.700000 And let's just go over those really quickly. 0:05:43.700000 --> 0:05:49.260000 I've got my database and my database is in a server, so we can draw those 0:05:49.260000 --> 0:05:53.740000 out again. We'll shorthand a little bit. 0:05:53.740000 --> 0:05:58.920000 This is my server, this is my database. 0:05:58.920000 --> 0:06:02.140000 We can get real nice and say DB. 0:06:02.140000 --> 0:06:08.360000 Now within my database, let's say I've got a table, and that table has 0:06:08.360000 --> 0:06:18.880000 some columns. And let's say one of those columns has sensitive data, okay? 0:06:18.880000 --> 0:06:22.460000 Now what I need to do is I need to encrypt that. 0:06:22.460000 --> 0:06:26.140000 And there's two different keys that are used. 0:06:26.140000 --> 0:06:31.460000 At the immediate level, there is what's called a column encryption key. 0:06:31.460000 --> 0:06:34.560000 And that's stored in the metadata on the server. 0:06:34.560000 --> 0:06:43.800000 I've got my, let's see if I can get this right, there we go, column, let's 0:06:43.800000 --> 0:06:45.720000 say column key, it's my column encryption key. 0:06:45.720000 --> 0:06:51.540000 Now that is stored within your database system, but it is not stored in 0:06:51.540000 --> 0:06:59.020000 plain text. That column key is encrypted by a master key. 0:06:59.020000 --> 0:07:07.560000 And that master key is by definition stored externally to SQL Server. 0:07:07.560000 --> 0:07:10.260000 There's two different ways the master key can be stored. 0:07:10.260000 --> 0:07:18.880000 It can be stored in a certificate store, such as the Windows certificate 0:07:18.880000 --> 0:07:27.220000 store. Or it can also be stored separately in a key vault. 0:07:27.220000 --> 0:07:31.060000 But either way, again, it's simply separate. 0:07:31.060000 --> 0:07:35.120000 Now within the column key, there's also, when that column key is created 0:07:35.120000 --> 0:07:37.940000 and stored, it's used to encrypt the data. 0:07:37.940000 --> 0:07:42.860000 And there's metadata that will point a client over to the appropriate 0:07:42.860000 --> 0:07:44.440000 location, right? 0:07:44.440000 --> 0:07:47.420000 So if I've got that in a key vault, it doesn't give you access to the 0:07:47.420000 --> 0:07:50.900000 key vault, but that metadata that's in the column key will point you over 0:07:50.900000 --> 0:07:55.700000 there, right? So again, even if you were able now to get directly through 0:07:55.700000 --> 0:08:00.980000 the server to that data, unless you have access to that key as well, you're 0:08:00.980000 --> 0:08:01.880000 not going to get the data. 0:08:01.880000 --> 0:08:03.740000 Well, you'll get an encrypted version of it. 0:08:03.740000 --> 0:08:07.200000 And what I'd like to do now is show you how to implement it. 0:08:07.200000 --> 0:08:10.420000 Again, there's some complex concepts going on here, but the implementation 0:08:10.420000 --> 0:08:13.000000 is actually relatively simple. 0:08:13.000000 --> 0:08:16.000000 I've got a table, and what I'm going to do is I'm going to encrypt the 0:08:16.000000 --> 0:08:17.920000 column on that table. 0:08:17.920000 --> 0:08:21.740000 And then I'm going to show you how you would access that data. 0:08:21.740000 --> 0:08:27.240000 Now to access the data, I'm actually going to use a .NET framework client, 0:08:27.240000 --> 0:08:33.100000 all right? And I typically use .NET core functionality, but it's actually 0:08:33.100000 --> 0:08:37.220000 not fully implemented right now within .NET core. 0:08:37.220000 --> 0:08:41.440000 Okay, so the always encrypted or the column level encryption, if you're 0:08:41.440000 --> 0:08:44.620000 going to use that from a .NET environment, then you're going to have to 0:08:44.620000 --> 0:08:46.760000 use the .NET framework. 0:08:46.760000 --> 0:08:48.980000 And again, that's at the time of this recording by the time you watch 0:08:48.980000 --> 0:08:50.460000 this, maybe that'll change. 0:08:50.460000 --> 0:08:52.280000 I certainly hope that it will have. 0:08:52.280000 --> 0:08:55.440000 It's something if you go and you kind of do a search, you'll find there's 0:08:55.440000 --> 0:09:01.000000 a lot of demand for this capability in .NET core. 0:09:01.000000 --> 0:09:04.200000 All right, now if you're using another tool, really, it's just a matter 0:09:04.200000 --> 0:09:07.880000 of looking at the drivers for whatever your framework is and making sure 0:09:07.880000 --> 0:09:09.920000 the drivers support that. 0:09:09.920000 --> 0:09:13.380000 But in any case, let's take a look at how we would implement this. 0:09:13.380000 --> 0:09:17.880000 I have an Azure SQL database. 0:09:17.880000 --> 0:09:23.020000 And I've got a database named Always Encrypted. 0:09:23.020000 --> 0:09:24.060000 It doesn't have to be. 0:09:24.060000 --> 0:09:25.880000 That's just high-named it. 0:09:25.880000 --> 0:09:28.360000 I've got a table in that database named Customers. 0:09:28.360000 --> 0:09:29.420000 It has three columns. 0:09:29.420000 --> 0:09:33.600000 It's got an ID column, a name column, and a PII column. 0:09:33.600000 --> 0:09:37.540000 I have already inserted a record here, and you can see that's just plain 0:09:37.540000 --> 0:09:43.840000 text from here. And if I retrieve that data, okay, I can see that data. 0:09:43.840000 --> 0:09:48.740000 Now what I want to do, though, is I want to go ahead and I want to encrypt 0:09:48.740000 --> 0:09:54.440000 the PII column. Now, to do that, the easiest way to do this is to go through 0:09:54.440000 --> 0:09:57.020000 a wizard. The tool that I'm in, by the way, if you're not a SQL Server 0:09:57.020000 --> 0:09:59.160000 person, this is SQL Server Management Studio. 0:09:59.160000 --> 0:10:03.900000 It's kind of your standard management tool for databases. 0:10:03.900000 --> 0:10:09.780000 And right down here, I right-click the column, and I go to Encrypt Columns. 0:10:09.780000 --> 0:10:15.120000 Now, by the time you watch this, this wizard probably will change somewhat, 0:10:15.120000 --> 0:10:18.220000 but the basics of it will not. 0:10:18.220000 --> 0:10:21.520000 All right, I'm going to go here, Column Selection. 0:10:21.520000 --> 0:10:23.160000 So what column do I want to encrypt? 0:10:23.160000 --> 0:10:27.060000 I want to encrypt the PII column. 0:10:27.060000 --> 0:10:29.380000 And I have to choose the encryption type. 0:10:29.380000 --> 0:10:31.060000 Now, there's two types of encryption. 0:10:31.060000 --> 0:10:33.980000 There's deterministic and randomized. 0:10:33.980000 --> 0:10:37.520000 Okay, with deterministic, the same encryption is going to be implied for 0:10:37.520000 --> 0:10:42.960000 every row. Now, what that means is it is possible, if somebody were to 0:10:42.960000 --> 0:10:45.960000 get the encrypted data, that if they see repeating patterns, they may 0:10:45.960000 --> 0:10:47.720000 be able to guess it. 0:10:47.720000 --> 0:10:51.360000 Okay, so from a security standpoint, the randomize is going to be better. 0:10:51.360000 --> 0:10:57.040000 However, for certain things, for example, sorting or joining onto a column, 0:10:57.040000 --> 0:11:00.520000 if it's encrypted, then it needs to be deterministic. 0:11:00.520000 --> 0:11:04.400000 Generally speaking, I'm going to architect so that I have data that does 0:11:04.400000 --> 0:11:06.940000 not need to be deterministic. 0:11:06.940000 --> 0:11:09.120000 So I'm going to make this randomized. 0:11:09.120000 --> 0:11:12.720000 So the PII is going to be randomized. 0:11:12.720000 --> 0:11:16.660000 Next, now there's two different ways that I can store it. 0:11:16.660000 --> 0:11:19.940000 I can store it in the Windows Certific Store, which by the way, once it's 0:11:19.940000 --> 0:11:24.620000 there, you could also export it, or I can store it to Key Vault. 0:11:24.620000 --> 0:11:28.380000 And if I store it to Key Vault, it's going to have me sign in to Azure 0:11:28.380000 --> 0:11:29.540000 and then select the Key Vault. 0:11:29.540000 --> 0:11:33.680000 But I want to actually just put this in the Windows Certificates Store. 0:11:33.680000 --> 0:11:41.000000 And next, and I can either generate a PowerShell script so I could automate 0:11:41.000000 --> 0:11:42.040000 this process later. 0:11:42.040000 --> 0:11:46.100000 Or now we're going to go ahead and just proceed to finish. 0:11:46.100000 --> 0:11:49.320000 And it's going to give me a summary of what I'm doing and I'm fine with 0:11:49.320000 --> 0:11:54.280000 that. And now we wait. 0:11:54.280000 --> 0:11:59.220000 All right, the wizard has completed. 0:11:59.220000 --> 0:12:02.320000 And my column is now encrypted. 0:12:02.320000 --> 0:12:03.940000 Let's take a look. 0:12:03.940000 --> 0:12:12.660000 So I'm going to try and rerun this select statement again from a mm from 0:12:12.660000 --> 0:12:14.120000 SQL Server Management Studio. 0:12:14.120000 --> 0:12:15.960000 Notice what this sees now. 0:12:15.960000 --> 0:12:17.640000 This just sees encrypted data. 0:12:17.640000 --> 0:12:20.400000 It sees kind of binary data. 0:12:20.400000 --> 0:12:24.640000 And so I don't just have, even though I'm in right now as a system administrator 0:12:24.640000 --> 0:12:29.400000 for the server and an owner for the database, it doesn't matter. 0:12:29.400000 --> 0:12:33.020000 Unless I have access to that key, not going to do any good. 0:12:33.020000 --> 0:12:36.680000 All right, now I have an application. 0:12:36.680000 --> 0:12:41.580000 This is a .NET framework console application. 0:12:41.580000 --> 0:12:45.220000 Very simple. And it connects to the database. 0:12:45.220000 --> 0:12:50.560000 And key to that connection is this setting right here. 0:12:50.560000 --> 0:12:53.880000 Column Encryption setting equals enable. 0:12:53.880000 --> 0:12:59.580000 So that's going to let this encryption or this connection work with encrypted 0:12:59.580000 --> 0:13:01.020000 columns. Simple as that. 0:13:01.020000 --> 0:13:06.820000 Now what's cool is I don't have to tell it anything about where the certificate 0:13:06.820000 --> 0:13:11.340000 is. Yeah, because that's all going to be embedded in the metadata of the 0:13:11.340000 --> 0:13:13.200000 encrypted column itself. 0:13:13.200000 --> 0:13:14.840000 But I say, okay, that's fine. 0:13:14.840000 --> 0:13:15.980000 We want to use that. 0:13:15.980000 --> 0:13:17.580000 And I've got a few statements here. 0:13:17.580000 --> 0:13:20.580000 I've got a couple of inserts that occur. 0:13:20.580000 --> 0:13:22.540000 And again, we don't really need the details of that. 0:13:22.540000 --> 0:13:26.420000 And then I select the results out from that table. 0:13:26.420000 --> 0:13:30.280000 So if all goes well, it'll have that one record plus two more and I'll 0:13:30.280000 --> 0:13:31.880000 be able to see them all. 0:13:31.880000 --> 0:13:33.480000 So I'm going to go ahead and start that up. 0:13:33.480000 --> 0:13:36.340000 I think it's going to open on the other screen. 0:13:36.340000 --> 0:13:39.840000 Come over here and there we go. 0:13:39.840000 --> 0:13:44.480000 Not perfectly formatted, but you can see there are three records here. 0:13:44.480000 --> 0:13:50.480000 And because I am using the connection to the encrypted component of the 0:13:50.480000 --> 0:13:53.060000 connection, I am actually just viewing that text. 0:13:53.060000 --> 0:13:58.340000 So again, it becomes transparent with the exception that I added one component 0:13:58.340000 --> 0:13:59.980000 to the connection string. 0:13:59.980000 --> 0:14:01.720000 Now that component is important. 0:14:01.720000 --> 0:14:05.800000 And in fact, as I said, when I was trying to do this recently, Test and 0:14:05.800000 --> 0:14:10.960000 C, but still worked on ASP net core, just dot net core in general, it 0:14:10.960000 --> 0:14:14.160000 didn't. That wasn't yet recognized. 0:14:14.160000 --> 0:14:20.600000 Now, again, if I go back, show you again, nothing up my sleeve here. 0:14:20.600000 --> 0:14:25.900000 I execute that. So now I'm getting all the data, but that data is encrypted. 0:14:25.900000 --> 0:14:29.800000 This, of course, would should hopefully also beg the question. 0:14:29.800000 --> 0:14:30.600000 Okay, that's great. 0:14:30.600000 --> 0:14:32.520000 But what about that certificate? 0:14:32.520000 --> 0:14:38.340000 Well, when I run the wizard and I select that I want the Windows store, 0:14:38.340000 --> 0:14:42.300000 what it's going to do is it's going to generate a certificate. 0:14:42.300000 --> 0:14:46.060000 And it's going to put that certificate in my personal store. 0:14:46.060000 --> 0:14:51.120000 Okay. So I opened up a MMC console. 0:14:51.120000 --> 0:14:54.300000 I opened up for certificate management. 0:14:54.300000 --> 0:14:56.140000 I added the snap in for that. 0:14:56.140000 --> 0:14:57.740000 If you're not familiar with that, you can look this up. 0:14:57.740000 --> 0:14:59.820000 This is all well documented. 0:14:59.820000 --> 0:15:04.060000 And then I go to the certificates for the current user, and I go to my 0:15:04.060000 --> 0:15:05.320000 personal certificates. 0:15:05.320000 --> 0:15:10.200000 And here is an always encrypted auto certificate, and always encrypted 0:15:10.200000 --> 0:15:11.740000 auto certificate one. 0:15:11.740000 --> 0:15:16.400000 Now, what this means is that right now, the only place that I can connect 0:15:16.400000 --> 0:15:21.140000 to this database, and I can encrypt and decrypt the data for that column 0:15:21.140000 --> 0:15:23.740000 is on this machine. 0:15:23.740000 --> 0:15:27.900000 What I would need to do in this case is I would actually need to go and 0:15:27.900000 --> 0:15:31.820000 export. Where are we there? 0:15:31.820000 --> 0:15:33.400000 Somewhere around here. 0:15:33.400000 --> 0:15:35.500000 Oh, open it. That's right. 0:15:35.500000 --> 0:15:41.640000 Open it and then export, copy to file, this certificate. 0:15:41.640000 --> 0:15:50.360000 And I would go and export the private key, and create a personal information 0:15:50.360000 --> 0:16:09.700000 exchange file, and go ahead and password protect it. 0:16:09.700000 --> 0:16:32.120000 And pop it out to a file. 0:16:32.120000 --> 0:16:37.920000 And then I would have to take that file, and I would have to, of course, 0:16:37.920000 --> 0:16:41.980000 distribute it. Now, just to show you that this is required, if I go and 0:16:41.980000 --> 0:16:47.400000 delete this certificate from my local certificate store, and if I try 0:16:47.400000 --> 0:16:55.660000 to run this again, it comes up pretty quickly. 0:16:55.660000 --> 0:16:58.920000 In fact, so quickly that I couldn't move it over here in time, but you'll 0:16:58.920000 --> 0:17:01.720000 see lots and lots of errors. 0:17:01.720000 --> 0:17:04.580000 And if you look right up towards the top, it says certificate with thumbprint, 0:17:04.580000 --> 0:17:06.800000 and a whole bunch of letters not found. 0:17:06.800000 --> 0:17:11.700000 Right? And so what that means is that, yes, I have the encrypted column. 0:17:11.700000 --> 0:17:15.200000 I was trying to connect to it, but when it tried to get that metadata 0:17:15.200000 --> 0:17:18.860000 from the column, it looked for this certificate, and it wasn't there. 0:17:18.860000 --> 0:17:23.840000 So now I've got data that's protected, and at the moment, really, actually 0:17:23.840000 --> 0:17:27.520000 that data is not accessible from anywhere, because I don't have that certificate 0:17:27.520000 --> 0:17:29.780000 installed anywhere. 0:17:29.780000 --> 0:17:36.640000 And that's what always encrypted does, and it's an incredibly important 0:17:36.640000 --> 0:17:39.900000 concept, if you have sensitive information, because again, what it does 0:17:39.900000 --> 0:17:45.500000 is it separates out the encryption from the core access. 0:17:45.500000 --> 0:17:50.300000 Right? So even if you access the data as a system administrator, not going 0:17:50.300000 --> 0:17:53.900000 to do you any good, unless you also separately have that key.