1 00:00:00,640 --> 00:00:01,940 Welcome back. 2 00:00:01,940 --> 00:00:03,840 In this lesson, we are going to continue 3 00:00:03,840 --> 00:00:06,380 our discussion of some Azure services 4 00:00:06,380 --> 00:00:10,490 and we are going to introduce Azure Synapse Analytics. 5 00:00:10,490 --> 00:00:11,900 So in this lesson, 6 00:00:11,900 --> 00:00:16,100 we're going to talk about Azure Synapse Analytics. 7 00:00:16,100 --> 00:00:17,120 So what is it? 8 00:00:17,120 --> 00:00:18,960 Let's give you a brief introduction. 9 00:00:18,960 --> 00:00:20,450 We're going to talk about the architecture 10 00:00:20,450 --> 00:00:22,570 of Azure Synapse Analytics, 11 00:00:22,570 --> 00:00:24,530 and then we're going to take a little bit of time 12 00:00:24,530 --> 00:00:26,000 to actually jump into the portal 13 00:00:26,000 --> 00:00:28,920 so you can see what it looks like. 14 00:00:28,920 --> 00:00:31,650 So let's start off with our introduction. 15 00:00:31,650 --> 00:00:33,660 Azure Synapse Analytics. 16 00:00:33,660 --> 00:00:35,930 What is it exactly? 17 00:00:35,930 --> 00:00:39,560 Well, to start off with it's SQL. 18 00:00:39,560 --> 00:00:41,130 And so when we talk about SQL 19 00:00:41,130 --> 00:00:43,610 if you remember from a previous lesson, 20 00:00:43,610 --> 00:00:47,670 we talked about structured data, relational databases, 21 00:00:47,670 --> 00:00:50,070 (that chart) and we talked about SQL and NoSQL. 22 00:00:50,070 --> 00:00:51,060 This is SQL. 23 00:00:51,060 --> 00:00:53,853 So you need to keep that in mind. It's very important. 24 00:00:54,860 --> 00:00:57,050 It's more than SQL, though. 25 00:00:57,050 --> 00:01:00,800 Azure Synapse Analytics actually includes data integration, 26 00:01:00,800 --> 00:01:05,060 enterprise data warehousing, and big data analytics. 27 00:01:05,060 --> 00:01:08,210 So Microsoft likes to say Azure Synapse Analytics 28 00:01:08,210 --> 00:01:10,840 brings worlds together with a unified experience 29 00:01:10,840 --> 00:01:14,410 to ingest explore, prepare, manage, and serve data 30 00:01:14,410 --> 00:01:18,173 for immediate BI and machine learning needs. 31 00:01:19,250 --> 00:01:21,630 Basically what Microsoft wanted to do 32 00:01:21,630 --> 00:01:26,220 was take SQL Data Warehouse along with some other services 33 00:01:26,220 --> 00:01:28,750 like Data Lakes, Data Factory, 34 00:01:28,750 --> 00:01:31,680 and actually build a unified service 35 00:01:31,680 --> 00:01:33,940 that's going to allow you to stay in that service 36 00:01:33,940 --> 00:01:38,070 and really do an entire data engineering experience 37 00:01:38,070 --> 00:01:39,010 from start to finish. 38 00:01:39,010 --> 00:01:42,580 That was the goal for Azure Synapse Analytics. 39 00:01:42,580 --> 00:01:44,810 And so when we look at the architecture 40 00:01:44,810 --> 00:01:49,810 for Azure Synapse Analytics, we have 4 main components. 41 00:01:49,840 --> 00:01:52,070 We have our ingest phase, 42 00:01:52,070 --> 00:01:54,730 so this is going to be basically taking Data Factory 43 00:01:54,730 --> 00:01:58,100 and putting it into Synapse Analytics. 44 00:01:58,100 --> 00:01:59,810 We have our store phase, 45 00:01:59,810 --> 00:02:02,020 which is going to allow you to use Data Lakes, 46 00:02:02,020 --> 00:02:04,893 Blob storage, and SQL databases. 47 00:02:06,460 --> 00:02:07,670 Prep and train; 48 00:02:07,670 --> 00:02:09,800 so this is kind of combining Databricks 49 00:02:09,800 --> 00:02:11,893 and Azure Machine Learning together. 50 00:02:13,240 --> 00:02:15,130 And then our model-and-serve phase, 51 00:02:15,130 --> 00:02:16,540 which is going to be modeling data 52 00:02:16,540 --> 00:02:19,503 in a SQL Data Warehouse-like experience. 53 00:02:21,850 --> 00:02:23,320 So seeing that in action, 54 00:02:23,320 --> 00:02:25,140 this is what it actually looks like. 55 00:02:25,140 --> 00:02:27,380 You can see that we have our ingestion source 56 00:02:27,380 --> 00:02:29,930 and we're going to use something like Data Factory 57 00:02:29,930 --> 00:02:31,230 to pull that data in. 58 00:02:31,230 --> 00:02:33,000 And I'll show you this in just a second. 59 00:02:33,000 --> 00:02:35,430 And then we are going to store that data 60 00:02:35,430 --> 00:02:37,460 using our big data storage services 61 00:02:37,460 --> 00:02:41,600 like our Data Lakes, prep and train using Databricks, 62 00:02:41,600 --> 00:02:44,090 and then we're going to use our model and serve, 63 00:02:44,090 --> 00:02:45,730 which is your SQL Data Warehouse, 64 00:02:45,730 --> 00:02:48,570 again, all of this is Azure Synapse Analytics. 65 00:02:48,570 --> 00:02:50,590 And then all of that gives you outputs 66 00:02:50,590 --> 00:02:52,960 into either kick-off other apps 67 00:02:52,960 --> 00:02:55,060 or it allows you to give insights 68 00:02:55,060 --> 00:02:56,963 into something like Power BI. 69 00:03:00,970 --> 00:03:02,170 And here is a chart 70 00:03:02,170 --> 00:03:04,910 kind of showing this in a different image. 71 00:03:04,910 --> 00:03:07,250 So you can see all of the different services 72 00:03:07,250 --> 00:03:10,200 and how we kind of have a cyclical approach, 73 00:03:10,200 --> 00:03:13,250 so that we can stay within Azure Synapse Analytics 74 00:03:13,250 --> 00:03:16,510 and effectively manage our data warehouse experience 75 00:03:16,510 --> 00:03:18,810 or our data engineering experience, 76 00:03:18,810 --> 00:03:20,970 assuming we're using things that would belong 77 00:03:20,970 --> 00:03:23,173 only in Azure Synapse Analytics. 78 00:03:25,150 --> 00:03:27,700 Now don't get too hung up on this, 79 00:03:27,700 --> 00:03:30,920 we are actually going to have an entire section, 80 00:03:30,920 --> 00:03:34,360 where we talk a lot about Azure Synapse Analytics, 81 00:03:34,360 --> 00:03:37,240 we're going to talk more about Data Factory. 82 00:03:37,240 --> 00:03:39,520 So again, this is the introduction section, 83 00:03:39,520 --> 00:03:41,140 so don't get too bogged down 84 00:03:41,140 --> 00:03:43,290 into having to know all the details. 85 00:03:43,290 --> 00:03:44,400 What I want you to understand is 86 00:03:44,400 --> 00:03:47,780 the overarching architecture, and what's kind of happening 87 00:03:47,780 --> 00:03:49,453 within each of these services. 88 00:03:51,030 --> 00:03:53,450 So with that, let's jump on over 89 00:03:53,450 --> 00:03:56,383 and let's take a look at Synapse in the Azure portal. 90 00:03:57,740 --> 00:03:59,810 So I've opened up the Azure portal, 91 00:03:59,810 --> 00:04:03,163 and I've clicked on Azure Synapse Analytics Studio. 92 00:04:04,530 --> 00:04:06,500 So rather than go through all of these buttons, 93 00:04:06,500 --> 00:04:08,890 let's actually jump over here to the left, 94 00:04:08,890 --> 00:04:10,960 this is more of a visual representation 95 00:04:10,960 --> 00:04:13,740 of the guide over here on our panel. 96 00:04:13,740 --> 00:04:15,090 So let me expand that. 97 00:04:15,090 --> 00:04:16,880 And over here we have our Home, 98 00:04:16,880 --> 00:04:21,560 Data, Develop, Integrate, Monitor, and Manage sections. 99 00:04:21,560 --> 00:04:23,290 In our Data section, 100 00:04:23,290 --> 00:04:27,040 you can actually see any databases that you have linked. 101 00:04:27,040 --> 00:04:28,480 This is going to be a very nice way 102 00:04:28,480 --> 00:04:31,970 to go through and actually see your data 103 00:04:31,970 --> 00:04:35,310 with a directory approach or hierarchical namespace, 104 00:04:35,310 --> 00:04:37,390 if you remember our Data Lake. 105 00:04:37,390 --> 00:04:41,180 So we have our Storage Gen2 here. 106 00:04:41,180 --> 00:04:42,820 This is just the standard database 107 00:04:42,820 --> 00:04:44,270 and then it would open up and show you 108 00:04:44,270 --> 00:04:46,370 all of the data inside here if we had it, 109 00:04:46,370 --> 00:04:47,887 and we could, kind of, drill down 110 00:04:47,887 --> 00:04:50,720 into where our databases and files live 111 00:04:50,720 --> 00:04:52,633 and then actually see those as well. 112 00:04:54,110 --> 00:04:57,130 Develop. Develop is a place that we can go in 113 00:04:57,130 --> 00:05:01,373 and we can create SQL scripts or notebooks or data flows. 114 00:05:03,390 --> 00:05:06,220 And then when we're not working with our data directly, 115 00:05:06,220 --> 00:05:08,130 we can come into our integrate 116 00:05:08,130 --> 00:05:10,940 and we can actually use pipelines. 117 00:05:10,940 --> 00:05:12,330 So we've been talking about this 118 00:05:12,330 --> 00:05:13,690 in Data Factory for a while, 119 00:05:13,690 --> 00:05:16,480 this is almost an exact replica of Data Factory, 120 00:05:16,480 --> 00:05:18,760 but it's in Synapse Analytics. 121 00:05:18,760 --> 00:05:21,230 So we can click on this and we can actually 122 00:05:21,230 --> 00:05:24,580 start to build some different pipelines. 123 00:05:24,580 --> 00:05:26,210 So let's just say, for example, 124 00:05:26,210 --> 00:05:28,560 that I have 2 different data sources, 125 00:05:28,560 --> 00:05:30,680 and again, I'm going to show you how to do all this later, 126 00:05:30,680 --> 00:05:31,513 I just want to give you 127 00:05:31,513 --> 00:05:34,420 kind of a high look at how this works. 128 00:05:34,420 --> 00:05:36,710 So let's say I had 2 different ingestion sources 129 00:05:36,710 --> 00:05:39,350 and I'm going to ingest data into the Cloud. 130 00:05:39,350 --> 00:05:42,440 I could take those and move those 2 sources 131 00:05:42,440 --> 00:05:44,520 into a Data Lake, 132 00:05:44,520 --> 00:05:45,960 and then I could kick off 133 00:05:45,960 --> 00:05:48,410 a notebook as well if I wanted to. 134 00:05:48,410 --> 00:05:52,563 And so we can start to build some different pipelines 135 00:05:55,130 --> 00:06:00,130 that we can use for our Azure Synapse Analytics workloads. 136 00:06:02,020 --> 00:06:03,750 And then finally we can come into Monitor 137 00:06:03,750 --> 00:06:05,650 and we can see all kinds of information 138 00:06:05,650 --> 00:06:09,390 about our databases and about our pipeline runs, 139 00:06:09,390 --> 00:06:11,790 so we can kind of see what's going on over here. 140 00:06:12,770 --> 00:06:14,380 So again, very high-level look, 141 00:06:14,380 --> 00:06:16,350 but I wanted to give you just a kind of a sneak peek 142 00:06:16,350 --> 00:06:19,320 of what you'll be seeing later on in the course. 143 00:06:19,320 --> 00:06:22,240 So in review, what do you need to know? 144 00:06:22,240 --> 00:06:24,360 First, you need to know what Synapse is 145 00:06:24,360 --> 00:06:25,883 and what it's used for. 146 00:06:27,950 --> 00:06:30,110 Synapse architecture. You need to keep in mind 147 00:06:30,110 --> 00:06:33,610 the main components that are in Azure Synapse Analytics, 148 00:06:33,610 --> 00:06:35,350 so that you can kind of get a feel for what is 149 00:06:35,350 --> 00:06:37,600 and is not possible in Synapse. 150 00:06:37,600 --> 00:06:39,110 Keeping in mind, of course, 151 00:06:39,110 --> 00:06:41,780 SQL, structured, relational databases, 152 00:06:41,780 --> 00:06:43,323 keeping that in mind as well. 153 00:06:45,250 --> 00:06:48,020 And then as we move through the course, 154 00:06:48,020 --> 00:06:49,620 it's not as important for right now 155 00:06:49,620 --> 00:06:52,590 that you understand what you saw in the portal, 156 00:06:52,590 --> 00:06:54,100 but as we move through and, especially, 157 00:06:54,100 --> 00:06:56,640 as you start getting more practice and hands-on labs, 158 00:06:56,640 --> 00:06:58,840 you need to get a good feel for, 159 00:06:58,840 --> 00:07:00,200 "Okay, I know where this is, 160 00:07:00,200 --> 00:07:02,140 "I've clicked on these different things, 161 00:07:02,140 --> 00:07:03,890 "I understand how it works," 162 00:07:03,890 --> 00:07:06,510 because that's going to help everything click together 163 00:07:06,510 --> 00:07:08,623 when you're taking the exam itself. 164 00:07:09,630 --> 00:07:12,010 Alright, that's it for our introduction 165 00:07:12,010 --> 00:07:13,860 to Azure Synapse Analytics, 166 00:07:13,860 --> 00:07:15,410 I'll see you in the next video.