Congratulations, you have made it
to the Section Recap.
In this lesson, we are going to be looking back
at the lessons in section 2.
Some things to keep in mind.
First, hey, it's all a review.
If you are having struggles
as we go through one of these concepts,
go back and re-watch the video.
Keep in mind that for this section,
there's not a whole lot of depth, and that's okay.
This is really around focusing on the DP-203,
which for this, means understanding what the services are,
where they live, and some high-level looks at what they do.
If you don't know something, review.
Go back, and check it out again.
All right, let's dive in and start reviewing.
First, we talked about data engineering,
and if you remember,
everything begins with Sally, the shopper,
and the piece of data that she creates.
That data then needs to move somewhere.
And that data movement creates data engineering.
Data engineering is really about storing, moving,
and pulling meaningful insights out of data.
Next, we talked about data transformation,
and for the DP-203,
that's primarily going to be
Synapse Analytics and Databricks.
And remember that data transformation
consists of pulling data from various sources
and transforming it
so that it looks like one uniform data source.
So that could be correcting, that could be removing nulls.
It could be quite a few different things,
but the process of transforming and curating data
is data transformation.
We also talked about structured versus unstructured.
Recall SQL versus NoSQL.
SQL being that relational database
with fixed schema complex queries
or the ability to process complex queries reasonably fast.
And then, we talked about that vertical scaling
versus the horizontal scaling of NoSQL.
This one's actually really important.
So for the DP-203,
make sure you understand the differences
between structured and unstructured
and then the types of services
that would live in either one.
I find it helpful just to kind of create
a little notebook of the services and then just a brief,
hey, this is what this one does.
So when we talk about SQL,
we would be talking about which database type?
If you said Synapse Analytics, you would be correct.
NoSQL would be more like Cosmos DB.
Cosmos DB, at least at this point,
does not appear on the DP-203,
but that's going to be more of an example of NoSQL.
Talked about Data Factory.
Data Factory is all about those pipelines.
So, creating pipelines,
which is a logical grouping of activities
to create our cloud projects.
Those activities are going to be data movement,
data transformation, and data control.
That's also very important for the DP-203.
Understanding not only the pipelines,
but the types of activities that you're going to see.
And then, as we look at datasets,
datasets, remember, are the data structures
that we have within those data stores.
So basically, it's where the data we need lives.
And then we have the linked services,
which is just that connection string
to point to the data that we need to get to.
We also talked about Azure Synapse Analytics.
So it's going to be that combination of our Data Lake,
Data Factory, and then our big data warehouse as well.
So just keep that in mind.
Stream Analytics, remember Stream Analytics
deals with streaming data, of course.
And with an important concept for Streaming Analytics,
remember it all comes down to that input, query, and output.
And when we talk about input,
the 3 pieces you need to remember,
Event Hubs, IoT Hubs, and Blob storage.
And then finally, windowing.
And again, we'll dive into windowing
at a much deeper level later.
Just remember that windowing helps us
to determine how data comes through
and how we're going to group or arrange it together
to get the views that we need to make business decisions.
Finally, we talked about Azure Databricks.
And we talked about data engineering
being the main component that you really care about.
And then we talked about
the transformational aspect of Databricks to clean up,
correct, curate, or process our data
so that it all looks the same
so that when we pull insights out of it,
or we do machine learning or whatever the next step is,
we have a clean set of data
that's all going to look the same.
So, in summary, let me point out just a few things.
First, this is the foundation.
We're going to integrate data engineering concepts
as we move forward.
But understanding these key services
and where they fit within Azure,
and some of the basics about data,
is going to help you
as we introduce data engineering concepts.
Next, focus on the DP-203.
So make sure that you are
reviewing the Microsoft exam requirements,
and you will actually see those right here.
So if you go to Google and just search for DP-203,
you can actually find the Microsoft DP-203 exam.
Scroll down a bit,
and you will see download exam skills outline.
Make sure that you are referencing this.
Now keep in mind, of course,
we've referenced this as well, very heavily.
But these are the topics and the skills
that you are going to be measured on for the exam.
So make sure that you're looking at that
and that you understand the concepts that fall in there.
And what you can also do
is you can also go in and look at Microsoft Docs.
So, for instance, if I type in Azure Data Lake,
there we go. You will see
this is our introduction to Azure Data Lake,
docs.microsoft.com.
If I click on that one, it's going to pull this up.
And I would suggest that you actually go through
and you skim through the concepts.
We've covered all of this material,
but skim through the concepts, go through the overview.
That's going to help give you a feel for how Microsoft
writes. And that's going to help you a lot on the exam.
So make sure that you're doing that as well.
Finally, don't forget the labs.
Each of the sections as you go through,
as you have those labs,
make sure that you not only have completed the lab
but that you've completed it in a way
that you know how to do that on your own.
That's going to help you both on the exam
and help you in your career.
And that's true whether Microsoft
has labs on the exam or doesn't.
Building that confidence and doing it for yourself
is really going to give you
better understanding of how it works.
All right, so this has been a fairly long review,
but we had a lot of stuff in this section to cover,
and it was a very broad section
covering a lot of different services.
So make sure that you have a good foundation
before moving forward.
And lastly, if you would do Landon and I a huge favor
and just leave a thumbs up as you go through these videos,
that really helps me to know
that the concepts and content
that I'm building for you is helpful.
So if you can do that, I would greatly appreciate it.
So, hey, that's it, you have completed section 2.
Landon is going to be picking up in section 3,
so he'll see you in the next section,
and I'll see you in a couple of sections down the road.
All right.