Welcome back.
So what is data.
I mean we've talked about it already and we should have an idea of what data is.
And we also know why business care about data right.
We live in a world now where data is valuable and some of the most valuable companies are valuable because
they collect data.
Companies like Facebook and Google sell data to make money.
They provide products for customers by using the data to improve their products.
Data is the new gold so businesses care because by understanding their data it could be part of the
product such as creating a YouTube recommendation engine.
We can ask if the company is doing OK to monitor the company's sales and sticks and you can also use
data to improve the company and ask can we do better.
What can we improve on.
How can we make our apps faster so businesses care a lot about data.
But I want to take a bit of time to really understand what data engineers do by manipulating or collecting
this data so let's talk about the four different types of data we learned that there's things called
structured data.
Now this is data that is actually organized well for us to understand and we because data gets collected
in different ways from different sources.
Right.
But once on a computer there's a standard way usually in a table or what we call a matrix a table that
makes it easy to read.
Attributes are usually columns and rows are usually instances and sometimes we have the outputs in there
right.
Most column like we saw with machine learning where we predict an output based on the inputs and structure
data usually comes from things such as relational databases like my askew all post Cress and other relational
databases that allow us to do something called S Q Well queries.
That is to perform actions on the structure data to collect information then there's semi structured
data.
There's still structure data but often in something like an Excel CSP which we've seen or Jason form
and these are just different extensions and different ways to store data in different files for example
a really popular Web sites to grab data that's freely open online and find datasets maybe even do competitions
or see other people's notebooks and talk about machine learning topics.
Well Kaggle allows us to actually look at datasets that are available for us to use and play with and
in here if I type something like Excel you'll give me all of the information or all the data.
Let's make this bigger that are an ex email format.
So for example if I click on blood cell images and I take a look over here.
I see that the data sources there's some CSP files but there's also some accidental files.
And if I scroll down there's also some images as well.
So different formats of data that we can use.
By the way if you click on filter here you'll actually see some of the file types available for data.
Now we know about CSO fees.
There's also Jason where Jason if we take a look and let's go to this beer styles one if I scroll down
we see that it's an Jason format and Jason format.
You can see has this tree like structure to explore data.
So just a little bit different than CSC.
Just another way to organize data like I've mentioned on Kaggle you'll find all these types of datasets
using different things.
Ask you a light for example is a type of database and using Eskew you also structured data.
So if I do ask you a light over here and search for something using relational databases
you'll see that this uses ask you up and it tells us what kind of comps it has.
So lots of ways to explore Kaggle but these two types of data are usually well organized and easy for
us to manipulate with tools such as pandas for Sears fees and you can actually do Eskew all as well
with pandas.
The third kind is unstructured data.
And Daniel has already mentioned this.
Right.
This idea of an unstructured data are well they're not usually in simple formats that we can really
manipulate and analyze easily.
These are often things such as emails or PDA FS or a sort of documents where perhaps actually understanding
them is a little bit more difficult.
It's unstructured.
Finally we have a binary data.
These are things such as audio files image files video files they're in binary.
That is ones and zeros that computers can understand.
But for us to categorize them that's really really difficult.
As you can see it goes from top to bottom from really organized well labeled data to something that
is a little bit harder to organize especially for machines.
So why do we have to talk about these four types of data.
Well because most businesses the bigger they get aren't only going to have structure data they're going
to have all sorts of data all sorts of inputs that needs to somehow all work together.
So one of the jobs of a data engineer is to essentially use the fact that there's all these types of
data and somehow combine them or organize them in a way that is useful to the business.
Let's take a break here and explore some more in the next video.