Welcome back.
Let's talk about the types of databases that exist now.
Here's the thing.
The world of databases is extremely complicated.
There's so many databases out there so many new innovations different databases for different things.
It's impossible to know all of them.
So we're just going to do a brief overview of what exists and why we have so many databases.
So let's start off with the first one.
Why do we have so many databases.
I mean look at this little diagram that I found that is a lot.
Well because we have different needs with data different databases are good at different things.
So there's many different databases because we need them from different solutions.
But the first thing we need to talk about when talking about databases is this idea.
There's three main categories of databases.
Now remember databases are just computers their computers with a hard drive essentially just like your
computers is able to store files and have files available anytime.
That's what a database is every time you turn off your computer and turn it back on.
You expect your files to be there and a database is that you expect that your data persists.
That is the data that you safe whether we turn off the computer or turn it back on.
It's still going to be there.
And then we can access it.
We can modify it.
That's the role of a database and we have three main types.
One was the original type of database which was relational database a relational database is a database
like post Cress or my sequel that allow you to use Eskew well to make transactions.
That is to write to the database to read to the database using the ASCII well language.
Now relational databases were all great until around thousands when initially we just had one database
one master database that held all the information that we needed.
But starting in the 2000s we got more and more data we collected more and more data and all of a sudden
all that information cannot be stored on just one database.
So we had an issue we now needed two databases to hold that information.
And although I'm simplifying things the idea was that now because we need distributed databases that
is we needed more than one database to store all that data relational databases didn't have the necessary
tools to have this idea of multiple databases working together.
So no sequel databases were born out of that databases like Mongo D.B. became popular because for the
first time it allowed us to do distributed databases.
That is we can have different machines 10 15 machines all working together as one database.
Finally more recently we have this idea of a new ask you all database.
What does that mean.
Well new rescue all databases are still fairly new but they tried to get the best of both worlds.
You see relational databases are really really good for something called acid transactions.
They have some really nice guarantees when we read and write data to make sure that it's always accurate.
No Eskew all databases were a little bit more disorganized but it allows us to store information on
distributed databases that is as our data gets bigger and bigger.
We can have a no Eskew database on different machines so that it scales up with our data new rescue
of databases like vault D.B. or cockroach D.B. are trying to have the distributed nature of no Eskew
all databases so we can scale up our data but also have some of the guarantees of relational databases.
Now these new rescue wells are still in the works but definitely keep an eye on them.
Another way to talk about databases is using databases for their own specific needs.
For example there's some data storage or databases that are specifically used for searches.
For example elastic search or solar are used as searching databases.
So we put data in there and it allows us to search for things really fast.
Then we have things like computational databases something like Apache Spark that allows us to use the
data that we have in the database and make computations and calculations on them and we'll get to Apache
Spark shortly.
Finally I just want to introduce you these terms that you might hear when it comes to data engineering.
Oh well T.P. and all that you can think of oil teepee databases as relational databases or Eskew old
databases that allows us to make transactions.
For example if you have a web app and that web app has a user that you can store user information you
can upload photos and have that application interact with a database for a user that's oh well DTP databases
and all that database is usually used for analytical purposes.
Now I'll some resources if you want to really dive deep into this topic.
But as you can see there's many ways to describe databases.
We have many different types of databases but at the end of the day each database is good for its own
specific use.
If we look back at our diagram databases are used everywhere.
For example these streams these are usually oil tepee databases like relational databases that collect
information let's say a user on a mobile phone updates their user name.
All that goes to a relational database and oil TB database and then a data lake well a data lake is
another database.
Usually something like Hadoop which we're going to talk about and then that data goes into a data warehouse.
All of these things that we see here are essentially databases we collect information from different
streams.
We have data lakes which our databases and data warehouses which are databases so over the next couple
of videos knowing what we know about databases and how we have different types of databases let's try
and figure out how this whole picture of a data engineer works.