‫And let's talk about durability, durability, in a nutshell, is the process of persisting the rights
‫that clients make to the database to a nonvolatile system storage system that means anything that we
‫write is stored.
‫So that I mean, even if I lost power or even if I crashed as a client after my committing that transaction,
‫if I come back, I see the change.
‫If the database crashed after the commit, I restore the database.
‫I see that change.
‫I see it persisted in desk if my computer crashed.
‫I come back, I see this change.
‫That's what durability is.
‫And you might think of that as this, OK?
‫Of course, everything is durable.
‫Not really.
‫A lot of database system actually play it with that.
‫They play with the durability says OK, durability is slow because to make things durable, that means
‫you have to be persistent to disk and disk are inherently slow.
‫So a lot of databases play around.
‫This durability is okay.
‫I'm going to write my stuff to memory and occasionally on the background.
‫I'm going to snapshot and write a disk.
‫So I make I'm going to make my race faster.
‫Can you get the database that does that right is right?
‫Is does that right as an option?
‫Obviously.
‫Right.
‫But durability is one of the stronger relational databases must have right.
‫You have to be durable.
‫Right.
‫And we're going to define what that means, really.
‫So durability changes made by a committed transaction, emphasis on committed has to be committed if
‫it's not committed.
‫Really, all bets are off must be persistent in and under an adorable, non-volatile storage like SSD
‫or hard drive.
‫Durability techniques, there are many durability techniques.
‫There is the wall.
‫The right to head log, which really you don't want out when I show you all the tables and indexes and
‫stuff like that.
‫These are really the representation of the data, which is what people read, what would what clients
‫read?
‫But these are really large if you think about it.
‫So if you if you rewrite these things to disk, which we, well, durability, committing a transaction
‫takes time because this is huge.
‫These data and the data structures and the B trees and stuff like that.
‫Stakes done.
‫So what databases improvement is a version of the changes that happen only, which is built as effectively
‫as you can think of it as the deltas?
‫And then this is a write only right to head log in what's called a wall and any changes.
‫It goes to this immediately.
‫First, we flush it immediately to disk.
‫And this guarantees persistence.
‫So yeah, we didn't really update the data table right on on another data structure yet, but we can.
‫If we have a crash, we have this beautiful wall.
‫We can go back and read all the wall entries and then rebuild this state effectively, easily from a
‫persistent estate.
‫We still persist this.
‫We still ride this durable right to disk like the actual table itself, the changes.
‫But we don't rely on that because that could be slow.
‫And there's another approach, which is asynchronous snapshots or even synchronous snapshots for that
‫matter, where we're as we as we write, we keep everything in memory.
‫But then asynchronously on the bag and on, we snapshot everything to desk at once.
‫And that is also another concept Regice uses.
‫Actually, both of these things append only file is another option, which is very similar to the wall
‫really that Redis uses, which is keeps tracks of the changes of what happens and then writes these
‫things.
‫This guarantees that we have a very, you know, lightweight way of storing data in a in a very fast
‫way, but also in case of a crash, I can always read back and construct the state of the table completely
‫without story, without the need of storing the likes and the idea and the string and the blob and all
‫that stuff, right?
‫We change all.
‫We store only these changes and eventually we're going to override that as well.
‫Let's talk about the wall a little bit.
‫Writing a lot of data to disk is expensive, as I talked about that a little bit.
‫We have indexes, we have data files, we have columns, rows, you know, we have system file system
‫tables, all this stuff.
‫Or I think if you imagine writing all the changes and the amplification of the rights itself, that's
‫going to take a long time.
‫You can do it, but it's going to be a lot.
‫So we need a better way.
‫We need a more optimized compact way.
‫So they compress this version of of these changes, known as wall right ahead log segments.
‫So the only released log log segment and this this better be persisted because this is what tells us
‫what to change effectively.
‫Right.
‫So these are just tell you, Hey, we changed this value to this.
‫We change this value to this, which is this value to this and we we even compress that story.
‫So we get it in a very tiny segments.
‫So there is something very critical in database system, and this is one of the problem the operating
‫systems do.
‫And this is called the OS cache when when you actually write to the OS, you ask the operating system,
‫what are the Windows, a Windows or Linux?
‫When you win the Davis, ask something to ask the OS to write to disk.
‫The OS actually does something shady, it writes not to this rights, to its own memory cache.
‫Right?
‫And the reason is because it will want to batch these rides and flush it to disk at once for for performance
‫reasons.
‫Because most applications do does a lot of right and they they want to kind of batch these supplies
‫and do it once.
‫So because if you batch it, you did, you do less Io and if you do a less ill, as I always talk about
‫this, it was better.
‫But what's the problem with that?
‫What if I actually you say commit and the database?
‫Asked the database.
‫Ask the the OS to write the wall segments to the disk, but always say, Hey, I'm going to write it
‫to the cache instead.
‫The OS will tell the database that, hey, I wrote successfully, despite it, only it only wrote to
‫the cache.
‫It didn't write to the disk wall till the database that hey database or application doesn't know.
‫The database, just not an applications that ask her to write something and it will ask the database,
‫Hey, by the way, yeah, I'm the voice and I wrote your changes successfully, but it only brought
‫it to the cache, so the database will say, OK, sure, the database will say your transaction committed
‫successfully, but the problem is what if you get a crash there?
‫The database told us it committed successfully.
‫You get a crash, your computer crashed, you restart and then the odds, obviously.
‫The data was in the cache and the oice cache when you restart, it was in the RAM, obviously.
‫So look, we lost the data.
‫And as a result, the database lied to us.
‫The database in that particular situation is not durable because you wrote the transaction because the
‫transaction told us that it was committed, but it wasn't durable.
‫Why?
‫Because if there was a problem here.
‫So when the rights go to the OS and the OS crashed machine restart could lead to loss of the data.
‫And that's a big problem that Davies's fight for.
‫So there is this command that bypasses the OS cache and ask the database to ask the operating system,
‫Hey, you dumb os, I don't trust you.
‫I tell you to flush these wall changes, flush them directly to the disk.
‫This is called the F Sync Command, so I've set command just forces the right always to do so.
‫It's going to be slow.
‫It's going to be very, very slow.
‫So we have to do it in a smart way because you're bypassing the OS gash.
‫And you might say why?
‫Why this?
‫The OS uses the cache right isn't the dangerous component of most of the situations, like you don't
‫really get a crash that often.
‫So if you save award file on a notepad or something like things like that, it's OK to use the OS cache.
‫Most of the applications use the OS cash, and it's very rare that this crash can happen.
‫Database system is a different.
‫They want to guarantee durability, and as a result, they have to bypass the OS cat and as a result,
‫it becomes slow.
‫You see the trade off, guys, right?
‫There is always a trade off.
‫So if it can be expensive and slow, does commence, right, this is another problem, right?
‫If you commit, you want to commit as fast as possible.
‫So let's summarize the final lecture here, durability is the ability to persist that changes when the
‫transaction commenced, the emphasis on the one transaction commits.
‫So when the transaction commence, I want you to have persisted on desk.
‫I don't care.
‫I don't care where he persisted.
‫Here's what I want when I when you tell me it's commit, I can shut off the power at that exact moment.
‫And when I come back, my data better be there.
‫That's what durability means.
‫I don't care going to store it on, on, on the desk.
‫And this is the only storage on, I don't know, on the air.
‫Who cares?
‫I don't care.
‫I want my data back.
‫That's what durability is.
‫A lot of databases play a trade off with this properties because durability is slows, writes down because
‫now I'm going through this physical medium to process my changes, you know?
‫And as a result, it will be slower than if you brought to memory.
‫So riders, for example, plays some games or that gives you option actually, that none of the databases
‫that I know of even relational gives you that option.
‫Because guess what?
‫It gives you the option.
‫It says, Hey, you want to be completely durable, strongly durable.
‫Who do you want to be here?
‫Partially durable, not even partial posture was the wrong one.
‫Eventually, durable, I guess I just made up that word, but eventually is going to persist to disk.
‫So there is a slight few seconds delay that if you lose power off of or if you crash, you could lose
‫your data.
‫Some people trade that off their three seconds with a fast rights because, hey, I rather get fast,
‫right?
‫And OK, my data is not that important.
‫I'm stalling just logs off of my application IoT devices.
‫Who cares if I lose a few things?
‫But that's what right?
‫Most of the time.
‫Nobody reports loss because crashes are very rare.
‫If you think about it when it comes to the actual databases, those guys build the systems to not really
‫crash.
‫But you can't take that guarantee because if it's Murphy's Law, it says if it never happened, it will
‫eventually happen.
‫All right, guys.
‫That was the last lecture of this.
‫We're going to jump into many other lectures I've got to talk about.
‫I've enjoyed courses.
‫I'm going to show you examples here.
‫So go with another lecture.
‫It's called Essid example by practice.
‫I believe that's what I called it and show.
‫See how I do this in real time with the actual database.
‫Show you all these beautiful properties can also talk about eventual consistency, and I'm going to
‫take take, take a stab at the quiz and enjoy the rest of the course.
‫Thank you so much.
‫Goodbye.