1 00:00:00,060 --> 00:00:05,760 ‫So essentially, before we jump into shouting, just the idea of traditionally how we do things is we 2 00:00:05,760 --> 00:00:10,160 ‫have one big database with a huge tables, right. 3 00:00:10,440 --> 00:00:14,190 ‫And we would you might have some indexes on those tables. 4 00:00:14,400 --> 00:00:19,500 ‫You query and then you get results is always a centralized one database. 5 00:00:19,500 --> 00:00:27,960 ‫Your pin point is one server and query becomes slower and slower the more data you have because what 6 00:00:27,960 --> 00:00:31,170 ‫gets what you get and have indexes and those indexes are going to grow. 7 00:00:31,170 --> 00:00:36,910 ‫Raj and as your tables grow or large and and that kind of slow down your queries. 8 00:00:36,910 --> 00:00:37,170 ‫Right. 9 00:00:37,180 --> 00:00:38,430 ‫And you need more memory. 10 00:00:38,560 --> 00:00:39,790 ‫You need more CPU. 11 00:00:39,810 --> 00:00:41,370 ‫So it's just data scourer. 12 00:00:41,370 --> 00:00:42,020 ‫It just makes sense. 13 00:00:42,030 --> 00:00:47,790 ‫So if I want to make a select YORO from this table where you are like, well, this let's say this is 14 00:00:47,790 --> 00:00:50,040 ‫like a U URL shortener website. 15 00:00:50,040 --> 00:00:50,430 ‫Right. 16 00:00:50,730 --> 00:00:52,310 ‫Which is like one huge table. 17 00:00:52,380 --> 00:00:54,560 ‫This is the orals and this is the code. 18 00:00:54,900 --> 00:01:00,410 ‫So essentially what you do is like, hey, give me the URL for this code so I can visit that page. 19 00:01:00,720 --> 00:01:00,980 ‫Right. 20 00:01:01,000 --> 00:01:03,330 ‫That's what the URL Shawano does. 21 00:01:03,330 --> 00:01:03,650 ‫Right. 22 00:01:03,960 --> 00:01:08,040 ‫And then you query that that's a huge table to say there are one million rows. 23 00:01:08,280 --> 00:01:14,970 ‫So that's that square is going to get slower and slower, slower the more rows you have, because even 24 00:01:14,970 --> 00:01:17,550 ‫with indexes, that still is going to get slow. 25 00:01:17,550 --> 00:01:20,420 ‫Right, because it's just then it's going to grow and grow and grow. 26 00:01:20,430 --> 00:01:23,540 ‫So you want how do we speed things up? 27 00:01:23,910 --> 00:01:30,160 ‫So people invented the idea of sharding less blubbed that table and put it on multiple databases. 28 00:01:30,390 --> 00:01:31,740 ‫It sounds very simple. 29 00:01:31,740 --> 00:01:33,200 ‫And that's that's what it is. 30 00:01:33,210 --> 00:01:34,530 ‫It's very simple. 31 00:01:34,530 --> 00:01:36,180 ‫It's not really rocket science. 32 00:01:36,450 --> 00:01:45,690 ‫You partition your table row based partition based on certain field, certain column called Partition 33 00:01:45,690 --> 00:01:46,740 ‫Key or Shati. 34 00:01:47,160 --> 00:01:51,390 ‫And we throw those rows on multiple tables uniquely. 35 00:01:51,390 --> 00:01:51,710 ‫Right. 36 00:01:51,930 --> 00:01:59,340 ‫So let's say 200 K UARS goes here, another two hundred Keiko's here, another 200 K goes here. 37 00:01:59,340 --> 00:02:05,850 ‫But the question is, how do you Shahd, how do you distribute like I mean if you have customer table, 38 00:02:05,850 --> 00:02:07,190 ‫that's easy, right. 39 00:02:07,200 --> 00:02:12,450 ‫You can and do like on the zip code like these are the zip code for the westcoast. 40 00:02:12,450 --> 00:02:14,340 ‫This is a zip code for the East goes. 41 00:02:14,670 --> 00:02:22,020 ‫Or if you have a bunch of users, maybe users from one to five hundred million goes to this database 42 00:02:22,020 --> 00:02:28,050 ‫and from five hundred and one to one million goes to this database, that's another kind of sharding. 43 00:02:28,050 --> 00:02:28,290 ‫Right. 44 00:02:28,290 --> 00:02:29,430 ‫And that's kind of easier. 45 00:02:29,760 --> 00:02:33,210 ‫But what if your key is a string like this. 46 00:02:33,210 --> 00:02:33,590 ‫Right. 47 00:02:33,810 --> 00:02:40,530 ‫Which database server is this 550 O.J. is on. 48 00:02:40,620 --> 00:02:40,980 ‫Right. 49 00:02:40,980 --> 00:02:42,400 ‫So it's like, how do you know that? 50 00:02:42,450 --> 00:02:45,450 ‫So that's where we're going to talk about consistent caching. 51 00:02:45,450 --> 00:02:51,330 ‫But there is just a function and you can always find out it's not really hard or I can build it on function 52 00:02:51,330 --> 00:02:59,010 ‫to determine how to consistently hash this input to a server and this case. 53 00:02:59,010 --> 00:03:00,750 ‫Hey, I got to server three. 54 00:03:00,990 --> 00:03:03,390 ‫So let's we're going to talk about a little bit that. 55 00:03:03,860 --> 00:03:04,140 ‫Right. 56 00:03:04,180 --> 00:03:09,780 ‫And then once you find out which server you want to connect to, you connect to only that server and 57 00:03:09,780 --> 00:03:12,120 ‫you send your query to only that server. 58 00:03:12,330 --> 00:03:13,460 ‫Sounds simple, right? 59 00:03:14,220 --> 00:03:19,620 ‫You do not want to send your query to all databases to get essentially a bunch of messages. 60 00:03:19,620 --> 00:03:19,950 ‫Right. 61 00:03:20,280 --> 00:03:23,040 ‫You want to connect to the rights over that. 62 00:03:23,040 --> 00:03:27,030 ‫You're sure that your database or your employees is actually there? 63 00:03:27,360 --> 00:03:27,650 ‫Right. 64 00:03:27,660 --> 00:03:31,320 ‫And how do you find out that need consistent caching?