1 00:00:04,060 --> 00:00:11,230 ‫What is going on, guys, my name is Hussein, and I want to do another system design video because 2 00:00:11,230 --> 00:00:20,260 ‫it's been a while and I bet Twitter as a sister design and the only thing that we know today is the 3 00:00:20,260 --> 00:00:21,460 ‫features we're going to build. 4 00:00:21,910 --> 00:00:26,350 ‫We want to build ability to post a tweet, 140 characters or more. 5 00:00:26,380 --> 00:00:27,800 ‫I think it's now to 80 percent. 6 00:00:27,820 --> 00:00:28,250 ‫Sure. 7 00:00:28,990 --> 00:00:31,390 ‫I want the ability to follow people and timeline. 8 00:00:32,050 --> 00:00:33,490 ‫I haven't prepared for this video. 9 00:00:33,940 --> 00:00:36,040 ‫I don't know how I was going to end up. 10 00:00:36,220 --> 00:00:41,050 ‫And I want it to be organic because there is no perfect system design. 11 00:00:41,350 --> 00:00:50,740 ‫As you guys know, and it's always better when you see my way of thinking and how I develop things, 12 00:00:50,970 --> 00:00:57,610 ‫and instead of me giving you a preform thought right, because it's like, OK, how did you end up with 13 00:00:57,610 --> 00:00:58,110 ‫this right? 14 00:00:58,120 --> 00:01:01,810 ‫In order to do that, guys think about these features, right? 15 00:01:02,140 --> 00:01:04,570 ‫I want the ability to post the tweet, right? 16 00:01:04,570 --> 00:01:08,440 ‫So to post, first of all, I need to know who you are. 17 00:01:08,920 --> 00:01:12,940 ‫So there are some sort of authentication going on, right? 18 00:01:13,420 --> 00:01:16,180 ‫So these are called the technical requirements. 19 00:01:16,210 --> 00:01:22,570 ‫These are the high level functional functional, the high level features that the user requested. 20 00:01:22,870 --> 00:01:29,560 ‫But now as a result, you end up with with tens, if not hundreds, of technical requirements that you 21 00:01:29,560 --> 00:01:33,570 ‫just need to deal with and the user and customer doesn't really care about, right? 22 00:01:33,880 --> 00:01:35,560 ‫That's the thing thinking you have to do. 23 00:01:35,740 --> 00:01:36,580 ‫So that's the first thing. 24 00:01:37,060 --> 00:01:38,450 ‫Some authentication system. 25 00:01:38,470 --> 00:01:39,310 ‫No problem. 26 00:01:39,640 --> 00:01:40,930 ‫Post a tweet, right? 27 00:01:40,930 --> 00:01:43,030 ‫Well, I need to store these tweets somewhere. 28 00:01:43,090 --> 00:01:44,920 ‫So they must be a database. 29 00:01:46,000 --> 00:01:50,680 ‫I'll pick relational just because I'm old school and follow people. 30 00:01:52,270 --> 00:01:54,770 ‫Yeah, I need to be able to follow people. 31 00:01:54,940 --> 00:01:57,940 ‫And that implies that people can follow me. 32 00:01:59,020 --> 00:01:59,290 ‫All right. 33 00:01:59,290 --> 00:02:02,080 ‫So we need some sort of a the data to store that. 34 00:02:02,920 --> 00:02:05,050 ‫These are the people I am following, right? 35 00:02:05,620 --> 00:02:09,070 ‫A table was should do and timeline. 36 00:02:09,220 --> 00:02:09,580 ‫Right? 37 00:02:10,150 --> 00:02:19,750 ‫And yeah, I want to be able to read everyone my time last, actually, and this is come composed of 38 00:02:19,750 --> 00:02:23,120 ‫all the tweets from everybody I'm following. 39 00:02:23,140 --> 00:02:26,530 ‫This is the home timeline that lets us be very specific about this. 40 00:02:26,530 --> 00:02:27,400 ‫A home timeline. 41 00:02:28,150 --> 00:02:30,670 ‫And there is a specific use our timeline. 42 00:02:31,150 --> 00:02:32,620 ‫We can worry about that later. 43 00:02:32,710 --> 00:02:32,950 ‫All right. 44 00:02:32,950 --> 00:02:41,800 ‫So let's start with so let's let's build the overall architecture of this first right and let the higher 45 00:02:41,800 --> 00:02:42,910 ‫level of first level. 46 00:02:43,660 --> 00:02:50,770 ‫So I'm assuming there would be some sort of a database here that will be storing my tweets, my, my, 47 00:02:50,770 --> 00:02:55,790 ‫my profile while my users, people, I follow all that stuff, right? 48 00:02:56,380 --> 00:02:58,750 ‫And there must be a client. 49 00:02:58,990 --> 00:03:04,930 ‫They don't say which client to use or could be a mobile client could be a web client. 50 00:03:05,120 --> 00:03:06,170 ‫I mean, any kind of coin. 51 00:03:06,200 --> 00:03:06,460 ‫Right. 52 00:03:06,460 --> 00:03:10,960 ‫So based on that, I can pick the language of choice. 53 00:03:11,380 --> 00:03:15,420 ‫But since we live in a web world, I'm going to pick the beautiful protocol. 54 00:03:16,480 --> 00:03:16,840 ‫Right? 55 00:03:17,770 --> 00:03:18,790 ‫They didn't say that, right. 56 00:03:18,790 --> 00:03:22,870 ‫But the moment I say it, should it be, it just works with everything, right? 57 00:03:23,170 --> 00:03:24,970 ‫Because this is a very standard protocol. 58 00:03:25,480 --> 00:03:26,800 ‫So this is my client now. 59 00:03:27,610 --> 00:03:31,030 ‫That's my client supposed to be a phone. 60 00:03:31,630 --> 00:03:32,110 ‫All right. 61 00:03:32,710 --> 00:03:34,030 ‫You can tell I'm old school. 62 00:03:35,260 --> 00:03:35,590 ‫All right. 63 00:03:35,590 --> 00:03:37,150 ‫So this is the phone, right? 64 00:03:38,110 --> 00:03:42,190 ‫Or the web browser cannot connect to the database directly. 65 00:03:42,220 --> 00:03:42,490 ‫Right? 66 00:03:42,520 --> 00:03:45,840 ‫Because the database doesn't speak A2DP as long. 67 00:03:46,210 --> 00:03:49,660 ‫I mean, couch DB does, but a little bit. 68 00:03:49,660 --> 00:03:55,200 ‫Not it's not mature enough, I think, to to be exposed directly, to be honest, right? 69 00:03:55,510 --> 00:04:01,090 ‫So you need some sort of a third tier here, which is my beautiful server. 70 00:04:05,290 --> 00:04:09,660 ‫This is a debate that database is declined. 71 00:04:11,280 --> 00:04:13,030 ‫That's the high level architecture, right? 72 00:04:13,360 --> 00:04:17,200 ‫And you would think that I can build the rest API. 73 00:04:17,460 --> 00:04:19,750 ‫Maybe that's enough to me, right? 74 00:04:20,680 --> 00:04:21,220 ‫Why? 75 00:04:21,230 --> 00:04:24,530 ‫Why don't I pick GOPAC, for example, or PopSockets? 76 00:04:24,550 --> 00:04:27,280 ‫Well, there is no real time requirement. 77 00:04:27,550 --> 00:04:34,180 ‫Those guys don't don't want real time updates or server center events or anything like that fancy. 78 00:04:34,180 --> 00:04:36,990 ‫So a PSDB purist tip will do so. 79 00:04:37,030 --> 00:04:41,020 ‫Rest will live and prosper. 80 00:04:41,560 --> 00:04:43,660 ‫Yeah, the rest is not dead bodies. 81 00:04:44,320 --> 00:04:48,010 ‫Yeah, you can go a fancy with the graphic here, but do you really need it? 82 00:04:48,880 --> 00:04:49,600 ‫We'll find out. 83 00:04:49,610 --> 00:04:50,530 ‫We'll find out soon. 84 00:04:50,530 --> 00:04:51,040 ‫We'll find out. 85 00:04:51,070 --> 00:04:52,300 ‫We can always change. 86 00:04:52,390 --> 00:04:55,210 ‫And that's why we keep thinking and just thinking together. 87 00:04:55,220 --> 00:04:57,850 ‫And I'm just thinking with you right now, guys. 88 00:04:58,100 --> 00:05:06,150 ‫And so if the client want to, let's say, poster two, we assume authentication is trivial. 89 00:05:06,160 --> 00:05:08,120 ‫We know what we have done that many times, right? 90 00:05:08,560 --> 00:05:10,590 ‫So I want to post a tweet, right? 91 00:05:11,210 --> 00:05:12,950 ‫I'm going to do a post, right? 92 00:05:13,510 --> 00:05:15,940 ‫And I'm going to in the body. 93 00:05:15,940 --> 00:05:17,110 ‫I'm going to put the tweet frame 94 00:05:20,090 --> 00:05:25,690 ‫right that will go to the server, the deep web server. 95 00:05:26,140 --> 00:05:27,160 ‫Any language? 96 00:05:27,760 --> 00:05:28,450 ‫Will do. 97 00:05:28,480 --> 00:05:29,560 ‫Because guess what? 98 00:05:29,620 --> 00:05:31,240 ‫Because I'm using this TV protocol. 99 00:05:31,330 --> 00:05:37,270 ‫If your language knows how to deal with HDTV, if you build a server that deals with the. 100 00:05:38,290 --> 00:05:44,500 ‫You can do this language doesn't matter as long as we speak the HDTV protocol, both in the client and 101 00:05:44,500 --> 00:05:44,950 ‫server. 102 00:05:45,070 --> 00:05:45,550 ‫Well, good. 103 00:05:45,880 --> 00:05:47,240 ‫And that's the beauty of the TV, Brooke. 104 00:05:47,260 --> 00:05:47,770 ‫All right. 105 00:05:48,670 --> 00:05:49,090 ‫We'll get it. 106 00:05:49,360 --> 00:05:51,610 ‫We're going to talk about which kind of protocol it is. 107 00:05:52,150 --> 00:05:54,820 ‫There should be two or one, one or three. 108 00:05:55,120 --> 00:05:55,350 ‫Right? 109 00:05:56,560 --> 00:05:58,350 ‫So yeah, I post here. 110 00:05:58,360 --> 00:06:08,170 ‫So that's a web request goes to the server, to the web server, and the server now immediately turns 111 00:06:08,170 --> 00:06:10,840 ‫around and speaks the language. 112 00:06:11,350 --> 00:06:17,230 ‫Let's say Typekit boss Chris, so it needs to understand the Postgres wire protocol. 113 00:06:17,800 --> 00:06:24,700 ‫And now it sends an insert statement right to populate. 114 00:06:25,840 --> 00:06:26,740 ‫So that's the sequel. 115 00:06:27,040 --> 00:06:29,110 ‫It's inserts into the database. 116 00:06:29,110 --> 00:06:30,720 ‫And so that's sweet, right? 117 00:06:31,000 --> 00:06:34,930 ‫And meanwhile, the client's waiting right? 118 00:06:34,930 --> 00:06:42,880 ‫And so the DB, if once the DB it writes the database, we get a get back the results and the server 119 00:06:43,090 --> 00:06:44,820 ‫will receive that result. 120 00:06:45,190 --> 00:06:49,600 ‫And obviously it says hey installed successfully. 121 00:06:49,810 --> 00:06:54,040 ‫Now the server is as free to respond whatever it like. 122 00:06:54,040 --> 00:06:58,660 ‫So I say, OK, Jason, like a Jason sexists something like that, right? 123 00:06:59,440 --> 00:07:03,900 ‫Or failed based on the results, let's say I don't know. 124 00:07:03,910 --> 00:07:08,830 ‫Why would a tweet fail to post network error or something like that, right? 125 00:07:09,670 --> 00:07:15,760 ‫And here here is where you as a designer, a system designer, kind of flourish. 126 00:07:15,770 --> 00:07:16,810 ‫What's you? 127 00:07:16,870 --> 00:07:23,620 ‫We can just take this guys and spent two hours discussing this guy's right. 128 00:07:23,620 --> 00:07:25,690 ‫Forget about follow people and everything, right? 129 00:07:26,140 --> 00:07:27,400 ‫Just just like that. 130 00:07:28,150 --> 00:07:30,070 ‫What are the problems with this design? 131 00:07:31,300 --> 00:07:38,800 ‫And when I say problems, I'm not saying, well, not saying you cannot live with it. 132 00:07:39,130 --> 00:07:42,040 ‫It's just limitations, kind of. 133 00:07:42,040 --> 00:07:42,970 ‫So let's think about it. 134 00:07:43,240 --> 00:07:43,900 ‫Let's think about it. 135 00:07:44,230 --> 00:07:51,700 ‫So if I post here and something happened here, I don't know, failed to answer. 136 00:07:51,970 --> 00:07:54,220 ‫There was a problem with this, right? 137 00:07:54,550 --> 00:07:55,300 ‫What will happen? 138 00:07:55,900 --> 00:07:56,800 ‫It will fail, right? 139 00:07:57,520 --> 00:08:02,680 ‫We have the option as building the server to keep retrying. 140 00:08:02,680 --> 00:08:05,260 ‫Maybe we can build and retrace. 141 00:08:06,680 --> 00:08:07,040 ‫Right. 142 00:08:07,340 --> 00:08:08,060 ‫You can do that. 143 00:08:08,570 --> 00:08:09,390 ‫Hey, let's try again. 144 00:08:09,410 --> 00:08:10,490 ‫Maybe the arrows in the talk. 145 00:08:11,670 --> 00:08:12,470 ‫Maybe. 146 00:08:14,560 --> 00:08:16,090 ‫We can do this on other things. 147 00:08:16,120 --> 00:08:25,390 ‫Maybe we can say, OK, a client, a client, what you don't want to keep the client waiting while you're 148 00:08:25,390 --> 00:08:31,300 ‫retrying me because maybe you kept retrying too many times and the client is waiting for this time, 149 00:08:31,300 --> 00:08:31,590 ‫right? 150 00:08:31,930 --> 00:08:36,040 ‫And and and that's I think this is a feature that is very critical, right? 151 00:08:36,670 --> 00:08:42,120 ‫User experience, how do I feel as a client? 152 00:08:42,130 --> 00:08:48,040 ‫I posted this, but I'm getting an error right now. 153 00:08:48,130 --> 00:08:56,350 ‫You can think about it, says, OK, how about if I the moment I post, I immediately respond with the 154 00:08:56,350 --> 00:08:59,110 ‫user, right says All right. 155 00:08:59,160 --> 00:09:01,690 ‫Success, despite me, didn't write to the database. 156 00:09:01,690 --> 00:09:02,350 ‫But guess what? 157 00:09:02,800 --> 00:09:07,530 ‫And instead of writing into a database which could fail for any reason, right? 158 00:09:08,410 --> 00:09:10,930 ‫Network error, maybe primary key. 159 00:09:11,200 --> 00:09:13,690 ‫I don't think of primary care, but it could be. 160 00:09:13,700 --> 00:09:15,640 ‫It could be congestion database anything, right? 161 00:09:16,300 --> 00:09:20,950 ‫So here's where a message queue comes in handy, right? 162 00:09:21,970 --> 00:09:22,550 ‫Says OK. 163 00:09:22,660 --> 00:09:31,030 ‫Message queue is very it's almost like a database, but you just says, Hey, I'm going to push this 164 00:09:32,200 --> 00:09:38,550 ‫to the queue a request to post a tweet right to the message queue. 165 00:09:38,590 --> 00:09:39,250 ‫And that's it. 166 00:09:39,490 --> 00:09:43,540 ‫The moment you write to the queue, you're free to return it to the client. 167 00:09:44,080 --> 00:09:45,490 ‫And what's the benefits of this? 168 00:09:45,670 --> 00:09:51,340 ‫The benefits this is, the client immediately gives the respondent his his or her tweet has been posted. 169 00:09:51,370 --> 00:09:53,100 ‫Technically, it was not. 170 00:09:53,110 --> 00:09:54,760 ‫It is not in the database, right? 171 00:09:55,000 --> 00:09:59,200 ‫So there is a lag, but who's saying, what are you doing? 172 00:09:59,210 --> 00:10:00,790 ‫Why there is a queue here? 173 00:10:00,810 --> 00:10:02,020 ‫What the purpose of this? 174 00:10:02,500 --> 00:10:08,080 ‫The purpose of the queue is as we as we do the following rain. 175 00:10:08,530 --> 00:10:16,720 ‫You can have these tweets tweet one tweet or tweet three in the queue and then have another process 176 00:10:18,070 --> 00:10:23,200 ‫right here that picks this up and writes it to the database in the state. 177 00:10:23,440 --> 00:10:24,820 ‫And this is what it does. 178 00:10:24,820 --> 00:10:26,140 ‫The right tries for us. 179 00:10:26,470 --> 00:10:29,440 ‫So this what did we gain with this architecture now? 180 00:10:29,950 --> 00:10:31,960 ‫First of all, let's start with the cons. 181 00:10:32,170 --> 00:10:34,170 ‫It's more complex to implement, right? 182 00:10:34,900 --> 00:10:35,380 ‫Why? 183 00:10:35,590 --> 00:10:40,270 ‫Because you just introduced a new hop right in and use infrastructure to maintain. 184 00:10:40,600 --> 00:10:44,680 ‫And then the second thing is there is there is a maintenance part of it. 185 00:10:45,550 --> 00:10:47,740 ‫So that's that's that's hard. 186 00:10:47,890 --> 00:10:51,220 ‫The benefit of this is you can never fail a tweet. 187 00:10:51,370 --> 00:10:57,610 ‫Once the tweet reaches the server, assuming it reach the server, now you want to throw another monkey 188 00:10:57,610 --> 00:10:58,180 ‫wrench, guys. 189 00:10:59,260 --> 00:11:01,840 ‫What if this communication failed? 190 00:11:02,860 --> 00:11:03,190 ‫Right? 191 00:11:03,220 --> 00:11:04,290 ‫We'll go to this man. 192 00:11:04,750 --> 00:11:10,990 ‫What if this failed right at something in the client connection and didn't even post right? 193 00:11:11,260 --> 00:11:12,730 ‫Does the client lose the tweet? 194 00:11:12,730 --> 00:11:13,660 ‫We're going to talk about that. 195 00:11:13,780 --> 00:11:16,180 ‫Well, we'll keep to that, and I hope we don't forget. 196 00:11:16,430 --> 00:11:18,460 ‫OK, I'm going to add a note to you. 197 00:11:19,240 --> 00:11:26,800 ‫But this architecture with the message queue, you can immediately post and this is a very nice architecture 198 00:11:26,800 --> 00:11:33,820 ‫where it works across the globe, because if you have a single message queue that it's distributed or 199 00:11:33,820 --> 00:11:40,390 ‫the servers can just published, the message and the message queue depends which message you use, RAID 200 00:11:40,510 --> 00:11:46,750 ‫can take care of persisting to the database, and this is how why the selling point of Kafka sort of 201 00:11:46,750 --> 00:11:48,280 ‫servers like Kafka, right? 202 00:11:49,570 --> 00:11:52,160 ‫Or Rabbit MQ Kafka? 203 00:11:52,660 --> 00:11:52,990 ‫Right? 204 00:11:53,960 --> 00:11:56,830 ‫Obviously, building Twitter system design, you don't. 205 00:11:56,980 --> 00:11:59,020 ‫You didn't think that it will get introduced Kafka. 206 00:11:59,020 --> 00:12:00,820 ‫And I'm not saying we should go. 207 00:12:01,720 --> 00:12:03,280 ‫It's simplicity again. 208 00:12:03,280 --> 00:12:09,850 ‫Simplicity rules with everything is just let's I'm thinking this out loud with the you as I am. 209 00:12:09,880 --> 00:12:12,910 ‫I'm just drawing this and I'm thinking with you, right? 210 00:12:13,030 --> 00:12:15,760 ‫And you as as I'm thinking, just right in the comments. 211 00:12:15,760 --> 00:12:18,250 ‫I Hussein, that's B.S. Don't do this. 212 00:12:18,370 --> 00:12:21,430 ‫This is better or saying, don't do this, do this instead. 213 00:12:21,430 --> 00:12:24,100 ‫Or let's just think, right? 214 00:12:24,580 --> 00:12:25,870 ‫There is no perfect system design. 215 00:12:25,870 --> 00:12:27,490 ‫It's an art. 216 00:12:27,730 --> 00:12:28,520 ‫This is an art. 217 00:12:28,550 --> 00:12:34,540 ‫You basically stamp your own thinking in in the design you build. 218 00:12:34,960 --> 00:12:42,460 ‫And that's the that's the best craft that we have as engineers back and engineers, software engineers 219 00:12:42,460 --> 00:12:43,110 ‫in general, right? 220 00:12:43,870 --> 00:12:46,990 ‫It's just this is an art, and I want you to think like that, guys. 221 00:12:47,710 --> 00:12:56,380 ‫That's why I don't like to watch other people how they do something I never did before unless I attempted 222 00:12:56,380 --> 00:12:56,800 ‫first. 223 00:12:57,100 --> 00:13:01,690 ‫If I watched the viewers and then watch others, or this guy did it better than me, oh, this is a 224 00:13:01,690 --> 00:13:03,310 ‫better way on this. 225 00:13:03,310 --> 00:13:04,870 ‫I think the way I did it is better. 226 00:13:05,290 --> 00:13:11,530 ‫That's why watching it for the first time, even this video, I recommend that you do it first. 227 00:13:11,560 --> 00:13:12,520 ‫How do you do it? 228 00:13:13,510 --> 00:13:15,970 ‫There always be a pros and cons whatever I'm going to do. 229 00:13:16,000 --> 00:13:19,960 ‫There is also going to be pros and cons, and it depends on up to you as a system architect. 230 00:13:20,410 --> 00:13:23,620 ‫Which ones do you want and which one and you can live with? 231 00:13:24,520 --> 00:13:24,790 ‫All right. 232 00:13:25,480 --> 00:13:27,970 ‫So, yeah, so we can introduce this. 233 00:13:28,450 --> 00:13:32,680 ‫We can kill the whole part if we don't want to and keep it simple like that. 234 00:13:33,610 --> 00:13:33,970 ‫All right. 235 00:13:35,050 --> 00:13:43,150 ‫Another way, if we decided not to introduce a message queue, we can build in the idea of pooling here. 236 00:13:45,010 --> 00:13:47,320 ‫That's a little bit expensive, right? 237 00:13:48,460 --> 00:13:56,590 ‫And I wish someone does implements better implementation of connection and we talked about pulling guys. 238 00:13:56,590 --> 00:13:59,290 ‫Check out the video right here, right? 239 00:13:59,740 --> 00:14:09,040 ‫And essentially, we talked about connection pooling where we the server establishes a set of TCP connections 240 00:14:09,040 --> 00:14:11,260 ‫between the servers preheated. 241 00:14:12,080 --> 00:14:19,240 ‫These connections are TCP, a pre heated. 242 00:14:19,660 --> 00:14:20,410 ‫What does that mean? 243 00:14:20,710 --> 00:14:26,770 ‫It means that because the act of establishing this connection is expensive, so the server started starting 244 00:14:26,770 --> 00:14:32,860 ‫with 10 or four or five GCP connections, and now it sends the request in. 245 00:14:32,860 --> 00:14:34,630 ‫These DCP can actually receive a response. 246 00:14:34,630 --> 00:14:37,050 ‫So it's it's faster technically, right? 247 00:14:38,140 --> 00:14:40,900 ‫So that's I can do something like that, right? 248 00:14:41,380 --> 00:14:49,570 ‫Obviously, it's the worst thing is when you receive a post tweet request, you immediately go and establish 249 00:14:49,570 --> 00:14:50,680 ‫a TCP connection. 250 00:14:50,920 --> 00:14:52,420 ‫You don't want to do that, right? 251 00:14:52,600 --> 00:14:57,460 ‫That's just that will feel the LAG client feels the lag here. 252 00:14:57,760 --> 00:15:01,600 ‫And same thing with the message queue, by the way, guys, because this is a TCB connection, right? 253 00:15:02,080 --> 00:15:06,280 ‫If you establish a DCB connection between you and the server and the message key, but I pull these 254 00:15:06,280 --> 00:15:16,210 ‫things or at or or if the message queue like rabbit in queue and Kafka, I believe rabbit to give you 255 00:15:16,210 --> 00:15:18,820 ‫a Kafka supports something called channels. 256 00:15:21,720 --> 00:15:28,170 ‫I know this video turned to from the system design into and to on that back end concept, but let's 257 00:15:28,570 --> 00:15:32,990 ‫discuss it because this is you just watch this for fun and we're learning, right? 258 00:15:33,060 --> 00:15:34,500 ‫There's no perfect design. 259 00:15:34,770 --> 00:15:36,210 ‫We're just thinking about that stuff. 260 00:15:37,320 --> 00:15:37,590 ‫Yeah. 261 00:15:37,590 --> 00:15:45,480 ‫So as we if we have a single TCP connection with channels just like HDTV two with streams exactly the 262 00:15:45,480 --> 00:15:46,500 ‫same thing, right? 263 00:15:46,710 --> 00:15:50,670 ‫Where we have a single TCP connection and there are streams inside of it, right? 264 00:15:50,670 --> 00:15:57,510 ‫And I can send many requests in the same TCP connection and I and I tag them with the channel idea or 265 00:15:57,510 --> 00:15:58,290 ‫destiny mighty. 266 00:15:58,590 --> 00:16:01,060 ‫So the other guys pooling is important, right? 267 00:16:01,080 --> 00:16:05,630 ‫If you don't want to use cover to use this concept here for the database, and some of you might think 268 00:16:05,640 --> 00:16:07,860 ‫I was like, where's this caching layer that we're going to talk about? 269 00:16:08,220 --> 00:16:15,750 ‫Let's not introduce complicated things unless we absolutely have a problem, right? 270 00:16:15,960 --> 00:16:17,310 ‫As we think about it, right? 271 00:16:17,490 --> 00:16:20,430 ‫We're going to introduce caching, of course, if we need it right. 272 00:16:20,970 --> 00:16:23,700 ‫So now this is just posting, right? 273 00:16:23,730 --> 00:16:25,260 ‫This is just a right operation. 274 00:16:25,260 --> 00:16:34,380 ‫And let's discuss this part where the posting of the tweet failed right here between the connection 275 00:16:34,380 --> 00:16:36,630 ‫between the client and the server. 276 00:16:37,360 --> 00:16:39,870 ‫Let's change the color blue, the client. 277 00:16:41,340 --> 00:16:42,120 ‫So this has failed. 278 00:16:42,540 --> 00:16:43,770 ‫I want to post. 279 00:16:44,040 --> 00:16:49,890 ‫But the client was driving away from his home and he lost Wi-Fi connection. 280 00:16:50,310 --> 00:16:50,700 ‫Right? 281 00:16:52,890 --> 00:17:00,000 ‫Immediately lost the Wi-Fi connection and the posting of the tweet, right, the TCP connection establishment 282 00:17:00,000 --> 00:17:08,340 ‫was there, but that packet that contains the tweet or part of the tweet didn't get acknowledgement 283 00:17:08,340 --> 00:17:09,180 ‫from the server. 284 00:17:09,360 --> 00:17:13,890 ‫So the client says, Oh, looks like the connection is dead, right? 285 00:17:14,610 --> 00:17:17,130 ‫After a while, we found out that the TCP connection is dead. 286 00:17:17,490 --> 00:17:19,410 ‫And what do we do in this case, right? 287 00:17:19,680 --> 00:17:20,820 ‫We just failed. 288 00:17:21,240 --> 00:17:21,630 ‫Right? 289 00:17:22,320 --> 00:17:28,530 ‫And I just wrote this beautiful tweet that I cannot think of, and I posted it and I think the other 290 00:17:29,170 --> 00:17:30,150 ‫poster, but it didn't. 291 00:17:30,450 --> 00:17:30,660 ‫Right? 292 00:17:30,900 --> 00:17:32,970 ‫So the client just lost this tweet. 293 00:17:32,970 --> 00:17:35,280 ‫So this leads into a bad user experience. 294 00:17:35,280 --> 00:17:40,170 ‫And when I say user, I don't mean buttons and beautiful page. 295 00:17:40,170 --> 00:17:46,500 ‫No, I mean, how the user is experiencing their product and this is a bad user experience to us, right? 296 00:17:46,650 --> 00:17:51,870 ‫Because I clicked and I then I posted, Yeah, I got an error, but I lost my tweet, right? 297 00:17:52,560 --> 00:17:59,880 ‫So this is where it's a good idea to implement some sort of local database here at the client side. 298 00:18:00,090 --> 00:18:01,980 ‫A popular one is light. 299 00:18:02,220 --> 00:18:02,580 ‫Right? 300 00:18:03,480 --> 00:18:06,480 ‫And just stored the tweet locally. 301 00:18:06,660 --> 00:18:07,140 ‫Right. 302 00:18:07,530 --> 00:18:07,980 ‫What? 303 00:18:08,370 --> 00:18:13,650 ‫In parallel, because you're going to use writing on your client app, right? 304 00:18:14,670 --> 00:18:19,350 ‫Spin up a thread to write to the disk and another thread to send that. 305 00:18:19,800 --> 00:18:22,310 ‫And that's not going to take much. 306 00:18:22,320 --> 00:18:26,380 ‫And for much of your memory, because this is asynchronous request. 307 00:18:26,380 --> 00:18:29,160 ‫So you're going to send it and just wait might as well. 308 00:18:29,160 --> 00:18:31,200 ‫Just this is the same thing writing to disk. 309 00:18:31,920 --> 00:18:40,950 ‫So this way, if you write it to the desk, while just the act of posting the tweet will stored this 310 00:18:41,040 --> 00:18:42,210 ‫in the ciclo database. 311 00:18:42,210 --> 00:18:43,710 ‫So if this case failed? 312 00:18:44,220 --> 00:18:44,630 ‫Right? 313 00:18:45,300 --> 00:18:49,140 ‫Just have a process that stores it in the drafts, right? 314 00:18:49,290 --> 00:18:50,610 ‫That's the draft. 315 00:18:51,090 --> 00:18:54,540 ‫So those guys don't didn't tell us about this feature, right? 316 00:18:54,780 --> 00:19:01,800 ‫But might as well just do it right because that's they just expect it if I post a tweet, right? 317 00:19:01,950 --> 00:19:03,270 ‫You can think about all this stuff. 318 00:19:03,540 --> 00:19:05,520 ‫Guys, we're stuck right here now. 319 00:19:05,520 --> 00:19:11,550 ‫We didn't even move to the second feature, so I swear to God I can go in for two or three hours discussing 320 00:19:11,550 --> 00:19:14,190 ‫this step, but I'm not going to go through all that stuff, right? 321 00:19:14,200 --> 00:19:17,820 ‫Because this is going to take ages, but it's a good exercise. 322 00:19:17,940 --> 00:19:23,310 ‫So, yeah, it's a sequel is a very, very interesting concept, right? 323 00:19:23,340 --> 00:19:25,190 ‫We store this locally in all our guys. 324 00:19:25,240 --> 00:19:30,030 ‫This is a single data, a single user database, and it's designed for us. 325 00:19:30,090 --> 00:19:33,600 ‫Basically, it comes with almost ships with every phone, I guess. 326 00:19:33,960 --> 00:19:35,460 ‫And you can use it to store locally. 327 00:19:35,550 --> 00:19:35,880 ‫Yeah. 328 00:19:36,480 --> 00:19:40,980 ‫So yeah, if it failed, I can just pick up the next almost like a queue, right? 329 00:19:41,910 --> 00:19:45,180 ‫I pick up my next to a tweet and I try retry three try. 330 00:19:45,450 --> 00:19:47,310 ‫So that's a nice feature, right? 331 00:19:47,910 --> 00:19:52,920 ‫Or even though you don't have to add the feature to auto retry. 332 00:19:53,520 --> 00:19:53,850 ‫Right. 333 00:19:54,150 --> 00:20:01,230 ‫We don't want a feature to auto retry generally, just like save it to the draft to say, Hey, this 334 00:20:01,230 --> 00:20:07,980 ‫failed leather you don't know with an alert message, your user sop this failed. 335 00:20:08,370 --> 00:20:09,770 ‫You might want to try it again. 336 00:20:09,780 --> 00:20:14,010 ‫Click on that button and then the user will try again when that connection comes back. 337 00:20:14,100 --> 00:20:17,130 ‫All right, so that's that's a nice user experience post to it. 338 00:20:17,280 --> 00:20:21,900 ‫I think I'm going to leave it at that with the post tweet and stopped, too. 339 00:20:21,910 --> 00:20:24,170 ‫We didn't discuss the protocol between this and that. 340 00:20:24,180 --> 00:20:26,490 ‫Maybe he'll spend a few minutes discussing that. 341 00:20:26,820 --> 00:20:32,140 ‫And then just in the video right there. 342 00:20:32,610 --> 00:20:38,730 ‫So the protocol that we discussed, rest, we said rest is work, and I think it does for posting at 343 00:20:38,730 --> 00:20:39,350 ‫least right. 344 00:20:39,360 --> 00:20:42,330 ‫We didn't go through the rest stuff, but maybe with this will change. 345 00:20:42,630 --> 00:20:43,470 ‫No problem. 346 00:20:44,550 --> 00:20:54,150 ‫I might use the idea of scaling this thing right, scaling the posting process, right? 347 00:20:54,600 --> 00:21:00,570 ‫And if you think about it, you are one client and you're connected to one server and we introduce any 348 00:21:00,570 --> 00:21:04,320 ‫load balancers would introduce any of that stuff yet, right? 349 00:21:04,680 --> 00:21:13,830 ‫So if you think about it, you can introduce this is your back in server that communicate to the database. 350 00:21:14,220 --> 00:21:14,630 ‫All right. 351 00:21:14,640 --> 00:21:18,780 ‫And let's say we have one global database that might be OK. 352 00:21:20,180 --> 00:21:30,300 ‫You can introduce a load balancer that sits between the client and the server and and and takes and 353 00:21:30,300 --> 00:21:34,590 ‫takes these flood of tweets and scale them up. 354 00:21:34,830 --> 00:21:36,660 ‫How about we do that instead? 355 00:21:36,720 --> 00:21:37,010 ‫All right. 356 00:21:37,190 --> 00:21:43,410 ‫So in order to scale this, let's think about and I have an idea of just adding more servers, right? 357 00:21:43,770 --> 00:21:48,540 ‫I'm going to add the same copy of the server right here, right here and right here. 358 00:21:48,540 --> 00:21:49,880 ‫This is the web server, right? 359 00:21:50,640 --> 00:21:50,970 ‫And. 360 00:21:52,160 --> 00:21:57,270 ‫I'm still didn't discuss whether I'm using TV or HDTV to we're going to come to that right. 361 00:21:57,290 --> 00:22:00,020 ‫I don't know if I need that, but we'll see. 362 00:22:00,640 --> 00:22:10,010 ‫I know this is my client's right and my client will either have this idea, right? 363 00:22:10,340 --> 00:22:12,890 ‫We have multiple public servers, right? 364 00:22:12,920 --> 00:22:15,260 ‫Well, the specific domains and addresses. 365 00:22:15,560 --> 00:22:25,220 ‫And we can let the client be aware of these servers like Server one, Server two and Server three and 366 00:22:25,220 --> 00:22:30,330 ‫the server picks one of these servers, server one server to server thing. 367 00:22:30,380 --> 00:22:32,160 ‫I might say, who's saying, What do you what are you are doing? 368 00:22:32,180 --> 00:22:33,290 ‫That's not a load balancer. 369 00:22:33,770 --> 00:22:38,600 ‫Well, this is an idea of the client level load balance, and some people do that. 370 00:22:38,900 --> 00:22:42,060 ‫Netflix does that sometime to avoid another layer. 371 00:22:42,080 --> 00:22:42,350 ‫Right? 372 00:22:42,380 --> 00:22:48,060 ‫We're going to talk about both architecture and see what you like and you prefer this debate. 373 00:22:49,160 --> 00:22:56,930 ‫So the client here first makes it communication to this server, right to some sort of a server that 374 00:22:56,930 --> 00:23:01,580 ‫tells you, OK, these are the available servers, and then it starts stabilize in connection with one 375 00:23:01,580 --> 00:23:02,540 ‫of the servers, right? 376 00:23:02,930 --> 00:23:08,240 ‫This is the double TCP connection, which is basically the DTP protocol, right? 377 00:23:09,650 --> 00:23:10,700 ‫HDTV protocol. 378 00:23:11,360 --> 00:23:17,510 ‫So now establish this connection and I send my tweet and now if I have another client right here, it 379 00:23:17,510 --> 00:23:21,740 ‫does that it goes to this server and that a client goes to this summer. 380 00:23:21,770 --> 00:23:23,840 ‫It's just absolute random, right? 381 00:23:24,110 --> 00:23:32,660 ‫And this list can be obtained by basically literally hard coding this list into this device, right? 382 00:23:32,900 --> 00:23:33,740 ‫You can ask. 383 00:23:33,920 --> 00:23:39,950 ‫Another thing is that you can ask the server that you communicate with, Hey, give me the latest list 384 00:23:39,950 --> 00:23:41,040 ‫of servers and who's going to? 385 00:23:41,270 --> 00:23:45,440 ‫Oh, by the way, this one is now dead is still is three is available, right? 386 00:23:46,910 --> 00:23:50,610 ‫And this puts the load balancing at the client side. 387 00:23:50,630 --> 00:23:57,320 ‫There are benefits to this because with this, you don't have another layer of load balancer that you 388 00:23:57,320 --> 00:23:58,370 ‫have to worry about. 389 00:23:58,700 --> 00:24:06,800 ‫Plus, these servers are distributed and there is no other layer that communicates with the server directly. 390 00:24:07,070 --> 00:24:15,290 ‫But that but the disadvantage of this is obviously this awareness of the these servers to the client 391 00:24:15,290 --> 00:24:15,650 ‫side. 392 00:24:16,580 --> 00:24:25,610 ‫That's kind of bad because you cannot do through load balancing with this, you cannot do blue green 393 00:24:25,610 --> 00:24:29,890 ‫deployments or I guess is, OK, I want to connect to this guy. 394 00:24:29,900 --> 00:24:35,090 ‫But this I want this users to use this version versus this rhyme. 395 00:24:35,150 --> 00:24:36,470 ‫So that's one way to do it. 396 00:24:36,500 --> 00:24:43,100 ‫I just wanted to kind of talk about it before we jump into the other one, which is the most popular 397 00:24:43,100 --> 00:24:43,340 ‫one. 398 00:24:43,820 --> 00:24:47,950 ‫And that is the introduction of the load balancer, right? 399 00:24:48,830 --> 00:24:49,660 ‫And the load balancer. 400 00:24:49,670 --> 00:24:53,570 ‫There are many types of load balancers layer for layer seven. 401 00:24:53,690 --> 00:24:59,300 ‫Most of the time you're going to use a layer seven load balancer right following to post this tweet 402 00:24:59,300 --> 00:24:59,870 ‫right now. 403 00:25:00,500 --> 00:25:06,920 ‫I'm going to establish here is another disadvantage of this, but we're going to talk about it in a 404 00:25:06,920 --> 00:25:07,300 ‫minute minute. 405 00:25:07,610 --> 00:25:13,370 ‫So if I'm going to establish a connection between the client and the load balancer, that means talked 406 00:25:13,370 --> 00:25:14,450 ‫about encryption, right? 407 00:25:14,720 --> 00:25:17,600 ‫The server this load balancer need to serve me. 408 00:25:17,600 --> 00:25:18,980 ‫It's certificate, right? 409 00:25:20,000 --> 00:25:24,530 ‫Because I need to have the certificate of the load base or right here. 410 00:25:24,840 --> 00:25:25,040 ‫OK. 411 00:25:26,330 --> 00:25:30,470 ‫That's how that's how you establish deals between the client and the server. 412 00:25:30,620 --> 00:25:35,900 ‫In order to post that information, you need to encrypt stuff, right that we didn't talk about encryption 413 00:25:35,900 --> 00:25:36,230 ‫at all. 414 00:25:36,530 --> 00:25:37,400 ‫But that's the idea. 415 00:25:37,790 --> 00:25:43,100 ‫You encrypt that encryption communication between the client and the server, and then poof, you start 416 00:25:43,100 --> 00:25:43,800 ‫communicating. 417 00:25:44,540 --> 00:25:49,550 ‫And then the load balancer does that. 418 00:25:50,030 --> 00:25:50,990 ‫The following, right? 419 00:25:50,990 --> 00:25:53,420 ‫It connects with all of these servers. 420 00:25:53,940 --> 00:25:54,260 ‫Right. 421 00:25:54,740 --> 00:25:57,470 ‫And just shoves the connection. 422 00:25:57,620 --> 00:26:02,690 ‫And then if the client posted a tweet can either go to this server, which eventually ended that at 423 00:26:02,690 --> 00:26:03,950 ‫an airbase or the message queue. 424 00:26:03,980 --> 00:26:06,620 ‫It depends on which architecture, urban or goes to. 425 00:26:06,620 --> 00:26:08,750 ‫This all goes to this right? 426 00:26:09,530 --> 00:26:16,040 ‫So that's been going on depends on the algorithm you picked, whether this is a generic proxy or whether 427 00:26:16,040 --> 00:26:21,860 ‫this is traffic that will funnel the request to each of the servers. 428 00:26:22,550 --> 00:26:22,880 ‫Right. 429 00:26:23,220 --> 00:26:26,600 ‫And this is not one request, guys, right? 430 00:26:26,630 --> 00:26:28,450 ‫This is technically it's two. 431 00:26:28,460 --> 00:26:31,790 ‫It's between this and this and between this and this. 432 00:26:32,210 --> 00:26:36,540 ‫If it is a layer seven load bouncer. 433 00:26:36,740 --> 00:26:41,270 ‫And the reason is this is the final destination to the client. 434 00:26:41,300 --> 00:26:41,900 ‫That's it. 435 00:26:42,050 --> 00:26:44,060 ‫The the client connects to this. 436 00:26:44,070 --> 00:26:45,330 ‫This is the final destination. 437 00:26:45,350 --> 00:26:48,110 ‫That's why it's called also a reverse proxy, right? 438 00:26:48,870 --> 00:26:49,730 ‫Reverse proxy. 439 00:26:50,060 --> 00:26:50,810 ‫The client, this is. 440 00:26:50,840 --> 00:26:56,120 ‫The final destination to it and there the reverse proxy makes the request on behalf of the client, 441 00:26:56,120 --> 00:27:00,870 ‫so technically the server to the final destination back and server doesn't know their client, and that's 442 00:27:00,890 --> 00:27:02,420 ‫the definition of the reverse proxy. 443 00:27:02,940 --> 00:27:03,140 ‫Right. 444 00:27:03,140 --> 00:27:04,730 ‫So that's the layer seven proxy. 445 00:27:05,030 --> 00:27:08,930 ‫If this is, you need also the backend to be secure. 446 00:27:09,200 --> 00:27:13,450 ‫So you need this to understand how to do TLS right? 447 00:27:13,460 --> 00:27:16,910 ‫As a client frame, you need a good encryption. 448 00:27:16,910 --> 00:27:23,510 ‫Algorithms between this and this side of the good cipher suites at this one point three and above, 449 00:27:23,510 --> 00:27:25,490 ‫even for you, is going to use TLS 1.2. 450 00:27:25,820 --> 00:27:28,100 ‫Use a good ciphers and you should be safe. 451 00:27:28,100 --> 00:27:31,310 ‫But with this one boy three, you have fewer handshakes. 452 00:27:31,310 --> 00:27:35,390 ‫So that's the pre warming of this connection that we talked about that, right? 453 00:27:35,660 --> 00:27:42,780 ‫You still need some TCP connections on the back in here to accept establishment and unless if you don't 454 00:27:42,780 --> 00:27:43,850 ‫want to use pooling. 455 00:27:45,410 --> 00:27:51,380 ‫And I don't know which proxies support that you might want to use HDTV to at the back. 456 00:27:51,380 --> 00:27:52,670 ‫And let's think about that a little bit. 457 00:27:53,060 --> 00:27:57,650 ‫If I used to be two of the back end, then it's a beautiful one TCP connection. 458 00:27:58,160 --> 00:28:05,630 ‫And the more they do that, then I don't have a problem off at HDB smuggling, right? 459 00:28:05,870 --> 00:28:07,090 ‫Because there's no smuggling. 460 00:28:07,100 --> 00:28:12,500 ‫You have this idea with this where a client can smuggle connection if you are using pooling and if you 461 00:28:12,500 --> 00:28:16,010 ‫wanted the back end and I talk about this, which Steve is smuggling right here, check out this video, 462 00:28:16,610 --> 00:28:16,970 ‫right? 463 00:28:17,000 --> 00:28:19,130 ‫And well, to be too. 464 00:28:19,160 --> 00:28:22,480 ‫You have one TCP connection, you Shelvey's just like that. 465 00:28:22,490 --> 00:28:26,150 ‫The other thing multiple streams, multiple channels in the same connection. 466 00:28:26,150 --> 00:28:27,320 ‫So this more efficient. 467 00:28:27,620 --> 00:28:29,810 ‫So if you would, you want to do that load balancer? 468 00:28:30,020 --> 00:28:35,750 ‫Make sure your H2 all the way, because that's the future, right edge to here as to here. 469 00:28:36,140 --> 00:28:39,830 ‫This way you have one TB connection and then you shoves multiple stuff. 470 00:28:39,830 --> 00:28:42,440 ‫Let's say the user wants to post multiple tweets at once. 471 00:28:42,740 --> 00:28:43,070 ‫Right? 472 00:28:43,910 --> 00:28:47,060 ‫It's unlikely as a requirement. 473 00:28:47,060 --> 00:28:53,390 ‫So that's why I don't really it doesn't really matter to use a or not here because it's still a little 474 00:28:53,390 --> 00:29:00,350 ‫bit as a CPU intensive protocol, especially at this site, the front end of the load balancer. 475 00:29:00,650 --> 00:29:02,390 ‫So you got to be careful of that. 476 00:29:03,200 --> 00:29:05,600 ‫I guess it's all trade off, to be honest, right? 477 00:29:06,020 --> 00:29:10,130 ‫I can't believe we're just stuck right here discussing all that stuff. 478 00:29:10,670 --> 00:29:12,880 ‫So, yeah, there are many ways to solve a solution. 479 00:29:12,880 --> 00:29:17,180 ‫And guys and I don't know which one to pick a Lloyd Bentsen is definitely a popular one. 480 00:29:17,420 --> 00:29:18,800 ‫But guess what, guys? 481 00:29:18,800 --> 00:29:23,710 ‫This lot better will run out of connections really fast if you're using just x1 X2. 482 00:29:23,750 --> 00:29:28,220 ‫You got to get a little bit more money out of your book. 483 00:29:28,250 --> 00:29:30,440 ‫Is that they're saying, I don't think that's the same. 484 00:29:30,560 --> 00:29:36,230 ‫Yeah, you're going to get more out of your buck with your load balancer if you're using X2 because 485 00:29:36,230 --> 00:29:38,050 ‫it still has a fewer TCP connections. 486 00:29:38,340 --> 00:29:46,120 ‫If using edge one here and each one here, it should be one that is, then you can only send one request 487 00:29:46,130 --> 00:29:49,130 ‫per TB connection at a given time. 488 00:29:49,970 --> 00:29:53,240 ‫That's not true with pipeline, but pipeline have been proven to do. 489 00:29:53,240 --> 00:29:56,950 ‫Doesn't do, don't work with proxies as effective. 490 00:29:56,960 --> 00:29:58,880 ‫But yeah, we can use it. 491 00:29:58,880 --> 00:30:01,840 ‫But that that means a lot of TCP connection. 492 00:30:01,850 --> 00:30:02,930 ‫That means pooling. 493 00:30:02,930 --> 00:30:09,200 ‫As we discussed, pooling is good, but it's a very CPU hungry operation that means preheating connections. 494 00:30:09,200 --> 00:30:10,280 ‫That means more memory. 495 00:30:10,640 --> 00:30:13,720 ‫And this will this will definitely run out very quickly. 496 00:30:13,730 --> 00:30:19,340 ‫So if you can use QE2 on the back end and they should do in the front end, that will save you some 497 00:30:19,340 --> 00:30:19,790 ‫memory. 498 00:30:20,060 --> 00:30:23,540 ‫Those little bands that can only have so many TCP connection going. 499 00:30:23,650 --> 00:30:25,250 ‫So it's almost like a bottleneck. 500 00:30:25,490 --> 00:30:30,770 ‫That's why I like to flirt with the idea of having the client. 501 00:30:30,770 --> 00:30:39,500 ‫Does the load balancing by actually know that individual servers and just directly connect to them another 502 00:30:39,500 --> 00:30:44,300 ‫way, if using a layer four load balancer, then 503 00:30:47,240 --> 00:30:49,430 ‫it's not really one DC began. 504 00:30:49,430 --> 00:30:54,650 ‫Actually, if you think about it, there is a case where if you're using that, then it is a one TB 505 00:30:54,650 --> 00:30:57,680 ‫connection all the way, but it's very hard to deploy. 506 00:30:57,920 --> 00:31:04,430 ‫You get a change the gateway right here that you're ingress to point to that to that stuff, right? 507 00:31:05,450 --> 00:31:06,920 ‫Let's clean up this little bit. 508 00:31:07,010 --> 00:31:07,330 ‫All right. 509 00:31:07,340 --> 00:31:11,600 ‫Let's discuss layer four load balancing load right here. 510 00:31:12,500 --> 00:31:14,150 ‫So I am a client right here. 511 00:31:14,390 --> 00:31:15,740 ‫What kind of client is this? 512 00:31:17,090 --> 00:31:22,010 ‫The client makes it TCP connection between this, and this was designed initially to be connection. 513 00:31:22,400 --> 00:31:29,360 ‫But the beauty of the layer for connection is just it's stuck to one server, and any request that you 514 00:31:29,360 --> 00:31:31,400 ‫send always goes to the server. 515 00:31:31,940 --> 00:31:32,510 ‫It's not. 516 00:31:32,930 --> 00:31:37,430 ‫It is load balanced, but the first hop is on only load band. 517 00:31:37,430 --> 00:31:44,090 ‫So when you make the handshake Nissin request, the proxy will always remember it all. 518 00:31:44,090 --> 00:31:47,100 ‫Or this is based on your source IP source port. 519 00:31:47,120 --> 00:31:50,570 ‫I always get a forwarded to S1, so any request? 520 00:31:50,620 --> 00:31:54,760 ‫That you said always goes to the same server, always might be good or bad. 521 00:31:55,180 --> 00:32:04,210 ‫The beauty of this is you don't have to serve as share your keys and certificates with the load balancer 522 00:32:04,210 --> 00:32:11,260 ‫because the load balancer will have the keys right here now know that this back end server will have 523 00:32:11,260 --> 00:32:17,170 ‫the certificate and the certificate will be stored as part of the handshake, the TLS handshake just 524 00:32:17,170 --> 00:32:19,120 ‫with the back and yes, this one. 525 00:32:20,020 --> 00:32:23,880 ‫So it's almost like this acts like a man in the middle that doesn't know anything. 526 00:32:23,890 --> 00:32:27,670 ‫It's act like a pure proxy, transparent proxy. 527 00:32:27,670 --> 00:32:28,780 ‫That's another word for it. 528 00:32:28,810 --> 00:32:39,010 ‫Trans parent, proxy and your ISP is basically a transparent proxy because all packets go through it. 529 00:32:39,430 --> 00:32:39,700 ‫Right. 530 00:32:39,700 --> 00:32:40,540 ‫And that's that's it. 531 00:32:40,900 --> 00:32:48,280 ‫So this app all connections, you know, there is no connection sharing or the backend. 532 00:32:48,280 --> 00:32:50,450 ‫So it's almost like dedicated being angstrom. 533 00:32:50,470 --> 00:32:57,020 ‫Technically, it's more resource heavy and the backend, unless you're using a SH2, if you're using 534 00:32:57,020 --> 00:33:00,760 ‫it still, then you you're going to make fewer TCP connection from the client. 535 00:33:00,760 --> 00:33:01,600 ‫So that's good. 536 00:33:02,020 --> 00:33:09,280 ‫But yeah, especially on the web, if you here on the web and using a browser, right, the browser 537 00:33:09,280 --> 00:33:16,960 ‫does very freaky things with that when it comes to HDTV one and establish like six to 10 TCP connections 538 00:33:16,990 --> 00:33:18,160 ‫per domain. 539 00:33:18,430 --> 00:33:23,020 ‫And it just floods the server with a lot of connection, and we want to minimize this as much as possible. 540 00:33:23,020 --> 00:33:25,150 ‫So it is very good choice here. 541 00:33:25,330 --> 00:33:29,460 ‫I don't mention it because it's still experimental and I don't know much about it. 542 00:33:29,470 --> 00:33:34,630 ‫And I know it's three has the backbone to use TCP and instead of UDP. 543 00:33:35,470 --> 00:33:37,180 ‫So it's still still. 544 00:33:37,180 --> 00:33:43,930 ‫I need to read more about that, but I know UDP is being blocked on the firewalls and and a lot of stuff 545 00:33:43,930 --> 00:33:44,770 ‫that like them. 546 00:33:45,580 --> 00:33:46,090 ‫All right, guys. 547 00:33:46,090 --> 00:33:47,230 ‫I think we're going to stop here. 548 00:33:47,980 --> 00:33:54,640 ‫So we talked about just one feature and this is a 12 system design and we focused on the back and most 549 00:33:54,640 --> 00:33:55,180 ‫of the time. 550 00:33:55,180 --> 00:33:58,960 ‫Yeah, and that's what I like doing them because that's that's my my specialty. 551 00:33:59,440 --> 00:34:00,880 ‫I tor then go to specialty. 552 00:34:00,880 --> 00:34:01,630 ‫That's what I know. 553 00:34:01,990 --> 00:34:08,500 ‫I'm going to continue with part two where we discuss the other stuff here, where the follow to people 554 00:34:08,500 --> 00:34:14,800 ‫and home timeline and that might take us into another route and completely another game really might 555 00:34:14,800 --> 00:34:15,070 ‫change. 556 00:34:15,070 --> 00:34:15,790 ‫The design might. 557 00:34:16,720 --> 00:34:22,600 ‫That's why I'm I'm really surprised if someone does it with a system design and 20 minutes or 10 minutes, 558 00:34:22,600 --> 00:34:27,940 ‫you cannot unless you have sold this through for hours and hours and hours and days, right? 559 00:34:29,110 --> 00:34:35,290 ‫And then you compiled your perfect design into this compile 20 minutes. 560 00:34:35,740 --> 00:34:41,860 ‫Yeah, a little work, but I think I get confused when I watch videos like that because it's like, 561 00:34:41,860 --> 00:34:44,380 ‫Wow, how did you get into that decision? 562 00:34:44,410 --> 00:34:45,400 ‫How about this? 563 00:34:45,610 --> 00:34:46,830 ‫I have a lot of questions, right? 564 00:34:47,230 --> 00:34:51,550 ‫But you cannot you lose the history of what that was? 565 00:34:51,550 --> 00:34:53,290 ‫The designer did it. 566 00:34:53,380 --> 00:34:55,600 ‫You just you are given a design. 567 00:34:55,810 --> 00:34:58,150 ‫That's why I enjoy making things from scratch. 568 00:34:58,390 --> 00:35:04,330 ‫I enjoy the mistakes that I make because even this design decision that I make, they are not perfect 569 00:35:04,330 --> 00:35:09,870 ‫and we know we didn't even go through that database and how it is a scale sharding and all that stuff. 570 00:35:09,880 --> 00:35:17,200 ‫I don't like to do sharding from day one, to be honest, but if I anticipated a lot of load, then 571 00:35:17,200 --> 00:35:18,010 ‫I'm I might. 572 00:35:18,160 --> 00:35:18,790 ‫Who knows, right? 573 00:35:19,210 --> 00:35:23,470 ‫And just being pragmatic is is a good thing. 574 00:35:23,470 --> 00:35:28,570 ‫And there is a book called The Program a pragmatic programmer and what it means. 575 00:35:28,960 --> 00:35:40,120 ‫The gist of things is just be pragmatic means you have to follow the simple design and not chase shiny 576 00:35:40,120 --> 00:35:41,350 ‫things all the time. 577 00:35:41,530 --> 00:35:43,240 ‫Always think so. 578 00:35:43,240 --> 00:35:44,110 ‫It all it does. 579 00:35:44,110 --> 00:35:45,070 ‫Is this worth it? 580 00:35:45,250 --> 00:35:46,240 ‫Is this thing worth it? 581 00:35:46,240 --> 00:35:47,650 ‫Should I choose this? 582 00:35:47,950 --> 00:35:51,190 ‫Just because the technology is popular doesn't mean you have to use it right? 583 00:35:52,570 --> 00:35:56,020 ‫Use the basic stuff because the basic and simple stuff works. 584 00:35:56,200 --> 00:35:57,970 ‫Look at what's up are building. 585 00:35:58,210 --> 00:36:06,010 ‫They use the SMB protocol, I believe a custom version of it which uses SML, which is what is hated 586 00:36:06,010 --> 00:36:06,490 ‫these days. 587 00:36:06,490 --> 00:36:13,210 ‫People hate Excel flat out right because it's it is a text based protocol and it's heavy. 588 00:36:13,210 --> 00:36:15,610 ‫And it said we the list goes on. 589 00:36:17,440 --> 00:36:22,690 ‫But the app WhatsApp users and look at that billions and billions of users are using, I think, two 590 00:36:22,690 --> 00:36:26,890 ‫billion users now this this service has like how does it work? 591 00:36:27,550 --> 00:36:33,070 ‫Those guys squeeze every single performance of a sync, every single TCP connection that they have. 592 00:36:33,580 --> 00:36:34,330 ‫That's awesome. 593 00:36:34,990 --> 00:36:36,160 ‫And we want to think like that. 594 00:36:36,160 --> 00:36:37,390 ‫Guys be pragmatic. 595 00:36:37,750 --> 00:36:39,580 ‫Don't chase fancy stuff. 596 00:36:39,840 --> 00:36:40,210 ‫All right. 597 00:36:41,000 --> 00:36:45,520 ‫Understand the fundamentals because everything we discuss here are pure fundamentals. 598 00:36:45,790 --> 00:36:49,480 ‫I didn't throw, yeah, I did throw Kafka, but Kafka is a message. 599 00:36:49,480 --> 00:36:50,470 ‫Cura a message. 600 00:36:50,610 --> 00:36:56,040 ‫Rocker, think about it, and that's the basic fundamentals that we need to understand asynchronous 601 00:36:56,040 --> 00:36:59,610 ‫synchronous database, server, web server protocols. 602 00:36:59,910 --> 00:37:01,320 ‫That's what we need to learn. 603 00:37:01,870 --> 00:37:03,150 ‫OK, guys, that's it for me today. 604 00:37:04,170 --> 00:37:05,160 ‫Sorry for the long video. 605 00:37:05,580 --> 00:37:07,860 ‫I'm going to see in the next one, I'm going to see in the next one. 606 00:37:08,070 --> 00:37:08,940 ‫You guys stay awesome. 607 00:37:09,190 --> 00:37:09,510 ‫Goodbye. 608 00:37:13,640 --> 00:37:21,800 ‫What is going on, guys, my name is Hussein, and welcome to another system design VIDEO So this is 609 00:37:21,800 --> 00:37:23,030 ‫part two of. 610 00:37:24,060 --> 00:37:29,160 ‫I don't know how many parts what we're going to discuss, the Twitter system design, we have done part 611 00:37:29,160 --> 00:37:32,040 ‫one and we only discussed one feature. 612 00:37:32,910 --> 00:37:40,110 ‫This tells you how complex system design is right and my staff guys, if you don't know when it comes 613 00:37:40,110 --> 00:37:41,640 ‫to system design. 614 00:37:42,660 --> 00:37:51,270 ‫I like to go into a system design without without preparing anything, because that's what real life 615 00:37:51,270 --> 00:37:51,840 ‫is about. 616 00:37:52,590 --> 00:37:58,350 ‫The other videos on YouTube suffer from over preparation. 617 00:37:59,900 --> 00:38:04,130 ‫And here's what the problem with our preparation is, because if I. 618 00:38:05,260 --> 00:38:10,630 ‫Went off and spent days designing the best water design I can come up. 619 00:38:10,900 --> 00:38:12,370 ‫And I made a video about it. 620 00:38:12,640 --> 00:38:16,940 ‫You'll gain zero benefit, literally zero benefits. 621 00:38:16,960 --> 00:38:17,440 ‫Why? 622 00:38:18,010 --> 00:38:23,320 ‫Because you didn't know what I went through to actually come up with that design. 623 00:38:23,860 --> 00:38:27,250 ‫Even that that design might not be perfect. 624 00:38:27,490 --> 00:38:34,660 ‫So what I like to do if I'm going to do a system design VIDEO What I do is I start from scratch and 625 00:38:34,810 --> 00:38:36,790 ‫my brain does not. 626 00:38:37,030 --> 00:38:38,380 ‫I didn't prepare anything. 627 00:38:38,390 --> 00:38:48,610 ‫I literally come in blank and you see me stumble upon the problems and and go from one design to another 628 00:38:48,610 --> 00:38:50,320 ‫until I find what works. 629 00:38:50,590 --> 00:38:57,430 ‫And even though as you watch me do this and as I discussed what it's always like a discussion with you 630 00:38:57,430 --> 00:39:00,490 ‫guys when I do I, it will be a common discussion. 631 00:39:00,490 --> 00:39:01,150 ‫You see me. 632 00:39:01,150 --> 00:39:02,110 ‫I'll get Oh, was it? 633 00:39:02,110 --> 00:39:03,040 ‫Why did you do this? 634 00:39:03,040 --> 00:39:03,760 ‫Do this better? 635 00:39:03,760 --> 00:39:04,720 ‫I think this is better. 636 00:39:04,990 --> 00:39:09,760 ‫You will come up with even better design as you see me stumble upon this stuff. 637 00:39:10,060 --> 00:39:11,650 ‫Would I say that's my opinion? 638 00:39:11,650 --> 00:39:12,100 ‫I you. 639 00:39:12,310 --> 00:39:17,720 ‫You can agree or disagree as you like that, but that's what I prefer. 640 00:39:17,740 --> 00:39:22,240 ‫If you're new, especially to back an engineer for any system design, do it yourself. 641 00:39:22,480 --> 00:39:28,320 ‫Don't watch videos that have, like, laid down, Oh, this is the best way to do things. 642 00:39:28,480 --> 00:39:28,960 ‫And that's it. 643 00:39:29,320 --> 00:39:30,400 ‫No, there's no way. 644 00:39:30,550 --> 00:39:31,870 ‫There is no best design. 645 00:39:32,140 --> 00:39:33,460 ‫There is no final. 646 00:39:33,490 --> 00:39:34,210 ‫There's no perfect. 647 00:39:34,210 --> 00:39:39,630 ‫Because if that's what if there is a perfect design, we don't have jobs anymore, guys, because all 648 00:39:39,640 --> 00:39:43,470 ‫right, this guy has had just made a video that is a perfect system design. 649 00:39:43,480 --> 00:39:43,840 ‫Nope. 650 00:39:44,590 --> 00:39:44,950 ‫Nope. 651 00:39:45,130 --> 00:39:45,880 ‫Doesn't exist. 652 00:39:46,210 --> 00:39:52,840 ‫It does not exist because someone will always make a better design that design for certain use cases. 653 00:39:53,650 --> 00:39:56,440 ‫And might it, it really depends. 654 00:39:56,440 --> 00:39:58,600 ‫What are you trying to do? 655 00:39:58,990 --> 00:40:04,210 ‫And this is unlimited, and this is where creativity comes into the picture. 656 00:40:04,240 --> 00:40:07,600 ‫You are an artist, guys. 657 00:40:07,870 --> 00:40:13,810 ‫You are an artist and and you're you're you're you're building a system. 658 00:40:14,140 --> 00:40:15,340 ‫It's like an architecture. 659 00:40:15,640 --> 00:40:17,380 ‫You're building a system of architecture, right? 660 00:40:18,420 --> 00:40:19,230 ‫It's tough. 661 00:40:20,210 --> 00:40:20,840 ‫It's tough. 662 00:40:22,140 --> 00:40:26,280 ‫And learn to enjoy it with that intro out of the way. 663 00:40:26,310 --> 00:40:27,870 ‫How about we get it started? 664 00:40:27,870 --> 00:40:35,010 ‫Guys and guys are going to start seeing time chapter as I try my best to to separate this into time 665 00:40:35,010 --> 00:40:35,310 ‫chapter. 666 00:40:35,310 --> 00:40:36,840 ‫Sometimes it works, sometimes it doesn't. 667 00:40:37,740 --> 00:40:38,070 ‫All right. 668 00:40:38,070 --> 00:40:42,420 ‫So in the previous video, we did the post tweet and we discussed it. 669 00:40:42,420 --> 00:40:43,560 ‫Database design. 670 00:40:43,560 --> 00:40:43,980 ‫We did. 671 00:40:44,430 --> 00:40:47,220 ‫We didn't know what dancing we did proxy. 672 00:40:47,220 --> 00:40:48,300 ‫We did message queuing. 673 00:40:48,300 --> 00:40:48,600 ‫We did. 674 00:40:48,750 --> 00:40:50,640 ‫We were thrown so many ideas. 675 00:40:50,640 --> 00:40:51,450 ‫Some of them work. 676 00:40:51,450 --> 00:40:52,470 ‫Some of them didn't. 677 00:40:52,830 --> 00:40:53,100 ‫Right. 678 00:40:53,100 --> 00:40:54,840 ‫But well, that's the part of the fun. 679 00:40:55,590 --> 00:40:58,620 ‫How long do you think system design takes in real world? 680 00:40:59,250 --> 00:41:00,600 ‫It takes us weeks. 681 00:41:00,600 --> 00:41:03,920 ‫If it's all months to do a good system design, right? 682 00:41:04,020 --> 00:41:06,570 ‫Things doesn't just magically pop up, right? 683 00:41:07,650 --> 00:41:08,010 ‫All right. 684 00:41:09,710 --> 00:41:11,870 ‫Today on discuss, follow people. 685 00:41:13,530 --> 00:41:15,010 ‫This Poppy, these puppy. 686 00:41:16,990 --> 00:41:22,390 ‫So now I'm going to start, as I always do, I always start from the bottom up. 687 00:41:22,440 --> 00:41:23,770 ‫That's the database, right? 688 00:41:24,130 --> 00:41:25,020 ‫So let's let's talk. 689 00:41:25,030 --> 00:41:25,810 ‫Think about this. 690 00:41:28,140 --> 00:41:37,050 ‫Implement implying that I follow people means that there is some sort of a profile to follow. 691 00:41:37,080 --> 00:41:37,410 ‫Right. 692 00:41:37,770 --> 00:41:44,190 ‫So I'm imagining some sort of a table here that will have certain columns, right? 693 00:41:45,400 --> 00:41:47,140 ‫They will love the idea. 694 00:41:48,180 --> 00:41:50,190 ‫We will have your name. 695 00:41:52,310 --> 00:41:54,170 ‫It will have what what other stuff? 696 00:41:54,230 --> 00:41:55,100 ‫Picture. 697 00:41:57,460 --> 00:41:59,650 ‫Ride, mile ride. 698 00:42:02,230 --> 00:42:02,630 ‫That's it. 699 00:42:02,650 --> 00:42:04,450 ‫This is the database design now. 700 00:42:06,350 --> 00:42:07,580 ‫That's what I'm thinking here. 701 00:42:11,380 --> 00:42:17,110 ‫And we can add more schema as we go again, and I'm going with the relational because I'm old school, 702 00:42:17,500 --> 00:42:22,090 ‫feel free to to do it and no sequel, there is no right or wrong way. 703 00:42:22,330 --> 00:42:27,100 ‫There are benefits and and advantages and disadvantages when it comes to everything, right? 704 00:42:27,100 --> 00:42:30,490 ‫I like to have acid transaction all the time. 705 00:42:30,490 --> 00:42:35,080 ‫I know Mango and other NoSQL databases supports acid, but doesn't matter. 706 00:42:35,110 --> 00:42:36,790 ‫No, we're just building the schema. 707 00:42:36,790 --> 00:42:38,890 ‫I like to have explicit schema. 708 00:42:39,220 --> 00:42:41,680 ‫This is just how I how I at all personally. 709 00:42:41,890 --> 00:42:44,400 ‫Don't let it influence you, right? 710 00:42:45,040 --> 00:42:45,910 ‫Anything you want. 711 00:42:45,970 --> 00:42:47,040 ‫This could be PostgreSQL. 712 00:42:47,080 --> 00:42:48,580 ‫Misquote me, MongoDB. 713 00:42:49,150 --> 00:42:50,020 ‫Me cockroach db. 714 00:42:50,230 --> 00:42:50,710 ‫Who cares? 715 00:42:51,940 --> 00:42:56,180 ‫So now this is my my profile table. 716 00:42:56,200 --> 00:42:57,540 ‫It's called profile table. 717 00:42:58,480 --> 00:43:01,900 ‫Proofs, file profile. 718 00:43:02,050 --> 00:43:02,380 ‫Right. 719 00:43:03,430 --> 00:43:06,370 ‫And now how do I? 720 00:43:08,310 --> 00:43:17,550 ‫Kind of build a structure that tells me that this this guy is following this guy as popular as some 721 00:43:17,550 --> 00:43:19,230 ‫ideas, I.D. number one. 722 00:43:20,250 --> 00:43:20,970 ‫Hussein. 723 00:43:22,810 --> 00:43:23,950 ‫I.D. number two. 724 00:43:25,760 --> 00:43:27,020 ‫And I don't know. 725 00:43:28,140 --> 00:43:28,770 ‫Mary. 726 00:43:32,120 --> 00:43:40,130 ‫OK, so I want to add a link that this person is following mainly, for example. 727 00:43:40,790 --> 00:43:42,600 ‫What is the first thing that comes to your mind? 728 00:43:42,620 --> 00:43:43,040 ‫I don't know. 729 00:43:43,070 --> 00:43:44,180 ‫I'm just throwing ideas. 730 00:43:44,420 --> 00:43:48,110 ‫While should we add another column here that says OK now? 731 00:43:48,560 --> 00:43:49,610 ‫Followed by. 732 00:43:50,740 --> 00:43:55,510 ‫That won't work immediately, because followed, this is a column and what are you going to be going 733 00:43:55,510 --> 00:43:56,680 ‫to do it as an integer? 734 00:43:57,370 --> 00:44:01,270 ‫Then what if you want to follow multiple people that breaks up real quick? 735 00:44:01,370 --> 00:44:01,510 ‫I. 736 00:44:02,880 --> 00:44:07,170 ‫You might say, as I am, I'm going to use Mongo and I'm going to add a field that is at best. 737 00:44:08,290 --> 00:44:08,920 ‫Be my guest. 738 00:44:08,950 --> 00:44:12,280 ‫You can do that, but things get really complex, real fast. 739 00:44:12,500 --> 00:44:13,780 ‫When you when you do that. 740 00:44:15,850 --> 00:44:21,070 ‫I'm not sure about the performance of having a list inside a column, I have never done that before, 741 00:44:21,070 --> 00:44:22,630 ‫but do you consider it? 742 00:44:22,720 --> 00:44:23,650 ‫Consider everything. 743 00:44:24,040 --> 00:44:27,190 ‫This is again, this is you're having fun, guys. 744 00:44:27,580 --> 00:44:29,710 ‫This is not stressful task. 745 00:44:30,160 --> 00:44:31,420 ‫Don't beat yourself up. 746 00:44:32,430 --> 00:44:39,210 ‫Even someone with twenty five years of experience make mistakes, and I work with people with 30 and 747 00:44:39,210 --> 00:44:47,340 ‫40 years of experience, and sometimes they come up with an idea and someone with a junior come up with 748 00:44:47,340 --> 00:44:48,150 ‫a better idea. 749 00:44:48,600 --> 00:44:50,550 ‫I see it all the time. 750 00:44:51,720 --> 00:44:52,980 ‫All the time and it happens. 751 00:44:53,130 --> 00:45:00,630 ‫So don't don't don't stress yourself up is just have fun with this camp and obviously not everything 752 00:45:00,630 --> 00:45:02,670 ‫that might work right. 753 00:45:02,670 --> 00:45:04,410 ‫Sometimes work sometimes is not right. 754 00:45:05,730 --> 00:45:10,470 ‫So, yeah, so what I'm going to do here instead, I'm having another table. 755 00:45:10,600 --> 00:45:13,200 ‫OK, this is old school stuff. 756 00:45:13,200 --> 00:45:14,670 ‫We have done this so many times. 757 00:45:14,670 --> 00:45:17,130 ‫I'm going to do call this the following table. 758 00:45:19,220 --> 00:45:20,660 ‫And this table will be. 759 00:45:21,670 --> 00:45:25,390 ‫I'm expecting the stable to grow very large. 760 00:45:25,750 --> 00:45:27,270 ‫Let's talk about this, so I'd. 761 00:45:27,920 --> 00:45:28,610 ‫Let's call this. 762 00:45:31,730 --> 00:45:32,680 ‫Source Oiti. 763 00:45:33,300 --> 00:45:35,840 ‫I don't know, destination I.D. did. 764 00:45:37,320 --> 00:45:39,300 ‫That means this guy is following this guy. 765 00:45:39,360 --> 00:45:40,170 ‫That's what it means. 766 00:45:40,930 --> 00:45:42,700 ‫Well, this guy is following this gal. 767 00:45:43,320 --> 00:45:43,950 ‫That's what I mean. 768 00:45:44,280 --> 00:45:52,890 ‫So if I had a record that says one two, that implies that Hussein is following Mary. 769 00:45:53,070 --> 00:45:57,960 ‫If you want to add another record saying to follows one. 770 00:45:59,610 --> 00:46:01,600 ‫Then Mary is following the same, right? 771 00:46:01,650 --> 00:46:03,240 ‫So that's that's the idea here. 772 00:46:03,570 --> 00:46:11,430 ‫And you can see that this stable is going to grow extremely, extremely large. 773 00:46:12,870 --> 00:46:16,820 ‫Imagine imagine this is Taylor Swift. 774 00:46:19,020 --> 00:46:26,220 ‫So you going to have the source I.D. there's going to be duplicated all over here. 775 00:46:27,340 --> 00:46:30,070 ‫And the destination, it is going to have different numbers. 776 00:46:31,120 --> 00:46:37,630 ‫So this tells me immediately I need a multi multi index, right, multicolored index. 777 00:46:38,050 --> 00:46:45,340 ‫So I need an index that that basically have both in in a B three maybe, right? 778 00:46:47,100 --> 00:46:55,050 ‫So that I can I can search this structure much more or form because this will grow very large easily, 779 00:46:55,830 --> 00:46:56,050 ‫right? 780 00:46:56,730 --> 00:46:59,910 ‫This is a big, huge table, but it depends, guys. 781 00:46:59,910 --> 00:47:01,230 ‫What are we trying to do? 782 00:47:01,230 --> 00:47:02,430 ‫Let's think about this, right? 783 00:47:02,580 --> 00:47:03,730 ‫So yeah, I have no. 784 00:47:04,080 --> 00:47:09,510 ‫The ability to follow people, what do I do in order to follow someone? 785 00:47:10,580 --> 00:47:13,010 ‫I write I start a transaction. 786 00:47:13,580 --> 00:47:23,870 ‫I write my own I.D. into the source ID and I write the destination of whom I want to follow and the 787 00:47:23,870 --> 00:47:24,350 ‫destination. 788 00:47:24,670 --> 00:47:32,000 ‫So just like that, the post request, if you're going to build it, application is extremely lightweight 789 00:47:32,360 --> 00:47:36,380 ‫because it's going to contain maybe a JSON with the source and destination. 790 00:47:36,560 --> 00:47:37,910 ‫It's extremely lightweight. 791 00:47:37,910 --> 00:47:41,870 ‫So I like I like the lightweight aspect of this. 792 00:47:43,100 --> 00:47:45,900 ‫Right, so it's very light what I can and quickly follow this stuff. 793 00:47:47,050 --> 00:47:50,020 ‫And I can ride to this side transaction, right? 794 00:47:50,350 --> 00:47:55,480 ‫And then in transaction and commit and immediately, I follow this stuff. 795 00:47:56,620 --> 00:47:57,670 ‫Now this is the first time. 796 00:47:58,860 --> 00:48:00,510 ‫The second question is. 797 00:48:01,570 --> 00:48:02,040 ‫All right. 798 00:48:02,220 --> 00:48:03,240 ‫The first question is. 799 00:48:04,500 --> 00:48:05,490 ‫Follow people, right? 800 00:48:06,720 --> 00:48:07,290 ‫Which we did. 801 00:48:10,300 --> 00:48:12,190 ‫Let's see if our I have problems, I don't know. 802 00:48:12,340 --> 00:48:13,450 ‫We'll go through that. 803 00:48:15,130 --> 00:48:20,770 ‫And that's what I like, I like to see that we're going to have a problem otherwise. 804 00:48:20,950 --> 00:48:29,830 ‫If I eliminated a very bad design and I give you what I think is good, then then this this misses the 805 00:48:29,830 --> 00:48:30,520 ‫point, right? 806 00:48:30,820 --> 00:48:36,880 ‫This is again, this is this is not the final design by any stretch of the imagination. 807 00:48:37,420 --> 00:48:42,010 ‫This is why why you, you you need to train yourself to design. 808 00:48:42,250 --> 00:48:44,350 ‫You don't need to train yourself to memorize things. 809 00:48:44,350 --> 00:48:45,940 ‫Well, not memorizing anything, guys. 810 00:48:45,940 --> 00:48:47,620 ‫Here, we're having fun. 811 00:48:47,980 --> 00:48:51,490 ‫You're going to do it to relearn how to system design. 812 00:48:51,500 --> 00:48:54,700 ‫You just have to do it and then scratch it and do it again. 813 00:48:54,700 --> 00:48:55,600 ‫Scratch and do it again. 814 00:48:55,630 --> 00:48:57,220 ‫I have no idea how this will end up. 815 00:48:57,640 --> 00:48:58,480 ‫God knows. 816 00:48:59,020 --> 00:49:00,690 ‫So let's go follow people. 817 00:49:00,700 --> 00:49:01,600 ‫Will this work? 818 00:49:02,790 --> 00:49:03,900 ‫I think it's OK. 819 00:49:04,230 --> 00:49:08,880 ‫I think it will work even if this stable is two three. 820 00:49:09,960 --> 00:49:14,440 ‫Four billion rows inserting is always fast. 821 00:49:14,460 --> 00:49:17,570 ‫I know that for a fact, right? 822 00:49:17,910 --> 00:49:21,600 ‫Inserting into a table is always fast and this. 823 00:49:22,940 --> 00:49:25,490 ‫I don't think it's going to be a busy table. 824 00:49:26,210 --> 00:49:29,360 ‫Let's think about it how often people are following orders. 825 00:49:30,910 --> 00:49:32,590 ‫It's not happening every second, right? 826 00:49:32,740 --> 00:49:38,470 ‫It's like, yeah, you click, I know people are reading the information far more than they're writing, 827 00:49:38,470 --> 00:49:42,520 ‫so I'm fine by using a B-plus tree here. 828 00:49:43,390 --> 00:49:43,900 ‫Yeah. 829 00:49:44,830 --> 00:49:53,290 ‫It's going to it might hurt my my SSD in the long run because the three need to be rebalanced as I insert 830 00:49:53,290 --> 00:49:55,360 ‫more stuff in it, but it's not that bad. 831 00:49:55,930 --> 00:50:02,320 ‫It's not like I'm logging if if this is a log that that is being executed, like, I don't know, 100 832 00:50:02,320 --> 00:50:10,870 ‫million, that's a law 100 million, 100000 times a second, then I'm going to be worried about writing 833 00:50:10,870 --> 00:50:11,480 ‫performance. 834 00:50:11,500 --> 00:50:14,980 ‫I'm not worried about write performance here, so I'm fine using a battery. 835 00:50:15,190 --> 00:50:16,530 ‫That means all. 836 00:50:18,130 --> 00:50:24,310 ‫This also influenced the database choice, right, because if abusing bigotry I'm using, I'm fine with 837 00:50:24,340 --> 00:50:28,120 ‫using post because I don't want to go with my sequel, right? 838 00:50:28,780 --> 00:50:32,170 ‫There is, there is method the advantage of using all these features, right? 839 00:50:32,560 --> 00:50:36,580 ‫My school had the ability to to swizzle different database engine. 840 00:50:36,820 --> 00:50:38,340 ‫The problem is complexity. 841 00:50:38,890 --> 00:50:43,570 ‫Because once you have different database engine between two tables, you cannot query against them as 842 00:50:43,570 --> 00:50:44,590 ‫a transactionally. 843 00:50:44,980 --> 00:50:45,820 ‫That's a problem. 844 00:50:46,750 --> 00:50:48,350 ‫But let's stick with this. 845 00:50:48,700 --> 00:50:55,420 ‫A simplified insert, I'm fine with this, so follow people as basically the ability to insert a row 846 00:50:55,420 --> 00:50:56,230 ‫in this table. 847 00:50:57,330 --> 00:51:01,560 ‫Use you what that Instagram or Twitter they do not allow. 848 00:51:01,810 --> 00:51:11,290 ‫They don't not allow you to do this mass following of just because, first of all, spam second is performance 849 00:51:11,560 --> 00:51:13,330 ‫they don't want you to talk to. 850 00:51:13,600 --> 00:51:16,360 ‫They don't want to write multiple rows here. 851 00:51:16,600 --> 00:51:22,750 ‫So if they want to make their arrests, endpoint or their API as light as possible, because if you 852 00:51:22,750 --> 00:51:31,840 ‫have mass following, then you have to kind of build a huge Jason object or group object with a lot 853 00:51:31,840 --> 00:51:35,110 ‫of rows in order to just slam them all in. 854 00:51:35,980 --> 00:51:37,570 ‫This is just to avoid that. 855 00:51:37,840 --> 00:51:40,030 ‫Don't add this feature to do a mass follower. 856 00:51:40,200 --> 00:51:40,480 ‫Right? 857 00:51:40,660 --> 00:51:42,130 ‫Just follow one by one. 858 00:51:42,140 --> 00:51:43,130 ‫It's painful. 859 00:51:43,150 --> 00:51:44,160 ‫I know Instagram. 860 00:51:44,230 --> 00:51:45,520 ‫They make it painful on you. 861 00:51:45,730 --> 00:51:46,000 ‫Right? 862 00:51:46,360 --> 00:51:53,320 ‫But they also know that people don't usually follow people on a daily or on on a on a second per second 863 00:51:53,320 --> 00:51:53,710 ‫basis. 864 00:51:53,710 --> 00:51:54,910 ‫So we're fine with this design. 865 00:51:55,360 --> 00:51:56,440 ‫Second feature. 866 00:51:57,590 --> 00:51:57,910 ‫Right. 867 00:51:58,760 --> 00:52:01,610 ‫The second feature is let's do the list for our source. 868 00:52:03,780 --> 00:52:05,430 ‫Because I think this will do followers right. 869 00:52:06,950 --> 00:52:10,430 ‫I want to know my followers or any profile followers. 870 00:52:11,060 --> 00:52:13,400 ‫So as an input, you have what? 871 00:52:13,400 --> 00:52:14,030 ‫You have an ID. 872 00:52:14,420 --> 00:52:14,750 ‫Right? 873 00:52:15,760 --> 00:52:22,420 ‫Given an IED, a profile, I want to know their followers, first of all, let's do the count. 874 00:52:22,840 --> 00:52:24,640 ‫How do I get that count of them? 875 00:52:24,940 --> 00:52:27,550 ‫And then we're going to do the actual list of followers. 876 00:52:27,580 --> 00:52:29,440 ‫The count is always easier than that. 877 00:52:30,190 --> 00:52:33,850 ‫And again, I'm not talking about cash in cash, who will make this even better. 878 00:52:34,180 --> 00:52:36,280 ‫But let's just think about the basics. 879 00:52:36,280 --> 00:52:40,360 ‫Guys don't introduce complexity unless you absolutely need it, right? 880 00:52:40,570 --> 00:52:52,420 ‫Try to optimize the the the raw level fundamentals and then try to build this caching layers and other 881 00:52:52,420 --> 00:52:53,040 ‫stuff on top. 882 00:52:53,050 --> 00:52:54,760 ‫Don't don't over. 883 00:52:55,790 --> 00:52:58,580 ‫Complicate things prematurely. 884 00:52:58,790 --> 00:52:59,030 ‫Right? 885 00:52:59,300 --> 00:53:02,210 ‫So how do I do account, I won't I give it an ID? 886 00:53:02,630 --> 00:53:04,300 ‫I need to know the floor for hours. 887 00:53:04,670 --> 00:53:12,640 ‫If I do a query where ID is equal, the source is equal to one count of the ID, the destination I.D. 888 00:53:13,400 --> 00:53:18,380 ‫And that will give me all the count of how many people I am following. 889 00:53:18,440 --> 00:53:19,150 ‫Or what? 890 00:53:19,160 --> 00:53:19,610 ‫How many? 891 00:53:20,090 --> 00:53:21,050 ‫The profile of folks. 892 00:53:21,290 --> 00:53:23,240 ‫So that's that's that's pretty neat. 893 00:53:25,470 --> 00:53:28,560 ‫So you can know how much people are following you. 894 00:53:28,740 --> 00:53:32,040 ‫How do I know how many people are? 895 00:53:33,030 --> 00:53:35,010 ‫You are following it's different, right? 896 00:53:35,290 --> 00:53:36,720 ‫That means this. 897 00:53:37,740 --> 00:53:39,360 ‫You are following this, right? 898 00:53:39,510 --> 00:53:47,160 ‫So if you are following this, what do you need to do is basically you query using the D I.D. If you 899 00:53:47,160 --> 00:53:52,530 ‫want to flip, you want to know your whole whom you are following, then you flip the equation and you 900 00:53:52,530 --> 00:53:52,890 ‫do. 901 00:53:53,100 --> 00:54:00,510 ‫There is I.D. So that's essentially two queries to know your followers or your following, right following 902 00:54:00,900 --> 00:54:03,210 ‫flash followers following. 903 00:54:06,220 --> 00:54:09,010 ‫So this table gives us both essentially, right? 904 00:54:11,440 --> 00:54:12,040 ‫Here's the thing. 905 00:54:13,480 --> 00:54:14,410 ‫We can. 906 00:54:15,670 --> 00:54:17,290 ‫We can play this differently. 907 00:54:18,230 --> 00:54:23,060 ‫Today, I'm proposing we do two queries on this table, right? 908 00:54:23,810 --> 00:54:28,010 ‫So a query where is it equality count or query? 909 00:54:28,020 --> 00:54:29,240 ‫Where did equal? 910 00:54:30,230 --> 00:54:33,410 ‫The I.D. count, and this gives you either one of them. 911 00:54:34,640 --> 00:54:41,390 ‫That's to queries a little bit your additional costs to answer two questions which are valid. 912 00:54:42,020 --> 00:54:42,880 ‫What if? 913 00:54:42,980 --> 00:54:46,670 ‫Can I answer this question with one query, if possible? 914 00:54:48,200 --> 00:54:51,860 ‫And then compile the results locally. 915 00:54:53,420 --> 00:55:04,650 ‫I'm not sure we can, but let's give it a shot if I can compute the this and the client side by and 916 00:55:04,670 --> 00:55:13,760 ‫the client, here I am the back and not that app or the web browser, I can query where its ID equals 917 00:55:13,760 --> 00:55:19,340 ‫the I.D. or did equal the I.D. and this will give me a huge set of rows back. 918 00:55:20,660 --> 00:55:25,010 ‫Man, that's a huge setback, right, because you're either here or here. 919 00:55:25,960 --> 00:55:35,620 ‫And now you can loop through the cursor results, and then if you are if you are in the side, you add 920 00:55:35,620 --> 00:55:38,350 ‫one, if you are in the ID, you add one to the follower. 921 00:55:38,860 --> 00:55:39,160 ‫Right. 922 00:55:39,370 --> 00:55:42,940 ‫This way, you compute the following versus the followers, right? 923 00:55:43,570 --> 00:55:44,350 ‫That's how you do it. 924 00:55:44,560 --> 00:55:45,460 ‫That's another way. 925 00:55:45,550 --> 00:55:51,040 ‫But that's I think that's you're essentially doing a big show of what Owen of MN. 926 00:55:52,260 --> 00:55:53,730 ‫Through the whole results set. 927 00:55:54,760 --> 00:56:01,490 ‫It's not bad, but I honestly don't know which one is faster doing to close and let the database handle 928 00:56:01,490 --> 00:56:10,280 ‫it or pull everything back and do it, look locally in the client side and their client sign in here. 929 00:56:10,300 --> 00:56:18,490 ‫I mean, the the the database client that executing the query, the benefit of this is, yeah, it might 930 00:56:18,490 --> 00:56:19,630 ‫take you a little bit longer. 931 00:56:19,630 --> 00:56:20,290 ‫But guess what? 932 00:56:20,890 --> 00:56:29,590 ‫You now and not just two questions, actually, and solved four questions, because now if you want 933 00:56:29,590 --> 00:56:34,300 ‫to, you can build a local client cached list. 934 00:56:35,630 --> 00:56:39,590 ‫With all the followers list, which answer this question, number three. 935 00:56:41,190 --> 00:56:44,730 ‫List followers, not the count, just actual list the followers. 936 00:56:47,680 --> 00:56:49,360 ‫Followers following. 937 00:56:54,790 --> 00:56:55,900 ‫That's pretty neat, right? 938 00:56:56,050 --> 00:57:00,520 ‫Because now the next thing we're going to do is we're going to keep a cash, maybe that's right this 939 00:57:00,520 --> 00:57:00,880 ‫too. 940 00:57:01,130 --> 00:57:07,900 ‫And since we already bothered the database, let's take that cash information stored somewhere else 941 00:57:08,800 --> 00:57:09,940 ‫because we already took the head. 942 00:57:10,180 --> 00:57:12,400 ‫Someone already asked for this information. 943 00:57:12,610 --> 00:57:20,350 ‫So the next thing they might have to do is actually either drill down and looking for followers, which 944 00:57:20,350 --> 00:57:23,770 ‫is something that I don't know about you, but I don't do this often. 945 00:57:23,830 --> 00:57:26,290 ‫You actually drill down on the followers list all of the following. 946 00:57:26,500 --> 00:57:35,290 ‫So if you want to, if that drills is is is is a frequent operation in your user experience, and that's 947 00:57:35,290 --> 00:57:37,270 ‫a completely different story. 948 00:57:37,630 --> 00:57:43,840 ‫Building user experience, I'm building a system design on the back and building an app is difficult 949 00:57:44,110 --> 00:57:47,590 ‫to build a user experience that how would the app look like? 950 00:57:47,590 --> 00:57:53,410 ‫Some someone would say, Oh, the app should, should, should account and and call to the followers 951 00:57:53,410 --> 00:57:57,940 ‫and the following, and also lists the followers at the same time or in this case. 952 00:57:58,510 --> 00:58:01,390 ‫This approach is better, right? 953 00:58:01,900 --> 00:58:03,760 ‫Because you're going to execute it once. 954 00:58:03,940 --> 00:58:09,670 ‫So you took the head of the database once and you're doing client side processing at the application 955 00:58:09,670 --> 00:58:12,730 ‫level and then you already have this information. 956 00:58:12,940 --> 00:58:17,170 ‫So now you can serve it very easily to the backend. 957 00:58:18,920 --> 00:58:20,270 ‫And here's the problem with this. 958 00:58:20,690 --> 00:58:28,070 ‫The problem with this is if you using a stateless architecture like rest, this will not work. 959 00:58:28,220 --> 00:58:28,970 ‫I believe. 960 00:58:30,620 --> 00:58:31,550 ‫This will not work. 961 00:58:32,480 --> 00:58:32,990 ‫Why? 962 00:58:33,170 --> 00:58:39,260 ‫Because the first request that goes to to to actually query the database and the second request to actually 963 00:58:39,620 --> 00:58:47,170 ‫pull this information from the cache mine, my hit and other service, my head in another process, 964 00:58:47,180 --> 00:58:52,610 ‫another host altogether so that where there is information is not available, you might say, Hussein, 965 00:58:52,610 --> 00:58:54,800 ‫I'm going to throw it in a centralized right server. 966 00:58:54,800 --> 00:58:58,520 ‫But what can you offer complicating your architecture that much? 967 00:58:59,390 --> 00:59:08,300 ‫Can you solve it by storing the state, making your application stateful in the server memory temporary 968 00:59:08,840 --> 00:59:10,430 ‫so that the client? 969 00:59:11,580 --> 00:59:15,930 ‫Can get this information when when they ask for it next. 970 00:59:16,690 --> 00:59:18,820 ‫Thus, rest will won't work for this. 971 00:59:18,840 --> 00:59:26,280 ‫If you want, you can either use Group C, which is a stateful protocol in this case, where you can 972 00:59:26,280 --> 00:59:31,970 ‫push this information and even better use rest with this thing than you think. 973 00:59:31,980 --> 00:59:38,580 ‫Old Volcano I just learned about Work Canel, and they're using fdb to push. 974 00:59:39,980 --> 00:59:41,480 ‫This is pretty neat. 975 00:59:42,140 --> 00:59:44,360 ‫So what are we going to know in this case? 976 00:59:44,690 --> 00:59:47,510 ‫I'm going to ask for the followers count. 977 00:59:48,350 --> 00:59:55,550 ‫This will do the query on both site, retrieve all the information the client and get the count immediately. 978 00:59:55,880 --> 00:59:58,710 ‫Send it back to the user because it's a stateful connection. 979 00:59:58,730 --> 00:59:59,360 ‫This case, right? 980 00:59:59,720 --> 01:00:07,800 ‫So I'm going to respond back using it to try to push immediately as I finish calculating this information. 981 01:00:07,820 --> 01:00:14,630 ‫I'm going to push the list of the followers and I'm going to post the list of the following to their 982 01:00:14,630 --> 01:00:15,260 ‫client. 983 01:00:16,280 --> 01:00:20,270 ‫And this is the actual end user client app. 984 01:00:20,630 --> 01:00:24,890 ‫Android app, iOS app, browser using TV to push. 985 01:00:25,160 --> 01:00:26,270 ‫That's another technology. 986 01:00:27,430 --> 01:00:28,300 ‫That's pretty neat. 987 01:00:29,530 --> 01:00:34,720 ‫Well, hard to get right, though, in my opinion, this is a little difficult to get away. 988 01:00:35,380 --> 01:00:36,430 ‫Make things simple. 989 01:00:37,150 --> 01:00:40,180 ‫Execute four queries one two one two. 990 01:00:41,320 --> 01:00:44,650 ‫And now go, as you might say, Oh, they're saying this is these are just ideas, right? 991 01:00:46,420 --> 01:00:47,140 ‫Which ends. 992 01:00:47,650 --> 01:00:49,330 ‫What if I want to display the name? 993 01:00:49,330 --> 01:00:52,180 ‫All that stuff question. 994 01:00:52,480 --> 01:00:57,880 ‫That's where a schema less architecture works really nice. 995 01:00:57,880 --> 01:01:00,730 ‫Where where you can put actually the name also here. 996 01:01:01,240 --> 01:01:05,320 ‫But that's a cost, because if I put the name here, I need to index it. 997 01:01:05,320 --> 01:01:12,520 ‫And if I indexed it, then if I don't index it, then first of all, if I put two columns here with 998 01:01:12,520 --> 01:01:17,660 ‫the names, which is the most information that we going to put, plus the pictures and the pictures, 999 01:01:17,660 --> 01:01:20,140 ‫we can solve it easily with the URL request. 1000 01:01:20,140 --> 01:01:21,310 ‫That's that's not a big deal. 1001 01:01:21,760 --> 01:01:30,010 ‫But if you put the name, then if you don't index this rose, which is OK, then you when you ask for 1002 01:01:30,010 --> 01:01:35,110 ‫the name, you're going to head the actual row on disk, not the index, right? 1003 01:01:35,110 --> 01:01:42,580 ‫Because that's how roll level access to pull the names, and that's a little bit little bit slightly 1004 01:01:42,580 --> 01:01:43,420 ‫more expensive. 1005 01:01:44,440 --> 01:01:50,560 ‫The second thing is, if the user changed their name, you have to write a transaction to update this, 1006 01:01:50,560 --> 01:01:52,120 ‫which is also not so bad, right? 1007 01:01:52,150 --> 01:01:52,900 ‫You can do this. 1008 01:01:53,320 --> 01:01:54,220 ‫Get that. 1009 01:01:54,280 --> 01:01:59,320 ‫The second solution is not is to also add them, but also index them. 1010 01:01:59,320 --> 01:02:03,880 ‫That will increase your index size that will slow down your rights as well. 1011 01:02:04,950 --> 01:02:07,380 ‫The fourth solution is look at this guy. 1012 01:02:07,680 --> 01:02:11,940 ‫I'm just making up solutions because there are infinite number of ways to solve this problem. 1013 01:02:12,750 --> 01:02:15,630 ‫And this is just me if I am with another team. 1014 01:02:15,930 --> 01:02:19,500 ‫If you're if you're part of a team, that's the best thing you can do. 1015 01:02:19,890 --> 01:02:21,570 ‫Deal with a smart team. 1016 01:02:21,960 --> 01:02:23,640 ‫You can see the flow of idea. 1017 01:02:23,670 --> 01:02:26,070 ‫I love working with my team. 1018 01:02:26,250 --> 01:02:31,350 ‫I work because everyone throwing ideas is, Oh, this will not work, because there's this world because 1019 01:02:31,350 --> 01:02:32,820 ‫of this, oh, this will work because of this. 1020 01:02:33,840 --> 01:02:34,800 ‫I'm just alone here. 1021 01:02:34,800 --> 01:02:35,920 ‫I'm thinking out loud. 1022 01:02:35,940 --> 01:02:36,220 ‫Yeah. 1023 01:02:36,240 --> 01:02:42,030 ‫So if I don't put that in the index, then all I have to do is basically, if I don't add them at all, 1024 01:02:42,360 --> 01:02:49,350 ‫don't have the name, then I have this list and now I need to actually query the profile table and ask 1025 01:02:49,350 --> 01:02:50,010 ‫for the name. 1026 01:02:50,850 --> 01:02:55,110 ‫That is absolutely disastrous, in my opinion, because what are you going to do? 1027 01:02:55,470 --> 01:03:02,730 ‫Select from profile, where name, where ID is in you to an end. 1028 01:03:02,730 --> 01:03:05,880 ‫This list destroyed performance. 1029 01:03:06,840 --> 01:03:08,100 ‫I don't like this one bit. 1030 01:03:10,770 --> 01:03:12,210 ‫Yeah, I don't like it. 1031 01:03:12,270 --> 01:03:12,930 ‫I don't like it. 1032 01:03:12,930 --> 01:03:20,970 ‫So there is a way to actually write this list into another table and then join the profile and that 1033 01:03:20,970 --> 01:03:23,460 ‫log table and then get their names. 1034 01:03:23,460 --> 01:03:27,540 ‫But I don't like it, so I would prefer another method to do this stuff. 1035 01:03:28,050 --> 01:03:28,350 ‫Fine. 1036 01:03:29,040 --> 01:03:34,380 ‫Maybe since the name is information that we can capture this information somehow. 1037 01:03:36,270 --> 01:03:36,750 ‫But. 1038 01:03:38,020 --> 01:03:43,780 ‫I try to avoid as you you know it, as I try to avoid clashes as much as possible because they are hard. 1039 01:03:44,230 --> 01:03:45,940 ‫It's hard to manage the cash. 1040 01:03:46,210 --> 01:03:49,130 ‫So anything that is difficult and hard to maintain. 1041 01:03:49,150 --> 01:03:52,630 ‫I try to avoid it and see if there's a simpler way to solve the problem. 1042 01:03:53,410 --> 01:04:00,280 ‫But yeah, man, it looks like our best guess is to add that the names here, right? 1043 01:04:00,310 --> 01:04:01,750 ‫Or even worse. 1044 01:04:02,320 --> 01:04:04,310 ‫OK, here's what we're going to do. 1045 01:04:05,230 --> 01:04:06,310 ‫Another way we can do it. 1046 01:04:06,910 --> 01:04:11,470 ‫We're not going to obviously follow following the following we're not going to push. 1047 01:04:12,390 --> 01:04:18,750 ‫Every single this is four million rows, we're not going to push four million followers to the client 1048 01:04:18,750 --> 01:04:19,260 ‫throat. 1049 01:04:19,440 --> 01:04:20,460 ‫That's just bad, right? 1050 01:04:20,460 --> 01:04:22,600 ‫That's just bad idea, guys, guys. 1051 01:04:22,620 --> 01:04:27,240 ‫And I want to just kind of fuel you to do this your own right. 1052 01:04:27,750 --> 01:04:32,620 ‫Watch the video to get ideas, but to see the process? 1053 01:04:32,700 --> 01:04:33,270 ‫That's it. 1054 01:04:33,570 --> 01:04:34,620 ‫So this is an idea. 1055 01:04:36,170 --> 01:04:41,210 ‫As we start compiling this list, which is our first results, right, first solution. 1056 01:04:42,720 --> 01:04:44,640 ‫I will only push 10. 1057 01:04:45,560 --> 01:04:46,520 ‫What is not that bad? 1058 01:04:46,670 --> 01:04:53,630 ‫So I can query the profile table for for the list of 10 10 items, not that bad, so give me the names 1059 01:04:53,870 --> 01:04:56,030 ‫and even give me the picture right? 1060 01:04:56,240 --> 01:04:57,540 ‫And then and that's it. 1061 01:04:57,560 --> 01:04:59,870 ‫Don't don't ask for more information than you need, right? 1062 01:04:59,900 --> 01:05:00,470 ‫What what? 1063 01:05:00,620 --> 01:05:01,610 ‫What does the user experience? 1064 01:05:01,610 --> 01:05:06,950 ‫The display the name just shove the name back and then the followers list will be building OK. 1065 01:05:07,010 --> 01:05:11,160 ‫We'll begin to build the followers that that that the Dow, the name that they are going to live, do 1066 01:05:11,190 --> 01:05:11,440 ‫that. 1067 01:05:12,590 --> 01:05:17,150 ‫And that's where you as you scroll down, you page the result, you page. 1068 01:05:17,150 --> 01:05:21,950 ‫There is still the result, slowly pushing the rest of the information, right? 1069 01:05:22,190 --> 01:05:25,160 ‫That's why the HDTV push is a little bit as good, but it has. 1070 01:05:25,160 --> 01:05:27,000 ‫It's a double edged sword, right? 1071 01:05:27,910 --> 01:05:34,940 ‫Because if the client cannot handle the the information coming from the server, it can crash literally 1072 01:05:34,940 --> 01:05:37,010 ‫crash the app if it cannot handle. 1073 01:05:37,010 --> 01:05:42,230 ‫So you have to do this in a smart way as much as possible. 1074 01:05:44,070 --> 01:05:46,830 ‫Fourth way, guys, what what can we do else? 1075 01:05:47,040 --> 01:05:48,780 ‫So a fourth, what is the fourth method? 1076 01:05:49,500 --> 01:05:55,740 ‫The fourth method, guys, as the fourth feature is follow. 1077 01:05:58,460 --> 01:06:00,050 ‫I went to high profile. 1078 01:06:00,380 --> 01:06:08,360 ‫I want to Nicki Minaj's profile, and I want to know if I am following her. 1079 01:06:09,460 --> 01:06:12,310 ‫Because I want to add the follow button, right? 1080 01:06:14,460 --> 01:06:19,920 ‫Because this one is going to be follow if I'm not following her, it's going to be unfollow. 1081 01:06:20,190 --> 01:06:22,210 ‫If I'm already following it, I wish to unfollow her. 1082 01:06:22,380 --> 01:06:23,270 ‫That's how I do, right? 1083 01:06:24,300 --> 01:06:28,560 ‫So how do how expensive is this query? 1084 01:06:29,190 --> 01:06:29,970 ‫Let's take a look. 1085 01:06:30,480 --> 01:06:34,970 ‫So if I go to Nicki Minaj's profile, I hear you. 1086 01:06:35,010 --> 01:06:36,810 ‫I need to head this table, right? 1087 01:06:38,380 --> 01:06:44,740 ‫And by hitting the staple, where does that's what I have, I have the ID, I how do I find out I searched 1088 01:06:44,740 --> 01:06:46,890 ‫for her, which is something we didn't talk about search. 1089 01:06:46,900 --> 01:06:51,910 ‫When I search on another I.D. and on clicking on the I.D. the fact of clicking the idea, I have the 1090 01:06:51,910 --> 01:06:54,880 ‫I.D. information where you give me all the information here. 1091 01:06:55,510 --> 01:06:57,460 ‫I give the old information back. 1092 01:06:57,820 --> 01:07:03,310 ‫Can I return the fact that because I know my I.D. right? 1093 01:07:04,850 --> 01:07:06,140 ‫When I made that query. 1094 01:07:07,410 --> 01:07:08,520 ‫I to the stable. 1095 01:07:08,790 --> 01:07:10,410 ‫Can I at the same time? 1096 01:07:12,760 --> 01:07:14,290 ‫Ask whether. 1097 01:07:16,410 --> 01:07:17,970 ‫I am following this or not. 1098 01:07:20,270 --> 01:07:25,180 ‫I can can see the stable in a transaction transactions, I start begin a transaction, right? 1099 01:07:25,660 --> 01:07:32,350 ‫And I then to the stable and then immediately also query this table where basically, how do I find 1100 01:07:32,350 --> 01:07:32,920 ‫following her? 1101 01:07:33,010 --> 01:07:35,740 ‫So that means that the ID is equal. 1102 01:07:36,070 --> 01:07:39,700 ‫The Nicki Minaj is already at and my is it's OK. 1103 01:07:39,700 --> 01:07:40,330 ‫One more. 1104 01:07:40,690 --> 01:07:40,930 ‫OK. 1105 01:07:41,170 --> 01:07:41,860 ‫In this case? 1106 01:07:42,950 --> 01:07:46,760 ‫Account what I can do, ask for just the count, and that will give me the count. 1107 01:07:47,780 --> 01:07:53,030 ‫And if it's greater than one, if it's greater than zero, that means I'm following her. 1108 01:07:53,030 --> 01:07:56,960 ‫If as if it's exactly zero, that means I am not following her and that's what I do. 1109 01:07:57,500 --> 01:08:01,130 ‫And now I return this boolean with it, right? 1110 01:08:01,860 --> 01:08:08,590 ‫However, this is an extra cost query to this table, which is a huge right because this operation is 1111 01:08:08,600 --> 01:08:09,920 ‫very common. 1112 01:08:09,920 --> 01:08:17,660 ‫Guys viewing a profile is something you do a lot compared to viewing followers, right guys. 1113 01:08:18,200 --> 01:08:22,970 ‫It's always like you have to be pragmatic like this, how often I do this versus how often I do this 1114 01:08:23,300 --> 01:08:25,850 ‫operation and based on that, you assess. 1115 01:08:26,210 --> 01:08:26,570 ‫All right. 1116 01:08:26,810 --> 01:08:32,750 ‫So if I do this and I quit, I just added a few nanoseconds or milliseconds to that query. 1117 01:08:33,770 --> 01:08:40,220 ‫And for a lot of people viewing multiple full files, especially in the same time, then that this adds 1118 01:08:40,220 --> 01:08:41,270 ‫up to a lot. 1119 01:08:41,900 --> 01:08:42,740 ‫So can I. 1120 01:08:43,550 --> 01:08:45,770 ‫Can I do this asynchronously? 1121 01:08:46,520 --> 01:08:51,740 ‫Because if I click on a profile as a user, I want the profile to be immediately displayed. 1122 01:08:51,770 --> 01:08:53,750 ‫So I want just to head this table. 1123 01:08:54,050 --> 01:08:59,480 ‫So let's say if I only had this table, that will be a quick query and I come back immediately. 1124 01:08:59,750 --> 01:09:00,040 ‫Right? 1125 01:09:00,260 --> 01:09:02,840 ‫Yeah, we have a huge table, but we have an index too, right? 1126 01:09:03,230 --> 01:09:09,950 ‫But if I had two queries this and this, we just doubled, essentially doubled or even more than that 1127 01:09:10,070 --> 01:09:12,800 ‫at cost of viewing a profile. 1128 01:09:14,000 --> 01:09:15,680 ‫Do can you afford this? 1129 01:09:16,130 --> 01:09:19,700 ‫Can you afford this plus these followers and following what? 1130 01:09:19,760 --> 01:09:20,840 ‫You have so many questions. 1131 01:09:21,260 --> 01:09:23,150 ‫So here's an idea. 1132 01:09:23,900 --> 01:09:29,240 ‫If I view the profile, I can immediately give the result to the user so they can display it. 1133 01:09:29,240 --> 01:09:33,320 ‫And that is following button becomes grayed out because we still don't know. 1134 01:09:34,580 --> 01:09:42,800 ‫The client can then asynchronously ask the question separately, because this way you still saw the 1135 01:09:43,010 --> 01:09:47,570 ‫the picture while you're you're you're blazing on. 1136 01:09:47,570 --> 01:09:51,710 ‫The beauty of Nicki Minaj was just a profile picture. 1137 01:09:51,980 --> 01:09:54,320 ‫Why are you clicking on enlarging the picture? 1138 01:09:54,590 --> 01:09:58,520 ‫We have already asynchronously loaded the rest of the information. 1139 01:09:58,730 --> 01:10:00,350 ‫You can do this at the client side. 1140 01:10:01,070 --> 01:10:02,480 ‫Yes, additional inquiries. 1141 01:10:02,750 --> 01:10:08,990 ‫But it's it's it's a better user experience because, hey, I got all the information and that's what 1142 01:10:08,990 --> 01:10:10,050 ‫I noticed on Instagram. 1143 01:10:10,070 --> 01:10:11,600 ‫I don't know if you click on a profile. 1144 01:10:12,110 --> 01:10:15,890 ‫The rest of immigration takes a little bit time to populate this mind. 1145 01:10:15,890 --> 01:10:17,390 ‫They, my are doing something like that. 1146 01:10:18,260 --> 01:10:20,530 ‫One way is to do it this way, right? 1147 01:10:20,540 --> 01:10:22,140 ‫The client ask for more. 1148 01:10:22,160 --> 01:10:23,030 ‫Oh, give me this. 1149 01:10:23,030 --> 01:10:23,500 ‫Give me this. 1150 01:10:23,510 --> 01:10:24,020 ‫Give me this. 1151 01:10:24,530 --> 01:10:26,540 ‫Give me that is that I am I following this? 1152 01:10:26,750 --> 01:10:27,330 ‫Always. 1153 01:10:27,890 --> 01:10:28,670 ‫What are the followers? 1154 01:10:28,670 --> 01:10:29,390 ‫How many followers? 1155 01:10:29,390 --> 01:10:30,950 ‫All that stuff, right? 1156 01:10:31,790 --> 01:10:37,460 ‫So give the user that quick as quick as possible more information. 1157 01:10:38,360 --> 01:10:42,140 ‫Quickest way to give it so they can feel something. 1158 01:10:42,470 --> 01:10:44,960 ‫The rest of the that are less important. 1159 01:10:45,380 --> 01:10:46,040 ‫They can. 1160 01:10:46,730 --> 01:10:48,110 ‫They can ask for it later. 1161 01:10:48,110 --> 01:10:55,580 ‫By asynchronously, the application level, the app or there's another way to push it using it should 1162 01:10:55,670 --> 01:10:58,220 ‫be to push again a little bit dangerous, right? 1163 01:10:58,490 --> 01:11:00,130 ‫Or I don't know. 1164 01:11:00,140 --> 01:11:05,140 ‫I didn't talk about this user stateful protocol G, C, G or PC. 1165 01:11:05,140 --> 01:11:11,210 ‫You have a full two way communication web sockets, Java C or all DCP anything you like. 1166 01:11:11,570 --> 01:11:12,950 ‫Fine and then. 1167 01:11:14,500 --> 01:11:21,880 ‫And then just swizzle the information back and forth, because if it's no longer a request response, 1168 01:11:21,880 --> 01:11:26,780 ‫it's a request and could be multiple responses because it's stateful protocol. 1169 01:11:26,800 --> 01:11:29,530 ‫I can send you multiple stuff all day long. 1170 01:11:30,280 --> 01:11:31,240 ‫So don't worry about it. 1171 01:11:31,840 --> 01:11:33,920 ‫So, yeah, that's definitely possible. 1172 01:11:34,120 --> 01:11:35,860 ‫And I think, guys, I'm going to stop right here. 1173 01:11:36,100 --> 01:11:37,990 ‫So we discuss so many other things. 1174 01:11:38,260 --> 01:11:39,820 ‫Just one follow. 1175 01:11:39,820 --> 01:11:42,790 ‫One feature, one feature. 1176 01:11:43,270 --> 01:11:43,660 ‫Talk us. 1177 01:11:43,660 --> 01:11:47,260 ‫I don't know how many minutes we've been discussing, but I'm going to see you in the next one. 1178 01:11:47,470 --> 01:11:54,490 ‫You guys stay awesome, guys and keep designing, keep enjoying the process and going to see in the 1179 01:11:54,490 --> 01:11:54,820 ‫next one. 1180 01:11:54,960 --> 01:11:55,660 ‫You guys stay awesome.