1 00:00:00,420 --> 00:00:01,930 ‫What is going on, guys? 2 00:00:01,950 --> 00:00:09,540 ‫My name is Hossein and this video, I want to discuss whether databases should move to quick or not. 3 00:00:09,900 --> 00:00:11,160 ‫So I've already discussed that. 4 00:00:11,220 --> 00:00:11,610 ‫All right. 5 00:00:11,610 --> 00:00:21,030 ‫So Database's guys uses most of the databases, uses their own protocol for making communications. 6 00:00:21,030 --> 00:00:21,280 ‫Right. 7 00:00:21,300 --> 00:00:27,360 ‫So if you have a client that I want to connect to a database like a Postgres or Oracle or SQL Server, 8 00:00:27,390 --> 00:00:34,260 ‫already there is always most of the time 99 percent is a TCP connection. 9 00:00:34,440 --> 00:00:39,860 ‫And that database built their own protocol on par on top of TCP. 10 00:00:39,870 --> 00:00:44,940 ‫So they have this special headers and they have their special compression and they do their own thing. 11 00:00:45,540 --> 00:00:50,910 ‫I am not aware of a protocol for databases, a universal protocol. 12 00:00:50,910 --> 00:00:55,510 ‫Correct me, guys, if anyone, a database engineer there that there was any wrong. 13 00:00:55,530 --> 00:00:56,670 ‫There is actually a protocol. 14 00:00:57,000 --> 00:01:03,330 ‫The only protocol that I'm aware of was resp, and that's for retests, which is retests protocol, 15 00:01:03,330 --> 00:01:05,760 ‫something communication protocol, resp. 16 00:01:05,760 --> 00:01:06,930 ‫Our ISP. 17 00:01:06,930 --> 00:01:12,900 ‫I believe I might be wrong that might the other protocols so that that created kind of a chaos in the 18 00:01:12,900 --> 00:01:22,980 ‫database engineering community because there is no protocol like grb see with with web traffic or that 19 00:01:22,980 --> 00:01:27,960 ‫walji obviously doesn't be used for web traffic, but rest for what the traffic or HDB for web traffic 20 00:01:27,960 --> 00:01:28,860 ‫to be on is right. 21 00:01:29,070 --> 00:01:30,330 ‫To be very specific. 22 00:01:30,510 --> 00:01:32,980 ‫It is used for Web most of the time, right. 23 00:01:33,000 --> 00:01:36,720 ‫There's very elegantly designed a micro services. 24 00:01:36,720 --> 00:01:42,270 ‫Now we're seeing a movement that every service that can make it with another service use GOPAC because 25 00:01:42,270 --> 00:01:49,590 ‫it has kind of a schema ID as very well defined and but databases, guys, databases of raw databases. 26 00:01:50,370 --> 00:01:54,150 ‫There is there is no protocol fit for it yet. 27 00:01:54,990 --> 00:01:55,320 ‫Right. 28 00:01:55,320 --> 00:02:01,010 ‫Yet there are many problems with databases, especially when it comes to the Web. 29 00:02:01,020 --> 00:02:04,830 ‫What do you do if you spin up a Web application, the first thing you do? 30 00:02:04,920 --> 00:02:05,300 ‫Right. 31 00:02:05,790 --> 00:02:08,010 ‫And I'm talking about relational databases more. 32 00:02:08,010 --> 00:02:13,290 ‫And I think this applies to also know school databases, because what do you do? 33 00:02:13,290 --> 00:02:22,440 ‫Want to open up a framework or an API gateway ride that does a Web API gateway that essentially talks 34 00:02:22,440 --> 00:02:23,610 ‫to a back in database? 35 00:02:24,900 --> 00:02:35,010 ‫The idea is you spin up a pool of connections and every request that comes in a get request, you take 36 00:02:35,010 --> 00:02:37,620 ‫that request you reserve. 37 00:02:38,460 --> 00:02:45,280 ‫A connection from the pool, you execute the query and you return the connection back to the pool and 38 00:02:45,300 --> 00:02:46,080 ‫you might say. 39 00:02:47,260 --> 00:02:50,680 ‫Can't we just use one single connection for everybody? 40 00:02:51,190 --> 00:02:53,950 ‫Bad idea, guys, never do this. 41 00:02:54,520 --> 00:02:55,980 ‫You might get away with it. 42 00:02:56,140 --> 00:03:00,160 ‫The database will not yell at you, but there is a big problem with this. 43 00:03:00,160 --> 00:03:01,270 ‫What why? 44 00:03:01,270 --> 00:03:02,080 ‫Why is it? 45 00:03:02,710 --> 00:03:08,140 ‫Database's was designed in the 70s until this day to be a stateful. 46 00:03:08,560 --> 00:03:09,400 ‫What does that mean? 47 00:03:09,430 --> 00:03:16,630 ‫That means if I as a client, connect to a database, I am the only one sending stuff to it. 48 00:03:16,900 --> 00:03:25,300 ‫And probably one process will use one connection to send a query, a select star and insert a transaction. 49 00:03:25,450 --> 00:03:27,410 ‫That's one process, right? 50 00:03:27,790 --> 00:03:30,490 ‫It was designed to be the Singleton thing. 51 00:03:30,520 --> 00:03:30,860 ‫Right. 52 00:03:32,330 --> 00:03:34,850 ‫And I said it's been working fine forever. 53 00:03:36,020 --> 00:03:45,050 ‫Nobody anticipated this blowing up of Web applications where now you have simultaneous gets requests 54 00:03:45,050 --> 00:03:53,420 ‫or post requests coming, but now the Web servers actually responsible for making their way to the database. 55 00:03:53,450 --> 00:04:01,430 ‫Well, now, yes, you have a single process with a single connection to the database, but multiple 56 00:04:01,430 --> 00:04:04,100 ‫users are sending those requests. 57 00:04:04,110 --> 00:04:06,770 ‫So it's kind of acting like a proxy. 58 00:04:07,940 --> 00:04:08,900 ‫What does that mean? 59 00:04:09,020 --> 00:04:14,780 ‫That means if you use the single TCAP connection or debted this connection, here's what will happen 60 00:04:15,190 --> 00:04:22,580 ‫if I come in and ask for my pictures as I use and I'm authenticated and you're going to use the same 61 00:04:22,580 --> 00:04:28,580 ‫TCP connection, hoping you're going to hit the same server using let's say you are using OJARS and 62 00:04:28,760 --> 00:04:32,900 ‫someone else is asking for their pictures the same exact second. 63 00:04:33,200 --> 00:04:35,270 ‫And you hit that query, right. 64 00:04:35,270 --> 00:04:38,270 ‫And you executed that simultaneously. 65 00:04:38,300 --> 00:04:38,610 ‫Right. 66 00:04:38,900 --> 00:04:49,380 ‫So you cannot guarantee the response that comes back for is for which requester unless there is metadata 67 00:04:49,380 --> 00:04:53,320 ‫that I'm not aware of a metadata that is getting sent with the Skype connection. 68 00:04:53,750 --> 00:04:54,060 ‫Right. 69 00:04:54,380 --> 00:05:00,710 ‫So if you send a request from the same process, you can get my pictures. 70 00:05:02,210 --> 00:05:03,920 ‫And that's very, very bad. 71 00:05:04,040 --> 00:05:05,540 ‫That's why nobody does that. 72 00:05:05,720 --> 00:05:12,020 ‫Right, because the TCAP connection is is is there's no order per say to it. 73 00:05:12,020 --> 00:05:12,320 ‫Right. 74 00:05:12,320 --> 00:05:15,650 ‫Is you can send a request and the other request can go back. 75 00:05:15,650 --> 00:05:19,890 ‫And so it goes you're kind of using the same TCP connection to get the packets back. 76 00:05:20,150 --> 00:05:23,480 ‫So anything any problem with the ordering. 77 00:05:24,080 --> 00:05:24,460 ‫Right. 78 00:05:24,500 --> 00:05:27,530 ‫Can give you into a very bad state so you don't do this. 79 00:05:27,740 --> 00:05:28,790 ‫So what do you do instead? 80 00:05:29,240 --> 00:05:34,210 ‫Once you send a request to a single connection, you say, hey, don't touch this connection. 81 00:05:34,220 --> 00:05:40,310 ‫This connection is for this request and then you start using this pooling idea, a been up 20 connections. 82 00:05:40,490 --> 00:05:47,420 ‫And if you want to make a request, reserve a connection from the pool executer query, get it back 83 00:05:47,630 --> 00:05:49,760 ‫and then respond back. 84 00:05:50,420 --> 00:05:50,690 ‫Right. 85 00:05:50,750 --> 00:05:51,470 ‫So that's what we do. 86 00:05:51,470 --> 00:05:52,300 ‫We do pooling. 87 00:05:52,370 --> 00:05:52,730 ‫Right. 88 00:05:53,510 --> 00:05:54,590 ‫That being said. 89 00:05:55,840 --> 00:05:58,330 ‫You cannot trust this single connection, think. 90 00:05:59,290 --> 00:06:05,110 ‫I might be wrong that some delegates actually implement some sort of a safety right, but you cannot 91 00:06:05,110 --> 00:06:05,630 ‫trust it. 92 00:06:06,220 --> 00:06:09,400 ‫I'm not sure which to implement these safety things. 93 00:06:09,400 --> 00:06:09,620 ‫Right. 94 00:06:09,750 --> 00:06:10,750 ‫Just request. 95 00:06:10,840 --> 00:06:12,250 ‫And they are not out of order. 96 00:06:12,610 --> 00:06:15,770 ‫You did, because one request can get delayed. 97 00:06:15,790 --> 00:06:18,400 ‫One response can get delayed by a fraction of a millisecond. 98 00:06:18,400 --> 00:06:22,570 ‫And you can you have no idea what responses for which request. 99 00:06:22,810 --> 00:06:23,060 ‫Right. 100 00:06:23,320 --> 00:06:25,330 ‫That's why we have streams initiative, too. 101 00:06:25,480 --> 00:06:25,840 ‫Right. 102 00:06:26,050 --> 00:06:31,290 ‫That's why Initiative one, we block you from sending another request until the response comes back. 103 00:06:31,450 --> 00:06:32,380 ‫Why do we do that? 104 00:06:32,380 --> 00:06:33,340 ‫Because we don't know. 105 00:06:33,370 --> 00:06:34,870 ‫We don't know which request is for what. 106 00:06:34,870 --> 00:06:38,110 ‫Your response, which responses for each request is the same problem. 107 00:06:38,650 --> 00:06:38,980 ‫Right. 108 00:06:38,990 --> 00:06:44,260 ‫So we have had offline blocking because of that because to prevent this problem. 109 00:06:44,590 --> 00:06:48,820 ‫And now we invented a video, obviously beautiful design. 110 00:06:49,330 --> 00:06:50,350 ‫We have a stream. 111 00:06:50,360 --> 00:06:53,980 ‫So every request now you can use multiple. 112 00:06:54,920 --> 00:07:03,230 ‫Request on this NTSB connection, as long as you elegantly tag each request with a story middy. 113 00:07:03,440 --> 00:07:03,830 ‫Now. 114 00:07:05,710 --> 00:07:10,000 ‫We solve this problem on the web, but the database community says, wait a second, why is that? 115 00:07:10,010 --> 00:07:12,610 ‫Why are we still using this boiling garbage thing? 116 00:07:14,680 --> 00:07:15,960 ‫It's a general question. 117 00:07:15,970 --> 00:07:16,630 ‫I don't know. 118 00:07:16,870 --> 00:07:17,710 ‫I might be wrong. 119 00:07:17,860 --> 00:07:26,310 ‫I think we should bring streams to database connections to database protocols. 120 00:07:26,320 --> 00:07:28,480 ‫Don't you guys think it makes sense? 121 00:07:28,480 --> 00:07:28,820 ‫Right. 122 00:07:29,350 --> 00:07:31,840 ‫It just makes perfect sense, specially for the Web. 123 00:07:32,140 --> 00:07:32,950 ‫Oh, my God. 124 00:07:32,950 --> 00:07:40,210 ‫The performance we're going to get from this and obviously we cannot implement this to be to right databases 125 00:07:41,470 --> 00:07:41,970 ‫database. 126 00:07:41,980 --> 00:07:45,760 ‫People get really freaked out when you say it's beyond databases. 127 00:07:45,760 --> 00:07:48,630 ‫They don't like that couch killed that idea. 128 00:07:48,690 --> 00:07:52,930 ‫They were very brave and they implemented that because, you know, because of the extra here and I 129 00:07:52,930 --> 00:07:59,560 ‫understand the headers that is EDV to it doesn't really apply for databases and they are just an overkill. 130 00:08:00,190 --> 00:08:04,450 ‫But still, that's why both databases elegantly designed their own protocols. 131 00:08:05,200 --> 00:08:08,450 ‫But I think we have a good candidate. 132 00:08:08,450 --> 00:08:12,310 ‫And guys, quick, quick is a great protocol. 133 00:08:12,940 --> 00:08:17,260 ‫Those guys said don't say step two is great. 134 00:08:17,440 --> 00:08:18,910 ‫The stream is amazing. 135 00:08:19,330 --> 00:08:27,220 ‫How will we push this beautiful design at the lower level, at the network level? 136 00:08:27,430 --> 00:08:32,980 ‫So that's why Quick was born as a logical connection UDP. 137 00:08:34,010 --> 00:08:41,040 ‫Architecture that has dreams in it, so now as application which should be free, will just it will 138 00:08:41,070 --> 00:08:43,810 ‫be a thin layer that uses quick, right. 139 00:08:43,830 --> 00:08:45,410 ‫It just have all that stuff. 140 00:08:45,420 --> 00:08:48,800 ‫So a database could use quick. 141 00:08:49,970 --> 00:08:53,020 ‫Watch database we'll use quick first. 142 00:08:53,150 --> 00:09:01,610 ‫I'm just waiting desperately for that database that will be brave enough to use quick. 143 00:09:03,550 --> 00:09:09,870 ‫Quick has their own problem, though, UDP, the Internet doesn't like YouTube much, right? 144 00:09:10,810 --> 00:09:13,790 ‫And doesn't like DCP either. 145 00:09:14,410 --> 00:09:20,620 ‫But you get the ECB has its own problem with retransmission and controls, that the mistake that the 146 00:09:20,620 --> 00:09:23,100 ‫ECB did was implementing this as a low level. 147 00:09:23,110 --> 00:09:28,330 ‫To be honest, it should have been implemented by the application level right at the application layer 148 00:09:28,900 --> 00:09:32,920 ‫or maybe at a higher level, a little bit higher level. 149 00:09:32,920 --> 00:09:34,420 ‫And and we're going to take care of it. 150 00:09:34,420 --> 00:09:39,400 ‫But it's it was implemented as a lower level, so we were stuck with it. 151 00:09:39,790 --> 00:09:40,150 ‫Right. 152 00:09:40,480 --> 00:09:45,280 ‫And now when we tried to introduce streams with DCPI, we ran into the problem. 153 00:09:45,280 --> 00:09:52,150 ‫The famous problem with SGB, too, with TCP, which is I have many streams. 154 00:09:52,810 --> 00:09:53,290 ‫Right. 155 00:09:53,290 --> 00:10:02,290 ‫And one stream is shady and it's I don't know, one stinking packet is corrupted to stream one and stream 156 00:10:02,290 --> 00:10:05,740 ‫two is waiting because of the stinking packet. 157 00:10:06,250 --> 00:10:13,090 ‫Despite having no resemblance of commonness at all, they they have nothing in common. 158 00:10:13,360 --> 00:10:19,900 ‫They share only the same TCP connection, but they have to wait because of this thinking packet that 159 00:10:19,900 --> 00:10:24,940 ‫is bad and in stream one quick solves that and they logically separate. 160 00:10:25,060 --> 00:10:26,710 ‫How do you like this animation guys? 161 00:10:26,800 --> 00:10:29,350 ‫I don't, I don't know how to do editing in animation. 162 00:10:29,350 --> 00:10:30,910 ‫I'm going to get better at this, I promise. 163 00:10:30,910 --> 00:10:40,930 ‫But but you see the idea I just got done so every stream is completely independent of that. 164 00:10:41,120 --> 00:10:44,080 ‫How's that French independent? 165 00:10:44,500 --> 00:10:50,130 ‫Every stream is independent by itself and it's doing its own thing. 166 00:10:50,320 --> 00:10:56,350 ‫So now if that stream 10 has a bad beckert in it, we only retransmit right. 167 00:10:56,470 --> 00:11:02,620 ‫That packet in stream ten stream want to stream to stream three could just go normally and think of 168 00:11:02,620 --> 00:11:04,720 ‫streams just like as normal request. 169 00:11:04,720 --> 00:11:05,470 ‫I get requests. 170 00:11:05,470 --> 00:11:07,450 ‫No one returns the three. 171 00:11:07,450 --> 00:11:11,670 ‫You can simultaneously concurrently send all that stuff in. 172 00:11:12,070 --> 00:11:12,510 ‫So now. 173 00:11:12,540 --> 00:11:17,470 ‫Yeah I, I'm waiting for a database that implements quick at that level. 174 00:11:17,740 --> 00:11:18,100 ‫Right. 175 00:11:18,340 --> 00:11:19,000 ‫And now. 176 00:11:20,430 --> 00:11:25,020 ‫The observer can have one DCB connection that serves millions of users. 177 00:11:25,380 --> 00:11:32,310 ‫This might be, of course, we're going to run into problems where that there will be lots of conjecture, 178 00:11:32,550 --> 00:11:39,510 ‫congestion control over a single DGP connection, but that that will save us memory, guys. 179 00:11:39,750 --> 00:11:42,840 ‫That will save us round trip. 180 00:11:42,840 --> 00:11:43,440 ‫That will save us. 181 00:11:43,440 --> 00:11:46,110 ‫Processing of this overside, I think is a good idea. 182 00:11:46,140 --> 00:11:46,650 ‫What do you think? 183 00:11:46,660 --> 00:11:51,240 ‫What are the odds then, guys, if you build Web applications, have you run into this problem before 184 00:11:51,720 --> 00:11:57,590 ‫or do you just do a stateless architecture like some people do that they don't use pooling? 185 00:11:57,600 --> 00:12:05,040 ‫They say, hey, every request closes, an opening opens and closes its own DCB connection to the database? 186 00:12:05,490 --> 00:12:10,380 ‫That's a little bit of an overkill, but some people do that to be absolutely safe. 187 00:12:11,580 --> 00:12:15,390 ‫Still works then I see in the next one you guys stay out in.