1 00:00:04,820 --> 00:00:12,470 ‫In this section, we've covered a lot about Swarm. Swarm, Stacks and Secrets are kind of like a 2 00:00:12,470 --> 00:00:17,510 ‫trilogy of awesome features that can really make things much easier in production for us. 3 00:00:17,600 --> 00:00:23,120 ‫I want to show you what it might be like if you actually took your Compose files, organize them in 4 00:00:23,120 --> 00:00:24,710 ‫a couple of ways, 5 00:00:24,920 --> 00:00:27,210 ‫and this is what I basically call living the dream. 6 00:00:27,230 --> 00:00:32,690 ‫It turns out you can actually use a single file to do a lot of things, but sometimes your complexity 7 00:00:32,690 --> 00:00:36,080 ‫grows and you're going to need multiple Compose files. 8 00:00:36,080 --> 00:00:38,060 ‫I want to just run through real quick. 9 00:00:38,060 --> 00:00:42,620 ‫You don't actually have to do this yourself if you don't want to. I'm just going to show you an example of 10 00:00:42,620 --> 00:00:47,520 ‫how these Compose files might work together to build up your environment as you go. 11 00:00:47,780 --> 00:00:52,490 ‫In this scenario, we're going to use docker compose up for our local development environment. 12 00:00:52,490 --> 00:00:59,170 ‫We're going to use a docker compose up config and file for our CI environment to do integration testing. 13 00:00:59,390 --> 00:01:04,580 ‫Then when we're in production, we're going to use those files for docker stack deploy to actually deploy 14 00:01:04,580 --> 00:01:10,630 ‫the production environment with a stack. So, I'm on my local machine and we're going to be using the example 15 00:01:10,630 --> 00:01:14,330 ‫before of the Drupal scenario, with a database server, and a web frontend. 16 00:01:14,380 --> 00:01:19,630 ‫We have the Dockerfile we have used in our previous assignments. We're rebuilding a custom, yet simple, 17 00:01:19,690 --> 00:01:21,880 ‫drupal config with a template. 18 00:01:22,060 --> 00:01:25,540 ‫Then we're going to have this default compose file. 19 00:01:25,540 --> 00:01:26,990 ‫This is called an override, 20 00:01:26,980 --> 00:01:32,350 ‫what we're about to do. An override is where I have the standard Docker Compose file, and it sets the 21 00:01:32,350 --> 00:01:35,710 ‫defaults that are the same across all my environments. 22 00:01:35,710 --> 00:01:42,370 ‫Then, I have this override file that by default, Docker Compose, if it's named this exact name, docker 23 00:01:42,370 --> 00:01:48,060 ‫compose.override.yml, it will automatically bring this in whenever I do a docker compose up. 24 00:01:48,190 --> 00:01:53,200 ‫You'll see that in this scenario, we're assuming local development because we really want the hand-typed 25 00:01:53,200 --> 00:01:55,650 ‫commands we're going to type to be easiest locally. 26 00:01:55,720 --> 00:02:00,400 ‫Normally in your CI environment, it's all automated so we don't really care if those commands are 27 00:02:00,400 --> 00:02:01,470 ‫a little bit longer. 28 00:02:01,520 --> 00:02:04,750 ‫But we really want locally is the easy docker compose up. 29 00:02:04,750 --> 00:02:08,890 ‫The cool thing is Docker Compose will read this file automatically and it will apply this over 30 00:02:08,890 --> 00:02:13,750 ‫top, or override, any settings in the Docker Compose yml file. 31 00:02:13,750 --> 00:02:14,590 ‫And so here, 32 00:02:14,630 --> 00:02:19,780 ‫notice I don't have the image name, because that's already specified over here. In this override file, 33 00:02:19,840 --> 00:02:25,800 ‫I override by saying I want to build the image locally using the Dockerfile in this current directory. 34 00:02:26,020 --> 00:02:32,920 ‫I want to create a port on 8080 for local development. And I'm setting up some volumes, and you'll even notice 35 00:02:32,920 --> 00:02:39,190 ‫I gave you an example here of a bind mount, where I might be doing a custom theme. And I want to mount 36 00:02:39,190 --> 00:02:45,550 ‫my theme on my host into the container like we did in previous sections, so that I can change it locally 37 00:02:45,580 --> 00:02:48,130 ‫and then see it right away on the server. 38 00:02:48,130 --> 00:02:52,790 ‫And by the way, for this example, I don't actually know how to develop themes in Drupal. 39 00:02:52,810 --> 00:02:57,170 ‫I'm not exactly sure that if I change a file in there, it will automatically be reflected. 40 00:02:57,310 --> 00:03:02,050 ‫I just wanted to show an example of how when you're doing development in web, typically this is the way 41 00:03:02,050 --> 00:03:07,480 ‫you would do it without having to stop and start the Compose every time. Down here, under postgres, we 42 00:03:07,480 --> 00:03:12,610 ‫have the environment variable and the Secret settings like before. We have the defined volumes, and you'll 43 00:03:12,610 --> 00:03:13,470 ‫see down at the bottom, 44 00:03:13,480 --> 00:03:18,780 ‫I actually have the file-based secret because when we're doing local development, we have to use the file- 45 00:03:18,790 --> 00:03:19,690 ‫based secret. 46 00:03:19,960 --> 00:03:24,130 ‫Things get a little interesting when I look at this prod, or test files. The way this is going 47 00:03:24,130 --> 00:03:29,680 ‫to work is, remember that the .override.yml automatically gets picked up by the Docker Compose 48 00:03:29,680 --> 00:03:32,130 ‫command line. In prod or test, 49 00:03:32,140 --> 00:03:34,640 ‫I'm going to have to specify them manually. 50 00:03:34,810 --> 00:03:38,420 ‫So for test, we're going to have to use the -f command. 51 00:03:38,530 --> 00:03:44,770 ‫If you remember from earlier sections, the -f is when we do a docker compose that we want to specify 52 00:03:44,770 --> 00:03:47,590 ‫a custom file. I'll show you that in a minute. 53 00:03:47,590 --> 00:03:53,170 ‫Then in production, since we're not going to actually have the Docker Compose command line on a production 54 00:03:53,170 --> 00:03:59,260 ‫server, what we're going to do here is we're actually going to use a docker compose config command. 55 00:03:59,260 --> 00:04:05,590 ‫The config command is actually going to do an output by squishing together, or combining, the output of 56 00:04:05,590 --> 00:04:07,050 ‫multiple config files. 57 00:04:07,060 --> 00:04:08,290 ‫So that will be really cool. 58 00:04:08,290 --> 00:04:12,520 ‫Real quick, the test file just has the drupal and the postgres. 59 00:04:12,520 --> 00:04:18,820 ‫Imagine if this was your Jenkins CI or Codeship CI solution, where I want it to build the image 60 00:04:18,850 --> 00:04:24,430 ‫every time I commit my code, and I want to call it this, and I want it to be on this port for testing 61 00:04:24,430 --> 00:04:30,520 ‫purposes. Then I'm going to use a fake password. But I don't need to define any of the volumes because 62 00:04:30,520 --> 00:04:35,080 ‫I'm not going to actually try to keep named volume data because again, it's just a CI platform. 63 00:04:35,080 --> 00:04:39,040 ‫As soon as it passes tests or fails tests, it'll get rid of everything. 64 00:04:39,190 --> 00:04:44,950 ‫Then in this scenario, you might see that I've actually got this sample data scenario where maybe 65 00:04:45,040 --> 00:04:51,310 ‫in your CI solution, you have simply databases sitting there that come from either a custom Git repository 66 00:04:51,340 --> 00:04:56,470 ‫or maybe they're an FTP download. Or something happens during the initialization of your CI where it actually 67 00:04:56,470 --> 00:05:02,320 ‫downloads a database file. And instead of us having to create our database every single time we do CI 68 00:05:02,320 --> 00:05:08,260 ‫testing, we would just mount this directory of sample data into where our postgres data is supposed to 69 00:05:08,260 --> 00:05:08,840 ‫be. 70 00:05:08,920 --> 00:05:13,740 ‫And that way, we could guarantee we had the same sample data every single time we do a CI test. 71 00:05:14,050 --> 00:05:17,710 ‫I'm not going to go into that any further. I just wanted to show that that might be how this file 72 00:05:17,710 --> 00:05:19,320 ‫for CI would be different. 73 00:05:19,600 --> 00:05:25,840 ‫Then in production, we have all of our normal production concerns. We're specifying volumes for our 74 00:05:25,840 --> 00:05:32,260 ‫specific data. We're specifying our secret. And notice down at the bottom, we have the external secret. 75 00:05:32,260 --> 00:05:37,030 ‫Because we're going to have put the secret in already via the command line like we did in the earlier 76 00:05:37,030 --> 00:05:38,080 ‫assignment. 77 00:05:38,080 --> 00:05:43,690 ‫The point here is that all three of these configs are different in some way, but they all relate to 78 00:05:43,690 --> 00:05:51,010 ‫the core config, or the base config, which just defines the two services and their images. 79 00:05:51,250 --> 00:05:53,700 ‫You can see here. Here's a reminder of what it looks like. 80 00:05:53,710 --> 00:05:56,910 ‫Let's exit this and run a couple of commands. 81 00:05:57,090 --> 00:06:01,870 ‫If I look in this directory, you'll see that I have the base file and then the three override files. 82 00:06:01,960 --> 00:06:04,640 ‫Again, remember, that the override.yml is the default. 83 00:06:04,660 --> 00:06:12,590 ‫If I do a docker compose up, what it will actually do here is use the docker-compose.yml first, 84 00:06:12,710 --> 00:06:19,190 ‫and then it will overlay the override.yml one on top. I want to put a -d in so we can 85 00:06:19,190 --> 00:06:26,960 ‫take a quick look after it started. I'm going to do a docker inspect on the drupal image. 86 00:06:27,260 --> 00:06:30,800 ‫And what I wanted to show you was in here, it's got all the mounts listed. 87 00:06:30,800 --> 00:06:36,920 ‫So we know that it took the override file because the override file was where we defined all of these 88 00:06:36,920 --> 00:06:37,720 ‫mounts. 89 00:06:37,880 --> 00:06:39,460 ‫So we know that it picked that up. 90 00:06:39,590 --> 00:06:43,400 ‫Obviously, if it didn't pick up the base one, it wouldn't even know what images to use so it would 91 00:06:43,400 --> 00:06:47,050 ‫actually be complaining to us and saying that the Compose file was incomplete. 92 00:06:47,240 --> 00:06:48,160 ‫That one worked. 93 00:06:48,170 --> 00:06:50,710 ‫Let's do a down real quick. 94 00:06:50,900 --> 00:06:56,600 ‫If we were going to actually do the command we needed for our CI solution, what we would have to 95 00:06:56,600 --> 00:07:02,960 ‫do on our CI solution was to make sure that Docker Compose was there, and installed, and available so that 96 00:07:02,960 --> 00:07:07,610 ‫we could do docker compose commands. Then we can specify the -f, 97 00:07:07,790 --> 00:07:12,210 ‫and then the order of the f's is the base file always needs to be first. 98 00:07:12,260 --> 00:07:18,010 ‫So docker compose yml, then docker compose test. 99 00:07:18,290 --> 00:07:26,970 ‫If we did this with an up at the end and a -d, and then I went and inspected that same drupal, 100 00:07:27,050 --> 00:07:30,060 ‫you'll notice that there's no bind mounts. 101 00:07:30,060 --> 00:07:34,250 ‫They're completely missing because in the test file, we didn't specify those. 102 00:07:34,250 --> 00:07:38,270 ‫We didn't actually need Drupal to save information because it was going to be thrown away at the end 103 00:07:38,270 --> 00:07:39,760 ‫of our CI run. 104 00:07:39,980 --> 00:07:42,710 ‫And then third, we have the production config. 105 00:07:42,710 --> 00:07:44,980 ‫The production config is going to be a little bit different. 106 00:07:45,200 --> 00:07:50,450 ‫I'm going to hit the Up Arrow, and then here I'm going to do the prod. Instead of an up command, what I want to do 107 00:07:50,450 --> 00:07:53,120 ‫here is a config. 108 00:07:53,290 --> 00:07:58,640 ‫If you just look at the help real fast for the config, it has only two options and neither one of 109 00:07:58,640 --> 00:08:00,880 ‫those are really relevant in this situation. 110 00:08:00,890 --> 00:08:05,990 ‫What we want to just do is just run config by itself. What it's going to actually do is look at both 111 00:08:05,990 --> 00:08:11,080 ‫files and push them together into a single Compose file equivalent. 112 00:08:11,090 --> 00:08:16,010 ‫So what we could do here is just run this command somewhere in our CI solution. Then have 113 00:08:16,010 --> 00:08:23,690 ‫it output to a file, maybe with you know, something like that. That output file would be the one we would 114 00:08:23,690 --> 00:08:28,220 ‫use officially in production to create or update our stack. 115 00:08:28,220 --> 00:08:31,490 ‫However, I want to throw in a little caveat here. 116 00:08:31,550 --> 00:08:34,130 ‫This is all relatively new stuff. 117 00:08:34,160 --> 00:08:37,340 ‫We know that Secrets and Swarm Stacks are relatively new. 118 00:08:37,340 --> 00:08:38,620 ‫They're only a couple of months old. 119 00:08:38,630 --> 00:08:39,920 ‫As of this recording. 120 00:08:40,040 --> 00:08:41,980 ‫So there's a couple of rough edges. 121 00:08:42,050 --> 00:08:48,850 ‫We just ran that config command, and you'll actually notice that the secrets weren't listed in there. 122 00:08:49,070 --> 00:08:50,490 ‫That's a bug currently. 123 00:08:50,510 --> 00:08:53,870 ‫I'm actually working with the Docker team to see if we can't squash that bug. 124 00:08:53,870 --> 00:08:57,760 ‫So by the time you read this, it may have already been fixed. 125 00:08:57,770 --> 00:09:01,490 ‫Make sure you inspect that output of the config line before you go 126 00:09:01,480 --> 00:09:02,680 ‫deploying in production. 127 00:09:02,810 --> 00:09:10,070 ‫Secondly, the Compose Extends option, which I did not discuss here, is another way to override these 128 00:09:10,070 --> 00:09:16,580 ‫Compose files where you actually use the override file and you actually define an extends section in 129 00:09:16,580 --> 00:09:17,150 ‫there. 130 00:09:17,180 --> 00:09:23,120 ‫It's a little bit more declarative, so it's easier to understand. I'll provide a link in the references 131 00:09:23,120 --> 00:09:28,970 ‫for this section. Just know that that Extends option doesn't yet work in Swarms stacks. 132 00:09:29,000 --> 00:09:33,770 ‫I didn't mention it here because it doesn't really give you the full app lifecycle that we were hoping 133 00:09:33,770 --> 00:09:37,360 ‫for, but I do expect them at some point to do something about that. 134 00:09:37,370 --> 00:09:42,860 ‫Like either add it into Swarm or create a better workflow. Because that's really the idea 135 00:09:42,860 --> 00:09:48,800 ‫we're trying to get to with all of these tools, is to have a complete and easy lifecycle from development 136 00:09:48,830 --> 00:09:53,850 ‫all the way through test, into production, with the same set of configurations, in the same set of images. 137 00:09:53,850 --> 00:09:58,220 ‫I hope this got you thinking about how you might make your apps this way, and how you might extend 138 00:09:58,220 --> 00:10:00,980 ‫your own Compose files for complex scenarios. 139 00:10:00,980 --> 00:10:06,140 ‫Because nothing puts a smile on my face faster with Docker than when I see someone run a one line command 140 00:10:06,140 --> 00:10:09,550 ‫to do something that previously took multiple scripts to execute.