‫The Docker Health Check command. It was a new feature added in 1.12, which came out mid 2016, the
‫same time that Swarm Kit and Swarm Mode were available in Docker. It was added really as a part of
‫that toolkit, but it still works in all the different files like the Dockerfile, the Compose file,
‫the docker run command uses it, the Stack files support it, the service update and service create command
‫support it.
‫It's everywhere.
‫I highly recommend that when you're going production, you do engage in testing options for this health
‫check command.
‫It's going to work right out of the box with an exec.
‫It's going to execute that command inside the container just like if you were running your own exec
‫command.
‫So, it's not running it from outside the container; it's just running it inside which means that even
‫simple workers that don't have exposed ports, you can run a simple command in them to validate whether
‫they're returning good data or whatever.
‫It's a simple execution of a command, which means it gets a simple return. It expects a 0 or a 1.
‫In Linux and Windows, you have exit codes from commands and a 0 is a good thing.
‫It means everything was fine.
‫Anything other than a 0 is going to be an error in most applications.
‫But in Docker, we need that application to exit a 1 specifically. We'll show in a minute how you
‫do that.
‫There's only three states to a healthcheck in Docker. It starts out with starting. Starting is the
‫first 30 seconds, by default, where it hasn't run a healthcheck command yet.
‫Then it's going to run one.
‫If that returns a 0, it'll start with the healthy. It'll change to the healthy option.
‫It'll take that command and it'll run it every 30 seconds by default again. If it ever receives an unhealthy
‫return, like an exit 1, then it marks it as an unhealthy container.
‫We have options for controlling all of this including retries.
‫We'll see that in a minute.
‫This is a much better option than we've had in the past because Docker, until now, was just making
‫sure the application was still running.
‫It didn't have any insight into whether that application was doing what it was supposed to.
‫Now we can do that inside the Docker container itself.
‫But this isn't a replacement for your third party monitoring solution.
‫This isn't going to give you graphs, or status over time, or any sort of third party tooling that you
‫would expect out of a monitoring solution.
‫This is about Docker understanding if the container itself has a basic level of healthy.
‫So, in a Nginx, it might return a localhost of the root index file. A return of 200 or 300 is fine
‫and gives it an exit code of 0, and it considers it healthy.
‫That's not a super advanced uh, you know, monitoring tool.
‫But if it did return a 404 or 500 error, it would then consider it unhealthy and we can do something about
‫that.
‫Where are we going to see this Docker healthcheck in the GUI?
‫The first place is in container ls.
‫It'll just see it as this new option.
‫It's in the middle. We'll see in a second where it'll show us one of the three states if the health check
‫is running, and that's how we actually know that there's a healthcheck.
‫That's the easiest way, at least, to know. We'll see the history, the last five of that healthcheck,
‫show up in the inspect for that container. And we can see some basic trend over time there.
‫But the docker run command does not take action on an unhealthy container.
‫Once the healthcheck considers a container unhealthy, docker run is just going to indicate that in the ls
‫command, and in the inspect, but it's not going to take action.
‫That's where we expect the Swarm Services to take action.
‫So the stacks and services will actually replace that container with a new task, on a new host possibly,
‫depending on the scheduler.
‫Even in the update command, we see a little extra bonus by using the healthchecks because the updates
‫will consider the healthcheck as a part of the readiness for that container before it goes and changes
‫the next one.
‫If a container comes up, but it doesn't pass its health check, then the service update won't go to
‫the next one. Or it'll take action based on the changes you give it.
‫Let's look at a few examples before we go to the command line.
‫This is one that we're using on docker run.
‫This allows us to use an existing image that doesn't have a health check in it, and we're adding
‫the health check in at runtime.
‫In this case, we're using the Elasticsearch image. You can see the command is a cURL localhost
‫9200, which is the port that the Elasticsearch is running on inside the container, not the published port,
‫but inside the container. For Elasticsearch,
‫there is an actual health URL.
‫So, we can use that here.
‫You'll notice the two pipes with the false at the end of that command.
‫And that's going to be pretty common
‫if using something like cURL or another tool that will send out an error code that's other than 1.
‫Remember when I mentioned that while ago? We need it to exit with 1 if there's a problem. Because that's
‫the one error code that Docker is going to do something about.
‫We need to make sure that in this case, a shell will always return the false 1 exit code
‫if there's anything coming out of that command other than 0.
‫It's a nice way to get around that problem.
‫It just so happens with cURL, cURL will give other potential error codes and we don't want it to
‫do that.
‫In the actual Docker files, we can add the same command. The format's a little bit different. But you see
‫that we have these options here. We have the interval, the timeout, the start period (which is new), and retries.
‫The interval is what you would think it is. It's, by default, every 30 seconds.
‫How often it's going to run this health check. The time out is how long it's going to wait before it errors
‫out and returns a bad code, if maybe the app is slow. The start period is a new feature that allows us now
‫in 17.09 and newer, to give a longer wait period than the first 30 seconds of the duration. Before, it
‫would always just wait the long...the interval time before it started the healthcheck.
‫But maybe you have a Java app, or database, or something that takes a lot longer to start.
‫Maybe it takes five minutes.
‫You could add that start period in there. It'll still do healthchecks. But what it will do is it won't
‫alarm on an unhealthy check until that time has elapsed.
‫So if you set two minutes in there, even though it's health checking every 30 seconds, it's going to only
‫consider it unhealthy once it's past that two minute mark. The last one there, retries, means that
‫we will try this health check x number of times before we consider it unhealthy.
‫That gives maybe a potentially unstable app a chance to come back with a healthy and recover on
‫its own before we consider this a truly unhealthy container.
‫The basic healthcheck command you would use in a Dockerfile is called HEALTHCHECK,
‫all capital letters there. The same format exists where if we're just doing a simple cURL of the localhost
‫because maybe it's PHP app or something. We can do that.
‫This is how you would add all those options in to a Dockerfile so you would see how I add the
‫timeout interval and the retries before the command itself.
‫The first one there for the basic command, notice I don't have to put in a CMD if I'm just giving it the
‫command to run. But if I want to show options, if I want to give it custom options out of the box with
‫the timeout and so on, then I have to specify which one is the command.
‫Now these aren't two different lines.
‫Notice the back slash on the end of the first line there.
‫So don't get that confused.
‫Here we have a simple example of what it might be like if you had a static application running inside
‫an Nginx server.
‫You could set the interval and the time out from your Dockerfile, and you would just have it simply
‫do a cURL command on the localhost.
‫If it returns a 200 or 300, it considers that fine. If it returns a 4, or 5, or something else, it considers
‫that an error.
‫You notice here that I have an exit 1, which is the same thing as a false.
‫I did that just to show you that certain examples on the Internet will have a false. Certain examples
‫will have an exit 1.
‫They both do the same thing.
‫Here's a little bit more advanced example.
‫In this case, we're using a PHP app that's combined with Nginx.
‫What I've done is, in the resources, you'll find a link to this PHP example. I've added in a custom
‫Nginx config file that uses Nginx and PHP-FPM status URLs.
‫Both of those applications have their own status page and sort of a healthcheck ping URL.
‫You can use those in your apps if you're using PHP or Nginx. There are two different URLs,
‫but you can use both of them inside the same healthcheck.
‫In this case, we're using just one of them, and we're throwing in the localhost/ping, which is
‫actually a PHP-FPM
‫status command, but you have to enable that inside your PHP-FPM. Again, in the resources of this lecture,
‫there's a link to a PHP Docker Good Defaults.
‫You can go check that out on a GitHub where I've shown in this example in a little bit more detail.
‫Next we have a Postgres example so in the Dockerfile I can use a different URL.
‫Here we have a Postgres application where in the healthcheck command, I'm using a command of pg isready.
‫Now, with different apps, there's different tools. With Postgres, it comes with a built-in tool,
‫that's a very simple testing of a connection to a Postgres server.
‫It doesn't validate that you have good data, or that your database is mounted properly. It's simply
‫going to say, 'Does this database server allow connections? Yes or no?'
‫That's a neat one that you can do out of the box.
‫Here's what it would look like in a composer/stack file.
‫Very similar.
‫You'll notice that the start period down there requires a different version.
‫Since the healthcheck command came out in 1.12, it was actually supported in 2.1 of this Compose
‫file.
‫But, if you're going to use the start period, that means you have to update your Compose file to version
‫3.4 in order to support that. Because the start period came out over a year later after the healthcheck
‫command did.
‫Let's start out with some simple run commands.
‫What we're gonna do here is we're going to start a Postgres database server without the healthcheck
‫because by default, it doesn't come with one.
‫Then we're going to run it again with a manual healthcheck command that will add at the command
‫line, and we'll see the difference.
‫Here, we're just going to call the first one p1. We'll run it detached from the official Postgres
‫image.
‫If I do a docker container ls, you'll see that there's nothing indicating a healthcheck here.
‫If we do that same command again, and call it p2 this time, we're going to add a health command.
‫This time, we're going to use the pg isready, which we talked about earlier,
‫to test that the connections are available on this Postgres server. We're going to tell it that the
‫user we need is the postgres user. We don't actually need to give it a password. It's not going to try
‫to log in. It's just going to try to validate. We'll use the Postgres image.
‫Now if we do a docker container ls, and I zoom out a little bit, you'll see that it says, 'Up 4 seconds
‫health is starting.'
‫Now we get this additional feature in our status of our ls command. It will stay in the starting
‫state for the default
‫30 seconds until it runs the healthcheck command for the first time.
‫Now that we've waited over 30 seconds, you'll see that it's changed to status of healthy.
‫If we do a docker container inspect on that p2, we'll see at the very top of that that we had
‫this new health status option. In this case, I've only been able to run it twice.
‫You can see the output there, that it's showing it's accepting connections.
‫All right. Let's do some service create commands to that same database, in that same test healthcheck.
‫What we'll see here when we do this is that there are three different states that a service goes through
‫on starting up.
‫It's preparing, which usually means it's downloading the image. It's starting, which means it's executing
‫the container and bringing it up. Then it's running.
‫Without the healthcheck command, the starting and running are very quick. They're almost instantaneous.
‫We'll see that here with a docker service create name p1 postgres.
‫Once it's done preparing by downloading the image, you'll see that it goes immediately from starting
‫to running, because there is no healthcheck. It doesn't have anything else to do other than start the
‫container and say, 'Yep. The binary is running.'
‫But if we do that same command, docker service create, and call it p2 like before, and give it that same
‫health command.
‫We start this service with the healthcheck command built in.
‫What we'll see is that it'll go from preparing to starting, and it will sit at the starting state for
‫the default 30 seconds until the first healthcheck runs.
‫This is now the Docker service expecting a healthy state before it considers this service fully
‫running. After the 30 seconds is over, it'll shift to the running state. Then we get the last little
‫verify there, just to make sure that it's considered stable, and then we're done.
‫You can already see, out of the box, that with services, as well as service updates,
‫we're going to get this extra bonus of health concept if we use these commands whenever we can.