‫What do you think about distroless images?
‫Is it a good trade off between size and security?
‫I would say if you can do
‫distroless images, and what we mean by this, the term
‫is in flux.
‫I feel like distroless images
‫is complicated. I feel like just saying scratch images
‫might be an easier way to say it. But, it is correct to say
‫distroless image, meaning there is no base package
‫system. There's no apt. There's no Yum.
‫There's not all those different commands that you would
‫assume are in a container if you use one of the official
‫images. The thing is, distroless images
‫aren't very common because of two reasons.
‫One, most programs need all that
‫stuff in order to work.
‫They need apt package dependencies.
‫They need their own open
‫SSL libraries. They need all sorts of stuff in addition
‫to the program that's in there, right.
‫That's most applications today.
‫Golang, C, some other languages
‫can build single binaries
‫that basically have everything they need to run, other
‫than the kernel. Those things
‫are technically able to remove everything below it.
‫All of the layers and your image could go away.
‫I don't see them a lot in the wild yet because I don't work
‫with Google, and Netflix, and
‫huge teams of people that are super professional at this.
‫The two reasons here is, one, you have to have a language
‫that supports it, and you have to have a team that
‫understands how to build these properly.
‫Then, you have to really be good at troubleshooting and
‫how containers in namespaces work.
‫Because the minute that distroless image runs
‫in a container and you have a problem, you can't even shell
‫into it.
‫There's no shell. You can't get into it.
‫You can't look at it. So, now your tooling
‫is limited, and you have to start looking at other ways
‫to connect your tooling into those containers.
‫That exists. There are those things out there that will do
‫that for you, but it largely depends on your skill set
‫in terms of how you deal with namespaces.
‫How can you jump in as a sidecar into a running
‫container in its namespace from the command line without
‫using Docker?
‫Those are the kind of problems that creep up when you're
‫dealing with troubleshooting or system monitoring and
‫maintenance when you have distroless images.
‫For everyone that I work with that's starting out with
‫containers, their first couple of projects in containers, I
‫don't mention this stuff because size is just size.
‫You can always add more disk space to a server, right.
‫No reason to make a whole team run around learning all
‫sorts of new tooling and stuff just so that
‫you can have less disk space used on a server.
‫I personally don't care about size.
‫Security is a theoretical thing.
‫In theory, all those things
‫getting removed from a container image might make it more
‫secure. Let's talk about out in the wild, though.
‫How many opportunities does someone have to mess
‫around with other tooling in your container outside of your
‫app? Let's say that cURL has a vulnerability
‫in it, but your app doesn't use cURL.
‫It just happens to be in the image.
‫How is someone going to get access to that, right?
‫They're going to have to break through your app.
‫So, your app has to have an exploit.
‫Then in theory, they could run the cURL command in the
‫container, right. So, that's really what this distroless
‫stuff is about is the theoretical
‫security bonuses of not including other applications.
‫The reality is, though, once you build a static binary,
‫you're already including a lot of those libraries that
‫could be vulnerable like open SSL, which is a
‫common way to connect applications with
‫SSL security, right.
‫Well, if that has a vulnerability and you statically build
‫it into your app, it's still there.
‫It's just not a separate file on the container.
‫It's in your app in a single binary.
‫In fact, it actually makes it harder to security scan that
‫because now, every scanner that I know of won't
‫catch that open SSL binary if it's
‫built in. So, a lot of this stuff is theoretical.
‫I don't tend to push teams toward theoretical
‫security features. I look more at
‫look what's happening in the wild, right.
‫Docker daemons are getting exposed.
‫Kubernetes doesn't have proper authentication, so it gets
‫exposed. Those are the real problems that I see is, you
‫know, the things that focus on are the things that we keep
‫hearing about in the news that people are getting hacked
‫with, right. This kind of stuff with theoretical security
‫in removing files from images.
‫That's just more in the theoretical realm.
‫It's not a reason not to do it.
‫Just like this list, everything in this list,
‫I would do first before I would start going down
‫that road of hard stuff.
‫I do consider distroless images relatively hard.
‫Unless you're already a Go programmer and already
‫your whole team is really good at namespaces,
‫and Docker commands and stuff like that.
‫I don't just mean regular Docker commands.
‫I mean complicated stuff that helps you troubleshoot
‫containers that don't have shells in them, for example.
‫So, it's a great question.
‫It comes up all the time.
‫Everyone asks me about it when we have security discussions
‫because it's a popular topic. But, it's not even in my top
‫ten.
‫I hope that helps.