‫So enough with the announcements.
‫I want to get to a topic that I've been working on and focused on from my Node.js course.
‫And that is environment variables so like raise your hand in chat and you can by the way in chat you
‫can do a colon wave colon and then it will actually do a little wave hand wave your hand if you're dealing
‫with environment variables in your apps and you'd like easier ways that you're not exactly sure if you're
‫doing the best way for environment variables.
‫Because I struggled with that for quite a while.
‫I mean not just in containers but in all of distributed computing making sure that all of my apps have
‫at least some consistent way to do environment variables.
‫Getting information secrets your you know server your server DNS names you know you name it all that
‫stuff into your containers
‫so all right.
‫So it looks like a few of you are interested in this topic.
‫So this will be part of my node.js course.
‫And then I plan to take some of that and figure out how to release that information to the Web.
‫There's a couple of ways that you can go about it but the basics of this come out of 12 factor apps.
‫And if you haven't heard about a factor app it's been around for a decade or more and it's a concept
‫about 12 factors that make a distributed app more successful essentially and it's from a company that
‫learned early on in cloud and on the Internet how they needed to manage all the different points of
‫their app and how they needed to know the different concerns they had around best practices essentially
‫for writing distributed apps.
‫And chances are if you've got a database on one server and an API over here and a Web site over here
‫and a worker job over there but lets a distributed app right there's lots of definitions for distributed
‫that but generally most of us today if you're concerned about containers and you that's part of your
‫job and you're running them on servers you're probably dealing with some distributed apps.
‫So the interesting part about this is the config state.
‫So if you go back to the twelve factor dot net paid to throw this into chat you can definitely skim
‫through these later and then dive into each topic when you have to deal with that in your app.
‫But the one that I wanted to talk about today was the config and config was about storing the settings
‫for your environment.
‫So as you learn if you're taking my dog or Mastery course you know that you never want your your your
‫images or your apps themselves to have hard coded settings that would be different for each environment
‫and the environment might mean you're like a machine that the dev test server or the staging server
‫or the production server or if you like a lot of us you have multiple levels of production and multiple
‫copies of production like all these things.
‫If there's anything that might change in there that might be memory settings in Java that might be different
‫passwords for an API.
‫Different keys that might be storage keys for your cloud.
‫It could be anything from just the word of maybe those in your app is a little flag that indicates which
‫part of the system you're on and you need that set in a simple little environment variable.
‫All that stuff needs to be pulled out of your app and one of the rules of twelve factor is if strict
‫separation of config from code right.
‫And we've all seen that we've all probably done it putting you know hard hardcoded value like local
‫host because your app is always running on the same server as your database but that's before containers
‫then you might see a hardcoded app that says local host in code right.
‫And that's unfortunate because that makes them data that makes them very rigid and hard to deal with
‫because now you got to change code just a changed environment so you don't want to do that.
‫And one of the ideas of this is that twelve factor apps and if you're following the twelve factor principles
‫you tend to have a very easy time making them into containers the twelve factor apps ideally are using
‫everything through environment variables.
‫The reason is is because every OS and every system all supports that right.
‫If you have a Java app over here and you've got a no GSM over there and then you've got a database app
‫back here that's storing your databases and you've got a MIM caching all these things.
‫Every one of those has a different way of consuming its values its environment variables right.
‫The settings it needs the config it needs to get start up.
‫Those are all different.
‫And so that can be quite frustrating.
‫So the only thing that's consistent between all of them is environment variables.
‫Every shell supports it.
‫Windows Mac Linux they all support the same functionality.
‫It might be slightly different the way you set an environment variable or clear it but the key value
‫is the same and all of them.
‫Right.
‫So that's a core premise and that's what you can see throughout the doctor and the container ecosystem
‫as we've all adopted that as our standard.
‫Now what I means is is like if you have for example an application that has a demo file or an I and
‫I or some other sort of config file that the values need to get to then you have to write a middle layer
‫between the environment variables and then where they're going to go into the app.
‫Right.
‫And I have an example of this in my not so complete BHP app.
‫So over on GitHub in my PSP app images show you a quick literal example.
‫And then we're gonna get to your questions.
‫So in here what I have is a docker file and this sacrifice there's a lot of things but the things I
‫want to focus on is in this level app I have defaults that I set in the docker file for environment
‫variables.
‫Now the reason and the way that I'm setting them here is I prefer that the settings here default to
‫working in a production environment up to you whether you say hey look these defaults are just gonna
‫be the average production values or they might you might say hey this is gonna be the average developer
‫values I prefer to them sort of defaulting a little bit to the production side of things that way I
‫can override them with a compose file locally in my local machine no problem but it's up to you how
‫you do that but you definitely want to create consistent ones here that makes sense from the get go.
‫Now you're gonna overwrite a lot of these with other ways of setting it either through the docker run
‫command or through deploying a swarm or a Cuban network deployment or in your Ducker compose GMO file
‫you will overwrite those.
‫Right.
‫You can even overwrite these at the shell before you run those commands and it will suck those in.
‫You can also you using something like Docker compose you can set a dot EMV file that in the PSP world
‫is actually a standard that it will work in such that in the gamble as well.
‫So you can see that I have these here but obviously some of these go into the app they might go into
‫different files and let's back up a second
‫and I'll show you the next part.
‫So those are embedded in the doctor file when the image is built right now.
‫When I want to look at my doctor compose file this is going to be used for local settings right.
‫When I'm doing development and then I can look through here and you can see that I'm setting them all
‫and maybe either setting them differently.
‫I don't have to override them if I don't want to.
‫But in this case I'm choosing to set similar ones as an example.
‫If I didn't need to override the ones in the image I might just leave a lot of these alone.
‫So there's that.
‫All right.
‫So I'm setting them here and now these will be used on container startup.
‫So if I wanted to change them I have to restart the container.
‫I don't have to delete the container right.
‫I just had to restart it.
‫So you can use Dr. composed to do that then the next step is maybe let's see
‫maybe on my local machine or even in production I need those to be in specific files.
‫Right.
‫So when my kid my application starts up those environment variables need to be somewhere else.
‫They can't just live in environment variables for some apps right.
‫I would call these legacy apps or traditional apps where they're not looking to environment variables
‫by default.
‫Now if I was creating a new app my goal would be that that looks to environment variables and maybe
‫there's some config file or something that the app uses but I'm overriding those with enviroment variables
‫automatically.
‫So the idea is my app is smart enough to say hey if there isn't an environment variable for this I will
‫use this default from this file.
‫But if there's already an enviroment variable set I will use that.
‫Now if you're familiar with swarm and Cuban net is you know that once we get into secrets and configure
‫we can start overwriting those entire files and providing passwords and secrets as individual files.
‫So you also might want your app to be aware of those.
‫But if your app if you're not building your own custom app or you're using maybe stuff from Docker Hub
‫or official open source then what you need to look at is providing an entry point script.
‫Now the way that gets run is in your image you set the entry point script you want it to run and it
‫will run that at startup.
‫And what you can see here is a very simple example of me writing the violent variables and some other
‫hard coded data into a specific file on startup simply using a bash script.
‫Right.
‫Does have to be fancy.
‫You can use other methods to do this as well but it tends to be in the docker official images.
‫This app tends to be the standard.
‫All right.
‫So in this case I'm taking environment variables that were set either in the docker file or the compose
‫file which overrode the image Docker file it'll take those Passim into here and then this will echo
‫that stuff out to a file.
‫It's about image.
‫So this is for a legacy HP app that expects a p HP FP M config file and I need to set certain config
‫values for NPM that are unique to each environment in this case.
‫These are actually this how how much I want the FGM system to grow and how much how many sub processes
‫I want to spawn stuff like that.
‫Right.
‫So that would be different on the local development versus production on production I want it to be
‫to grow a lot.
‫Right.
‫So you would change that stuff here and this is how you take those legacy apps and make them work.
‫Now you can see this stuff in official images because that's how a lot of my stuff I learn from.
‫So if I just went into the my sequel you can see how they did it which is exactly the same way my sequel
‫does needs things in a config file.
‫Right.
‫It needs settings that are dynamic at runtime.
‫So if you looked in the docker file for the official my sequel and you scroll down you'll see that yes
‫it does set some environment variables here but it also has an entry point script that it copies in
‫it links it to a specific location and then it as it is the entry point.
‫So the command here is seeing the CMB you can't really see it.
‫The CMB is for just my sequel the entry point technically runs is before that and then it passes execution
‫to the CMT.
‫Once that script is done running now the way that I find that script in here is I just go up one level
‫and get hub and I click on this eight point so and then I will see an environment variable
‫see
‫and now we have the shell script that's run every time my sequel starts all right.
‫And in here it does things like take environment variables and make them available for the app.
‫In fact it's smart enough that if you add a little underscore file on the end of an environment variable
‫it will take whatever is in that file and turn it into the environment variable which is great for all
‫the applications that still need secrets an API keys and all that stuff.
‫And if you're using swarm secrets or Cuban entities secrets those things are provided as they can be
‫provided as files.
‫Now you can always use stuff like vault and other fancy systems that honestly file based but in containers
‫we tend to use things as files and Linux in general we then use things as files.
‫So you're passing these things in as individual files.
‫And then this little bit of shell script which is open source which means you can copy it and put it
‫into your app you can use that to copy the environment variables out of the text file and then stick
‫each one of them in an environment variable named appropriately and that way the app down later can
‫have environment variables that are written in to config files for the app and everything's wonderful.
‫Right.
‫And this would change on every start of the app because the inherent variables might be different.
‫And then it would rewrite those and that's how that works.
‫So hopefully understand those steps.
‫There's a lot more to it.
‫There's actually a let me get you real quick.
‫There is a great little summary of dealing more with environment variables inside of compose it's article
‫I wrote even get the link here.
‫Yeah this is over on the Riley site.
‫And I did a nice blog article for them.
‫Well I think it's nice.
‫But you know it's up to you to say give us lies.
‫And it talks about Dr. Campos because a lot of what we're dealing with for developers especially is
‫the local config.
‫How do I make my config specific for me.
‫And one of those things is an override file which I unfortunately don't have in this HP example I don't
‫think I don't have an override file but I should.
‫That would be another thing to add into here.
‫But in here I will put that in check.
‫Check that O'Reilly link out to talk about three different things and compose that will really help
‫you up your game on managing configuration and reducing replicated parts right templating means you
‫can add specific parts in the top your compose file and then repeat without having to repeat those over
‫and over again so if you have lots of the same environment variables because you're doing micro services
‫and you have maybe 20 micro services in a compose file templating is for you.
‫This is another thing that will really save you some time and you can have all the same parts that need
‫to be repeated over and over again at the very top of your compose file and then it's this is just a
‫gamble thing really the templating and then you can then insert that on each service at the appropriate
‫time.
‫The way that I do here there's other stuff in here as well I encourage you to check that out including
‫how to use its environment variables inside of composed files not just for setting environment variables
‫in the image but also anything else you want to change in your config file at runtime so check that
‫stuff out.
‫I hope you like it Michael.
‫What's up Michael.
‫Michael Ahrens on the call.
‫Call me.
‫What is this conference call there.
‫Dr. Captain.
‫Often what should not go into environment variables.
‫Who are you baiting.
‫The question.
‫To see what people come up with.
‫What should not go into environment variables.
‫Well so the run of the reasons that Docker and Cuban entities and other tools have created these ways
‫for us to put secrets in the files is so that they don't have to be an environment variables.
‫So ideally if you're creating your own apps for managing your own apps you're keeping those secrets
‫out of environment variables not because necessarily environment variables are inherently insecure it's
‫because we with environment variables we tend to leak them right.
‫They tend to they tend to show up in logs because maybe we do an environment variable dump somewhere
‫in our config or we do.
‫If you don't like a P HP was it there's a p HP test command PSP EMV or I came in what it's called anymore
‫someone I'm sure on here is and remember and tell me but where it'll dump your configuration to a web
‫page there's stuff like that and then if there's any secrets in any environment variables they will
‫be exposed there and you may not intend that especially they leak into logs or in the monitoring system
‫somehow and you.
‫If you have errors sometimes you know if you've ever had applications where if you set it to debug and
‫errors out this is the true thing in dot net where it will then put a whole bunch of variation on the
‫screen and might leak environment variables on the screen because you forgot to lock it down into production
‫mode there's all sorts of ways environment variables can end up in the wrong hands.
‫So a good a good first step on not putting stuff into environment variables would be the secrets and
‫with swarm and communities you can keep those in files and that way you can teach your app how to just
‫get them out of the files.
‫The next step there is you if you can get them from your secrets and config files into the files that
‫the app needs.
‫So instead of storing an environment variable that then eventually gets like I should example eventually
‫gets written to a file you could use can figs and store the configure and secrets and files and then
‫they go directly to the files that maybe my sequel or your app need.
‫Right.
‫So they never technically get into environment variables.
‫So at least not on the server itself.
‫Right.
‫So that's a great topic I'm glad you brought that up.
‫But the thing is is I could probably write an entire course on config secrets environment variables
‫but basically all the things you need for an environment and how to deal with them.
‫Right.
‫And I think we're even if you go to stuff like vault for you and now you're talking about a secret API
‫that you're calling it even gets more complicated.
‫So definitely there's lots to talk about there.
‫And if anyone has good resources that they like feel free to throw and chat or throw them in the comments
‫of the YouTube video for later.
‫All right good question Michael good to see you.
‫Why would would you.
‫Stephen says Would you Ducker file not be in source control though I don't favor having production creds
‫such as user name password and source control.
‫That's why those values are just defaults inside of your doctor files Steven.
‫You wouldn't put real secrets in that file.
‫Right.
‫As you can see the one in my example it's just over here everything in the dagger file is just examples
‫or samples.
‫Now if it's a simple number or something that just needs to be set and there's a default a sort of reasonable
‫default like these right here these p p p.m. settings then sure put in the production ones that you
‫prefer and then you can overwrite them as needed.
‫That way you don't have to set them every single time manually.
‫But yeah if you go down here to any of these ones that are unique to a specific app or a specific environment
‫I'm just putting in dummy values here.
‫I've definitely don't recommend that inside of your code repo.
‫You put in your secrets and your your URLs and environment you know the domain name settings and stuff
‫like that.
‫All right.
‫I agree but your Ducker file itself would be in with your app.
‫Yeah that's where you want to keep it.
‫All right.
‫Good question.
‫Good question yeah.
‫I mean unless they're encrypted right you could do hashing on them if your app accepts that sort of
‫thing.
‫But I still wouldn't keep them in there.
‫I don't I keep environment settings elsewhere but you do need defaults.
‫And what I tend to do in a compose file with Docker compose is I set defaults that work with a local
‫developer out of the box.
‫So Docker compose up would just work for them.
‫Now it may mean they need to get a dot in V file from somewhere else maybe a secure store where you
‫have sort of developer secrets you know that sort of thing.
‫But I really work hard to also not include that like ideally if you're local developer you have everything
‫you need locally.
‫You don't require remote api as remote things and you hopefully have sample data and things that can
‫just be on your local machine and you don't need a lot of secrets that you have to worry about you just
‫have dummy values for everything and you can just get buy locally.
‫Obviously not all of our apps can do that as sort of a utopia.
‫So you're going to come up with a different way to store those secrets and that's why there's so many
‫these ways to set environment variables at runtime so that you can keep them elsewhere and then just
‫get them in at the last second through the environment variables all right good questions on environment
‫verbs.
‫I'm glad we've got this topic today.
‫Let's see
‫what is the simplest way to implement the E and V file.
‫As well I showed you the example of the my sequel app right.
‫And in here they take this script and they there is no EMV file.
‫What this is designed for is this is when you use secrets or config.
‫So if you're using the native secrets and config features of Docker swarm or config maps and secrets
‫in communities you would maybe use this as a way to get those the information out of those files specifically
‫the secrets really and get them into environment variables.
‫If you need them right you may not need to get them out into environment variables but if you did then
‫you could use this little bit of shell script to do that.
‫I hope that answers your question but I wouldn't be.
‫There's no need to create a specific file.
‫Exactly.
‫If you're talking about how to do swarm secrets because swarm secrets create files for you then check
‫out my Docker mastery course at the URL.
‫Brett Fisher dot com slash talker which is down in the description.
‫I talked about that there as well in my swarm course.