News for hackers. Stuff that matters.
Re:Twelve-factor app development on Google Cloud by ethagnawl on Thursday October 31, 2019 @11:41PM (#21416106)
> A better approach is to store configuration in environment variables.

It's about damn time, Google! I've been working on a number of GCP projects this year and have consistently bumped up against the awkwardness that was Google's (former?) preference for configuration files over environment variables.

Re:Twelve-factor app development on Google Cloud by emmelaich on Thursday October 31, 2019 @11:47PM (#21416125)
This is the weakest or least well-defined aspect of 12factor.

How can you 'store' anything in something as ephemeral as environment variables? Where do these environment variables come from? When are they set?

You should of course store everything in version control; the point of this guideline is that the scope should not be the same as the app or an instance.

There is particular tension between this guideline and X. environments should be a similar as possible and I. there should be a single codebase for all deployments.

Re:Twelve-factor app development on Google Cloud by tybit on Friday November 01, 2019 @02:50AM (#21416698)
Another option is rather than storing configuration in config files, is to store it in code. After experimenting with different approaches, storing configuration in code and switching on a single environment variable ENV (or STAGE or whatever you like to call it) this is probably my favourite.

This doesn't work for everyones infrastructure setups, but it's worked well for mine.

Re:Twelve-factor app development on Google Cloud by icebraining on Friday November 01, 2019 @06:17AM (#21417417)
A file that stores configuration is a configuration file, even if the encoding format happens to be the same as the main programming language and not toml or yaml :)

That works fine if there's a limited number of environments, but it becomes a pain if you're running the same app in too many, or if someone else needs to deploy it independently of the developers.

Re:Twelve-factor app development on Google Cloud by solipsism on Saturday November 02, 2019 @05:09AM (#21426645)
it becomes a pain if you're running the same app in too many,

No it doesn't. It's trivial to generate the full set of configs using a script, if you have too many to manage manually.

or if someone else needs to deploy it independently of the developers

And how exactly do envvars help with that? The set of envvars still needs to get pushed with each deployment. envvars are just a special config file.

Re:Twelve-factor app development on Google Cloud by mixedCase on Friday November 01, 2019 @01:19PM (#21421318)
Convenient enough for dynamic languages. Not so much compiled ones.
Re:Twelve-factor app development on Google Cloud by emmelaich on Thursday October 31, 2019 @11:50PM (#21416134)
Also ... it's a bit circuitous.

> requires strict separation of config from code. Config varies substantially across deploys, code does not.

What's the difference between config and code? Config is that which changes between deployments / environments.

Re:Twelve-factor app development on Google Cloud by pushrax on Friday November 01, 2019 @12:27AM (#21416276)
Config also benefits from review and version control, and it's inconvenient to use multiple systems for these. It's been working out well for us to store secrets encrypted with public key crypto alongside source, or in a deployment repository using the same source control. The private key is inserted into the infrastructure as a K8s secret.
Re:Twelve-factor app development on Google Cloud by thomascgalvin on Friday November 01, 2019 @01:00AM (#21416391)
One of the concepts twelve factor pushes is that you should be able to release your source code right now without compromising any credentials.

Basically, you'd have a git repo for code, and a separate git repo for environment scripts. I've seen organizations use a whole separate infrastructure for config, but that always seemed like overkill to me.

Re:Twelve-factor app development on Google Cloud by pushrax on Wednesday November 06, 2019 @09:42PM (#21469910)
This is true with encrypted credentials, if security is compromised by release of the repo then the encryption is not secure. I think the spirit of the principle is met, and in cases where you do later release the source code without having planned for it, it's easy enough to move the secrets.

It's definitely more convenient to use the same repo, especially in the declarative containerized world; you can make sweeping changes or roll them back in a single atomic commit.

That said, if open source is the intent, certainly use multiple repos from the beginning so you avoid git gardening when releasing later.

Re:Twelve-factor app development on Google Cloud by unlinked_dll on Friday November 01, 2019 @12:32AM (#21416298)
>How can you 'store' anything in something as ephemeral as environment variables? Where do these environment variables come from? When are they set?

Set at build time. They come from a build config file/script as part of the build. They're not perfect, but just about every tool ever made has a way of understanding environment variables - the same can't be said for any given config file.

Here's a real world example: I want one source of truth for my build configuration, but my build itself pulls together libraries and binaries in a number of different languages and build systems. How would you handle that with anything but environment variables and a master shell script?

Re:Twelve-factor app development on Google Cloud by emmelaich on Friday November 01, 2019 @01:41AM (#21416519)
> They come from a build config file/script as part of the build.

But the script is in VC surely?

> How would you handle that with anything but environment variables and a master shell script?

Again, where is that master shell script? What args do you give it?

My preference is that you have one environment variable, and your config is keyed off that.

If that environment variable is not supplied, fail noisily.

Re:Twelve-factor app development on Google Cloud by alchemism on Friday November 01, 2019 @06:18AM (#21417421)
Kubernetes assists with this aspect of passing environment variables to immutable containers. There you add the variables to the deployment code or edit them live on active deployments.

You can take it one step further and define a single ConfigMap resource per environment, containing all of the required variables across your application, and then have all the various containers reference it ‘blindly’ when they are deployed. Said configuration resource would exist in version control but deploy independently from the functional codebase(s).

Re:Twelve-factor app development on Google Cloud by snicker7 on Friday November 01, 2019 @05:26PM (#21423867)
Configuration isn't stored in environment variables. Rather, they are exposed as environment variables.

The idea is that the application environment / infrastructure should own the config data, not the application code itself. 12 factor practices require that devs draw a boundary between app and infrastructure.

Re:Twelve-factor app development on Google Cloud by lostcolony on Friday November 01, 2019 @12:22AM (#21416257)
I actually had this question earlier today.

Given a config file per 'environment', with environment being an arbitrary identifier, what benefit do environment variables have over it? I've read some docs on 12 factor apps looking for an answer but didn't really find anything. Just curious what I'm overlooking

Re:Twelve-factor app development on Google Cloud by paulddraper on Friday November 01, 2019 @02:12AM (#21416599)
Env vars are easy and ubiquitous.

1. Bash, Python, C, Java...super easy to access; no libraries needed.

2. They automatically propagate through shell scripts and worker processes.

3. They are also super easy to specify when starting programs, in shell, docker, child process from programming languages, Lambda functions, etc.

4. They can easily exist independently of other env variables on the same system.

5. The model is dead simple. Key-value. Simplicity is a feature.

If you need to specify something complex (e.g. hundreds of tuned timeouts), consider using files or code for the heavy parts and env vars to select parts.

Re:Twelve-factor app development on Google Cloud by lostcolony on Friday November 01, 2019 @10:51AM (#21419423)
Thanks! I had thought of about 3 of those, but they aren't really wins for most of my deployment environments (and some of the downsides still are major downsides for me; one thing I love about config files is I can provide a readme or schema in the same place to define what all the config params are and what they do).

For #4, what do you mean? If running in a container it's not really an issue, but if I'm loading a couple of different dependencies/apps in the same container, or running without containers, name collisions have always been a concern. Set up a chroot, or start renaming things (i.e., "PORT" conflicts really quickly). Or are you saying supply them in the shell, or...?

Re:Twelve-factor app development on Google Cloud by paulddraper on Friday November 01, 2019 @11:24AM (#21419773)
> For #4, what do you mean?

I mean that one process can have COLOR=blue and another can have COLOR=red with no conflict.

    PORT=123 COLOR=blue ./myapp
    PORT=456 COLOR=red ./myapp
Yes you can chroot, run in containers, use only relative file paths, etc.

There aren't any irremovable differences between env vars and files; env vars are basically /proc/ files with widespread runtime support.

Re:Twelve-factor app development on Google Cloud by manigandham on Friday November 01, 2019 @12:56AM (#21416372)
Env vars are universally supported, globally available, and simple to manage. Files mean yet another deployment artifact that has to be stored somewhere, and now tied to a location.

Sometimes that location is communicated through an env var, and that point why not just set all config that way?

Re:Twelve-factor app development on Google Cloud by AlphaSite on Friday November 01, 2019 @01:02AM (#21416402)
If you have something like k8s it becomes trivial to use files vs end vars.
Re:Twelve-factor app development on Google Cloud by manigandham on Friday November 01, 2019 @03:08AM (#21416771)
How so? You still have to edit the file, save it somewhere, then mount it as a volume and agree on a location.
Re:Twelve-factor app development on Google Cloud by yencabulator on Friday November 01, 2019 @11:50AM (#21420078)
The configuration file can be inline in the yaml, ConfigMaps take care of shuffling bytes around for you, and for a specific app it's config can always be at a fixed path.
Re:Twelve-factor app development on Google Cloud by jaysh on Friday November 01, 2019 @12:47AM (#21416351)
As long as your (production) config files are not checked in with the code, then little difference. The challenge then is: where do you store them?

Environment variables had the early upside of being first class in any virtually any deployment strategy - from Bash wrappers, supervisord to Docker and Kubernetes - so it's pretty much plug and play. For example, Docker has config files, but they're mounted to a specific place in your container, so that means your application is now married to that location.