The authors of the 12 factor app methodology seem to disagree:

"Another aspect of config management is grouping. Sometimes apps batch config into named groups (often called “environments”) named after specific deploys, such as the development, test, and production environments in Rails. This method does not scale cleanly: as more deploys of the app are created, new environment names are necessary, such as staging or qa. As the project grows further, developers may add their own special environments like joes-staging, resulting in a combinatorial explosion of config which makes managing deploys of the app very brittle." -http://www.12factor.net/config

I don't think the two approaches are as far apart as they appear at first glance. Later on in the article they talk about removing Rails.env.production? from their code.

The twelve factor app calls for strict separation of config from code. I suspect that 37 signals has that, or pretty close.

Actually storing the config in environment variables is a Heroku-ism. There are other ways of doing it; chef is another good way. What matters is the strict separation.

I think that this is right. I also think that storing configuration information in environment variables doesn't scale past a handful of settings. I'm also not sure that it's the best way of storing secrets. If you're working in a real SOA environment, you may have dozens of external services to talk to, and possibly secrets for each one. At that point, you probably can't use environment variables. And, even if you do -- you'll be managing them in some sort of persistent storage, probably with versioning.

Another advantage of using environment rather than a config file for configuration is not leaving a file full of secrets lying around in your environment. And why is loading your environment so difficult? If I can put foo=bar in a config file, I can put export FOO='bar' in a startup script, and delete the startup script once the application has started.

This is wrong and dangerous. From whom are you trying to keep these variables secret? Assuming a sane sysadmin, sensitive files would only be readable by the user which needs to run the application. And environment variables can be read from the procfs file system by the same user that launched them: http://serverfault.com/questions/66363/environment-variables...

Not to mention that if an attacker has shell access to your application user account, you're already owned. So again, who are you trying to hide these variables from? Because it seems like this is just security theater and additional inconvenience (why delete and recreate the file on each deploy?) for literally no benefit.

The custom configuration part of that is the most interesting. Sadly, we have a lot of Rails.env.production? scattered everywhere. I'm not sure I understand the examples DHH provided though.

He provides a more complete example on the github page: https://github.com/dhh/custom_configuration/

I have written a configuration plug-in that I think is slightly better (for example allows local overwriting of the settings, not just per-environment):

https://github.com/jaroslawr/dynamic_configuration

Yes, that is nice and all, but could we please remove the assumption that there will only be one database.

Yes, it is possible to work with several at once, but automating the setup of an environment that have several databases, each with migrations, is no fun at all.

What would be really nice would be to have one list of databases, and one map between environments and databases.

Is anyone else deeply concerned that 37signals puts actual production data into their non-production environments (which are inherently insecure given the nature of those environments and who needs access to them)?

We treat all environments as production. However we add additional security to staging since it's the most experimental. (Additional security meaning it can only be accessed via VPN and after two factor authentication.)

To be very clear: Beta, production, rollout and staging environments are secure. They run all the same front end security provisions, same data center, same patches, etc. The difference is they run differing versions of the code base based on development.

No. If your non-production environments are less secure than your production environment, then you're doing something wrong. In fact, non-production environments generally have a much smaller population of users and they are tend to be more security minded. Or should be.

You're missing the point. In non-production environments a large user base (eg developers, etc) have direct access to the database as needed for development.

In production environments, they do not.

This is exactly why data masking technologies exist. To mask/transform production data in non-production to that non-production has meaningful data but not REAL data

I'm certain all 37s employees have access to the production database anyway; they're still a fairly small company.

The only real additional risk here is running non-production code against live data; e.g. the risk of a feature branch sending extra email to customers. Given the nature of their products this is probably manageable, assuming they don't run batch jobs (via eg. resque)

Fair enough, but if they were large enough to require that sort of privilege separation, then the entire post would make a lot less sense. Furthermore, I think the target audience has the same demographics; relatively small companies where the development team is also the production team.

Thanks for the reference to data masking; it should be more common.

Hypothetically speaking if the load-balancer is directing the traffic depending on the cookies then the environments can be very well separated behind the balancer.

My other thought on this was if they use any separate static web asset management or it's all one single environment-switching for ruby.

These days one could easily work with the stable REST server and all the web assets are switched between staging/beta/etc envs.

You're thinking about external threats, I'm talking about internal threats.

See my response to gyepi below for more info

IMO, the existance of code to configure more then one production-like environment is an anti-pattern. The remedy is using configuration management (Chef/Puppet/whatever) to put the correct environment specific configuration into place in a non-executable config file. Any other policy will lead to environmental drift.

I'm not sure there is a substantive difference between "code" and a "non-executable config file". The latter is something that is read by an executable and which results in specific execution that differs by the content of the file; the difference between this and interpreted code is semantic rather than substantive.

E.g., for any combination of a YAML configuration file and a Ruby function which reads and applies it, there is a corresponding Ruby function which achieves exactly the same result without the configuration file.

There are other reasons to prefer the config file approach, but a policy which deploys the correct code to each environment will have exactly the same behavior with regard to environmental drift as one which deploys identical code the correct config file.

This is what we do, but we can also do his rollout example in production by putting both new and old into production side by side and then targeting percentages of new users.

Tried this and it didn't work for me. I found each additional RAILS_ENV value was an annoying point of divergence between environments that should be as like as possible.

Eg, the whole point of staging is to be as like to production as possible, and running the staging server in RAILS_ENV=production is a dead easy way to accomplish this goal. For the very few differences (eg database config), I further refine the production environment by consulting a SECONDARY_ENV variable that may be set to either production or staging.

Eg, ideally CI is as close to the developer's test environment as possible, so that tests that pass locally pass in CI and vice-versa, and running the CI jobs in RAILS_ENV=test is a dead easy way to accomplish this goal. For the very few differences (eg using xvfb in CI), I further refine the test environment by consulting a CI variable that may be set to either true or false.