Modern web apps connect with increasingly numerous third-party libraries and external software services. My own little website, Oxbridge Notes, sends web traffic stats to Google Analytics; accepts payments via a PayPal integration; saves large binary files to Amazon’s S3 service; notifies exceptions to our Slack chatroom via a webhook; indexes the text body within our notes files using a search-as-a-service partner; delivers email through robust specialist services like MailChimp; filters spam messages with Askimet, etc.
In order to integrate with these various third parties, my code has to supply appropriate configuration and authentication details for each service. Herein lies the problem: With the tech ecosystem being the thriving exemplar of diversity that it is, nearly every service provider (/third-party library/other programmer who exists) will instruct you (e.g. in their project READMEs) to handle configuration for their project in a different way. For example, the library I installed for saving images wants me to enter my S3 credentials in a YAML file; my sitemap generator library expects environmental variables for the same; and another library’s README (dubiously) suggests to hardcode these same S3 API details into my source code.
Without a deliberate plan for how to handle and organise configurations, it’s easy to get pushed around by everyone’s inconsistent demands. As a matter of fact, when I was a junior, “I-just-follow-the-README-and-never-think-for-myself” programmer, I used to take the “pinball approach” to configuring successively installed libraries:
Log into the admin dashboard and fill in the S3 details there.
This is wretched, disgusting, and abhorrently messy coding. It would have been infinitely better to have a consistent way for dealing with all our configurations and then writing tidbits of adapter code when necessary to connect to these various APIs. But what general configuration system should we choose? Given that there are so many approaches to configuration out there, we ought to first take a step back and ask ourselves what we might want from our config systems. Once we’ve figured out what’s important to us, we’ll then know what strategy to accept (and what trade-offs we’re willing to tolerate).
Preparing the Shopping List
Wishlist 1: One home for all configurations
What happens if we blindly follow the whims of each library or service? We end up with a labyrinth of configuration sources—a knotty mess that disperses credentials across our codebase. This tangle attracts duplication—for example, when we forget that some set of credentials was already given using a different configuration strategy expected by a different library. We can sidestep these duplication problems by storing all our credentials in one central place.
Wishlist 2: Consistent interface for reading or modifying configurations
Related to the last point, it would be ideal if we always set/updated configurations with the same signature technique, e.g. you might always use code like:
This consistency would enable us to easily search through our codebase for places where we modified our configuration, which would be invaluable in debugging, refactoring, and meta-programming.
Wishlist 3: Configuration ought to vary depending on the deployment environment (development/testing/live)
It’s almost never a good idea to connect your development and testing environments to the same third-party accounts you use for production. The risk of clobbering and polluting your production data is way too high, not to mention the possibility of carrying out financial transactions. For this reason, your configuration strategy ought to be capable of seamlessly swapping out details according to whichever environment it currently finds itself in.
Wishlist 4: Access control for sensitive information
Certain configuration data contains highly sensitive information, like log-in credentials. Obviously we don’t want an untrusted new teammate to have access to our payment provider API credentials just in case they go rogue and transfer all our money to a bank in Mexico. Likewise, we don’t want the public to see our AWS credentials in plain text within our open source code repository—unless we specifically WANT to payroll some random hacker’s botnet.
Heroku co-founder Adam Wiggins writes in his Twelve-Factor App manifesto that there should be a “strict separation of config from code” such that “the codebase could be made open source at any moment, without compromising any credentials”. He achieves this by storing his credentials in environment variables. Then, by restricting server access to only the most trustworthy team members, these environment variables are shielded from prying eyes.
Wishlist 5: Capable of delegating configuration to admin staff
Not all configurable attributes are created equal. Whereas security is paramount when dealing with credentials to S3 etc., ease of modification by admin staff may trump security with regard to relatively inconsequential configurations, such as how often your application sends past customers discount coupons or which specific email address your server prints in the emails’ “from” fields, or which pan-website special notification we want to set today.
As an entrepreneur (and especially as a lifestyle entrepreneur), you need to delegate. It is a terrible use of your time if you personally have to load up your servers and tweak environmental variables every time a slight configuration change is needed—e.g. to set a pan-website special message, or to enable a Christmas vacation email. Wouldn’t it be great if someone else could update these settings on your behalf so that you can keep out of day-to-day business operations and focus on doing more important things (such as doing nothing at all)?
As such, your overall approach to configuration could be built to simultaneously allow for heterogenous demands. For example, your admin panel could be empowered to change a handful of non-sensitive environment variables but stopped from even seeing that the more sensitive configurations exist.
Wishlist 6: Configuration changes should leave an audit trail
Ideally, configuration changes should leave some sort of trail so as to aid in debugging. This could be as simple as a message in the logs or as complicated as a full database history of configuration state. Security considerations may complicate or perhaps even obliterate your attempts at leaving a nice audit trail.
Wishlist 7: Configuration stored outside of source control
When configuration is stored within a source-controlled file, you require a fresh commit and fresh deploy to deliver this change to your production server. This muddies up your commit history and creates an unnecessary deploy—something I always find stressful regardless of how many batteries of automated tests I’ve got going. As such, I would avoid checking configurations which often change into source control, preferring to set these through other means (such as administrative panels).