You should be able to throw a key-value store on top of settings to override certain values at runtime. Replace django.conf with django_dynconf?