I think generic deployment system can be very difficult to build. For example, there can be problems with library dependence or data format dependence. I think you should select one 'family' of models, for example narrow down to only python scikit-learn models, and try to build a system for them.