Snakemake works very well for me on our batch processing cluster, but for other scaling we still package the whole flow in a docker and only scale on sample. Yeah, it's not very efficient. This (AGC) would really help though.