VoltaML is a relatively vanilla diffusers-based backend, so its not a hairy monster to hack like you may have seen with SAI-based UIs (like Comfy, Fooocus and Automatic)

The AITTemplate code is a lightly modified version of Facebook's example dynamic AIT script, to get rid of small issues like VRAM spikes: https://github.com/facebookincubator/AITemplate/tree/main/ex...

InvokeAI is also diffusers based, but they seem to mess with the pipeline a bit more.

Anyway, all that may be better as a reference for interesting features rather than a backend to try and adopt.