On a whim I tried the vps docker setup on the pi. Runs much smoother (seems like the db was the bottleneck but).

edit: Aaaaaaand now I can't login because I foolishly tried to setup webdav and nextcloud is adding 30 seconds to my login and it never actually cool downs. Why do I even bother.