"In my experience the 64-bit OSes all run much slower on the RPi3/+ than Raspbian."

This should be backed up by some numbers. I have seen plenty of benchmarks now that pretty much show that many workloads on ARM64 are a lot faster.