I fully understand wanting native silicon to do development on and to work on some architecture specific branch. It's easy to just recompile your working copy locally. The original poster was talking about "build machines". I'm not seeing the utility as "build machines", especially if they have far less power than AMD64 machines.
In theory you can set up the build system to correctly know which things should be compiled native and which should be cross-compiled. Sometimes you even need native and cross-compiled versions of the same thing.
In practice, this is a capability that few people use and even if everything is set up correctly at some point most people contributing patches do not think about or test maintaining the ability to cross compile and it gets broken.
When you're building hundreds or thousands of packages for a distro such as Debian or Fedora this is a huge problem.
Building natively in a full system emulator or on real hardware is the only way, in practice.
amd64 machines do have the advantage that you can get them with 32 or 64 cores and hundreds of GB of RAM -- at a price, especially in power consumption.
The three year old HiFive Unleashed uses around 5W to 6W at the wall when building flat out, considerably less than the maybe 25W a quad core i7 or Ryzen will use, with lower performance running qemu-system.
The new HiFive Unmatched probably has the same or lower power consumption, at 50% higher performance.
Running qemu on a Pi 4 would use about the same power as the RISC-V chip but be waaaaaaay slower.