Are many of those devices actually reprogramming the FPGA at runtime from the OS running on the soft processor, though?

My impression was that you tended to define pretty static blocks of functionality (you know, hash calculators for your bitcoin mining or whatever), and then just communicate to those from the OS using interrupts and a shared memory interface like any other peripheral.