I await the day when I can be like "nah, pin 2 will be a 16-bit 10MHz PWM output controlled by this register and pin 7 will be a frequency counter" and then access that with code on the microcontroller as if it was a built in peripheral.
I'm kind of skeptical I'll ever really get into FPGAs otherwise, my use case for them is always so simple, at the level of... well... PWM outputs and PID loops. But a microcontroller is fast enough for most PID loops so really to me the advantage is customizable function-pin mappings. I can't imagine trying to implement something like an ethernet stack on an FPGA though, although I guess you can run DOOM on one so maybe there will be a compiler from C++ to FPGA somehow that's not comically bad. ;-)
Are you familiar with the Cypress PSoC line? They have that.
https://www.cypress.com/products/32-bit-arm-cortex-m3-psoc-5...
It's an older part, so the performance isn't incredible, but there are some very interesting things you can do with it.
http://papilio.cc/index.php?n=Papilio.PapilioDUOHardwareGuid...
edit: yep...still for sale http://store.gadgetfactory.net/fpga/
The "FPGA acceleration", on the other hand, is pretty limited. Arduino has one board -- the MKR Vidor 4000 [1] -- with an onboard FPGA. The FPGA can be configured with one of a couple prepackaged bitstreams to add I/Os or perform some specific video processing tasks; it does not accelerate processing on the Arduino MCU. The documentation for the board is minimal, and support for generating your own bitstreams is basically nonexistent. The whole thing has a half-baked feel to it; I'd avoid it.
[1]: https://store.arduino.cc/usa/mkr-vidor-4000