Whatever you go with, as a beginner you're going to want a lot of help from pre-written software. The Arduino IDE and Wiring/Arduino standard API is a good example. The mistake I made with my first purchase was not making sure that a good audio library was available for the hardware I picked. (I might consider writing one for fun, but not in C++ since I don't have much experience in that language.) But it looks like Teensy has a nice audio library, and there are other projects that look intriguing like Bela and Axoloti.
I expect (speculating) that a similar rule would apply for an FPGA; it would only be appealing to most beginners if someone more experienced came up with an interesting design that makes use of an FPGA and documented it pretty well, so you're not starting from scratch.
It's hard to compare working with an HDL to working with a programming language like C; they are fundamentally different. The short answer to that question is that using an FPGA can lead to massive but situational performance increases. You can't use an FPGA for everything, but when you need it, you need it.
>testing then export designs into hardware
This is another common use case. Some designs can be tested on an FPGA before committing that design for production as an ASIC. There's also the case where you may need to reconfigure the hardware, so you can never truly commit to an ASIC, and continue using the FPGA.