As skybrian says, Teensy has a lot to offer for audio/DSP work. The Teensy 4 generation runs like a bat out of hell compared to an Arduino or ESP32, but it's a much lighter-weight development platform than a RPi or BeagleBone.

Basically, if you want an Arduino that's roughly a thousand times faster, you want a Teensy 4.