When doing arithmetics you can lose as much as 1 bit of precision per floating point operation. So, it pays off to do use more precision for calculations even when the storage format doesn't persist it.
Besides I remember reading somewhere that 32-bit floats instructions aren't actually any faster than 64-bit (except of course the cache...). They even might be slower because the FPU has to extend them to 64-bit before performing the operation
Nowadays when you do computations with single float/double values in your registers they are equally fast.
The biggest difference comes from memory bandwith and the ability to vectorize. Your CPU can calculate either 4 floats or 2 doubles in one instruction (assuming pre AVX X64 processor). With AVX it's 8 floats or 4 doubles.
If it's something you're worried about, I'd suggest measuring the difference in your actual application, both in terms of performance and accuracy. If I can come up with a decent test case I can legally publish, I'll post results, but your mileage may still vary.
So I could use a chip that costs more, results in shorter battery life, more thermal problems, and is slower so smaller BW processed and less audio filtering possible. But, at least I'm using doubles instead of floats, which doesn't even matter when it results in no measurable signal metric improvement while every other measurable performance and economic metric tanks downward.
You can do a lot of DSP and plain ole signal processing with fixed point of course, and sometimes doing weird things requires doubles.
Even CD-quality audio only has 16 bits per sample. My digital camera only records 12 bits per pixel, and that's before the image is converted to a more reasonable 8-bit-per-channel format.
Single precision is still overkill in a number of applications.