For signals (as opposed to images), you should have a look at "Understanding Digital Signal Processing" by Richard G. Lyons (ISBN-13: 978-0131089891). I enjoyed this very much and if you grasped the contents from it, you should be able to understand audio-specific books easily.

I second Lyons. In addition, "Digital Signal Processing: A Practical Guide" by Steven Smith is also quite approachable -- immersion in calculus not required.

"A Digital Signal Processing Primer" by Ken Steiglitz is a nice but rigorous intro to the subject. Written by an EE academic, it's more mathematically rigorous than Smith or Lyons.

Allen Downey's "Think DSP" is also worth a look, though its focus is more conceptual than practical, IMO.

DAFX by Zölzer covers all the fundamental audio DSP building blocks. The book is eponymous with the annual international audio DSP conference.