It's true that a 2D discrete FFT can also be used to produce an orthogonal expansion of an input matrix, but a truncated reconstruction using it (like a band-pass filtering) won't have the rank properties (e.g. optimal low-rank approximation) of a truncated SVD-based approximation.

They are relatable in many ways if you are interested in data filtering and analysis (viewing matrices as data structures rather than as operators in a function space). Generally the SVD is nicer to have, but the FFT is easier to compute (O(n^3) vs O(n^2 log (n)).

Another way to think about them:

An FFT will describe your data in terms of a fixed orthonormal basis: sine and cosine functions of varying frequency.

An SVD will describe your data using a basis that is also orthonormal, but is entirely based on your data.

If you are interested in this stuff, check out Strang's Linear algebra course lectures (mit opencourseware iirc).

He goes through SVD and FFT in terms of matrices. Really illuminating course.

This one? http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-... Do you know if there is a written version? Videos are sometimes a bit harder to follow at your own pace, I find.

Chapters from his textbook are available online.

Nick Trefethen's "Numerical Linear Algebra" is another great source, and a pretty standard read in an upper-level undergraduate or graduate sequence.