I like this way of thinking about it, but I think it is not quite accurate for discrete Fourier Transforms. In this case, we're not projecting onto the space of all sinusoids, only the space of sinusoids whose period is a multiple of (1/N). We could probably prove (if we wanted to try) that those form a basis for the vector space of N-long complex vectors, so using any more sinusoids would be redundant.

However, I believe the continuous Fourier Transform works exactly like that.

Very true, and if you want to be more specific my definition only works for finite duration continuous time signals with finite second moment. I use this definition as it also works well for understanding other transforms such as the Laplace transform.

It is both the least squares approximation using periodic functions of this sort (i.e. the projection you mentioned), and an interpolant - a very nice combination of properties.

As is explained, you don't even need to project. What you do is a change of basis, which is to say no information is lost.

That's a really great way of thinking about it, actually - particularly if you're used to vector mathematics.