The mathematical term "Embedding" refers to finding a lower dimensional representation of some high(er) dimensional data, whether this is done with single function (maybe hand chosen) or a neural network doesn't matter.

I don't think that's quite right.. According to wikipedia, embedding is: https://en.wikipedia.org/wiki/Embedding

So Projection sounds closer to what you describe. https://en.wikipedia.org/wiki/Projection_(mathematics)

So my original comment is not quite right either. The embedding is a structure preserving map. A projection isn't necessarily structure preserving. ''' To be an embedding, such a mapping must preserve order "both ways": ''' http://mathworld.wolfram.com/Embedding.html

I think that a projection from p dimensions to q<p dimensions isn't structure preserving.

At this point a mathematician may be able to explain more.

SVMs project the data into a higher dimensional space, not lower. A mixed blob of dots on a plane can't be cleanly separated by a line into two colors, but if you blow them out into a cube by some arbitrary projection, a plane will be able to separate them cleanly.