Right, I don't deny that you get something sort of vector-ish and some operations that make intuitive sense some of the time. That's cool enough on its own! There's no need to pretend that we've actually constructed a vector space. Just say that we've arranged words on a hypersphere with an inferred distance metric and call it a day.

(Also, by "scalar product" I meant "scalar multiplication"--the product of a scalar and a vector, not the dot product. It's pretty clear how to make some sense out of the dot product, but it's pretty hard to make consistent sense out of scalar multiplication. Apologies for being unclear.)