Is that really true? I wouldn't exactly feel comfortable with "x -20", though I suspect you're right about at least a large number of languages determining the meaning of the hyphen through local syntactic context ("I just saw an identifier; that must be a non-unary hyphen").
Now I'm interested in whether the corpus of existing code shows any bias between "y = -x" (perfectly allowed, I think) and "y = -1 * x".
I know this for a fact since I've studied their grammars.
Infix notation and - are really the reasons why nobody does this, some people like to use spacing to indicate precedence too (writing code such as "a - 1*g") so it would break some workflows and realistically having a language which is whitespace agnostic except for identifiers AND the minus operator just seems too irregular for people to commit to it.
Now, I grant the point that it is doable. But the point is it complicate things. Now, fair, we have some of these complications already by virtue of the fact that we allow numbers in variable names. "foo3" is already allowed in many languages, and that clearly gets altered as you add space between the characters.