The general idea (using word frequencies in context) is Firth's distributional hypothesis [0]. Related to this is the idea of a distributional representation which is covered by Gardenfors in his book on conceptual spaces.

For word2vec and such, several papers in this year's TACL explain why these methods should work [1,2].

[0] https://www.aclweb.org/aclwiki/index.php?title=Distributiona... [1] https://transacl.org/ojs/index.php/tacl/article/viewFile/742... [2]https://transacl.org/ojs/index.php/tacl/article/viewFile/809...