News for hackers. Stuff that matters.

A comment on Do Machine Learning Models Memorize or Generalize?. Parent comment

Re:Do Machine Learning Models Memorize or Generalize? by pillefitz on Thursday August 10, 2023 @01:01PM (#37078711)
That is essentially what embeddings do
[ Parent | on HN ]
Re:Do Machine Learning Models Memorize or Generalize? by nightski on Thursday August 10, 2023 @01:49PM (#37079365)
Maybe, except from my understanding an embedding vector tends to be much larger than the source token (due to the high dimensionality of the embedding space). So it's almost like a reverse compression in a way. That said I know vector DBs have much more efficient ways of storing those vector embedding.
[ Parent | on HN ]
Re:Do Machine Learning Models Memorize or Generalize? by jncfhnb on Thursday August 10, 2023 @02:09PM (#37079657)
Tokens are not 1:1 with vectors.
[ Parent | on HN ]