Re:The Super Effectiveness of Pokémon Embeddings Using Only ...
by dcl on Monday July 01, 2024 @06:56PM
(#40851734)
Because then you couldn't use a pretrained LLM to give you the embeddings.
If you added these numerics as extra dimensions, you would need to train a new model that somehow learns the meaning of those extra dimensions based on some measure.
Re:The Super Effectiveness of Pokémon Embeddings Using Only ...
by rahimnathwani on Tuesday July 02, 2024 @03:07AM
(#40854228)
The embedding model outputs a vector, which is a list of floats. If we wrap the embedding model with a function that adds a few extra dimensions (one for each of these numeric variables, perhaps compressed into the range zero to one) then we would end up with vectors that have a few extra dimensions (e.g. 800 dimensions instead of 784 dimensions). Vector similarity should still just work, no?
Is it because:
A) It's easier to just embed everything, or
B) Treating those numeric fields as separate dimensions would mean their interactions wouldn't be considered (without PCA), or
C) Something else?