News for hackers. Stuff that matters.
Re:The Super Effectiveness of Pokémon Embeddings Using Only ... by rahimnathwani on Monday July 01, 2024 @04:50AM (#40843830)
There seem to be a lot of properties that are numeric or boolean, e.g.

    "base_happiness": 50,
    "capture_rate": 190,
    "forms_switchable": false,
    "gender_rate": 4,
    "has_gender_differences": true,
    "hatch_counter": 10,
    "is_baby": false,
    "is_legendary": false,
    "is_mythical": false,
Why not treat each of those properties as an extra dimension, and have the embedding model handle only the remaining (non-numeric) fields?

Is it because:

A) It's easier to just embed everything, or

B) Treating those numeric fields as separate dimensions would mean their interactions wouldn't be considered (without PCA), or

C) Something else?

Re:The Super Effectiveness of Pokémon Embeddings Using Only ... by dcl on Monday July 01, 2024 @06:56PM (#40851734)
Because then you couldn't use a pretrained LLM to give you the embeddings. If you added these numerics as extra dimensions, you would need to train a new model that somehow learns the meaning of those extra dimensions based on some measure.
Re:The Super Effectiveness of Pokémon Embeddings Using Only ... by rahimnathwani on Tuesday July 02, 2024 @03:07AM (#40854228)
The embedding model outputs a vector, which is a list of floats. If we wrap the embedding model with a function that adds a few extra dimensions (one for each of these numeric variables, perhaps compressed into the range zero to one) then we would end up with vectors that have a few extra dimensions (e.g. 800 dimensions instead of 784 dimensions). Vector similarity should still just work, no?