News for hackers. Stuff that matters.
Re:Jev: New frontier model 40-400x cheaper and 20-200x faster by torginus on Tuesday September 15, 2026 @05:06PM (#49718912)
I was thinking about something similar (maybe) - generally speaking, embeddings for LLMs tend to learn real world concepts - things like 'fruit' or 'France' or 'city' as directions in embeddings.

But in things like programming, most concepts are abstract - 'if hungry eat an apple' in programming terms would look like

'if hunger > 50 {apples--; hunger-=30;}'

and compilers work with 'concept erasure' - to them, tokens (which are like llm tokens) look like

'if var1 > 50 {var2--;var1-=30}'.

They don't care about how these things map to real concepts. So all the embedding directions used to encode real-world concepts are just noise to LLMs when programming. This greatly reduces dimensionality and training costs. So does a token representation tuned for programming constructs, rather than natural language would probably have a more efficient encoding.

Re:Jev: New frontier model 40-400x cheaper and 20-200x faster by ta988 on Tuesday September 15, 2026 @05:12PM (#49718975)
Current models go beyond the simple embedding because you start to encode groups of concepts in the context-aware part of the model (attention heads or any other method). So it is never simply words/tokens in isolation anymore.