News for hackers. Stuff that matters.

A comment on Astra can read and write weights to improve Qwen. Parent comment

Re:Astra can read and write weights to improve Qwen by AustinDev on Monday November 02, 2026 @05:44PM (#50345101)
Short answer is tokenizers, so to the LLM your query was roughly:

[5299][1991][428][885][306][181747][3]

[How][ many][ r]['s][ in][ blackberry][?]

Hope that helps. Using the open source gpt-tokenzier.

[ Parent | on HN ]

There are no comments on this comment yet.