News for hackers. Stuff that matters.
Re:Character Prefix Conditioning by kcarnold on Saturday January 11, 2025 @12:45PM (#42667503)
This was the subject of https://arxiv.org/abs/2412.03719. (I suspect you can do simpler than the paper's solution if you're only interested in the top-k.)

A related topic is "token healing", although some implementations (unfortunately including the one in HuggingFace Transformers) make some big assumptions that aren't always true (like treating spaces as special).

There are no comments on this comment yet.