News for hackers. Stuff that matters.

A comment on Flash-Moe: Running a 397B Parameter Model on a Mac with 48GB RAM. Parent comment

Re:Flash-Moe: Running a 397B Parameter Model on a Mac with 4... by hrimfaxi on Sunday March 22, 2026 @09:00AM (#47477067)
Why does this G character appear to prefix most of the output? ("Ġlike")
[ Parent | on HN ]
Re:Flash-Moe: Running a 397B Parameter Model on a Mac with 4... by frwickst on Sunday March 22, 2026 @09:31AM (#47477372)
It is a tokenizer artifact most likely (https://github.com/huggingface/transformers/issues/4786). So the output is not properly decoded in this case, it should just be a space.
[ Parent | on HN ]
Re:Flash-Moe: Running a 397B Parameter Model on a Mac with 4... by kgeist on Sunday March 22, 2026 @09:30AM (#47477362)
The original tokens have Ġ instead of space. I had this issue too when writing an inference engine for Qwen. You have to "normalize" those special characters.
[ Parent | on HN ]