Wouldn’t "visually identical” tokens still be valid utf8?

It depends on the meaning of 'valid UTF-8'. They could be 'UTF-8 that is invalid for my application', but still be 'strings that are valid under the UTF-8 specification'.

There is no such thing as "UTF-8 that is invalid for my application". Any properly encoded sequence of UTF-8 codepoints is valid UTF-8. Your application may have constraints on the codepoints that it should accept, but that does not suddenly make them improperly encoded. Using visually-similar lookalikes does not make a UTF-8 string invalid.

> There is no such thing as "UTF-8 that is invalid for my application".

In fact, yes there is! Your application may vary. :)

UTF-8 doesn't vary, it's just an encoding.

That's not what invalid UTF-8 means. It doesn't mean replacing codepoints with other codepoints, it means that the encoding of the codepoints is invalid. Since JavaScript uses UTF-16 strings, it's possible to create JavaScript strings that don't have a valid UTF-8 representation (I believe unpaired surrogates can do this). Also, if you are working with raw bytes, there is no guarantee that it has any encoding at all (this seems to be what the article is about). There is WTF-8 [0] that proposes a possible solution to the UTF-16 problem, but purely as an internal representation and not as an interchange format. I think the Rust standard library uses WTF-8 to represent Windows wide-strings, which allows ASCII and UTF-8 algorithms to work on them, while still being able to represent any UTF-16 string.

[0]: https://simonsapin.github.io/wtf-8/