News for hackers. Stuff that matters.

A comment on How fast can you validate UTF-8 strings in JavaScript?. Parent comment

Re:How fast can you validate UTF-8 strings in JavaScript? by orangepanda on Saturday January 20, 2024 @03:52PM (#39072186)
Wouldn’t "visually identical” tokens still be valid utf8?
[ Parent | on HN ]
Re:How fast can you validate UTF-8 strings in JavaScript? by wolverine876 on Saturday January 20, 2024 @03:54PM (#39072210)
It depends on the meaning of 'valid UTF-8'. They could be 'UTF-8 that is invalid for my application', but still be 'strings that are valid under the UTF-8 specification'.
[ Parent | on HN ]
Re:How fast can you validate UTF-8 strings in JavaScript? by LoganDark on Saturday January 20, 2024 @03:58PM (#39072250)
There is no such thing as "UTF-8 that is invalid for my application". Any properly encoded sequence of UTF-8 codepoints is valid UTF-8. Your application may have constraints on the codepoints that it should accept, but that does not suddenly make them improperly encoded. Using visually-similar lookalikes does not make a UTF-8 string invalid.
[ Parent | on HN ]
Re:How fast can you validate UTF-8 strings in JavaScript? by wolverine876 on Saturday January 20, 2024 @04:06PM (#39072322)
> There is no such thing as "UTF-8 that is invalid for my application".

In fact, yes there is! Your application may vary. :)

[ Parent | on HN ]
Re:How fast can you validate UTF-8 strings in JavaScript? by LoganDark on Sunday January 21, 2024 @12:18AM (#39075806)
UTF-8 doesn't vary, it's just an encoding.
[ Parent | on HN ]