Validating UTF-8 strings using as little as 0.7 cycles per byte

Re:Validating UTF-8 strings using as little as 0.7 cycles pe... by bumblebritches5 on Wednesday May 16, 2018 @02:48PM (#17085273)

Have you ever written your own unicode library, or are you just parroting what you've heard?

I have written my own unicode library, normalizing strings as UTF-32 is MUCH MUCH MUCH easier than trying to do the same in UTF-8 or UTF-16.

not to mention casefolding, and it also allows you to design better interfaces, so that the actual algorithm only needs to be implemented once.

[ Parent | on HN ]

Re:Validating UTF-8 strings using as little as 0.7 cycles pe... by greglindahl on Wednesday May 16, 2018 @06:41PM (#17087045)

Sounds like your library is the poster-child for:

> it blows up memory usage for ASCII and BMP (4 bytes per codepoint versus 1~3)

[ Parent | on HN ]