News for hackers. Stuff that matters.

A comment on Fast UTF-8 validation. Parent comment

Re:Fast UTF-8 validation by qppo on Tuesday October 20, 2020 @05:30PM (#24842544)
The check only needs to be done once, when the iterator is constructed. You can also zero pad the buffer by 3 bytes to avoid a bounds check if you really want.

This seems like a premature optimization, unless you're only iterating over tiny strings.

[ Parent | on HN ]
Re:Fast UTF-8 validation by oconnor663 on Tuesday October 20, 2020 @09:26PM (#24844019)
That's a good point. I know in the case of Rust in particular there are other invariants, like that each 32-bit code point is guaranteed to have a valid value. (The compiler knows this and may use unset bits to stash enum state or something like that.) But maybe in other languages without such strict validity guarantees, it's less of a big deal?
[ Parent | on HN ]