News for hackers. Stuff that matters.

A comment on Make grep 50x faster. Parent comment

Re:Make grep 50x faster by acdha on Sunday December 15, 2013 @01:58PM (#6910357)
> it's smart enough to recognize that the UTF-8 search can be correctly performed as a byte search

It shouldn't that simple – it'd also need to confirm that the pattern wouldn't match any combining characters or normalization would still be necessary.

[ Parent | on HN ]
Re:Make grep 50x faster by Joeri on Sunday December 15, 2013 @02:29PM (#6910493)
The non-leading utf-8 bytes all are in an easily detected range that doesn't overlap with ascii.
[ Parent | on HN ]
Re:Make grep 50x faster by acdha on Monday December 16, 2013 @11:25AM (#6915721)
Yes - it's not hard to do but it does require someone to remember to check before attempting the optimization.
[ Parent | on HN ]