Make grep 50x faster

Re:Make grep 50x faster by comex on Sunday December 15, 2013 @01:17PM (#6910221)

Note that, at least as of GNU grep 2.14, if you don't use -i, the discrepancy doesn't show up, so it's smart enough to recognize that the UTF-8 search can be correctly performed as a byte search. I suspect the insensitive version can also be done correctly much faster, though.

[ Parent | on HN ]

Re:Make grep 50x faster by acdha on Sunday December 15, 2013 @01:58PM (#6910357)

> it's smart enough to recognize that the UTF-8 search can be correctly performed as a byte search

It shouldn't that simple – it'd also need to confirm that the pattern wouldn't match any combining characters or normalization would still be necessary.

[ Parent | on HN ]

Re:Make grep 50x faster by Joeri on Sunday December 15, 2013 @02:29PM (#6910493)

The non-leading utf-8 bytes all are in an easily detected range that doesn't overlap with ascii.

[ Parent | on HN ]

Re:Make grep 50x faster by acdha on Monday December 16, 2013 @11:25AM (#6915721)

Yes - it's not hard to do but it does require someone to remember to check before attempting the optimization.

[ Parent | on HN ]