News for hackers. Stuff that matters.
Re:2000x performance win by Confusion on Sunday December 11, 2011 @05:45AM (#3339817)
It's not necessarily grep's responsibility to normalize strings. They could very well stipulate that it is the responsibility of the user to make sure the both the input and the search expression are already normalized. That way, the normalization could be done by a separate program, resulting in the reusability and separation of concerns that is The Unix Way.
Re:2000x performance win by barrkel on Sunday December 11, 2011 @10:20AM (#3340198)
I fully agree, if you redefine grep as a byte search program, rather than for text search. Rule it out for most languages apart from English.

To have any usable grep with applicability beyond programming languages, English-language log-files, etc. (i.e. massive Anglocentrism), the normalizing would need to be built-in. Sketched:

    #!/usr/bin/bash
    regex="$1"; shift
    unicode-normalize-to-utf8 "$@" | byte-grep "$(utf8-byteify-regex "$regex")"