Re:Setting the TZ environment variable avoids thousands of s...
by butlerm on Thursday January 12, 2023 @02:22AM
(#34350266)
Presumably you can set LC_ALL=C.UTF-8 in most environments to deal with that problem.
Re:Setting the TZ environment variable avoids thousands of s...
by loeg on Thursday January 12, 2023 @02:52AM
(#34350450)
C.UTF-8 doesn't save you anything locale-wise? I believe you're still on the slow path for character matching.
Re:Setting the TZ environment variable avoids thousands of s...
by moring on Thursday January 12, 2023 @03:30AM
(#34350715)
> Those tools are supposed to act locale specific.
Another response mentioned decimal points. In a locale that uses a decimal comma, how does (for example) grep whether an ASCII 46 symbol is a decimal point or full-stop?
Re:Setting the TZ environment variable avoids thousands of s...
by theamk on Thursday January 12, 2023 @09:45AM
(#34354199)
Luckily, this does not matter for most tools -- for "grep", ASCII 46 is nether decimal point nor full-stop, it is "match any" character. "sort" is commonly used with non-number data (filenames, ISO dates) or with integer values (file sizes), and most uses of "find" can avoid floating point entirely (use -mmin instead of-mtime). And since both "expr" and bash's "$((...))" don't work on non-integer data, there is a great incentive to avoid floating point in shell scripts.
And yes, sometimes you have no choice but to care about locales, maybe you are pulling data from badly designed API or parsing files generated by someone who didn't know about "stat -c" and did "ls -l" instead. But I'd argue that in this case you should be explicit about setting locale env, and you shouldn't rely on system settings. It's not like an API will suddenly start returning different data if you move your fetching script to a machine with a different locale.
Locales and encodings (at least ones that could be applied using LC vars) generally all behave the same as long as (1) the characters in question are in ASCII (0-127) range and (2) searches are cases insensitive or occasional false match is acceptable. In my experience, this applies to the vast majority of the grep/find invocation I have seen.
In other words: don't put LC_ALL=C in the script which searches your music or document collections. Most other cases are fine with it.
Same applies to things like Java source code: as long as you are searching for FactoryConstructorIndirectorSingleton, you can treat UTF-8 as ASCII, because ASCII is a subset of it. { is code 123, no matter if its latin-1, "C locale", UTF-8 or iso-8859-3.