Re:tolower() in bulk at speed
by londons_explore on Monday June 27, 2022 @05:54PM
(#31901350)
Worth patching libc?
[ Parent
| on HN ]
Re:tolower() in bulk at speed
by Findecanor on Monday June 27, 2022 @06:11PM
(#31901536)
This type of bit manipulation will only work with pure ASCII, which had been designed to make this transformation simple with only bit manipulation — just not in parallel.
Most systems default to using Unicode these days, for which the problem is much more complex even when the language is set to English.
I was really disappointed when I found out the author didn't have to deal with the dotless I problem.
Re:tolower() in bulk at speed
by mananaysiempre on Monday June 27, 2022 @06:08PM
(#31901498)
The standard tolower() / toupper() take a single character only; bulk strlwr() / strupr() are nonstandard (if commonly present) and—more importantly—virtually unused. I suppose implementing this technique in the C/POSIX-locale version of nonstandard stricmp() / POSIX strcasecmp() might be helpful in some cases, because people do use that one, but I still expect that any situations that truly call for an ASCII-only case-insensitive comparison (parsers?) will be doing much more work per byte for other reasons (state machine, perfect hash, etc.).
It's not the same, libc's tolower takes an int as an individual character
Re:tolower() in bulk at speed
by Findecanor on Monday June 27, 2022 @06:19PM
(#31901610)
Although ... the existence of a libc with SIMD versions of its functions is not implausible. There are compilers that would produce such functions beside the normal ones if the source is decorated with the right #pragma.
Such functions would be called only from within vectorised loops (or other SIMD versions of functions).
Indeed. For instance, glibc's math library has vectorised trig functions, and gcc can generate calls to them if it sees fit. For the rest of us, there is sleef.
Re:tolower() in bulk at speed
by vinkelhake on Monday June 27, 2022 @06:07PM
(#31901486)
libc tolower() works on a single char at a time. This post is about the gains you can get by converting multiple chars at once.
Re:tolower() in bulk at speed
by londons_explore on Tuesday June 28, 2022 @05:47AM
(#31905885)
Presumably with the right __inline stuff it could be vectorized by a compiler, since tolower is normally called from a loop?
Although I guess since it's often a dynamically linked library, said optimizations would have to happen at load time, and the dynamic loader doesn't do such things... :-(