Re:Wc2: Investigates optimizing 'wc', the Unix word count pr...
by brainwad on Friday June 21, 2024 @07:51AM
(#40748593)
According to the authors it's also faster on files full of 'x' or ' ', so there must be more than just better unicode support.
[ Parent
| on HN ]
Re:Wc2: Investigates optimizing 'wc', the Unix word count pr...
by yencabulator on Friday June 21, 2024 @10:08AM
(#40749805)
Even something as simple as wc calling a libc function for parsing utf-8, that doesn't get inlined, would destroy its performance relative to anything optimized.
Personally, I'd expect SIMD to win over all of these. wc sounds like kind of challenge that's very easy to partition and process in chunks, though UTF-8 might ruin that.
Re:Wc2: Investigates optimizing 'wc', the Unix word count pr...
by teo_zero on Saturday June 22, 2024 @01:22AM
(#40756506)
> very easy to partition and process in chunks
Which counters to increment at each byte depends on the previous bytes, though. You could probably succeed using overlapping chunks, but I wouldn't call it very easy.
Re:Wc2: Investigates optimizing 'wc', the Unix word count pr...
by yencabulator on Saturday June 22, 2024 @09:30AM
(#40759014)
That sort of "find the correct chunk boundary" logic was very common with all the mapreduce processing that was done when people still used the phrase big data.