Re:How quickly can you remove spaces from a string?
by Sharlin on Friday January 20, 2017 @04:01PM
(#13446945)
Even if the regex library compiled the expression down to the very simplest state machine possible, it would still be at most as efficient as the first handwritten code in the article, handling one byte at a time. I'm fairly sure no regex engine in existence is able to detect "simple enough" regexes and optimize them into code that handles 64 or 128 bit blocks at a time.
[ Parent
| on HN ]
Re:How quickly can you remove spaces from a string?
by burntsushi on Friday January 20, 2017 @08:39PM
(#13448318)
They exist. In fact, if you hand Rust's regex engine the pattern ` |\r|\n`, then it will compile it down to a very simple alternation of three literal bytes. It will never touch the actual regex engine at all. Once the regex engine knows its a simple alternation of three bytes, it will actually use a variant of memchr called memchr3[1], which looks almost exactly like the OP's "portable" approach.
So it's not quite the fastest possible because it doesn't use SIMD, but this is mostly because we don't have good explicit SIMD support on stable Rust yet. Once we do, this immediately becomes a strong candidate to optimize to something like the OP's fastest version (and also probably generalizing to AVX2 instructions as well).
N.B. My sibling comment linked to my blog post on ripgrep, which uses Rust's regex engine. ;-)
[1] - https://github.com/BurntSushi/rust-memchr/blob/master/src/li...
Re:How quickly can you remove spaces from a string?
by Sharlin on Saturday January 21, 2017 @04:36AM
(#13449503)
Awesome, thank you!
Re:How quickly can you remove spaces from a string?
by anowell on Friday January 20, 2017 @05:16PM
(#13447416)
You might checkout the literal optimizations that ripgrep does. IIRC, the underlying regex lib is optimizing a few classes of "simple" regexes with SIMD to achieve better perf than the first handwritten code in that article.
Re:How quickly can you remove spaces from a string?
by GregBuchholz on Friday January 20, 2017 @06:14PM
(#13447727)
Time to break out the HDL then...