Re:The Boyer-Moore Fast String Searching Algorithm
by pletnes on Friday April 23, 2021 @03:10PM
(#26918063)
Wow, cool!
Since rg (which is awesome) understands which programming languages map to which file suffixes, could it make sense to do per-language frequency analysis? {}[] are common in C-style languages, but much less so in lisp, python, etc.
Re:The Boyer-Moore Fast String Searching Algorithm
by burntsushi on Friday April 23, 2021 @06:08PM
(#26919771)
It could. But there's a lot of code complexity there because ripgrep is made from de-coupled components. Implementing your idea would require making that frequency distribution available as an overridable element all the way down into where substring search is actually implemented (a transitive dependency of ripgrep).
On top of that, it's unlikely to make much of a difference. Not only would the frequency distributions between languages look pretty similar, but it's still just a guess at the end of the day. There is such a thing as overfitting too. :-)
https://github.com/rust-lang/regex/blob/3db8722d0b204a85380f...