Any idea why the compiler doesn't hoist the constant single-character regexp automatically?
1) It's not a constant expression, it's a function call.
2) The compiler does not know if there are collateral effects or not.
3) Ocaml is strict eval - no memoisation
Therefore this would be an unsafe optimisation if the compiler tried to perform it, which could lead to unintended behavioural changes.
The only way to fix this properly would be to add an effects system in some sense, at least a way to mark pure/impure functions.
I moved the regexp instantiation outside the loop, see the results below ("orig" is your code, "proper" has the regexp parsed only once):
I'd say the rest of the difference is probably due to the standard library performance itself. BTW, compilation time for ocamlopt is 70ms.