News for hackers. Stuff that matters.
Re:Python, Catastrophic Regular Expressions and the GIL (2013) by Narann on Monday November 14, 2022 @05:12AM (#33592078)
I've been in this situation many times.

> However, the regular expression module is a C module that doesn’t release the GIL before running.

I would love to know why.

Re:Python, Catastrophic Regular Expressions and the GIL (2013) by byroot on Monday November 14, 2022 @05:24AM (#33592171)
Mostly a guess, but:

  - Releasing/acquiring the GIL isn't cheap. So you'd need to know in advance whether it's worth it or not.
  - You need the GIL acquired to instantiate the resulting "
match data", captures, etc.
Re:Python, Catastrophic Regular Expressions and the GIL (2013) by formerly_proven on Monday November 14, 2022 @06:12AM (#33592509)
Stuff like re.sub frequently calls into Python code as well, so it would seem to be rather hairy to get rid of the GIL for this. It could of course add the usual workarounds of cooperative multitasking, like yielding (releasing the GIL) from time to time, just like the main interpreter loop does (sys.setswitchinterval)
Re:Python, Catastrophic Regular Expressions and the GIL (2013) by asah on Monday November 14, 2022 @09:29AM (#33594387)
assuming it's legal to do so, GIL release seems like a reasonable flag to add to possibly-expensive functions like re.()

... or even better, have re. track its own performance and release the GIL if it forecasts that it statistically-might use a lot more computation... a trivial strawman algorithm: if len(input) > MAX then release GIL first.