That still requires you to know whether either is being called with user-provided values before you can say it's unsafe.

If, for example, someone had a utility which resized user-uploaded images you couldn't say simply calling exec to run something like ImageMagick was unsafe before checking whether it used the user's filename.

I think that's what they're saying in those quoted lines though. That 36% appeared to be statically-safe (i.e. the string sent to eval or exec was generated using only strings defined in the code) and the others were not.

True — it wasn't clear to me whether e.g. a random temporary filename would be considered static or only actual fixed strings in the code. (Similar questions would arise if e.g. a web app saved things using a database primary key so it's technically not static but the attacker doesn't control it even if they control the file contents)

Ahhhhh nice, I hadn't considered the "random temporary filename" or "db primary key" aspect either. That's something that gives me a small chill, but it's not the end of the world compared to "arbitrary user input piped to exec()"

Who knows how elaborate their static analyzer actually is in practice? I wrote a Python analyzer in grad school, and the results were both pretty interesting (type inference tracing through function calls) and pretty mediocre. It was also fiddly-as-hell to get working. Plus... the first rule of statically-analyzing dynamic languages is that the results get weird after the first eval()--anything could happen!

Yeah, as a developer I see plenty of appeals to dynamic languages but they’re certainly a nightmare for tool authors. I know OpenStack was working on a security scanner for Python – I wonder what percentage of the open issues start with “this is probably very hard but…”