These only apply for specific problems, and very few applications are purely CSV parsing or purely matrix math operations. In the real world, you often spend more time marshaling your Python data to C than you save by doing your computation in C.
> But as you note, bottleneck is the I/O, and a program waiting for I/O in Python and I/O in C will wait the same time after the computation is done.
The bottleneck in a static site generator is I/O. The fact that Python, Ruby, etc based implementations take tens of seconds or more while Go and Rust finish instantly for an I/O bound problem is pretty damning.
> If you are writing software that can parallelize well independently (eg. web apps) and your memory pressure is not the most important thing, you simply run multiple Python processes to max out the CPU (this avoids the GIL unlike async Python).
The goal isn’t to saturate the CPU as much as it is to complete requests in a timely fashion. If it’s just some light translation between HTTP and database layers, Python is fine, but if you have to do anything computationally significant at all, it can range from “a huge pain” to “virtually impossible”. I gave the example earlier of a web service that was struggling to complete requests in even 60s (despite using Numpy under the hood where possible) while a naive Go implementation completed in hundreds of ms.
My point was that if this was the case, your Python code is probably suboptimal.
Sure, you are comparing against naive implementation as well, but if performance is a concern, don't do naive Python :)
> I gave the example earlier of a web service that was struggling to complete requests in even 60s (despite using Numpy under the hood where possible) while a naive Go implementation completed in hundreds of ms.
Yes, it's easy and sometimes even idiomatic to write non-performant Python code. Getting the most out of pure Python is hard and it means avoiding some common patterns.
Eg. simply using sqlalchemy ORM (to construct rich dynamic ORM objects) instead of sqlalchemy core (tuples) to get 100k+ rows from DB is 20x slower, and that's still 2x slower from pure psycopg (also tuples using basic types). There are plenty of examples like this in Python, unfortunately.
I agree, and I'll go further: if performance could be a concern and you aren't certain that even optimized Python is up for the task, don't do Python. :)
I don't know how optimized these SSGs are, but given how frequently this complaint occurs and how popular they are, I would expect that someone would have tried to optimize them a bit. Even assuming naive implementations, tens of seconds versus tens of milliseconds for an I/O-bound task is pretty concerning.
> Yes, it's easy and sometimes even idiomatic to write non-performant Python code. Getting the most out of pure Python is hard and it means avoiding some common patterns.
It probably shouldn't be easy for someone to write non-performant Python code when they're trying desperately to write performant Python code. :)
> Getting the most out of pure Python is hard and it means avoiding some common patterns.
And even then, you're probably going to be coming in 10-100X slower than naive Go/Java/C#/etc unless your application happens to be a good candidate for C-extensions (e.g., matrix math) or if it really is I/O bound (a CRUD webapp). It honestly just seems better to avoid Python altogether than try to write Python without using "common patterns" (especially absent guidance about which patterns to avoid or how to avoid them).
I am sometimes frustrated that I can't just loop over a string character by character and not get crappy performance, but the "problem" you (and me) are seeing in existing codebases is that Python is very inviting to beginners, and they are not frustrated with this because they don't know it :)
But as you note, bottleneck is the I/O, and a program waiting for I/O in Python and I/O in C will wait the same time after the computation is done.
If you are writing software that can parallelize well independently (eg. web apps) and your memory pressure is not the most important thing, you simply run multiple Python processes to max out the CPU (this avoids the GIL unlike async Python). And you keep your dependencies low.