Re:Progress on the Gilectomy
by matthewmacleod on Friday June 02, 2017 @05:05AM
(#14468532)
Many machine learning people use Python, and they need all the CPU power they can get.
The compute code for ML is emphatically not written in Python – Python is essentially a lightweight API wrapper around it.
That's bit of a circular argument.
Agreed that compute code for ML is not written in Python, but if GIL wasn't a problem there would be one less reason to drop down to C. Many users would appreciate that. Cython helps with the C part but to a degree.
You are forgetting python is (depending on the benchmark) still 100-400x slower than a language compiled to native machine code. Removing the GIL might slightly help in some cases but anything CPU bound requiring
high performance will still need to drop down to something faster.
That's a CPython problem. PyPy is much faster at number-crunching.
The python libraries ML people use (scikit-learn, tensorflow, numpy) all do the heavy work in C though
Yes, but the light work, run on a single Python thread, will in some cases take a significant part of execution time.
If you have 16 CPUs in your system, seeing your load at 6% for the first 10% an last 10% of running time, and at 100% in-between surely will make people think there must be easy wins in that Python code. More so since many users will have effectively _zero_ knowledge of C.
Python added the "horrible event-driven programming model of Node" in Python 3.6. The syntax is said to be better, though.