Re:Yes, Python is Slow, and I Don’t Care
by nhumrich on Friday September 08, 2017 @10:07AM
(#15200230)
Python does multiprocess very well. You can easily use all cores on your machine. Pythons main "disadvantage" is threading because of the GIL. But each process gets its own GIL. So when you multi process, your not limited to one core.
Re:Yes, Python is Slow, and I Don’t Care
by classybull on Friday September 08, 2017 @10:23AM
(#15200352)
This. I had a problem where I needed to scrape roughly 20,000 html documents daily, which is normally a pretty slow task. You have to open the file, load it into memory, parse the DOM, and then run all of your selection methods. Sequentially, it took about 60 minutes daily. Multithreading slowed it down because it was CPU bound. Multiprocessing allowed me to run 12 processes across 8 cores. That took the total processing time down to about 4 minutes or so. And I was able to write the code in a day. Writing something similar in Java or C++ would have taken me a week.
Re:Yes, Python is Slow, and I Don’t Care
by aeorgnoieang on Friday September 08, 2017 @12:42PM
(#15201748)
The point is that (modifying existing code or writing new code to) "just use more cores" may be less efficient, for the business or organization that is employing the programmer, given that programmer salaries, over even a fairly short amount of time, can be more expensive than hardware.
Right tool for the right job I suppose.