News for hackers. Stuff that matters.

A comment on Show HN: A Python Script to Download Thousands of Wallpapers at Once. Parent comment

Re:Show HN: A Python Script to Download Thousands of Wallpap... by mynewtb on Monday April 23, 2018 @12:26PM (#16904855)
Sure thing...

    from multiprocessing import Pool
    with Pool(8) as p:
        p.map(function, sequence))
[ Parent | on HN ]
Re:Show HN: A Python Script to Download Thousands of Wallpap... by myroon5 on Monday April 23, 2018 @02:31PM (#16906110)
Neat.

I see that you explicitly specify 8 as the number of processes here. In Java, parallelStream() will pick a sane default for you if you haven't previously specified (based on the number of available processors, I believe). Is something like that possible in Python?

[ Parent | on HN ]
Re:Show HN: A Python Script to Download Thousands of Wallpap... by jstarfish on Monday April 23, 2018 @02:38PM (#16906184)
The ThreadPool class picks a sane default (number of cores), but I believe it uses Python threads instead of processes.
[ Parent | on HN ]
Re:Show HN: A Python Script to Download Thousands of Wallpap... by pddubs on Monday April 23, 2018 @12:33PM (#16904944)
You could just use threads here too - CPython releases the GIL during IO.

  from concurrent.futures import ThreadPoolExecutor
  with ThreadPoolExecutor as executor:
      executor.map(function, sequence)
[ Parent | on HN ]