News for hackers. Stuff that matters.
Re:Python Concurrency From the Ground Up (2015) [video] by shepardrtc on Wednesday November 11, 2020 @10:50AM (#25059324)
You're right. To me they just feel like fibers because they can't run in parallel.
Re:Python Concurrency From the Ground Up (2015) [video] by helgie on Wednesday November 11, 2020 @11:19AM (#25059597)
If you use numba (or cython, c extensions, etc) you can make them run without requiring that they hold the GIL, and they can run in parallel. Here's an example that should keep a CPU pegged at 100% utilization for a while:

  import numba as nb
  from concurrent.futures import ThreadPoolExecutor
  from multiprocessing import cpu_count

  @nb.jit(nogil=True)
  def slow_calculation(x):
      out = 0
      for i in range(x):
          out += i**0.01
      return out

  ex = ThreadPoolExecutor(max_workers=cpu_count())
  futures = [ex.submit(slow_calculation, 100_000_000_000+i) for i in range(cpu_count())]
Re:Python Concurrency From the Ground Up (2015) [video] by shepardrtc on Thursday November 12, 2020 @08:31PM (#25077776)
> and they can run in parallel.

Even without requiring the GIL, these are still child threads of the main process, correct? And because of that, wouldn't the OS keep them all on the same core? And if that's the case, would ProcessPoolExecutor solve that problem?

Re:Python Concurrency From the Ground Up (2015) [video] by shepardrtc on Wednesday November 11, 2020 @12:20PM (#25060353)
I had no idea that existed, thank you!