News for hackers. Stuff that matters.
Re:Meta pledges Three-Year sponsorship for Python if GIL rem... by netheril96 on Saturday July 08, 2023 @09:45AM (#36644338)
GIL has been a major pain point when training machine learning models, where every library is Python first. The data preprocessing phase must happen quickly enough and in parallel with respect to the GPU operations. The simplest option is doing data processing in several threads other than the one driving GPU operations, but that is not possible with GIL. TensorFlow and PyTorch each has their workarounds for this problem, but many difficulties remain. Removing the GIL will be greatly beneficial to this domain at least.

There are no comments on this comment yet.