News for hackers. Stuff that matters.
Re:A viable solution for Python concurrency by cletus on Friday October 15, 2021 @10:01PM (#28884983)
This feels like Schrodinger's Cake to me (you know, having it and eating it too).

> ... the first of which is called "biased reference counts" ... With this scheme, the reference count in each object is split in two, with one "local" count for the owner (creator) of the object and a shared count for all other threads. Since the owner has exclusive access to its count, increments and decrements can be done with fast, non-atomic instructions. Any other thread accessing the object will use atomic operations on the shared reference count.

So if the owner needs to check the shared ref count every time it changes the count of the non-atomic local count, isn't that basically all the negatives of a single shared atomic counter?

Every design has pros and cons. Of course the GIL has well-known costs but it also has benefits. It makes developing C modules that integrate into Python relatively safe and trivial. And this is where Python shines: as plumbing where CPU intensive work (which is still single-threaded) is done by C modules. This is how the likes of Mercurial, Jupyter, numpy and scipy. And probably PyTorch but I know less about that.

My personal view is that the world has largely moved on from dynamically typed languages for anything non-trivial or that isn't essentially plumbing. For good reason. Of course people will bring up Javascript but it has a captive market as being the only thing that'll universally run code in a browser and the likes of TypeScript can ease that burden anyway.

This just feels fighting the seemingly inevitable fate of Python.

Re:A viable solution for Python concurrency by SoylentOrange on Friday October 15, 2021 @10:16PM (#28885031)
> My personal view is that the world has largely moved on from dynamically typed languages for anything non-trivial or that isn't essentially plumbing. For good reason. Of course people will bring up Javascript but it has a captive market as being the only thing that'll universally run code in a browser and the likes of TypeScript can ease that burden anyway.

This is not correct. There is a lot of data analysis code in the research and scientific community written in python. A lot of PyCon attendees and speakers come from these communities. Oftentimes, it’s not easy to write code to perform a task entirely in numpy, and then you incur massive slowdowns (often 20-100x). This is a common and contemporary problem in the python ecosystem.

GVR initially didn’t think that python needed to be faster either, but recently changed his mind. You can find a presentation on his motivations here: https://github.com/faster-cpython/ideas/blob/main/FasterCPyt...

Sam Gross comes out of that community so he’s familiar with the motivations for making raw python faster.

Re:A viable solution for Python concurrency by qstwall on Saturday October 16, 2021 @03:47AM (#28886250)
It will still be slow if the threaded code is in pure Python.
Re:A viable solution for Python concurrency by SoylentOrange on Saturday October 16, 2021 @09:51AM (#28887883)
> When pointed to the "ccbench" benchmark, Gross reported speedups of 18-20x when running with 20 threads.

I would be very happy with that, together with a 10% speed increase in single-threaded performance that they said would come as a result of these changes.

Re:A viable solution for Python concurrency by pansa2 on Friday October 15, 2021 @10:25PM (#28885066)
> So if the owner needs to check the shared ref count every time it changes the count of the non-atomic local count, isn't that basically all the negatives of a single shared atomic counter?

I think the owner only needs to check the shared count when it changes the local count to zero, not every time it changes the local count.

Re:A viable solution for Python concurrency by willseth on Saturday October 16, 2021 @11:08AM (#28888368)
> This just feels fighting the seemingly inevitable fate of Python.

Which is what, exactly? If you’re suggesting that devs will lose interest in Python then you’re simply not paying attention. Python is _gaining_ in popularity, over Java, C, even JS. It’s not that control over typing isn’t an issue, but developers are undeterred.

Re:A viable solution for Python concurrency by masklinn on Saturday October 16, 2021 @05:23AM (#28886606)
> So if the owner needs to check the shared ref count every time it changes the count of the non-atomic local count

It does not, it only needs to check the shared refcount when the local count reaches 0, to decide whether it can free the object immediately or whether it should mark it as "unowned".

Re:A viable solution for Python concurrency by DSingularity on Friday October 15, 2021 @10:10PM (#28885012)
The shared ref count is in the same cache line making the check basically free. Once you shift to shared then it is equivalent to the atomic shared ref count.