News for hackers. Stuff that matters.
Re:Our Plan for Python 3.13 by btilly on Thursday June 15, 2023 @11:08AM (#36341592)
My point of view is that anyone who wants to write multithreaded code, shouldn't be trusted to. Making it easier for people to justify this kind of footgun is a problem.

Also, no matter how much you wish it otherwise, retrofitting concurrency on an existing project guarantees that you'll wind up with subtle concurrency bugs. You might not encounter them often, and they're hard to spot, but they'll be there.

Furthermore existing libraries that expect to be single-threaded are now a potential source of concurrency bugs. And there is no particular reason to expect the authors of said libraries to have either the skills or interest to track those bugs down and fix them. Nor do I expect that multi-threaded enthusiasts who use those libraries in unwise ways will recognize the potential problems. Particularly not in a dynamic language like Python that doesn't have great tool support for tracking down such bugs in an automated way.

As a result if "no GIL" ever gets merged, I expect that the whole Python ecosystem will get much worse as well. But that's no skin off of my back - I've learned plenty of languages. I can simply learn one that hasn't (yet) made this category of mistake.

Re:Our Plan for Python 3.13 by samsquire on Thursday June 15, 2023 @11:29AM (#36341939)
My deep interest is multithreaded code. For a software engineer working on business software, I'm not sure if they should be spending too much time debugging multithreaded bugs because they are operating at the wrong level of abstraction from my perspective for business operations.

I'm looking for an approach to writing concurrent code with parallelism that is elegant and easy to understand and hard to introduce bugs. This requires alternative programming approaches and in my perspective, alternative notations.

One such design uses monotonic state machines which can only move in one direction. I've designed a syntax and written a parser and very toy runtime for the notation.

https://github.com/samsquire/ideas5#56-stateful-circle-progr...

https://github.com/samsquire/ideas4#558-state-machine-formul...

The idea is inspired by LMAX Disruptor and queuing systems.

Re:Our Plan for Python 3.13 by btilly on Thursday June 15, 2023 @11:33AM (#36341982)
And your approach can be built into a system that does multi-threading away from Python, thereby achieving parallelism without requiring that Python supports it as well.

That's basically what all machine learning code written in Python does. It calls out to libraries that can themselves parallelize, use the GPU, etc. And then gets the answer back. You get parallelism without any special Python support.

Re:Our Plan for Python 3.13 by zo1 on Thursday June 15, 2023 @12:17PM (#36342772)
Just to add a bit of my opinion after reading your comment in the context of this thread and not to the merit of your idea. You are precisely the type of person I'd keep very very far away from multithreading in any business software project and also why I advocate the GIL to stay. If you want to do that, go solo in your own time, or try apply for a research position in some giant tech Co.
Re:Our Plan for Python 3.13 by coldtea on Thursday June 15, 2023 @11:17AM (#36341734)
>My point of view is that anyone who wants to write multithreaded code, shouldn't be trusted to. Making it easier for people to justify this kind of footgun is a problem.

It's 2023 already. 1988 called.

Re:Our Plan for Python 3.13 by btilly on Thursday June 15, 2023 @11:39AM (#36342081)
Did you know that Coverity actively REMOVED checks for concurrency bugs?

It turns out that when the programmer doesn't understand what the tool says, managers believe the programmer and throw out the tool. Coverity was finding itself in situations where they were finding real bugs, and being punished for it by losing the sale. So they removed the checks for those bugs.

I'll revisit my opinion of multithreaded code when things like that stop happening. In the meantime there are models of how to run code on multiple threads that work well enough with different primitives. See Erlang, Go, and Rust for three of them. Also, if you squint sideways, microservices. (Though most people set up microservices in a way that makes debugging problematic. Topic for another day.)

Re:Our Plan for Python 3.13 by biorach on Thursday June 15, 2023 @01:11PM (#36343637)
> Did you know that Coverity actively REMOVED checks for concurrency bugs?

Source?

Re:Our Plan for Python 3.13 by btilly on Thursday June 15, 2023 @01:20PM (#36343753)
Re:Our Plan for Python 3.13 by yjftsjthsd-h on Thursday June 15, 2023 @02:07PM (#36344373)
Specifically, on the last page of that:

> As an example, for many years we gave up on checkers that flagged concurrency errors; while finding such errors was not too difficult, explaining them to many users was.

(And thanks; I was also wondering about that)

Re:Our Plan for Python 3.13 by yjftsjthsd-h on Thursday June 15, 2023 @11:56AM (#36342367)
This is very witty and all, but what does it mean?
Re:Our Plan for Python 3.13 by DonHopkins on Thursday June 15, 2023 @01:45PM (#36344090)
I think he was subtly Rick Rolling you.

https://en.wikipedia.org/wiki/Never_Gonna_Give_You_Up

On 12 March 1988, "Never Gonna Give You Up" reached number one in the American Billboard Hot 100 chart after having been played by resident DJ, Larry Levan, at the Paradise Garage in 1987. The single topped the charts in 25 countries worldwide.

Re:Our Plan for Python 3.13 by coldtea on Thursday June 15, 2023 @02:51PM (#36344959)
Actually it just meant that this is a tired old argument when C/C++ programmers were new to multithreaded code.

Now we have languages and language facilities (consider Rust, Haskell, and others) to make it much safer. Same with green threads and what Go and now Java does.

Re:Our Plan for Python 3.13 by btilly on Thursday June 15, 2023 @07:56PM (#36348821)
And yet, the actual proposals for how to do multithreading in Python are essentially the same as the old C/C++ approaches. And since that hasn't changed, there is no reason to expect better results.
Re:Our Plan for Python 3.13 by iskander on Thursday June 15, 2023 @12:26PM (#36342913)
>My point of view is that anyone who wants to write multithreaded code, shouldn't be trusted to. Making it easier for people to justify this kind of footgun is a problem.

Out of curiosity, have you done any Rust programming and used Rayon?

It's hard to convey how easy and impactful multi-threading can be if properly enclosed in a safe abstraction.

Re:Our Plan for Python 3.13 by kortex on Thursday June 15, 2023 @12:49PM (#36343268)
Python is dynamic AF and Rust's whole shtick is compile-time safety. Python was built from the ground up to be dynamic and "easy", Rust was meticulously designed to be strict and use types to enforce constraints.

It's hard to convey how difficult it would be to retrofit python to be able to truly "enclose multithreading in a safe abstraction".

Re:Our Plan for Python 3.13 by btilly on Thursday June 15, 2023 @01:08PM (#36343586)
I have only read about and played a tiny bit with Rust. But as I noted at https://news.ycombinator.com/item?id=36342081, I see it as fundamentally different than the way people want to add multithreading to Python. People want to lock code in Python. But Rust locks data with its compile-time checked ownership model.

See https://blog.rust-lang.org/2015/04/10/Fearless-Concurrency.h... for more.

Re:Our Plan for Python 3.13 by hklzr100 on Friday June 16, 2023 @06:21AM (#36354217)
This is a good point. With tsan and helgrind multithreading becomes manageable in C/C++, nothing like that exists (or will exist) for Python.