This is the first time I've seen threads referred to as a way to avoid pain.
Feel free to burn me at the stake.
I think a large part of that for me is because early in my career I got involved in Unix kernel work, first writing device drivers and later doing Unix ports, including both porting a swapping Unix to paged hardware (and rewriting pretty much all process and memory management from scratch) and porting a paging Unix to swapping hardware.
That was followed by a few years largely doing embedded stuff on systems too small to have operating systems or memory management units. For those I'd write a small kernel that supported threads and scheduling (sometimes cooperative and sometimes preemptive) and then write the embedded applications much like I would have written them on Unix, using threads instead of processes.
So for me threads are easy and things like async/wait seem overly complicated and less clear because I came at concurrency from the operating system side of things and the hardware side of things (e.g., dealing with interrupts) where thread-like thinking is natural and perhaps required.
Maybe it is conceptually harder for people who come at it from the application side of things?
On the app side, "I want to do this thing that might take a while without blocking the UI, and then do something with the result, and I have zero need to communicate with the code that's running the task beyond giving it its arguments and getting its result back," is the most common use case. Task parallelism is tailor-made for that kind of problem.
Using a dedicated thread, on the other hand, gives you some raw material you can use to solve that problem, but leaves you to handle all the messy details on your own. And doing it the easy way - one short-lived thread for every thing you need to do in the background - is expensive, while doing it the correct way and implementing a thread pool is essentially just committing to Greenspunning your own personal task parallelism library.
E.g. GUI code -> async, because you must do everything on the UI thread. Heavy lifting in a GUI app -> threads. Traditionally you'd do this with a huge mess of callbacks, but these days it's reasonably easy to just write the GUI portion using async and have background computations run in threads signal their results through futures into the async code, which makes everything so much easier.
There are times when one might want async executors like node's event loop or rust's tokio/async-std, but in other situations one might want os threads or "green threads" with a RwLock or Mutex.
What about a few long running threads that use async internally? ;-)
Parallelism and concurrency are orthogonal concerns. Multiple threads can run on the same core, while concurrent awaits can run on separate cores (depending on the language). Or the opposite!
Threads definitely solve wasted time waiting on I/O, but OS-level threads are a very limited resource in many environments, which can cause problems more quickly in programs with unbounded concurrency, and they tend to have a lot of startup cost, so they have to be created judiciously to avoid hurting performance.
Languages like Go and Elixir give the programmer access to unlimited lightweight “threads”, and that works great for concurrency, without requiring developers to re-color their functions.
One additional note is that some OSes (including Linux) have struggled to support true nonblocking File I/O for many years, so you literally have had to use multiple OS threads for I/O concurrency... one for each file operation in progress. Otherwise you would block the async executor! Languages often handle this implementation detail for the programmer, so you just don’t realize you’re using a thread-per-await sometimes.
(io_uring seems to finally solve this in a meaningful way on Linux)
STM puts the cost of locking into everything, ie slower default operations, whilst still being concurrency unsafe. Deadlocks can occur.
There are proven good solutions for fast and safe concurrency with ownership tracking and a good compiler/scheduler. No, not Rust. And not STM. We fought with this for years.
Do you have any pointers to these solutions for the uninitiated?
But I’m not sure this is tractable in either implementation effort or runtime overhead.
If so, you can't do IO inside a transaction (it won't compile), and any value not on a TVar is thread-local.
No?
I'm responding to the idea of the 'colour' of functions and suggesting how I think it should be in Ruby - all side effects transactional, but IO irrevocable. Drop the idea of TVar and make all mutable locations transactional.
It is correct that the standard library provides you with a newtype-based identity monad. But as you have said, you can also write own `type Id a = a`. What's wrong with that?
The fact that it doesn't work:
> - The type synonym 'Id' should have 1 argument, but has been given none
> - In the instance declaration for 'Monad Id'
> | instance Monad Id whereBut I get your argument now. What you are saying is that we want some kind of a way to define functions that are either pure or return a monadic value. Currently that's not doable because these two have distinct types. It may be difficult to unify them via the type system. This is even more difficult for higher-order functions that take other functions.
Indeed I did, sorry, unfortunately too late to edit. The following reproduces the error:
{-# LANGUAGE TypeSynonymInstances #-}
type Id a = a
instance Monad Id where
x >>= f = f x
return = id
(The last two lines can be ommitted, since GHC aborts before checking the instance body. Edit: which is why I didn't notice I had the arguments to >>= backward; fixed?)> It may be difficult to unify them via the type system. This is even more difficult for higher-order functions that take other functions.
IMO, the main difficulty in unifying `a ~ Monad m => m a` is inferring `m ~ \a1-> a1` in a way that allows `\a1-> a1` to inherit `Id`'s instances. It's a understandable deficiency, but it is a deficiency.
The idea in the end is to provide a language for working on memory concurrently. You either choose speak it or not.
In any case, the unsafeness of unsafePerformIO and importing an FFI function as a pure function are not reflected in the type in Haskell. These are things that can and do happen.
My point was that the ability to go outside the bounds of a language does not necessarily make the language useless.
There are no implications, that's the point.
> the unsafeness of unsafePerformIO and importing an FFI function as a pure function are not reflected in the type in Haskell
Yes, that is correct behaviour; if it was reflected in the type it'd be call safePerformIO.
It's reflected in safe Haskell annotations. Unfortunately nobody cares about those. Haskell has a culture problem regarding safety, just like most other languages, both high level and low level (with the exception of Java, that don't make data races UB)
No, in any other language, one has to have an understanding of the codebase and what is mutating and what is not instead of relying on the language to do this work for them, which, granted, can be a lot more work.
dismissing things because 'i dont want to deal with it because it is hard' is not the same as dismissing it because 'this is not possible', this argument is the former.
see also: ocaml / scheme style FP, which can be 99.9% as pure-functional and type-checked as haskell if one wants to build their system that way.
That's not a reasonable thing to expect.
What this proposal misses is that reasoning about software transactional memory in functions all but demands being able to abort and retry code to make progress, and that therefore means having a high level separation between transactional code and IO code.
Suppose you had a banking application, you would like to use STM (as in the "ractor" example in this article) to move balances from one to another. In Haskell, you'd perform STM actions on those TVars (transactional vars) and use the default retry policy of STM to automatically construct an order of operations such that each operation appears to occur atomically. (STM doesn't impose a clock based ordering but I don't think it would be tremendously difficult to do so.)
If you don't use Haskell's "check" or "retry", you find that your code will abort and fail with near certainty. You can't perform lots of transactions on tvars and not, occasionally, read from a var another transaction is writing.
So you add "check" calls to verify your preconditions and postconditions such as balances cannot go negative, and you add "retry" calls to automatically retry transactions that fail because of concurrency.
Now imagine doing that in a modern Ruby codebase, or a modern JavaScript codebase. There's so much mutation that happens in these environments, can you guarantee within a Ruby function that it hasn't mutated another variable with ease? That it hasn't written to disk, made an API call, or changed some global state?
Haskell pioneered the use of STM. It's easier to reason about in Haskell because the world of the STM can be isolated from the world of effects due to the IO monad. It's easy to declare "this function is blue" and "this function is red" and the never the twain shall meet.
In any other language, all bets are off.