Take for example logging or tracing - we almost always in a modern backend application want an ambient trace or span ID and a log destination. What we don't want is to have to add those as parameters to _every function_.

So we want to paint these functions with the "logger" and "traced" colors, which respectively allow a function to send messages to an ambient logger via log(message: str) and to get the current trace context via getTraceContext() each with no other arguments.

The compiler will tell us if we call these functions from an unlogged, untraced function, so at a very high level - say at the RPC request level, we paint them early on. In Haskell we might even be able to do something like

    handleReq req = 
      withTraceSpan (...)
      . withLogging (...)
      $ handleReqInner req

Having too many effect handlers is never a problem either - you can always call an untraced function from a traced function. The "trace" effect handler just falls off at that call. Or in other words: you can always call a function whose colors are a subset of the call-site's.

This is somewhat a foreign concept to people who haven't worked with type systems like this and whose only concept of the color of function is purely binary - either async or not. In a good typed effect system, the compiler will assist you in knowing where you're missing an effect handler, and it will be easy to add those effect handlers sufficiently far from the business logic that you won't have to think about it.

The way effect systems are typically added to languages without this level of type system is that the effect "fails". Like making a database call in Python without "with use_database(...):" that provides the database context. In those languages and with those effects, typically the default behavior is the effect's methods are always available, but may either be a no-op/return a nullish value or throw an exception.

As an example, the way tracing works on Node.js is that you have to explicitly declare a span (or use a plugin for a framework which does that for you) and inside that span, you can do something like getSpanId() and it'll return a value. Outside a span, it returns null.

The worst case scenario is you have a language and framework where it's both difficult to annotate the types and difficult to provide some ambient context. In those cases, you're back to (essentially) passing your trace span, your logger, etc as arguments to every function again. Not ideal.

The only ways I know of to reliably make this kind of change (or just about any nontrivial change) to an interface are increasing version numbers and letting clients suffer or having a monorepo and fixing all the clients yourself

It can help in a closed code base. In an open code base, you could inadvertently change a public API, breaking callers that are unknown to you, if the type signature of something public is inferred.

If a function needs capability X then it must require that capability. Simple as that.

The effects system effectively acts as an abstraction for some side effect, like an interface, and gets ride of a lot of the boilerplate code needed for mtl or custom transformer stacks.

Also, in strict typing it's pretty easy to refactor with modern linters and such, it actually makes refactoring an API change delightfully simple, just get rid of the red squiggle lines telling you you types are wrong.

The only thing for it is to push a new, incompatible version and other people will have to migrate. So you're pushing off the work onto them.

I don't see how an effects system helps with this much? It might help you better understand how you painted yourself in a corner, but I don't see how it helps you get out of it.

With a type system which understands effects, at least the compiler can give you very accurate help in fixing call sites.

Suppose you have have a language with two categories of functions, those that can fail (returning an error) and those that can't.

It's nice that within the "functions that can fail" category, you don't have to worry about what kind of error it might be. Error propagation can happen in a generic way. Adding a new kind of error doesn't change any API's.

If instead, you have different kinds of errors and declare them everywhere, you end up with a situation like Java's checked exceptions. The problem is being too precise, which doesn't leave room for changes later.

Similarly, we could be very generic about effects. Maybe we could just say "this function has effects" and treat them all the same? By not being precise about what the effects might be, we aren't promising too much, so we allow ourselves room for change.

The downside is that the caller has to assume any effect is possible. This is a fundamental tradeoff between caller and callee convenience. You need to be specific enough for the caller to be able to deal effectively with the effects you declare, but not too specific, or you're painting yourself in a corner.

Also, breaking API changes happen in every language, on every codebase, ask the time. At least with static types and a common linter I can refactor and be a hundred percent sure it works since it compiles, whereas in Python for example I can't be sure unless I add tests and stuff, even then...

I don’t believe that’s true. The very existence of Stack and Stackage is testament to how significant the compatibility problems had become in the Haskell ecosystem.

Even if it were true, Haskell world is tiny and (in)famously has the motto “Avoid success at all costs”, so it isn’t necessarily a representative example of practices that would be effective in other contexts.

The advantage with a type system as explicit as Haskell’s is that at least if there is a breaking change in an interface you depend on, you get quite good information about where the problem is and what you need to do to fix it.

Also, breaking API changes happen in every language, on every codebase, ask the time.

Some people take backwards compatibility extremely seriously, and code that depends on their interfaces still works just fine many years after it was written. The pace of development is usually slower when robust standardisation is part of the process, but that isn’t necessarily a bad thing given the stability you get in return.

At least with static types and a common linter I can refactor and be a hundred percent sure it works since it compiles

This is a common claim in the Haskell world, but of course it’s not really true. It always reminds me of Knuth’s famous quip, “Beware of bugs in the above code; I have only proved it correct, not tried it.”

Dependency migration is the sort of thing that's easy enough in small examples and gets harder as you scale up.

No. Just use stack + Stackage and you have a huge set of packages which are mutually compatible.

(You can also use cabal-install with a Stackage snapshot, I think. Not sure though. Cabal-install has also improved massively and doesn't really do "global" installation of dependencies, so it'll help a lot... but you may still run into difficulties making different versions of your project's dependencies work together.)

Less so than it used to be. Cabal received a big update a few years ago, and the new-style commands now operate more like many other package managers in terms of isolation etc.

The standard head function is partial, yes, but that's not a big deal at all.

We've been there with Java. An API takes a Runnable. You need to do something that does IO, so you catch the exception... and then what? Log and suppress it? This is how bad code happens. At best you get chained exceptions.

With Callable, they got smarter and declared it to throw Exception, which means it can only be easily propagated by other methods that declare they throw Exception. So it's a viral declaration, unless you engage in workarounds like exception wrapping.

You don't need an effect system for this to happen, though. In Go, you don't have the problem with different kinds of errors, but you can still classify functions into two categories: those that may return an error, and those that (apparently) always succeed. If you need to fix the API, you might end up with lots of refactoring [1].

Mistakes at the API level can be hard to fix without touching lots of API's. This is unsolvable in general. The only way to avoid workarounds (via escape hatches or by mocking things out) is to standardize whatever you can so it works together.

[1] https://www.dolthub.com/blog/2020-11-16-panics-to-errors/

Now, instead of having a single function that has some effects, you have a family of closely-related functions, each with different effects. These functions are incompatible with each other. The function-coloring problem has gotten worse! :)

(Generic types are still useful though.)

Never understood why the browser didn’t ship a full access to something like an EventEmitter. Since you can dispatch events on window document DOM elements etc seems like being able to subscribe to events in a more arbitrary fashion that worked in parallel to the other events would have been useful and solved callback hell all the same

the idea should generalize to whatever thread-pool/effect constraints you want to express, but the paper and implementation are focused on keeping UI effects on the android main thread.

Think of the classic "zlib" style C libraries. Often they can't assume anything about the target platform, even basic stuff like memory allocation or I/O, because there's such a huge variety compilers, standard libraries, and restrictions.

So just about every such library exports an "API context" struct, full of function pointers. These then allow the caller to pass in implementations for malloc, free, threading, I/O, etc...

The trick is that these can have default implementations. So the function pointer in the context for allocation can simply point to "malloc()". This can of course be overriden, so the capability is there, but the caller need not do anything by default.

Features like logging and tracing can be similarly sent through to default implementations that do nothing or call standard APIs in some naive (but sufficient) way.

The common modern example being async functions, I would say it's actually quite surprising that JavaScript can't seem to provide the sane default behavior of "block this non-async function until the async function it calls returns a value".

The reason it can't is simply that it chooses not to provide a runtime with a true threading model - blocking a non-async function means literally blocking the entire runtime. But there's a possible world in which a "colored" (async) function could be used directly in a synchronous function using an escape hatch provided by the runtime, and then async functions would no longer be colored in the same way.

    R process<R, E extends Exception>(ThrowingFunction<Long, R, E> processor) throws E { ... }

Checked exceptions a "bolted on" effect that sits outside the usual type system in Java, and their presence on a method signature "colours" it in a way that an effect described inside a type system would not. More of a stain than a tint, if you will.

Java 8 resolved it by quietly pretending checked exceptions never happened, which works fine until (1) you get unhandled unchecked exceptions in all their glory at runtime, and (2) you want to use one of the Java 7 or earlier methods as a "processor" but, oh no, it throws an exception.

(It's a deeper colouring than JavaScript's async/await keyword, which is just syntactic sugar around continuations in callbacks and returning promises - you can pass an async "processor" function to a "process" function that has no idea what it's getting and get back the promise you'd expect from it.)