I think a part of the whole monad confusion comes from the fact that monads "add something" to a computation, which is valuable in Haskell but not in many other languages. Any function in C/Java/Python/Ruby is already free to log to stdout, truncate tables in the database, phone home to the NSA, modify files on the filesystem, throw exceptions, modify global state in the program, etc etc etc. You could say they all live in some `ErrorT (StateT s) IO` type of monad by default.

In Haskell, it's almost exactly the opposite: by default a function can do nothing but access its argument and do computations, but by adding more and more monadic contexts it can do more. This is useful for things like lazy evaluation, because the compiler can infer many more optimizations for pure functions than for functions that might do "anything". It also has benefits for security: a function with type `a -> a` is (almost) certainly not going to access the database or the filesystem. You can have the compiler confirm many assertions about the program when all side effects have to be declared through the type system.

For (say) a method in Ruby, which can already do anything, monads are not very useful IMO.

Additionally, monads, applicatives, and functors with currying and without currying will look very different. In a language that differentiates between single variable functions and a multivariate functions, even functors are weirder.

Haskell simplifies this all by just currying everything so that a function of two variables looks like a single variable function. Then you just have to define “applicative” and you get fmap that can apply to functions of any arity.

In python, by contrast, any equivalent definition of Applicative would be super non-idiomatic. On the other hand, you could write a very idiomatic ApplicativeOfTwoVariables or ApplicativeOfThreeVariables and so on.

The difference between programming with curried and non-curried functions runs really really deep, and I don’t think we’ve found the right abstractions yet that makes functor/applicative/monad analogs actually simplify developers’ lives in uncurried languages.

The real challenge for newcomers, I think, is to reconciliate this almost magical "monads allow side effects in pure functions" with the very mundane "a monad is a type with a 'return' and a 'flatMap' aka >>= method".

The reconciliation is simple: Monads don't allow side effects in pure functions (in Haskell). I know it's a very popular idea but it's also just wrong.

(Yet another in the long list of Very Popular But Also Just Wrong ideas surrounding this not-that-complicated concept.)

What the monadic interface to the IO type (very carefully phrased) in Haskell allows is building up a value to be interpreted at run time with a convenient API that looks very like it is an imperative programming language. But it actually constructs a large, complicated value that (per the linked article) is really just a big data structure, which at runtime Haskell lazily walks through (laziness being key here since the vast majority of programs have effectively infinite loops which mean the data structure representing the program would be infinite in size if strictly manifested in memory) to run the program.

In practice, you don't need to think about this distinction and you program with the IO type in the top-level IO context as if it is "really" an imperative language that is really doing what you tell it to do.

But "monad" has nothing to do enabling side effects. Monad is just an interface. Interfaces lack the power to fundamentally rewrite the nature of a language. If you just hacked out the place in the Haskell base library where IO declares that it is an instance of Monad, you could still do everything Haskell does today. Nothing about IO itself would change. What would change is that all the generic functions that have a Monad constraint would no longer work on IO, and you'd have to clone all of https://hackage.haskell.org/package/base-4.16.3.0/docs/Contr... into a new specialized set of functions that work on IO. But that's all that would happen, and a whole lot of code would have to be mechanically translated to use these new non-generic functions. Haskell would not break. Haskell would not suddenly become unable to speak on the network or read files. It would merely become less convenient to use for no benefit.

To build on this, it's worth noting that you can't implement your own IO monad in Haskell (or at least, not Haksell 98) on top of unsafePerformIO. At root you need some special magic to ensure that A's side effects are sequenced before B's in (A >> B). The type of (>>) does not guarantee this.

Well, you have seq. The type of seq also doesn't guarantee this, but the implementation does

I don’t think so? Seq is strict in its first argument, but that doesn’t provide a way of guaranteeing the order of side effects that I can see.

Edit: You may be thinking of pseq, but I don't think that's in the Haskell 98 report.

"Just Wrong" is a bit harsh. Monads are there to simulate side effects within the language, as opposed to the side effects that are built in to the language.

WARNING: the below contains math. Ignore if math makes you mad.

If "side effects" means the availability of certain operations such as mutating state, throwing errors etc, then monads certainly do have something to do with side effects in that these operations can often be described by an algebraic theory and algebraic theories have a direct correspondence to certain kinds of monads (the monad of free algebras of the theory).

No, they aren't. This is like saying "Iterators are in the language to walk over hash tables." No, they aren't. It is one of the things they can do, but it isn't what they are there for. What they are there for is to present a sequence of values. They aren't there to walk over trees, or walk over hash tables, or indeed, walk over data structures at all. Even if you think that last one, you don't understand them, because there are iterators that don't involve data structures at all.

You'll be stuck with a wrong understanding of them as long as you think that's what they are. That is one of the things they can be used for, but that's all. It isn't what they are. There are plenty of "monads" that are not being used for "side effects" and are perfectly pure no matter how you look at them, such as the list instance of monad. Rather than arguing with me about how if you bend the list a bit and sort of redefine "side effect" and if you squint a bit, the list instance really is related to "side effects", just don't look at it that way.

Yep! It's as if Haskell had an I/O DSL that built a data structure - a tree with all interaction with the outside world - and the runtime interpreted this data structure. (of course the compiler is allowed to optimize that away). That way, Haskell can achieve purity - it does I/O by building an IO value technically using pure functions!

Haskell then decided to use this DSL for arbitrary types, again just to build data structures. Then, people noticed that monads were really cool.

> Any function in C/Java/Python/Ruby is already free to log to stdout, truncate tables in the database, phone home to the NSA, modify files on the filesystem, throw exceptions, modify global state in the program, etc etc etc. You could say they all live in some `ErrorT (StateT s) IO` type of monad by default.

I think it's worth expanding this point out a bit. As well as your other point about knowing that a function from `a -> a` is sound. It's the declarative nature of monads that are truly valuable, especially when communicating with other devs through code. The explicit declaration of side-effects really comes into its own when applications are large. Often it means we don't have to go and 'look inside' a function to see what its interaction with the world will be.

Obviously there's the 'other stuff' too:

* Compositionality

* Package up complex boilerplate so it's never written again

* Ability to optimise the core implementations later without rewrites (I'm thinking for more complex domain managing monads, rather than the 'built-ins')

> For (say) a method in Ruby, which can already do anything, monads are not very useful IMO.

That's probably true for dynamic languages where there's no declarative value to using monads; although the encapsulation of common behaviour is still useful.

I develop language-ext [1] a functional framework for C# that tries to bring the benefits of monads (and other functional goodness) into C#. It is true that a programmer can still launch nuclear missiles in between lines of code if they want to. But with a bit of self-discipline it's entirely possible to reap similar benefits to what you'd see in Haskell. LINQ in particular is very powerful for this. The compiler doesn't get a look in, like it does for Haskell, and that is definitely a downside - because it can't ever be optimised as well as Haskell can, but that's just a trade-off.

Of course it isn't Haskell, but it's still possible to get 95% of the benefits.

[1] https://github.com/louthy/language-ext

Oh I understand the usefulness of individual monads and infact am very happy that the NodeJS community finally switched to promises from callbacks (even though they violate some of the monad laws, you still get a lot of the same benefits).

I just don't think the generalization of those concepts is very interesting. Monoids seem a lot more interesting to me, especially when it comes to splitting up computations (mapreduce) or partial computation caching, you can really come up with quite a few useful ones

Monads in comparision seem boring.

I tend to agree on this. Understanding the power introspection trade-off is important. Monads, while supplying a lot of power, aren't terribly introspectable. Applicative functors have some degree of introspectability. While functors are the most introspectable, but have the least amount of power. Understanding these trade-offs is important when deciding to use the right of my power when solving a problem.

That's pretty typical for understanding mathematical concepts. You start knowing the definition (interface), but with absolutely no intuition for it, so it seems mysterious. Then you work though some examples, theorems, use-cases etc. to build understanding, and the concept seems to become more than the sum of its parts. At some point, you look back at the definition with your new intuition and realize: There really isn't anything more to it than this definition! Everything else just follows from that!

I do know what concepts map to it, I understand all common monads (including IO) as well as monad transformers. I think ZIO style monads (combined reader, error and effect monad) are pretty awesome for building backend apps, especially the reader bit which solves the common "contextual data" problem typically solved by threadlocalstorage or CLS or OOP in other languages

I still find "monad" as a concept underwhelming and meaningless. I can put `flatMap` and `wrap` on a lot of things but that won't make them interesting or useful. The individual use cases are interesting, the concept isn't - seems like an unnecessary overgeneralization.

I agree with you that there's no inherent magic to the monad concept, but there really is something neat about the monad generalization -- it's not just a senseless abstraction. That meme "a monad is just a monoid in the category of endofunctors" has merit here. If you already believe that monoids are a useful enough concept, and also if you believe functors are useful enough, then this is a basic structure that pops out. It's a higher-order one, which makes it that much harder to grasp, which only amplifies any possible disappointment :-)

One place this generalization is useful is in defining other structures, for example traversable functors, which as part of their definition need to commute with every other monad. Sure, you won't be using most monads, but it's easier to define a traversable functor that works for all monads instead of the small subset of computationally useful ones.

The deep part to monads is that they (well, a large class of them, the "finitary" ones) end up classifying Lawvere theories. This means that each monad is secretly encoding some suite of composition laws for operators of arbitrary arity. I'll just give an example with the list monad. If you have a list [1,2,4,2], then you can think of it as being a function, where `map` takes each integer (a "variable") and substitutes it for some value, yielding a list -- so in this way the list is an operator with an input per int. The monad part is that you can compose it with other operators by using `flatMap`. For example, if 1 -> [1], 2 -> [2, 3], and 4 -> [2], then the `flatMap` gives a new "function" [1,2,3,2,2,3] from integers to lists by splicing in other operators (and `wrap` is a way to tell `flatMap` that you want to preserve a particular variable, so to speak). I think this is a way to think about why monads have anything to do with computation -- a value of `m a` where `m` is a monad is a computation with "`a`-shaped holes," and `flatMap` is a way to plug other computations into these holes.

Anyway, that's a lot to say for this, but I'd come across Lawvere theories earlier this year and it gave me a new appreciation for monads after years of just using them while programming. (Disclosure: I'm also a pure mathematician, so I understand if others won't share this appreciation.)

To expand on the “classifying Lawvere theories“ part, the relevant concept here is algebras over a monad. An algebra a over m is determined by a function merge :: m a -> a which “plays nice“ with the monad structure [0].

For example, integers with the sum function form an algebra over the list monad, because

1) for any list of ints, the sum is an int

2) the sum of [n] is just n

3) summing recursively is the same as summing over the flattened list.

More generally, the algebras over the List monad are exactly the monoids, with ‘merge [a, b]‘ as the monoid operation. Algebras over the Maybe monad are pointed sets, with ‘merge Nothing‘ as the distinguished point. A lot of algebraic structures (Lawvere theories) can be defined by a monad this way.

[0] Laws that ‘merge‘ should satisfy: merge (return x) == x fmap merge (merge mx) == merge (join mx)

I don't understand how this is related to monads - there is nothing in the general monad interface that even allows you to go `m a -> a`, so something being a monad doesn't seem sufficient to be able to define that algebra.

Seems to me that the entire interesting bit is "provided" by `a` being a monoid. (I agree that monoids are really interesting)

Here's another way to say the general idea, by way of example. Secretly, list is "the" exemplary universal monoid. Given any `a`, then `list a` gives you the free monoid generated by `a`, with composition given by `++` and the identity given by `[]`. Now, let's say you have some `a` such that you're able to come up with a function `ev : list a -> a` that's a monad algebra (i.e., it satisfies the two axioms given by yccs27). Then you're able to conclude that `a` is a monoid(!) and you can extract the monoid structure in the following way: `\ (x y : a) -> ev [x, y]` is the binary operation and `ev []` is the unit. (In particular, `ev` must be some `list.fold`.)

I called it `ev` because it's an evaluator/interpreter. To be able to write such a `m a -> a` function, `a` needs to have the same kind of structure that `m` is somehow representing, and you need to be able to map the operation that `m` "contains" to corresponding operations for `a`.

This is that business about free monads being useful for organizing how you write interpreters I believe.

> Then you're able to conclude that `a` is a monoid

Yep, which is why I'm saying all the interesting bits come from `a` being the monoid in that example.

In the free monad interpreter example, I have no idea though.

If there's a point to any of this, I think it's monads let you package up some algebraic structure in the form of a function `m a -> a` that satisfies certain properties, and then you can pass such a function around without any reference to the fact that `a` implements that algebraic structure. This function is the interface. (Is there real practical significance? I'm not sure, but I'm not trying to argue that :-) )

> all the interesting bits come from `a` being the monoid in that example.

Yeah, there's nothing deep there. I just said what I said to underscore that the converse is true, that if anyone were to hand you such a `list a -> a` function then it's encoding some monoid structure on `a` (one of often very many).

> there is nothing in the general monad interface that even allows you to go `m a -> a`, so something being a monad doesn't seem sufficient to be able to define that algebra.

You've got the direction backwards: the point is to characterize all `m`-algebras, not to construct particular ones.

This is closely related to the usual duality between types and values. It's very common for people new to TypeScript, for instance, to think that `{a: string} | {b: number}` should be `{a: string, b: number}`, because they're thinking primarily in terms of records as sets of fields, not types as sets of values. But making the type bigger makes the set of things you can say about its values (i.e. the space of functions out of it) smaller. Algebras are similarly the "value-like" counterparts to "space-like" monads.

I have no idea what you mean.

The important point is that each monad (e.g. list) corresponds to an algebraic structure (like monoid). To study the set of algebraic structures, you can also study monads

It is a pretty good concept to scare away non-monadic ppl. Cool word too

This is an underrated observation IMO. Mathematics gets mystified so often when it really is just a very useful (and aesthetically appealing) method to approach problems. There's no secret sauce.

In the definitional sense it is underwhelming, like with many algebraic structures the diversity of instances leads to interesting results. The behaviour of these instantiations have a completely different character even though they instantiate the same pattern

  replicateM :: Int -> [a]      -> [[a]]
  replicateM :: Int -> Cont r a -> Cont r [a]

  sequenceA :: [IO a]             -> IO [a]
  sequenceA :: Maybe (x -> a)     -> (x -> Maybe a)
  sequenceA :: Map key (Parser a) -> Parser (Map key a)

In fact both of them require only an Applicative pattern (aka n-ary lifting) which is weaker than Monad

  liftA0 :: Applicative f => (a)                -> (f a)
  liftA1 :: Functor     f => (a -> b)           -> (f a -> f b)
  liftA2 :: Applicative f => (a -> b -> c)      -> (f a -> f b -> f c)
  liftA3 :: Applicative f => (a -> b -> c -> d) -> (f a -> f b -> f c -> f d)
  liftA4 :: Applicative f => .. 
  where
    liftA0 = pure
    liftA1 = fmap

but has more interesting properties than Monad. Because there is no dependency between Applicative computations they can be run Concurrently[1] or Backwards[2]. They are also closed under Compose-ition[3]. This is not even mentioning what makes an Applicative or a Monad. Functor instances are unique[4] but a single type constructor can have different law-abiding Applicative instances.[5]

[1] https://hackage.haskell.org/package/async/docs/Control-Concu...

[2] https://hackage.haskell.org/package/transformers-0.6.0.4/doc...

[3] https://hackage.haskell.org/package/base-4.16.3.0/docs/Data-...

[4] https://stackoverflow.com/questions/19774904/are-functor-ins...

[5] https://hackage.haskell.org/package/idiomatic

Yes. They aren't much more complicated in any other context, either.

But it is interesting how widely widely that interface is supported. Eg functions are also monads (where function composition corresponds to monad's fmap.)

> it turned out they're just a generic interface

Yep. It's a generic interface for the interpreter pattern. (That one code pattern from OOP fame that is as powerful as macros.)

All of the importance of the type comes from what it represents, not on its own complexity. All the hype people get once they understand it comes from learning how to write interpreters.