News for hackers. Stuff that matters.
Re:Monads are a Class of Hard Drugs (2008) by csense on Saturday December 29, 2012 @12:56AM (#4981294)
Anyone who understands monads is on a class of hard drugs.

I'm still looking for someone who can present a cogent explanation of monads using only Python.

(I'm willing to replace "Python" in the above challenge with "any language I know well," which would include Python, C, C++, Java, JavaScript, and quite a few others which I won't bother to list here. Notably lacking from this list are Lisp, Scheme, Caml, Haskell, Prolog, Erlang, and most other languages whose view of the world is "weird." NB not that I'm against learning such languages; it's more a matter of available tutorials and time.)

Re:Monads are a Class of Hard Drugs (2008) by tel on Saturday December 29, 2012 @11:43AM (#4982553)
They're not smart to do in python. Monads are an expressive, simple pattern of coding that can be type-checked. In dynamic languages it'll just look like a lot of unnecessary line noise. In HM type systems it's a great way to help you pass contexts around in a way lets you focus on your values.

But if you just want to speak Python, let me try to translate.

---

As a motivation for monads in Python, we're going to try to make "total" Python. To do so, we have to eschew execptions. This means we'll write stuff like

    def mypop(l):
      if len(l) > 0:
        return l.pop()
      else:
        return None
which, if we pass it a list of numbers, is guaranteed to return either a number or None---I've sidestepped the empty list exception. Now let's say I want to compose this function. For instance, my list of numbers is a list of bids and I have a function where I can try buying something with my bid.

    def buy(tendered):
      effective = tendered*1.2 # there's a discount!
      if market_open():
        if effective > 60:
          return True
        else:
          return False
      else:
        return None
So I believe that buy takes numbers and returns Booleans, so I might think that

    def trybuy(l): return buy(mypop(l))
takes lists of numbers and returns a boolean as to whether I've bought it. Of course, I have to be smart enough not to send in an empty list otherwise the discount in 'buy' will cause an error. I also have to be sure that if I try buying on days the market is closed to handle the 'None'. In Haskell, we'd write these constraints into the types and values using a Maybe type.

Here's a Pythonic Maybe type (kind of not pythonic though with the magic sentinel value, but we don't want to be able to sneak 'None's in).

    class Maybe(object):

      __sentinel__ = object()

      def __init__(self, obj = __sentinel__):
        self.isNothing = False
          if obj == self.__sentinel__:
            self.isNothing = True
          else:
            self.just = obj

    def just(x): return Maybe(x)
    def nothing(x): return Maybe()
Now we can have 'mypop' and 'buy' to return Maybe types to cover the cases of empty lists and closed markets.

    def mypop(l):
      if len(l) > 0:
        return just(l.pop())
      else:
        return nothing()

    def buy(tendered):
      effective = tendered*1.2 # there's a discount!
      if market_open():
        if effective > 60:
          return just(True)
        else:
          return just(False)
      else:
        return nothing()
So now we've abstracted the 'None's out a bit. 'mypop(l)' is always a "Maybe(Bool)" and you can inspect it to determine what the case is like 'mypop(l).isNothing'. Of course, this isn't any different from using None's and checking with "x == None" except it's more explicit: if you forget that 'mypop' returns Maybes then your program will throw an error on every use... instead of just the ones where you could have gotten a None without noticing for a while. (This is where the "You Probably Already Invented Monads" idea comes from, btw.)

In Haskell we'd say that mypop has type

    mypop :: [Float] -> Maybe Float
and buy has type

    buy :: Float -> Maybe Bool
but this makes it harder to make 'trybuy :: [a] -> Maybe Bool' since buy doesn't accept Maybes. It's also pretty clear, though, that if we send in an empty list we shouldn't even attempt to buy anything (our paycheck hasn't come in yet), so let's write that as a failure mode.

    def bind(maybe_something, maybe_process):
      if maybe_something.isNothing:
        return nothing()
      else:
        return maybe_process(maybe_something.just)
and now we can write trybuy such that it respects maybes with almost no additional noise

    # trybuy :: [Float] -> Maybe Bool
    def trybuy(l): return bind(mypop(l), buy)
The pattern we've captured here---the creation of lots of explicit context to help make functions more total and fail more quickly if you misuse them and the use of 'bind' to write functions that don't need to know about context on their input---is the Monad pattern. In Haskell, we recognize that many, many kinds of context are Monadic and thus can have default compositions programmed in. 'bind' is heavily overloaded and whenever you're writing code with context you use it often to compose functions without paying the complexity cost of routing that context around.

It's obviously not a really great Python pattern. This largely has to do with the fact that Python isn't terribly explicit about what's going on: you have to remember where exceptions and edge cases can fall out. In Haskell, we just use types to encode all of that into the documentation and the type checker ensures that we never mess up.

Re:Monads are a Class of Hard Drugs (2008) by csense on Saturday December 29, 2012 @03:48PM (#4983392)
Thanks for the lengthy reply!

So Monads are a complicated error-propagation framework that has a harder-to-understand conceptual model and results in longer code compared to Python's exceptions.

Look at any serious program in the C language -- there are enormous amounts of code devoted to error handling.

Look at the problem with Java's checked exceptions. The other day I needed to call a static method reflectively. This is what I ended up with:

  // java code to invoke a named method on a named class
  String the_class_name;
  String the_method_name;

  try
  {
     c = Class.forName(the_class_name);
     result = (Value) c.getDeclaredMethod(the_method_name).invoke(null);
  }
  catch (ClassNotFoundException e)
  {
     throw(new RuntimeException(e));
  }
  catch (IllegalAccessException e)
  {
     throw(new RuntimeException(e));
  }
  catch (IllegalArgumentException e)
  {
     throw(new RuntimeException(e));
  }
  catch (InvocationTargetException e)
  {
     throw(new RuntimeException(e));
  }
  catch (NoSuchMethodException e)
  {
     throw(new RuntimeException(e));
  }
  catch (SecurityException e)
  {
     throw(new RuntimeException(e));
  }
The point that Python's Exception model gets right, that so many other languages get wrong, is that by default, you want to pass errors to the caller.

You don't want to pass errors through the same pipeline that results flow through, since then every joint in the plumbing needs error checking. The very name "exception" is chosen to denote an "extraordinary control flow" specifically for error handling.

Re:Monads are a Class of Hard Drugs (2008) by tel on Saturday December 29, 2012 @04:43PM (#4983589)
No, of course not. Error handling is just one example. The monad interface makes handling that problem you mention---wiring errors through the value pipeline---much easier.

And the monad part is only just the "just" and "bind" functions. Everything else is about trying to make Python more explicit with its errors: "explicit it better than implicit", right?

Monads allow for easier composition of "values in context". That context might be error, or nondeterminism, or continuation, or state, or logging, or parsing, or parallelism, or simulation, or probability, or graphs, or prolog-like logic, or streaming, or many combinations over the previous.

And in every case, you have the same interface.

Re:Monads are a Class of Hard Drugs (2008) by fusiongyro on Sunday December 30, 2012 @03:57AM (#4985171)
If your only standard for utility is applicability to Python there's no point discussing monads or anything else that comes up in conversations about languages besides Python. Maybe the reason you haven't seen a cogent explanation of monads using only Python is the same as the reason you haven't seen a cogent explanation of how to use the clutch in an automatic car, or how to desalinate fresh water, or how to make lemonade with nothing but a jug and ice water. Maybe the problem isn't the explanation or the one producing it.
Re:Monads are a Class of Hard Drugs (2008) by jayferd on Saturday December 29, 2012 @01:43AM (#4981359)
What I wish someone had explained to me earlier was that Monads are an abstraction level up from most things you ever program with. Learn specific ones, and the general picture will start to make more sense. Avoid the IO monad until you understand, in order:

+ the List monad (probably the easiest for python people, since it's very similar to list comprehensions)

+ the Maybe monad (kind of a degenerate case)

+ the State monad

And it's much easier if you do it in Haskell, trust me. "Learn You A Haskell For Great Good" is a nice easy intro to the basics.

Re:Monads are a Class of Hard Drugs (2008) by pi18n on Saturday December 29, 2012 @05:41AM (#4981725)
Most information on Monads start out with IO, which I agree isn't helpful. I found one that explained it with Maybe and Either, and that was where I began to understand it.

I think Maybe is the easiest to grasp for people who work in procedural languages, since there's a very common idiom of returning a nil object if some operation doesn't succeed. You can look at Maybe and see immediately; "oh this is like a type-safe version of that". Then from there, Either makes since almost instantly, and you can sneak in List and some others before they realize they shouldn't be able to understand it :)

Since I like your order of Monadic instruction, do you perhaps have a preferred starting point for learning Arrows? I can see that it's a way to compose sequences of computation, but I don't get what problems they solve that are difficult to solve with other methods.

Re:Monads are a Class of Hard Drugs (2008) by jayferd on Saturday December 29, 2012 @12:49PM (#4982753)
This is the part where I admit that I don't quite get Arrows. Anyone else?
Re:Monads are a Class of Hard Drugs (2008) by epidemian on Saturday December 29, 2012 @09:30PM (#4984323)
Anyone else who doesn't get arrows? Count me in! I wish i understood this comic's joke: http://ro-che.info/ccc/12.html :(
Re:Monads are a Class of Hard Drugs (2008) by fusiongyro on Sunday December 30, 2012 @04:08AM (#4985193)
There was a blog post that came out around that time in which it was claimed that arrows are easy to understand because they're just functions. There was a lot of backlash, and this comic is trying to point out that the reasoning behind it is backwards. Every function is an arrow, but not every arrow is a function. I believe the blog post in question is here:

http://blog.romanandreg.com/post/2755301358/on-how-haskells-...

That's about the limit of my comprehension of arrows. When you see a type (Arrow arr) => arr a b, you should read it as "an arrow from a to b", just like a function (a -> b) can be read "a function from a to b". But an arrow can encompass a monad, a function a -> m b is also an arrow, a Kleisli arrow with type Kleisli m a b. So this means you can use the arrow composition functions with normal functions as well as monadic functions, as well as arrows of other types which need not be functions at all.

Beyond that, all I know is that arrows provide composition functions that allow you to build a digraph of computations where inputs come in, get split off, passed through other arrows and recombined in interesting ways, and there is an arrow syntax which is so convoluted I've never been able to understand the relationship between it and the fundamental arrow combinators. I have never run into a situation where knowing anything about arrows was useful, other than because (&&&) is a neat combinator if you're want tuples. Practice doesn't seem to have caught up to the theory here, and things like applicative functors, which are simpler than monads in some sense, are a lot more useful than arrows, which are more complex. This blog post illustrates a case where arrows were handy, but he winds up using applicative functors in the end also:

http://jaspervdj.be/posts/2012-01-14-monads-arrows-build-sys...

It's a curious thing about functional programming that often what's simpler in theory (monoids, functors) turn out to be more useful in complex situations in the real world than what's more complex in theory.

One last note, I wrote a blog post about this in 2007 with FizzBuzz. Forgive me if it's horrible; I haven't given it much thought since, but here it is:

http://old.storytotell.org/blog/2007/04/08/haskell-arrows.ht...

Re:Monads are a Class of Hard Drugs (2008) by pi18n on Sunday December 30, 2012 @07:04AM (#4985475)
Thanks for the link, I hadn't seen the second one. I felt like I almost understood what they are for, and the constructors for the arrow datatype aren't very cryptic, but then it hits the code examples, which has stuff like

    y <- readFileA "unicorns.txt" -< ()
and it's unlike anything I'm used to. I'd love to find a post that described the utility of arrows as well as his did and then maybe having a middle step where they are used without the arcane symbols, and then finally with the arcane symbols.

Your post was very clear (although your code blocks have a tiny font on my computer for some reason). The &&& and >>> combinators make sense, especially with your diagram... but in the fizzbuzz case I don't get why it's preferable to have the arrow over something like

    fizzbuzz x = combine (three x) (five x)
Although maybe it's just a case of making it simple to compose functions when there are tens of inputs or something, so the simple case wouldn't really demonstrate anything like that.
Re:Monads are a Class of Hard Drugs (2008) by fusiongyro on Sunday December 30, 2012 @09:45AM (#4985791)
I think we're on the same page not seeing a real use for them. :)
Re:Monads are a Class of Hard Drugs (2008) by pi18n on Tuesday January 01, 2013 @09:31PM (#4995011)
It's just so weird, because monads are obviously useful :)
Re:Monads are a Class of Hard Drugs (2008) by fusiongyro on Tuesday January 01, 2013 @11:02PM (#4995260)
If they're meant to be, somebody will cook up a few great instances and show why monads and other abstractions were insufficient. Until that happens I don't think they need to be in people's standard repertoire. Applicative functors seem to be much more handy.

I'd like to know more about comonads too, but the only interesting article I've seen about them I can recall was this one about using comonads to build a zipper for managing location in a site menu.

http://fmapfixreturn.wordpress.com/2008/07/09/comonads-in-ev...

Re:Monads are a Class of Hard Drugs (2008) by pi18n on Friday January 04, 2013 @10:21AM (#5007960)
I read that and I am none the wiser. I guess I need to find a tutorial on comonads too :/
Re:Monads are a Class of Hard Drugs (2008) by codewright on Saturday December 29, 2012 @03:56AM (#4981551)
>+ the Maybe monad (kind of a degenerate case)

Hey. Hey. heyyyyy.

I've used bind + maybe monad 'ish stuff in Python to clean up code previously littered with redundant if checks.

Re:Monads are a Class of Hard Drugs (2008) by tikhonj on Saturday December 29, 2012 @04:36AM (#4981615)
He probably means its a degenerate case of the list monad. Which is a useful way to look at it: a Maybe value is like a list of length at most 1.

You can also look at it as a degenerate case of Either. (That is, Maybe a ~ Either () a.)

This is "degenerate" in the mathematical sense, much the same way you can think of a point as a degenerate circle.

Re:Monads are a Class of Hard Drugs (2008) by jerf on Saturday December 29, 2012 @12:51PM (#4982763)
Yes, the point is that it's just about the simplest Monad implementation that actually does something useful.

I'd observe in my code that I rarely have an enormous do block using Maybe; instead what I see a lot is use of the monadic interface to tie things together that would normally be a lot of if blocks, all on one line into one expression. You start missing that in other languages real fast. Also the Applicative instance is often quite useful, allowing you to easily tie together several things that return Maybes into a call of some sort that does the right thing if one of them is a Nothing, almost "free" from a syntax point of view.

Re:Monads are a Class of Hard Drugs (2008) by jayferd on Saturday December 29, 2012 @12:47PM (#4982745)
Haha, yeah, I meant "degenerate" in the mathematical sense. Totally useful, but in explaining it you're going to get a lot of "well, it's trivial in this case because...", which isn't that helpful if you don't yet understand the underlying concepts :)
Re:Monads are a Class of Hard Drugs (2008) by Chattered on Saturday December 29, 2012 @05:20AM (#4981692)
> I'm still looking for someone who can present a cogent explanation of monads using only Python.

Personally, I don't do monads in any other language than Haskell. I find monads painful even in Ocaml. I'm pretty stupid as a programmer. And --- especially when doing things such as monad transformers --- I need lots of type checking and lots of type inference to help me over my mistakes (which are pervasive).

The cool thing is that when I've written these things called "monad transformer stacks" (see Real World Haskell for what I guess is the best explanation for those), I find that when my code gets through the type checker, it really does just work. It works, even when I've lost any sense of what my code is actually doing underneath. Without the type system, I couldn't have any confidence of this, and the runtime bugs you can potentially get with monads are so subtle that I would be terrified to use them in a language like Python (note again, I'm stupid, and YMMV).

The other reason why I wouldn't do monads without types is because, to me, monads are the type. The semantics of a monad is pretty much just that type (plus some crucial axioms). Types in Haskell are much more expressive than in most languages. The way "operator overloading" works in Haskell means that by specifying types, you are often generating runtime code (think C++ templates, but not for too long). That's why you see more type annotations in Haskell compared to (say) Ocaml. Though if you want to do the same thing in Ocaml, you'll be knee deep in the even more verbose world of these things called functors.

Re:Monads are a Class of Hard Drugs (2008) by gizmo686 on Saturday December 29, 2012 @06:22AM (#4981798)
Consider the pattern of using None (or null) as an error. You want to compute: var x x=f(x) x=g(x) x=h(x)

however, if either f,g, or h return None, then the final value of x is None. The standard way of doing this is: def f(x): if x==None: return None ...

The monadic solution is the Maybe monad. Maybe has two constructors, Just(x) and Nothing. The above code would be equivilent to var x x=Just(x) x=x.apply(f) x=x.apply(g) x=x.apply(h)

The definition of Maybe is pretty simple. The Just object takes a value of a given type, and will run that value through a given function: Class Just(): def __init__(val): this.val = val

def apply(f); return f(x)

The Nothing object is even simpler: Class Nothing(): def apply(f): return Nothing()

Using this, we can see that in our original example, once one function returns Nothing, we skip all the other functions and end up with Nothing. When a function does return something, it needs to wrap it in Just, so we have:

def f(x): ... return Just(x)

In general, to be a Monad you need to satisfy the type class Monad(): def apply(f)

Of course, using Monads like this can end up being somewhat messy, so there is much syntactic sugar around them.

Re:Monads are a Class of Hard Drugs (2008) by chongli on Saturday December 29, 2012 @03:28AM (#4981522)
>Anyone who understands monads is on a class of hard drugs.

It's so frustrating to hear you say that. Monads are such a simple concept but they're difficult to explain because they're far more general and abstract than most programming topics. The best explanation I have is that monads are a way of storing some kind of computation with a value. When the user wants access to that value, he uses a monadic operator which forces the monad's computation to be run before returning the value.

Re:Monads are a Class of Hard Drugs (2008) by brazzy on Saturday December 29, 2012 @04:52AM (#4981638)
So basically monads are like a Java object that has a private field and a getter method that has some extra logic. Hey, that's really simple, you're right!

duck & runl

Re:Monads are a Class of Hard Drugs (2008) by chongli on Saturday December 29, 2012 @07:19AM (#4981891)
No, objects in Java are more complicated than monads. They include the concepts of identity, mutable state and inheritance; all of which monads lack.
Re:Monads are a Class of Hard Drugs (2008) by tel on Saturday December 29, 2012 @11:53AM (#4982585)
Almost, it's more like an "applyer" method with some extra logic. You're never allowed to get "raw" values out, you can only pass functions inward to modify the value.

If you want to get the raw values out (and you almost always do) then you have to add some extra logic to the object above its "monadic" nature.

And yeah, it is really, really simple!

Re:Monads are a Class of Hard Drugs (2008) by csense on Saturday December 29, 2012 @03:55PM (#4983426)
I'm trying to parse this explanation. What's the difference between a monad and overriding __getattr__ in Python?
Re:Monads are a Class of Hard Drugs (2008) by chongli on Sunday December 30, 2012 @04:31AM (#4985241)
Apples and oranges. A monad is just an idea; that of storing computations as values. This is useful because these computations can be composed together in a manner not unlike that of a pipeline in shell scripting. Different monads define different ways of composing these computations leading to vastly different semantics.

Here's a nice video that explains monads fairly simply:

http://www.youtube.com/watch?v=ZhuHCtR3xq8

Re:Monads are a Class of Hard Drugs (2008) by codewright on Saturday December 29, 2012 @03:55AM (#4981550)
Python it is. I've explained monads using Python many times.

http://www.dustingetz.com/2012/04/07/dustins-awesome-monad-t...

http://www.dustingetz.com/2012/10/02/reader-writer-state-mon...

(I'm not dustingetz though, he's named dustingetz on HN.)