Haskell doesn't have mutable state; it still has closures.
Mutable or immutable is somewhat orthogonal; I admit, coming from Erlang to Javascript I was -shocked- and very facepalmy when I learned that closures close over variables that can still vary. That is, I couldn't just pass back a function and expect it to behave the same way depending when I called it. I had to add an additional scope in place and bind the values anew (i.e., a new function, passing the values being closed over into it as params). Immutable is much easier to reason about, much easier to write with, and despite being more limited in what you can do with them, I'd argue just as useful. Everything it prevents you from doing there is some other, safe way of achieving; the same as comparing mutable vs immutable in other contexts.