I think there may be some confusion about what a space leak in Haskell is.

When you apply a function f to a, that is not actually evaluated. Rather, a "thunk" is created that will evaluate `f a` only when that value is actually needed.

If, in your program, you never need anything, or don't "force" your function calls and data structures ("pretend" to need something) in intermediary stages, then the thunks may take up a non-trivial amount of memory. This is not a memory leak in the traditional sense, just temporarily increased memory usage.

It is very easy to (pre-emptively) handle most space leaks in Haskell, but you do need to know how they arise.

There are two very simply rules you can follow that take care of the vast majority of space leaks:

1. Make data fields strict unless you actually want them to be lazy, i.e. instead of:

    data Foo = Foo
        { bar :: String
        , baz :: Int
        }

write

    data Foo = Foo
        { bar :: !String
        , baz :: !Int
        }

2. When you write recursive functions that depend on values which are not forced (e.g. pattern matched against) in each function call, use either `seq`/$! or bangpatterns to make sure the value is evaluated (to HNF) rather than building up excessive thunks. For example, instead of:

    acceptLoop :: Socket -> Int -> IO ()
    acceptLoop sock connNum = do
        econn <- accept sock
        _     <- case econn of
            Left err   -> printf "Error accepting connection %d: %s" connNum err
            Right conn -> forkIO $ runConn conn
        acceptLoop sock (connNum+1)

write either

    acceptLoop :: Socket -> Int -> IO ()
    acceptLoop sock !connNum = do
        econn <- accept sock
        _     <- case econn of
            Left err   -> printf "Error accepting connection %d: %s" connNum err
            Right conn -> forkIO $ runConn conn
        acceptLoop sock (connNum+1)

or

    acceptLoop :: Socket -> Int -> IO ()
    acceptLoop sock connNum = do
        econn <- accept sock
        _     <- case econn of
            Left err   -> printf "Error accepting connection %d: %s" connNum err
            Right conn -> forkIO $ runConn conn
        acceptLoop sock $! connNum+1

to make sure that connNum is always just a single value rather than a series of unevaluated thunks. That way you won't get a space leak if you rarely have problems accepting new connections.

Thanks, helpful explanation.

Now, that begs the question, why lazy by default and not opt-in lazy?

From the outside looking in it seems that deep expertise is required in order to launch a Haskell production app with any degree of confidence (i.e. to quickly dig yourself out of runtime issues like space leaks where the means to avoid them may be known, but the means to resolve them when they occur, non-trivial).

> Now, that begs the question, why lazy by default and not opt-in lazy?

Very good question. Actually, I think most haskellers agree that laziness complicates things more often than not, and if we could start over we wouldn't make Haskell lazy by default. (Although that's not to say there won't be even simpler ways to "strictify" things in the future. Also, many libraries already provide functions that are strict in their arguments by default.)

However, there is also agreement that Haskell's laziness is the reason the language got purity right: there was simply no other way, since laziness meant evaluation order was unclear.

> From the outside looking in it seems that deep expertise is required in order to launch a Haskell production app with any degree of confidence (i.e. to quickly dig yourself out of runtime issues like space leaks where the means to avoid them may be known, but the means to resolve them when they occur, non-trivial).

As someone who writes Haskell for a living, I really just follow a few rules like this without thinking too much about laziness, and I tend to not have any problems. I have had maybe one nasty space leak in the past five years.

Yes, sometimes they do come up, but it takes ~5-10 minutes to pinpoint the problem spot with the heap profiler. It is not nearly as messy as using Valgrind to find actual memory leaks.

> I think most haskellers agree that laziness complicates things more often than not, and if we could start over we wouldn't make Haskell lazy by default.

Following Haskell's evolution somewhat from the outside, this is surprising. (And also somewhat disappointing, as laziness always seemed an important part of Haskell's elegance.) Is laziness now considered something of a failed experiment?

> laziness always seemed an important part of Haskell's elegance

It's part of it, but to a lesser extent than you would expect. The much more important part of Haskell is its no-corners-cut separation of effects, which happened chiefly because, without it, laziness meant that there was no way to know when fireTheMissiles() would actually happen.

> Is laziness now considered something of a failed experiment?

To some extent, yes. No one is saying laziness doesn't make the implementation of some algorithms and data structures extremely elegant, just that, most of the time, you don't actually gain much by leaving your function calls and data types lazy.

I enjoy being able to make infinite/self-referencing data structures, or leaving fields lazy and "performing" all of the function calls in the initialization of the "struct", but have only the functions producing results that will actually be needed matter performance-wise. However, if you don't use any strictness annotation, that benefit doesn't outweigh the problems that can be caused by space leaks.

If you do use strictness annotation, I don't think it matters too much whether the language is lazy or strict, you just have to write strictness annotation instead of laziness annotation.

I think this might be overstated. It is certainly an opinion with some mindshare; I would hesitate to guess whether it's in the majority or minority, and it's definitely not universal.

FWIW, Simon Peyton Jones agrees laziness isn't the boon it's made out to be.

While not universal, I feel confident stating that the opinion is shared by the majority of long-time Haskell programmers.

Because the downsides of strictness are great as well. Laziness lets you "decomplect" production from consumption, with a lot of safety, most of the time. I think few who've spent most of their time working in strict-by-default languages recognizes the pain of it. Those who do write opt-in lazy language properties like generators.

So the question becomes whether strict-by-default or lazy-by-default is better given that both kinds of evaluation have their place. I personally have come to believe that lazy-by-default is nicer since there are fewer places where I really demand strictness... but it also basically forces your language to be pure.

Without laziness the Haskell community would likely not have maintained functional purity for very long. Looking at the history of all other programming languages it's difficult to find one that hasn't succumbed to the temptation of impurity at some point or another. Without this enforced purity, there would not have been the same pressure to develop technologies such as Applicatives and Monads.

The other advantage of laziness is that it is a big aid to composition, modularity and concision. It allows you to perform common sub-expression elimination which can be a major boon for code readability. See this article for some examples:

http://augustss.blogspot.ca/2011/05/more-points-for-lazy-eva...

    data Foo = Foo
        { bar :: !String
        , baz :: !Int
        }

This is actually a great illustration of why "making data fields strict" is a lot more tricky than it sometimes looks. All that

> bar :: !String

is going to get you is the first cons cell in weak head normal form.

If my string is "Hello", then in unevaluated form I have

<THUNK>

Using the strict data field gets you weak head normal form, which is

<THUNK>:<THUNK>

Probably not what you had in mind. As far as Haskell is concerned those two thunks might be

undefined:undefined

Using a strict Text will get you where you want to go, because when you reduce that to weak head normal form you will get a fully formed Text.

Watch out with lists, which is all that a String is.

Right, to fully evaluate something you'd use Deepseq. With data structures like Data.HashMap.Strict, Data.Text, etc., the HNF gained from strict data fields tends to be sufficient to avoid most issues caused by space leaks -- but even for something like !String, i.e. ![Char], you'd have to try pretty hard to blow up the stack. (Evaluating the first cell is also more useful than it might seem at first glance.)

I find myself using deepseq the most when I want to be sure a data structure (and any exceptions any pending operations might throw) has been fully evaluated before passing it off to another thread, not to prevent space leaks.

I am surprised that your comment was downvoted. I upvoted you. I feel that you are 100% correct when you write:

"I'd take a NullPointerException over a memory (edit: space) leak any day of the week; the former is instantly resolved, the latter, god knows."

If someone really feels that your remark should be downvoted, I hope they post an explanation about why.

I think it was because - in the words of The Dude, "Yeah, well, you know, that's just, like, your opinion, man."

> I'd take a NullPointerException over a memory (edit: space) leak any day of the week.

That's your preference and I can understand that preference from your point of view because in order to debug a space leak in a Haskell program you would first have to learn Haskell.

I'm not the one who downvoted, but I also didn't think it contributed much =(

Sorry, it's not an opinion, it's a fact ;-)

Why? NPEs are easily solved, even for beginners. Stack trace says blah blah blah occurred at line X in class Y. Easy fix, totally low hanging fruit for beginners and experts alike.

The space leak issue, OTH, may be easily avoided (if one is well versed in Haskell best practices), but resolving them when they occur is something else entirely. Seriously, you have to break out a memory profiler in order to find out where the issue _may be_, not exactly where it is on line X of class Y.

So, yes, I never get them (in Scala), but I'll stand by taking an NPE over a space leak any day of the week.

thirsteh summed up my feelings in his response to your comment but I'd also like to add that a preference is an opinion and not a fact. You may prefer apples over oranges. It is not a fact that apples are better than oranges.

For instance, what if your null pointer in language x causes a silent fail? Now your apples are beginning to look like oranges.

This is really quite silly since the choice between NullPointerExceptions and space leaks is a false one. You can have both in either language.

The choice between NullPointerExceptions or occasional space leaks isn't really apples-to-apples. You could similarly ask if you'd rather have solely impure functions and no isolation of effects, or occasional space leaks. Clearly (probably?) the answer would be the latter.

Yeah, an apt comparison would be race conditions/spooky action at a distance vs. space/resource leaks. Pick your poison.

If you're going to have errors anyway (and you are), then it's better to have errors that fail fast and early and in a clearly identifiable way. NullPointerExceptions are easy.