News for hackers. Stuff that matters.
Re:Is Haskell the Cure? by gregwebs on Monday October 03, 2011 @09:14AM (#3066127)
I believe you are attributing a library issue to a language. Before today (and by today I literally mean a month ago when Yesod released a cross-platform development server that automatically re-compiles your web application) there wasn't a productive set of libraries and tools to build a web application with in Haskell. 3 years ago when you started, and even until 1-2 years ago the library situation was absolutely horrible. Web frameworks with very little to offer, mediocre templating languages, not even an attempt at a database ORM. Tutorials would have you write a bunch of code to achieve a detail taken for granted in libraries used in web frameworks of other languages.

Please take a look at doing real-world, productive web development with Yesod. http://www.yesodweb.com

You are still going to take a productivity hit in Haskell due to lack of libraries in comparison to Ruby, Python, etc. So the practical reason for using Haskell today is to take advantage of the amazing performance, take advantage of Haskell non-web libraries in the backend, or for a high assurance project where its type system can rule out most common web development bugs.

oh, and Yesod is even faster than the mentioned Snap framework which is already much faster than Node (and unlike Haskell, Node does not scale to multi-core). Although Yesod isn't going to automatically cache the fibonacci sequence for this artificial benchmark because in the real world I have never once been tasked with writing code like that for a web application.

Re:Is Haskell the Cure? by danieldk on Monday October 03, 2011 @10:00AM (#3066279)
I believe you are attributing a library issue to a language.

Reasoning about laziness? Polymorphism that can only be implemented using existential types plus Typable? Even purity is a double-edged sword (some algorithms are inherently mutable)[1]. Some of Haskell's problems in real-life projects can definitely be attributed to the language itself.

So the practical reason for using Haskell today is to take advantage of the amazing performance,

My experience with everything from simple checksum functions to parameter estimators (ML) is that Haskell is generally at least 2-10x slower than C (even when introducing strictness where necessary, unboxing constructors, etc.). So, in practice you'll often end up doing heavy lifting in C anyway (whether it is a database server or a classifier that works in the background), and in the end it doesn't matter so much whether you use Haskell or a dynamic language (performance-wise) if a significant amount of time is required processing requests.

where its type system can rule out most common web development bugs

Right, this is where Haskell currently has an edge, because it does not only make it easy to make DSLs (as e.g. Ruby), but typechecks everything as well.

oh, and Yesod is even faster than the mentioned Snap framework which is already much faster than Node

Yes, but the benchmarks you implicitly point to (the pong benchmark) is very synthetic and says fairly little about real-life use. Until we see Snap and Yesod more in production, the jury is still out.

[1] Sure, you can do quicksort in the ST monad, but it will require a lot of unnecessary copying.

Re:Is Haskell the Cure? by gregwebs on Monday October 03, 2011 @10:38AM (#3066441)
Yes, reasoning about laziness and difficulties using types are library issues. Particularly if a library is forcing you to learn about existential types. In Yesod we are very conscientious about what types (even just polymorphism) that is exposed to the user, because they can make error messages, etc difficult.

I don't think the Pong benchamark http://www.yesodweb.com/blog/preliminary-warp-cross-language... is that synthetic - I think it demonstrates concurrency capabilities fairly well. We just have to keep in mind which web applications benefit from high concurrency.

As for raw performance of a single request, I agree that the average web application won't see a great difference for the 80% case. However, for most Ruby web applications that I have worked on I have had to spend time re-writing slow parts of the application because Ruby was truly the bottleneck, and I would have been much better off using almost any compiled language with types.

Ruby applications I have worked on always have more complicated deployments, worse response times, and huge memory usage due to the lack of async IO. Async IO is possible in Ruby & Python, but it still sucks because it is extra work and you have to always be on guard against blocking IO. So I hope we can at least agree that async IO is a big win, and that Haskell & Erlang are the best at async IO because it is built into the runtime and no callbacks are required. And likewise deployment to multi-core is no extra effort in Haskell/Erlang, whereas in Node, Ruby, or Python you will need to load balance across multiple processes that are using more RAM.

Re:Is Haskell the Cure? by danieldk on Monday October 03, 2011 @11:23AM (#3066607)
Yes, reasoning about laziness and difficulties using types are library issues.

I disagree, if the language were strict by default, this was not an issue. It is a language problem that is forced on libraries.

However, for most Ruby web applications that I have worked on I have had to spend time re-writing slow parts of the application because Ruby was truly the bottleneck,

My point was that Haskell is often a lot slower than C of C++, so people will rewrite CPU-intensive code anyway. Look at many of the popular Haskell modules where heavy-lifting is done (from compression to encryption), most of them are C bindings. That code will be nearly equally fast in Haskell as in, say Python.

BTW. I am not arguing that Haskell not faster than Python, Ruby, Clojure, etc. But for computationally intensive work C/C++ are still the benchmark, and that is what people will use in optimized code. Whether it is Haskell or Python.

Particularly if a library is forcing you to learn about existential types.

But why is it? Because the language does not support the kind of polymorphism that is commonly used, in an intuitive fashion. People need containers with mixed types that adhere to an interface in some applications. And a commonly-used method to realize this in Haskell is by using existential types.

we can at least agree that async IO is a big win

Yes.

And likewise deployment to multi-core is no extra effort in Haskell/Erlang, whereas in Node, Ruby, or Python you will need to load balance across multiple processes that are using more RAM.

Since most modern Unix implementations do COW for memory pages in the child of process of a fork, this is not so much of an issue as people make it out to be. The fact that you mention Erlang is curious, since spawn in Erlang forks a process, right? Forking is more expensive than threading, but again, in most applications negligible compared to handling of the request.

Re:Is Haskell the Cure? by gregwebs on Monday October 03, 2011 @12:20PM (#3066863)
The biggest reason why there are Haskell packages wrapping C libraries is not for performance, but to reuse good C libraries, and because Haskell has an excellent interface for C libraries. Many people prefer to write Haskell for computationally intensive tasks than C/C++. Depending on the problem it is possible to get within 2x the raw speed of C and you much nicer code to maintain and much easier concurrency/parallelism opportunities.

I have not found it to be the case that existential types are commonly needed (and need to be forced on the user). Maybe you are in a different problem domain. I find Haskell's regular polymorphism to work very well for 95+% of my use cases.

Fork is not negligible to handling a request, but pre-forking theoretically could be. In practice, COW fork does not automatically solve multi-core. The Ruby garbage collector is not COW friendly and thus there is little memory savings from COW (unless you use the REE interpreter which has a slower COW friendly garbage collector but saves on memory and GC time). I haven't looked at this for other languages but I assume this is still a limiting issue. Also, you are still stuck doing load-balancing between your processes, which will limit or complicate your deployment. I don't know much about Erlang other than async IO is built into the language, which is why I mention it in the same breath as Haskell.

Re:Is Haskell the Cure? by pjscott on Monday October 03, 2011 @05:43PM (#3068235)
In case anybody is wondering: both Erlang and Haskell have very lightweight user-space threads built in, which are mapped onto a small pool of OS threads to take advantage of however many cores you have. It's very slick and fast, and probably the Right Thing.
Re:Is Haskell the Cure? by gtani on Tuesday October 04, 2011 @12:07PM (#3071535)
Yes, there's also akka which is being incorporated as scala standard lib, and F# MailboxProcessor. The thing is that erlang/OTP and its behaviors have many people pounding on heavily loaded apps in production and improving its toolchain, whereas GHC and akka have recently (last couple years I think) been working ot get the stack working: dispatchers and load balancing (like erland reds), and bring GC up to snuff
Re:Is Haskell the Cure? by pjscott on Tuesday October 04, 2011 @08:05PM (#3073572)
Akka looks pretty sweet, but it looks like you still have to worry about blocking code in external libraries. In Haskell (and Erlang, IIRC), blocking code is deferred to a background thread automatically so you don't have to be consciously on-guard for it. You also get proper pre-emptive multithreading, while Akka looks like a hybrid of an event loop and a thread pool.

Is this a substantial headache with Akka, in practice?

Re:Is Haskell the Cure? by gtani on Wednesday October 05, 2011 @10:51AM (#3075868)
(late reply)

it's pretty hard to google akka deployments but:

http://www.quora.com/What-companies-are-using-Akka-commercia...

http://groups.google.com/group/akka-user/browse_thread/threa...

and in terms of memory overheads and how many erlang process-type things you can spin up:

http://akka.io/docs/akka/1.1/scala/tutorial-chat-server.html

Re:Is Haskell the Cure? by thesz on Monday October 03, 2011 @12:03PM (#3066784)
Why use quicksort over arrays when you can do mergesort over lists and get 1) stable behavior and 2) solution to maximum and k-max problems due to laziness? Do you really need arrays?

And quicksort for arrays in ST monad wouldn't copy anything unnecessary.

Actually, I've seen many claims that some algorithms are inherently mutable. So far none stand close scrutiny.

Matrix operations? You better copy intermediate results, that way you'll be safer and faster (parallel algorithms). Good compilers do that behind the curtain (array privatization).

Sorting? Use maps or lists, that way you won't forget something important.

Graph operations? Immutable (inductive) graphs are slower by a constant multiplier and sometimes are faster than their mutable counterparts (tree-based maps are faster for changes than arrays).

The last one is even more amusing when applied to compiler optimizations (i.e., to non-trivial graph algorithms): http://lambda-the-ultimate.org/node/2443 Pure version is less buggy, faster (!) and allows more optimizations.

Re:Is Haskell the Cure? by danieldk on Monday October 03, 2011 @12:20PM (#3066870)
Why use quicksort over arrays when you can do mergesort over lists

Sure, you can do merge sort. Except that the list split step in Haskell is O(n) in time, while it is constant when using arrays. As well as merging lists, since you have to 'reattach' the second list as the tail of the first list.

And quicksort for arrays in ST monad wouldn't copy anything unnecessary.

You have to copy the data from whatever representation you had to something that lives in a memory block in the ST monad.

Actually, I've seen many claims that some algorithms are inherently mutable. So far none stand close scrutiny.

You have probably never read Okasaki...

The rest of your argument proposes that slow is better because of persistence. First, persistence is often not required, second persistence can also be implemented in a mutable language.

Re:Is Haskell the Cure? by thesz on Tuesday October 04, 2011 @06:38AM (#3070258)
>Except that the list split step in Haskell is O(n) in time, while it is constant when using arrays.

Oh, no. You shouldn't split list by calculating length.

Try this instead:

   even (x:_:xs) = x : even xs
   even xs = xs
   odd = even . drop 1

   splitList xs = (even xs, odd xs)
Voila! Completely lazy, O(1).

So for merge. See here: http://lambda-the-ultimate.org/node/608?from=0&comments_... The solution contains proper merge algorithm.

And yes, I never read Okasaki in full. But, I use Haskell semi-professionally from 1999 and professionally from 2006.

Re:Is Haskell the Cure? by eru on Monday October 03, 2011 @01:15PM (#3067163)
I agree with you in most cases.

> Sure, you can do merge sort. Except that the list split step in Haskell is O(n) in time, while it is constant when using arrays. As well as merging lists, since you have to 'reattach' the second list as the tail of the first list.

It's no problem writing a merge-sort in Haskell that uses O(n log n) time. So who cares what the asymptotics of the individual elements of the algorithm are? (You may care about the actual speed of the whole thing and its parts, though.)

Re:Is Haskell the Cure? by eru on Monday October 03, 2011 @01:00PM (#3067101)
If you want a canonical example of an algorithm that's imperative and harder to do in a functional setting, cite union-find (See "A Persistent Union-Find Data Structure"). Searching is optimal in a functional setting, too. Just not the classic quicksort.
Re:Is Haskell the Cure? by eru on Monday October 03, 2011 @01:00PM (#3067102)
If you want a canonical example of an algorithm that's imperative and harder to do in a functional setting, cite union-find. Searching is optimal in a functional setting, too. Just not the classic quicksort.
Re:Is Haskell the Cure? by jberryman on Monday October 03, 2011 @01:13PM (#3067151)
> Polymorphism that can only be implemented using existential types plus Typable?

I'm curious where you came across this. In an external library you were using, or in the process of trying to implement some kind of dynamic typing in your own code?

Re:Is Haskell the Cure? by danieldk on Monday October 03, 2011 @01:44PM (#3067243)
Both :). To give one specific example: I was working on a transformation-based learner for learning tree transformations. Say that a rule consists of an action and a list of condition that makes the action fire if they are true for a particular tree node. Obviously, you'll want to be able to add new conditions, so you make a type class for conditions:

    class Cond a l where
      applies :: a -> TreePos Full l -> Bool
Now, say that a rule contains a list of conditions which belong to the type class Cond (Cond a l => [a]). You can see the problem coming. Say I provide a condition of the type MyCondition, then the list will be of type [MyCondition]. However, in practice it would be inflexible to restrict a list of rules to one type. You want to be able to add new conditions outside the module or package binary. So, instead I used existential typing for conditions:

  data Condition l =
    forall c . (Cond c l, Eq c, Show c, Typeable c) => Condition c