It sounds like you just haven't used Haskell enough to have the full extent of the reusability become apparent. I agree that on the surface it seems like you're right. But that particular example of reuse is just the tip of the iceberg. It's actually a very pervasive pattern that has tremendous positive impacts on your code at every level from small 3-line loops to large overarching abstractions.
This isn't just me making baseless assertions. It has been backed up by real world experience. There was an ICFP presentation by a company that rewrote their entire application in Haskell. With Haskell their reuse was an order of magnitude higher as measured by the number of external library dependencies! The rewrite also decreased the size of their code base by something like 80%. Oh, and they saw a massive reduction in bugs and increase in performance.
> But that premise has not been shown to be true. The lack of magically bug-free, FP OSs, drivers, control software, and large applications, shows that even the biggest supposed benefits of languages like Haskell, are yet to be demonstrated in the real world.
It's only been in the last 5 or so years that Haskell has really become commercially viable. This is too recent for the large apps you're talking about to have been written. But don't worry, we're working on it. The other day our newest hire made the comment that at our company each individual developer is responsible for roughly the equivalent of an entire team at all his previous jobs.
If we're talking real world -- as we should -- then this should be contrasted with the tradeoffs and alternatives. The most important kind of code reuse, IMO, actually happens at a larger scale, and in the past few years has worked remarkably well, since the advent of open-source. I'm talking about reusing projects like ZooKeeper, Hadoop, etc. Being able to use such a project might save you millions of LOC, regardless of the level of internal code reuse. The lack of Haskell libraries and bindings relative to other languages reduces this most important kind of reuse. In fact, using ZooKeeper also reduces a large number of bugs.
> But don't worry, we're working on it.
Excellent. I'd love to get a full picture, tradeoffs and all, of a big, important software project in Haskell (which isn't a compiler).
> The other day our newest hire made the comment that at our company each individual developer is responsible for roughly the equivalent of an entire team at all his previous jobs.
I can say the same for my current job, even though we're mostly doing Java (and some Clojure), and at our previous jobs we also used Java. This mostly has to do with the quality of developers, company structure and size, and less with the choice of programming language.
There's a hidden instance of begging the question here, which is that the reason why "large-scale" code reuse is "more important" in the real world is that OO has largely been a failure for mid-scale code reuse. You can build enormous frameworks in OO that serve a need (at some cost in constraining your options, but one that can be worth paying), and it can do certain fairly small-scale reuse like 'a generic red-black tree' or other data structures, but it has not done well in the middle. Therefore, since OO has been the dominant paradigm for decades now, we do not see any middle-scale reuse in the "real world". However, it is difficult to disentangle whether that is a fundamental characteristic of "reuse" or whether it's a fundamental characteristic of OO. (I can argue in favor of both, and I don't mean "both are true", I mean I could argue in favor of either one.)
I'm not a blind fan of any one type of software methodology but this assertion is completely wrong. Java and C++ world is replete with widely used code libraries that are used and reused in millions of diverse software projects. This fact is so obvious that I don't feel the need to list out the examples here, but I will if challenged.
This is an instance of begging the question too. Newer, less popular languages will always have fewer libraries and bindings relative to more popular languages. It might be an argument against using Haskell at the current point in time, but it has nothing to do with the merits of the language itself.
But the reality is that Haskell is actually doing quite well in terms of libraries and bindings. Pick anything you can think of...odds are probably pretty good that Haskell has bindings to it. As an example, let's look at the two projects you mentioned: ZooKeeper and Hadoop. Yep, there are Haskell bindings to both of them. For ZooKeeper there's hzk (http://hackage.haskell.org/package/hzk), and for Hadoop there's hadron (https://github.com/Soostone/hadron) and there's even a nice presentation about it (http://vimeo.com/90189610).
The merits of the language itself don't matter that much. A language isn't a piece of art to be admired. And as long as Haskell is used by such a tiny group of people, the merits of the language itself can't even be argued well, because it's unclear how they work in practice for different kinds of software and different kinds of developers. And the tradeoffs people have to make aren't clear, and if the tradeoffs aren't clear, the merits aren't clear either. They're at best hypotheses of potential merits, or merits in the eyes of the very particular group that's currently using the language.
I'm guessing Haskell is used by more people than you think. There are 6500 packages on hackage. The Haskell subreddit has more than 16,000 subscribers. And there are usually more than 1300 people in the #haskell IRC channel at any given time. That's more than #ruby and ##javascript and about the same as #python. Also, Standard Chartered bank has roughly 1.3 million lines of Haskell code running in production.
> And the tradeoffs people have to make aren't clear, and if the tradeoffs aren't clear, the merits aren't clear either. They're at best hypotheses of potential merits, or merits in the eyes of the very particular group that's currently using the language.
This statement applies equally to pretty much every language out there. We don't understand the tradeoffs between mainstream languages either. We have no idea how those choices are actually going to end up affecting our projects. To claim otherwise is intellectually dishonest. How do you know that Ruby or Perl would be better than Brainfuck? The tradeoffs and merits aren't clear. They're at best hypotheses of potential merits. So how do we actually make these decisions in practice? We look at the language features and think about how they can help us manage common situations in software development, then we go with our best guess. We can do that for Haskell just as well as we can do that for Java, C, Ruby, etc.
Maybe some people do, but when I pick a programming language, language features are among the last things on my mind. I look at the available tools (profilers etc.), available libraries (particularly in the domain that interests me), quality of documentation, the size of the community and its vibrancy. I also look at similar projects done in the language. Only after all these do I look at the language features, but they don't matter so much, because I know I'll be very productive in a language with a very large selection of libraries and a large user community. I'm saying this after 20 years of developing software in languages like BASIC, Pascal, x86 asm, C, C++, Matlab, Java, Clojure, Scheme and more. I can't think of a single case where language features mattered more than any one of the things I listed.
All this is, of course, for serious projects intended for production. For a toy project I might look at the language features first.
Nobody in this thread is talking about toy projects. Developers who use mainstream programming languages do not have a monopoly on real work. The consistent assumption made in these discussions that FP practitioners don't work on "serious projects intended for production" is misguided and insulting.
Haskell is a prime example of a rather adventurous research language. It's an experiment that's been run for about 15 years now. First the early steps, and now people are experimenting with real world applicability. I'm really interested in the results, so I'd really like more people to run the experiment, but I have no doubt that Haskell itself will never gain widespread adoption. If it does its job well, its best ideas will be copied by other, non-research languages.
Now, I'm certainly not saying that Haskellers don't work on serious projects. But it is a fact that Haskell is much more common in academia than it is in the industry. Again, this is intentional. Haskell is meant to venture into unchartered territory and explore. Some of the things it aims to do work well and might be adopted in the industry, and others don't and won't. If Haskell teaches us even a few things, then it is a great success. What I don't like is that some Haskellers deny the true purpose of the language, and instead of examining it as researchers should do, they preach it.
Haskell has been around for nearly 25 years, not 15, and the ML family has been around for nearly 40. Strong type systems are not a research curiosity, and have not been for decades. They have significant and well-understood benefits, in practice.
I wish your hypothesis that the best ideas from research would be copied by other languages were true, but the history does not bear this out. A notable recent example is Swift: though it does make progress compared to its reference class, it still does not use any research more recent than roughly the 1970s. Clearly, the "pragmatic" languages are not keeping up. In what sense is it "pragmatic" to constrain yourself to decades-old technology?
The Haskell language is not constrained by some God-given purpose to be a research language for all time. Haskellers aren't "deny[ing] the true purpose of the language", they're choosing to use a useful tool, because doing so is pragmatic.
Merits of programming languages can be argued well, because there is an entire theoretical discipline dedicated to the understanding of programming languages (PLT). Your lack of knowledge of this field does not mean that it does not exist.
"One is code reuse: yes FP code is definitely more reusable. The problem is that the kind of code it enables reuse of more than OO is very small, simple loops."
Combinator libraries sure are pretty useful, and they're more than small simple loops.
"Writing the same 4 line method -- that could have been abstracted with a monad -- 10 times in a 50-150K LOC project is never a big problem"
All our goes code through a bunch of monads (mostly Future + Writer). Very puzzled how you're using monads if they only appear 10 times in 50KLOC.
"Another is state management"
I think you're referring to STM here. That's been extensively explored in the Haskell community.
Useful -- yes. Useful enough to pay the price -- IMO, no. OO is decent at code reuse, and the most effective, most important code reuse, is that of big libraries with rich functionality, which is done rather well in many programming paradigms (i.e. pure, statically typed FP has little to no advantage here).
> Very puzzled how you're using monads if they only appear 10 times in 50KLOC.
I'm not. But OO code that refactoring with monads would have saved me does not amount to much.
> I think you're referring to STM here.
I was talking about more limited transactional (or "concurrently mutable") data structures like Clojure atoms and agents, or restricted STM like Clojure's refs and in-memory DBs (general purpose STM is pretty much dead). I know Haskell explores STM. I'm just saying you can enjoy the same benefits in Clojure or even Java.
Your basic argument seems to come down to Haskell (the only pure statically typed language with any widespread adoption) vs any other language. That's not an argument I'm particularly interested in.
As for OO vs FP in combinator libraries -- the same pattern is often called a fluent interface in OO. This gets to the larger point that FP is as much a programming style as a language that supports that style, and you can program in a FP style in any language. The question I'm interested in is "is FP style useful?" to which I give a resounding "yes!"
The only language that is purely FP is Haskell. It doesn't have anything that we could mistake for an object system, its functions all the way down!
Fluent interfaces should die a horrible death; it is a confusing style in OOP and quite easy to make lots of errors without the equational reasoning that supports combinators in FP.
I'm not saying it's a complete OO implementation, but instead that it's not so far as to be completely unconfusable, though it might be hard to, say, do Smalltalk-in-Haskell.
My point that OOP is essentially "thinking in terms of named objects" whereas most of what Haskell focuses on is "expressing math by composing anonymous values." I believe this starting from a design perspective is the best way to highlight the real differences between OOP and pure FP.
I don't find definitional wars interesting, and I'd, in the stream of everyday things, be very sympathetic to "OO and FP are different because they were built from different POVs". The only reason I am being a stickler is that I'm pretty interested in the idea that both FP and OO have a representation in type theory/category theory/logic and thus can be put into common representation.
Haskell definitely isn't the best OO language, but if you squint hard enough you can see OO in it much like you can see FP in C's function pointers: painfully. Seeing FP in C helps you examine the implementation challenges of FP. Seeing OO in Haskell helps you pick apart the different semantic pieces which build to form OO.
A type theorist's definition of OO will be quite different from a generalist's. For type theory, various kinds of polymorphisms mostly overlap between paradigms (e.g. generics and structural subtyping are useful in OOP), but nominal typing is both commonly associated with OOP and relatively useless in FP languages to be a distinguishing feature. This also dovetails with my named-oriented definition of OOP: objects have identity, but their types have identity also. If you come at it from the FP perspective, identity for objects AND types aren't very desirable...there are no meaningful names in math!
So really what I believe is that Haskell does a better job of being OO-FP than Scala does. I think there's still a happier medium between the two, though.
I'm also really eager to explore the type theorist's definition of OO in order to tease and bend the generalists definition. I'm not sure I completely agree about the idea of names that you outline, though. Existential types (and their dualized universal presentation) present unique types which could be named. State-transformer style (and the state-like monads) present a notion of identity-as-entity.
Again from the perspective of Haskell being a stab in the direction of a type theorists FP-OO language I find all of these features really interesting.
data Complex = { rotate :: Degrees -> Complex }
-- thus
-- rotate :: Complex -> Degrees -> Complex
where `rotate` is a state transformer and thus produces a notion of identity. Complex -> Radian -> Complex
offers two points of view. You can see it as a function which transforms complex numbers or a function representing an update to the internal state of a Complex object. The second highlights the need to keep track of identity. Throwing an STRef into the mix just gives you a particular kind of history—one which only retains the "now".Which is all sort of obvious and meaningless, but I still think it's interesting to think about. I think coalgebras tend to force thinking in terms of identity in this kind of way.
;-)
[To the downvoters: this is me replying to the conversation I've had in the previous posts in this thread. If you can't handle my surprise and happiness at reaching some agreement hit up 4chan.]
I've been reading religious advocacy of FP for almost 15 years now and yet there still don't seem to be many non-trivial apps written in any of these languages. Certainly many individual features of FP have been adopted to good effect in more mainstream languages but commercial Haskell or OCaml apps are still conspicuously thin on the ground. I think at this point it's up to the FP advocates to back up their claims with results. I'm sure there are a lot of programmers that would happy to roll up their sleeves and learn something new if it truly would make them 3x more productive.
Reddit was written in a Lisp originally. But like many major FP projects, ended up being rewritten in a more common language. I used to work at a place that had hundreds of thousands of lines of CL code that was all rewritten in Perl and was on its way to getting rewritten into Java by the time I left there.
I agree with the major thrust of your argument, FP seems to find itself mired in weird niches. I'm not sure if it's because FP advocates seem to not want to write mainstream end-user software or it just hasn't leaked out into the mainstream, but it's pretty tough to find lots of major FP success stories outside of a constant stream of anecdotal advocacy stories (none of which ever seem to be attached to some piece of software I can download and actually touch for day-to-day usage).
I suspect the problem is that lots of the kind of software end users want is difficult/hard to write in an FP style, but an order of magnitude easier in an imperative language (even if it's less "right")...FP is just the wrong paradigm for general purpose programming, even if it works better than imperative languages in certain niches. I think this is an illuminating summary of all this [1]. However, here's an engine written in Haskell [2].
It seems that FP ideas will continue to end up in non-FP languages long before FP languages will become a major success. That might be a success by itself?
To save us some time, here are some pretty woeful lists where this exact question is asked and answered.
https://programmers.stackexchange.com/questions/22073/is-fun...
http://lambda-the-ultimate.org/node/2491
You are probably thinking of ITA and their Orbitz service, using Franz Common Lisp. That is not really a FP language, but a good feather in the hat for Lisp nonetheless.
Foursquare uses Scala extensively. LinkedIn is using Scala extensively for its new projects. The commercial and successful use cases for Scala are widespread at this point. For Haskell and OCaml it's going to take a while.
Am curious to see how Scala 3/Dotty backed Scala works out. Enabling better tooling and faster build times via a generally less kitchen sink-y/more regular language should bring Scala adoption to new levels. Dotty is under active development, even saw a Github issue talking about the migration tool[1]. It's going to happen, just a matter of when.
I don't think I've ever heard of LabVIEW described as a functional language (and it's clearly not a "purely" functional language), but it programs like one. Programs are written by connecting VIs (functions in LabVIEW-speak) with wires in a data-flow diagram. VIs don't contain any state (usually, unless you explicitly use local or global variables--which all the documentation warns you never to do upon pain of something really awful). VIs mostly don't have side-effects (unless you're explicitly doing something with the filesystem, or data acquisition, or motion control, or... ).
One of the difficulties of using FP for "real" things is that you give up something you got for free with imperative languages, namely control of timing. LabVIEW works around this with something called a sequence diagram, which forces VIs to be called in a deterministic order (because, being FP, you can't otherwise divine in what order VIs will be called).
LabVIEW also encourages the use of a trick to enforce sequencing without using sequence structures (and this trick is absolutely mandatory if you don't want a write-only program): The abstract name of an instrument resource is passed to a VI, and then returned unchanged. Since the returned values of a function aren't determined until it is run, you can chain several functions together in this way so that they run in sequence (which, IMHO, is bad practice because this abstract resource thing being passed about is something which is needed by each function, and can be modified by each function, but SHOULDN'T be modified by any function).
Annoyances aside, I've written a number of medium-sized programs in LabVIEW that implement entire data acquisition systems, including control of external instruments, positioning of mechanical parts, high voltage power supplies (no side effects, heh, sure thing), pulsed lasers (same), and a synchronous detector of my own design. I would also note that the data acquisition systems do not use any local or global variables (as LabVIEW uses those terms), so it could be argued that they are have a very high "functional purity," though that was not my intention when I wrote them.
My personal opinion is that the productivity claims of the FP camp are largely inflated. Maybe FP is better suited to problem domains that exist entirely inside the computer.
Technical merits aside, I think this actually re-enforces the parents point.
I mean, you're basically saying that one of the best examples of production-ready Haskell is a Haskell IDE, developed by a company a good >80% of HN readers may never even have heard of.
Perhaps you are not looking hard enough? Sure, if your problem space is CRUD web apps, then you won't find many implementations. Try looking into more difficult problem spaces, such as high-frequency trading, and you'll suddenly find more FP examples.
Also, there's Erlang, which was developed by Ericsson to solve a very specific, real-world problem. Look it up.
I doubt any of these languages are significantly more popular in "more difficult problem spaces."
There's also Super Monao Bros (which used to be called Super Nario):
I agree with the sentiment that for a paradigm that has been advocated for 60 years it is really odd how little non-academia code is written in it.
In my experience you find functional code shines (and is therefore common) in places where prove-ability is of the utmost importance or where writing custom DSLs provide a lot of value. Things like nuclear systems, extremely high security applications (think custom OS level requirements) or financial modelling software.
That said in most of the places I've found the FP program is a high value prototype or proof of concept, not the actual system. I don't know if that is just a resource allocation thing or not.
Of course there are ways of abstracting those side effects and acting like they are the same as pure functions. Or conversely instead of hiding them, you could point them out and make them "exceptional", which is what I assume you mean by making them explicit. These machinations (of which FRP is a clear example) to deal with side effects have a price, and in a game like SMB it is entirely possible that the vast majority of the work would be this overhead.
Again, of course you can do it, but it is a problem domain not particularly suited to FP in and of itself. I've written short running, fast starting, memory constrained applications in Java, but it was a fight to do it. If the fight is worth the other advantages, it can be worth it.
No, those are precisely effects. They can only be side effects once you've defined how you're distinguishing side effects from effects generally.
A common way is labeling intended results "effects" and unintended results "side effects". By this definition, they're clearly effects not side effects, though this equally clearly isn't the usage in question.
Another way - common in Haskell - is to label effects that do not appear in a function's type signature "side effects". By this definition, it clearly depends on the implementation and there are "side effect free ways of changing pixels on the screen".
I don't think there is another useful definition to be had.
I'm not a big FRP proponent though. In my own work [1], I believe that manipulating pixels and doing physics should be effects...even side effects that are at least explicit during run-time. Of course, to achieve this, I needed to do special things to achieve commutativity and idempotence, which is otherwise taken for granted in a pure functional program.
[1] http://research.microsoft.com/pubs/211297/managedtime.pdf
Many FP paradigms (including Haskell's) impose an overhead when doing operations that are not pure. In the case of a SMB clone a huge percentage of the work of the program may end up not being pure, therefore a large overhead may be imposed. Further, many of the advantages of FP paradigms (including Haskell's) are dependent on pure functions.
In the case of a SMB clone, a large percentage of the application may be intended to be impure, therefore the "worst case scenario" comment.
That's the second sense I described. It's only meaningful if you're actually talking about a particular function. And you're still left with the question of the actual domain of the function. Semantics, but relevant semantics.
"Many FP paradigms (including Haskell's) impose an overhead when doing operations that are not pure. In the case of a SMB clone a huge percentage of the work of the program may end up not being pure, therefore a large overhead may be imposed. Further, many of the advantages of FP paradigms (including Haskell's) are dependent on pure functions."
That's somewhat true of FRP, basically false about Haskell IO generally, but either way if you can do it in PyGame someone can do it in Haskell - it's mostly a matter of working in a high level language, not a matter of functional-programming-imposed overhead.
I would recommend listening to what John Carmack had to say about implementing games in Haskell (and also Lisp, IIRC).
I am currently trying to write something in F#, which is still OO, only the rest of my team won't let me because there is "no proof it's better". Well, no. And there won't be if everyone says that. And where is the proof C# was better than VB.NET? Oh, right, the proof is VB.NET doesn't look like Java! Curly braces, the only evidence we need!
As long as the majority can hold back the minority that WILL learn something new just to see if it's better, everyone stays with the status quo.
"Just don't make me learn anything new" <- the developer motto.
Yes people seem to learn much better by example. The worst teachers (we have all had them) are those who jump straight into the abstraction that solves the problem instead of going through some examples first. If they cover a few specific examples with specific solutions then the student will almost invent the abstraction themselves. The same thing came up recently about pitching startups. If you don't describe the problem first with specific examples then people won't even recognise the solution (and often these people are far from stupid). In general for most people if they are trying to do something innovative they are rarely definitive about it at first. If you asked them to explain it or prove it they couldn't. They start with a hunch, do something ill defined, and then refine it as they go. Even mathematics is like this, except that often once someone has discovered and understood something, they explain it as if they were definitive about it all along - which can be misleading to say the least.
I really don't think you do understand, as an example in the last thread we ran into each other you completely missed the fact that Futures allow concurrency for free since you're already in a monadic context 90% of the time.
https://news.ycombinator.com/item?id=7751605
> The problem is that the kind of code it enables reuse of more than OO is very small, simple loops. This is never a big issue.
This is called a straw man argument. People aren't claiming that FP helps you write four line loops in one line. This article and others talk about orders of magnitude improvement in LOC on a large scale.
>"transactional" mutable state is simpler than the pure FP one, and just as safe.
You clearly don't know what these words mean. Mutable and purity are not mutually exclusive. Purity is a slang term for referential transparency. You can have mutable, referentially transparent functions. You can also have transactional memory in pure languages.
>Finally (and I've said this before on HN), a language like Haskell discounts the very useful choice of reasoning about your code after it runs, favoring, instead, all-upfront reasoning, often at the expense of facilitating the former.
Citation needed. You’re making things up.
The bottom line is you’ve created a product that solves many of the same problems functional programming solves. Your product looks very nice and advanced, but your unfamiliarity with functional programming makes you fear it, so you’ve decided to spread FUD across HN to the detriment of the community.
Overall i find Common Lisp Macros to be a better way for me to code, you can always construct your problem domain easily instead of choosing between two.
Research in Programming Languages giving us new tools to reason about our software development and FP is one of those nice tools.
> I used to like functional programming.
you should have said
> I used to like Haskell.
It's caused a lot of confusion in this thread. Also, I think it's unfair to compare Haskell vs. Every Other Programming Paradigm. Because by doing so you are only picking and choosing parts of other paradigms which work as well as the Haskell equivalent and not judging the whole of Haskell vs. Language X.
In fact I find the nature of functional code very much helps out here. With clojure, in a complicated system I can typically take any subcomponent and run it in the repl without much fuss/mocking or worrying about the state of the system. Since usually functions aren't modifying state I can rerun, modify, rerun over and over again without restarts.
So, yes, Haskell and Clojure development has a similar flow.
That's sort of the Erlang approach, right?
Which is generally considered an extremely pragmatic language.
I guess it all goes back to the Actor model and the origins of OO in Simula.
One is code reuse: yes FP code is definitely more reusable. The problem is that the kind of code it enables reuse of more than OO is very small, simple loops. This is never a big issue. Writing the same 4 line method -- which could have been abstracted with a monad -- 10 times in a 50-150K LOC project is never a big problem, and these methods rarely contain bugs. On the other hand, OO code is often easier to refactor and modify. Sometimes -- in Java for example -- dynamic linking combined with OO, lets you modify/add functionality without even re-compiling existing code. Heck, it let's you add and load new polymorphic type implementations at runtime. It is much more malleable than FP code.
Another is state management: yes, FP is one solution to the problem of managing state, especially in a multicore environment. But it is neither the only solution, nor is it the best. The Clojure solution of -- for lack of a better name -- "transactional" mutable state is simpler than the pure FP one, and just as safe. I.e. there are ways to make side-effects safe without restricting them so much that they become a nuisance (after all, all software, possibly with the exception of compilers, exists for the sake of its side effects).
Finally (and I've said this before on HN), a language like Haskell discounts the very useful choice of reasoning about your code after it runs, favoring, instead, all-upfront reasoning, often at the expense of facilitating the former. There are some domains where figuring everything up front is very important. Others, where trial and error is far more productive.
So pure FP increases code reuse, but of code that's not that important to reuse. It helps deal with the hard problem of state management, but other, simpler solutions exist. Finally, it makes it hard to "feel" how your code runs, to debug it, to profile it, and more. I think it is based on the premise that if programming could be made more formal -- more mathematical , if you will -- it will become easier/"better". But that premise has not been shown to be true. The lack of magically bug-free, FP OSs, drivers, control software, and large applications, shows that even the biggest supposed benefits of languages like Haskell, are yet to be demonstrated in the real world.
EDIT: Just to clarify: Unfortunately FP does not have a definition, so, when I was saying "FP", I meant "FP as the article's author practices", which means "statically typed, pure FP". FP using Java streams, FP in Clojure/Erlang, and FP in Haskell mean very different things.