This is incorrect. One of the former VPs said he would not use Scala. Twitter uses Scala massively and has published and continues to maintain several open source Scala projects.
> LinkedIn definitely tried Scala, failed, and went to Java
This is incorrect. A newly hired VP insisted on moving the front end from Scala to Java. In the offline backend stack Scala is used extensively. Check Kafka, Samza from LinkedIn or better yet ask current LinkedIn employees.
Apart from these Spotify, Foursquare, ebay, several banks use Scala. FP is huge in financial institutions, check K/KDB+. Jane Street etc. Thats where most FP programmers work. Also verizon, jet.com and walmart.com. There are several dozen large companies using FP.
FP is mainstream now. Traditionally, it was financial institutions using some flavor of ML/Haskell. With Scala/Clojure on the JVM, Si Valley has moved into FP in a big way.
The reason for this pattern is because companies look for easily replaceable developers.
To get into functional you need to be good at abstracting the business domain, this requires seniority which is hard to find. Most of the time, when you are working using functional programming you are implementing the business vocabulary. Entry level developers are not good/interested in this. Their worries are in how they can use Angular 2 in this project, or why we need to still use SQL Server instead of converting to {insert the newest database technology}.
It is easy to find (and replace) junior developers that are happy because they created something with the latest technology and it provides the immediate value intended. It sounds good, but it misses the point.
Software development in a company happens to support/enhance the business. This means that it has to stand the test of time, not only in reliability but also in adaptability. The only constant in software is that it will change. Here is where building a vocabulary that represents the business subjects pays off. Change now is a definition-change and, unless the business is pivoting, definitions rarely change; instead, they have constant adjustments. Functional shines in creating a vocabulary and having this adaptability.
Of course, you can adapt using OOP, but the question is not if adaptability is possible, it is instead how to make changes at the lowest risk.
>Here is where building a vocabulary that represents the business subjects pays off. Change now is a definition-change and, unless the business is pivoting, definitions rarely change; instead, they have constant adjustments. Functional shines in creating a vocabulary and having this adaptability.
I tend to use the same arguments to make the case against FP for business modelling.
I tend to say that FP is good at adding new operations to existing things, while OOP is good at adding new things to existing operations. However when Software Development is used to model a business - as it should be for reasons you accurately explained - the reality it meets is the one of the business world in which business needs evolve and requirements get added. You start by supporting local groceries and then you grow to support global supermarket chains. OOP is more suited to model business because you can naturally extend existing types and apply existing operations to them (ie. pass a supermarket everywhere a shop was expected + things such as polymorphism). In FP, every time you add a type, you need to add switch cases to your existing operations (at best), or redefine them create / new ones (at worse), since the ubiquitous assumption of FP development is immutability and known types.
I'd be curious to know how you approach this conflict in a way that makes FP work in real world business modelling which needs to be extension-oriented. It seems to me that the more operations using a type you have, the more you will be screwed when you want to extend this type. The only reason I can see this wouldn't be the case would be to say that I approach FP with the wrong philosophy and that there is no such things as extending types in FP. But in this case I'd question if FP modelling really is the best tool to model real world business for which it is natural to define things as extensions of other things.
What you described is exactly the _expression problem_. And it has some solution.
See https://oleksandrmanzyuk.wordpress.com/2014/06/18/from-objec...
The functional and strongly typed value proposition of FP is very appealing on paper but only shines when you have a level of certainty and knowledge on requirements that you typically do not have in real world software development. OOP by design embraces this uncertainty (but as a trade off is less assertive on correctness than FP).
That being said, certain businesses do not have a high level of uncertainty in their developments while correctness is a big deal for them (banks, scientific researchers, etc...). It makes sense for them to reinvest this slack in correctness. However, for those iterating consumer facing products (which represents the majority of developments), OOP modelling usually is the way to go. Because of the expression problem (which should probably be called the expression dilemma to better highlight the fact that the assessment should be continuous and situational rather than ideological and/or philosophical)
"OOP is more suited to model business because you can naturally extend existing types and apply existing operations to them (ie. pass a supermarket everywhere a shop was expected + things such as polymorphism)"
The reason why you feel is more suited is because you are thinking in an object oriented way. It is not a bad thing, but it is not functional.
"... extend existing types and apply existing operations to them". This means you have things, and you do things to them. This is the OOP way, not the FP way.
In FP the things are immutable, so you don't do things to them because they cannot change; instead, you create new things out of what you have.
In OOP this is what you have, this is what it can do, and you put all the gears (or objects) together and you get your engine.
In FP you have pipes, this is what I have and if I put it through this pipe it becomes something else.
When a local shop there is a sale, there is a function that takes a bunch of arguments and produces something called a sale, Maybe is the total, maybe is a tuple with the different pieces of the sale.
When you move that to a supermarket, then the question is: how what the shop owner calls a sale in the local shop is different from what the supermarket owner calls a sale? How my vocabulary changed? for a lot of things, you will see that the meaning is the same, just over more data; and also, there are new words that didn't exist in the smaller context. A sale is in both cases the sum of the quantity multiplied by the price of the groceries, apply the tax, spit out the result. But maybe the meaning of Tax changed, instead of been a fixed percent, it needs to be the lookup in a table depending on where the supermarket is.
If you need to constantly add switch cases, then probably you are using a functional language using an OOP mentality.
The way that I have introducing FP to people is using Excel. Think on this syntax:
A1: 1 B1: 2 C1: A1 + B2
You don't need new cells, C1 does not mean the same thing and the formula have to change... the formula or is it the function?
When you model the real world, you can focus on the objects, map them to types, and go from there. Or you can focus on the meaning of the words, then you need just a few types, see how those meanings compound, and go from there.
"But in this case I'd question if FP modelling really is the best tool to model real world business for which it is natural to define things as extensions of other things."
I agree, FP is bad at extending things... but it is not FP fault that you see the world as things that need to be extended.
A middle step between what you are saying and FP will be Flow Programming (https://en.wikipedia.org/wiki/Flow-based_programming). It is not FP, nor OOP, but somewhere in between.
>Or you can focus on the meaning of the words, then you need just a few types, see how those meanings compound, and go from there.
>I agree, FP is bad at extending things... but it is not FP fault that you see the world as things that need to be extended.
I should probably give another look to FP and continue to study the subject, I have likely been approaching it with a OOP mindset as you suggest. One last question though, what about code duplication ? If you focus on the meaning of words and compounding these meanings to create new things, wouldn't it most of the time create lot of duplicate code ? For example in the case of the local shop and the supermarket, wouldn't you have two functions (or a switch case) containing exactly the same code to calculate the quantity multiplied by the price of the groceries ? One would take a local shop as parameter, and the other one would take a supermarket.
Splitting the sale processing function in two parts - one summing up the groceries and the other one calculating the taxes - so that you can reuse the first function implies knowing in advance the existence of the supermarket structure (which you will only know many years after putting your product in production). Not doing it in advance either means duplicate code or refactoring every time you add an operation. The extension mindset initially comes from the need to avoid code duplication which is known to make maintenance difficult.
Now maybe the strongly-typed experience of FP and its compile-time checks are so convenient in practice that they make the maintenance of this kind of code easily manageable ? Maybe it's not even really duplicate code since what would be a basic number in OOP would be two different types here in FP ? Or maybe it's still thinking about FP from an OOP perspective ?
What goes through the sale function pipe is not the store and its size, it really does not matter for the meaning of sale were the sale happened. What matter for the sale is the cart. The cart might be more or less sofisticated because the size of the store, but is still a cart.
So the sale function is a pipe that eats a cart and spits out a receipt.
Or if you want to put it in another way, a sale is something that might happen to a cart over time. Functions are the clock on the system. In a functional system, times does not pass until a function is executed (does it sounds like lazy evaluation?)
In functional programming, we avoid duplication of code by appliying functions to the data so it spits what the next function needs. If you have a function sale that can eat a simple cart, but now you have a complex cart, one solution is to create a function that converts the complex cart into the simple one that the sale function needs; another way is to make the complex cart having the same data as the simpler cart. What you choose to do is proportional to how different the meaning of cart is, or maybe the cart remains the same. In general terms, this type of thinking tend to produce less lines of code needed per similar acomplishment.
So in OOP your are using objects to put together the store, while in FP your are descriving what happens in the store by connecting pipes betweem the different data. It is not about what can happen to the data, but what makes the data travel. It is not about what steps makes it work, but the fluidity of the data life span.
In particular your demonstration was very convincing from a DDD perspective. I can see that FP is not less natural than OOP for domain modelling while it approaches things from a very different angle. However FP brings to the table rich domain semantics, high correctness guarantees, and I feel productivity gains too. And with the mindset you've presented, it is also clear that the unknown seems to be accounted for in FP too. As an architect, this last point was the grey area and the deal breaker for me. However now I see no reason not to invest in heavily leveraging FP moving forward. Thanks again jorgeleo.
Which is definitely part of the "company at scale" equation the GP referred to.
Java has had lambdas and streams for awhile now. A pretty common design pattern has also become anemic domain models operated on by groups of functions, which fits in the spirit of functional programming. Modern java can, and is used in a functional style.
On the JS side, functional concepts are everywhere. Given how heavily FB and Twitter are invested in JS, I don't think they have abandoned functional programming at all.
Finally, IMO Scala is actually a pretty poor example of a language that can be used at scale. Without a highly enforced set of rules it can quickly become a write once/read never language. I'm sure there are exceptions, but given the choice between scala and java 8+, I would go with java. Clojure may be an option though.
Pojo's which are just data structures or a glorified HashMaps for holding data. And, if allowed, using Lombok to get rid of all the getter/setter cruft.
The second category of class types are just Service classes. These classes hold no state and are a logical grouping of methods that typically operate on a given data type. However, the class is based more on a specific role it performs than the type(s) it operates on. Also, there's considerable thought given to reuse of the Service classes outside the given context at hand.
https://static.googleusercontent.com/media/research.google.c...
Imperative/object oriented languages are often more practical for real-world use (MapReduce itself I believe is written in C++), however it's the ideas of functional programming that matter most. Those with experience working in functional languages are at an advantage over developers who've never had that exposure, because they have a wider range of mental tools at hand with which to solve problems.
Erlang was developed for Ericsson phone switches with a large proportion of uptime.
Whatsapp also use Erlang.
[1] http://harmful.cat-v.org/software/OO_programming/why_oo_suck...
Then, my thesis supervisor said "But you're wrong, Erlang is extremely object oriented". He said object oriented languages aren't object oriented. I might think, though I'm not quite sure if I believe this or not, but Erlang might be the only object oriented language because the 3 tenets of object oriented programming are that it's based on message passing, that you have isolation between objects and have polymorphism.
..."
Read the rest. It's good.
I've heard of inheritance as a corrupting influence which has taken over and corrupted OO for years/ Funny enough, even most OO folks eventually realized that and moved to banish inheritance as a viable pattern.
Ruby is OOP. Elixir/Erlang is purely functional (although dynamically-typed... some will prefer typed... but the results speak for themselves).
If you are a long-time developer like me and are just coming across this, this news should strike you somewhere between "astonishing" and "pure fucking magic."
Here's the deal- Paradigm shifts, most people only see them AFTER they've finished. If you want to be ahead of the curve on this one and you want to reap the benefits, go functional (Haskell, Clojure, OCaml, F#, Elm, Erlang/Elixir). Today. That is my #1 piece of advice right now. (Well, that, and to avoid Scala and Go. Scala is just too overly complicated to keep your cognitive load down as a developer, and Go... Well, the name even says it: "Go!" As in, "don't stay!" ;) Go will suffer the same long term problems that all OOP/procedural langs share, compounded by its idiotic take on error handling /opinion)
Elixir: http://elixir-lang.org/
Phoenix: http://www.phoenixframework.org/
There are a lot of interesting aspects of functional programming, but there's no particular miracle here: Ruby, particular Ruby on Rails, is slow.
[1] https://www.techempower.com/benchmarks/#section=data-r13&hw=...
[2] https://www.iron.io/how-we-went-from-30-servers-to-2-go/
JavaScript, for example, is ubiquitous in the industry, and, especially with the advent of ES6, can be written quite functionally. ReactJS is also widespread and can also enjoy an FP approach.
I personally find it much easier to navigate and fix bugs in an Erlang code base than in the equivalent C++/Java/C# one. It mainly boils down to how state is updated. When I see NewState = update(Value, State) it is clear what is happening. In OO code, I usually need to remember what object inherits from, what are is the instance state so far (what other members have been called) and how it got there. I find that much harder to handle, for example.
Another approach is to run experiments but they are universally terrible. Their methods have so little relation to the way real software is developed, that it is hard to take any conclusions seriously.
I would also like some hard data, but don't hold your breath. Technology decisions will probably need to made based on other factors for the foreseeable future.
There was a recent post here on HN about empirical language design and I thought that was a refreshing change. I'd like to see a change in software culture towards an evidence based approach. It's long overdue. Alan Kay called the software world "pop culture" and I think he is right. Software is more fashion driven than Vogue magazine.
Academic research could be a vehicle for a more empirical approach. Research in the hard sciences embraces empiricism. Sadly, the academic research in software doesn't, generally, adhere to the same level of rigor.
From the article "in this case for massive distributed systems Functional Languages are the right tool ...
Immutability at scale is really important because it allows you to go from 30k members to a staggering 2.5 Million+ and not even feel blip within 3 months, like we did here at Jet from the end of July to October. That’s powerful!"
For example: I've been developing for the last 3 years for the iOS and we been using FP and FRP almost everywhere with different teams and companies. Compared to the standard OO or Apple -way of doing things we have been able to make much better tests, stability/maintainability of the apps has increased etc.
Designed to replace and address flaws of an older language but ends up being huge and unmaintainable, with very questionable design decisions and a tool chain that's simply not up to standards in the 21st century (slow compiler, poor IDE integration that keeps breaking version after version, etc...).
In C++, some of these design decisions were driven by legacy pressure but in Scala, a lot of design decisions were driven by the pressure on the team to write research papers. Either way, both languages end up being monsters that implement all known programming language features known on Earth.
Your C++ to C analogy doesn't hold, because C++ is an (almost) superset of C, while Scala is not a superset of Java at all. Modern C++ compiler can compile most of C code with at most minor, cosmetic changes.
As for the toolchain - I'm working in it daily and so far it has been great. Ok, compiler is not as fast as Java (point taken), but this doesn't matter as long as I can compile my code fast enough. IDE support is stellar compared to most other popular languages, particularly dynamic ones like Python or PHP.
Odersky himself gave up on scalac and he's writing a compiler and a new language from scratch.
* C++ language specification: 1374 pages (and still includes plenty of "unspecified behavior" / "undefined behavior")
* Java language specification: 788 pages
* Scala language specification: 191 pages
> Odersky himself gave up on scalac and he's writing a compiler and a new language from scratch.
Odersky created a new language to be able to do research without sacrificing Scala stability, which is already mainstream and should not undergo any more major revolutions. So Dotty is for research / incubating language features, while Scala stays on the stable side of things, letting companies like Twitter or LinkedIn develop without fear that 2.13 will not compile their 2.12 code.
> also poorly designed overall
This is an opinion, not a fact.
> With so many features that they interact with each other in ways that are pretty much unpredictable.
Name three. I'm coding in it for several years, and didn't notice a single one issue. I heard this complaint about Scala being too hard a few times very long time ago (when Scala was at version 2.6, and its tooling was in fact terrible), but it was always from programmers, who had also major problems coding in Java, so I never treated this seriously.
That's really the main issue per the original question. It's not that functional somehow fails on its merits to deliver, it's just that "at scale" means having enough developers who know it. The reality is that imperative/OO languages have the greatest mindshare, and thus it's far easier to find developers, and far less wind-up time, if you're using such a language.
I'm not sure if I can call this "the Haskell community consensus", but a lot of the Haskell community finds Scala too complex to be worthwhile. They find it introduces a lot of complexity above and beyond Haskell, but at the same time, fails to take proper advantage of it and the result is quite dirty, full of special cases, and consequently hard to reason about. So you get a lot of the costs, perhaps even more costs for compliance than a Haskell code base, but you get fewer benefits.
Also demonware [2], which is owned by Acitivision/Blizzard and manages online game lobbies for all their titles (ie Call of Duty series across all platforms), uses Erlang. They switched from C++ to Erlang before Call of Duty Black Ops was released. They said if they hadn't decided to switch before the release the company would have never been able to handle the 5 months of user growth the game brought to their platform [3].
[1] https://github.com/janestreet/core [2] https://www.demonware.net/ [3] https://news.ycombinator.com/item?id=14120506
This is a good video, from Jane Street, on why they use OCaml:
Google primarily uses C++/Java. Facebook: Hack. Twitter tried Scala and as far as I know, abandon it and went to Java. LinkedIn definitely tried Scala, failed, and went to Java. Square is Ruby/Java.
I'd like to have something to point to as a success story of a company at scale with functional programming. Anyone know of one?