I never understood why people rave so much about functional programming wrt Erlang/Elixir, when its functional programming is clearly only a means to an end (fast and safe message passing requires immutable data, which requires FP) and not a driving design goal in its own right.

I mean, unlike in typical hard FP languages like Haskell or Elm, mutable state is rampant in your average Elixir app, it's just spread out across many (global singleton) little processes. Only inside a process you're doing "true" FP but given how small the average process is in scope, in practice the only real big difference is that you can't to an `i++` style for loop. Oh no!

But once you leave the process boundary, and often even before it, all bets are off. The amount of Elixir forum messages I've read that go "you can't do X at that point, because Y hasn't completed yet" is nuts.

Eg you can't broadcast in a phoenix channel `join` because the channel hasn't been fully initialized yet. So you send yourself an :after_join message and do the broadcast in there. I don't know about you but to me this feels a lot more like C++ than like Haskell.

Or consider the library module Agent which is exactly identical in semantics to a global singleton variable in an OO/imperative language. It's just a blob of data that you can get or set.

Now, I don't think any of these are disadvantages. I did mostly C# and JavaScript before Elixir, so I'm used to the occasional mutable state flying around.

But I'll never understand that people like Elixir for being FP. You just get such a small subset of the usual advantages of FP that it feels like an implementation detail. There's lots of advantages, but freedom from thinking about state isn't one of them.

If you look closely enough at Haskell, you’ll realize that it also can have a lot of mutable state. Haskell just puts state into various monads and STM to make its functions pure. You can even spawn numerous isolated threads, all with their own state, and have them communicate with one another like you do in Elixir. It does provide a lot of structure and guarantees compared to your standard imperative language, but I assure you mutable state is still there.

To be clear, the Elm model of putting everything into a big tree and and transforming that at every user input is very unusual among even FP languages, and is not the model typically used in Haskell. This might be what you are thinking of.

Edit: I also want to point out that Elixir processes can be registered globally to act as singletons, but by default you can spawn any number of agents or other processes at runtime, meaning they are not singletons.

Actor frameworks with message passing, just like direct mutation, don't compose.

On Haskell, yes, you can just work off a thread and write shared buffers. But you are better off using STM or Async (these are a bit like futures), which compose, and you can write pipelines out of them.

Everything that you pointed out can be achieved in Elixir, quite easily too. If I understood you correctly.

Not sure what you mean by "actor frameworks don't compose"?

Actor frameworks don't compose, because actors are an atomic compute unit. You can't really take a horse actor and a bird actor and turn them into a pegasus actor. There are non-trivial interactions between message handling logic that would prevent such a thing with some fairly trivial-to-generate breaking cases.

Maybe composition for compositions' sake isn't that important?

> Maybe composition for compositions' sake isn't that important?

Definitely. I was about to ask "but when did you actually need to compose actors and how does that even make sense?" -- and while I am sure there are people who would find a scenario I feel that would still be tarrying on minutiae.

Obviously the actor model is not a panacea. But for my commercial work Elixir -- and thus Erlang's OTP -- has been a true blessing. There aren't many commercial scenarios where OTP is a very poor fit. They do exist but I'd dare saying they are no more than 10-15% of everything you can stumble upon out there.

Yep I love Elixir too. Just, you know, I try to only sip the kool-aid.

Well, I am just now arguing with people in another thread that nobody claims that Elixir is an end-all be-all.

It's quite sad how fiercely the average HN audience is attacking people who are trying to show an alternative way of doing things.

You can write great and useful software in non-composable languages. People write good stuff in C or Go.

But if you have the choice, composability always seems much nicer to me and stops me wanting to tear my hair out.

This is definitely true, I am just not sure how well it applies to actors. For all data-modelling scenarios and OOP/FP ways of doing polymorphism, composability is a life-saver though.

It's a good question. Let me meditate on actors and STM. Perhaps there's a way to get something like vectors that's composable.

The point of the actor model, at least in OTP, is not to have the actor model as a unit of computation (as was imagined by Hewitt, et, al) but as a failure domain. Erlang was not designed to be an actor system, it was designed to be a robust system and they later discovered that what they had built was more or less the actor model (if an impure version) and it was easy to communicate the architecture's computational capabilities and mental model as actors. Actually I think if anything it's led to a fair share of misunderstandings, and truly misses the important and unique aspects of Erlang processes.

In the sense of being failure domains, OTP "actors", or processes, ARE composable. With very little boilerplate, that is built in as primitive BIFs in the standard library, I can bundle failure domains together and orchestrate them using links, monitors, and supervision strategies, with the ultimate failure domain being a single erlang VM operating inside a cluster of erlang VMs, all supported without third party libraries as a part of the way of doing things.

Interesting idea, but I'm not sure I agree. I was doing a lot of telephony/mobile back in those early 1990s and there was a huge amount of formalism. We tried to model everything as FSM that transitioned states based on received messages. We even used code generators to generate code directly from diagrams. I believe Erlang grew out of that same approach. And initially, systems that failed would simply restart by default.

Then the idea of adding a supervising and observability was a natural addition to these systems.

That said, I think OTP is a great tool for modeling failure domains and I think it does a great job.

a process is a singleton even tho the code it runs is reused

I’m not sure what you mean. A singleton is something your program only has one of. You can have any number of processes. By your logic, every object in an object-oriented language is also a singleton.

> But I'll never understand that people like Elixir for being FP.

To me the answer is: using mutable state is opt-in. I disagree that "mutable state is rampant".

By opting in to the mutable state constructs you are basically saying "I know what I am doing, let me do my work" which is IMO quite fine because "pure" FP languages like Haskell can be a huge hassle when you actually need to deal with the real world.

To me Elixir is a very nice compromise.

Mutable state is essential to the actor model. Local arguments to a tail-recursive message loop which change based on the last received message and the previous arguments, and determine behavior (i.e. messages sent and side effects), are equivalent to local mutable state. State machines are a lot better than unstructured, freely mutable global variables, but they are still mutable state.

Sure. That's absolutely unavoidable in any FP language. All their compilers invisibly produce a lower-level code that's intrinsically using the mutable paradigm. It's how our hardware works currently.

No, in an actor based model the mutable parts that the comment you replied to mentioned are surfaced to the user.

It's not about how the Erlang VM is implemented. You could implement it in pure Haskell and compile it so some hypothetical pure CPU. Wouldn't change that part of how you interact with it that is stateful as described.

(And I liked programming in Erlang.)

And I still disagree with that comment that "mutable state is rampant". There are realities which we all have to take into account. To give an extreme example, should we get rid of all mutable databases and use append-only journals? That might help eliminate another class of bugs but there are a number of (practical and political) limiting factors in commercial projects.

But maybe we'll talk the same language if you give a few examples. I was under the impression that my parent poster made the point of "but your FP code gets compiled to imperative mutable code so FP is not good" or something. If I was mistaken in my interpretation then we're talking past each other.

My point was that the actor model has the semantics of local mutable state. The fact that an actor model language is compiled for a von Neumann machine is irrelevant to my point.

Okay, that's true, but especially in Erlang's BEAM VM the mutable state's access is serialized / centralized. An actor's state is not a globally modifiable volatile variable a la the C/C++ ones.

Yes.

Though I don't think anyone ever suggested the actor model (or Erlang) was doing C/C++ craziness.

As far as I can tell, that's not what the great-grand parent comment was about.

Have you ever programmed in Erlang?

About your first paragraph: not all databases should be append only, but it would be a good idea to make append-only the default and only deviate for a good reason. (Instead of making mutable the default, and only weirdos go for append-only.)

I work almost exclusively with Elixir for 3.5 years and love it. But I am not seeing that poster's point still.

I understand the point as: being able to send a message to a particular actor, passing it a value that you can later retrieve by sending that same actor another message, is semantically exactly the same as calling a setter on an object reference, and later calling a getter to get the value back. In both cases you have a pocket of mutable state, potentially accessible by multiple unrelated places in code, that isn't reflected in function signatures.

Mostly true, minus the fact that the mutable state's access is serialized / centralized due to the nature of Erlang's actors (lightweight threads; usually called fibers in other languages, and even that is not a good analogy since they are preemptive and not cooperative). So the semantics being similar is not strictly and 100% true. You can't do non-atomic volatile modification like you can in C/C++.

> You can't do non-atomic volatile modification like you can in C/C++.

Yes, exactly. No one was implying that Erlang's model is mimicking those particularly weird languages. The statement was much more narrow.

> I mean, unlike in typical hard FP languages like Haskell or Elm

What is a "typical" FP language? Are Scheme, Common Lisp, OCaml, SML, atypical?

> mutable state is rampant in your average Elixir app

I don't think this is generally true, or at least not on my experience, but I guess it may depend on what would one consider rampant.

Well, my main Elixir project is the backend of a web service. The input is JSON from the frontend and JSON or XML from a number of third party APIs. We process it, hit the db with select, update, insert and usually return a value. The same function with the same inputs usually returns different values because the db is stateful.

What I like of Elixir is not functional programming, it's the extensive use of pattern matching. What I like less is the convoluted syntax of GenServer with all those handle_cast/calls that obfuscate the real code. As a GenServer is usually a way to store a state, they should have had the courage of calling them objects and give them an object like syntax. After all they initially sold Elixir to developers coming from OO languages. If they had given it a Java like syntax maybe it would be 10 or 100 times as big by now.

Thankfully they didn't give it Java like syntax! The syntax is currently a bit more verbose and explicit on dealing with local/micro state which I now like, but without using 100 lines of boilerplate getter/setters. Using pipes gets one enough "OO" syntax feel.

While GenServer's store state they are, in my mind, more akin to microservices than Java/C++ Objects. They're like microservices, but without needing a separate service bus, global naming system (pg2), etc. Now I do wish Dialyzer/Dialyxer also had better support for checking GenServer handler's and messages, especially intra-process. You have a point the syntax their could be spruced up some perhaps. The process / GenServer paradigm kind of remind me of Smalltalk in a way, where you're passing messages that object may or may not want to respond to. That's not really possible with C++/Java objects.

I think you are misunderstanding what GenServers are and are for.

GenServers do have their own state, but they are used for way more than just storing state. It's probably best to literally think of it as a server in your running application, analogous to a server running on a network.

If storing state is all you're trying to do you might want to look at Agent or possibly ETS.

Further, GenServers come from Erlang, not Elixir. They're intended to model all sorts of runtime properties of code.

Practical tools often achieve that practicality by avoiding dogma.

"Dogmatic" is a great description of OTP, though. You have to adopt the OTP mindset, adhere to the OTP principles, and organize your architecture in the OTP style. There's a long initiation process where you learn esoteric vocabulary, become familiar with bespoke tooling, etc.

This isn't meant as criticism -- I think Erlang/OTP is a brilliant piece of work. But arguably Erlang/OTP is a practical tool that achieved practicality by embracing dogma.

I mean, yes? OTP is a framework—a paradigm for writing your code in, essentially—but it's an optional one. That's why it's split out from Erlang itself. You can write Erlang however you like. Most people choose to write it in the OTP paradigm. But sometimes that's not the best choice (e.g. the type of code that leex/yecc generates, does not obey OTP principles, nor would it help it in any way if it did.)

Fair point. Big frameworks and dogma go hand in hand, and I suppose Erlang without OTP is at least as practical as any other language.

I wonder though if Erlang might have been a footnote in programming history if it weren't for the mindshare that OTP generated. "Simple functional language" is attractive, but "simple functional language with world-class platform" turned out to be a game-changer.

> I wonder though if Erlang might have been a footnote in programming history if it weren't for the mindshare that OTP generated.

It absolutely would be. There's plenty of "simple FP languages" out there. OTP is very definitely the main selling point of Erlang, and nowadays of Elixir as well.

I think it's more accurate to say that OTP's dogma comes specifically from it having chased down pragmatism to a fault, to the point where whether or not OTP's doing something is a reasonably-reliable indicator of whether or not that something is a good idea (at least for Erlang's typical use cases) and therefore arguably warrants the dogmatism.

That's a good way to put it. I think we're in agreement that dogma wasn't avoided in this case.

> You just get such a small subset of the usual advantages of FP

Yeah but you're using that small subset in 90% of your code.

Look, this is not a thing to worry about in elixir:

    def p(my_array):
      do_something_with_this_array(my_array)
      return my_array

what is my array? I don't know. It could be anything. The company I work for just hired a sloppy python programmer that I don't want anywhere near my code, and you know what, if we change a section of our code to Elixir I am way more willing to have him work on our team.

I have a prototype system where I am passing over 100k msg/sec between a dozen backend services written in python (asyncio+redis) and I keep on wondering when my bottleneck will become functional programming and safe message passing by making copies. When will the madness end?

i don't think erlang is requiring FP + immutable + message passing for performance, but rather for correctness. Making copies is actually more expensive in terms of perf.

me too, and I have no clue as to how changing language can increase correctness in my case. I can think of other use cases where in-process concurrency and mutating data structures could cause problems, but I avoid those scenarios entirely anyway.

Tradeoffs. In the setup you describe I'd pick correctness over performance any day.

If you hit a performance bottleneck you might as well just use RabbitMQ or Kafka to queue up stuff and process it as it comes along. Or apply back-pressure if your current code allows for it.

I actually use redis hashsets to queue stuff when I need correctness. Much more performant than rmq's mnesia

I was in the same boat until discovering elixir/erlang it has by far been the most approachable functional language for me and it has been a gateway drug of sorts into experimenting with ocaml and various lisps.

I personally think elixir is really special and an amazing fit for any project that needs high IO concurrency.

IMO erlang (and arguably even more, elixir) is "functional for the working programmer". It doesn't drown itself in academic abstractions, uses functional programming as a tool to guardrail you from mistakes (in the same way that C guards the programmer from making asm mistakes), with escape hatches that are battle-tested and justified based on decades of experience.

I would say the only other functional language that has the same bent is Julia, which is "functional for the working scientist". It makes different choices about where to expose state, understandable, since scientific computing has different tradeoffs from systems programming.

Yeah I think that's a great way to frame it, historically one of the turn offs for me with FP was that it was overly academic when I am much more about how practical a tool the language is for what I'm trying to accomplish. Elixir was definitely the first FP that felt that way for me.

That said I have grown an appreciation for more esoteric languages and enjoy seeing the way they handle various problems it's just likely they'll never have a place in my toolkit.

I think Scala is the ultimate "gateway" FP language, for better or worse. Kotlin would probably meet the requirements, though I've never used it.

Agreed! I found learning Elixir to be a great gateway to learn Erlang as well. When I first tried learning Erlang (before Elixir was released), it became a bit too much to wrap my head around. It was a lot easier when I had gotten the fundamentals of FP down through Elixir, though!

Yes definitely, I would have never even bothered with Erlang before using Elixir and having it introduce me in a friendly way to Erlang, which is a shame because honestly the EVM and OTP are amazingly strong at what they both do.

On the Elixir/OTP side I've really enjoyed this course too: https://pragmaticstudio.com/elixir They have a current discount code LIVEVIEW (not affiliate), which is advertised on their currently free early access Phoenix live view course.

One of the really nice things about shared-nothing concurrency is that it scales to multiple machines. The same code that works locally will work across a data center. Along the same lines, shared-nothing tends to scale better as you add cores. With lock based programming, as you add cores, many times your contention increases minimizing the benefits of the additional cores.

Same. Or similar, it’s taken longer than it should for me to appreciate the functional paradigm.

Rust and Scheme have been gateway drugs ha. I found I really like OCaml-y languages.

Now, I’m very interested in Erlang (and to some degree elixir). I’m learning as much as I can about Erlang and the ecosystem; I’m trying to answer the question, “why isn’t Erlang more popular?”

Any guesses? Reasons? (syntax aside)

I might sound bitter but after about 3.5 years with Elixir my answer is very simple and boils down to:

Habit, confirmation bias, sunk cost fallacy.

Namely: people have gotten a lot of battle scars by working with what pays their bills -- PHP, Ruby, Python, C#, Java -- and they refuse to look at an alternative because that would render their huge time and energy investment moot (in their eyes at least; I don't see why this has to be the case but plenty of people have been adamant about this without giving an explanation).

I've only become a better programmer since I adopted Elixir but I never stopped using other languages.

All of that plus what PG calls "the middlebrow dismissal" are the main reasons IMO. People are just too set in their ways.

I've got those scars but I still find erlang refreshing when working with it. Unfortunately, as software engineers, we work in groups and there are many people having these scars. However, the erlang experience changes one forever.

As you said, unfortunately programming turned out to be quite the tribalistic activity indeed.

I got severely disheartened that 2-3 casual mentions of Elixir were enough for several people in this thread to attack and quickly stereotype me. I think I'll just keep quiet, or at least not mention what I work with. This seems to get the message across much better.

Interesting. Is FutureLearn something analogous to Coursera and edX?

Yes, very much the same as Coursera. Almost 1 to 1.

Did you go to Northeastern? Glad to see a fellow Husky in the wild!