Other than distributed/concurrent system use-cases, could you share what kind of products are best when built with elixir/erlang compared to easier to write languages like Go, for example.
Go, and any other language, lowers the barrier to running concurrent code, but there is much more to concurrent servers than concurrency: fault tolerance, isolation, shared state management, instrumentation, introspection, clustering, process migration. The BEAM and its ecosystem gives you all of that out of the box.
Also, the BEAM offers an immutable, functional environment. Data races are impossible, which are the biggest pain and source of heisenbugs in any kind of system with > 1 concurrent thread. This is huge. You can model your entire system as concurrent processes without ever having to deal with concurrency issues. You only ever have to think in "single-threaded" mode.
As far as the application code goes, but most systems have databases which opens you up to all kinds of race conditions. Does Elixir help in that case?
Much more common instead are race conditions on local storage, external APIs and traditional memory races when using unsophisticated languages.
Also using serializable isn’t free.
You do need to manage this somehow, but the building blocks are there. Of course, the building blocks for transactionless database access are there too.
For example, if you use blocking RPCs between two processes, they can deadlock waiting for each other's responses. Try implementing Dining Philosophers, you will learn a lot.
The answer is not to use blocking RPC, but that will challenge your brain topology. It takes some time to be able to lower yourself into the hot bath of full asynchrony, but when you do, it feels very very good.
I characterize it as the co-problem (in the sense of the dual of a problem). Distributing a problem, starting processes and pushing messages is easy. But it may be hard to know when you're finished. Everything is easy, but termination is difficult. It's the Erlang/Elixir Halting Problem.
It is not often mentioned as a strong feature for concurrent programming, but Erlang/Elixir have timeouts built into the language. And when you add OTP supervisor restarts, you can avoid some common programming mistakes through random evasion - don't do this by design, but it does help resilience.
My opinion is not that rare, so instead of listing my credentials, you can search and find many others that have found this platform to be a great fit, simply because it has been designed to solve this very problem since 1986 when everybody else was focusing on single core, isolated systems.
For everything else, which is not a lot, there's Rust, Scheme, Lisp and many other fun languages to explore. My focus these days is on my business rather than consulting, so I have a lot of freedom.
The drawbacks are basically in recruiting and a few other areas and quite manageable.
Edit: There's a book for this particular purpose, convincing the suits, https://pragprog.com/titles/tvmelixir/adopting-elixir/ .
Yes, we probably could have done some things better if we had a bit more Erlang experience on our team; at least while I was there, none of our applications were properly packaged as OTP applications, and maybe that could have been useful. But overall, we were smart, experienced server people who were willing to learn Erlang and we were handed a tool that fit our needs very well, so we all got Erlang books and figured it out. If you can recruit smart, experienced server people who are willing to learn a new language, you don't have a recruiting problem. I haven't personally worked with Elixir, but I feel like most of the unfamiliarity is going to come from the underlying BEAM and OTP, so same difference; Elixir just has different syntax and macros are more heavily used, IMHO both syntaxes are going to be unfamiliar to most.
It takes a bit of getting used to pattern matching and the overall functional/declarative approach when coming from a more imperative background, but I believe learning the ins and outs of the business niche will generally be harder and take more time.
In some companies handling recruitment issues isn't enough, investors or shareholders might be worried about 'exit' and how to maximise it for themselves, and refuse tooling they perceive as obscure and expect to lower bids to buy them out.
I would love the chance to learn more Erlang (looked at the beginning of "Learn you some Erlang for great Good") or Elixir (used in last year's AoC) on a job and get to use OTP, watch it run my function calls on multiple machines and all that. I know a lot about functional programming, as I do it in my free time (big Scheme fan).
As it is currently, I cannot apply my skills at the job. For example when I think that some code should not mutate some state, but rather use pure functions and the tests should be simply function calls and checking the output, then I don't get the time to do that, nor the time to show how this would look like and how it would make things simpler. No one aside me on the job seems to be interested in purely functional data structures/persistent data structures either, which sooner or later are necessary, if one wants to make things purely functional. So basically I am the only person with that knowledge and cannot apply it. It is so dull.
Talk about under-appreciated.
Any sufficiently complicated concurrent program in another language
contains an ad hoc informally-specified bug-ridden slow implementation
of half of Erlang.
https://rvirding.blogspot.com/2008/01/virdings-first-rule-of...Elixir matches the way my brain thinks somehow, with (kind of) pure functions transforming data step by step, and when you can break down some task in such a way its only a simple set of pipelines (|>) its really great and feels great. It is also exceptionally cool if you spawn Agents to hold some global state where multiple processes talk with it, plus the integrated stateful introspection/debugging is just chefs kiss. In the context of pg's "Blub Paradox" Elixir is an acceptable Lisp (not homoiconic, but with modern tooling). You can solve very complicated problems in very clean ways. Also often underrated: Phoenix/LiveView is probably the best escape hatch out of JS frontend hell alltogether, and leads to better outcomes (performance, scalability, sanity, maintainability, ...) compared to JS frameworks.
On the other hand, sometimes I have a very dirty real life thing I want to achieve. Like doing some Unix stuff, interacting with some ugly APIs, implementing an given imperative algorithm to brute force a problem quickly within reasonable constraints, manipulate some image file, automate some adhoc outlook365 process, ... you name it. The weakness of Go (extremely simple/plain, verbosity, boilerplate, ...) here is actually the strength of it in these cases, but took me a while to realize. In Go, I don't even care anymore to make anything "elegant" (which is very tempting in Elixir!), but write the absolute straightforward series of steps in brutal directness, including nested loops and very long functions. This leads to rapid dirty work solving, and has the added benefit of trivial distribution (cross-compile to a single self contained binary) to other folks that don't have dev dependencies installed. Also I rely solely and the compiler/linter for this type of code and have zero tests for it (I am not going to mock the filesystem interactions and all that for basically a better ad hoc shell script).
So for the big/complex/scalable projects, I think Elixir/Phoenix is just perfect in terms of a "web stack", only a few rough edges left in iE the docs for beginners looking into LiveView. But for the small ad-hoc stuff or in situations where I can "brute force" my way through an ad hoc problem, Go is it.
Just my 2 cents.
Most people? I have no love lost for Go, but most (if not all) programmers have experience with imperative code, making it very easy to learn, even if the syntax is ugly. Elix is from a different paradigm, so you have to learn that in addition to the syntax.
I haven't used Golang for things like binary protocols so I can't really compare, but Elixir or Erlang would be a good fit since they're very good for expressing grammars and fundamentally treat strings as byte sequences.
If pattern matching helps you express your problem domain succinctly they're also a good fit. Same goes for macros. My impression is that Golang commonly requires quite verbose or complex code compared to Elixir.
I expect raw number crunching performance to be better in Golang, but BEAM processes are very lightweight so it might win on either performance or developer ergonomics if the task can be solved in parallel.
What about “distributed systems” is elixir actually shining at? From my reading, it seems rather like a good option to _avoid_ distributing (thanks to the light processes and concurrency handling)
A process can be anywhere in the cluster, and you can interact with it as if it was on your local node.
I use all of these facilities in production on my product with ease. I’m happy to answer your specific questions.
Need raft concensus? Paxos? Something else? Want to place work with consistent hashing? Want to hydrate/dehydrate with persistence vs memory only?
Those are all important decisions and the languages do not force you to one option.
The structure of the system is such that you can call any function on any node in the cluster.
In order to do it well, you need to understand your own abstractions. If we have a cluster are we better off passing every function around the cluster or running some locally? Is the network overhead worth it in every case or only in some?
It's up to your code to make those determinations about your application, but everything you describe is fairly simple to implement on the BEAM.
Erlang provides "process groups" that allow you to structure groups of processes across a cluster.
It doesn't try to be magic. It's not a workload scheduler or "automatic distributed machine".
You structure things the way you want, on top of it's primitives. It's just that all of the primitives you need are built in and work really well.
Second, as the other comments say, Elixir/Erlang have very transparent built-in ways to address other nodes, and the programs will scale as expected: more cpus v. network data distribution cost. So good for small tasks that require lots of compute. The native clustering will run to a few dozen nodes over a trusted LAN.
But soon, you will want some layer managing the cluster, try Swarm:
I have only finished one Elixir app. It was a lot fun and the app has proven rock solid. For me it felt a bit like using microservices but with lots of the footguns and ops overhead removed. I would love to use it for more than a hobby project.
[1] https://pragprog.com/titles/sgdpelixir/concurrent-data-proce...
"The Soul of Erlang and Elixir • Saša Jurić • GOTO 2019" - https://www.youtube.com/watch?v=JvBT4XBdoUE
When you feel sufficiently amazed by this demo, I also recommend the Elixir in Action book, by this same author, to get started with this incredible ecosystem and paradigm.
---
It's hard to imagine a better platform than Erlang's for writing distributed and networked systems of any kind. I fell in love with Elixir in 2016 and have been using it full time since, first adopting as CTO in my previous company, now powering my new solo business. It's such a good language, with great community and stewardship, running on a rock solid platform that's so advanced compared to most popular languages today.
Erlang was created 38 years ago, and it's good to see the ideas of Armstrong, Virding and Williams to be vindicated today. They simply were too far ahead of their time.