This is my main problem with elixir. Even for senior developers it's hard to know how the building blocks and runtime works and since all abstractions are leaky, you'll end up with mysterious issues that only make sense once you understand all the layers. And in case of elixir there are lots.
In my first 6 months of Elixir development, I got very far with just modules, structs, and functional programming.
A common pitfall for new Elixir developers is to reach features that are overkill.
Metaprogramming, gen servers, protocols, etc, are powerful, but you don't need to know them to start building apps with Phoenix.
GenServers I disagree with you on even more so since they can handle arbitrary messages (handle_info) it makes it a lot easier to do things like run a continuous process that does an action every n seconds as an example and also to handle life cycle messages from other processes. Oh my downstream consumer says things are finished handle a message that cleans up the children processes and shuts down the supervisor so we don't keep polling.
I use them to great effect in web crawling as a producer for things like Broadway and things like Agent or Task don't have the flexibility at the level I need it.
All I can offer are anecdotes but I've worked at one company with a sizeable rabbitmq cluster where we were fortunate enough to have a guy in a different team who was a BEAM maintainer in his free time. He was the only one able to debug issues (mostly around unexpected memory usage).
At another place, we had some in house elixir services. Developers were very happy and productive with it, but we also ran into unexpected memory usage and crashes due to that and to the day their root cause are unclear, even after adding detailed telemetry to monitor process counts and memory profiles.
I guess I have a general issue with VM based languages, but that might be due to the fact that I spend more time debugging issues than writing software and I appreciate the common libc/syscall api used in VM-less languages.
Number one cause of hard-to-track memory usage in elixir, especially with services that parse JSON. Someone is ingesting JSON and caching a snippet of the JSON in an ets table, which means the entire JSON is never GC'd.
To identify:
erlang:memory() shows unexpectedly large word count for binaries.
Crawl all your ets tables and look if referenced_byte_size(Binary) is significantly larger than size(Binary) (you need to check the keys as well as the values, and you'll need to write something to crawl through your complex values, of course)
If you don't find it there, it's probably references held in processes. You can crawl processes with process_info(Pid, binary), but it's a little tricky because the value you get back is undocumented.
If you find a lot of memory used by binaries, the solutions are all pretty simple:
a) if it's binaries held in processes, and those processes really don't hang onto binaries, just receive and send them, and you're running older erlang (before 19?), try to upgrade --- there's a GC change around then to include refC binary size when checking to see if process GC should run.
b) anywhere else, do StorableBinary = binary:copy(Binary) and store StoredableBinary instead of Binary. This gets you an exactly sized binary (heap or ref-counted), instead of a binary referenced into another.
Parsing larger binaries (like JSON) is one way to get these overlarge binaries, but they're also pretty easy to get with idiomatic code like shown in the efficiency guide[1]; when the system builds a appendable refC binary, it makes appending efficient, but storage inefficient.
[1] http://erlang.org/doc/efficiency_guide/binaryhandling.html#c...
Elixir compiles down to Erlang which is compiled down to BEAM code.
But in short, yes, Elixir and Erlang are more akin than they are not. As both compile to an intermediate form called Erlang Abstract Format.
Here is the info: https://elixirforum.com/t/getting-each-stage-of-elixirs-comp...
that being said, the powed of elixir comes from how easy it is to learn and use it, the macro expansions (which in my opinion is the killer feature and is not talked enough) and the seamless way it managed to leverage and enhance the erlang ecosystem.
But annotations are just tags that do nothing, and god knows where the code that actually does anything as a result of them resides.
Isn't it well known that GenServers can become severe bottlenecks unless you know the inner workings of everything to the point where you're an expert?
I'm not an Elixir expert or even used a GenServer in practice but I remember reading some warnings about using GenServers around performance because they can only handle 1 request at a time and it's super easy to bring down your whole system if you don't know what you're doing.
This blog post explains how that happens: https://www.cogini.com/blog/avoiding-genserver-bottlenecks/
And I remember seeing a lot of forum posts around the dangers of using GenServers (unless you know what you're doing).
It's not really as easy as just copy / pasting something, adding 1 line and you're done. You need to put in serious time and effort to understand the intricacies of a very complex system (BEAM, OTP) if you plan to leave the world of only caring about local function execution.
And as that blog post mentions, it recommends using ETS but Google says ETS isn't distributed. So now suddenly you're stuck only being able to work with 1 machine. This is a bit more limiting than using Python or Ruby and deciding to share your state in Redis. This really does typically require adding a few lines of code and now your state is saved in an external service and now you're free to scale to as many web servers you want until Redis becomes a bottleneck (which it likely never will). You can also freely restart your web servers without losing what's in Redis.
I know you can do distributed state in Elixir too, but it doesn't seem as easy as it is in other languages. And it's especially more complicated / less pragmatic than other tech stacks because almost every other tech stack all use the same tools to share external state so it's a super documented and well thought out problem.
What Erlang/Elixir bring to the table is that there is a good vocabulary and introspection tools to observe these issues. For example, if you have a GenServer as a bottleneck, you can start Observer (or the Phoenix LiveDashboard or similar), order processes by message queue, and find which one is having trouble to catch-up with requests. So we end-up talking about it quite frequently - it is easier to talk about what you see!
If you need distributed data, then by all means, use Redis or PostgreSQL or similar. ETS is not going to replace it. What ETS helps is with sharing data within the same node. For example, if you have a machine with 8 cores, you may start 8 Ruby/Python instances, one for each core. If the cache is stored on Redis, you will do a network roundtrip to Redis every time you need the data. Of course you can also cache inside each instance but that can lead to large memory usage, given each instance has its own memory space. This may be accentuated depending on the data, such as caching geoip lookups, as mentioned in the post you linked.
In Elixir, if you have 8 cores, it is a single instance. Therefore, you could cache geoip lookups in ETS and it can be shared across all cores. This has three important benefits: lower memory usage, reduced latency, and increased cache-hit ratio in local storages. At this point, you may choose to not use Redis/DB at all and skip the additional operational complexity. Or, if you prefer, you can still fallback to Redis, which is something I would consider doing if the geoip lookups are expensive (either in terms of time or money).
In any case, ETS is completely optional. If you just want to go to Redis on every request, you can just do that too! And for what is worth, if I need distributed state, I just use to the database too.
What if the geoip lookup took 1.5 seconds to look up from a remote API? Is ETS still the right choice?
Based on your statement, it sounds like you wouldn't since that's a long time (relative to a 25ms response). But if ETS is meant to be used as a cache, wouldn't that defeat the purpose of what it's meant to be used for?
Like, if I wanted to cache a PostgreSQL query that took 1 second to finish. Isn't ETS the primary place for such a thing? But 1 second is a long execution time. I know Cachex (the Elixir lib) uses ETS to cache things, so now I'm wondering if I've been using it for the wrong thing (caching database calls, API call results, etc.).
Normally in Python or Ruby I would have cached those things in Redis and lookups are on the order of microseconds when Redis is running on the same $5 / month VPS as my web server. It's also quite speedy over a local network connection too for a multi-server deploy. Even with a distributed data store in Elixir, you'd hit the same network overheard right?
> if I need distributed state, I just use to the database too.
This part throws me off because I remember hearing various things in Phoenix work in a distributed fashion without needing Redis.
I would use ETS to cache local lookups (for all cores in the same node). Then fallback to Redis to populate the ETS cache. But again, feel free to skip one of ETS or Redis. The point is that ETS adds a different tool you may (or may not) use.
> Like, if I wanted to cache a PostgreSQL query that took 1 second to finish. Isn't ETS the primary place for such a thing?
Here is the math you need to consider. Let's say you have M machines with N cores each. Then remember that:
1. ETS is local lookup
2. Redis is distributed lookup
If you cache the data in memory in Ruby/Python, you will have to request this data in PostgreSQL M * N times to fill in all of the caches, one per core per node. Given the amount of queries, I will most likely resort to Redis.
In Elixir, if you store the data in ETS, which is shared across all cores, you will have to do only M lookups. If I am running two or three nodes in production, then I am not going to bother to run Redis because having two or three different machines populating their own cache is not an issue I would worry about.
> > if I need distributed state, I just use to the database too.
Apologies, I meant to say "persistent distributed state" as not all distributed state is equal. For ephemeral distributed state, like Phoenix Presence and Phoenix PubSub, there is no need for storage, as they are about what is happening on the cluster right now.
I wouldn't cache database queries unless the query is expensive, or the results rarely change but are frequently accessed.
Generally though, whether to store something in ETS or not is situational - your best bet is actually measuring things and moving stuff into ETS later when you've identified the areas where it will actually make a meaningful difference.
> This part throws me off because I remember hearing various things in Phoenix work in a distributed fashion without needing Redis.
This is true, but it depends on what kind of consistency model you need for that distributed state. The data you are referring to (I believe) is for Phoenix Presence, and is perfectly fine with an eventually consistent model. If you need stronger guarantees than that, you'll need a different solution than the one used by Phoenix - and for most things that require strong consistency, its better to rely on the database to provide that for you, rather than reinvent the wheel yourself. There are exceptions to that rule, but for most situations, it just doesn't make sense to avoid hitting the database if you already have one. For use cases that would normally use ETS, but can't due to distribution, Mnesia is an option, but it has its own set of caveats (as does any distributed data store), so its important to evaluate them against the requirements your system has.
GenServer is immaterial - it's the single process that can be a bottleneck. Just like a single thread can be in other languages. If you need multiple threads in other languages, you'll need multiple processes in Erlang. The nice thing here being the cost of spinning up more (even one per request) is negligible, the syntax is trivial, and the error model in the event something goes wrong is powerful, whereas threads in other languages are so heavy weight you have to size the pool, they can be complicated to work with, and if things go wrong your error handling options tend to be limited.
Use ETS in the places you'd use in memory caching. That's it. That's what it's meant for. If you need distributed consistency, it's not the right answer. If you need a single source of truth, it's not the right answer. But if you need a local cache, that does not require strong consistency(and that's very, very common in distributed systems), it works great.
Do you have an example of the syntax?
Basically, how would you wire things up so it becomes difficult or maybe even impossible to shoot yourself in the foot with GenServer bottlenecks?
Also if you had to guess, why would the blog post author of the post I link not do this trivial syntax adjustment instead of rewriting everything to use ETS?
Unfortunately posts like theirs are what comes up first when you Google for things like GenServer vs ETS.
The intuition I've built so far is that holding state in a process (GenServer being an abstraction around a process) is reserved for stuff like "The current state of a games table" where you would have one process per game being played. This state is only being read and manipulated by itself and will be thrown away when the game is over (which might make it sound like an object!). If you suddenly had a requirement to show live stats from thousands of games being played, then one option would be to start sending it to ETS.
One big difference to point out is that to store state in a process you need to use the language's data structures (lists, maps, keyword lists) which can get slow when they grow HUGE. ETS is an actual key-value store with incredibly fast lookup (and can easily be read by multiple processes).
I hope that makes sense—I'm also testing my own knowledge here :)
And I think potentially even more sense if Live View is just creating a GenServer under the hood.
This would explain why if you had 1,000 people connected to your site through Live View, you would have 1,000 stateful processes running on your server. That would be 1 GenServer for each connected client, each in their own universe with their own un-shared data / state.
Also yes, LiveView does indeed have one process for each one! The GenServer API is available in your LiveView modules.
- Be wary of places where you have N:1 process dependencies, where N is large and the number of messages exchanged between each member of N and the single process are frequent/numerous. Since each process can only handle received messages sequentially, there is little point in spawning a lot of tasks in parallel if each process has to talk to the same upstream process to do anything
- Set up telemetry that samples the number of messages sitting in the process mailbox; if a process is becoming a bottleneck, it is going to be frequently overloaded and have a lot of messages in its mailbox. If you have the telemetry, you can see when this starts to happen, and take steps to deal with it before it starts causing problems for you. Likewise, its probably useful in general to have telemetry on how long each unit of work takes in server-like processes, so you can get a sense of throughput and factor that data into your design.
- Avoid sending large messages between processes, instead spawn a process to hold the data and then send a function to that process which operates on the data and returns only the result; or store the data in ETS if you have a lot of concurrent consumers. It can also be helpful to denormalize the data when you store it in ETS so you can access specific parts of it without copying the entire object out of ETS on every access. The goal here is to make messaging cheap and avoid copying lots of data around.
- Take steps to ensure process dependencies in your design are structured as trees, i.e. avoid dependency graphs that can contain cycles. It is all too easy for a change to introduce the possibility of deadlock if you play fast and loose with what processes can talk to each other. If your process dependencies mirror your supervisor tree, then you can protect against this by only allowing dependencies between branches in one direction (usually toward the parts of the tree that were started earlier in the supervisor tree)
I think the problem is that Elixir is still relatively young, and due to the language evolution and the lack of established documented doctrine from the Erlang community, there is a lot of techniques, tips, design patterns, etc., that are being rediscovered; likewise there are a lot of seemingly good ideas that turn out to be not so great in practice, but are encountered on the road to the truly sound patterns. So you get a lot of people writing about the lessons they are learning, and because of the gaps in knowledge, the result is that the information may be missing things, or providing a more complex solution when there is a simpler one, etc. Ultimately this is an important process, and now that Elixir has largely stabilized, this will only improve (and its is already pretty good, certainly far better than when I first started with the language years ago).
Do you have any code examples or practical applications on how to apply most of those bullet points?
Those are all very daunting things to approach without examples but they sound very important.
In Python or Ruby I would have just throw things into Redis as needed without thinking about it and this hasn't failed yet in years of development time with tens of millions of events processed (over time). Send ID of DB object to worker, let the worker library deal with it, look up the ID in the DB when the work is currently being done, let worker library deal with the rest and move onto the next thing.
And for caching, it's just a matter of decorating a function or wrapping some lines of code to say it should be cached and everything works the same with 1 or 10 web servers when the state is saved into Redis (major web frameworks in Python and Ruby support this with 1 line of configuration).
Where you could hit a bottleneck might be in the background processing though, take for example the following scenario:
- A pool of N background job worker processes each pull an item off a queue, and spawn a process to perform the task in isolation - A singleton process S provides exclusive access to some resource - Each task calls some code which needs to interact with the resource controlled by S.
The problem with the above is that all of that concurrency/parallelism is nullified by the fact that the tasks are all going to block on S to do their work, the bottleneck of the design.
To be clear, you should always gather telemetry first, but lets assume that you've gathered that and you can clearly see that this bottleneck is an issue (the process mailbox has frequently got many messages waiting to be received, the average time to completion for jobs is increasing). To solve this depends on why the resource is held by S in the first place.
If its because the resource is not thread safe and requires exclusive access, then unless you can find a way to avoid needing the resource in every task, there isn't much you can do, but this should be fairly uncommon in practice.
If S exists because you needed to store some shared state somewhere, and someone told you that an Agent or GenServer was the way to go, then you could move that data to ETS and make it publically accessible so that functions which operate on that data read it from ETS directly rather than call the process. Now you've removed that bottleneck entirely.
If S exists because it needs to protect access to some data, but not all of it, and most tasks don't need to access the protected data, then you can move the parts that do not need to be protected into ETS, and keep the rest in the process. This might reduce the amount of contention on that singleton process by a huge amount, but if even half the processes no longer need to block on accessing it, then you've regained at least that much concurrency in the task processing code.
---
The example above is something I've seen numerous times, but the important pattern to note is that you have some task that you've tried to parallelize by spawning multiple processes, but that task itself depends on something that is not, or cannot be done concurrently/in parallel.
Any time this pattern arises, you need to either find a way to enable concurrency in that dependency, or you should avoid doing the task in parallel in the first place. This is ultimately true of any parallelizable task - its only parallelizable if all of the tasks dependencies are themselves parallelizable, otherwise you end up bottlenecked on those dependencies and you've gained little to no benefit.
Where it becomes a bigger problem is when you consider the system at a higher level. Bottlenecks reduce throughput, which may end up, via backpressure, causing errors on the client due to overload, or depending on the domain, data being dropped because it can't be handled in time (e.g. soft real-time systems).
I don't have any code examples that really encompass all of this in one place, if you are interested in something specific, I can try to throw something together for you. Or if you have specific questions I can point you to some resources I've used to help understand some of these concepts.
The danger is using a single process of the GenServer instead of multiple, so you can get single process bottleneck that won’t use multiple cores. You don’t have to know any intricacies of BEAM or OTP to know and design around using multiple process instances of the GenServer.
> I know you can do distributed state in Elixir too, but it doesn't seem as easy as it is in other languages.
You can use Redis in Elixir as well. Saying that Elixir is worse at distribution than Python/Ruby because ETS isn’t distributed is a bit like saying Python is bad at distribution because objects are not distributed. It’s especially strange since Elixir ships with a distribution system (so you can access ETS from other machines) while your other example languages do not.
Totally but a lot of folks say "but Elixir is good / easier because you don't need tools like Redis". But then when you try to do it without Redis you need to account for many things yourself and end up re-inventing the wheel. This is time spent developing library'ish code instead of your business logic.
It's sort like background job processing tools. Sure you can do everything in Elixir, but when you want uniqueness guarantees, queues, priorities, exponential back-off retries, periodic tasks, etc. you end up using a dedicated background processing tool in the end because it's very difficult and time consuming to write all of that from scratch.
But in practice almost all of those things end up being a requirement in a production grade system. You end up in the same boat as Python and Ruby.
Well, there is definitely a spectrum. For example, in some communities you will hear the saying "you cannot never block the main thread". So if you want to do a file export? You need to move to a background worker and send the export to an e-mail. Sometimes sending the e-mails themselves is done in the background while it could be done on the go.
Languages like Java, Go, Erlang, Elixir, will be just fine with performing those tasks during the request, so you don't need to push those particular actions to a background job. And as you noted in the other comment, Phoenix ships with PubSub and Presence that work out of the box without a need for an external tool too.
But if you need a uniqueness, queues, back-off, etc, then do use a background job tool! But some of the places you thought you needed one, you may not actually need it.
distribution, for example, is a much lauded feature of elixir/erlang but if you look into the implementation it's really just a persistent tcp connection with a function that evals code it's sent on the other end. you could easily write the equivalent in ruby or python or java but you probably wouldn't because it's not actually a very good idea. there's no security model, the persistent connections won't scale past a modest cluster size and the whole design ignores 30 years of experience in designing machine to machine protocols
similarly, people will mention mnesia or ets as a replacement for a database or redis. these are both very crude key/value stores with only very limited query capabilities. you should use them where you would use an in process cache in another language and not as a replacement for an out of process shared cache. they were never designed as such. and as in process caches they are really nothing special
in fact, a lot of elixir's marketing comes down to "do more with less" with a lot of focus on how you can do on a single machine what other languages take whole clusters to do. this is (partially) true. elixir/erlang are excellent for software that runs on appliance style hardware where you can't simply add machines to a cluster. it is, in fact, what erlang was designed to do. what this ignores though is that this is a terrible model for a service exposed over the internet that can run on any arbitrary machine in any data center you want. no one will advise you to run a single vm in aws or a single dyno on heroku for anything that matters.
elixir/erlang's features that increase it's reliability on a single machine are a cost you pay not an added benefit. the message passing actor model erlang built it's supervision tree features around are a set of restrictions that are imposed so you can build more reliable stateful services on machines that don't have access to more conventional approaches to reliability (like being stateless and pushing state out to purpose built reliable stores)
if you're building systems that need to run in isolation or can't avoid being stateful then perhaps elixir/erlang has some features that may be of interest. the idea that these features are appropriate for a totally standard http api running in aws or digital ocean or whatever backed by a postgres database and a memcache/redis cluster is not really born out by reality however. if it were surely other languages would have incorporated these features by now? they've been around for 30 years and the complexity (particularly of distribution and ets) is low enough you could probably implement them in a weekend
I mean, this is just straight up incorrect. Yes the underlying transport is TCP, but using remote evaluation is definitely _not_ the common case. Messages sent between nodes are handled by the virtual machine just like messages sent locally, that is the main benefit of distributed Erlang - referential transparency. Yes, you _can_ evaluate code on a remote node, which can come in handy for troubleshooting or orchestration, but it is certainly not the default mode of operation.
> there's no security model
I mean, there is, but it isn't a rich one. If one node in the cluster is compromised, the cluster is compromised, but the distribution channel is very unlikely to be the means by which the initial compromise happens if you've taken even the most basic precautions with its configuration. It would be nice to be able to tightly control what a given node will allow to be sent to it from other nodes (i.e. disallow remote eval, only allow messaging to specific processes), and I don't think there are any fundamental blockers, its just not been considered a significant enough issue to draw contribution on that front.
> the persistent connections won't scale past a modest cluster size
I mean, there is already at least one alternative in the community for doing distribution with large clusters, Partisan in particular is what I'm thinking of.
> these are both very crude key/value stores with only very limited query capabilities
What? You can literally query ETS with an arbitrary function, you are limited only by your ability to write a function to express what you want to query.
You shouldn't use them in place of a database, but they are hardly crude or primitive.
> elixir/erlang are excellent for software that runs on appliance style hardware where you can't simply add machines to a cluster. it is, in fact, what erlang was designed to do. what this ignores though is that this is a terrible model for a service exposed over the internet that can run on any arbitrary machine in any data center you want
I think you are misconstruing the point of "doing more with less" - the point isn't that you only need to run a single node, but that the _total number of nodes_ you need to run are a fraction of those for other platforms. There are plenty of stories of companies replacing large clusters with a couple Erlang/Elixir nodes. Scaling them is also trivial, since scaling horizontally past 2 nodes doesn't require any fundamental refactoring. Switching from something designed to run standalone in parallel with a bunch of nodes versus distributed _does_ require different architectural choices, and could require significant refactoring, but making that jump would require significant changes in any language, as it is a fundamentally different approach.
> elixir/erlang's features that increase it's reliability on a single machine are a cost you pay not an added benefit. the message passing actor model erlang built it's supervision tree features around are a set of restrictions that are imposed so you can build more reliable stateful services on machines that don't have access to more conventional approaches to reliability (like being stateless and pushing state out to purpose built reliable stores)
I'm not sure how you arrived at the idea that you can't build stateless servers with Erlang/Elixir, you obviously can, there are no restrictions in place that prevent that. Supervisors are certainly not imposing any constraints that would make that more difficult.
The benefits of supervision are entirely about _handling failure_, i.e. resiliency and recovery. Supervision allows you to handle failure by restarting the components of the system affected by a fault from a clean slate, while letting the rest of the system continue to do useful work. This applies to stateless systems as much as stateful ones, though the benefits are more significant to stateful systems.
> the idea that these features are appropriate for a totally standard http api running in aws or digital ocean or whatever backed by a postgres database and a memcache/redis cluster is not really born out by reality however. if it were surely other languages would have incorporated these features by now? they've been around for 30 years and the complexity (particularly of distribution and ets) is low enough you could probably implement them in a weekend
The reason why these features don't make an appearance in other languages (which they do to a certain extent, e.g. Akka/Quasar for the JVM which provide actors, Pony which features an actor model, libraries like Actix for Rust which try to provide similar functionality as Erlang) is that without the language being built around them from the ground up, they lose their effectiveness. Supervision works best when the entire system is supervised, and supervision without processes/actors/green threads provides no meaningful unit of execution around which to structure the supervision tree. Supervision itself is built on fundamental features provided by the BEAM virtual machine (namely links/monitors, and the fact that exceptions are implemented in such a way that unhandled exceptions get translated into process exits and thus can be handled like any other exit). The entire virtual machine and language is essentially designed around making processes, messaging, and error handling cohesive and efficient. Could other languages provide some of this? Probably, though it certainly isn't something that could be done in a weekend. No language can provide it at the same level of integration and quality without essentially being designed around it from the start though, and ultimately that's why we aren't seeing it added to languages after the fact.
when i say there's no security model i mean there's no internal security model. you can impose network based security (restricting what nodes can connect to epmd/other nodes) or use the cookie based security (a bad idea) or you can even implement your own carrier that uses some other authentication (i believe there a few examples of this in the wild) but the default is that any node that can successfully make a connection has full priveleges
as for ETS, you can query any data structure with arbitrary functions. that's exactly what i mean when i say there's limited query capabilities. all you can really do is read the keys and values and pass them to functions
my experience and the experience of others is that elixir and erlang are not significantly more efficient than other languages and do not lead to a reduction in the total number of nodes you need to run. whatsapp is frequently cited as an example of "doing more with less" but it's compared to bloated and inefficient implementations of the same idea and not with other successful implementations. facebook certainly wasn't using thousands of mq brokers to power facebook chat. no one is replacing hundreds of activemq brokers with a small number of rabbitmq brokers
you can absolutely build stateless servers with erlang/elixir (and you should! stateless is just better for the way we deploy and operate modern networked services). my point is that many of the "advantages" of elixir/erlang are not applicable if you are delivering stateless services
when i said you could deliver erlang/elixir features in a weekend, i did not mean all of them. i meant specifically distribution and ets. you are right that the actor model, supervision trees and immutable copy-on-write data structures are all necessary for the full elixir/erlang experience. i generally like that experience and think it is a nice model for programs. i don't think however it is very applicable to writing http apis. java, rust, python, go, ruby and basically every other language are also great at delivering http apis and they don't have these same features
They are not equivalent at all, sending a message is sending data, evaluation is execution of arbitrary code. BEAM does not implement send/2 using eval. Spawning a process on a remote node only involves eval if you spawn a fun, but spawning an MFA is not eval, it’s executing code already defined on that node.
> as for ETS, you can query any data structure with arbitrary functions. that's exactly what i mean when i say there's limited query capabilities. all you can really do is read the keys and values and pass them to functions
You misunderstood, you can _query_ with arbitrary functions, not read some data and then traverse it like a regular data structure (obviously you can do that too).
> my experience and the experience of others is that elixir and erlang are not significantly more efficient than other languages and do not lead to a reduction in the total number of nodes you need to run.
I’m not sure what your experience is with Erlang or Elixir, but you seem to have some significant misconceptions about their implementation and capabilities. I’ve been working with both professionally for 5 years and casually for almost double that, and my take is significantly more nuanced than that. Neither are a silver bullet or magic, but they excel in the domains where concurrency and fault tolerance are the dominant priorities, and they are both very productive languages to work in. They have their weak points, as all languages do, language design is fundamentally about trade offs, and these two are no different.
If all you are building are stateless HTTP APIs, then yes, there are loads of equally capable languages for that, but Elixir is certainly pleasant and capable for the task, so it’s not really meaningful to make that statement. Using that as the baseline for evaluating languages isn’t particularly useful either - it’s essentially the bare minimum requirement of any general purpose language.
i used erlang for 4 years professionally and elixir for parts of 5. i think both are good, useful languages. i just take issue with the misrepresentation of their features as something unique to erlang/elixir
advocates should talk up pattern matching, supervision trees and copy-on-write data structures imo. those are where erlang and elixir really shine. instead they overhype the distribution system, ets, the actor model and tools like dialyzer which are all bound to disappoint anyone who seriously investigates them
Not really. The distribution can only call code that exists in the other node. So while the system can be used as if it was an evaluator, it is not because of its primitives, but rather due to functionality that was built on top. If you nuke "erl_eval" out of the system, then the evaluation capabilities are gone.
I agree it is a thin line to draw but the point is that any message passing system can become an evaluator if you implement an evaluator alongside the message passing system. :)
> elixir/erlang's features that increase it's reliability on a single machine are a cost you pay not an added benefit
Agreed! Erlang/Elixir features should not be used to increase the reliability on a single machine. Rather, they can be used to make the most use of individual machines, allowing you to reduce operational complexity in some places.
There is a mostly API-compatible distributed version of ETS in OTP, called DETS. And a higher-level distributed database built on top of ETS/DETS called Mnesia, again, in OTP. So, no, you aren't.
> I know you can do distributed state in Elixir too, but it doesn't seem as easy as it is in other languages. And it's especially more complicated / less pragmatic than other tech stacks because almost every other tech stack all use the same tools to share external state so it's a super documented and well thought out problem.
You can use the same external tools in Elixir as on platforms that don't have a full distributed database built in as it is in the OTP, so, I don't see how the fact that those external tools are widely used on other platforms makes Elixir harder.
DETS is the disk based term storage, it is as distributed as ETS.
Can you explain something more about why you think this is not a good thing?
Is it like Laravel (PHP)? Lots of magic so it is very easy to get started but a hell to maintain?
I really like to get started with Elixir (for webdev) but I am not sure if I should start with Phoenix.
1 - How ok are you with just accepting things as documenting, like having to put `use HelloWeb, :controller` in each controller. Personally, I _had_ to understand what this was doing and how, but I imagine some people don't care so much.
2 - Do you need / want to do anything outside what's an idiomatic Phoenix. As a simple example, I think Plug.Parser is wrong to raise an exception when invalid json is given to it (shitty user data is hardly "exceptional" and I don't need error logs with poor signal to noise).
You should check the `:plug_status` field of the exception and only log/report the error if it's in the 5xx range.
Re: 2, your entire plug chain is accessible and configurable if you don’t like the defaults. I’ve never tried to modify the specific behaviour you’re referring to, but my other experience with modifying plus behaviour has generally been pretty smooth.
It makes for great demos but it basically means “this just works so you don’t need to know how it works”.
The problem is you always need to know how your tools work, often sooner rather than later, as you inevitably end up with edge cases and bugs in your application that simply can’t be resolved without that understanding.
only because we reserve the use of the word "magic" for the times we find it bad/unsuccesful.
When it's good/successful, we call it "powerful, mature, and robust high-level abstractions and APIs".
That "magic" is always bad is just a syllogism because a value judgement on the success of the abstractions or APIs is built into the term as used. Abstractions that are confusing, or unexpected, or inconsistent internally or with the platform they live on, or especially leaky or buggy, or in practice hard for beginner developer-users to develop proper mental models to guide use around and to understand how to debug the use of -- get called "magic". But nobody set out to provide such abstractions, they set out to provide nice, polished, predictable, simple to understand, consistent, powerful tools at the right level of abstraction for the domain -- when someone thinks they've failed, with abstractions at a fairly high level or higher than the speaker thinks is appropriate for the domain -- they call it 'magic'.
"Magic" is not something the developer doesn't understand, it's something the developer cannot possibly understand, because you didn't document it properly (or at all).
Nope. "magic" is poor execution of something, one way or another. It's a category of something done unsuccesfully, not some kind of difference in kind of thing.
This matters for how we talk about 'magic', like, you shouldn't have put all that magic in there, did you think we want magic? No, nobody wants something poorly documented and nobody thinks they do. Nobody sets out to make something unsuccessful, "magic" is a failure.
Is it magic when it does something convenient for you based on conventions without you writing the steps? As long as 'magic' is clearly explained in the docs I don't see how it's different from a layer of abstraction like a compiler or a car.
There is no line; that's the definition. (I'm agreeing with you here, btw) It's a blub issue; if one doesn't understand it, it's derisively called "magic". So you don't get this argument because it's totally based on the observer.
I use Django at work and I hate it because it's magic all over the place. And I recognize that me saying it is simply I haven't had the time or impetus to go figure out the stuff I don't have figured out yet, so I can bucketize all that as "magic".
I've had a similar experience in Ruby where metaprogramming was used in favor of objects with functions. When I had issues, it was very hard to debug.
Maybe it comes down to the quality of the abstraction. I think the point of a good abstraction is you don't have to go down to the layer below.
Or maybe this is a board full of programmers who love to fret over unimportant minutia.
The idea has a well understood meaning, just use that meaning. Yes, maybe somewhere on the edges it's fuzzy, learn to live with fuzzy.
If we are debating an API design and a colleague says "it's too much magic", is that bad thing? If I write deliverEmail(email) and the e-mail "magically" gets sent, what's wrong with that?
On the other hand, if we debate the abstraction itself--for example behavior doesn't match user expectations--then we can do better at improving the API design.
A car is not magic. It does exactly what it purports to do and nothing else. David Blaine levitating is magic. It works in a very limited set of circumstances and no more. It works to amuse some spectators. It doesn't work to, for example, cross the Hudson River, despite outward appearances.
Similarly, in programming we use "magic" to describe something that seems to defy what the underpinning (language, framework, etc.) would allow. We know that it's using some trick to pull off what it's doing, and that trick likely has constraints that won't allow the magic to ever become a true abstraction.
Note that I'm not calling Phoenix magic as I haven't used it enough to know
For example, I actually do understand http, but doing anything not explicitly designed for by the framework of choice can be painful. It's not that I don't understand the underlying tech, it's that the "magic" is expecting a specific use case that I don't want to do.
Also applies to scenarios where a developer does not understand something not because they don't want to, but because the vendor has actively tried to obscure how it works and is asking you to take it as a given that it will always just work "by magic" for every use case, which invariably nothing ever does.
When do you learn about the JVM in a Java environment? At what point should you dig into be Ruby source code?
At some point you have to treat the abstraction below you as a black box.
I've personally found it helpful to accept the black box and only dig in after I've gained more experience or at the point I feel like a lack of understanding is getting in my way.
When the abstraction can be considered 100% leak-proof in practical terms. Also, if you have alternative tools when you encounter a case where "X" doesn't fit, you do not need to learn about "X" as long as you have "Y".
But when a solution is presented as "this must always be used", the solution should have been around for a long time with a track record of never breaking. If that's where the abstraction is, it's usually solid enough to not go any deeper.
In my experience, most developers don't talk about "magic" as if it's the same thing as "abstraction". "Magic" usually means an abstraction that leaks under non-trivial conditions.
The only thing I couldn't figure out how to override was streaming uploads.
But this describes all manner of legitimately useful features. Garbage-collection, for instance, or dynamic-dispatch, or closures, or await/async.
As technicolorwhat and vendiddy already said, it's also poorly defined. In languages like Haskell, or SMT-solving languages, the programmer might know very little about how the language is implemented.
> you inevitably end up with edge cases and bugs in your application that simply can’t be resolved without that understanding
You can run into trouble if you have an incomplete understanding of your programming language, but this isn't particular to 'magic' abstractions.
You'll lose some things (no built-in auth but there's libs) and win some (easy websockets and pub/sub).
Also LiveView is fun.
Currently I am working with Blazor (C#) and I really like that you can forget about Javascript (most of the time). I believe Blazor, LiveView and others are the future for webdev.
See below (jswny) for more clarification.
I use both dialyxir and Credo in my projects
Most libraries and frameworks I’ve used enforce things on your modules via behaviors which have nothing to do with naming. Perhaps you are talking about naming functions for behaviors? Because I think it’s the same in most languages that “overridden” or “implementations” of functions for modules/classes that use some kind of interface must be named the same as the interface definition.
Yes. In typescript, you can have abstract classes or interfaces to implement which gives you more information (intellisense, visible errors, type information) and make the contract more explicit. I was trying out phoenix and I didn't get any hints about the methods to implement in the module. I was relying on documentation for that. If there was a way to say use @some_spec and get information about it in the editor. It would have helped out.
There's also Dialyzer (and various IDE integrations) for some level of static type-checking.
elixir people really should learn erlang.
if you ever saw a coffeescript person struggling to do things that are straightforward in js, because they didn't bother to learn js? that's what you're doing now
If you learn the base language, you get more powerful in the descendant language as well
Only you DO use languages that end up as machine code, and the idea of understanding machine code to better utilize something like C isn't outlandish.
No need for extra build tools, FFI with the platform language, additional debugging tools, the urge to create wrapper libraries that feel more idiomatic into the guest language, waiting for the guest language to come up with ways to expose new platform capabilities,...
Guest languages are good for experimenting new ideas that eventually get adopted by the platform and then the world moves on, and they turn into the next CoffeeScript.
In erlang these are called "behaviours"
.
> I was relying on documentation for that
Consider reading the Erlang documentation. The Elixir people really don't understand Erlang very well, and are trying to manufacture Ruby on top of a language that already gives them solutions to their problems that they just can't see.
.
> If there was a way to say use @some_spec and get information about it in the editor. It would have helped out.
Basically every IDE that speaks Erlang knows behaviors. They're as fundamental to the language as header files are to C/C++.
Elixir tried to "do away with them" because Elixir thinks they're "confusing" but they're actually really important tools.
That's completely inaccurate. They are an important part of Elixir and they are fully documented.
Elixir even has annotations that allow you to say "this function maps to a behaviour's callback". For example:
@behaviour GenServer
@impl true
def handle_info(msg, state)
So we do support behaviours and we have added more static guarantees compared to what you would find in Erlang.
> The Elixir people really don't understand Erlang very well
It is a bit depressing it has come to this but my long list of contributions to Erlang/OTP through pull requests, discussions, EEPs, etc say otherwise.
Including Joe Armstrong himself... https://joearms.github.io/published/2013-05-31-a-week-with-e... ^_^
You don’t have to go to Erlang documentation to learn about them since they are documented in Elixir as well [1] [2].
[1] https://elixir-lang.org/getting-started/typespecs-and-behavi... [2] https://elixir-lang.org/getting-started/typespecs-and-behavi...
As soon as I add @behaviour to a module, I get IDE autocomplete and compiler warnings if a function has not been implemented.
I do agree that I have not seen a lot of behaviour use in the wild, even in cases where it's clear that they would be beneficial. Elixir developers seem to have some blindspots.
It is exactly what I was looking for. Thanks for mentioning it.
I am looking at the answers from this stackoverflow question: https://stackoverflow.com/questions/1483550/in-erlang-what-a... and the Erlang docs.
I don't get the impression that they are comparable.
> Note that using the Process Dictionary:
> - Destroys referencial transparency
> - Makes debugging difficult
> - Survives Catch/Throw
> So:
> - Use with care
> - Do not over use - try the clean version first
Seasoned Erlang developer here too (10+ years).
First of all, the original post wanted to share data across requests. Given each request is typically handled in separate process, you simply cannot use the process dictionary to share this data. So your original comment completely missed the mark.
Second of all, I would say most of the Erlang community actually agrees with the Erlang official documentation linked above. For example, for a long time, Cowboy had this in its README:
> No parameterized module. No process dictionary. Clean Erlang code.
And the reason is simple: the process dictionary is about state within a single process. This data cannot be easily shared (you can with erlang:process_info/2 but that adds a process lock). Therefore, most times people resort to the process dictionary is to pass data within a process, implicitly, and passing it functionally (the clean way) is often better. Using ets for the same purpose would also be frowned upon. The process dictionary should only be used almost exclusively for metadata.
You probably won't be inclined to take my word for this, so here is a proposal: send a pull request to remove or update the linked notes above from Erlang/OTP docs and see how well it will be received.
How is it for performance as compared to
- Hand written single-threaded C - Python code stringing numpy call together.
(I know these are very different things). Whenever I dip into concurrency for performance I always feel like things could be so much better.
Erlang/Elixir would be fine as an orchestration language calling out to C code to actually do the number crunching, but you likely won't see much of the benefit of the language doing that unless you're doing something more complicated in that layer. If it's a super thin layer of glue, Python is better suited (not least because Numpy is familiar and meant to work with the language)
Rust certainly seems quite interesting, but also quite daunting.
I can’t tell if it’s trolling, FUD or just ignorance blinded by arrogance.
I don't think anyone would take issue with your comments if that's how you approached it.
Instead you're being an asshole, and for no reason I can understand. It's not like we're talking politics here...
I got voted into the floor for saying "you're a better elixir programmer if you also learn erlang," which is a simple fact.
I do think this. I think that most micro-language communities are bizarrely intolerant of criticism, and unwilling to cope with the idea that every language has problems and that you can only get good at a language if you face them with open eyes.
Imagine a C programmer getting offended if you warned someone learning C "by the way, if you're doing string parsing, you're gonna have a bad time."
.
> Instead you're being an asshole
I love how I've given measurable facts and technical claims, and in order to resist, the members of this community are trotting out swearing and insults, then imagining that they've made a valuable case
.
> and for no reason I can understand.
This we agree on
I don't think that's the case. As an active participant in the various Elixir communities, I've found that the general advice is that learning Erlang, while not necessary, is definitely worth it. Many of the community 'members' do or did plenty of Erlang programming, and the stories of all the 'goodies' and how much simpler Erlang can be at times have more than once gotten me to play around with it. I generally find very little hostility between Elixir and Erlang programmers. Far from it.
I also strongly contest that you're just saying "you're a better elixir programmer if you also learn erlang". At best, some of your comments approach the sentiment, but then end with judging the commenter a blub programmer, or something similar.
I can understand if maybe you've had bad experiences with 'micro-language communities' and you're reading that into what Elixir programmers are saying, but by and large I very rarely come across Elixir programmers who thumb their nose at Erlang. More than anything the sentiment is a kind of reverence and maybe even a degree of 'embarrassment' over finding it hard to get over Erlang syntax.
> I do think this. I think that most micro-language communities are bizarrely intolerant of criticism, and unwilling to cope with the idea that every language has problems and that you can only get good at a language if you face them with open eyes.
That's perhaps true, but Elixir is among the few communities where I find much less of that than elsewhere. The forums and slack are rife with day-to-day Elixir/Erlang programmers who lament the lack of static typing, argue about how Phoenix is too magical, and so on. One reason I was drawn to Elixir was precisely the relative lack of "this is the best thing ever and everything else sucks" sentiment that I find in so many other programming language communities.
I was assuming you were just trolling, because it's hard for me to believe that you don't see the discrepancy between what you say you are saying, and what you're actually saying. It's very discrepid!
That includes not tossing in swipes like "This just makes me laugh" and so on. Your comments have been provocative to the point of being trollish, as well as outright nasty in places. That's not cool, regardless of how much you know about Erlang.
Can you imagine responding to "everyone dies" with "please post specifics, thanks"?
From what I can see, this poster stated that no programming language is perfect, and I'm struggling to see why they need to back that up.
It had criticisms without any explanations.
If someone makes a statement I'm skeptical of, I want to push for examples.
Notice the ram usage of the machine go to the moon.
Then there's this thing about immutability being guaranteed by processes. No it isn't, though? You just copy the data and operate on a copy (or a fragment, sometimes, in the case of binaries.) The original can be mutated, and if you've scaled anything in erlang, you've seen this cause a bunch of problems around the array module
Erlang isn't actor model (sometimes you hear it called pi calculus, which it also isn't.) These are things that seem casually, superficially similar to what Erlang is, but then, the whole point of the actor model and the pi calculus is to give mathematical rigor to guarantees that Erlang doesn't offer, and then you've confused yourself into thinking that Erlang has properties it doesn't actually have.
Engineering by failed metaphor is tremendously dangerous, because it makes people feel safe in ways they aren't, and just due to the nature of systems, the way you find out is often under terrible pressure and non-rarely unfixable.
These "everybody knows" beliefs are really fucking bad. REALLY bad. You can see some examples here: https://aphyr.com/posts/288-the-network-is-reliable
.
"not ever worrying about or messing up concurrency"
Like. How new do you have to do be to Erlang and Elixir to think that message passing means you can't mess up concurrency?
If you ask a C programmer to name some threading problems, five dollars says one of the first three they name is some variation on lock contention.
Messaging does not solve this. (Arguably it makes it worse, since every blocking process is now effectively a lock, which is a generally discouraged design strategy in most languages but encouraged as a primitive in this language.)
Indeed, just as OO problems got more common in C++ vs C because of how easy OO became to do, and how frequent it became, I actually end up seeing a lot more concurrency related problems in Erlang than in most languages, just because there's a lot more concurrency than in most languages.
And this isn't be complaining about erlang; I love it. But also, if you've just upgraded from bows to rifles, you have to understand that with that improvement will also come an increase in accidental bullet injuries.
Saying you'll never have concurrency problems in erlang is like saying you'll never have allocation problems in C++. It suggests that everything you know about the language is reading chapter three's discussion about how tools are made available to mitigate this (in C++'s example it'd probably be a SFINAE discussion or something naive about automatic deletion, or if it's an old enough document, maybe smart pointers.)
That that tooling exists suggests there's an active problem being actively mitigated, not that the problem is solved in blub and not everywhere else. There's a reason you don't see cut outside prolog.
In order to abuse one of my favorite community metaphors, "the reason footguns exist is that sometimes you actually do need to shoot lots of animal at once; don't take it out of the case unless Dracula shows up, but, in the balance, eventually you'll be glad you know [Burt Gummer](https://www.youtube.com/watch?v=SdwFp2D5Eg8)."
.
> Then you'll wan to do something distributed, and everything you've learnt about single-node development becomes applicable to distributed systems.
This just makes me laugh, sadly.
"Because I passed messages in a message passing language, I'm now ready to write distributed systems" is just the most delightfully naïve claim
There are red flags. Red flags are a thing that exists.
And the thing is, you see them all the time in places you shouldn't. [This is correct](https://martin.kleppmann.com/2015/05/11/please-stop-calling-...) and yet my trainers at two FAANG companies claimed that they had internally solved this at large scale within the company, on down from on high from the biggest people in the space
You see fucking Oracle get that wrong
Like the very first three things you ask about distributed systems are:
1. how well does it tolerate partitions, 2. how well does it perform over a WAN, and 3. did you design for both city loss and provider loss
With erlang, the traditional answers are
1. no-ish (that is, only just barely and by throwing away everything you used to love about the language to get a hokey election system that barely works and seems designed for the explicit purpose of causing merge failures (it's the CVS of rejoins, i swear to your god) ) 2. (loud, plaintive sobbing) 3. we stood up three copies of the system, does that count? what do you mean transactional
The people saying that Erlang solves distributed systems are the people who stood up an ejabberd instance on their desktop and one on their laptop and declared themselves the vice president of MS Azure Engineering
like yes, it does get rid of a few problems, and it has some convneiences, but
what would you do if someone told you haskell solved all logic errors? some offered responses include:
1. believe them, drop everything, and make all life a burrito 2. say "wow, this person isn't very good at haskell, are they?"
.
> The synergy between the runtime, standard library and language, backed by the actor model + immutability is a huge productivity win. It's significantly different (to a point where I think it's way more accurate to group Ruby with Go than with Elixir), but, as I've tried to explain, very approachable.
I read this paragraph out loud and suddenly I had two people making toothpaste commercials working for me
I don't understand what happened. Am I a marketer now?
i learned a ton about distributed systems in my time writing erlang and elixir, but it was almost always because i was forced into it by misapplying some erlang/elixir feature and backing myself into a corner. the idea that erlang/elixir make concurrency easy is extremely dangerous. what they do is make it accessible. you still need to understand what you are doing
Elixir is layered, making it easy to learn and master. You can get pretty far with Phoenix without ever understanding (or even knowing about) the more fundamental building blocks or the runtime. In large part, this is because of its ruby-inspired syntax. You'll have to adjust to immutability, but that's pretty much it.
Then one day you'll want to share state between requests and you'll realize that the immutability (which you're already comfortable with at this point) goes beyond just local variables: it's strictly enforced by these things called "processes". And you'll copy and paste a higher-level construct like an Agent or Genserver and add the 1 line of code to this root supervisor that was just a file auto-generated in your project. But that'll get you a) introduced to the actor model and b) thinking about messaging while c) not ever worrying about or messing up concurrency.
Then you'll want to do something with TCP or UDP and you'll see these same patterns cohesively expressed between the runtime, the standard library and the language.
Then you'll wan to do something distributed, and everything you've learnt about single-node development becomes applicable to distributed systems.
Maybe the only part of Elixir which can get complicated are Macros / metaprogramming. But you can get far without ever understanding this, and Phoenix is so full of magic (which isn't a good thing), that by the time you do need it, you'll certainly have peaked behind the covers once or twice.
The synergy between the runtime, standard library and language, backed by the actor model + immutability is a huge productivity win. It's significantly different (to a point where I think it's way more accurate to group Ruby with Go than with Elixir), but, as I've tried to explain, very approachable.