Re:Programming language comparison by reimplementing the sam...
by pg_bot on Sunday October 23, 2022 @05:33PM
(#33310489)
The elixir implementation seems super slow. The first thing that jumps out to me is that we should be using streams instead of just reading the file. The second would be the `schedule_for_route` which should just be a GenServer lookup instead of reading from ets.
[ Parent
| on HN ]
Re:Programming language comparison by reimplementing the sam...
by losvedir on Sunday October 23, 2022 @05:42PM
(#33310561)
Yeah, I should benchmark streams. In my experience streams are rarely faster, though, and are instead something you should reach for if 1) you can't fit the file in memory or 2) you might stop the enumeration early.
Regarding GenServer, I'm not so sure about that. I suppose I should benchmark it, but intuitively I expect ETS to be better here. There's more overhead in getting the data, but it allows you to concurrently read it from different processes. A GenServer, meanwhile, could respond to a given message faster, but now you're serializing the (e.g.) 50 concurrent virtual users through a single bottleneck. I could have multiple copies of the data in several GenServers, I suppose, at the expense of much more memory use.
Re:Programming language comparison by reimplementing the sam...
by pg_bot on Sunday October 23, 2022 @05:54PM
(#33310656)
I would guess that there's too much work being done at runtime. I would expect all of the data to be cached in such a way that it is a read operation without any transformation. It's doing multiple lookups and unnecessary maps when the data should just be formatted a single time and cached. You could even skip the JSON transformation by just doing it once if you want to get super fast.
Re:Programming language comparison by reimplementing the sam...
by Kabie on Monday October 24, 2022 @02:47AM
(#33313627)
Using stream can be faster here. Because you don't have to rebuild the same huge list again and again.
Re:Programming language comparison by reimplementing the sam...
by innocentoldguy on Sunday October 23, 2022 @10:31PM
(#33312402)
Agreed. I've seen significant performance improvements by using streaming with binary pattern-matching to parse files rather than loading them into memory and using String functions to get at the data.
These Elixir numbers seem suspect to me, based on my company's extensive benchmarking on many of these same languages. Elixir is somewhat slow at some tasks, but serving up web requests isn't one of them.