Re:Elixir RAM and the Template of Doom
by chrismccord on Tuesday April 26, 2016 @03:04PM
(#11574623)
The techmeme results are not representative of the framework as they report extremely high error rates in their results. We don't know what caused the errors and unfortunately we didn't have a chance to collaborate with them on a true run as they didn't publish their preliminary results like they normally do before the final set. A few months ago they added Phoenix in a preview, but it was a very poor implementation. They were testing JSON benchmarks through the :browser pipeline, complete with crsf token generation. They had a dev DB pool size of 10, where other frameworks were given of pool size of 100. And they also had heavy IO logging, where other frameworks did no logging. We sent a PR to address these issues, and I was hoping to see true results in the latest runs, but no preview was provided this time and we weren't able to work with them on the errors.
tldr; these results are not representative of the framework.
Re:Elixir RAM and the Template of Doom
by brightball on Wednesday April 27, 2016 @07:50AM
(#11579522)
Thanks for clearing that up! Now I'm excited again.
actually no framework in techempore is a representative of the framework. also the test cases say nothing about the real performance either.
Re:Elixir RAM and the Template of Doom
by out_of_protocol on Tuesday April 26, 2016 @03:06PM
(#11574636)
Long story short: last round was made in a hurry (due to hardware issues), no feedback loop at all, results were vastly different from preliminary tests. Hope the next round will be done sane way
Usually we see Elixir compared directly to Go and across the board Phoenix looks like it's not even coming close. I realize it's not the end all of comparisons, but it's just about the best thing we have going at the moment for comprehensive cross-language benchmarks.