News for hackers. Stuff that matters.
Re:Lithium ranked the highest performance web application pl... by andybak on Saturday August 29, 2020 @07:39AM (#24315371)
Is this a criteria that's important enough to warrant this much effort?

I can think of several metrics that would come first - even for sites with fairly high performance requirements.

How often is the web framework genuinely the bottleneck (at least in a way that's not easily fixable with caching or any other similar sledgehammer)?

Re:Lithium ranked the highest performance web application pl... by rishav_sharan on Saturday August 29, 2020 @08:16AM (#24315570)
4/6 tests in there account for database requests. So yes, I would say that Tech empower is pretty important to gauge the performance and efficiency of web frameworks.

Personally, it helped me realize that my mistaken belief that nodejs was hella fast and that old Java based frameworks are slow. It also introduced me to Kotlin (I started on it to try Kooby) and overall love that I can at a glance see that for my chosen set of languages, which farmeworks-dbs are a good bet.

Re:Lithium ranked the highest performance web application pl... by mattmanser on Saturday August 29, 2020 @11:11AM (#24316609)
The greatest trick that javascript ever pulled was to convince the latest generation of programmers that it was fast...
Re:Lithium ranked the highest performance web application pl... by dathinab on Saturday August 29, 2020 @04:07PM (#24319037)
The problem is they account for _simple_ database requests.

Which surely are common but long(er) running database requests or delegation to another service aren't super rare either.

Similar some of the frameworks might have major reliability problems which can randomly "crop up" due to thread pinning and similar.

I believe it should have some form of "chaos" test where instead of interacting with the DB it interacts with a another test server on both ends and the front and back connected server run through long random "chaos" simulation (and another version where a pre-recorded run is used for reproducibility).

Re:Lithium ranked the highest performance web application pl... by matt42 on Saturday August 29, 2020 @08:04AM (#24315513)
The benchmark show that the frameworks has a big impact on performance. While all the frameworks are using the same database server, performances varies from 1x to 100x.

While the benchmark focus on more req/s, we can also see it the other way around: for the same req/s, you can use smaller hardware, i.e. save money and energy.

Re:Lithium ranked the highest performance web application pl... by ghostwriter on Saturday August 29, 2020 @08:28AM (#24315635)
Not every framework presented in the benchmark perform the same amount of necessary protocol verifications. For instance, here's the take from Servant perspective - https://www.reddit.com/r/haskell/comments/6b3dlt/techempower...
Re:Lithium ranked the highest performance web application pl... by stagas on Saturday August 29, 2020 @08:00AM (#24315487)
In a typical web app you probably wouldn't notice any difference but in the niche where you're handling millions of reqs/s even slight improvements there could possibly translate to a non-trivial amount of money savings. On the other hand if you've already scaled to such a degree then those savings wouldn't matter that much. So, what's left is maybe for educational or research purposes, it's good to have an idea where the upper bound limit is while still maintaining a decent API, so there's value there, it's a great feat regardless.
Re:Lithium ranked the highest performance web application pl... by weego on Saturday August 29, 2020 @08:05AM (#24315519)
At millions of reqs per second the thin contract of the internals of http request/response will never be the layer that will have any fruit left to pick to make gains.
Re:Lithium ranked the highest performance web application pl... by sulam on Saturday August 29, 2020 @11:00AM (#24316531)
You’re making a statement here that is only true if you assume the developers of such a service have optimized their framework. If instead they are using one of the slower frameworks, they will pay proportionally — you can’t assume that just because they take Mrps that the service will be super complex past http. Often the exact opposite is true and they get to that scale by keeping the service very, very simple. Http processing becomes -more- of their latency budget in such cases.
Re:Lithium ranked the highest performance web application pl... by orthoxerox on Saturday August 29, 2020 @02:46PM (#24318468)
Yes, and the top frameworks are already bandwidth limited on the simplest benchmarks. The big gains are now made in DB access, which is a much more complicated topic than "ORM is bad for performance".
Re:Lithium ranked the highest performance web application pl... by brightball on Saturday August 29, 2020 @11:06AM (#24316568)
It's also a question of what's going on when you dive in. Out of curiosity, I've looked into several parts of benchmark source code before and it's questionable especially where the database is concerned. You'll see some things that seem to go against their own rules, like presorting and batching inserts for efficiency but their rules seem to indicate that it's not allowed.

It has lead me to treat these benchmarks with a grain of salt.

There are very few circumstances that I can imagine that would send me running for C++ for a web project. I'm much more concerned about balancing efficiency of the developer environment with performance, security and maintainability, than pure performance. At the point that I'm not, my priorities are out of whack (aside from very niche situations). The side effect of these benchmarks, while we all love benchmarks, is the idea that any of these faster frameworks are actually a better choice because of a benchmark result.

Re:Lithium ranked the highest performance web application pl... by matt42 on Saturday August 29, 2020 @07:35PM (#24320163)
My motivation in writing yet another web framework was to get pure performance while providing maintainability of more dynamic languages. Check the implementation of lithium, it is one of the most concise implementation of TFB: https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...
Re:Lithium ranked the highest performance web application pl... by Areading314 on Saturday August 29, 2020 @01:23PM (#24317713)
There are reasons why you might need to bolt an http api into a c++ project, so it's nice to know there are high performance options available
Re:Lithium ranked the highest performance web application pl... by austincheney on Saturday August 29, 2020 @10:53AM (#24316493)
On the client side, at least now, the framework and/or means of API access are always the primary performance bottleneck.
Re:Lithium ranked the highest performance web application pl... by Alex3917 on Saturday August 29, 2020 @08:22AM (#24315610)
Django is already vastly faster than human perception. An entire roundtrip to my Django app takes 70 - 110ms. The entire roundtrip is already faster than human perception even in the worst case, and of that Django itself probably contributes something like 2 or 3ms.
Re:Lithium ranked the highest performance web application pl... by w_t_payne on Saturday August 29, 2020 @11:14AM (#24316630)
110 ms is not faster than human perception. It's over a tenth of a second, and is very noticeable. The flicker fusion threshold for human vision lies in the 60Hz-90Hz range, corresponding to between 11 and 17ms. If I recall correctly, EEG experiments show differences up to around 100-120Hz, so on the order of 8ms. In any case, network latency itself adds on the order of 10ms per 1,000km of distance; 35ms or so for a London to New York round-trip, so the point about framework speed being a bit of a moot point other than in exceptionally high traffic scenarios still stands.
Re:Lithium ranked the highest performance web application pl... by Alex3917 on Saturday August 29, 2020 @11:52AM (#24316932)
> 110 ms is not faster than human perception.

OK yes, it a literal sense you can see things that happen in less than 100ms. What I mean is that you can't tell whether the nominal amount of flicker on page load is from the time it takes to paint the DOM or from the network request. Unless you were asking a professional front end developer, I don't think most people would even have any intuition about what was happening.

Re:Lithium ranked the highest performance web application pl... by ksec on Saturday August 29, 2020 @10:23AM (#24316303)
>and of that Django itself probably contributes something like 2 or 3ms.

Unless it is super tuned, running on a specialise VM, I seriously have doubt about that. It is already pretty damn good if PHP/Python Django and Ruby Rails could do 10ms request excluding the Database on a moderately large App. Generally they are 20ms+.

And recently shopify even have their performance matrix allowing for 300ms per request.

Re:Lithium ranked the highest performance web application pl... by fmajid on Sunday August 30, 2020 @06:46PM (#24326636)
At my company we used to have an OpenRTB bidder backed by Redis that would do 9ms 99th percentile for 4B requests/day spread over 8 machines.
Re:Lithium ranked the highest performance web application pl... by ghostwriter on Saturday August 29, 2020 @08:37AM (#24315668)
At this point it's mostly about (cost) efficiency, like how many EC2 instances you will need to cover your workload with a service implemented in a particular framework. It also assumes that developers can pick any of them with the same amount of effort, which is probably a proper assumption for a benchmark that measures performance.
Re:Lithium ranked the highest performance web application pl... by Alex3917 on Saturday August 29, 2020 @08:49AM (#24315722)
How much cost efficiency is there? I’ve worked at multiple of the top 100 websites in the U.S., and none of them had more than a dozen web servers. For any website, 90% of traffic is logged out and probably doesn’t even hit the web servers.

And web servers are generally the least expensive component of hosting costs.

Re:Lithium ranked the highest performance web application pl... by ghostwriter on Saturday August 29, 2020 @09:00AM (#24315783)
These costs are more meaningfull for those who bootstrap their businesses and projects at their own expense. I have a personal anectdote where switching from a scripting web framework to a compiled one reduced the runtime size, and the deployable environment went from 800mb to 10mb, which allowed me to fit all my deployments into a free-tier monthly quota of AWS ECR.

Given I already knew both frameworks from fulltime jobs, it's a great net benefit.

It also means that whoever wants to use self-hosted open-source alternatives of user-tracking services (notes, calendars, emails, etc), they can do it cheaper or even for free.