Keep in mind, Wikipedia is at scale that most web applications will never, ever get to.
It's important to think about performance and scale, but it's not the only important trade off engineers should concern themselves with.
The win here was about individual page load time. And page load time is just as important, if not more so, for something new trying to vigorously grow as it is for the big sites.
(Disclaimer: HHVM alum.)
His machine could easily handle the traffic thrown at it, but the page load time was slow. 2 seconds at best, with all machines involved being almost 100% idle. With various common tweaks such as caching, we got it down to about 800ms. We eventually replaced it with our own solution and got it to 50ms.
Scale never entered the picture because from our testing, we had the machinery in place to handle enough traffic to be wildly profitable. The user experience at 2 second page load times was vastly different from the user experience at 50ms though.
MySQL over Postgres because that's what we had experience with at the time. Redis as both a cache and ephemeral store.
So, more request can be served with a smaller number of servers.
The un-shortened URL is https://ganglia.wikimedia.org/latest/graph.php?r=year&z=xlar...
(I should've put a disclaimer there: I am one of the WMF engineers involved in the project).
It's also not necessarily about scale, as in number of users that one needs to serve. There are lots and lots of problems where performance is very relevant, from startups that can't afford to scale horizontally, to projects in an industrial setting that have to be rock solid, to participating in real-time bidding on RTB platforms, to games in which frames being dropped are a disaster, to the platforms on top of which we build stuff.
Quite the contrary, I believe that too many software developers are focusing on building front-ends to a database, but this isn't necessarily because these problems are solving real needs.
As I read the article I thought exactly this. Scaling is a nice problem to have. I keep playing around with Nginx, elasticsearch, and other tech that's supposed to help PHP's performance problem, but from personal experience, missing database indexes for complex joins have been the issue every time. I'm unlikely to get to use the tech I play with unless I switch jobs.
But back to the point, this is a great writeup, and I think a big step forward for both PHP and HHVM
This will also make tiny MediaWiki installs just as much faster.
I don't think Wikipedia is really a typical case. Most websites probably don't have the CPU burden Wikipedia faces.
[0]https://en.wikipedia.org/w/index.php?title=Barack_Obama&acti...
Perhaps MediaWiki should figure out how to use a weaker form of computation?
According to that 99% of magento2 tests are passing.
I'd be more interested to see what, if any changes they have made to their other server components. Are they using NGINX vs Apache? What kinds of caching systems are they using? What is their database layout? Is their data sharded? These kinds of things I think are much more interesting, and at their scale at least as impactful.
The medium-term goal is to store all revisions as HTML, so that views and HTML-based saves can become storage-bound operations with low double-digit ms latencies. This will mostly eliminate the remaining 2-4% of cache misses, and should get us closer to actual DC fail-over without completely cold caches.
There is still a large amount of work to be done until all the puzzle pieces for this are in place, but we are hard at work to make it a reality. A major one, the bidirectional conversion between wikitext and HTML in Parsoid (https://www.mediawiki.org/wiki/Parsoid), is already powering VisualEditor and a bunch of other projects (https://www.mediawiki.org/wiki/Parsoid/Users). Watch this space ;)
GD is using it to store user-generated content, and it works very well, most responses (lookup and processing) were sub 20ms consistently under heavy load... (I'm hoping they write a DNS service that does similar).
Also, speaking from a few years' personal experience with HBase / Cassandra, the support for non-Java languages on these two NoSQL databases is small (though it is getting better.)
Though node.js tends to be very flexible in terms of wrapping a friendlier interface around a less friendly one.
Yes, three seconds for a page render is still uncomfortably slow, but it's substantially faster than the original implementation, and unsurprisingly frees up CPU for additional concurrent requests. It's a shame Wikimedia didn't have this platform available to them earlier.
Today web developers have many high-performance platform options that offer moderate to good developer efficiency. Those who use low-performance platforms may do their future selves a service by evaluating (comfortable) alternatives when embarking on new projects.