News for hackers. Stuff that matters.
Re:A lot of complex “scalable” systems can be done with a si... by riyadparvez on Sunday December 29, 2019 @08:22PM (#21909966)
One thing to keep in mind that their work-load is very ready-heavy which eases things a lot when scaling the system. The same is true for Wikipedia. Scaling a write-heavy workload is way more complex than scaling a read-heavy workload.
Re:A lot of complex “scalable” systems can be done with a si... by manigandham on Sunday December 29, 2019 @09:01PM (#21910153)
SO content changes all the time. Votes, comments, moderation, edits, tagging, search and recommendations, etc. There are also real-time community features. It's not as simple as it seems.
Re:A lot of complex “scalable” systems can be done with a si... by RhodesianHunter on Monday December 30, 2019 @12:13AM (#21910971)
They said read heavy, not read only.

I guarantee you the bulk of traffic to SO is hitting a page and performing zero writes.

Re:A lot of complex “scalable” systems can be done with a si... by manigandham on Monday December 30, 2019 @12:59AM (#21911147)
They log every single pageview to SQL Server. There are plenty of writes to various other counters, recommendation system, message inboxes, analytics, etc. Also the ads system, although I'm not sure how much of that is still 3rd-party.

Yes it's read-heavy but there's still plenty of work done in assembling a page. It's definitely not as simple as caching at the CDN edge for every hit.

Re:A lot of complex “scalable” systems can be done with a si... by dvirsky on Monday December 30, 2019 @01:43AM (#21911311)
IIRC most of the "real time" stuff is done on Redis, for which this QPS load is usually a joke (depending on requests of course, but counters etc are easy)
Re:A lot of complex “scalable” systems can be done with a si... by fock on Monday December 30, 2019 @10:17AM (#21913818)
so, would you mind showing any non-FANG, but write heavy site? Besides that, even the operation of most social networks should be very easy to be split into town-sized instances with batch-updates in-between instances (see mastodon specs https://github.com/tootsuite/documentation/blob/master/Runni...). What actually costs a lot is constantly surveilling all user interactions, data-warehousing that and playing out ads. None of which is necessary, and a 20ct/per user/per month should cover the actual service nicely...
Re:A lot of complex “scalable” systems can be done with a si... by jjeaff on Monday December 30, 2019 @03:08PM (#21916233)
Most any saas offering is going to be very write heavy in comparison.
Re:A lot of complex “scalable” systems can be done with a si... by rbanffy on Tuesday December 31, 2019 @05:34AM (#21920110)
All that can be implemented as separate services. Doing that also enables graceful degradation, as a failure of, say, voting doesn't prevent me from reading the question.
Re:A lot of complex “scalable” systems can be done with a si... by hliyan on Sunday December 29, 2019 @08:47PM (#21910083)
There is a significant amount of question submission, commenting and voting going on on SO.
Re:A lot of complex “scalable” systems can be done with a si... by yawn on Sunday December 29, 2019 @09:03PM (#21910162)
A slim subset of humans typing things is not what I would label "write heavy". Write heavy is more like 100k+ devices out in the field sending their current position every 10s. That's still very manageable, but requires some thoughtful design.
Re:A lot of complex “scalable” systems can be done with a si... by Retric on Sunday December 29, 2019 @10:27PM (#21910582)
10k simple updates per second is hardly high bandwidth. I knew people handling similar workloads on a 1U server 10 years ago.
Re:A lot of complex “scalable” systems can be done with a si... by pushpop on Monday December 30, 2019 @04:38AM (#21912148)
Indeed and I literally was. And the back end was written in mod_perl (which isn’t exactly known for performance). Even back then 10k/s was our smallest service - both on income as well as traffic - so there wasn’t the incentive to rewrite it in a faster language.
Re:A lot of complex “scalable” systems can be done with a si... by stefan_ on Sunday December 29, 2019 @11:46PM (#21910884)
The Raspberry Pi 3 in my cupboard without a heat sink on can do 80000 Redis write ops/sec.