News for hackers. Stuff that matters.
Re:Cassandra at Apple: 1000s of Clusters, 300k Nodes, 100 PB by tpmx on Saturday October 08, 2022 @05:50PM (#33136347)
> Apple has a lot more data than just a list of devices. [...]

Of course. That is not the point here.

Re:Cassandra at Apple: 1000s of Clusters, 300k Nodes, 100 PB by echelon on Saturday October 08, 2022 @06:05PM (#33136473)
Perhaps you'd be better convinced with a service breakdown.

Breaking monoliths into service boundaries yields easier ownership, maintenance, migration, and resilience.

One "tiny" company with a few verticals can be comprised of thousands of microservices, each handling their own dedicated objective. Authentication, reverse proxy, API gateway, SMS, email, customer list, marketing email gateway, CMS for marketers on product X, feature flags, transaction histories, GDPR compliance handling, billing intelligence, various risk models, offline ML risk enrichment, etc. etc. Each will have its own data needs and replication / availability needs.

This Apple number might seem crazy, but I'm not phased by it. I can picture it.

Re:Cassandra at Apple: 1000s of Clusters, 300k Nodes, 100 PB by tpmx on Saturday October 08, 2022 @06:19PM (#33136577)
I can also picture it, but not really in the way you're outlining it.

It's a sad and very inefficient picture though. Apple does not need this this much data processing. It's a grotesque amount per device. Or maybe they're just wasting insane amounts of energy doing lots and lots doing of stupid analytics...

Re:Cassandra at Apple: 1000s of Clusters, 300k Nodes, 100 PB by echelon on Saturday October 08, 2022 @06:29PM (#33136647)
Sometimes things have to be built as layered abstractions in order for humans to reason about them at scale.

See also the natural stochastic gradient ascent that produced our crazy complicated metabolic pathways (and all of biology).

Re:Cassandra at Apple: 1000s of Clusters, 300k Nodes, 100 PB by threeseed on Saturday October 08, 2022 @09:30PM (#33137552)
> It's a grotesque amount per device

Again. It is not just for device data.

There are backend services which your device interacts with e.g. Maps, Siri, Weather.