News for hackers. Stuff that matters.

jjirsa

Re:The road to ACID transactions in Cassandra 6 on Saturday August 22, 2026 @03:08PM (#49402787)
Feels like a thread for me

I’ve run more Cassandra than anyone on the planet. Also more FoundationDB. And more Postgres than I can count. Likely 3 orders of magnitude more than you.

1. Any db at that scale requires a team. Thousands of cores is gonna be either paying saas or paying a team and I don’t care what the tech is. You’re gonna fuck up any db on the market letting your app eng run a thousand cores of Postgres or MySQL or couchbase or cockroach or tidb.

2. Cassandra at scale has a demonstrably better cost and availability slo than literally annything else I’ve ever tried to run, as long as you have a competent team. And I had a great team. But we had a great team because we understood that at million core scale it’s a core of the product and you invest in your core. But a good team of a dozen people can run a million cores at 6-8 nines, at lower cost per byte or cost per query than any db-aas you can buy online from any cloud vendor. I know this because we checked, and I’ve been ahead every time.

3. The db isnt fun to run. But it’s getting way better. If you think this is 2009, it’s not. The folks working on it in 2026 know what they’re doing and the last decade has made it measurably safer and easier.

Nothings free, but I’d bet my salary I can run damn near any company on Cassandra cheaper than any of your alternatives. Even all the vendors who keep selling against Cassandra. A competent team on Cassandra is going to beat your vendor product 90% of the time.

> Waymo is now completing 450,000 weekly driverless rides across six cities, with a peer-reviewed study showing statistically significantly lower crash rates than human drivers across 56.7 million rider-only miles. When Waymo’s vehicles were caught passing school buses, the company filed a voluntary recall within weeks. Tesla, on the other hand, is now on its second deadline extension just to tell NHTSA about FSD’s traffic violations.

That, in itself, is saying more than anything else

Re:IBM Plans to Acquire DataStax on Tuesday February 25, 2025 @03:32PM (#43176995)
Congrats datastax folks. Happy for you.
Re:ScyllaDB moving to a source available license on Wednesday December 18, 2024 @07:36PM (#42457104)
Youre not far off, just use your original sentence and acknowledge that Scylla isn’t the OSS here, its the company that came in, forked a volunteer driven project, and tried to pretend its theirs:

> Or more often companies with money come to fork an open source project from the developer, and continue pretending it's their own now.

Re:ScyllaDB moving to a source available license on Wednesday December 18, 2024 @05:59PM (#42456450)
> Copycat is a bit of a stretch here imho, considering it's in different language, and even the general architecture. But I agree they are very compatible on the protocol level, which they used as an advantage as people don't need to rewrite their code when they migrate

It's not a stretch. They literally copied the java code and re-implemented it class-by-class with Seastar/c++.

It's literally in the ORIGIN file in their repo: https://github.com/scylladb/scylladb/blob/dc375b8cd3e8c7e85d...

Re:ScyllaDB moving to a source available license on Wednesday December 18, 2024 @05:35PM (#42456226)
> I would say it's opposite. No one contributes to open source and the developers have to make money on the side to work on the project

You missed the important part of his sentence:

> All those open source copycat projects, using open source to get exposure then switching

The original open source here was Cassandra. Scylla exists to pick off that market share. They launch with a free license, pull customers, then swap license. The actual adoption of Scylla would have been a fraction had it been released under this license to start, which everyone understands.

> Or more often companies with money come to fork an open source project from the developer, and continue pretending it's their own now

It had already been AGPL, so it's not like they're protecting against competition. They already had a license that avoided the forking problem. This isn't protection against AWS (keyspaces likely has more OSS cassandra code in the protocol tier than it has Scylla code, if it has either). It's protection against free consumption.

Re:Ask HN: I want to create IMDB for Open source projects on Monday April 15, 2024 @05:01PM (#40045535)
Remarkable to me that ctrl-f, "Sourceforge" had 3 hits on this page. Would have expected a dozen.
Re:Apple releases Pkl – configuration as code language on Saturday February 03, 2024 @01:29PM (#39243235)
Some companies have both massive files and a massive number of files
Re:Apple releases Pkl – configuration as code language on Saturday February 03, 2024 @10:45AM (#39241394)
> I do wish they just contributed to nickel or something else though rather than doing their NIH as usual.

They’re contributing to a ton of OSS that’s NIH:

https://opensource.apple.com/projects/

K8s, spark, cassandra, netty, zookeeper, solr, containerd

Re:Apple releases Pkl – configuration as code language on Saturday February 03, 2024 @10:37AM (#39241300)
Yes! Congrats Phil!
Re:Pkl, a Programming Language for Configuration on Saturday February 03, 2024 @10:33AM (#39241258)
Regrettably the folks who have been using it for years aren’t going to be give a lot of specifics, but generation of k8s yaml / jsonnet in particular was exceptionally common. One example from the other thread:

> My team migrated several kloc k8s configuration to pkl with great success. Internally we used to write alert definitions in pkl and it would generate configuration for 2 different monitoring tools, a pretty static documentation site and link it all together nicely.

https://news.ycombinator.com/item?id=39235425

Re:Pkl, a Programming Language for Configuration on Friday February 02, 2024 @09:54PM (#39237170)
Congrats Apple folks who worked on this, I know it’s been a long time coming
Re:RavenDB 6.0.2 (A Jepsen Report) on Wednesday January 31, 2024 @11:25AM (#39205669)
Cassandra’s jepsen is over 10 years old now.

That database spent 3-4 years primarily focused on correctness from 2018-2022.

The industry moves fast, our memories are slow. But there are millions of instances of cassandra in production across most of the fortune 500, and half of this thread has never heard of RavenDB

Re:RavenDB 6.0.2 (A Jepsen Report) on Wednesday January 31, 2024 @11:21AM (#39205602)
It’s disappointing to me that the technologist desire to experiment with new DBs continually puts naive customers at correctness and durability risk they don’t (won’t) understand.
Every database engineer I know, and that’s … basically all I do anymore … says every database is bad, except when they’re being paid to promote one their employer builds.
> Apple's underlying economics haven't changed, so they have not had any reason to over-hire in the first place.

The bloomberg article has a visual showing revenue-per-employee going up from 1.17M/employee (2017-2019) to 2.51M/employee (2020-2022), so ... some of the underlying economics changed somehow?

Re:Musk orders Twitter to cut infrastructure costs by $1B on Thursday November 03, 2022 @11:20PM (#33461672)
Tell me you’ve never run infrastructure at scale without telling me you’ve never run infrastructure at scale.
Re:Cassandra at Apple: 1000s of Clusters, 300k Nodes, 100 PB on Friday October 14, 2022 @01:12PM (#33206269)
Just wanted to say thanks for understanding how hard this is.

It's a fun sub-thread to read.

As I mentioned elsewhere, I'll see if I can get permission to talk publicly about the actual numbers.

Re:Cassandra at Apple: 1000s of Clusters, 300k Nodes, 100 PB on Friday October 14, 2022 @01:17AM (#33199903)
I'll see if we can get permission to discuss this publicly.
Re:Cassandra at Apple: 1000s of Clusters, 300k Nodes, 100 PB on Sunday October 09, 2022 @08:31AM (#33140396)
You’re wrong again here, in basically every single comment

If you have 3 and you for CL 3 instead of quorum, and only 2 are up, it won’t partially write - it’ll throw an unavailable exception instead of attempting it

If you have 3 and specify quorum and 1 is down, it’ll write to the two, and propagate to the third in one of three ways:

- via hints (the timed thing you describe),

- via active repair (times and tombstones don’t matter at all, and data can’t magically come back to life with reasonable config settings), or

- or via read repair when you read from a quorum that overlaps the down node, again, regardless of pretty much anything else, this always happens.

Re:Cassandra at Apple: 1000s of Clusters, 300k Nodes, 100 PB on Saturday October 08, 2022 @11:40PM (#33138079)
Cassandra is very good when all of these are true:

- You're willing to give up some common SQL-isms for leaderless, multi-dc, high-availability

- You're going to grow to need more than one machine taking writes - if you can fit on one normal commodity machine, just use something designed for one machine. Cassandra sacrifices a lot to scale to ~thousand-machines-per-cluster, so if you can use one machine, don't bother with Cassandra.

- You can model your data in a way that does SELECTs without JOINs, and always uses AT LEAST the partition key in the WHERE clause. Denormalizing and duplicating your data is PROBABLY ok.

- You're willing to run Java (so you understand that you may need to tune the JVM eventually), and you're willing to learn about data modeling before you just start writing code

If all of that is true, Cassandra starts being interesting.

Re:Cassandra at Apple: 1000s of Clusters, 300k Nodes, 100 PB on Saturday October 08, 2022 @09:53PM (#33137663)
Except they definitely do use it and many actually need it
> I think Cassandra - certainly inter alia - simply sits at a level of complexity which is beyond the comprehension of any one human being

I think there are a handful of humans that actually get it. But it is a true handful, not more than 5.

Re:A pure Go embedded SQL database on Saturday January 29, 2022 @11:55PM (#30134161)
The cardinal rule of databases: http://www.dbms2.com/2013/03/18/dbms-development-marklogic-h...

Ignoring that leads to long postmortems about how your embedded db failed in a way you didn’t expect: https://blog.roblox.com/2022/01/roblox-return-to-service-10-...

SQLite is good

Rocksdb is good

You don’t need pure go.

Re:Albanian prosecutors probe huge suspected leak of personal data on Wednesday December 22, 2021 @12:26PM (#29652365)
Targeting for home invasion

Targeting for extortion

Targeting in general

Re:Ask HN: Why is Docusign a $50B company? on Saturday November 27, 2021 @06:25PM (#29364627)
The “extraordinarily difficult to do very well” part gets ignored here by most people

DocuSign’s feature list is huge. It’s not just overlaying a signature on a doc. It’s a massive and complex app that most people will ignore or trivialize, but that shit just works.

Mark all the signature locations?

Multi-user?

Chained dependencies?

Recording and redownloading later?

SSO

Audit / identification

Salesforce integration?

Contract generation?

Orange site minimizing huge successful company because they care about one feature seems super on brand.

Re:Benchmark results: Cassandra 4.0, 3.11, Scylla 4.4 [video] on Thursday September 30, 2021 @01:15AM (#28703422)
I don’t think anyone avoiding talking about single threaded bottlenecks. The main one that matters is on sstable streaming during bootstrap, and the zero copy streaming was directly to address that. There’s not really any single threaded pieces in cassandra that matter beyond that? If anything the opposite is true - thread pools everywhere leading to context switching and data copying for no reason.

On a deeper note, it probably says something about the user base that people who don’t run it in prod think the perf is really bad and yet there are very few people submitting perf improvement patches. Maybe most of the power users aren’t that worried about perf because they either know how to tune a JVM or they happen to size their clusters based on bytes on disk?

Re:Benchmark results: Cassandra 4.0, 3.11, Scylla 4.4 [video] on Thursday September 30, 2021 @12:59AM (#28703347)
Your options for scylla are AGPL or pay scylla. Nobody risks AGPL in prod so people running it are probably paying scylla.
Re:Benchmark results: Cassandra 4.0, 3.11, Scylla 4.4 [video] on Tuesday September 28, 2021 @02:17AM (#28679834)
Modern Cassandra is more like Dict[SortedList[Row]], where the row is basically a SQL row just like you’d expect in any SQL database
Re:Benchmark results: Cassandra 4.0, 3.11, Scylla 4.4 [video] on Monday September 27, 2021 @09:20PM (#28678201)
Something something bigtable

Older »