News for hackers. Stuff that matters.
Re:What every software engineer should know about Apache Kafka by commandlinefan on Sunday May 17, 2020 @08:25PM (#23216861)
Not just Kafka, either. Mongo, Cassandra, couchbase, Redis, SOLR and elastic search are all mistaken as replacements for an RDBMS.
Re:What every software engineer should know about Apache Kafka by 411111111111111 on Monday May 18, 2020 @10:26AM (#23222678)
Cassandra can replace an rdbms in a lot of cases if eventual consistency works for your data.

It's not a drop in replacement though, that's for sure.

Re:What every software engineer should know about Apache Kafka by commandlinefan on Monday May 18, 2020 @10:46AM (#23222972)
Better than the other suggestions, but my experience is that you run into Cassandra's limitations really quick: you can't query on non-primary columns, and you can't join tables without pulling all the data down to the client and merge manually.
Re:What every software engineer should know about Apache Kafka by 411111111111111 on Monday May 18, 2020 @02:37PM (#23226599)
> can't query on non-primary columns,

That's incorrect unless I'm misunderstanding what you mean with primary columns.. It's just not as efficient.

And missing joins are by design and one of the reasons cassandra is as fast as it is. And as I said before: it's not a drop in replacement. You need to architect your application around it's strength to leverage it's performance.

It is however usable as a rdbms replacement if you know what you're doing and your data is fine with eventual consistency.

And knowing what cassandra does with your data is important as well. It's actually a key-value store on steroids. Once you get that, it's limitations and strengths become obvious

Re:What every software engineer should know about Apache Kafka by commandlinefan on Monday May 18, 2020 @03:05PM (#23226971)
> That's incorrect unless I'm misunderstanding what you mean with primary columns

I'm referring to "Cannot execute this query as it might involve data filtering and thus may have unpredictable performance. If you want to execute this query despite the performance unpredictability, use ALLOW FILTERING"

> It's just not as efficient.

That's what I thought, too, so I "allowed filtering". And crashed the database. (Apparently the correct solution here is to use Spark).

Re:What every software engineer should know about Apache Kafka by 411111111111111 on Monday May 18, 2020 @03:44PM (#23227442)
haha, yes - thats a real possibility.

cassandra is at its heart a key-value store. for every queryable field, you need to create a new in-place copy of the data. so you're basically saving the data n-times, for each way you wish to query the data.

if you however try to query on a field which hasn't been marked as query-able, the cluster will have to basically select everything in the table and do a for loop to filter for the 'where' clause :-)

But i haven't used in production yet, so you've got more experience then i do