News for hackers. Stuff that matters.
Re:Kafka vs. Redpanda Performance – Do the claims add up? by skyde on Monday May 15, 2023 @01:31PM (#35951639)
Exactly I will never ever try MongoDB because of that. A database that do not fsync should not be called a database.
Re:Kafka vs. Redpanda Performance – Do the claims add up? by nezirus on Monday May 15, 2023 @02:45PM (#35952765)
Kafka doesn't do any stupid tricks, but uses the underlying platform for the full potential: https://kafka.apache.org/documentation/#linuxflush

With the usual recommended settings, XFS filesystem, 3 replicas, 2 "in-sync" replicas, etc., it is rather safe. You can also tune background flush to your liking.

The above tradeoffs are very reasonable and Kafka runs very fast on slow disk s(magnetic or in cloud), and even faster on SSD/NVMe disks.

Re:Kafka vs. Redpanda Performance – Do the claims add up? by uberduper on Monday May 15, 2023 @03:11PM (#35953161)
MongoDB moved on from mmap at version ~3.6. WiredTiger can be configured to fsync every commit. Enjoy trying MongoDB!

PS: I really miss working with mongodb. It's been almost 7 years since I last used it. I'm surprised I don't see it mentioned very often anymore.

Re:Kafka vs. Redpanda Performance – Do the claims add up? by AtlasBarfed on Monday May 15, 2023 @04:37PM (#35954151)
Last I heard of MongoDB it was getting utterly buried by the Jepsen guy, and for anyone that follows distributed systems at some technical level, that is damning. He finds stuff wrong with everything, but that one was particularly damning.

MongoDB has always seemed to place write consistency secondary to other priorities (mostly sales / read / features) which is frankly a crap way to do a database, much less a distributed one. And I am so sick of MongoDB basically saying "no it's fixed in the new version" which is always a major red flag.

Right now it's getting its lunch eaten by Postgres's document interface from what I can tell.

Re:Kafka vs. Redpanda Performance – Do the claims add up? by threeseed on Monday May 15, 2023 @04:45PM (#35954245)
a) Every distributed database has had serious issues with Jepsen.

b) MongoDB has been growing revenue ~40% year on year for the last few years.

c) PostgreSQL is only a serious competitor for MongoDB if you have small datasets. After all these years PostgresSQL still is ridiculously poor when it comes to clustering, replication etc. Everyone's solution of "just buy a bigger instance" is just laughable.

Re:Kafka vs. Redpanda Performance – Do the claims add up? by KptMarchewa on Monday May 15, 2023 @07:29PM (#35955939)
Growing revenue of a owner company as a argument for database? We have an Oracle fan here.
Re:Kafka vs. Redpanda Performance – Do the claims add up? by AtlasBarfed on Monday May 15, 2023 @10:09PM (#35957054)
Jepsen does find stuff with everything. Thus you have to know what is being discussed is serious and blatantly bad, or just the usual "wow distributed is hard".

Which is why his papers are so great.

But the MongoDB one was "wow this is bad".

Re:Kafka vs. Redpanda Performance – Do the claims add up? by threeseed on Monday May 15, 2023 @11:47PM (#35957542)
Every distributed database has been "wow this is bad".

I assume you have an example of one that wasn't ?

Re:Kafka vs. Redpanda Performance – Do the claims add up? by AtlasBarfed on Tuesday May 16, 2023 @12:26AM (#35957737)
"MongoDB’s default level of write concern was (and remains) acknowledgement by a single node, which means MongoDB may lose data by default."

Cassandra doesn't do that, consistency level is fundamental to the documentation and user guide. That is AWFUL.

"Curiously, MongoDB omitted any mention of these findings in their MongoDB and Jepsen page. Instead, that page discusses only passing results, makes no mention of read or write concern, buries the actual report in a footnote, and goes on to claim:

    MongoDB offers among the strongest data consistency, correctness, and safety guarantees of any database available today.
"

That is fraud. That is clownshow. Enjoy your increasing revenue.

Re:Kafka vs. Redpanda Performance – Do the claims add up? by threeseed on Tuesday May 16, 2023 @04:27AM (#35959025)
The default write concern for the last 2 years has been majority.

And single node is a perfectly fine default for most use cases.

After all Cassandra's default consistency level is 1.

Re:Kafka vs. Redpanda Performance – Do the claims add up? by dorlaor on Tuesday May 16, 2023 @06:53AM (#35960021)
Most users of ScyllaDB/Cassandra use quorum or local_quorum
Re:Kafka vs. Redpanda Performance – Do the claims add up? by threeseed on Tuesday May 16, 2023 @11:51AM (#35963541)
And most users of MongoDB use majority write concern.

Hence my point there is no difference in defaults between MongoDB and Cassandra.

Re:Kafka vs. Redpanda Performance – Do the claims add up? by doctor_eval on Tuesday May 16, 2023 @02:27AM (#35958327)
Although it was some time ago and I may be misremembering, I seem to recall reading the Jepsen article on RedPanda and thinking that it (and Postgresql) were among the better reports.

Certainly, not all Jepsen reports are all that bad, and tbh I'm at leaast as interested in the way the vendors respond (some of which have been terrible).

Re:Kafka vs. Redpanda Performance – Do the claims add up? by datadeft on Monday May 15, 2023 @03:41PM (#35953438)
Kafka is not a database....
Re:Kafka vs. Redpanda Performance – Do the claims add up? by postalrat on Monday May 15, 2023 @05:00PM (#35954418)
Maybe you could say that if it acted like redis pub/sub and nothing was stored.
Re:Kafka vs. Redpanda Performance – Do the claims add up? by threeseed on Monday May 15, 2023 @04:42PM (#35954207)
MongoDB has been doing fsync by default for over a decade now .

And those that actually had tried it were aware that every client enabled fsync out of the box. So in fact the entire situation was seriously overblown.

But sure let irrational ideology affect your technology decisions. That will work out well.

Re:Kafka vs. Redpanda Performance – Do the claims add up? by hnfong on Tuesday May 16, 2023 @08:54AM (#35961203)
Avoiding a database that has a proven historical record of disregarding data consistency and resorting to marketing gimmicks is "irrational ideology"?

Not everyone has time to review every single line of code in their tech stacks. Past reputation is important, and your replies here don't seem to be of much help to MongoDB's reputation as far as I can tell.