News for hackers. Stuff that matters.
Re:The database ruins all good ideas by bobthebuilders on Saturday July 17, 2021 @04:09PM (#27868419)
CockroachDB is a terrible technology that causes almost guaranteed data corruption due to its lack of ACID guarantees and is written in a language with a GC which contirbutes to GC pauses. The dev team refuses to listen to feedback to port their code to C ;(.
Re:The database ruins all good ideas by Guvante on Saturday July 17, 2021 @04:47PM (#27868728)
Because "just rewrite your program in C to avoid GC pauses" is such a flawed argument when it comes to any production system that it isn't even worth discussing.

The reality is there are limited resources to work on any given project and "rewrite" is generally not the correct way to fix a given problem.

Especially since the first thing you are going to need to do is show that a network system isn't resilient to GC pauses or that GC pauses are frequent enough to impact throughput (very very few applications are actually materially impacted by sub second spikes in latency).

Re:The database ruins all good ideas by pkolaczk on Saturday July 17, 2021 @05:08PM (#27868888)
GC pauses are not the only GC issue. Thrashing the caches and blowing up memory usage by an order of magnitude can be also very bad for performance. In a database system memory is very precious - the more of it you can use for caching / buffering users data, the better the performance.

As for the subsecond spikes in latency, these tend to multiply in a distributed system. If serving a client request takes N internal requests, the likelihood of hitting a GC pause somewhere is much larger than if you did only one local request.

Not sure about Go, but none of the "free" Java's GC guarantees low pauses. There is STW fallback even in the most advanced ones like ZGC. So you never know when it stops for more than 1s.

Re:The database ruins all good ideas by ngrilly on Saturday July 17, 2021 @07:44PM (#27869692)
Go GC pauses are bounded at 0.5 ms:

https://blog.golang.org/ismmkeynote

Re:The database ruins all good ideas by pkolaczk on Sunday July 18, 2021 @01:17AM (#27870920)
Is it a hard guarantee or just a soft goal like in G1, which tries to stay within the target pause milliseconds, but there are many ways it can fail to? Can it handle 100+ GB large heaps?

Even if it is a hard guarantee, then, from the link you posted, it is not even generational, so it will scan the whole heap quite frequently, and that is going to influence average performance quite visibly - you definitely dont want a database system to access all its cached memory once in a while.

Database systems are really all about memory and I/O management. You shouldn't outsource those core features to a universal algorithm, unless you wish to forgo any competitive advantage (at least in performance department). So this pushes the devs into the off-heap manual memory management territory, where dragons live (at least in Java, again - maybe Go is better in that regard). I've been there, and I don't recommend.

Re:The database ruins all good ideas by redis_mlc on Saturday July 17, 2021 @05:15PM (#27868926)
> Because "just rewrite your program in C to avoid GC pauses" is such a flawed argument

False. All popular databases are written in C, and continue because of the GC issue. I would not use a general-purpose database written in Java because of GC, for example. We'll see how well Go works in practise.

> sub second spikes in latency

Go is supposed to be sub-second GC pause latency, but understand that most SQL queries are sub-millisecond, so GC latency is still a significant issue compared to query times.

Go might be acceptable now for niche databases like column-store for certain use cases, though.

Also, see the excellent comment above about distributed systems and server cache issues. You can't do application performance analysis with GC literally everywhere.

The puerile knee-jerk hatred for C on HN has to stop - almost every software you use is written in C, from scripting languages to operating systems to web servers to databases.

Source: DBA who's worked with current databases, as well as a custom database written in Java with significant (ie. brutal) GC problems that required a total refactor (rewrite) to "work" at all.

Re:The database ruins all good ideas by mappu on Saturday July 17, 2021 @07:15PM (#27869549)
I'd like to reply specifically to the Go is supposed to be sub-second GC pause latency part - Go's GC is crazy fast, it pauses for only microseconds, comparable to the socket transport overhead. It can't be thought of as Java-like at all.
Re:The database ruins all good ideas by ngrilly on Saturday July 17, 2021 @07:46PM (#27869707)
Re:The database ruins all good ideas by ngrilly on Saturday July 17, 2021 @07:51PM (#27869727)
The fact that database engines written in Java exhibit high latency is not a reason to dismiss all other programming languages except C and C++. It still makes sense to consider modern languages like Rust, Go, Zig, etc.
Re:The database ruins all good ideas by lmm on Sunday July 18, 2021 @07:53AM (#27872191)
I've run production systems on H2 that ran rings around the same company's dedicated-DBA systems.
Re:The database ruins all good ideas by hugofirth on Saturday July 17, 2021 @08:05PM (#27869776)
This is daft. CockroachDB exhibits extremely strong isolation and consistency levels. I believe it is strictly serializable under most circumstances?

Also - anyone who says that x database must be rewritten “because GC” is just making an incredibly un nuanced argument about a nuanced problem. People have built production ready databases in both Java and Go. If you care about low/predictable tail latencies then you have a bunch of other more important problems to solve before you worry about the behaviour of a modern garbage collector. For example: how good is your cache hit ratio? How are your synchronous replication protocols affected by grey failures? That kind of thing.

Re:The database ruins all good ideas by _benedict on Sunday July 18, 2021 @03:13AM (#27871257)
I believe it only guarantees serializable isolation. You may get strict serializable, but it doesn’t appear to be the case that you will know for sure if any transactions were not linearizable.
Re:The database ruins all good ideas by andreimatei1 on Saturday July 17, 2021 @06:19PM (#27869274)
> its lack of ACID guarantees

Our transactions implementation is our crown jewel. You might want to check your sources.