News for hackers. Stuff that matters.

A comment on ClickHouse cost-efficiency in action: analyzing 500B rows on an Intel NUC. Parent comment

Re:ClickHouse cost-efficiency in action: analyzing 500B rows... by codexon on Monday January 06, 2020 @08:17PM (#21975707)
Fsync is not synonymous with transactions. Not using fsync anywhere means there's a wide window that can be over 10 minutes long when data can be lost when a server gets an unplanned shutdown.
[ Parent | on HN ]
Re:ClickHouse cost-efficiency in action: analyzing 500B rows... by valyala on Tuesday January 07, 2020 @06:44PM (#21986543)
Btw, it should be easy adding fsync to ClickHouse. For instance VictoriaMetrics uses similar file format as ClickHouse, and it issues proper fsyncs at least every second, so it may lose only the last second of data on unclean shutdown such as OOM, hardware reset or `kill -9`. [1], [2].

[1] https://medium.com/@valyala/wal-usage-looks-broken-in-modern...

[2] https://medium.com/@valyala/how-victoriametrics-makes-instan...

[ Parent | on HN ]