News for hackers. Stuff that matters.
Re:Faust: Stream Processing for Python by peterwwillis on Tuesday July 31, 2018 @05:40PM (#17657465)
> Tables are stored locally on each machine using a superfast embedded database written in C++, called RocksDB.

I haven't used RocksDB. Does being a LevelDB fork give it similar corruption issues?

Re:Faust: Stream Processing for Python by vineetgoel1992 on Tuesday July 31, 2018 @06:44PM (#17657961)
We have been using RocksDB for some time and it works fairly well. While we haven't seen any corruption yet, as you correctly pointed out, it is definitely a possibility.

We use RocksDB only as a cache and use log-compacted kafka topics as the source of truth. In the case of RocksDB corruption, we can simply delete the local rocksdb cache and the faust app should recover (and rebuild the rocksdb state) from the log compacted Kafka topic.

Re:Faust: Stream Processing for Python by TuringNYC on Tuesday July 31, 2018 @08:54PM (#17658741)
How swappable is the underlying DB from RocksDB to something else? Faust seems awesome, but I imagine in many corporate settings having multiple DB vendors in Production can be difficult.