Re:Faust: Stream Processing for Python
by vineetgoel1992 on Tuesday July 31, 2018 @06:44PM
(#17657961)
We have been using RocksDB for some time and it works fairly well. While we haven't seen any corruption yet, as you correctly pointed out, it is definitely a possibility.
We use RocksDB only as a cache and use log-compacted kafka topics as the source of truth. In the case of RocksDB corruption, we can simply delete the local rocksdb cache and the faust app should recover (and rebuild the rocksdb state) from the log compacted Kafka topic.
How swappable is the underlying DB from RocksDB to something else? Faust seems awesome, but I imagine in many corporate settings having multiple DB vendors in Production can be difficult.
I haven't used RocksDB. Does being a LevelDB fork give it similar corruption issues?