Re:Stream and Go: News feeds for 300M users, built on RocksD...
by tbarbugli on Saturday January 20, 2018 @11:00AM
(#16193772)
We currently use the Raft Go implementation from Hashicorp (the article links to the Github project). We found Hashicorp implementation mature enough and easier to work with than the one from etcd.
Using Rocksdb with Go has indeed its own challenges because CGO calls are far from cheap and have an important consequences on Go applications. For performance reasons, we ended up moving some logic from Go to C in a few hot-spots (eg. moving the retrieval of large amount of keys from Go to C).
Disclaimer: I work for Stream and I am one the two co-founders
Re:Stream and Go: News feeds for 300M users, built on RocksD...
by StreamBright on Saturday January 20, 2018 @02:46PM
(#16195052)
Could you elaborate on Keevo? Could you compare it to Riak or Cassandra for example? Do you have any plans to open source it?
Re:Stream and Go: News feeds for 300M users, built on RocksD...
by espadrine on Saturday January 20, 2018 @05:23AM
(#16192859)
> since they mentioned that rocksdb is written in C++, I am interested to know the cgo issues
On that subject, is there a Badger[0] based distributed key-value store, equivalent to etcd, ready for production?
Re:Stream and Go: News feeds for 300M users, built on RocksD...
by xstartup on Sunday January 21, 2018 @08:40AM
(#16197986)
I wonder why people don't use badger over rocksdb.
Re:Stream and Go: News feeds for 300M users, built on RocksD...
by StreamBright on Saturday January 20, 2018 @02:45PM
(#16195043)
I disagree. I think it is great to see how people build stuff and everybody can get the conclusion. I am super interested in Keevo and hoping to get to know more. Rocksdb is a rock solid foundation (no pun intended), some of the most amazing developers are working on it. The only missing part is how to create a service on the top, sort of like LevelDB + Riak. Actually it would make sense to Riak guys to look into using RocksDB instead of LevelDB.
Re:Stream and Go: News feeds for 300M users, built on RocksD...
by xstartup on Sunday January 21, 2018 @08:41AM
(#16197991)
Why not badger since it's clearly superior to RocksDB in terms of performance and written in Go: https://github.com/dgraph-io/badger
On the Go side, since they mentioned that rocksdb is written in C++, I am interested to know the cgo issues they might had when using Go and rocksdb together, but of course they didn't mention that. Same goes for the GC stw pauses.
I strongly believe that articles like this largely used as soft ads materials without much detailed technical meat should be banned from Hack News. If it is all about "we built system X using building blocks Y and Z in language T!", what is the real value for readers?