Re:Migrating Messenger storage to optimize performance
by wenc on Tuesday June 26, 2018 @07:46PM
(#17404796)
Anyone know why Facebook moved off HBase? (article doesn't address this)
I get that MyRocks is truly amazing, but I'm wondering what issues they were facing with HBase. I heard HBase was picked over Cassandra (developed at FB) because it was strongly consistent vs Cassandra's eventual consistency.
Re:Migrating Messenger storage to optimize performance
by ryanobjc on Wednesday June 27, 2018 @02:10AM
(#17406087)
at the time they picked HBase because it was easier to repair missing replicas. Hdfs namenode juat schedules block replication. Cassandra (at least the version they were considering) had to work a lot harder at replacing a missing replica. Remember they had a version that didn’t have workable merkel trees.
I also heard they had an issue with network flaps causing unrepairable inconsistency. I never got the full scoop on this.
I had worked with the engineers directly responsible for the HBase choice back when. All water under the bridge now it seems.
Re:Migrating Messenger storage to optimize performance
by tomnipotent on Tuesday June 26, 2018 @08:14PM
(#17404927)
My guess is operational experience, though the article mentions it was necessary to take advantage of Lightning. I can't think of another business on the planet that has pushed MySQL (and RDBMS' in general) further than FB.
Re:Migrating Messenger storage to optimize performance
by midom on Tuesday June 26, 2018 @01:47PM
(#17402638)
+ app server / schema rework
Re:Migrating Messenger storage to optimize performance
by godelmachine on Tuesday June 26, 2018 @02:39PM
(#17403014)
Have they published a paper on this on Facebook Research?
Re:Migrating Messenger storage to optimize performance
by midom on Tuesday June 26, 2018 @02:51PM
(#17403114)
doesn't really qualify much as research - just lots of engineering to cover all corners
[dead]
Re:Migrating Messenger storage to optimize performance
by alexbeloi on Tuesday June 26, 2018 @05:48PM
(#17404254)
Yes they do/have research. Facebook had 12 papers at the last computer vision conference I attended (CVPR '17) and won the awards for best paper and best student paper.
Re:Migrating Messenger storage to optimize performance
by wenc on Tuesday June 26, 2018 @07:53PM
(#17404822)
They have a very strong research arm covering a significant number of research areas.
https://research.fb.com/
Re:Migrating Messenger storage to optimize performance
by tomnipotent on Tuesday June 26, 2018 @05:06PM
(#17403987)
You are aware that Facebook is the largest website on the planet?
[1] https://code.facebook.com/posts/989638804458007/introducing-...