News for hackers. Stuff that matters.
Re:Migrating Messenger storage to optimize performance by tomnipotent on Tuesday June 26, 2018 @01:36PM (#17402564)
TL;DR - Moved from HBase to MyRocks engine on MySQL sitting on top of NVMe storage via their Lightning Server [1], which is a JBOF (just a bunch of flash) setup using x16 PCIe.

[1] https://code.facebook.com/posts/989638804458007/introducing-...

Re:Migrating Messenger storage to optimize performance by wenc on Tuesday June 26, 2018 @07:46PM (#17404796)
Anyone know why Facebook moved off HBase? (article doesn't address this)

I get that MyRocks is truly amazing, but I'm wondering what issues they were facing with HBase. I heard HBase was picked over Cassandra (developed at FB) because it was strongly consistent vs Cassandra's eventual consistency.

Re:Migrating Messenger storage to optimize performance by ryanobjc on Wednesday June 27, 2018 @02:10AM (#17406087)
at the time they picked HBase because it was easier to repair missing replicas. Hdfs namenode juat schedules block replication. Cassandra (at least the version they were considering) had to work a lot harder at replacing a missing replica. Remember they had a version that didn’t have workable merkel trees.

I also heard they had an issue with network flaps causing unrepairable inconsistency. I never got the full scoop on this.

I had worked with the engineers directly responsible for the HBase choice back when. All water under the bridge now it seems.

Re:Migrating Messenger storage to optimize performance by tomnipotent on Tuesday June 26, 2018 @08:14PM (#17404927)
My guess is operational experience, though the article mentions it was necessary to take advantage of Lightning. I can't think of another business on the planet that has pushed MySQL (and RDBMS' in general) further than FB.
Re:Migrating Messenger storage to optimize performance by midom on Tuesday June 26, 2018 @01:47PM (#17402638)
+ app server / schema rework
Re:Migrating Messenger storage to optimize performance by godelmachine on Tuesday June 26, 2018 @02:39PM (#17403014)
Have they published a paper on this on Facebook Research?
Re:Migrating Messenger storage to optimize performance by midom on Tuesday June 26, 2018 @02:51PM (#17403114)
doesn't really qualify much as research - just lots of engineering to cover all corners
[dead]
Re:Migrating Messenger storage to optimize performance by alexbeloi on Tuesday June 26, 2018 @05:48PM (#17404254)
Yes they do/have research. Facebook had 12 papers at the last computer vision conference I attended (CVPR '17) and won the awards for best paper and best student paper.
Re:Migrating Messenger storage to optimize performance by wenc on Tuesday June 26, 2018 @07:53PM (#17404822)
They have a very strong research arm covering a significant number of research areas. https://research.fb.com/
Re:Migrating Messenger storage to optimize performance by tomnipotent on Tuesday June 26, 2018 @05:06PM (#17403987)
You are aware that Facebook is the largest website on the planet?