Twitter, PayPal reveal database performance

Re:Twitter, PayPal reveal database performance by dkhenry on Friday October 05, 2012 @08:00AM (#4616486)

I always knew the company I work for does cool things with MySQL, but I never knew how far we really have pushed the envelope until I see how much more we push MySQL then the twitters and facebooks of the world. Doing some napkin math I would say our largest cluster of MySQL does about 320 Billion inserts a Day across 80 nodes and some of our nodes can peak at about 200K queries per second ( that would be 16 Million QPS if all the nodes were fully loaded ). Meaning at one of our installs we do the same volume as two facebooks !

[ Parent | on HN ]

Re:Twitter, PayPal reveal database performance by ddorian43 on Friday October 05, 2012 @09:39AM (#4616801)

Curious what type of servers, what type of writes, do you use compression, how you shard, what type of indexes? Write an article for highscalability and get exposure?

[ Parent | on HN ]

Re:Twitter, PayPal reveal database performance by ericcholis on Friday October 05, 2012 @08:04AM (#4616496)

Out of curiosity, what industry do you work in?

[ Parent | on HN ]

Re:Twitter, PayPal reveal database performance by dkhenry on Friday October 05, 2012 @08:07AM (#4616511)

Performance Monitoring.

[ Parent | on HN ]

Re:Twitter, PayPal reveal database performance by aristus on Friday October 05, 2012 @10:33AM (#4617024)

Coincidently I work on performance monitoring at Facebook. We gave up logging perf data to MySQL because it was a management headache at scale. So instead my colleagues built Scuba, which is essentially a search engine they taught to perform aggregate functions.

http://gigaom.com/cloud/exclusive-facebooks-scuba-project-di...

[ Parent | on HN ]

Re:Twitter, PayPal reveal database performance by dkhenry on Friday October 05, 2012 @11:12AM (#4617201)

Lots of people gave up trying to do their own performance logging, it turns out its a non trivial problem when you get to large scale systems. Which is why the company I work for exists.

[ Parent | on HN ]

Re:Twitter, PayPal reveal database performance by aristus on Friday October 05, 2012 @11:39AM (#4617296)

Does the name start with New and end with Relic, by chance? :) I'd love to compare notes sometime. My email's in my profile.

[ Parent | on HN ]

Re:Twitter, PayPal reveal database performance by dkhenry on Friday October 05, 2012 @11:52AM (#4617354)

No, but I understand why you might think that. I sent you an E-mail.

[ Parent | on HN ]

Re:Twitter, PayPal reveal database performance by nc17 on Friday October 05, 2012 @01:22PM (#4617757)

Your company is easy to find on Google. It's not stealth, and it's venture-funded. Curious why you can't or won't mention the name here.

[ Parent | on HN ]

Re:Twitter, PayPal reveal database performance by dkhenry on Friday October 05, 2012 @03:32PM (#4618321)

I am in no way an official spokesperson for them, just a developer, I don't know how much publicity I am allowed to draw to our back end architecture without getting into trouble. It is after all the thing that separates us from everyone else out there. (BTW I have posted the Who's Hiring post a couple times so my Username is pretty easy to track to whom I work for)

[ Parent | on HN ]

Re:Twitter, PayPal reveal database performance by ericcholis on Friday October 05, 2012 @08:09AM (#4616523)

Ah, that'll do it.

[ Parent | on HN ]