Re:SQL Databases Don't Scale
by stuff4ben on Monday July 06, 2009 @10:34PM
(#690811)
I thought it was a good summary of the problems facing SQL databases as organizations grow. However I would have appreciated some solutions to the problem.
[ Parent
| on HN ]
I think one of the points made at the end was that there kinda isn't one - people have been trying to come up with the perfect solution to scaling RDBMS's for ever and the current state of the art (pretty much) is as described in the article - i.e, not great.
People with really massive MySQL setups (eBay, for example) basically just do a huge amount of the stuff described here, but with lots of message-queue type stuff to glue it together and loads of hard work; i.e. you can make it scale but it's really really hard work because RDBMS's aren't inherent scalable.
the current state of the art (pretty much) is as described in the article - i.e, not great
That simply isn't true.
For me, thousands of transactions per second and 10s of terabytes of data on a single database is normal. It's unremarkable, it's everyday, it's what we do, we have done it for years. And I know of installations handling 10x that. It's only people who's only "experience" is websites that whinge about how RDBMS can't handle their tiny datasets.
What hardware is this running on, roughly? What the author was getting at, I think, was not that you can't do it, just that it's hard/expensive; you need fairly beefy hardware and experienced DBA's to manage horizontal scaleout manually.
Re:SQL Databases Don't Scale
by DanielBMarkham on Tuesday July 07, 2009 @07:16AM
(#691283)
If that were the case, then I think the concluding sentences should be changed.
When hundreds of companies and thousands of the brightest programmers and sysadmins have been trying to solve a problem for twenty years and still haven’t managed to come up with an obvious solution that everyone adopts, that says to me the problem is unsolvable.
That's a very curious statement, especially for HN. Why would you consider needing good, experienced people to be a disadvantage?
Because that kind of hardware+DBA team is prohibitively expensive for a startup?
Hardware yes, but the basic premise of a startup is that your people are top-notch. Scalability doesn't imply building something up-front that can handle enormous loads - it means building something that can grow with you. As opposed to "oh crap, we've got some load now, better start again" a la Twitter.