jsr
MongoDB Hadoop Connector goes 1.0
1) They used Infiniband interconnects. Running on ethernet is likely to yield less impressive results.
2) Their benchmark does simple primary key lookups. If you start doing joins or transactions that need to hit multiple data nodes, things will slow down. Depending on your workload, this may or may not be an issue.
3) NDB is an in-memory storage engine, so you're limited to the aggregate RAM in your cluster for max storage size.
4) AFAIK, MySQL Cluster doesn't re-balance, you need to pre-determine how data is partitioned and changing it at runtime is hard. I don't know if this has changed in the later releases.
Why are software development task estimates regularly off by a factor of 2-3?
Top job trends for 2011
Cloudera raises $40M series-D
10gen is the primary sponsor of the MongoDB open source database. We're hiring rapidly in all roles and we'd love to hire you!
Current list of openings here: http://jobvite.com/m?3dFtbfwg
Configuration mistake takes down Microsoft Cloud
The hilarious product market fit of U2
Transactions in MongoDB
Last chance: send your name to mars
Getting started with Play Framework, Scala and MongoDB/Casbah
Tutorial: VMware Cloud Foundry + Rails + MongoDB
MongoDB on EC2 and EBS
Great online curation of systems papers
Vertical scaling of a DB is definitely an option for many people and has been used to scale many applications. However, the cost curve associated with buying bigger and bigger hardware is super-linear; doubling CPU & Memory in a single system leads to more than doubling the hardware cost. This can be problematic for many businesses whose revenue growth is exceeded by cost growth of the database.
Sharding is also an option for scaling, leveraged to great success by Facebook, Yahoo, and many others. However as the article points out, sharding prevents the developer from using many of the features that make a relational database a productive development environment. There are lots of foot guns that emerge in a sharded SQL environment and if you have not set up your development constraints appropriately, you can slow the pace of development considerably. This again leads to a cost problem because the incremental costs of adding features grows as you add more things like sharding around your database.
SQL is not useless and not hopeless. In a large number of cases, SQL is the right solution. However the techniques used to scale SQL tend to be options only to very large budget organisations. NoSQL solutions tend to be more cost effective in their scaling approach (scale out vs. scale up) without crippling the developers productivity. For these reasons, NoSQL solutions tend to be the better choice for the cost-conscious.