On moving from CouchDB to Riak

Re:On moving from CouchDB to Riak by philwise on Monday March 07, 2011 @01:25PM (#2298111)

"We store a lot of data... 2TB"

Given that a pair of 2TB drives is less than $250 on ebuyer right now, 2TB of data is not 'big data'. You could comfortably stuff that in any decent database (SQL Server for example, I'm sure PostgreSQL would work too).

Just because a tiny machine on slicehost isn't big enough doesn't mean that your data won't fit in a normal database.

[ Parent | on HN ]

Re:On moving from CouchDB to Riak by defroost on Monday March 07, 2011 @02:07PM (#2298272)

> You could comfortably stuff that in any decent database (SQL Server for example, I'm sure PostgreSQL would work too).

You may have noticed that the OP's requirements were: * a REST interface * sharding The costs of SQL Server are unbelievably high, and which you can do horizontal partitioning and sharding, Riak is designed to be used in this manner. On the other hand, re: SQL Server according to this article the "horizontal partitioning feature requires Enterprise or Datacenter licenses which have a retail price of 27,495 and 54,990 per processor." I think the OP made a wise choice. http://www.infoq.com/news/2011/02/SQL-Sharding

[ Parent | on HN ]

Re:On moving from CouchDB to Riak by JoachimSchipper on Monday March 07, 2011 @03:19PM (#2298500)

You don't necessarily need to shard if the data fits comfortably on a standard disk... and the parent did mention PostgreSQL.

[ Parent | on HN ]

Re:On moving from CouchDB to Riak by grncdr on Monday March 07, 2011 @05:59PM (#2299087)

Sharding (for free) on PostgreSQL is surprisingly painful

[ Parent | on HN ]

Re:On moving from CouchDB to Riak by JoachimSchipper on Tuesday March 08, 2011 @04:32AM (#2300369)

I thought people just built sharding into the application level when using MySQL stuff - I'd imagine the same thing works just fine for PostgreSQL. Or were you talking about something else?

[ Parent | on HN ]

Re:On moving from CouchDB to Riak by prodigal_erik on Tuesday March 08, 2011 @05:02AM (#2300425)

Anyone who volunteers for app level sharding is going to find themselves deep in Special Hell. Step one is to rewrite all your queries so they no longer expect really exotic use cases like "SELECT ... WHERE" to actually work until you somehow figure out which nodes to run them on (and if it was supposed to be a join between entities crossing shards, good luck with that). Step two is to find working XA-aware drivers (haha). For step three you need a knife and a goat....

Apparently really high-end clustered databases solve this problem correctly, i.e., the schema is extended to specify where any record can be found, the cluster uses that to work out a minimally-stupid query plan, and record->shard mapping becomes merely a tuning decision. I've never had the opportunity to work with one. But I'm not bitter.

[ Parent | on HN ]

Re:On moving from CouchDB to Riak by JoachimSchipper on Tuesday March 08, 2011 @08:41AM (#2300785)

Actually, if you use MySQL as a "great hashtable in the sky (cloud)", it's not all that much more painful to shard things. That said, if you do that you're already in a state of sin/pain...

[ Parent | on HN ]

Re:On moving from CouchDB to Riak by philwise on Tuesday March 08, 2011 @11:59AM (#2301584)

'Sharding' and 'a rest interface' are not really business requirements. Sharding is only a solution to handling certain types of high load, and REST is but one solution for IPC.

[ Parent | on HN ]

Re:On moving from CouchDB to Riak by erikpukinskis on Monday March 07, 2011 @05:37PM (#2299004)

Just because your data will fit in a normal database doesn't mean it will be fast enough.

[ Parent | on HN ]