News for hackers. Stuff that matters.
Re:Databases at 14.4Mhz by dchichkov on Wednesday December 10, 2014 @02:34PM (#8731177)
I'm only familiar with other key-value storage engines, not FoundationDB, but it seems like the goals are: "distributed key-value database, read latencies below 500 microseconds, ACID, scalability".

I remember evaluating a few low latency key-value storage solutions, and one of these was Stanford's RAMCloud, which is supposed to give 4-5 microseconds reads, 15 microseconds writes, scale up to 10,000 boxes and provide data durability. https://ramcloud.atlassian.net/wiki/display/RAM/RAMCloud Seems like, that would be "Databases at 2000Mhz".

I've actually studied the code that was handling the network and it had been written pretty nicely, and as far as I know, it should work both over 10Gbe and Infiniband with similar latencies. And I'm not at all surprised, they could get pretty clean looking 4-5us latency distribution, with the code like that.

How does it compare with FoundationDB? Is it completely different technology?

Re:Databases at 14.4Mhz by jrallison on Wednesday December 10, 2014 @02:39PM (#8731210)
Don't know much about RAMCloud, but from the description:

"Many more issues remain, such as whether we can provide higher-level features such as secondary indexes and multiple-object transactions without sacrificing the latency or scalability of the system. We are currently exploring several of these issues."

Sounds it doesn't provide multi-key ACID transactions at the very least.

Re:Databases at 14.4Mhz by dchichkov on Wednesday December 10, 2014 @02:54PM (#8731284)
Yes, It looks like it doesn't support multi-key ACID transactions. As per:

https://ramcloud.atlassian.net/wiki/display/RAM/Deciding+Whe...

> If your application requires the higher-level data model features of a relational database, such as secondary indexes and transactions spanning multiple objects, then it may be difficult to get that application running on RAMCloud. The current RAMCloud data model is a key-value store with a few extension (such as conditional writes, table enumeration, and multi-read/multi-write operations); it does not yet support secondary indexes and multi-object transactions. Most of these operations can be implemented on top of the existing RAMCloud features, but it will take extra work and these operations will not run as fast as the built-in RAMCloud operations. We are currently working on adding higher-level features to RAMCloud.

Still. Aside from that, it is free, and open source and gives hundred times better latency?

Re:Databases at 14.4Mhz by misframer on Wednesday December 10, 2014 @03:04PM (#8731340)
> Still. Aside from that, it is free, and open source and gives hundred times better latency?

It doesn't have the same guarantees. If you need those guarantees, then there's no comparison and it doesn't make sense to compare latencies.

Re:Databases at 14.4Mhz by dchichkov on Wednesday December 10, 2014 @03:10PM (#8731375)
I'm not so sure. Have the latencies claimed by FoundationDB been measured on multi-key transactions?

And if you need these guaranties, it seems like "Most of these operations can be implemented on top of the existing RAMCloud features". I would expect it to be possible to implement that and stay well withing the latency budget, provided that a single-key durable write is 100 times faster.

Re:Databases at 14.4Mhz by nlavezzo on Wednesday December 10, 2014 @03:16PM (#8731410)
All of the transactions in the test noted in the blog are multi-key transactions. (20 key updates)
Re:Databases at 14.4Mhz by dchichkov on Wednesday December 10, 2014 @03:32PM (#8731495)
When claiming the 14.4MHz number, is that per 20-key update? Or a single key update in the 20-key update? And what's the latency on the 20-key update?
Re:Databases at 14.4Mhz by itp on Wednesday December 10, 2014 @06:15PM (#8732303)
The writes we are measuring are individual keys, modified atomically in transactions of 20 keys at a time. So this test is doing 14 million writes per second as 720 thousand transactions per second.
Re:Databases at 14.4Mhz by jrallison on Wednesday December 10, 2014 @03:25PM (#8731460)
Multi-key transactions means coordination. Coordination is not cheap. You'll be paying a hefty cost somewhere.
Re:Databases at 14.4Mhz by imaginenore on Wednesday December 10, 2014 @05:02PM (#8732000)
But it doesn't store the data on disk.

It should be compared to Memcached, not to a real storage engine.

Re:Databases at 14.4Mhz by dchichkov on Wednesday December 10, 2014 @07:18PM (#8732522)
Unlike Memcached or Redis, RAMCloud provides durability:

  * Automatic replication, crash recovery, and fail-over 
    (no loss of availability if a server fails)
  * Durability guarantee: data is always replicated 
    and durable before operations return, without
    significant performance hit (subject to the 
    requirement for persistent buffers on backups).
Re:Databases at 14.4Mhz by grogers on Wednesday December 10, 2014 @11:07PM (#8733144)
You can't make a durable write to disk in 15us. SSDs are around 2 orders of magnitude slower than that. 15us is about right for in memory on two nodes in a single data center though. Most people wouldn't consider that durable though.
Re:Databases at 14.4Mhz by imaginenore on Wednesday December 10, 2014 @08:58PM (#8732806)
So? Power goes out in your data center, all the data is lost.

Nothing stops you from replicating memcached.