News for hackers. Stuff that matters.
Re:A NoSQL Database with ACID Transactions by krenoten on Monday March 04, 2013 @12:10PM (#5319163)
The Flow language they implemented for this is interesting. http://www.foundationdb.com/white-papers/flow/

I think we are going to see eventual consistency deflate as people relearn to love transactions. SQL also makes users of a database a lot more productive. The bottom line is you don't want your devs spending time and effort reasoning about database behavior, you want them focusing on business logic. In 5 years hopefully the world will catch up a bit with Google's Megastore and Spanner.

Re:A NoSQL Database with ACID Transactions by Dave_Rosenthal on Monday March 04, 2013 @12:29PM (#5319279)
FoundationDB founder here. Flow sounds crazy. What hubris to think that you need a new programming language for your project? Three years later: Best decision we ever made.

We knew this was going to be a long project so we invested heavily in tools at the beginning. The first two weeks of FoundationDB were building this new programming language to give us the speed of C++ with high level tools for actor-model concurrency. But, the real magic is how Flow enables us to use our real code to do deterministic simulations of a cluster in a single thread. We have a white paper upcoming on this.

We've had quite a bit of interest in Flow over the years and I've given several talks on it at meetups/conferences. We've always thought about open-sourcing it... It's not as elegant as some other actor-model languages like Scala or Erlang (see: C++) but it's nice and fast at run-time and really helps productivity vs. writing callbacks, etc.

(Fun fact: We've only ever found two bugs in Flow. After the first, we decided that we never wanted a bug again in our programming language. So, we built a program in Python that generates random Flow code and independently-executes it to validate Flow's behavior. This fuzz tester found one more bug, and we've never found another.)

Re:A NoSQL Database with ACID Transactions by gngeal on Monday March 04, 2013 @01:17PM (#5319553)
"FoundationDB founder here. Flow sounds crazy. What hubris to think that you need a new programming language for your project? Three years later: Best decision we ever made."

I don't think that should come as a surprise to anyone. In the long run, language-oriented approach seems to be the best approach to handling the intricacies of complex software systems. The only thing stopping it seems to be a certain lack of skills and experience on part of most developers, and the inadequacy of current language-building tools. (By any chance, are you familiar with the "let's simplify language construction"-related work of people at vpri.org?)

Re:A NoSQL Database with ACID Transactions by gruseom on Monday March 04, 2013 @01:22PM (#5319595)
How does the Python program know what the random Flow code is supposed to do?
Re:A NoSQL Database with ACID Transactions by Dave_Rosenthal on Monday March 04, 2013 @02:16PM (#5320094)
Hmm, that's the tricky bit to explain in a quick post. It has an independent code interpreter that works on the internal representation of the fuzzed code. The output of that internal representation in fed through the Flow compiler (then C++). There is basically a "check([random number])" statement on every other line of code which should be reached in a known order. We compare the check log of the fuzz tester's interpreter to the check log of the compiled code.
Re:A NoSQL Database with ACID Transactions by gngeal on Monday March 04, 2013 @02:19PM (#5320123)
I guess that the Python program generates some sort of internal AST that is either trivial or at least easier to interpret than the fuzz code (enhanced C++, blah...) generated from the AST.
Re:A NoSQL Database with ACID Transactions by rzidane on Tuesday March 05, 2013 @01:28AM (#5323249)
I read the performance analysis comparing it to Go, Erlang etc. Has any comparison been made with Akka (http://akka.io)? How much would this extra efficiency matter when the communicating processes are on different machines? Since speed in communicating between local processes is important to you guys, I am assuming that the actor model is then used for all local and remote communication?
Re:A NoSQL Database with ACID Transactions by EugeneOZ on Monday March 04, 2013 @04:04PM (#5320923)
"4GB RAM" in hardware requirements.. Why? It's not Java, it's C++, so why??? 4GB just to store definition of empty storage?

Is Debian 6 supported?

Re:A NoSQL Database with ACID Transactions by melchebo on Tuesday March 05, 2013 @05:29AM (#5323814)
Reminds me a bit of OkCupid's Tamer : https://github.com/maxtaco/tamer

Though I believe they go less all out on the actor model, but just focus on concurrent processing, essentially using futures.

Re:A NoSQL Database with ACID Transactions by andrewflnr on Monday March 04, 2013 @02:00PM (#5319936)
At first I thought you were talking about http://www.flowlang.net/p/introduction.html, a different Flow language designed around data flow to allow automatic parallelization.
Re:A NoSQL Database with ACID Transactions by awjr on Monday March 04, 2013 @09:30AM (#5318323)
It's one of the 'bug bears' I find with new technologies where they try and use it for something it wasn't really designed for. The most recent being the idea of doing e-commerce with a no-sql db (mostly MongoDB). You are just using the wrong type of DB for this.

It's nice to find a NoSQL solution that addresses this issue.

Re:A NoSQL Database with ACID Transactions by VexXtreme on Monday March 04, 2013 @10:14AM (#5318536)
A big part of the problem are developers who think that every new technology is a golden hammer and who try to shoehorn those same technologies into roles they are not well suited for.

A pretty obvious example is people using Redis and Mongo for applications that would really benefit the most from relational databases. If you have a highly relational model that would benefit from things like referential integrity and normalization then use a relational database. If you need a massive highly scalable KVP database, use NoSQL. I don't see why people try to use a single approach as a solution to every problem.

I thought this video about NoSQL fanboys was really hilarious: http://highscalability.com/blog/2010/9/5/hilarious-video-rel...

Re:A NoSQL Database with ACID Transactions by gav on Monday March 04, 2013 @11:49AM (#5319046)
I think it's more of an issue that developers consider their options to be SQL or NoSQL when the option of using both often is a great choice. You can pick the best technology to solve individual aspects of the system.

For example, the current system I'm working on uses Oracle and Mongo, the previous used SQL Server and Redis. The main theme is to use a relational database for the structured data that we need transactional support for, e.g. orders, then use a NoSQL data for unstructured data and caching.

Re:A NoSQL Database with ACID Transactions by jt2190 on Monday March 04, 2013 @10:54AM (#5318768)

  > If you have a highly relational model that would benefit
  > from things like referential integrity and normalization
  > then use a relational database. If you need a massive 
  > highly scalable KVP database, use NoSQL. I don't see why 
  > people try to use a single approach as a solution 
  > to every problem.
Part of the problem is that people present this as a choice between relational and non-relational models, when the choice is actually about how the database will scale when the data set gets very large.
Re:A NoSQL Database with ACID Transactions by obviouslygreen on Monday March 04, 2013 @11:49AM (#5319047)
You're turning one false dichotomy into... I don't know the word for it, but while how it scales can be part of choosing a database system, it definitely is not the main difference between the relational and noSQL models. How your data storage will scale is an important but tangential topic that should be addressed as complementary to deciding which class of storage will help you most effectively and efficiently model your data.
Re:A NoSQL Database with ACID Transactions by jt2190 on Monday March 04, 2013 @01:54PM (#5319885)

  > ...[database scaling is] definitely is not the main
  > difference between the relational and noSQL models.
The post I was responding to strongly implies that one simply has to decide whether they want to use a relational model or not. I contend that framing the question this way is contributing to the misunderstandings of what the advantages and disadvantages of the popular noSQL databases are, which become very apparent when one thinks of what happens when the data set gets very large. Relational databases don't work nearly as well when you have to split the schema across multiple databases, and you can no longer use the database provided transactions or joins.
Re:A NoSQL Database with ACID Transactions by papsosouid on Monday March 04, 2013 @12:06PM (#5319140)
Except that virtually everyone can easily have everything in a single, fully normalized relational database on a single server. People have convinced themselves that relational databases can't scale, based on nothing at all.
Re:A NoSQL Database with ACID Transactions by giulianob on Monday March 04, 2013 @10:51AM (#5318747)
The issue I see the most is that NoSQL has become synonymous with distributed databases which is really where making ACID promises becomes a challenge. If you had a single node document DB, it probably wouldn't be very challenging to make it ACID. SQL or not, as soon as you want to make it automatically shard and distributed, then you will suffer large performance impacts with transactions. Are there any single node document DBs which provide ACID guarantees out there? I know Redis can do it with multi/exec but Redis isn't really document oriented though it's an awesome piece of software.
Re:A NoSQL Database with ACID Transactions by misframer on Monday March 04, 2013 @10:54AM (#5318763)
Redis doesn't support transaction rollbacks, so they're not really "atomic."
Re:A NoSQL Database with ACID Transactions by giulianob on Monday March 04, 2013 @11:29AM (#5318950)
They do have DISCARD so if you call MULTI and want to bail out you can say DISCARD. What they don't have is automatically rolling back if there's a failure while executing commands. However, failures in Redis are mainly going to happen if you issue a bad command (something you should catch during development). IIRC, if there's a power failure or something happens before the transaction finishes, Redis is able to discard the transaction during startup.
Re:A NoSQL Database with ACID Transactions by lucian1900 on Tuesday March 05, 2013 @03:43AM (#5323588)
True. For example, CouchDB is ACID, as long as you consider versioned data equivalent to transactions.
Re:A NoSQL Database with ACID Transactions by stephenpiment on Monday March 04, 2013 @10:04AM (#5318486)
Transactions are great not only for e-commerce but for a broad range of applications. Take a look at our Transaction Manifesto: http://www.foundationdb.com/white-papers/the-transaction-man...
Re:A NoSQL Database with ACID Transactions by mercurial on Monday March 04, 2013 @01:22PM (#5319602)
I like the emphasis on transactions. However, the fact that it doesn't support transactions longer than 5s and puts the onus on the client to enforce is kind of a turn-off. How is the client supposed to know how long the server will take for a given operation at a random load?
Re:A NoSQL Database with ACID Transactions by itp on Monday March 04, 2013 @02:07PM (#5320012)
Our performance page (http://foundationdb.com/performance/) shows how we've worked very hard to provide predictable and low latencies, even at saturating workloads [1] (on the order of 1ms for a read and 10ms for a commit). This means that transactions with 50 serial reads are still going to complete in two orders of magnitude less time than the maximum transaction duration (and futures make it easy to parallelize most reads). This is more than enough for client operations -- regardless of your database, you usually don't want to have a user wait for 5s, hold a lock for 5s, or be subject to conflicts for 5s.

[1] Under saturating load, FoundationDB will queue transactions before assigning them a read version (starting the 5 second window), so that latencies within the transaction stay low. This explicit queuing also makes it easy to prioritize transactions, so you can mix latency-sensitive and saturating batch workloads safely.

Re:A NoSQL Database with ACID Transactions by mercurial on Monday March 04, 2013 @02:32PM (#5320249)
Thanks for the quick answer. Regarding long transactions, it depends on what you are doing. Say you want to change your schema, you won't be able to do this in a single transaction (something which, eg, Postgres lets you do).
Re:A NoSQL Database with ACID Transactions by makmanalp on Monday March 04, 2013 @10:42AM (#5318682)
A bit of a tangent but does anyone have any info / experience on the similar NuoDB (NoSQL + ACID), both in terms of functionality and in general?

http://nuodb.com/

Does this compare?

Re:A NoSQL Database with ACID Transactions by andrewflnr on Monday March 04, 2013 @01:53PM (#5319867)
What? The page you linked says "100% SQL".
Re:A NoSQL Database with ACID Transactions by SeanKilleen on Monday March 04, 2013 @10:21AM (#5318575)
I see C, node, ruby, and Java support...any plans for a .NET API as well? This seems like something that would be nice to mess around with there.

Would also be nice to understand what the intended pricing model will be (if any). If the plans are to keep this free, that would be pretty great.

Re:A NoSQL Database with ACID Transactions by egeozcan on Monday March 04, 2013 @10:26AM (#5318601)
"We will offer FoundationDB as both a free community edition and a licensed version with support and larger cluster capability. The community edition will include the full capabilities of FoundationDB and will allow production deployment.

FoundationDB licenses will have reasonable and linear pricing. The license cost for a cluster, including support, will be similar to the operational cost of the commodity hardware in the cluster."

From: http://www.foundationdb.com/faq/

Re:A NoSQL Database with ACID Transactions by EugeneOZ on Monday March 04, 2013 @03:53PM (#5320838)
"You may not deploy or use the Software in a production environment" - from TOS http://www.foundationdb.com/BetaLicenseAgreement.pdf
Re:A NoSQL Database with ACID Transactions by Dave_Rosenthal on Monday March 04, 2013 @10:29AM (#5318619)
.NET and PHP are now our most-requested APIs on our community site; I think it's safe to say that those will be coming. We've already had a couple of people whip up quick .NET wrappers on top of our C APIs themselves.
Re:A NoSQL Database with ACID Transactions by ibejoeb on Monday March 04, 2013 @03:24PM (#5320647)
@Dave_Rosenthal can you address the licensing? I'd consider porting a nontrivial application if the terms were agreeable. I'm a little confused, though.

First:

"FoundationDB Beta 1 is ready for production use."

Then:

"The community edition will include the full capabilities of FoundationDB and will allow production deployment."

Finally:

"FoundationDB grants you a...revocable license...for test purposes only in a non-production environment"

Re:A NoSQL Database with ACID Transactions by Dave_Rosenthal on Monday March 04, 2013 @03:54PM (#5320846)
Yes, thanks. You are correct, the "Beta Evaluation License" only covers evaluation use, not production. Our "GA" release will include transparent pricing and a free community license for smaller clusters.

If you are interested in using FoundationDB Beta in production today (as some of our customers are) you should give us a call. We can get you a license for production use and support at a very reasonable cost.

Re:A NoSQL Database with ACID Transactions by rcknight on Monday March 04, 2013 @10:31AM (#5318628)
RavenDB is also ACID I beleive? The foundation site makes it sound like they are the only one.
Re:A NoSQL Database with ACID Transactions by Dave_Rosenthal on Monday March 04, 2013 @10:40AM (#5318670)
FoundationDB founder here. We think that it's awesome that RavenDB also supports multi-node ACID transactions. However, on page 9 of https://s3.amazonaws.com/daily-builds/RavenDBMythology-11.pd... they warn against relying on this capability:

"RavenDB supports multi document (and multi node) transactions, but even so, it isn’t recommended for common use, because of the potential for issues when using distributed transactions."

So, the differentiator is that FoundationDB is built from the ground up to support these type of transactions at high performance levels with no "potential issues".

Re:A NoSQL Database with ACID Transactions by Maarten88 on Monday March 04, 2013 @02:48PM (#5320363)
That remark seems a bit unfair. That document is two years old, from long before version 1.0. RavenDB is now at version 2.0. It was also built with transaction support from the start. The default setting for TransactionMode is Safe (http://ravendb.net/docs/server/administration/configuration) so most users will use that, if there were any issues with it that would certainly be known by now.
Re:A NoSQL Database with ACID Transactions by fry_the_guy on Monday March 04, 2013 @03:27PM (#5320681)
The current RavenDB documentation still warns against using "system transactions" because of performance reasons (http://ravendb.net/docs/client-api/advanced/transaction-supp...), so I think it is still fair to say that RavenDB was not designed for applications requiring high performance cross node transactions.
Re:A NoSQL Database with ACID Transactions by Maarten88 on Monday March 04, 2013 @04:02PM (#5320904)
System.Transactions refers to support by RavenDB of the Distributed Transaction Coordinator service on Windows. This makes it possible to enlist transactions in multiple systems (MSSQL, MSMQ, RavenDB, NServiceBus) in one single transaction with the possibility of a rollback. It is not needed for transactions inside RavenDB, and I think foundationdb does not even support something like that (it is Windows specific, although Mono supports it on other platforms).
Re:A NoSQL Database with ACID Transactions by rcb on Monday March 04, 2013 @12:57PM (#5319417)
Very nice, and congratulations. Reading the "Data Modeling" documentation brings to mind some similarities with the MUMPS storage model (Intersystems Caché / GlobalsDB / GT.M).. is this a correct observation? Thanks.
Re:A NoSQL Database with ACID Transactions by Dave_Rosenthal on Monday March 04, 2013 @02:05PM (#5319978)
Absolutely. Good eye.
Re:A NoSQL Database with ACID Transactions by toolslive on Tuesday March 05, 2013 @02:56AM (#5323468)
I searched the site a bit and saw no information whatsoever on what is behind it, so I'm going to guess:

For the local store, probably something like an append-only (aka copy on write aka persistent) B-tree-ish data structure. This guarantees good behaviour on SSDs, and cheap/easy transactions. It can also be scaled linearly over multiple spindles.

For the distributed aspect, I guess they use some paxos variation (I guess this from the separation between logging the transaction and durably applying this)

But again: I am merely guessing.

With regard to flow: Is it just me, or does it look like a poor man's concurrency monad.

Re:A NoSQL Database with ACID Transactions by xb95 on Monday March 04, 2013 @03:57PM (#5320874)
I really want to be excited by this.

For better or worse, after being in this industry for a long time now all I can say about a new database launch is: cool, I'd love to talk to you in 3-5 years when you've had time to work out the hinks and people have figured out what your strengths and weaknesses are and where the real gotchas will getcha.

Until then: good luck. A distributed, linearly scalable, easy-to-admin NoSQL database with transaction support and full ACID would be delicious indeed.

Re:A NoSQL Database with ACID Transactions by Dave_Rosenthal on Monday March 04, 2013 @04:36PM (#5321144)
I totally understand this perspective. FWIW, we're trying to be as open as we can about those "gotchas". You might be interested in our:

- Known limitations (http://foundationdb.com/documentation/beta1/known-limitation...)

- Anti-features (http://foundationdb.com/white-papers/anti-features/)

- Performance considerations (http://foundationdb.com/documentation/beta1/developer-guide....)

Talk to you in a few years :)

Re:A NoSQL Database with ACID Transactions by xb95 on Wednesday March 06, 2013 @09:08PM (#5335516)
FWIW, those documents are a fantastic start and contain the sort of things that I wish I had known about MongoDB and Riak before embarking on quests to use them.

I hope you do well -- the world really needs a good amalgamation of strong data integrity (ACID, transactions) and the ease-of-life that distributed databases generally give you. Since I don't work in an industry where I need microsecond reads, I'm perfectly happy to pay the cost of "it might take 10s of milliseconds to commit this" if I can avoid having to go down the "okay, now I have to shard my project" path for an eighth time. :-)

Re:A NoSQL Database with ACID Transactions by wbl on Monday March 04, 2013 @10:19AM (#5318559)
MVCC does not provide serializability of transactions without a bit more work. Since when can you get away without giving guaranties about what you support?
Re:A NoSQL Database with ACID Transactions by Dave_Rosenthal on Monday March 04, 2013 @10:32AM (#5318636)
I agree that MVCC does not itself guarantee transaction isolation, but FoundationDB does indeed do the "bit more work" to guarantee all of the ACID properties.
Re:A NoSQL Database with ACID Transactions by wbl on Tuesday March 05, 2013 @09:18PM (#5328897)
Ah, I found the page where it claims serializability. http://www.foundationdb.com/documentation/beta1/developer-gu...
Re:A NoSQL Database with ACID Transactions by JoelJacobson on Monday March 04, 2013 @12:58PM (#5319423)
Any plans on releasing an SQL-layer to allow querying FoundationDB using normal SQL? Would be nice with both SQL and ACID in the same database.
Re:A NoSQL Database with ACID Transactions by Dave_Rosenthal on Monday March 04, 2013 @02:42PM (#5320324)
No plans, but that would be nice. Others like Clustrix and NuoDB are more directly targeting the distributed SQL database market. Often we describe FoundationDB's core to people as more of a "storage substrate" than a database. It is possible to build an efficient SQL database as a layer on top of that substrate. For example, SQLite4 is choosing a transactional ordered key-value abstraction for its internal storage engine (which exactly matches FoundationDB's API). Of course, a SQL database is a big project involving a lot more than just a storage engine!
Re:A NoSQL Database with ACID Transactions by colinhowe on Monday March 04, 2013 @10:03AM (#5318473)
Took me a while to find how FoundationDB thinks about data... it's an ordered key-value store with nice scaling properties.
Re:A NoSQL Database with ACID Transactions by EugeneOZ on Monday March 04, 2013 @02:12PM (#5320058)
"APIs for C, Python, Ruby, Node.js, and Java"

And where is PHP?

Re:A NoSQL Database with ACID Transactions by itp on Monday March 04, 2013 @02:23PM (#5320177)
PHP and .NET are the clear front-runners when we asked this question on our community site, so while I can't commit to any specific timeline, it's safe to say we'll be adding them soon.
Re:A NoSQL Database with ACID Transactions by wildchild on Tuesday March 05, 2013 @05:03AM (#5323764)
Too complicated to just download it. I'll stay postgres instead.
Re:A NoSQL Database with ACID Transactions by loeg on Monday March 04, 2013 @03:28PM (#5320684)
NoSQL YesACID LBJ IRT USA LSD LSD LBJ FBI CIA LSD LBJ…
Re:A NoSQL Database with ACID Transactions by goloxc on Monday March 04, 2013 @02:26PM (#5320196)
could not have been more disappointed