Bullshit, and I will elaborate.
Oracle has two "scalable offerings:"
RAC relies on a single large san to "scale."
Exadata relies on super-fast interconnects to get multiple machines to look more like a single huge one, which has the obvious speed-of-light limitations as well as the price one.
Neither approach is the kind of scale out with commodity hardware that something like Cassandra gives you.
Invoking the mythical "pay Oracle enough money and they will make it scale" mantra is frequently done by people who either don't know better or are deliberately muddying the water, but that doesn't make it right.
Did you not understand what I wrote, or do you think that "large san" or "fast interconnects limited to machines in the same couple of racks" count as "scaling?"
When it comes to scaling, there's no rule which says you have to scale in any particular way. Many database customers consider scaling _up_ a form of scaling, and probably 99% of the world's database users will never go beyond what can be achieved on a single machine. They don't care about your pet project, about how the way it works is cooler and technologically purer. They consider databases a tool, and they don't really care how it works. I don't think too deeply about how a can opener works. I'm able to open more cans faster with a more powerful electric can opener. Even scaling up is still scaling.
There are customers with bigger needs. For some of them, SAN based scaling is just what they need; more IOPS let their database keep running and they can get on with their lives. If they need more IOPS, they add more drives to their SAN. Scaling with a SAN is still scaling.
Some customers have more complex demands, and look to Exadata or Netezza. They might start with just a few nodes, and add more nodes as their load increases. It's still scaling.
Now, we in CompSci circles get excited about scaling using clusters. That's _sexy_ scaling. Cassandra and FathomDB can both change the rules of the game, and I'm excited about what FathomDB can do here, just as I can see you're passionate about Cassandra. But let's not pretend that most customers really care about how it scales; they care about what we can do. To customers, if it looks like a duck, and it quacks like a duck, it is a duck. But when they're choosing a database, scaling is not the only requirement, and certainly scaling in a certain way is unlikely to be on their list of requirements. If you've ever read an RFP, there's a bewildering number of questions that have nothing to do with technology at all, and often the technology section is a depressingly short list. Although part of the RFP game is to try to get the purchaser to write in requirements that only you can deliver, purchasers are considerably more savvy than that.
You can jump up and down like Rumpelstiltskin arguing that yours is the only database that's _really_ scaling and the Oracle solution didn't meet the requirements, and that they should have chosen you. At the end of the day all you're left with is one person jumping up and down screaming about the rules of the game, and a customer that got what they needed and an Oracle salesperson that earned their commission.
Instead of curling up in semantics discussions, how about simply answering a few of those? I think you could do that without revealing anything about the magic sauce of your product.
To reiterate:
* Does your system support the full SQL vocabulary, including all join types?
* Is it ACID?
* Do I really not have to arrange my data in any special way (schema-, or partition-wise)?
* Does it really scale near-linear, regardless of the workload that I apply?
* Why do you bother with MySQL-Hosting as a secondary product if your new db scales down just fine?
One I can definitely answer: yes, full SQL vocabulary support (though we haven't implemented everything yet!)
Oh well, a simple yes or no would be fine, really.
However, I guess it's safe to assume then, that you're not ACID and that data will have to be rearranged to accomodate your system. That's my take-home because a simple "yes" to either question would not have revealed anything.
Well, thing is, part of your claim (if I understood it correctly) is that I just throw my data and my queries at your wonder-device and it will magically be fast and scale. You never mention any special care I must take, you even explicitly say it's not partitioning.
This is decidedly not the case with what said Oracle salesperson will sell to me. The sale will normally include an expensive support-contract and probably a dedicated DB Admin. Because operating a RAC- or Exadata Cluster is, for all I know, far from trivial. You still have to make a bunch of critical decisions and tradeoffs upfront or it will probably suck.
Moreover you're effectively saying that you solved what Amazon and Google either couldn't or didn't bother to solve with their SimpleDB and Appengine Datastore offerings.
Still sounds like quite a bold claim to me. But as said elsewhere, I appreciate that you're standing by it and will keep looking forward to what you will unveil.
Just one more thing that I really don't understand: If you have this doomsday device at your hands, then why on earth do you bother with MySQL hosting as a secondary product?
The relational model != ACID. The relational model != SQL. The relational model has nothing in it implicitly that ruins it's ability to obey CAP. You are right, that you will have to make some consistency tradeoffs to get partition tolerance. Again, to repeat, this has nothing to do with the relational model itself, just with the CAP theorem.
Just because it is hard to build a RDBMS that loosens ACID but scales and is much easier to build a ad-hoc non-formally-defined model like that seen in Cassandra or BigTable does not mean it is impossible. You simply have to pay for it, in software, hardware, and consulting fees.
Nothing is stopping the open source world from making PostgreSQL or MySQL scale out other than time, energy, and perhaps a seminal paper or two out of acadamic that provide foundational models that are necessary to implement it correctly. Odds are, they've done this work at Oracle, but it's not available for obvious reasons.
Well, it seems you're misreading mine now.
does not mean it is impossible.
I never said that.
If you re-read my post you will see that I was merely wondering about their claim of not just meeting the goalposts set by Oracle et al but of exceeding them by a large margin. As I understood it their new system is supposedly not only maintenance-free, but also efficient regardless of the workload or schema that I, as a user, might throw at it. And all that supposedly hosted on commodity VMs in a cloud (probably EC2).
That's about as bold a claim as it gets in database-land, so if I was reading too much into it then feel free to correct me on that, but please don't suggest I said something that I never said.
To put it into perspective (again): I would think both Google and Amazon have probably also thought long and hard about this problem. A full fledged SQL Store would be a much more sexy offering than their crippled SimpleDB/Datastore solutions after all. There's also a good amount of academic research in the area, with Cassandra (Dynamo/BigTable) seemingly being what many people consider the best we can currently do in terms of linear scalability on commodity hardware - at the cost of not having SQL and no ACID.
So here comes this startup out of the blue, claiming to have solved all these things.
Yes, leaps like that happen sometimes. Google happened, too, after all. But I think my skepticism is not completely unjustified either?
Hence my question which tradeoff FathomDB chose to make, to get at least a basic idea about what to expect.
So, if somebody misquotes the CAP theorem, it's an indication to me that perhaps the thread isn't going to go anywhere. I'll point them to the place where they can read about it (the proof was where the CAP theorem was formalized, not Eric Brewer's original presentation), and move on. In this case, that wasn't fair to btilly, who just made a typo.
So, as I said in the presentation and elsewhere here, if you have a hair-on-fire problem with scaling your database, and are willing to be an early customer of our new database, and would make a good reference customer in a few months, we'd love to work with you today. Otherwise, you'll have to wait till we open it up more widely.