News for hackers. Stuff that matters.
Re:What PostgreSQL has over other open source SQL databases:... by rdtsc on Saturday October 24, 2015 @08:32PM (#10445432)
In general that is not something that can be easily slapped on later easily. There are some solutions but they feel like add-ons.

For software some things can be added on later, but some can't. Things like fault tolerance, distribution, security can't be easily bolted on after the product has already been developed (as those usually cut right through the whole stack).

People who like Postgres and make fun of NoSQL products for not having well ... SQL, usually forget that NoSQL isn't as much about not having SQL as about having a good distributed/scalable backend story.

Re:What PostgreSQL has over other open source SQL databases:... by takeda on Saturday October 24, 2015 @11:26PM (#10445847)
The whole SQL vs NoSQL is misunderstood (most likely to the silly name). It is not about that Relational (SQL) database was designed in a way that is hard to make it distributed. It's all about what trade offs are you ok with. If you want ACID (Atomicity, Consistency, Isolation, Durability - if you are not sure, then you do want it) then you go with a relational database. If you are ok with data not always being consistent, or sometimes even lost (for example sessions, statistics, user tracking etc) you can use NoSQL.

There are also databases coughMongoDBcough which claim to have it all, but in reality they have neither[1][2].

[1] Performance of single instance: http://www.enterprisedb.com/postgres-plus-edb-blog/marc-lins...

[2] Scaling out: http://www.datastax.com/wp-content/themes/datastax-2014-08/f...

Re:What PostgreSQL has over other open source SQL databases:... by manigandham on Sunday October 25, 2015 @12:53AM (#10446020)
There's no such thing as "NoSQL" - it's a meaningless term. It's far better to just refer to the actual type of database: relational, key/value, wide-column, graph, rdf, etc.

Any one of these database types can have a SQL interface (in various dialects and standards), ACID, different consistency models and scale-out options. It's just up to the specific database product in what it actually offers.

Re:What PostgreSQL has over other open source SQL databases:... by takeda on Sunday October 25, 2015 @01:48AM (#10446120)
That's a good point, I do hate that term, but there's not a good term that encompasses them all. Perhaps it should be relational vs specialized database.

Perhaps it should be said that relational is DB that's for general purpose, while specialized (key/value, graph, etc) requires you to understand your data so you can make proper trade-offs when picking specialized db. You are not getting those features advertised by other databases for free.

Re:What PostgreSQL has over other open source SQL databases:... by spacemanmatt on Sunday October 25, 2015 @05:18AM (#10446431)
They're all specialized DBs. There is no general case.
Re:What PostgreSQL has over other open source SQL databases:... by manigandham on Sunday October 25, 2015 @11:15AM (#10447077)
You don't need any terms, they're all just databases.

Relational is specialized too - in dealing with relational data, foreign keys, normalization, joins, etc.

ACID is just a feature of a product, not tied to relational as there are relational databases that don't offer it or can have it disabled. Tradeoffs and understanding your data apply to every single type of database.

Re:What PostgreSQL has over other open source SQL databases:... by threeseed on Sunday October 25, 2015 @12:48AM (#10446006)
I think you are the one adding to the confusion by tarring all NoSQL databases with the same brush.

HBase is strongly consistent and given that it a core part of the Hadoop stack and used by Facebook for Messages would indicate it is hardly has systemic data loss issues. Cassandra likewise is used for PSN, Steam, EA Online, Spotify and is quite capable of running in strongly consistent mode. That's just two of the many NoSQL databases that are strongly consistent and being used for major systems where data loss would be unacceptable.

Also your comparisons between PostgreSQL and MongoDB are meaningless. The complete overhaul of the default engine in MongoDB 3 (WiredTiger) has increased performance massively with a notable reduction in disk space usage. It's basically a whole new database.

https://www.mongodb.com/blog/post/performance-testing-mongod...

Re:What PostgreSQL has over other open source SQL databases:... by takeda on Sunday October 25, 2015 @01:42AM (#10446114)
HBase is not ACID and therefore not strongly consistent. The consistency in CAP is different than consistency in ACID.

Regarding the Mongo, the second link I provided was comparing WiredTiger. In link you provided they made an improvement, but 2.x was already slow. Initially MongoDB gained its popularity because it supposed to be fast. And it was when it did not worry about making sure the written data ends up on disk. Once people started losing data when system crashed they changed defaults, but then became much slower. And worst of all it still was losing data[1]. There's no good reason to use it, even if you would want to use it as a cache, there are far better solutions.

[1] https://aphyr.com/posts/322-call-me-maybe-mongodb-stale-read...

Re:What PostgreSQL has over other open source SQL databases:... by lobster_johnson on Sunday October 25, 2015 @02:16AM (#10446156)
> HBase is not ACID and therefore not strongly consistent. The consistency in CAP is different than consistency in ACID.

You're arguing the word "consistent" is used wrong, and then you go on to mixing ACID and CAP — two acronyms which use the word "consistent" with completely different meanings.

The "C" in ACID stands for data consistency, which is that you shouldn't be allowed to store any data which isn't internally consistent with the schema's consistency rules. For example, you're not allowed to store a "null" value in a "not null" column, and you can't delete a row that is pointed to by a different table's foreign key constraint.

The "C" in Brewer's original CAP theorem really represents atomic consistency. It's the idea that if you submit data, you should get the exact same data back, even if you ask a different node. It sort of represents both the "A" (atomicity, in the sense that either everything or nothing is updated) in ACID, as well as the "I" (isolation, in the sense that nodes don't see partial writes) and "D" (durability, in the sense that data written stays written).

In CAP terms, HBase is strongly consistent, I believe, as long as you query the primary region server and not a replica. But HBase is not transactional, and therefore it's pretty meaningless to talk about ACID.

Re:What PostgreSQL has over other open source SQL databases:... by threeseed on Sunday October 25, 2015 @02:46AM (#10446210)
As mentioned I think you are confused about what consistent means.

By the ACID definition a database can either be consistent or not. It can't sometimes be one or the other (unless there is some bug. I was referring to the CAP definition which I assume is what you are referring to. And by that definition HBase is absolutely strong consistent.

And that second link was pretty bad (with all respect to the great guys over at Datastax). It is benchmarking a release candidate build of MongoDB (2.8.0-rc4) where quite a number of important fixes went in before GM. They should have just waited until 3.0 was released.

Also it would be good for that Call Me Maybe test to be redone considering that stale-read bug has been fixed:

https://jira.mongodb.org/browse/SERVER-18022

Re:What PostgreSQL has over other open source SQL databases:... by lobster_johnson on Sunday October 25, 2015 @01:58AM (#10446130)
> It is not about that Relational (SQL) database was designed in a way that is hard to make it distributed

Not entirely accurate. Relational database schemas are typically normalized into many tables and make heavy use of joins; it's fair to say that the concept of joins are integral the relational model. But joins, by definition, require fast access to data; it's extremely inefficient to join across network boundaries.

So-called NoSQL databases aren't just about ACID or CAP, but also about denormalization. In fact, I would go so far as to say that the single founding tenet of all of "NoSQL" was denormalization across multiple nodes. Google's Bigtable (on which HBase is based, and which arguably started the public's interest in "NoSQL") depends on extreme denormalization in its data model‚ which does away with tables and instead treats everything as columns, and also discards referential integrity (the "C" in ACID, which isn't the same as in CAP) and isolation (the "I"). Sharding flows naturally out of denormalization. Eventual consistency came later with Amazon's Dynamo paper; Bigtable was strongly consistent.

A consequence of denormalization is that you can hit any database node and get consistent performance. No query will ever get bogged down in complicated, long-running nested loops and hash joins because the mechanisms don't exist. It creates an even playing field for queries because they're mostly individual key lookups; the result is (in theory) predictable, stable performance across the cluster. The complexity of join logic that was in the relational database is moved into the client, which gets the burden of untangling data. Which is why Google also invented early solutions like MapReduce and Sawzall, because once you have a huge, denormalized, distributed cluster of data in this poor man's relational model, your individual clients can no longer access the data efficiently. So denormalization also leads to parallel computation, and especially precomputation, rather than the ad-hoc queries that relational databases are good at.

It's all about scale, and fitting the data model to work in your favour at that scale.

So yeah, I would definitely say that relational databases are designed in a way that make them hard to distribute. The above is just the tip of the iceberg. How do you enforce referential integrity (foreign keys, etc.) across nodes? How do you assign unique primary keys? You talk about making tradeoffs, but would you really want a distributed relational database without distributed transactions?

Can you implement a performant distributed database with joins and other relational features? Yes. VoltDB and Vertica come to mind. They are designed from the ground up to do this, and have complicated logic for partitioning queries. They make other tradeoffs: for example, VoltDB is in-memory and to perform well, all transactional access code has to be written as a "stored procedure", a Java JAR file that you upload to the server, and transactions are expected to be very short-lived. But they will, pretty much by definition, not perform at scale the way that "NoSQL" databases are designed to.

Re:What PostgreSQL has over other open source SQL databases:... by manigandham on Sunday October 25, 2015 @11:18AM (#10447080)
Take a look at MemSQL - a much better distributed relational database with regular mysql protocol access.
Re:What PostgreSQL has over other open source SQL databases:... by lobster_johnson on Sunday October 25, 2015 @01:07PM (#10447453)
Thanks, but I'm not going to consider a database that's not open source, sorry.