News for hackers. Stuff that matters.

A comment on What PostgreSQL has over other open source SQL databases: Part II. Parent comment

Re:What PostgreSQL has over other open source SQL databases:... by lobster_johnson on Sunday October 25, 2015 @02:44AM (#10446205)
There are a bunch of commercial engines built on Postgres that implement sharding and parallel querying at huge scale: Amazon Redshift, Truviso, Netezza, ParAccel, Aster Data and CitusDB come to mind. It's a shame that none of this has tricked down into the open-source version.

There's Postgres-X2 (formerly Postgres-XC), sponsored by NTT, which implements fully consistent multimaster replication and partitioning. After many, many years apparently still isn't production-ready nor particularly scalable, and the likelihood that it will ever be merged into mainline Postgres is zero (its planner changes alone are apparently several hundred thousand lines of code).

More interestingly, there's Postgres-XL, which is apparently a merger of Postgres-XC and a different implementation called StormDB that was bought by a company called TransLattice. (TransLattice also sponsored or acquired an earlier project that went nowhere, Postgres-R.) Unlike XC/X2, they say they aim to contribute changes back to the mainline, and they also claim their distributed query model is superior. With the commerical support behind it (it's used as the basis of a commercial product), it's possible that this is something that will be usable. Unfortunately, it seems very quiet and not very open-sourcy; most of the development seems to be by just one guy [3], and nobody seems to be using it in production at this point.

[1] https://github.com/postgres-x2/postgres-x2

[2] http://www.postgres-xl.org/

[3] http://git.postgresql.org/gitweb/?p=postgres-xl.git;a=summar...

[ Parent | on HN ]
Re:What PostgreSQL has over other open source SQL databases:... by jacques_chester on Sunday October 25, 2015 @09:38AM (#10446837)
> It's a shame that none of this has tricked down into the open-source version.

Greenplum is an MPP database forked from the Postgres 8.x codebase. It'll be opensourced soon.

[ Parent | on HN ]
Re:What PostgreSQL has over other open source SQL databases:... by cmrdporcupine on Sunday October 25, 2015 @09:58AM (#10446870)
Oh wow, I had no idea Greenplum was being open sourced, that's great news. I fiddled with it 6 years agocomparing it and Vertica for some large scale analytics work and it was pretty impressive.. I wonder how hard to merge Postgres head features back into Greenplum though, or vice versa. Probably not gonna happen?
[ Parent | on HN ]
Re:What PostgreSQL has over other open source SQL databases:... by jacques_chester on Sunday October 25, 2015 @11:51AM (#10447179)
It's hard for me to answer you because I work for Pivotal Labs and I've heard stuff in confidence about the current thinking of what to do next. But once it's opensource I think all sorts of possibilities open up legally, technically and economically.
[ Parent | on HN ]
Re:What PostgreSQL has over other open source SQL databases:... by dorfsmay on Sunday October 25, 2015 @01:21PM (#10447508)
Are the newer PostgreSQL development been merged back in greenplum, or is it based on an older version?

Is the recent Dell acquisition putting the open sourcing plans at risk?

[ Parent | on HN ]
Re:What PostgreSQL has over other open source SQL databases:... by jacques_chester on Sunday October 25, 2015 @01:43PM (#10447617)
I'm afraid I couldn't answer these questions even if I knew.

Blah blah a lot of it I don't know rhubarb doctrine of confidence murmer cough professional ethics mumble mutter SEC regulations yadda yadda.

[ Parent | on HN ]
Re:What PostgreSQL has over other open source SQL databases:... by lobster_johnson on Sunday October 25, 2015 @01:21PM (#10447507)
That's interesting. 8.x is pretty antique, though; has anything from the 9.x mainline been merged back in? I'd love to hear about a release date, too.
[ Parent | on HN ]
Re:What PostgreSQL has over other open source SQL databases:... by jacques_chester on Sunday October 25, 2015 @01:51PM (#10447645)
We announced our opensource plans in February[1]. I don't know when Greenplum will be ready to be opened up, as I work in Labs, which is a different division.

So far we have opened Apache HAWQ (incubating)[2], which is an SQL query engine for Hadoop based on the MPP query planner in Greenplum.

And Apache Geode (incubating)[3], which is based on Gemfire, which is the most successful in-memory grid database you've never heard of.

Edit: I suppose I should mention, entirely in my own self-interest, that we are hiring in our Big Data division across all our products. You can look at our webpage (https://pivotal.io/careers) or email me ([email protected]).

[1] http://pivotal.io/big-data/press-release/pivotal-introduces-...

[2] http://hawq.incubator.apache.org/

[3] http://geode.incubator.apache.org/

[ Parent | on HN ]
Re:What PostgreSQL has over other open source SQL databases:... by lobster_johnson on Sunday October 25, 2015 @02:48PM (#10447865)
Looks like you got GemFire from Gemstone, which is also owned by VMware? That explains why it's written in Java, whereas I was happily surprised that HAWQ is written in C++.

It's great that you're releasing stuff as open source. Perhaps you could persuade a colleague at your other division to talk about Greenplum in this thread?

[ Parent | on HN ]
Re:What PostgreSQL has over other open source SQL databases:... by jacques_chester on Sunday October 25, 2015 @03:33PM (#10448049)
We (Pivotal) had a bunch of EMC and VMWare companies, technologies and technologies spun into us when the company was founded a few years ago.

Including: Cloud Foundry, Greenplum, Spring, Gemfire and of course the namesake -- Pivotal Labs.

Email me and I will connect you with our director of engineering for Big Data.

[ Parent | on HN ]