New in PostgreSQL 10

Re:New in PostgreSQL 10 by jacques_chester on Thursday September 21, 2017 @06:58PM (#15308038)

I expect either of Citus or Greenplum can deal with that kind of volume. Citus acts as an extension, Greenplum is descended from an earlier fork (now merging its back to mainline as part of overall development).

Citus specifically cite analytics and write throughput as their strengths. I believe them. They do seem to charge for shard rebalancing and some other features.

Greenplum comes with an inbuilt data science toolkit (MADlib) and has a long history of dealing with intensive workloads. I don't know if there is a commercial version with extra features. I don't think so, but I haven't looked closely.

Greenplum also gave birth to HAWQ, which uses the Greenplum distributed/parallel query planner to plan queries over Hadoop.

Disclosure: I work for Pivotal, we do most Greenplum development.

[ Parent | on HN ]

Re:New in PostgreSQL 10 by user5994461 on Friday September 22, 2017 @05:11AM (#15310448)

And citus will be a nightmare to setup and maintain, for what elasticsearch could have done easier and better out of the box. Use the right tool for the job.

[ Parent | on HN ]

Re:New in PostgreSQL 10 by jacques_chester on Friday September 22, 2017 @07:36AM (#15311176)

I was pointing out that 2TB/day is not a very high write volume for modern OSS RDBMSes, especially distributed ones.

As for elasticsearch, I have found it to be temperamental. The "out of the box" experience was usually unhappy. For logging I've seen people move to other tools (splunk, stackdriver) more than once.

[ Parent | on HN ]

Re:New in PostgreSQL 10 by user5994461 on Friday September 22, 2017 @11:58AM (#15313313)

I have found elasticsearch to be one the easiest database I ever had to setup and maintain. Some tuning to do when you have TB on multiple boxes but nothing hardcore.

2 TB/day is fairly easy... as long as you have a bit of experience AND one of the database that can scale horizontally AND the company buys the amount of hardware that is required.

StackDriver/Sumologic/logentries are cloud solutions, so obviously one doesn't have to deal with the database ever.

[ Parent | on HN ]

Re:New in PostgreSQL 10 by jacques_chester on Friday September 22, 2017 @07:27PM (#15316864)

I think it's anecdote vs anecdote, but clearly you have more experience. In a fulltext indexing situation I'd probably look at elasticsearch again; I don't deliberately hamstring myself to try and be right on HN.

On the other hand, if I have structured-enough data, I'm going to prefer relational tools for it. Hammers and algebraic nails and so on.

[ Parent | on HN ]