News for hackers. Stuff that matters.
Re:Postgres 11 – A First Look by gaius on Saturday September 22, 2018 @03:50AM (#18044994)
With Postgres you don't need MongoDB, InfluxDB, or any other trendy thing, Postgres does it all, and better than all the wannabes.
Re:Postgres 11 – A First Look by sz4kerto on Saturday September 22, 2018 @05:31AM (#18045256)
Is that really true? Postgres is great, but it doesn't cater for _all_ use cases. For example, horizontal scalability is -- understandably for an ACID RDBMS -- not as trivial as it is for some document stores.
Re:Postgres 11 – A First Look by gaius on Saturday September 22, 2018 @06:06AM (#18045342)
Check out what Citus are doing with PG https://www.citusdata.com
Re:Postgres 11 – A First Look by threeseed on Saturday September 22, 2018 @07:39AM (#18045605)
That's a fork of PostgreSQL.

So it doesn't help you when it comes to using cloud hosted services e.g. AWS or Azure. It's also a commercial product that isn't exactly cheap ($890/node).

Re:Postgres 11 – A First Look by stareatgoats on Saturday September 22, 2018 @09:35AM (#18045904)
It's not a fork (according to their promotion), it's an 'extension', but not sure I can explain the difference. Their solution looks interesting but pricing wasn't very visible on their website, what you quote is prohibitive for me (where does it say?). But I at least assumed the 'open source' would be free.
Re:Postgres 11 – A First Look by threeseed on Saturday September 22, 2018 @10:10AM (#18046021)
Pricing is available here: https://www.citusdata.com/pricing

And the point still stands that this isn't core PostgreSQL. So you can't use managed services like AWS RDS with this. And you're reliant entirely on the vendor for support.

So from the user perspective it is basically a fork.

Re:Postgres 11 – A First Look by manigandham on Saturday September 22, 2018 @08:39PM (#18048753)
PostgreSQL is completely open-source with no backing company so everyone is reliant on some vendor if they want support. It makes no difference that Citus is one of them, with some core PG devs onboard.

Why would you talk about AWS RDS and then skip over Citus Cloud which is a managed service? If you consider this a fork then any plugin for PG is also a fork, which then makes it hard to have a productive discussion.

Re:Postgres 11 – A First Look by threeseed on Saturday September 22, 2018 @09:40PM (#18048955)
The reason I am talking about AWS RDS is because most companies aren't multi-cloud. Data needs to be housed and secured within that private VPC. And so Citus Cloud is irrelevant because it is an externally housed product.

If Citus allowed their proprietary product to be run within existing clouds that could be something but it isn't. And again since it is a significant add-on i.e. beyond just a simple plugin that almost demands vendor support it is effectively a fork.

Re:Postgres 11 – A First Look by manigandham on Sunday September 23, 2018 @12:25AM (#18049399)
What does RDS have to do with being multi-cloud? It's a hosted offering which only works in AWS. And Citus Cloud does offer VPC connectivity so you can access it privately. As stated before, the extension is also open-source and you can run it in any cloud or datacenter yourself, for free. Why would you claim that's not allowed?

You seem to be stuck on a strange intrepretation of vendor support when PG has no backing vendor, so you can pick whichever one you want. If you aren't comfortable with Citus and prefer AWS then that's fine, but you make the trade-off in the features as well. Not every managed postgres service chooses to support every PG extension so it's nothing unique in that regard, and is ultimately no different than the choices involved in picking any other database with vendor backing.

You've posted several comments now that seem to show that you do not know much about Citus or are willingly misleading people about it. I'm not sure why but perhaps it would be better if you read up on the details or disclosed your angle more clearly.

Re:Postgres 11 – A First Look by stareatgoats on Saturday September 22, 2018 @10:50AM (#18046183)
OK thanks for this. The product seems promising even if it is not core PG. I'd have to dive into the deep end and test drive to check if the trade-offs would be worthwhile. The 'open source version' seems free, so I guess I could at some point.
Re:Postgres 11 – A First Look by manigandham on Sunday September 23, 2018 @12:19AM (#18049381)
An extension means you're just running normal Postgres and load this on top. All of the normal features of Postgres are available and you can upgrade that as new versions are released without waiting for some other vendor to update their own specialized version of it. And yes, Citus is open-source and free for you to run yourself.
Re:Postgres 11 – A First Look by mslot on Saturday September 22, 2018 @06:17PM (#18048303)
Citus is an open source plug-in that you load into vanilla PostgreSQL, similar to PostGIS and other extensions.
Re:Postgres 11 – A First Look by Avamander on Saturday September 22, 2018 @07:09AM (#18045506)
Could you please elaborate how to use Postgres as a time series database (like InfluxDB)?
Re:Postgres 11 – A First Look by manigandham on Saturday September 22, 2018 @08:46PM (#18048783)
There is absolutely nothing special about "time-series" to be an actual type of database. It's all hype.

Time-series data is data that has a time component which is usually the primary property to query by. Almost every database can handle this, like MongoDB/Redis for lightweight use, Cassandra for write-heavy/global replication, ElasticSearch for raw search-style quering, or an OLAP columnstore (Redshift, MSSQL, Snowflake, Clickhouse) for serious ingest and querying. Monitoring systems like Prometheus and Netdata even have time-series storage built-in because it just isn't that hard.

InfluxDB is only really useful in the context of the integrations that it provides with the common monitoring software stacks. You can easily just create a regular relational table with "time" as a column and get great performance with an index, and then use partitioning to break up the table to get great performance over lots of data.

pg_partman is an extension that makes partitioning automatic. Timescale is an extension that makes time-focused partitioning automatic. Citus is an extension that makes partitioning across multiple nodes automatic. Or use one of the distributed OLAP systems mentioned above.

Re:Postgres 11 – A First Look by scrollaway on Saturday September 22, 2018 @07:48AM (#18045628)
The TimescaleDB plugin is extremely promising: https://www.timescale.com/

Natively without a plugin, it's not possible though.

Re:Postgres 11 – A First Look by StreamBright on Saturday September 22, 2018 @09:19AM (#18045862)
Create a table and have a timestamp for your data? Not sure what you are asking.

On hand we have: 15+ years old mature product that is widely supported and it can be used as a time series database

Other hand: n+1 young database engine with all of the problems that any young data storage engine has (only exception is FoundationDB because those guys actually understood the challenges of writing a reliable data store)

The question is, what can InfluxDB offer over Postgres for storing time series data?

Re:Postgres 11 – A First Look by threeseed on Saturday September 22, 2018 @10:01AM (#18045982)
Well InfluxDB actually is a time series database. That's all it does.

The query language is optimised for time based queries. The storage is optimised for time series. It is part of the metrics/monitoring ecosystem so it has integrations with tools like Grafana.

PostgreSQL doesn't have any time series capabilities OOTB. It's just people modelling the concept in a relational fashion. No different to how Excel can be used as a time series database.

And seriously everyone should just get over FoundationDB. There are plenty of people who have invented new databases that got it right e.g. Cassandra, HBase, Redis etc.

Re:Postgres 11 – A First Look by mslot on Saturday September 22, 2018 @06:51PM (#18048413)
PostgreSQL has excellent time series capabilities. It can load millions of rows per second, efficiently scan by time range, build rollup tables, has expressive SQL with excellent support for time (timezones, ranges, timestamps, intervals, conversion, etc.), it can combine multiple indexes to query large volumes of time series data quickly, it can do sampling, and it can expire data efficiently through partitioning. Ok, it's not entirely ootb. To make it work nicely you may need extensions like pg_partman to automate partitioning and Citus to scale out, but once you do you have a time series database that's faster and more powerful than anything else on the market.
Re:Postgres 11 – A First Look by threeseed on Saturday September 22, 2018 @09:34PM (#18048937)
Congratulations. You just described the capabilities of ANY SQL database.

But again there are capabilities that exist only in InfluxDB because all it does is time series data. It's not multi-purpose. Which again is why you see it all over the place in the metrics/monitoring ecosystem but you never see PostgreSQL, MySQL etc.

Re:Postgres 11 – A First Look by xdanger on Sunday September 23, 2018 @01:04PM (#18051981)
Actually you can use PostgreSQL in those with timescaledb plugin. ;)
Re:Postgres 11 – A First Look by gaius on Saturday September 22, 2018 @08:24AM (#18045719)
Could you please elaborate how to use Postgres as a time series database (like InfluxDB)?

There's some good discussion of it here https://news.ycombinator.com/item?id=16539317

Re:Postgres 11 – A First Look by threeseed on Saturday September 22, 2018 @07:37AM (#18045598)
This is simply untrue.

MongoDB is a much better JSON store. It's ridiculously faster, has much richer update semantics and it's simply not comparable when it comes to the ease of clustering.

As for time series unless I am confused but PostgreSQL doesn't have any OOTB capabilities.

Re:Postgres 11 – A First Look by gaius on Saturday September 22, 2018 @08:28AM (#18045727)
MongoDB is a much better JSON store

That hasn't been true for a long time, if it even ever really was when you consider that the key role of a database is to reliably persist your data https://www.enterprisedb.com/node/3441 (2014)

Re:Postgres 11 – A First Look by threeseed on Saturday September 22, 2018 @08:50AM (#18045788)
Think about this for a second. You just posted a link that is over 4 years old.

It's completely irrelevant. Especially since I am referring to version 3.0+ where they completely replaced their entire engine with WiredTiger. And from that point on the performance for tuple level updates for example is faster than I've seen from ANY database.

And this nonsense about reliable persistence of data was a case of bad defaults many, many years ago. It was also fixed many, many years ago and so these tired memes just make you look bad.

Re:Postgres 11 – A First Look by gaius on Saturday September 22, 2018 @09:16AM (#18045853)
You just posted a link that is over 4 years old.

Yes, to make my point that it hasn't been true "for a long time". Yes Mongo had JSON support before PG did, but PG soon surpassed it.

these tired memes just make you look bad

Nah, they make me look like what I am, an extremely experienced data guy. Maybe Mongo has some use cases in webdev, I wouldn't know.