Foreign data wrappers: PostgreSQL's secret weapon?

Re:Foreign data wrappers: PostgreSQL's secret weapon? by thinkpad20 on Wednesday July 08, 2020 @11:01AM (#23770541)

On the subject of foreign data wrappers, there's an extension for storing columnar data in Postgres which is implemented via this concept: https://github.com/citusdata/cstore_fdw

I haven't used it myself but it's pretty cool that it's out there.

[ Parent | on HN ]

Re:Foreign data wrappers: PostgreSQL's secret weapon? by mildbyte on Wednesday July 08, 2020 @11:33AM (#23770896)

Splitgraph co-founder here. cstore_fdw is great (we even use it in Splitgraph to store data [0])! It's not as fast as purpose-built columnar stores like MonetDB. However, it plugs seamlessly into PostgreSQL and supports all types, even those added via extensions. Read performance for OLAP workloads like aggregations is better than PostgreSQL [1] and it has a much smaller IO load and disk footprint (you can get long runs of similar values in column-oriented storage, which compress better). As a nice bonus, you can simply swap cstore_fdw files in and out of the database without having to "load" them into PostgreSQL. We use this idea too to enable data sharing.

[0] https://www.splitgraph.com/docs/concepts/objects

[1] https://tech.marksblogg.com/billion-nyc-taxi-rides-postgresq...

[ Parent | on HN ]

Re:Foreign data wrappers: PostgreSQL's secret weapon? by senderista on Wednesday July 08, 2020 @12:33PM (#23771488)

Alternative storage formats like columnar storage are better served by Postgres’s new custom access methods (introduced in PG12 I believe).

[ Parent | on HN ]