News for hackers. Stuff that matters.
Re:Some SQL Tricks of an Application DBA by munk-a on Wednesday July 29, 2020 @02:35PM (#23990100)
All of those tips and no mention of MATERIALIZED VIEWS[1] it's a pretty wonderful tool to use for data caching (potentially highly safe data if you use on action trigger based refreshing) that can allow you to have a nice normalized data layout to write into and a beautiful de-normalized cross-table index supporting place to read from. If you've never heard of mat views please do look them up[2] and play around a bit.

1. As they're called in postgres at least

2. https://www.postgresql.org/docs/current/sql-creatematerializ...

Re:Some SQL Tricks of an Application DBA by gerhardi on Wednesday July 29, 2020 @03:52PM (#23990878)
Materialized views are also nice for many modern DW applications where you don't live in legacy "once in a hour/day/week" ELT/ETL environment, but more in a real timeish or micro batch world. You can have data marts / reporting tables composed with these materialized views over many kinds of underlying DW models. Just dropping this to promote the Materialized View functionality :)
Re:Some SQL Tricks of an Application DBA by chishaku on Thursday July 30, 2020 @02:29AM (#23995271)
dbt (data build tool) is awesome for managing this.
Re:Some SQL Tricks of an Application DBA by aargh_aargh on Thursday July 30, 2020 @06:46AM (#23996645)
Re:Some SQL Tricks of an Application DBA by djrobstep on Wednesday July 29, 2020 @06:30PM (#23992302)
I've found them to be pretty limited as a feature - really they are just some syntactical sugar over "create table as select ... ".

If you start to need any kind of flexibility over refreshing the table, you'll need to move to using a plain old table anyway.

Re:Some SQL Tricks of an Application DBA by asah on Wednesday July 29, 2020 @07:55PM (#23992854)
This was my impression too - without incremental update, I don't see the benefit ?
Re:Some SQL Tricks of an Application DBA by gavinray on Wednesday July 29, 2020 @08:12PM (#23992980)
I'm someone with limited DB wizardry, but a large admiration for Postgres so take this for what it's worth:

It's my understanding that there is a proposed feature, with a proof-of-concept floating around, called "Incremental View Maintenance", which is auto-update of dependent Materialized Views when their dependent base-tables change.

https://wiki.postgresql.org/wiki/Incremental_View_Maintenanc...

  Materialized view with IVM option created by CRATE INCREMENTAL MATERIALIZED VIEW command. Noted this syntax is just tentative, so it may be changed.

  When a materialized view is created, AFTER triggers are internally created on its all base tables.

  When the base tables is modified (INSERT, DELETE, UPDATE), this view is updated incrementally in the trigger function.
I think there are some performance/technical things that are getting sorted out with this, but it would be a killer feature.

It's the one thing I wish Postgres had that it doesn't. You have to use triggers to do this currently.

Re:Some SQL Tricks of an Application DBA by Normal_gaussian on Wednesday July 29, 2020 @09:12PM (#23993323)
Yes, this is a killer feature. There are many instances of what are essentially application side caches that try (and often fail) to do this; its non-trivial to do right, and I am greatly looking forwards to it.

In particular, an event-sourced world becomes much easier to implement and maintain with the DB handling all the incremental magic.

Re:Some SQL Tricks of an Application DBA by Mouse47 on Wednesday July 29, 2020 @08:38PM (#23993132)
For what it's worth SQL server supports incremental updates for materialized views. I think they're called indexed views, though
Re:Some SQL Tricks of an Application DBA by lazulicurio on Wednesday July 29, 2020 @09:48PM (#23993485)
Yep! And they're pretty magical. Beyond materializing joins, you can use COUNT_BIG to greatly speed up common DISTINCT queries and the two row trick[1] to enforce complex constraints.

[1] https://spaghettidba.com/2011/08/03/enforcing-complex-constr...

Re:Some SQL Tricks of an Application DBA by hultner on Thursday July 30, 2020 @06:39PM (#24003616)
Well they are magical until you need to do something a bit more complex, like window functions, where they’ll not work again. I’ve worked around this by using indexed views for sub parts of a larger regular view but it feels a bit hacks since you need to hint the server to actually use the indexed view or it will just use it as a regular view which I don’t understand the reasoning behind.

But for simple things they’re magical.

Re:Some SQL Tricks of an Application DBA by lazulicurio on Saturday August 01, 2020 @06:41PM (#24024537)
> Well they are magical until you need to do something a bit more complex, like window functions.

The best method I've personally found for window functions is cross/outer apply + narrow indexes with included columns. A lot of times you can get away without an indexed view at all.

> you need to hint the server to actually use the indexed view or it will just use it as a regular view which I don’t understand the reasoning behind

SQL Server Enterprise will use indexed views automatically. But you gotta shell out the big bucks for that improved query planner.