News for hackers. Stuff that matters.
Re:Are triggers really that slow in Postgres? by hinkley on Tuesday May 22, 2018 @09:04PM (#17130785)
This sort of sentence is uttered by someone who is so far down a hole that all they can see is stars. It’s a big hole that a lot of people are in but it’s the consequence of a nasty trade off.

All of this is trivial if your team decides that a shared database for dev work is bad for repeatability and thus bad for scaling the team.

You should be able to spool up a local database with good sample data in it. To do that your schema, indexes and triggers would be under version control, and a data dump is somewhere people can get it.

Once you have this your CI system runs Postgres locally or in a container, runs the drop create scripts, and then runs your integration and end to end tests on the canned data.

In addition to getting a CI solution for next to free you get rid of the concurrent access Wild West and this particularly painful conversation:

Why did this break and why didn’t you notice it before you pushed? Oh I saw that problem the other day but I thought someone else was changing data (and not my code being broken).

If the data is on your machine and it gets broken, then it is only your machine that could have broken it. You can’t delude yourself into thinking it was someone else mucking around. The problem is either in your code or in your latest pull from master. You are responsible for determining the cause, not me, not the release manager, not QA. You.

Re:Are triggers really that slow in Postgres? by manquer on Tuesday May 22, 2018 @11:38PM (#17131320)
Not all bugs are logical. Many times CI workflow does not catch poorly written but logically correct queries because sample db has few records in it, emulating large data sets with data patterns like production to get the same kind of query plans is hard.
Re:Are triggers really that slow in Postgres? by hobs on Wednesday May 23, 2018 @10:14AM (#17134310)
I would say this is the first objection that is on point in this thread, not that figuring out deployment isn't hard, but the blog posts on how to solve it for your platform are multitudinous.

Getting useful data loads to test queries is much more difficult but you have a few options (at least coming from a SQL Server approach):

* Query hints to emulate larger sets of data (so you can see what type of IO you would get, of course multiplication is your friend)

* Check if your database offers something along the lines of DBCC CLONEDATABASE https://docs.microsoft.com/en-us/sql/t-sql/database-console-... (gets you a copy of statistics from prod without the data)

* Building a data masking process so that the data is somewhat representative (in volume, mocking it such that the histogram of values is the same in your database is WAY harder)

* Building an isolated load test environment with something like distributed replay https://docs.microsoft.com/en-us/sql/tools/distributed-repla... (record workload, replay, measure, make your change, replay, measure, yes - it is tedious but that's a performance regression test for you)