News for hackers. Stuff that matters.
Re:How PostgreSQL is tested [video] by stonemetal on Wednesday February 15, 2017 @06:17PM (#13656428)
Postgresql testing works the same way it has worked for the last 20-30 years. Each test consists of a SQL script, an expected result file, and a result file. A shell script runs the SQL script through the pg commandline, and dumps the results to the results file. Diff then tells you if the test pass. Also the shell scripts were rewritten in C for cross platform support. Not available via SQL, ie vacum? No tests. Concurrent modifications? No tests. Testing query plans that don't use order by have fun.

They have a new test suite built around TAP (https://testanything.org/) and perl so now they have automated tests for more things like certs, and replication.

Re:How PostgreSQL is tested [video] by ioltas on Wednesday February 15, 2017 @06:39PM (#13656589)
There are three basic suites of tests:

- pg_regress, which is as you say a set of SQL scripts run on a running server with generated output compared to the expected output. There could be multiple expected outputs, see for example float tests.

- pg_isolation_test, which is a part you are misunderstanding. This facility has been introduced with SSI (serializable transactions) to test concurrency.

- TAP tests (author of the replication/backup/recovery part here), which has roughly three test suites:

-- src/bin/ to test all the in-core binaries

-- src/test/ssl to test SSL configurations. This has proved to be useful lately when moving all the SSL parameters to be reloadable.

-- src/test/recovery to test all the kind of replication scenarios.

There are as well modules in src/test/modules that use TAP tests to run regressions on pg_dump, commit timestamps and extension interactions.

Re:How PostgreSQL is tested [video] by rattray on Thursday February 16, 2017 @12:25AM (#13657932)
Thanks for the helpful detail!

Are there, in fact, no tests on VACUUM and others? Is that a problem? Are there plans to add them? Would you be looking for contributors to do so?

Re:How PostgreSQL is tested [video] by ioltas on Thursday February 16, 2017 @06:50PM (#13664725)
The query VACUUM is being tested via SQL. Now if you are referring to autovacuum, there are no tests directly designed with this purpose in mind. autovacuum is running in any instance running the regression tests as it is always enabled by default.
Re:How PostgreSQL is tested [video] by coolgeek on Thursday February 16, 2017 @09:32AM (#13660081)
> Testing query plans that don't use order by have fun

diff $(sort output) $(sort expected)