News for hackers. Stuff that matters.
Re:Thinking Psycopg3 by ghostwriter on Wednesday March 11, 2020 @08:52PM (#22552414)
> when I switched the simple script to asyncpg I gained 5x over psycopg2.

This should really be a comparison between asyncpg and psycopg2+gevent+psycogreen to be technically aligned, as otherwise it's a comparison between two scripts with non-blocking and blocking IO.

Re:Thinking Psycopg3 by 1st1 on Wednesday March 11, 2020 @11:22PM (#22553387)
Here's a benchmark [1] against aiopg, which is psycopg2+asyncio.

[1] http://magic.io/blog/asyncpg-1m-rows-from-postgres-to-python...

Re:Thinking Psycopg3 by ghostwriter on Thursday March 12, 2020 @06:46AM (#22555435)
Unless aiopg patches psycopg2 around its C extension with available callback hooks (see [1] and [2] - psycogreen uses it to register gevent callbacks), this comparison is still between blocking and non-blocking IO scripts. Quick GitHub search for `set_wait_callback` didn't return any result in the aiopg repository - https://github.com/aio-libs/aiopg/search?q=set_wait_callback...

[1] https://www.psycopg.org/docs/extensions.html#coroutines-supp...

[2] https://www.psycopg.org/docs/advanced.html#green-support

Re:Thinking Psycopg3 by ddorian43 on Thursday March 12, 2020 @04:29AM (#22554830)
You can't scale that. Adding clients will have a lot of overhead (ex: 100K clients to 1 db server). Only way is autocommit with proxy (pgbouncer etc).
Re:Thinking Psycopg3 by wnoise on Thursday March 12, 2020 @02:53AM (#22554441)
The point is to compare the blocking vs non-blocking though.
Re:Thinking Psycopg3 by ghostwriter on Thursday March 12, 2020 @06:34AM (#22555384)
I was addressing OP's impression that psycopg2 was slow "in any case" and that it cannot be used in non-blocking fashion.

> psycopg2 is slow in any case due to the blocking nature of the network calls.