Great work, thanks for sharing. Very impressive speed.

I didn't know about uvloop, that's a cool project:

http://magic.io/blog/uvloop-blazing-fast-python-networking/

Yep, it's also much more stable now, 3 months after that article was posted. I plan to make a new blog post about it soon.

Do you have any numbers on queries that return millions of rows? If it's still fast, asyncpg could be nice for some ETL-type workloads.

We haven't benchmarked this specific use case. However, cursors in asyncpg do data prefetching (the exact number of prefetched rows is configurable), so I expect that asynchronous iteration should be pretty fast (and still faster than psycopg2).

May be I am reading the benchmarks wrong, but does it mean I can get a 2x speedup in my end to end query execution from moving from aiopg to asynpg alone without any server side changes or query optimizations?

We intentionally benchmarked simple queries to measure the overhead of the driver implementation. So, the 2x speedup really means 50% reduction in overhead.

I'd love to read the docs for the protocol itself, is it planned? Do you have a link explaining it otherwise?

Great work!

The protocol is documented in the official PostgreSQL documentation: https://www.postgresql.org/docs/current/static/protocol.html

I meant a doc explaining roughly the components at https://github.com/MagicStack/asyncpg/tree/master/asyncpg/pr... and how they interact with each other at a high level as it doesn't seem to have comments.

Thanks for the link though, I'll have a look

Yeah, documenting internals sounds like a good idea. At least to make it easier for people to contribute and further improve asyncpg. We'll try to find some time to do that.

How can I use it with SQLAlchemy instead of psycopg2?

SQLAlchemy ORM is not async, so there is no way currently to use any async dialect for it. However, it is possible to write an SQLAlchemy Core adapter for asyncpg.

It might even be easier than writing an adapter if you just use SQLAlchemy Core's DSL as the generative mechanism for a SQL statement that is sent to asyncpg.

There appears to be an adapter for Core already: https://github.com/CanopyTax/asyncpgsa

Author of asyncpgsa here, I am in the process of writing that, let me know if you have questions.

Does this automatically escape/convert things like datetimes in a query or does that need to be done separately? The docs are a little sparse on details.

Looks fantastic anyway!

Yes, query arguments are converted automatically. Results are converted into Python types as expected (including arrays and composite types).

Bit offtopic, but what are the plots made with?

Handmade with d3.js