Show HN: 1M rows/s from Postgres to Python

Re:Show HN: 1M rows/s from Postgres to Python by 1st1 on Thursday August 04, 2016 @06:49PM (#12229001)

A lot of code in asyncpg is datatypes parsers, buffer abstractions, API facade and other IO independent things. So good news is that it would be possible to reuse a lot of code.

Protocol is implemented in two layers -- the base layer is IO and framework independent. The higher level is designed for asyncio.

Long story short, it's possible to add a synchronous version, but it still will require a lot of work. The API will also have to be split in two.

[ Parent | on HN ]

Re:Show HN: 1M rows/s from Postgres to Python by zzzeek on Thursday August 04, 2016 @07:24PM (#12229144)

It's possible to make a really nice async sqlalchemy core too, and even some of the ORM. huge amount of work, i started on just the connection pool. Would be nice but require all its own test coverage and everything. If i was the CEO of sqlalchemy inc, it would have been done. But that's not where my salary comes from :)

[ Parent | on HN ]

Re:Show HN: 1M rows/s from Postgres to Python by allendoerfer on Thursday August 04, 2016 @08:10PM (#12229304)

SQLAlchemy is clearly one of the best ORMs (maybe there is something in Java-land that I don't know). It is beyond me how you are not guilted by cash getting thrown in your direction into making this your job.

Well, I like what bountysource.com and others are doing, maybe we will get there.

[ Parent | on HN ]

Re:Show HN: 1M rows/s from Postgres to Python by infinite8s on Friday August 05, 2016 @10:25AM (#12232525)

It's the tragedy of open source. See - https://news.ycombinator.com/item?id=10905845

[ Parent | on HN ]

Re:Show HN: 1M rows/s from Postgres to Python by allendoerfer on Thursday August 04, 2016 @07:04PM (#12229074)

I think you should open a bounty for it. As questions like "Can I use this with Django/SQLAlchemy/…?" showed, people love drop-in replacements that speed up their code, especially when it is at a low level like this.

Do not want to sound ungrateful, though. Thank you for your work also on you other contributions. Would definitely use this when applicable. Are more performance related projects like this and uvloop planned?

[ Parent | on HN ]

Re:Show HN: 1M rows/s from Postgres to Python by 1st1 on Thursday August 04, 2016 @07:18PM (#12229129)

> I think you should open a bounty for it. As questions like "Can I use this with Django/SQLAlchemy/…?" showed, people love drop-in replacements that speed up their code, especially when it is at a low level like this.

We'll consider doing that. We have to keep in mind that supporting two IO modes will add a lot of maintenance overhead. Anyways, we can certainly continue the discussion on GH, and maybe we can find a champion to do the heavy-lifting.

> Do not want to sound ungrateful, though. Thank you for your work also on you other contributions. Would definitely use this when applicable. Are more performance related projects like this and uvloop planned?

Thank you.

Right now most of our resources are consumed by getting EdgeDB alpha out as soon as possible. We also have plans to create a micro web framework based on httptools and few other exciting ideas. Stay tuned :)

Also check out PEP 525 -- another cool thing we're trying to make happen in Python 3.6.

[ Parent | on HN ]