The asyncio cancel is convenient but can be really deceiving. If you want to reliably propagate the cancel across the wire, you likely have to invoke new await-calls inside your CancelledError-block. But doing this requires you to re-await the cancelled function again!
async def process():
try:
await db.slow_operation()
except CancelledError:
synchronous_functions_work()
await db.cancel() # This future will not complete on timeout
p = process()
try:
await asyncio.wait_for(p, timeout=1)
except TimeoutError:
await p # Required for db.cancel() to run!!!
Edit: Seems they actually fixed/changed this in 3.7: https://bugs.python.org/issue32751. So instead you have to write robust except-blocks that must never timeout.
A co-worker of mine actually added support for timeouts to a database we were using. (It is a smaller, less-well known DB.) I added it to the Python side.
Good cancellation support in the language is really critical here, I found. In Python, it was a breeze to add timeouts and get rid of long running requests, if, say, the network connection dropped: you cancel the future, and that cancellation propagates to all the sub-futures. It is even hookable so that one can — if the network protocol supports it — propagate that across the wire to other services.
The DB in our case was written in Go, however, so that was tougher. Golang's best method (that we learned of at the time) is to thread a "Context" object through your code paths. We were working with existing code, of course, and it lacked this, and it's harder to add in hindsight.
Of course, once we got the server to stop hanging on queries of doom and return a more appropriate "that's a query of doom, and would hang the server" error, the complaint was that the server wasn't executing those queries anymore…