I worked in a Python web-scraping team doing 3 million requests per day. Their solution to manage concurrency was also one pod per crawl, each with its own DB connection. It really struck me as crazy that everyone thought this was a good idea.