> The one thing I genuinely don't understand is why there is no single task queue that works across ruby and python.

Not sure I can recommend it, but Gearman does fit the bill. There’s client and worker libraries for a dozen languages, including Python3 and Ruby.