I ran a quick test, an entry-level box from Digital Ocean can sustain 350+ connections/sec and maxes out at around 7,200 open connections.

Having root access to the machine would allow you to optimise some things and probably raise those numbers, but then you don't get the convenience of using Heroku.