> I tried a couple different server libraries on Python but they all seemed to struggle with even tiny loads. Not sure what was up with that

Could the problem have been the "global interpreter lock"? The GIL makes it impossible for multiple threads to simultaneously perform certain basic operations, effectively making the system single-threaded.