Even so, we could easily - without a sweat - get 2000 people on a server. I wasn't even using more workers than we had cores. Make that a bigger box and you get a few thousand more. Then use a few different servers that are federated with each other for DEFCON and you have instant scale.
The matrix.org people run the matrix.org homeserver for over ~200k active users. You can see an outdated version of the stats here: https://matrix.org/faq/#what-is-the-current-project-status and they've said their active userbase has grown 300% in the past year.
The original plan for HOPE was to run multiple homeservers, but we didn't even begin to need to.
EDIT: And eventually p2p which might change everything.
DEFCON should be trivial by comparison, and as others have said, the HOPE server (2500 users or so) was a resounding success - so much so that it’s been set up as permenant fixture now post conference.
Now, it’s true that the Synapse server implementation has to be tuned - for large userbases you need worker processors and to dial up the caches a bit. And needless to say you need postgres rather than sqlite.
Kinda depressing that nobody from DEFCON synced with the Matrix team (to my knowledge) on how best to do this, rather than just writing it off as slow and bloated.