Re:CIO update: Post-mortem on the Skype outage
by jpablo on Wednesday December 29, 2010 @10:57AM
(#2049037)
If 20% of your network goes down and you can still serve clients normally, it means that you have a big reserve of machines useful only in case in big outages. I don't know if you can justify it economically.
Just spin more EC2 instances ?
Re:CIO update: Post-mortem on the Skype outage
by michaelbuckbee on Wednesday December 29, 2010 @11:33AM
(#2049181)
That's an interesting thought: in case of outage Skype could switch from user supplied resources (Supernodes eating users bandwidth and processing) to emergency Skype hosted supernode services.
Re:CIO update: Post-mortem on the Skype outage
by toumhi on Wednesday December 29, 2010 @11:24AM
(#2049151)
Yes, if you use an elastic cloud, by all means, spin more instances :-) Most existing companies still have real servers however.
Re:CIO update: Post-mortem on the Skype outage
by bonzoesc on Wednesday December 29, 2010 @11:47AM
(#2049237)
Most existing companies don't run P2P voice chat networks, either. Using EC2 or some other elastic cloud for emergency supernodes makes a lot of sense, since they can outsource the risk of those machines sitting idle to Amazon.
You can also gracefully degrade performance, by rejecting client connections, disconnecting progressively some clients, accepting loss of consistency etc. It depends how far you can go without infuriating your customers.
We discovered that large-scale real-time systems(in our case, currently 400.000 concurrent connections) are really hard to stabilize against presence storms, network problems and buggy clients, among others.