Re:Command-line tools can be faster than a Hadoop cluster (2...
by La1n on Thursday January 30, 2020 @05:35AM
(#22190393)
As someone who doesn't know a lot about this stuff, is there a way to have the base load internal and use a cloud provider for elastic load?
Re:Command-line tools can be faster than a Hadoop cluster (2...
by krab on Thursday January 30, 2020 @06:48AM
(#22190707)
Yes, with a caveat which is latency. For example it's hard to split a typical webapp so the application servers are far from the database. The latency kills performance.
On the other hand, for stuff like CI jobs, batch processing or anything else which doesn't depend on tight synchronization with the other location, you can mix and match.
And we'd need to have staff to manage the physical servers, run our network infrastructure, etc.
We've been through the numbers many times and the cloud always wins, even when compared to running our own servers for base load and have some hybrid cloud that lets us expand into AWS for peak loads. (which saves some dollars in servers, but that doesn't make up for the added complexity).
And nearly for "free", we have a cold standby site that has a replica of our hot data (a small portion of our full dataset), so if there was a full region outage, we could be back up in running in 30 minutes with reduced functionality.