I think people forget that the standard back then in the industry was very high margin servers with redundant everything (psu, network/nic, disks, etc).
Going to (eventually) mostly dependable simple servers with no redundancy was a big leap.
Re:Software Engineering at Google
by shadowgovt on Monday August 14, 2023 @01:12PM
(#37123621)
There were a couple places in Google's internal infrastructure where they had picked up traditional "big iron" architectures (mostly by acquisitions), and it was funny to observe the sheer plane between their regular best practices and those systems (when they hadn't yet been replaced).
One high-ticket item relied on a huge Oracle database. Every other aspect of that product was patch-on-the-fly, silent-release new versions, forward-and-backwards compatibility of software... But the whole product had regular scheduled outages on the weekends because the Oracle part could not be patched on the fly, had a rigid schema baked into its relational architecture, and had to be brought offline for updates and migrations.
It was like everyone was zipping around in racecars and then they all had to pause for Grandpa to cross the street.
There is one underlying technology special sauce they refined to an art form, which is building a reliable system on incredibly unreliable components. This comes from their inception when the two guys building out the company knew how to write code but not build custom machines; their initial attempts at racks (hard drives and motherboards mounted to flexible plywood) are hilarious-looking by modern standards and broke down all the time, so they had to learn how to write software that was fault-tolerant at every link in the chain. That resulted in building an infrastructure that was extremely supportive of experimentation (if the system can survive half of it disappearing, it can survive a software bug crashing half the machines), and the rest of the shape of the company kind of flows from that.
It is worth repeating a lot that most companies do not operate at the scale or constraints Google does, and their approach to engineering is emphatically not one-size-fits-all.