Re:Mistakes I've Made in AWS
by BackBlast on Saturday September 11, 2021 @05:09PM
(#28496058)
I've been there. I shut down a bunch of what looked like idle instances doing nothing to reduce spend. 80% of which were, in fact, doing nothing. I did drop off two vms that were supporting critical infrastructure.
Everyone who had done any work on them was long gone. I had done my due diligence to identify what they could possibly be.
Still, the day of reckoning came, and we got calls of services down a week after I turned them off. I spun them back up, and they were going again without any real impact to the business.
This turned out to be a blessing as the very next week the cert these same services depended on expired and if I hadn't learned about the system by turning them off we never would have known which boxes held up those services.
Also a lesson in what happens when people leave without any documentation on where the work they did lives and how it works.
Everyone who had done any work on them was long gone. I had done my due diligence to identify what they could possibly be.
Still, the day of reckoning came, and we got calls of services down a week after I turned them off. I spun them back up, and they were going again without any real impact to the business.
This turned out to be a blessing as the very next week the cert these same services depended on expired and if I hadn't learned about the system by turning them off we never would have known which boxes held up those services.
Also a lesson in what happens when people leave without any documentation on where the work they did lives and how it works.