We incurred a 6 figure bill when the API an authentication token handling lambda was updating from was taken down. The lambda went into a crazy loop self invoking, as it had a retry mechanism and a CRON schedule, which piled invocations on top of retries. (over worked team, poor design, etc.)

So far we have gotten no concessions from AWS, and we have annual bills in the millions, just not for this application whose budget now has an awkward and obvious spike.