Then last month I got an email saying "Hey, those quotas you were setting using the API documented to set quotas, those were actually not being enforced the whole time because of undocumented issues with our systems." So basically you can't rely on the documented behavior of these systems, there's no good way to test whether your code is correct or whether your limits will work without actually exceeding your budget for real, and the whole thing is a clusterfuck. When you get a surprise bill you just have to throw yourself at the mercy of whichever first line billing support rep is randomly assigned to your case.
Limiting your bill to something less than "potentially infinite" is just a basic fundamental feature that shouldn't require rolling your own bill-monitoring service relying on poorly documented and malfunctioning APIs with no provision for testing. There's no excuse strong enough to explain why the cloud providers can't do something reasonable here.
The "tiny bit of lag" between usage and billing calculation explodes when there's a lot of usage - in my case, a broken job tried resubmitting itself continuously, and the lag increased to 8 hours and $5000 just when I needed the alert the most. My team's response time was 5 minutes... After the 8 hour GCP lag.
Very similar to this guy's story: https://blog.tomilkieway.com/72k-1/
I had to go back and forth with them on email for weeks, and ultimately threaten them with a draft blog post with a lot of graphs and screenshots of their recommendations for them to cancel the bill.
I’d love it if GCP’s official method were to disable billing if your bill went over a limit.
Sadly, I suspect it would just disable systems instead.
Is this like asking the phone company "When I reach my plan limits, stop charging me money but let me keep making calls?"