News for hackers. Stuff that matters.
Re:Burnt $72k testing Firebase and Cloud Run and almost went... by dijit on Thursday December 10, 2020 @06:15PM (#25380109)
> but man do I wish there was a way to put down a spending limit and simply disable anything that goes over that limit.

Literally did this my first week when trying out GCP for my company. It is entirely possible and documented (with code):

https://cloud.google.com/billing/docs/how-to/notify#cap_disa...

Re:Burnt $72k testing Firebase and Cloud Run and almost went... by dathinab on Thursday December 10, 2020 @06:48PM (#25380386)
> Note: There is a delay of up to a few days between incurring costs and receiving budget notifications. Due to usage latency from the time that a resource is used to the time that the activity is billed, you might incur additional costs for usage that hasn't arrived at the time that all services are stopped. Following the steps in this capping example is not a guarantee that you will not spend more than your budget. Recommendation: If you have a hard funds limit, set your maximum budget below your available funds to account for billing delays.

(Source link in parent post, emphasis mine).

In this case they had a additional cost due to delay of $72k. Which, lets be honest means this feature kinda useless for anything but the somewhat harmless case.

Only by combining this with resource limits in load balancers, instance and concurrency limits and similar can the maximal worst cost be limited. But tbh. this partially cripples auto-scaling functionality and it's really hard to find a good setting which doesn't allow to much "over" cost and at the same time doesn't hinder the intended auto-scaling use-case.

Re:Burnt $72k testing Firebase and Cloud Run and almost went... by donmcronald on Thursday December 10, 2020 @07:40PM (#25380856)
> it's really hard to find a good setting which doesn't allow to much "over" cost and at the same time doesn't hinder the intended auto-scaling use-case

> I created a new GCP project ANC-AI Dev, set up $7 Cloud Billing budget, kept Firebase Project on the Free (Spark) plan.

There's a lot of middle ground between $7 and $72k. Your quote explains it perfectly though. They flat out can't because the accounting and reporting is badly designed and incapable of providing (near) real-time data.

IMHO the easiest solution to this is government regulation. If you set a budget for a pay-what-you-use online service there should be legislation forbidding companies from charging you more than that.

I also find it (sort of) hilarious they can magically lock the whole thing down once payment fails, but not before the CC is (presumably) maxed out. Lol. Talk about a good deal for Google.

Re:Burnt $72k testing Firebase and Cloud Run and almost went... by throwclowd on Thursday December 10, 2020 @07:10PM (#25380613)
There's something uncanny about understanding the situation enough to turn on the budget alerts, while at the same time not realizing it's not going to help in time if your system runs amok.
Re:Burnt $72k testing Firebase and Cloud Run and almost went... by ggthrowaway2020 on Thursday December 10, 2020 @07:52PM (#25380951)
I'm not sure if you meant it this way, but your tone makes it seem like the parent just needs to "read the docs".

Unfortunately for all of us, your solution doesn't work, per the huge disclaimer on the page that says those alerts can be days late. You can rack up an almost unlimited $ bill in hours.

Re:Burnt $72k testing Firebase and Cloud Run and almost went... by alasdair_ on Thursday December 10, 2020 @06:29PM (#25380220)
The article says that they had a limit in place but that in practice the billing limit lags up to 24 hours behind the "real" number.
Re:Burnt $72k testing Firebase and Cloud Run and almost went... by merb on Thursday December 10, 2020 @07:17PM (#25380673)
thats not the best thing you can do. the best thing you can do is put excessive time into quotas. aws has way better quotas for starters than gcp has, sadly