News for hackers. Stuff that matters.
Re:Burnt $72k testing Firebase and Cloud Run and almost went... by tunesmith on Thursday December 10, 2020 @05:27PM (#25379654)
Deeply dissatisfying to read. Ex-Googler uses connections to get his (understandable!) cloud mistake refunded.

Every time I read one of these stories, I get more and more convinced I will just simply never use scalable cloud tech for my side projects. I'm not going to risk my family's retirement savings on the all-too-possible chance that a small deep-implication error will cause runaway charges.

Re:Burnt $72k testing Firebase and Cloud Run and almost went... by sudcha on Thursday December 10, 2020 @06:54PM (#25380449)
OP here.

Your assumption is incorrect. I haven't been in touch with anyone in Google, and used 0 internal connections. Happy to make another post with my conversation + documentation to support this.

I reached out to the GCP through their regular channels. This is not a paid post, and we are not sponsored by Google in anyway.

Re:Burnt $72k testing Firebase and Cloud Run and almost went... by tunesmith on Thursday December 10, 2020 @08:07PM (#25381066)
You might want to take another look at your paragraph:

> Having been a Googler for ~6.5 years and written dozens of project documents, incident reports, and what not, I knew how to put the case for Google team when they would come back to work in 2 days.

That certainly reads as an advantage that most non-Googlers would not have.

Re:Burnt $72k testing Firebase and Cloud Run and almost went... by sudcha on Thursday December 10, 2020 @09:36PM (#25381602)
Thanks for sharing, I see now what you mean.

I'll share the doc I prepared and sent to Google in my consult, in one of the next posts.

Re:Burnt $72k testing Firebase and Cloud Run and almost went... by tunesmith on Friday December 11, 2020 @12:45AM (#25382730)
FWIW, I didn't intend "deeply dissatisfying" as criticism against your writing, although I phrased that poorly so I can understand it coming across that way. If anything I feel for you when that unfair surprise hit you. It just sucks that it's possible, and that the odds feel against us when we're seeking a refund.
Re:Burnt $72k testing Firebase and Cloud Run and almost went... by swyx on Saturday December 12, 2020 @03:14AM (#25396664)
it's an advantage, sure, but still he was right in saying he did not use internal connections as alleged. calm down everyone.
Re:Burnt $72k testing Firebase and Cloud Run and almost went... by betwixthewires on Thursday December 10, 2020 @08:48PM (#25381304)
Yeah, that would be a very good idea. The way it is stated gives that impression, and helping people how to understand how to resolve an issue like this would be priceless IMO.
Re:Burnt $72k testing Firebase and Cloud Run and almost went... by tbodt on Thursday December 10, 2020 @08:49PM (#25381310)
Yes, please make a post sharing your knowledge on how to convince them. I suspect most people would have no idea how to pull this off.
Re:Burnt $72k testing Firebase and Cloud Run and almost went... by Exuma on Thursday December 10, 2020 @06:59PM (#25380497)
You only wrote half the story and no technical details...... why even write a post at all, it's just clickbait and you give no other information how what went wrong, how it got fixed, what exact technical problem was...
Re:Burnt $72k testing Firebase and Cloud Run and almost went... by dsymonds on Thursday December 10, 2020 @07:33PM (#25380798)
Did you read Part 2, linked from the bottom? It has plenty of technical details.
Re:Burnt $72k testing Firebase and Cloud Run and almost went... by CamelCaseName on Thursday December 10, 2020 @08:11PM (#25381092)
Up until this comment, I did not realize that was a link either.

Perhaps OP can edit the bottom to make it more clear?

Re:Burnt $72k testing Firebase and Cloud Run and almost went... by aidenn0 on Thursday December 10, 2020 @07:39PM (#25380842)
I didn't realize that was a link; seriously why does every website have to make links look different.
Re:Burnt $72k testing Firebase and Cloud Run and almost went... by betwixthewires on Thursday December 10, 2020 @08:49PM (#25381311)
Lol to me it looked like part 2 wasn't written yet, but I clicked it anyway just to check and the page loaded, so I read it. No real downside in getting a 404.
Re:Burnt $72k testing Firebase and Cloud Run and almost went... by joshmanders on Thursday December 10, 2020 @06:09PM (#25380039)
I'm absolutely astounded that cloud providers allow stuff to get this way and don't even go "ope, this looks out of the ordinary, we should look into it" Nor do they offer the ability to straight up kill all services if exceeding a certain price set by the customer.

Cloud is still good though, I believe it's the future. I just don't believe in deceiving your customers to hopefully rack up a high bill with them.

I view not having the ability to say "Shut everything down if I go over $100/mo" the same as pre-checked hidden cross sales that MasterCard/Visa cracked down on in the adult industry few years ago. Just money grabs.

I will definitely be putting such measures in my cloud platform.

Re:Burnt $72k testing Firebase and Cloud Run and almost went... by betwixthewires on Thursday December 10, 2020 @08:45PM (#25381287)
I'm with you. I have yet to use any of these cloud computing providers for building or testing anything, and it is partly due to this (and partly due to privacy and confidentiality considerations).

"Yay! NBD! Google is the best! All I had to do was work there for a few years, rub elbows, make connections and ask for favors!" Really, google would be the best if there was no way to accidentally go over your stated budget cap by 86 million percent, or at the very least have a policy to refund people who can demonstrate that this is what happened.