Yep, we had an incident with Mongo cloud where a bug in their synchronization protocols for Mongo Realm resulted in an insane amount of traffic. This was a development cluster with almost no application load somehow pumping around many TB over the course of a few days. The bill was many thousands of dollars. Their support did the right thing. And we actually ended up with some credits because we were having a rough time with bugs in their software. Ultimately, we gave up on Mongo Realm because it was just not working as advertised for us (high CPU usage on the device, lots of bandwidth, we experienced data loss in the managed cloud storage, etc.). But their support team was great.

Their interests is keeping you as a long term customer. So, they will help you if they can. Unexpectedly high bills like that can end the relation in no time. And 10K is not a lot on a yearly basis. That's a few months of normal usage for lots of companies. So, protecting that revenue is worth something to them. That's also worth realizing when you deal with cloud providers: you are spending non trivial amounts of money on their services and support is part of that deal.

A developer on a team I worked with many years ago accidentally committed our AWS keys in a repo. Got a $30k bill due to a an enormous amount of EC2-instances being spawned. We contacted AWS and they were very understanding and reduced the bill to $50.

I had this happen to me once on Digital Ocean, and I contacted them - they were rather understanding that the bill I had was clearly "atypical for my account and not intended" and refunded it.

I'll second that.

I've seen several cases on both Azure and AWS that bills got weaved after someone opened support ticket starting with "oops, I just did..."

I got an $800 aws expense (one line item) waived after I contacted them and they asked me to explain why it happened and how I’ll prevent it from happening in the future. I think it’s a once per account thing they’d probably do and Troy should definitely do it.

> Secondly, there's cost alerts. I really should have had this in place much earlier as it helps guard against any resource in Azure suddenly driving up the cost.

He did not enable alerts.

Every online course that requires you to use a public cloud to deploy something should first have you set up a billing alert that notifies you when costs start to creep past something reasonable, like $20 or $50 (depending on the course and work involved).

OP do this! It works, they are usually very generous (same for gcloud!)

Op is Troy hunt, an ms MVP. You can bet there are people from MS doing it for him as soon as they got wind

100% do this. Azure has a surprisingly responsive billing support team and will likely eat this as goodwill (honestly with this on the front page of HN they'll probably do it proactively). Just open a ticket in the portal.

Also there is 0% chance serving this traffic cost Microsoft anything near $10000.

As opposed to all those other customers who are not good faith actors?

You’re trying to be snarky to GP, why exactly? Yes there are bad faith actors that might try to get some free cash out of cloud refunds. And other customers can also be good faith actors and included in the assertion.

The post applies to everyone and I’d second it. Ask nicely for a refund in these situations, the worst that can happen is they say no.

Where did they say that “only Troy Hunt shall receive a refund, for only Troy Hunt is a good faith actor, so say we all”?

No snark intended. It's just that I would assume that all customers of such services are in principle good faith actors, not just Troy.

The one thing that is special about Troy is that he is providing a service for the public good but that has nothing to do with being 'good faith' or not.

The response was snarky but it's meaning is true. Troy Hunt will get the refund without any problems because he is a public figure. But if a John Doe will make the same mistake he can only hope someone at AWS/Azure/GCP will lift his fees too - which is not guaranteed!

I think there's a dozen people in this thread who have gotten refunds and not a person saying they were refused?

The point is that if they give refunds to all the good actors they won't make any money.

If you have a long time customer (especially one who brings in as much good publicity as Troy Hunt) and you look at their billing history this spike would be a clear anomaly. Writing off an $8k bill to keep a customer around and happy for years to come is worth more than that bill.

Only if their regular services run at a loss and their business model relies on people making mistakes.

But Azure’s regular prices are definitely high enough that they’re not a loss leader.

> The point is that if they give refunds to all the good actors they won't make any money.

That assumes the only way they make money is on peoples understandable mistakes or lack of care. Doesn't seem to be the case for these services (unlike, say, many gym subscriptions).

It seems far more likely that if they refunded all the well documented issues like this one, their bottom line wouldn't be impacted.