News for hackers. Stuff that matters.
Re:Ask HN: Is S3 down? by JPKab on Tuesday February 28, 2017 @04:40PM (#13758211)
I've used both Google Cloud and AWS, and as of a year or so ago, I'm a Google Cloud convert. (Before that, you guys didn't at all have your shit together when it came to customer support)

It's not in bad taste, despite other comments saying otherwise. We need to recognize that competition is good, and Amazon isn't the answer to everything.

Re:Ask HN: Is S3 down? by eknkc on Tuesday February 28, 2017 @05:58PM (#13759054)
We were on GCP for around a year, it was my decision I really wanted to love GCP and I initially did. But we recently switched to AWS.

I think there is little GCP does better than AWS. Pricing is better on paper, but performance per buck seems to be on par. Stability is a lot worse on GCP, and I don't just mean service outages like this one (which they had their fair share) but also individual issues like instances slowing down or network acting up randomly. Also lack of service offerings like no PostgreSQL, functions never leaving alpha, no hosted redis clusters etc... Support is also too expensive compared to AWS.

Management interfaces are better on GCP and sustained use discount is a big step up against AWS reservations. Otherwise, I think AWS works better.

Re:Ask HN: Is S3 down? by aardshark on Wednesday March 01, 2017 @11:34AM (#13765154)
I haven't used AWS, but my experience with AppEngine and by extension GCP is similar.

Just last week I got an email saying that they'd discovered an issue on Google Cloud Datastore where certain (strongly consistent!) queries could have been returning incorrect results for a week long period and that I should check my logs to see if anything important had been affected in my application.

That's not the sort of behaviour that inspires confidence in a service.

Re:Ask HN: Is S3 down? by fidget on Tuesday February 28, 2017 @06:20PM (#13759247)
Functions are going beta this week
Re:Ask HN: Is S3 down? by Moru on Tuesday February 28, 2017 @06:50PM (#13759475)
And discontinued next year?
Re:Ask HN: Is S3 down? by benley on Tuesday February 28, 2017 @07:00PM (#13759543)
The standard "lol google will kill it in 6 months anyway" troll doesn't really apply to Google Cloud services. They know better than to be fickle with infrastructure offerings.
Re:Ask HN: Is S3 down? by themihai on Tuesday February 28, 2017 @11:51PM (#13761241)
Are you sure? Have you used appengine or any of their cloud libraries? Just found out that some of the services I wrote 3 years ago don't work anymore due various breaking changes. It also very much applies to cloud services themselves! What happened to the email service(appengine powered)?I'm telling you: it no longer exists! Compare that with AWS SES which gets better and better. I could go on and on all day long. Google cloud is nice on paper but fails in practice. If you consider the lock-in it is not worth it on paper either
Re:Ask HN: Is S3 down? by davidjnelson on Wednesday March 01, 2017 @12:21AM (#13761397)
That happened to me and my sites were down for four months. I lost 15 years of seo, and $30/mo revenue... They just shut down the Python app engine sites with no notice. I'll never use google cloud.
Re:Ask HN: Is S3 down? by scrollaway on Wednesday March 01, 2017 @09:58AM (#13764115)
What? That sounds like nonsense. Some sources please.
Re:Ask HN: Is S3 down? by davidjnelson on Thursday March 09, 2017 @01:01AM (#13827333)
I wish it were. It was an enormous amount of effort to get my pagerank that high. I suppose I could post my google analytics before and after, but that's not really data I share with the public.
Re:Ask HN: Is S3 down? by scrollaway on Thursday March 09, 2017 @08:30AM (#13829071)
I'm not doubting your site was "down for four months". I'm doubting the part where you said "They just shut down the Python app engine sites with no notice".

Most notably, I know many people who run these types of sites and outside of GAE being mediocre, I've never heard them complain about anything like that.

Re:Ask HN: Is S3 down? by davidjnelson on Monday March 13, 2017 @12:25PM (#13859244)
This was the very first version of app engine that rolled out in 2008 or so. It showed some type of "incompatible version" notice in the admin ui when I noticed it, and when I tried to redeploy the sites using the command line deploy tool. I switched everything to s3.
Re:Ask HN: Is S3 down? by benley on Wednesday March 01, 2017 @01:57AM (#13761830)
How much advance notice did they give when they were shutting down that email service? I bet it was like eight months or more. Things change, you have to deal with it sometimes. Seems kinda normal.
Re:Ask HN: Is S3 down? by themihai on Wednesday March 01, 2017 @11:52PM (#13771514)
People don't like to rewrite infrastructure code just b/c the provider decided it's not worth it anymore. When you sign up you consider the whole ecosystem not individual services. The cloud platform is marketed as reliable and rock solid that you can trust. It may be the case with AWS but on Google you should expect experimental, cheap and a high risk to get broken or even deprecated all together. It behaves like a start-up with customers paying for experiments. It's OK for some use cases but you should be aware of that.
Re:Ask HN: Is S3 down? by Moru on Tuesday February 28, 2017 @07:05PM (#13759580)
No trolling, just tired of setting up things that just stop working and forcing me to work on a fix. I dont work for a big company with a dev team of 20 people, its just me and customer support. Im close to a burn out as it is, I dont need help with it.
Re:Ask HN: Is S3 down? by foxylad on Tuesday February 28, 2017 @08:02PM (#13759973)
I've used Appengine since 2009. Early on they deprecated the original master-slave datastore, but apart from that I've had zero refactoring around their services.

Other services are a different story - from my perspective Google are better at supporting legacy interfaces than most.

Re:Ask HN: Is S3 down? by iampims on Tuesday February 28, 2017 @10:28PM (#13760792)
They deprecated their alpha search api 2 years in iirc
Re:Ask HN: Is S3 down? by fidget on Tuesday February 28, 2017 @07:16PM (#13759669)
They have a minimum of 1 year deprecation policy in their terms of service.
Re:Ask HN: Is S3 down? by ChristianBach on Tuesday February 28, 2017 @07:29PM (#13759757)
I hear you, discontinuing products that you're dependent on is painful, but discontinuing services that you built your infrastructure around is an outright killer.
Re:Ask HN: Is S3 down? by beambot on Tuesday February 28, 2017 @08:05PM (#13759993)
Counterpoint from an email I received just last week (Feb 21st):

> We are writing to inform you that we are winding down sales and renewals of Google Site Search (GSS). Starting April 1st, 2017, new purchases and renewals of GSS will not be available.

Re:Ask HN: Is S3 down? by kyrra on Tuesday February 28, 2017 @08:24PM (#13760117)
GSS isn't under Cloud and doesn't have the same deprecation policy. Cloud explicitly states that it has a minimum 1-year turndown on any feature they disable.
Re:Ask HN: Is S3 down? by beambot on Tuesday February 28, 2017 @08:51PM (#13760239)
> They know better than to be fickle with infrastructure offerings

Site Search seems like an infra offering to me.

Re:Ask HN: Is S3 down? by inimino on Tuesday February 28, 2017 @09:15PM (#13760356)
Cloud is still competing with on premises solutions, right? One year is nothing, try ten or twenty.
Re:Ask HN: Is S3 down? by newjersey on Tuesday February 28, 2017 @11:32PM (#13761134)
> Cloud is still competing with on premises solutions, right? One year is nothing, try ten or twenty.

Not an expert by any means but I would put more weight to Google's ONE year promise over (to give an example) HPE's twenty years promise. I know it is a cheap shot because I am pretty sure HPE will be bought and sold at least once in the next twenty years.

Re:Ask HN: Is S3 down? by xchaotic on Wednesday March 01, 2017 @11:32AM (#13765145)
FWIW I am now dealing with a system that is supported by HPE for over 10 years now. Even if they get bought out, someone will inherit those support obligations. I am also in the camp of not trusting Google with anything.
Re:Ask HN: Is S3 down? by epalmer on Wednesday March 01, 2017 @10:56AM (#13764687)
GSS shutdown gives us a year also. I just migrated to GSS a year ago. And they decommissioned they big appliance Google Enterprise Serarch I think it was called.

We were users of the Google Mini Search appliance, went to a 3rd party in-house installed search solution that we did not like and then a year ago went to GSS. We are looking again for something suitable. The best part of the Google Site Search was search fidelity.

Re:Ask HN: Is S3 down? by allizad on Wednesday March 01, 2017 @05:01PM (#13768903)
Try open-source search solutions? :)
Re:Ask HN: Is S3 down? by ExactoKnight on Tuesday February 28, 2017 @10:48PM (#13760923)
They did it to their Google Analytics API. We spent four months building a dashboard off of it then Google fucking deprecated it. Thanks Google.
Re:Ask HN: Is S3 down? by anotherturn on Wednesday March 01, 2017 @12:50AM (#13761524)
Not entirely true. App engine depreciations happen all the time and they give about 1 years notice. Most recently the channel api, before that prospective search, backends, etc etc
Re:Ask HN: Is S3 down? by benley on Wednesday March 01, 2017 @01:58AM (#13761835)
"oh no I only have _an entire year_ to deal with an API deprecation what ever will I do?" :-P
Re:Ask HN: Is S3 down? by bartread on Wednesday March 01, 2017 @02:26AM (#13761987)
Well, if it's something you've built your entire offering around it could be a simple fix, or it could be months of work, and that will vary by project. Bear in mind that this is completely non-value-adding work that you didn't plan on just to bring your project back to a functioning state.

I.e., some douchebag who has no interest or stake in what you do has just dumped a potentially substantial amount of technical debt into your product backlog and, quite possibly, prioritised it all the way to the top.

As somebody else noted above: I don't need people creating more work from me. I can do that quite well enough on my own, thanks very much, and for side-projects this kind of chopping and changing is a pain in the ass.

By definition, with side-projects time is limited, so you absolutely have to focus on the most valuable activities to the exclusion of all else. For this reason, I only consider AWS and Azure for my projects: Google are just too fickle. Lucky you, if you have the time to deal with their nonsense.

(Btw, I'm not dissing Google on a technical level - they obviously do great, interesting work, and they're certainly one of the pioneers of PaaS. I just don't need the hassle of having to fix stuff because they keep killing APIs, projects, services.)

Re:Ask HN: Is S3 down? by anotherturn on Wednesday March 01, 2017 @03:46AM (#13762296)
Yes exactly. The app specifically affected by Channels API depreciation is a side project that serves a few thousand people. It marches along perfectly well, and I pay Google money for it each month - though the project itself makes no money. Now, I need to consider whether the shift from Channel API to Firebase (and the few days work it'll take to do) is worth the investment, or if I should just shut it down.
Re:Ask HN: Is S3 down? by imron on Tuesday February 28, 2017 @10:06PM (#13760632)
> doesn't really apply to Google Cloud services

It might not, but doing it so much for other services destroys trust across the entire brand.

Re:Ask HN: Is S3 down? by dstroot on Tuesday February 28, 2017 @10:13PM (#13760681)
I, for one, would love to see Redis and Postgres on GCP. That would be enough to get me to switch I think.
Re:Ask HN: Is S3 down? by doubleorseven on Wednesday March 01, 2017 @12:06AM (#13761327)
Wait for the NEXT event. Radis will be announced.
Re:Ask HN: Is S3 down? by espeed on Tuesday February 28, 2017 @05:25PM (#13758766)
Me too. We switched to Google Cloud years ago at its inception and have never looked back -- always viewed it as a competitive advantage due to its solid, more advanced infrastructure -- faster network, reliable disks, cleaner UI that's easier to manage. Just a cleaner operation all the way around.
Re:Ask HN: Is S3 down? by snackai on Tuesday February 28, 2017 @08:07PM (#13760004)
What indeed is bad taste is your choice of Google Cloud over AWS. No I really like GCP, use it at core of many apps, but if people really want a decentralized web we need to use more than one provider. Don't "convert". Use booth, redundancy ffs.
Re:Ask HN: Is S3 down? by kiallmacinnes on Wednesday March 01, 2017 @04:02AM (#13762358)
Pity I can't upvote more than once! :)

This whole idea of being angry at a vendor for deprecating something with 1yr notice is just ridiculous!

People need to realize they are choosing lock-in, and are choosing the risk of deprecation every time they decide to use a cloud service with no drop in competition/open source/etc.

Own your choices people, don't blame others...

Re:Ask HN: Is S3 down? by unclebucknasty on Wednesday March 01, 2017 @10:57AM (#13764695)
Sounds great on paper, but this is infrastructure level stuff with real world constraints.

The expectation of stability beyond a year is certainly not unreasonable when you're asking people to build their businesses/infrastructure on your platform.

And, building redundancy across providers can be impractical, owed to learning curve, cost duplication, higher outbound bandwidth costs, effort duplication, solution complexity, etc.

Re:Ask HN: Is S3 down? by adjkant on Wednesday March 01, 2017 @01:03AM (#13761595)
What's the point of decentralizing by putting 50% on one, 50% on the other, and no overlap of the groups? You used the word redundancy, but who is willing to actually do that true redundancy?
Re:Ask HN: Is S3 down? by advisedwang on Tuesday February 28, 2017 @06:34PM (#13759353)
I work in GCP support. I'm really curious: what do you feel changed that led to such improved support? I'd like to make sure we keep doing it.
Re:Ask HN: Is S3 down? by anotherturn on Wednesday March 01, 2017 @12:57AM (#13761567)
Chiming in as I noticed the change too. For a long time it was almost impossible to speak with a human - every query was directed to the extensive but often useless support pages. If a human did respond it often seemed like they weren't savvy enough to handle a microwave let alone solve infra issues.

Then, about a year or two ago - humans actually started responding to and fixing problems. A welcome change!

Re:Ask HN: Is S3 down? by advisedwang on Wednesday March 01, 2017 @06:36PM (#13769775)
Do you have one of the paid support packages, or is this your experience of our google groups/stack overflow etc?
Re:Ask HN: Is S3 down? by vacri on Tuesday February 28, 2017 @07:09PM (#13759617)
My experience of support with Google Apps for Business makes me very wary of using anything Google for critical business infra. Google products are nice, but as soon as you hit a problem or edge case, you're on your own in my experience.
Re:Ask HN: Is S3 down? by keithnoizu on Tuesday February 28, 2017 @08:07PM (#13760006)
This.

I used to work on the Azure Portal Team. As much negative things as I can say about Microsoft, they take making things just work for developers seriously, despite high prices and misc. service issues.

The since nixed compute container project I initially worked on really exemplified this.

I tend to use Colo or AWS when possible but I have a client that insisted on Google GCE and Endpoints.

I've spent so much time time digging through source code and working around broken dev tooling, and dealing with incorrect or out of date documentation thanks to that requirement.

In my personal opinion Google has a way to go in mature tooling. Silent failures, or worse failures that don't result in build failures are not acceptable. Requiring paid support contracts to resolve an issue in google infra is not acceptable. Incredibly poor support for local dev environments is not acceptable.

After dealing with this stuff, I find it unlikely that I will ever rely on their systems in the future. AWS/Colo or, with reservations, Azure all the way.

Re:Ask HN: Is S3 down? by spuiszis on Tuesday February 28, 2017 @07:27PM (#13759746)
Wish I could +1 this more. Any time I get some error, I spend hours sifting through old documentation and forum posts.
Re:Ask HN: Is S3 down? by DerpyNirvash on Tuesday February 28, 2017 @07:55PM (#13759925)
Why not just open a support ticket?
Re:Ask HN: Is S3 down? by foxylad on Tuesday February 28, 2017 @08:14PM (#13760051)
Exactly. Because they've usually only experienced support for Google's free services, people assume all Google support is minimal - but it isn't. We pay $150 a month for silver support, and in the extremely rare (several years apart) case we need help, we get it.
Re:Ask HN: Is S3 down? by vacri on Tuesday February 28, 2017 @08:34PM (#13760157)
Google Apps for Business is not free.
Re:Ask HN: Is S3 down? by foxylad on Wednesday March 01, 2017 @03:43AM (#13762281)
True, it costs $5 a month - nearly free. It comes down to the eternal truth that there is no such thing as a free lunch, and expecting a $100/hour support person to be at your beck and call for $5 a month isn't realistic.
Re:Ask HN: Is S3 down? by vacri on Tuesday February 28, 2017 @08:08PM (#13760019)
In my GApps support experience, it's slow and inexperienced.
Re:Ask HN: Is S3 down? by dbg31415 on Tuesday February 28, 2017 @07:12PM (#13759638)
Spot on.

And good luck getting accurate documentation.

Re:Ask HN: Is S3 down? by ehsankia on Tuesday February 28, 2017 @07:13PM (#13759643)
Honestly, if you're a big service that millions of people use, you should not put all your eggs in a single basket and should probably use a mix, in case one of the clouds goes down like in this case.
Re:Ask HN: Is S3 down? by Svenskunganka on Tuesday February 28, 2017 @07:41PM (#13759822)
One of the biggest reasons you go for a cloud is because you don't want to deal with reliability & scaling issues, and there's a premium price attached to that. I think most companies using S3 in this case believed they put their eggs in different baskets when they put their data in there.
Re:Ask HN: Is S3 down? by kiallmacinnes on Wednesday March 01, 2017 @04:12AM (#13762397)
I can't believe anyone would have thought a dependency on a single AWS region, or even single service provider, would count as having eggs in different baskets, at least, I really hope nobody could think that!

I suspect though that most people affected deemed the risks and costs of failure low enough to be acceptable, and for many people it still is - even with this outage. But that's a conscious decision, rather than plain ignorance.

Re:Ask HN: Is S3 down? by Salgat on Tuesday February 28, 2017 @07:16PM (#13759668)
That depends if you're willing to pay for the cost of hosting all your content twice and the development overhead of managing that. Twice the persistences means twice the chance of an issue occurring.
Re:Ask HN: Is S3 down? by nashadelic on Tuesday February 28, 2017 @09:56PM (#13760572)
That's where tech like kubernetes help in making your app/service portable. Or having common APIs like between s3 and google cloud storage.

Twice the persistence means always having at least one backup and thus the occurance of downtime reduces not up

Re:Ask HN: Is S3 down? by w8rbt on Tuesday February 28, 2017 @08:37PM (#13760168)
With containers, I think the devops overhead would be minimal.
Re:Ask HN: Is S3 down? by tejasmanohar on Tuesday February 28, 2017 @10:14PM (#13760691)
That's if _everything_ is in containers. Also, don't undermine how much of a difference the host machine configuration can make... Docker uses its kernel.
Re:Ask HN: Is S3 down? by hkmurakami on Tuesday February 28, 2017 @07:12PM (#13759641)
>(Before that, you guys didn't at all have your shit together when it came to customer support)

Sounds like it basically coincides with Diane Greene coming on board to run the show -- which is great news for all of us with increased competition on not just the technical front but also support (which is often the deal maker/breaker)

Re:Ask HN: Is S3 down? by 7ewis on Tuesday February 28, 2017 @07:24PM (#13759733)
Is Diane really that good?

I was at a talk last year, where she spoke, and as much as I love Google, it was one of the boat boring talks I've ever heard in my life. So monotone and uninteresting... and I'm probably one of the biggest Google fans out there.

Re:Ask HN: Is S3 down? by contingencies on Tuesday February 28, 2017 @07:29PM (#13759755)
I have no idea about the person in question, but stable and reliable infrastructure can be really boring. Unfortunately, it's also necessary.
Re:Ask HN: Is S3 down? by hkmurakami on Tuesday February 28, 2017 @08:24PM (#13760118)
I'm not familiar with her public speaking. But you want someone decidedly un-Google-like to run an enterprise software (non-engineering) operation.

Look at Safra Catz's public speaking (Oracle). Terrible public speaker, terrific operator [1].

[1] though we may easily disagree with their business practices.

Re:Ask HN: Is S3 down? by jamesblonde on Tuesday February 28, 2017 @08:55PM (#13760257)
I just wrote a piece reflecting on the s3 outage and the limitations of s3 metadata/replication:

https://medium.com/@jim_dowling/reflections-on-s3s-architect...

Re:Ask HN: Is S3 down? by jamesblonde on Wednesday March 01, 2017 @01:23AM (#13761684)
Re:Ask HN: Is S3 down? by themihai on Tuesday February 28, 2017 @11:44PM (#13761199)
GCP has always felt like a forever beta product. On top of that you get a lot of lockin so I would never recommend GCP for a long term project.