News for hackers. Stuff that matters.
Re:Apple cloud services outage by akulbe on Thursday June 02, 2016 @06:05PM (#11826484)
Maybe I'm crazy, but it seems too coincidental that multiple services

(Amazon, Facebook, Apple... that I know of, so far)

have been affected, on the same day.

Re:Apple cloud services outage by zalmoxes on Thursday June 02, 2016 @06:34PM (#11826670)
It's related to Akamai DNS.
Re:Apple cloud services outage by kornish on Thursday June 02, 2016 @07:43PM (#11827160)
Do you have a source?
Re:Apple cloud services outage by zalmoxes on Thursday June 02, 2016 @07:48PM (#11827194)
Trying to access some APIs I work with (DEP,V PP) earlier today and seeing 'akadns' as the common providrer.
Re:Apple cloud services outage by geerlingguy on Thursday June 02, 2016 @06:43PM (#11826727)
Charter (ISP) was also out for over an hour—first time this year it was a complete outage and not just a DNS issue.
Re:Apple cloud services outage by feld on Thursday June 02, 2016 @07:07PM (#11826892)
I've been on Charter all day VPN'd into work with no interruption. Although I have a business connection at home, so it's possible I'm avoiding whatever middleboxes they jam residential traffic into.
Re:Apple cloud services outage by vonklaus on Thursday June 02, 2016 @08:55PM (#11827562)
my ISP Cox, had server failure for several hours about 2 weeks ago. It was a DNS issue as well. I don't want to take 2 unrelated things and call them similar, but one has to imagine if some of the above posters are substantiated several huge players losing peering/connectivity and 2 very large ISPs having non-trivial multi-hour multi-region DNS outages is bad. Horrible coincidence.

I have been losing google docs intermittenly all day (and I have been working nearly exclusively in google docs all day) maybe just me, but I have been unable to upload photos (same one as yesterday to gdocs) and from drive/docs dash I would lose connectivity and have sever lag loading up a doc I just worked in. Had to close browser several times.

Re:Apple cloud services outage by Keyframe on Thursday June 02, 2016 @06:23PM (#11826596)
Maybe one of the backbone carriers has a bad day?
Re:Apple cloud services outage by jlgaddis on Thursday June 02, 2016 @06:33PM (#11826661)
Perhaps, but I (network engineer at an ISP) haven't heard anything like that, and all of these services have multiple upstreams and private peerings.
Re:Apple cloud services outage by astrodust on Thursday June 02, 2016 @07:55PM (#11827244)
Comcast has as few peerings as they can legally get away with. Redundancy isn't high on their list of things to care about, apparently.
Re:Apple cloud services outage by jlgaddis on Friday June 03, 2016 @05:12PM (#11833310)
> ... as they can legally get away with.

Can you clarify a bit? I'm not sure what you mean.

An ISP isn't, by law, required to peer with anyone.

Re:Apple cloud services outage by astrodust on Saturday June 04, 2016 @06:37PM (#11838501)
If they didn't peer with anyone they'd be a closed network and it would be illegal to advertise that as "internet access".

They're legally required to live up to the claims of their product. If they say X Mbit/s then there needs to be at least a plausible case made that they can achieve those results. Anything significantly less is to invite a class-action lawsuit.

Re:Apple cloud services outage by Raphmedia on Friday June 03, 2016 @09:49AM (#11830268)
Well that was strange. Wonder what it was.
Re:Apple cloud services outage by draw_down on Thursday June 02, 2016 @06:27PM (#11826614)
Why?
Re:Apple cloud services outage by cookiecaper on Thursday June 02, 2016 @06:34PM (#11826672)
New NSA program rolling out? O_O
Re:Apple cloud services outage by kchoudhu on Thursday June 02, 2016 @07:23PM (#11827012)
You laugh, but I guarantee you that the NSA has systems reliability issues just like the rest of us.
Re:Apple cloud services outage by jrockway on Thursday June 02, 2016 @07:48PM (#11827195)
The last person that guaranteed me something about the NSA is now exiled in Russia ;)
Re:Apple cloud services outage by cookiecaper on Thursday June 02, 2016 @09:19PM (#11827658)
This comment turned out to be surprisingly controversial for a pretty obvious joke. I've observed it at +4, down to +1, back up to +3, and now it's at 0! Heh. Didn't mean to rustle jimmies of any NSA contractors that may read HN. I'm sure in real life your rollouts are much smoother than this. :P
Re:Apple cloud services outage by linkregister on Thursday June 02, 2016 @09:30PM (#11827695)
I'm sure the NSA employees and contractors are just happy to see non-threatening post about the agency.

The downvotes probably came from people tired of the joke. It is an oft-repeated joke.

Re:Apple cloud services outage by draw_down on Saturday June 04, 2016 @08:27AM (#11836089)
Well, I agree it got too much attention, that's for sure.
Re:Apple cloud services outage by sam_pointer on Thursday June 02, 2016 @07:15PM (#11826961)
No. That was the Syria/Iraq/Iran BGP outages from earlier today.
Re:Apple cloud services outage by api on Thursday June 02, 2016 @06:06PM (#11826488)
Are they all on EC2? Maybe EC2 is throwing a fit.
Re:Apple cloud services outage by luhn on Thursday June 02, 2016 @06:09PM (#11826498)
Amazon is, but Apple and Facebook both operate their own data centers.

Edit: I take that back, Apple uses several cloud providers, including AWS.

Re:Apple cloud services outage by therein on Thursday June 02, 2016 @07:14PM (#11826956)
Apple does use S3 for storage. You didn't hear it from me.
Re:Apple cloud services outage by ghshephard on Thursday June 02, 2016 @07:46PM (#11827180)
You just have to run littlesnitch, and see all the AWS servers that Apple applications hit. I don't think it's a secret that a ton of Apple applications are backended by AWS.
Re:Apple cloud services outage by niels_olson on Thursday June 02, 2016 @08:35PM (#11827485)
All hail Little Snitch. So much insight available in that app. I wish there were rule sets for Little Snitch like there are for AdBlock Plus.
Re:Apple cloud services outage by AckSyn on Thursday June 02, 2016 @08:56PM (#11827566)
lost -i in the Terminal.app will do this as well.

With a little grep-fu you could have yourself a good watchdog.

Re:Apple cloud services outage by blowski on Friday June 03, 2016 @01:12AM (#11828461)
I get 'command not found' so I'm assuming I have to install something? But I can't find anything when I search Google because of the generic name.
Re:Apple cloud services outage by syassami on Friday June 03, 2016 @01:29AM (#11828515)
I believe he meant to write `lsof -i`
Re:Apple cloud services outage by blowski on Friday June 03, 2016 @04:00AM (#11828954)
That works a lot better! Thanks both.
Re:Apple cloud services outage by AckSyn on Friday June 03, 2016 @05:01PM (#11833254)
oh i'm still not used to the autocorrect on OS X. Thank you for correcting my error.
Re:Apple cloud services outage by blowski on Friday June 03, 2016 @01:08AM (#11828444)
Indeed, I've read multiple articles about it, some of which seem to have been fed by the Apple PR machine:

http://www.zdnet.com/article/apple-cuts-aws-spending-signs-w...

Re:Apple cloud services outage by RKearney on Thursday June 02, 2016 @07:20PM (#11826992)
They also use AT&T and Azure for storage too.

You did hear that from me, or anyone else with a firewall running on OS X.

Re:Apple cloud services outage by conradev on Thursday June 02, 2016 @09:36PM (#11827715)
and Google Cloud Storage as well!
Re:Apple cloud services outage by tedmiston on Thursday June 02, 2016 @08:42PM (#11827513)
...AT&T for data storage?
Re:Apple cloud services outage by sethhochberg on Thursday June 02, 2016 @09:32PM (#11827704)
https://www.business.att.com/enterprise/Portfolio/cloud/

They're a massive company. When you already own telecom networks and run your own datacenters, selling cloud infrastructure services isn't too far fetched.

Re:Apple cloud services outage by rconti on Thursday June 02, 2016 @07:41PM (#11827145)
They also use METRIC TONS of ultra high-end storage.

Actually that's not saying much, because a single cabinet is around a metric ton.

So it was unintentionally accurate.

Re:Apple cloud services outage by nstj on Thursday June 02, 2016 @08:38PM (#11827499)
Apple discusses who it uses for cloud storage when discussing iCloud in their security guide (valid as of May 2016):

> The encrypted chunks of the file are stored, without any user-identifying information, using third-party storage services, such as Amazon S3 and Windows Azure.

[0]: https://www.apple.com/business/docs/iOS_Security_Guide.pdf

Re:Apple cloud services outage by api on Thursday June 02, 2016 @06:10PM (#11826511)
BGP? Tier-1 issues?
Re:Apple cloud services outage by alanh on Thursday June 02, 2016 @06:20PM (#11826581)
And also Azure
Re:Apple cloud services outage by akulbe on Thursday June 02, 2016 @06:15PM (#11826542)
AWS status page is all green.

http://status.aws.amazon.com/

Re:Apple cloud services outage by 0xmohit on Thursday June 02, 2016 @09:34PM (#11827708)
I've heard that the Doppler effect https://en.wikipedia.org/wiki/Doppler_effect is responsible for AWS status page to show up green perennially.

On a serious note, why would AWS admit that it's services were not quite working well and subject itself to potential liabilities.

AWS reacts shamelessly even when your EC2 instance goes down on it's own. For a sample of how they react, see https://news.ycombinator.com/item?id=11822298

Re:Apple cloud services outage by cookiecaper on Thursday June 02, 2016 @06:27PM (#11826622)
Just for the record, our company has found that Amazon occasionally omits incidents from their status page, though I doubt they could get away with omitting something this significant.
Re:Apple cloud services outage by bigiain on Thursday June 02, 2016 @08:18PM (#11827395)
"Occasionally"? <laugh>
Re:Apple cloud services outage by ChristianBundy on Thursday June 02, 2016 @08:47PM (#11827529)
</laugh>

Holy shit, you were laughing for 28 minutes straight. Are you alright?

Re:Apple cloud services outage by inopinatus on Thursday June 02, 2016 @10:01PM (#11827797)
You can tell who grew up in the XHTML era rather than SGML or HTML5.
Re:Apple cloud services outage by bigiain on Friday June 03, 2016 @12:20AM (#11828266)
Cause you know <laugh /> is just a gulp shortcut for <chuckle>&nbsp;</chuckle> (which is fully IE6 compatible)
Re:Apple cloud services outage by ChristianBundy on Monday June 06, 2016 @11:33PM (#11852211)
XHTML 1.0 Strict all day.
Re:Apple cloud services outage by adt2bt on Thursday June 02, 2016 @06:52PM (#11826791)
All of the cloud providers have an incentive to downplay incidents and only post them to the status page when they're really bad. The bar is pretty high to get something on there.
Re:Apple cloud services outage by NeutronBoy on Thursday June 02, 2016 @10:36PM (#11827915)
I mean, they outline them in their fineprint. Amazon doesn't consider the unavailability of 1 EC2 availability zone an outage because their position is that you should be architecting around that. 2 or more AZ outages in the same region and it'll show up on their status page IIRC.
Re:Apple cloud services outage by evanriley on Thursday June 02, 2016 @06:10PM (#11826512)
The only Apple thing that I know was on AWS was the beats1 radio thing when it launched, not sure if it still is.
Re:Apple cloud services outage by ghshephard on Thursday June 02, 2016 @07:47PM (#11827186)
Photos uses AWS for storage.