News for hackers. Stuff that matters.
Re: by cjbconnor on Tuesday November 17, 2026 @01:07PM (#50535589)
Just got back from the Apple Store. They confirmed that everything was down, they could not process orders or returns, and new iPhones that people bought weren't able to be activated.
Re: by itopaloglu83 on Tuesday November 17, 2026 @01:27PM (#50535933)
Honestly I was surprised Apple stayed up during the Microsoft outage as it affected so many interconnected services and infrastructure. I presume that the Cloud providers are shifting some of the workload around and caused other critical services of their customers to fail.
Re: by rickreynoldssf on Tuesday November 17, 2026 @02:06PM (#50536543)
Apple runs their own cloud.
Re: by lrvick on Tuesday November 17, 2026 @02:11PM (#50536614)
A giant cluster of Puppet managd RedHat computers managed like it is 2007, mixed with Google Cloud.

Apple has literally no idea what they are doing with Linux and cloud.

Re: by Symbiote on Tuesday November 17, 2026 @02:20PM (#50536750)
Up to this point, it seems to have been working well.
Re: by mingus88 on Tuesday November 17, 2026 @02:21PM (#50536752)
They also have private compute cloud, not running Linux

They also use AWS and run servers in China using whatever cloud provider the Party uses

Like you’d expect a trillion dollar global tech enterprise to be, they run a ton of stuff geared toward whatever meets their business objectives

Re: by ibejoeb on Tuesday November 17, 2026 @02:25PM (#50536808)
Interesting. This is the first I've heard of broad service interruption at Apple in a while, as I can't even remember the last time. Do you suspect something about puppet or their overall security posture to be implicated in this?
Re: by rdtsc on Tuesday November 17, 2026 @03:01PM (#50537284)
Hell they were more stable than all the other clouds so far. Maybe a giant cluster of redhat computers managed like 2007 is actually pretty robust compared to super kube control plane v42 with 100 add-ons and layers thrown in on top.
Re: by prmoustache on Tuesday November 17, 2026 @03:03PM (#50537324)
I am not sure what is supposed to be wrong with puppet managed linux servers.
Re: by Nevermark on Tuesday November 17, 2026 @07:25PM (#50540252)
Say “puppet linux servers” really fast several times.

Yeah.

Re: by lrvick on Tuesday November 17, 2026 @08:44PM (#50540832)
Production servers should be hardened, immutable, and remotely attestable appliances. Everyone should be able to prove every line of code in memory at all times, with access to secrets gated on signed quorum approval from multiple reviewers. They should not have dev tools, ssh, a shell, and certainly not capable of blindly modifying themselves at runtime if a single admin of a Puppet control server asks them to. Such tactics mean a single compromised engineer laptop means a compromise of production, and thus a data breach is all but inevitable.

If someone even suggesting Salt, Puppet, or Ansible in a sysadmin interview today for production systems with a straight face I would fail them.

Re: by solyin on Tuesday November 17, 2026 @09:54PM (#50541273)
What's the hardened immutable infra you're using these days?
Re: by edf13 on Wednesday November 18, 2026 @02:16AM (#50542892)
Never worked with a legacy system?
Re: by prmoustache on Wednesday November 18, 2026 @02:47AM (#50543069)
I understand and subscribe to some of the things you say but:

> Such tactics mean a single compromised engineer laptop means a compromise of production, and thus a data breach is all but inevitable.

I hope you don't believe that an immutable image cannot be compromised. The commits done to the build system come from the same engineer laptops and, in a world where CVE are published ao frequently, said compromised image could be deployed largely in only a few minutes hours.

Also there is something to be said with lack of debugging tools. While I agree that reducing the surface of attack is the way to go, I am pretty sure the reason it takes so long for some of the big infra outages we have experienced from various cloud vendors comes from the inability of engineers to log on some systems, observe and potentially fix temporarily by hand to stop a race condition. There is often a race condition and serpent eating its own tail situation involved and logs might not even reach the monitoring infra anymore.

Going back to manage pets is not the way to go but I am not entirely certain removing ability for a subset of engineers to log on systems is exactly the way to go.

Re: by aprdm on Tuesday November 17, 2026 @04:53PM (#50538750)
What's the problem with running puppet in 2026 ?
Re: by electric_mayhem on Tuesday November 17, 2026 @10:18PM (#50541439)
You say that like every group in Apple does things the same way.

They don't.

Re: by swiftcoder on Tuesday November 17, 2026 @02:36PM (#50536972)
> Apple runs their own cloud.

That's not the entirely of their cloud exposure, however. A ton of their stuff runs in AWS, I wouldn't be surprised if they had things on other cloud providers as well.

Re: by mixdup on Tuesday November 17, 2026 @03:05PM (#50537353)
The Microsoft outage wasn't an Azure outage anyway, it was specifically tied to Exchange Online. No one running their own stuff in Azure had any issues during that event
Re: by gr33nq on Tuesday November 17, 2026 @04:01PM (#50538126)
Mostly true, but not entirely. Azure Communication Services (ACS) was down alongside Exchange Online and the other Microsoft 365 applications. I verified this by testing upwards of a dozen times throughout the outage window. There was absolutely no indication made by Microsoft though that ACS was impacted in any way.
Re: by blinding-streak on Tuesday November 17, 2026 @02:53PM (#50537187)
Apple is a massive Google Cloud customer for storage (iCloud) and has been since at least 2018.
Re: by etc-hosts on Tuesday November 17, 2026 @05:39PM (#50539290)
Amazon's internal direction on how they scale out AWS and build new features is heavily guided by "what will it do to one of our biggest customers that has a secret code name but everyone knows its Apple".
Re: by icecheese on Tuesday November 17, 2026 @02:07PM (#50536544)
Apple is notoriously good at avoiding these widespread outages, by doing things independently regardless of industry trends. I am pretty sure they don't outsource their hosting to Microsoft.
Re: by redwood on Tuesday November 17, 2026 @02:10PM (#50536602)
A company of their scale uses every cloud to some extent but you're right they are primarily ane AWS and Google Cloud shop
Re: by tsimionescu on Tuesday November 17, 2026 @03:43PM (#50537907)
Even if they had been heavy Azure users, the impact to Azure was basically 0. Only Exchange Online and connected services were down. Even OneDrive, SharePoint, and Teams worked in all the parts where they didn't interval with Exchange (e.g. Teams Calendar could show your pre-scheduled meetings and allow you to join them, just not create new meetings).
Re: by SOLAR_FIELDS on Tuesday November 17, 2026 @08:41PM (#50540806)
Apple does run quite a few of their own data centers as well
Re: by basch on Tuesday November 17, 2026 @02:08PM (#50536564)
Isn’t Patch Tuesday a simpler explanation? Something deep needs to be patched and rebooted. Or a coordinated malware removal effort, where anything less than full reboots cause propagation.
Re: by MiroslavPokorny on Tuesday November 17, 2026 @11:21PM (#50541832)
Lets pretend its patch tuesday.

Do you really think that when these events happen, every single computer is patched at the SAME time ?

Does that sound intelligent ?

Does this happen in other industries ? Does the electric company shutdown every single transformer/station/big machine i dont know the name and perform servicing at the SAME time ?

Of course not, if they do they shouldnt.

Re: by chickensong on Wednesday November 18, 2026 @04:16AM (#50543879)
<CrowdStrike has entered the chat>
Re: by MiroslavPokorny on Wednesday November 18, 2026 @04:47AM (#50544152)
Once isnt the same as always.
Re: by Bluescreenbuddy on Tuesday November 17, 2026 @02:10PM (#50536599)
I doubt apple uses Exchange Online. They have their own mail service
Re: by NichoPaolucci on Tuesday November 17, 2026 @05:08PM (#50538937)
I was AT the Apple store buying a replacement airpod while the exchange outage was happening.

Apple was fine, but this poor old woman was at the genius bar trying to get some accounts connected. The genius at the Apple store was trying to explain the outage - and it just was NOT clicking for the woman. He must have said 4 variations of "Yes your accounts SHOULD connect, but Microsoft is having a major outage right now so they do not."

Re: by lazyasciiart on Tuesday November 17, 2026 @11:22PM (#50541844)
High probability she had no idea what Microsoft had to do with her email.
Re: by N19PEDL2 on Tuesday November 17, 2026 @02:14PM (#50536656)
> new iPhones that people bought weren't able to be activated

Bricked from the very beginning.

Re: by garrettgarcia on Tuesday November 17, 2026 @04:25PM (#50538410)
not yet activated != bricked
Re: by SilentM68 on Tuesday November 17, 2026 @05:38PM (#50539278)
Oh, so that's why I could not login and had to reset my password. Ok, now I know it wasn't me.
Re: by apparent on Tuesday November 17, 2026 @06:00PM (#50539467)
Yeah I was trying to log in and it couldn't send 2FA. I tried all the methods and considered doing a reset, but I realized that the site was loading weirdly and wondered if it was systemic. Glad I waited...hopefully this will be over soon.