No idea if that's what's going on, but routing protocols are one of a few effectively global control planes that can go wrong very quickly like this.

Traceroute from "the internet" back to that IP reaches AS14593 just fine, and my endpoint doesn't get beyond the first hop of the local starlink router.

Whatever it is, it doesn't look like a peering problem

One of the promises of starlink was it would stay in space as long as possible before being downlinked, giving far lower latency, alas that hasn't happened yet, and traffic will run thousands of miles in the wrong direction before being downlinked. For example from one location to another I have 360ms via Starlink but just 200ms rtt via local provision (5g p2p wireless then optical). On another it used to downlink in Lagos, but now it downlinks in Nairobi, meaning traffic to Lagos routes Nairobi -> Marseille -> Lagos, taking far longer than it used to. A shame really.

But hey, if you haven’t caused an incident yet, that just means you’re still in onboarding. Those SLA downtime budgets are there to be spent.

It says it didn’t, and it says the “which way is down” thing hasn’t converged. Occasionally, the signal to noise ratio light in the app goes gray which means < 3.

It also rebooted itself.

Before the first reboot, 30% of pings went through. It’s almost like the azimuth or some other timely but cached data was corrupted.

Whoever is doing immediate global deployments and/or any prod deployments without verified testing is just wrong as a corporate culture.

It's not like you can have a tech go out and reboot it or reinstall windows.

Here's a great talk on roughly this topic: https://media.ccc.de/v/38c3-hacking-yourself-a-satellite-rec...

This is hybrid warfare, not a coincidence.

Russia needs to learn some manners, the hard way.

Does that count as "major" now?

Emergency service lines go down somewhere in the world almost every day. Often for longer than that.

HN didn't used to be a place for people who like to gin up conspiracies, and engage in speculative hyperbole.

Other social media networks may tolerate this but this is not HN worthy.

The parent's concern is with the "could" in your sentence.

Unlike other web sites, the people of HN used to pride themselves on presenting facts, not speculation. Rumors were left for places like Facebook and now Reddit.

  Starlink VP of engineering Michael Nicolls tweeted that the service has "now mostly recovered from the network outage" after two and a half hours.

  "The outage was due to failure of key internal software services that operate the core network. We apologize for the temporary disruption in our service; we are deeply committed to providing a highly reliable network, and will fully root cause this issue and ensure it does not occur again," the tweet, posted at 3:23 p.m. PT, says.

Edit: funny, can't even access the replies..

And of course, they have to downvote to hell. They need to keep the illusion that the US is still the most powerful military, the most powerful economy, and that AI will save their hegemony.

Really sad.

What is the illusion is that a powerful military is enough to win a conflict.

If anyone has benefited humanity over the last couple of decades, it's Elon Musk! I'd argue he's done more than anyone!

For most large businesses, 90%+ of major network events are caused by internally-driven network config changes.

Depending on how far along the business is towards automation, a proportion of the 90% can be attributed to a human going “off script” - I.e.: making a change that had not been reviewed.

Hope we haven't kicked off the world of AI induced software outages

[1] https://www.pcmag.com/news/spacexs-starlink-hit-by-major-out...

But thanks for the link anyway.

Additional points for a user to cause the loop by plugging an rj-45 where they weren't supposed to, despite no warning or label telling them not to!

I love when devs talk about networking.

Heh.

National telcos manage to take themselves out countrywide from time to time too, so I'd assume the same usual suspects apply: Routing problems, botched global configuration changes etc.

If anything, due to centralization, ubiquitous and effectively free connectivity, and centralization/automation of configuration changes, the blast radius of any given error or malicious action has probably been steadily increasing over the years.

You would expect to be able to at least connect to a satellite unless the satellite has no Internet connection so it does not make itself available to be connected to you.

Anyone have any deeper knowledge about this for the curious?