News for hackers. Stuff that matters.
Re:FAA lifted National Ground Stop at 0119Z by dnqthao on Friday November 13, 2026 @09:28PM (#50493059)
I don't understand why they did not test this before deploying on production. How can we sure that when they develop a whole new system, it will work?
Re:FAA lifted National Ground Stop at 0119Z by ChrisMarshallNY on Friday November 13, 2026 @09:36PM (#50493134)
> I don't always test my code; but when I do, I do it in production.

https://static.wingify.com/gcp/uploads/abtasty/test-in-produ...

Re:FAA lifted National Ground Stop at 0119Z by OhMeadhbh on Friday November 13, 2026 @09:37PM (#50493141)

   Because if it doesn't work, they call it out for bids
   again, select their cronies and get kickbacks.  If it does
   work, then they've missed an opportunity for grift.
Re:FAA lifted National Ground Stop at 0119Z by OhMeadhbh on Friday November 13, 2026 @09:50PM (#50493242)

   Lol.  Down-voted by people who work for government
   contractors.
Re:FAA lifted National Ground Stop at 0119Z by Jtsummers on Friday November 13, 2026 @09:58PM (#50493283)
[delayed]
Re:FAA lifted National Ground Stop at 0119Z by OhMeadhbh on Saturday November 14, 2026 @12:21PM (#50498770)

   But apparently  they don't downvote people  for commenting
   on people commenting on downvoting.
Re:FAA lifted National Ground Stop at 0119Z by sippingabonedry on Friday November 13, 2026 @10:08PM (#50493365)
No you are being downvoted for abusing monospaced font for attention.
Re:FAA lifted National Ground Stop at 0119Z by dfc on Friday November 13, 2026 @10:40PM (#50493584)
They use monospace a lot. I wish there was a way to flag someone for it.
Re:FAA lifted National Ground Stop at 0119Z by mitxela on Saturday November 14, 2026 @06:41AM (#50496207)
there is, it's a button called "flag" in little gray text above the comment if you are on the comment's page.

If you meant to report the user, you have to email [email protected]. They don't bite but they don't always do the thing you wanted.

Re:FAA lifted National Ground Stop at 0119Z by OhMeadhbh on Saturday November 14, 2026 @12:20PM (#50498748)

   What about flagging people for using proportional fonts for
   attention?  Seems vaguely arbitrary.
Re:FAA lifted National Ground Stop at 0119Z by dfxm12 on Friday November 13, 2026 @09:40PM (#50493163)
This admin can't be trusted with lining a pool. Why is their incompetence still a surprise to you after all this time?
Re:FAA lifted National Ground Stop at 0119Z by dboreham on Friday November 13, 2026 @09:53PM (#50493257)
Let's set that aside and instead ask why tf they rolled out anything on a Friday? Or was it that they rolled it out on Tuesday but failed to notice it was close to overload state until it actually overloaded on Friday?
Re:FAA lifted National Ground Stop at 0119Z by kasey_junk on Friday November 13, 2026 @10:14PM (#50493403)
Or it was Friday traffic that overloaded it.
Re:FAA lifted National Ground Stop at 0119Z by golbanno on Friday November 13, 2026 @09:55PM (#50493264)
Sounds like the new system effectively performed a DoS on the old system. They probably don't have a development environment where they can hook the two systems up. Even if they do, it might actually have been an out-of-the-way component that failed first, impacting another, and so on. If they have a test environment with the two systems, a complex failure like that would unlikely to be found (and would explain why it took hours to resolve).
Re:FAA lifted National Ground Stop at 0119Z by pknomad on Friday November 13, 2026 @09:58PM (#50493292)
As someone who works under a different umbrella in the federal government, how do you propose testing something so archaic and complex without replicating an exact replica of a 40+ YO technology?
Re:FAA lifted National Ground Stop at 0119Z by AnneTrotter on Friday November 13, 2026 @10:00PM (#50493315)
I've actually worked a system where the test env was scavenged old eq set up to mirror exactly the ancient prod env. It's possible, it just takes a lot of work. Scaling it might be touchy though.
Re:FAA lifted National Ground Stop at 0119Z by pknomad on Friday November 13, 2026 @10:04PM (#50493337)
> It's possible, it just takes a lot of work

Not only it takes lot of work, but to have the replica connect to the prod in some way to mirror identical behavior will possibly mean adjusting ATO, which is not a simple process in itself.

Re:FAA lifted National Ground Stop at 0119Z by dwattttt on Friday November 13, 2026 @10:30PM (#50493528)
And it sounds like the fault conditions only occur when you do scale up.
Re:FAA lifted National Ground Stop at 0119Z by madaxe_again on Sunday November 15, 2026 @05:02AM (#50505592)
That’s obviously not true.
Re:FAA lifted National Ground Stop at 0119Z by nik282000 on Friday November 13, 2026 @10:03PM (#50493330)
Emulate, computing is cheap. In an application as important as air traffic control it seems like it wold be worth it to _always_ have a working copy of the original system specifically for testing changes.
Re:FAA lifted National Ground Stop at 0119Z by kasey_junk on Friday November 13, 2026 @10:14PM (#50493399)
You think an error that brings down _national_ networks is something you can emulate?
Re:FAA lifted National Ground Stop at 0119Z by pknomad on Friday November 13, 2026 @10:18PM (#50493430)
Thank you, you get it.
Re:FAA lifted National Ground Stop at 0119Z by cgio on Friday November 13, 2026 @10:32PM (#50493543)
Not aware of specific system, but critical infrastructure systems have a quality assurance system that replicates production albeit not perfectly. You may replicate the core system but not all its interfaces, you might replicate it but scaled down etc. There are ways for solid tests to fail. For systems with downtime, a test on production is useful iif you have solid rollback procedures. There’s lots of art in the operation of these systems.
Re:FAA lifted National Ground Stop at 0119Z by pwndByDeath on Friday November 13, 2026 @10:08PM (#50493366)
Its possible, just not probable with who you can hire for gov work.
Re:FAA lifted National Ground Stop at 0119Z by dnqthao on Friday November 13, 2026 @10:56PM (#50493724)
If you cannot replicate everything then maybe have a very conservative rollout approach so that if it fails, it only affect a small subset of the traffic.
Re:FAA lifted National Ground Stop at 0119Z by ncr100 on Saturday November 14, 2026 @07:20AM (#50496460)
That's sounding like a good idea, simulate using a RTOS.
Re:FAA lifted National Ground Stop at 0119Z by indstinctdialog on Friday November 13, 2026 @10:32PM (#50493540)
I’m sure it worked on his machine.
Re:FAA lifted National Ground Stop at 0119Z by jmyeet on Friday November 13, 2026 @11:02PM (#50493780)
The Secretary of Transportation's claim to fame is having been on MTV's the Real World and being a sycophant. Is it really that shocking? Really? We live in the dumbest timeline.

I'll be interested to see what the root cause of this all is, assuming it doesn't get buried. What I find both perplexing and predictable is how the finger has been pointed at Congress for not funding a replacement. Well, who's controlled Congress for the last 2 years? Who was president in 2017-2021? Who controlled Congress in 2017 and 2018?

If you want to dig deeper, this administration has also (unsuccessfully thus far) claimed in court that they're not bound by Congress when it comes to spending under the Impoundment Control Act.