My dad spent his career as a natural gas pipeline technician. For a long time they maintained their own radio towers and VHF or UHF (I forget which) radio network. Each district had their own comms guy who maintained the network, and it was very resilient and reliable, allowing the entire company to be in contact essentially anywhere.

In the early 2000s as a cost cutting measure, the company decided to nix the radios for satellite phones on each truck. Of course this proved to be problematic, as the sat phones often had reception issues. They relied on cell phones as backups, which was also quite foolish as the remote compressor stations had very poor reception. They also had issues where someone would leave a voicemail and they wouldn’t get notified of it for days or weeks due to some issue with AT&T.

After a couple emergencies where communication was identified as a big issue, they proposed moving back to the old radio system, but they had already sold off the frequencies and dismantled the infrastructure. My dad retired not long after this, but the “corporate bean counter” trope rang quite true here, and in the long run we were all a little less safe because some executive with no field experience wanted to make a name for themself by saving a little money on something that proved to be mission critical.

That makes me sad to hear, because not only was this company left with an inferior solution, the man hours and effort that went into the original project (including purchasing part of the spectrum!) was essentially nullified.

It's an alarming trend among new grads as well, who see existing systems as bloated and in need of rewrites. Who would've thought the bloat was kind of important?

But then we wouldn't have single threaded wonders like node.js

It's part of our culture that the young kids dismantle the 'old people stuff' setup their new stuff to only re-invent the wheel again and again, run into the same problems that were solved decades ago to only be repeated again.

Remember how TCL was going to save us all from K&R C? Then it was perl to save us from TCL? Then Ruby? Python? javascript, Go or whatever is the flavour of the month?

Sure except you've managed to bitch about most major programming languages here without proposing any alternatives.

The poster's entire point was that all of them came from a desire to find another option when a reasonable alternative already existed.

The logical conclusion is that Ruby shouldn’t exist because shakes fist those darn kids should write all programs in C instead.

I wouldn't argue that bloat is important. I wold argue don't let bean counters make technical decisions.

Unfortunately those bean counters probably got large raises and bonus. Then moved on to their next job proclaiming their victories and got out into higher positions.

And the board let the financial wizard violate the old principle [1] of not taking a fence down until you fully understand the reason why it was put it up in the first place (i.e., don't assume that your predecessors were stupid just because you don't yet understand why they did what they did.)

Really sad results there, throwing away a sound investment & body of know how, for worse results everywhere, and less safety for everyone.

[1] https://www.chesterton.org/taking-a-fence-down/

BNSF railway still maintains its own point-to-point microwave network and VHF/UHF repeaters on mountaintop tower sites, if you pull the public FCC ULS database data for any western US state you'll see their 6 and 11 GHz band licenses.

> After a couple emergencies where communication was identified as a big issue, they proposed moving back to the old radio system, but they had already sold off the frequencies

This is one of those cases where, if it isn't broke dont fix it was a rule to be followed.

Now its hard for the company to fallback to the old system, since they need to buy another set of frequencies, before they can even plan to rebuild the radio tower network.

Since they had stopped with the good running radio network to cut costs (unwise since it ran fantastic) the chance that they will invest in restarting such a network is not really big, unless they get heavy lossses due to their unreliable current system of modern (but worthless) gadgetry

Seems like the solution is to improve cellular reception along the track of the pipeline and stations.

Great write-up, thanks for the link. TLDR: Race condition when switches reboot during high (read normal) traffic load.

I love the fact the "fix" was to reduce traffic load so that the issue was unlikely to occur and let things go back to normal.

> AT&T solved the problem by reducing the messaging load of the CCS7 network. That allowed the switches to rest themselves and the network to stabilize.

Race conditions strike again. It's just hard, whether it's multi threading or distributed systems.

https://en.wikipedia.org/wiki/Therac-25