In Maintenance, sometimes the dumb solution is the best. The servers that ran amazon.com had a memory leak and were subsequently restarted. This hack continues for years because it was the most cost-effective thing to do.

I suppose you might want to first rule out a break-in, and if there is one, recover evidence that won't survive a reboot.

Yeah, a place I worked at years ago had a cron job to restart some critical service at midnight because it leaked memory all day. That was there for years.

Another real world example: to this day, it's a good idea to limit requests per PHP process using the MaxRequestsPerChild(apache)/max_requests(php-fpm) config option to some sane limit. After that configured number of requests, the PHP worker is terminated and restarted to... "fix" memory leaks. I remember setting that in the late 90s on Apache...

I am happy to no longer use PHP for anything.

Oh, it's been a while but I remember that.

I've used PHP in two previous jobs and certainly the stuff that I was in control of had it set to 0 because we made sure we didn't leak memory. Other departments... I'm sure at least one had the limit set to something non-zero.

We had a mail server that ran on an old DEC minicomputer. I rebooted it every six months because it had a slow memory, or possibly resource, leak somewhere.

It was the only thing running on that box so it wasn't a big deal to reboot.

I worked on multi-million dollar airplanes. If a pilot called me with an issue and hadn't rebooted the plane (on the ground), I wouldn't even talk to them. Rebooting isn't always a bad solution to technology.

It's not a bad solution, it definitely should be tried. But I'm still rather shocked to see it in official notes.

It's not a solution at all. It's a symptom.

It’s not a symptom, it’s a work around until a more permanent fix can be deployed.

> it’s a work around until a more permanent fix can be deployed.

If it "fixes" your problem, then you didn't fix you problem you identified it. It's another symptom of a bloated industry that can't or won't innovate, and I think it's more "won't" than "can't."

For example, we still can't patch running Windows, and most of the information technology professionals I know are still convinced it's impossible. Billions upon billions of dollars in time have been lost, probably enough man hours to send men to mars several times over.

Yet, you can just buy a version of Windows Server that already does it. It's not only possible, they already have the code. They just don't care enough to integrated it into the product most people use.

(on the ground) is probably pretty key, though. Ha ha. I can imagine the conversation if this was in the air.

Rebooting is also the only solution for some problems.

looks at the Boeing 787

[delayed]

Thales or Collins or Honeywell or whoever provided the system would. The technician at the airport would only follow a procedure listed on a checklist, which probably includes rebooting the system.

Ironically there have been a fair few cases of aircraft incidents where components were reported as bad by pilots but the issue couldn't be reproduced on the ground and "it worked" after a reboot and self-test so they just kept using it (with this cycle repeating multiple times). Often there were clear physical problems (cracked solder joints, stuck microswitches, broken insulation) that were only discovered after a catastrophic failure but could've been discovered with a more inquisitive engineering culture.

I would argue a "just reboot it" culture leads to more problems -- engineers should be empowered to diagnose problems before they become catastrophic. In the world of software we even have mechanisms for extracting data from failed systems to root cause them later, and yet most people do not take that step before "just reboot[ing] it".

[1]: AirAsia 8501, Lauda Air 004, and SmartLynx Estonia 9001 come to mind.

Right, rebooting only really makes sense if you understand why the reboot is going to fix the problem. As in, corrupted volatile state is reasonably fixed with a reboot. Any hardware issues or persistent state problems could well get worse.

The following is a few decades old --

A physicist, an engineer, and a computer programmer are driving down a steep mountain pass when the car’s brakes fail.

The car speeds out of control around sharp turns, nearly flying off the edge before they miraculously manage to skid to a stop at the bottom.

Shaken up, they discuss what to do next:

• The physicist says: "We need to calculate the friction coefficients in the brake pads and model the temperature rise to figure out why they failed."

• The engineer says: "I have a few tools in the trunk. I’ll take a look underneath and see if I can fix the mechanical failure."

• The programmer says: "Why don't we get back in and see if it's reproducible?"

But these intermittent failures lead to delays and worse. The sheer cost of repeat events leads to action. If deferrable at all the standard troubleshooting is mostly remove and replace. The repair and reliability processes identify these rogue units fairly fast and the actual rates of that in a well functioning airline are very low.

Yes, I remember a flight that sat at the gate past departure time, the explanation to the passengers was that some essential piece of equipment wasn't functioning, they had tried restarting it, and it was going to be replaced. About 30 minutes late, we were on our way.

The vast majority of the problems were a system that missed the time window to report it's self test. A reboot got things in order.

It was a small fleet, so we had a pretty good idea if the same problem kept popping up so we could look more into that system.

A few fault codes were well known enough that if you called with that one the answer was to shut down and find a hotel, you aren't going anywhere and no reboot will take care of it. (bad detectors in the engines for about 10 years until a redesign)

Yeah. It works, almost always: https://en.wikipedia.org/wiki/Indonesia_AirAsia_Flight_8501

I was thinking of that one as an example of why we don't do it in the air...

What makes sense for something that is used for 20 hours at a time is different from what makes sense for a service that literally has "365" in its name.

In rail, we call it BOBO - battery off, battery on.

In German, we say "AEG-Effekt", ausschalten einschalten geht wieder.

Amateur hour. Everyone knows professionals say "power cycling" instead to make it sound complicated.

the layman understands both of those words and their meaning together.

we "bounce" the servers when on calls with customers.

Powercycling is a designed method to bailout Cisco routers. If you botch a configuration command and lose connection, you can have someone powercycle it to restore you last good configuration. Its not just a joke it really works more often than I'd like to admit.

~25 years ago I was a firewall administrator, and the Cisco PIX we used was notorious for gradually running out of memory, effectively cutting off the entire company from the Internet when it went down (and it was a very big company).

We absolutely should have scheduled reboots during off hours. Lessons learned.