News for hackers. Stuff that matters.
Re:Stratus: Servers that won’t quit – The 24 year running co... by trothamel on Sunday January 29, 2017 @03:36PM (#13516462)
Note that the space shuttle didn't use processors running in lockstep, like the Tandem machines do (from what I understand). It used 5 single-core computers. Four of them would run the primary software, with a single control channel controlled by each of the four.

For something like the elevons, control was accomplished by connecting 3 actuators to 3 of the channels, with voting being accomplished by physical force - if the 3 computers disagreed, the 2 would overpower the one. (Things like thrusters used electronic voting, close to the thruster itself.)

This seems closer to modern architectures with multiple computers, than the mainframe idea of redundant hardware.

Re:Stratus: Servers that won’t quit – The 24 year running co... by Trombone12 on Sunday January 29, 2017 @06:00PM (#13517309)
Note that all those computers shared the same bus, and so that the complete system wasn't as redundant as hoped as this story[1] retells:

"At 12:12 GMT 13 May 2008, a space shuttle was loading its hypergolic fuel for mission STS-124 when a 3-1 disagreement occurred among its General Purpose Computers (GPC 4 disagreed with the other GPCs). Three seconds later, the split became 2-1-1 (GPC 2 now disagreed with GPC 4 and the other two GPCs). This required that the launch countdown be stopped.

During the subsequent troubleshooting, the remaining two GPCs disagreed (1-1-1-1 split). This was a complete system disagreement. However, none of the GPCs were faulty. The fault was in the FA 2 Multiplexer Demultiplexer. This fault was a crack in a diode. This crack was perpendicular to the normal current flow and completely through the current path. As a crack opened up, it changed the diode into another type of component ... a capacitor.

Because some of the bits in the signal are smaller than they should have been, some of the GPC receivers could not see these bits. The ability to see these bits depends on the sensitivity of the receiver, which is a function of manufacturing variances, temperature, and its power supply voltage.

From the symptoms, it is apparent that the receiver in GPC 4 was the least sensitive and saw the errors before the other three GPC. This caused GPC 4 to disagree with the other three. Then, as the crack in the diode widened, the bits became shorter to the point where GPC 2 could no longer see these bits; which caused it to disagree with the other GPC. At this point, the set of messages that was received correctly by GPC 4 was different from the set of messages that was correctly received by GPC 2 which was different again from the set of messages that was correctly received by GPC 1 and GPC 3. This process continued until GPC 1 and GPC 3 also disagreed with all the other GPC."

[1] Adapted from https://c3.nasa.gov/dashlink/projects/79/wiki/test_stories_s...

Re:Stratus: Servers that won’t quit – The 24 year running co... by hashhar on Monday January 30, 2017 @08:02AM (#13520763)
Is there a website which collects/links to such anecdotes and surfaces interesting ones? I would to read more such stories.
Re:Stratus: Servers that won’t quit – The 24 year running co... by cjp222 on Monday January 30, 2017 @12:41AM (#13518959)
Tandem computers did not run software in lockstep to achieve fault tolerance. They were shared-nothing parallel clusters before shared-nothing parallel clusters were cool, with message-based communication between processes independent of the node each was running on. This gave near-linear scalability as the number of nodes grew. I worked on parallel sorting and parallel data loading back in the 1980s, scaling up to 256 nodes, and Tandem's NonStop SQL developed and commercialized similar parallel database query evaluation in the late 1980s, ahead of its time.
Re:Stratus: Servers that won’t quit – The 24 year running co... by ams6110 on Sunday January 29, 2017 @09:48PM (#13518300)
Also I think the processors ran different, independently developed software, the idea being that it would be unlikely for independent codebases to have the same bugs.
Re:Stratus: Servers that won’t quit – The 24 year running co... by sitharus on Sunday January 29, 2017 @11:41PM (#13518734)
The four main CPUs ran the same Primary Flight Software. There was a separate backup computer running the Backup Flight Software, which was developed by an independant team, to take over if the PFS failed.

There's a bit of debate as to what would have happened if it ever took over.

Re:Stratus: Servers that won’t quit – The 24 year running co... by jackvalentine on Monday January 30, 2017 @02:45AM (#13519326)
I expect they did know, but it'd be an interesting exercise to not tell either development team if they were writing the "real" or the "backup" software, and just swap them around to keep people on their toes.