News for hackers. Stuff that matters.
Re:Real System Failures by deathanatos on Thursday July 13, 2017 @08:59PM (#14766658)
On slide 10, faulty hardware introduces a standing wave onto a bus. Two CPUs are nodes, two at antinodes, causing a 2-2 disagreement in the state of the system.

Yet, the slide goes on to argue this is a software problem? It was my impression that Byzantine tolerant systems required agreement among ⅔ of the nodes; if the system is split 50/50, how can even a tolerant system not fail? (Or rather, is it the difference between failing gracefully and failing spectacularly, and the slide fails to elaborate on exactly how the system failed? But I don't see how we can expect this to succeed.)

Re:Real System Failures by gizmo686 on Friday July 14, 2017 @12:55AM (#14767577)
I am not sure if it is refering to the NASA incident you are refering to, but a situation consistent with said failure is described here [0] (section 5.3), although it seems like it still assumes some background I do not have to fully understand it.

In the situtation described in my link, there were 4 data sources, and 4 processors. A byzantine fault occurred with the link from one of the data sources to the processors. This cascaded into a failure of the entire 4 processor system when it fell into a 2:2 split. In concept, the processors could have communicated with each other and detected a disagreement in one of the four data sources.

The software bug is that a fault with one of the data sources cascaded into a fault with the entire processing unit. In a correctly functioning system, the fault should have been contained to just the one data source. That is, the problem should have been no worse then simply losing the bus entirely.

Assuming there was sufficient redundancy across the buses, the system could have continued functioning properly despite the fault. However, because the software did not correctly handle the fault, any other redundancy became useless.

[0] https://www.cs.indiana.edu/classes/p545-sjoh/post/lec/fault-...

Re:Real System Failures by jfoutz on Thursday July 13, 2017 @11:21PM (#14767183)
I kinda think thats the point. Not exactly a software problem, but an information theory problem. If you need to tolerate 2 failures, you need 6 machines.
Re:Real System Failures by Normal_gaussian on Thursday July 13, 2017 @11:38PM (#14767264)
5 machines. Even numbers of machines introduce a higher failure likelihood for no greater tolerance.
Re:Real System Failures by jfoutz on Thursday July 13, 2017 @11:51PM (#14767330)
You're thinking majority not Byzantine fault tolerance. 2/3 have to agree.
Re:Real System Failures by nickpsecurity on Friday July 14, 2017 @08:25AM (#14769206)
Many of us in high-assurance systems have witnessed triple, modular redundancy fail. I started saying 3 out of 5 for that reason. May be same for other commenter. I also want where possible for the computers to be in different locations, using different hardware, and with different developers working against same API.
Re:Real System Failures by vitus on Friday July 14, 2017 @08:40AM (#14769292)
Yes, but those aren't Byzantine failures. Byzantine failures present with incorrect values, not simply the absence of a value (as seen by a total hardware failure).

See the abstract in Lamport's original paper introducing the Byzantine generals problem [0].

We also see a similar issue in error correction -- an introductory undergrad course might teach this via Lagrange interpolation [1], where you need only n+k of the coefficients in the presence of erasure errors, but n+2k in the general case (where n is the size of the actual message, and k is the maximum number of errors to correct).

[0] https://www.microsoft.com/en-us/research/wp-content/uploads/...

[1] https://inst.eecs.berkeley.edu/~cs70/fa14/notes/n8.pdf

Re:Real System Failures by nickpsecurity on Friday July 14, 2017 @08:57AM (#14769413)
Partial hardware failures exist. The wrong values start going through the system. A bit flip is easiest example. NonStop has been countering both partial and total HW failures in their design a long time now.