News for hackers. Stuff that matters.
Re:AT&T Fiber in the SF Bay Area is flipping bits by miahi on Tuesday December 08, 2020 @12:38PM (#25348607)
Deleting the queues is an operational decision that I made to be able to put the system back online after the network device was replaced (the important part was the uptime/SLA). From a quick analysis of the logs the percentage of bad messages was ~90% (there was a ~50% chance that the original "touched" bit was 0 so no change was done, but the messages had multiple "touched" bits at fixed intervals).

There was an escape hatch, but the conditions to hit it were a bit complex. Implementing new message filtering of this kind at 2AM while the system was down was not feasible.

There are no comments on this comment yet.