911 emergency services go down across the US after CenturyLink outage

Re:911 emergency services go down across the US after Centur... by mekaj on Friday December 28, 2018 @12:50PM (#18778229)

There was an incident with similar consequences on April 10, 2014. The cause was a programmed threshold being breached and the impact was 6h of downtime.

Source: "The Coming Software Apocalypse" published by The Atlantic (https://www.theatlantic.com/technology/archive/2017/09/savin...)

walrus01 also linked to https://www.fcc.gov/document/april-2014-multistate-911-outag... in another comment.

[ Parent | on HN ]

Re:911 emergency services go down across the US after Centur... by PhasmaFelis on Friday December 28, 2018 @04:05PM (#18779407)

> Operated by a systems provider named Intrado, the server kept a running counter of how many calls it had routed to 911 dispatchers around the country. Intrado programmers had set a threshold for how high the counter could go. They picked a number in the millions.

I'm really curious if there's some explanation that makes this sound less catastrophically stupid, particularly the part where they picked a threshold less than INT_MAX.

[ Parent | on HN ]

Re:911 emergency services go down across the US after Centur... by swiftcoder on Friday December 28, 2018 @04:44PM (#18779703)

What do you mean, VARCHAR(8) is a perfectly valid way of storing counters...

[ Parent | on HN ]

Re:911 emergency services go down across the US after Centur... by ineedasername on Friday December 28, 2018 @05:56PM (#18780161)

much better to future proof it by using a BLOB. You know, just in case you ever need to shove something else in there.

[ Parent | on HN ]

Re:911 emergency services go down across the US after Centur... by AdamJacobMuller on Friday December 28, 2018 @06:15PM (#18780268)

if you're going to use a BLOB how can you know the type and how to decode it? Make sure to serialize that data with something like protobuf.

[ Parent | on HN ]

Re:911 emergency services go down across the US after Centur... by oneplane on Friday December 28, 2018 @07:06PM (#18780530)

Nah, describe it using some custom binary format and have a 1000 page spec to document it!

[ Parent | on HN ]

Re:911 emergency services go down across the US after Centur... by justinclift on Friday December 28, 2018 @08:22PM (#18781024)

Yay, seralised bluetooth! ;)

[ Parent | on HN ]

Re:911 emergency services go down across the US after Centur... by dharmab on Friday December 28, 2018 @05:28PM (#18779994)

Allocating pieces of a resource without accounting for reusing released resources?

[ Parent | on HN ]

Re:911 emergency services go down across the US after Centur... by MacroChip on Saturday December 29, 2018 @01:06PM (#18784835)

Great question. Maybe they did pick a sensible constant, but it still wasn't enough. I.e. INT_MAX wasn't big enough.

[ Parent | on HN ]