Re:Worst Computer Bugs in History: Therac-25 (2017)
by mhneu on Saturday August 11, 2018 @05:20PM
(#17741606)
I think unfortunately this is going to change with the advent of "AI" and related technologies, such as autonomous driving
Yes, the potential cost of software bugs is increasing as software does things that no hardware interlock can stop. And worse than that, as a society we largely haven’t realized we need to optimize for worst case (not average) performance of algorithms, because they WILL be attacked. If you’re lucky, they won’t be attacked by sophisticated, well resourced nation-state attackers. But sometimes that will happen.
The rise of complex algorithms to control complex processes is the real difficulty. Facebook’s banning algorithm is an example of something that has been exploited by attackers.
Let’s hope voting software is not the next target where bugs can be exploited. Because changing political decisions can and does produce life-changing effects.
Yes, the potential cost of software bugs is increasing as software does things that no hardware interlock can stop. And worse than that, as a society we largely haven’t realized we need to optimize for worst case (not average) performance of algorithms, because they WILL be attacked. If you’re lucky, they won’t be attacked by sophisticated, well resourced nation-state attackers. But sometimes that will happen.
The rise of complex algorithms to control complex processes is the real difficulty. Facebook’s banning algorithm is an example of something that has been exploited by attackers.
Let’s hope voting software is not the next target where bugs can be exploited. Because changing political decisions can and does produce life-changing effects.