The problem is that companies don’t want to invest the time in good security because it’s less profitable. Or, they are old and decrepit. If the company runs critical infrastructure, then peoples’ safety is at risk. If they don’t then surely a new fad can take their place. Why should we tolerate small fuckups and negligence that only serves a company's shareholders’ interests?
I do know places that don’t allow random individuals to tail you into a building and have cameras on entryways so you can’t sit there and attempt to pick the lock endlessly. And they have good locks. Just because it’s harder to “see” digital systems doesn't make them any less relevant to secure. People in software more-so on average just don't understand how the internet works so they don’t know where to put the cameras and how to identify the shady individual trying to tail someone through an entryway that requires authorization.
This problem is fixable but not when the prime directive is “ship this experimental product as fast as you possibly can or I’ll hire an offshore contractor”. And not when your extent of experience with operating software is running a cute web server on your dev laptop. People are actually doing really dumb shit. Don’t pretend that everyone is writing good software and following best practices and they just happen to get hacked by a script kiddie “oh no”.
These are sophisticated actors targeting mature but vulnerable companies that have chosen to selfishly forgo healthy security practices. Again, why do we want these machines participating in society? Why don’t we want them replaced with more secure versions?
I do agree with the rest of the post but this is definitely NOT true. Your house probably has small issues here and there if you are willing to pay for an inspector. Doctors fucking up things is not news too, plus you never know if there is "one small fuckup" as long as you don't feel particularly bad afterwards.
The thing is I think you guys regard real world engineering too high. Small fuck ups happen all day.
Youtube channel "MentourPilot" [0] has some excellent examples of how civil aviation was built to withstand "small fuckups": with "Swiss cheese" model. [1]
That’s my point. We would not tolerate companies that build bridges that collapsed all the time and keep letting them build bridges so why should we tolerate companies that get hacked repetitively by ransomeware groups? Insurers are running because it’s not good business to insure these risks. Sounds like the market is working just fine.
"Small fuckups" which result in catastrophic failure, property damage, injury, or death, are not.
For the latter, you'll find all manner of standards, codes, practices, maintenance, and inspections.
What we're discussing with regards to infosec aren't "small fuckups", they're repeat major catastrophies, bearing little or no risk to the firms and organisations perpetrating them.
A large component of the risk resides simply in collecting and retaining data, as well as exchanging it with other organisations. That turns out to be a very high-risk activity, though the risks are externalised. The point is to internalise those costs, and to sharply reduce the incentives for the practices which present those risks. This includes incentivising the IT products and services sector toward architectures which can be more adequately protected.
Pilots are allowed small fuckups all the time when flying planes. I've got around 1500 hours and haven't had a perfect flight yet. When I stop flying, I expect to have 3000 hours and still to not have had a perfect flight.
Aviation is designed around having redundancies, tolerances, and contingencies for these small fuckups.
Are there lessons for software engineering to be learned from aviation?
- Aviation incidents are investigated and the investigation is typically bent on discovering root causes, not assigning blame. Do the same thing with bugs & security breaches, you’ll learn more.
- Pilots spend time in simulators practicing unusual situations like water landings or engine failure—do the same thing in software, simulate failure scenarios and intrusion events and see how your engineers and technicians handle those events. You develop new training materials and redesign systems based on what you learned in simulations.
- Pilots rely heavily on checklists rather than memory to ensure that all steps have been completed. This translates well to QA, deployment, various tasks people you need to do in production like migrations or whatnot.
- Pilots require good UX to do their job and cannot be overloaded with too many tasks. Sometime an incident will be traced to dangerous things that don’t seem dangerous when you look at the UI (like thrust reversers), or traced to systems that assigned too many duties to pilots (who get overloaded). For example, if you have a button that deletes a database, it should require a serious confirmation step, like a molly guard on a plane.
- Tasks must be explicitly delegated to individual people (which is what happens on an airplane). For example, in the Gimli Glider incident, the flight engineer role was eliminated and the engineer's responsibilities were not explicitly delegated to the pilot or copilot, but assigned to “both”. Same should be true for tasks in software engineering and incident response.
We also talk about how medicine seems uninterested in learning these lessons, for various reasons.
“Problem Management is exercising emergency authority here” clears away any ambiguity and the confidence this gives them means they are free to act to resolve an incident rather than wonder who they need to call to give the blessing.
https://www.law.cornell.edu/cfr/text/14/91.3
When onboarding new members to that group, we go over this, including the origin story as part of helping them understand how (and that) we expect them to use this.
In other areas? Probably. Aviation tries not to bet lives on a single piece of equipment working perfectly. Where that’s unavoidable (like the Jesus nut and pin on a helicopter), inspection and maintenance procedures are more stringent.
For security, I don't think there are any. Aviation isn't great on security anyway.
In the cyber security world such machines absolutely do exist so we should be expected to design around them.
Most of the time, it's that a non-technical decision maker decides to take a badly mis-calculated risk. "Bob in sales can't access the files he needs." "IT give him admin rights! Now! no... I don't even understand or care to understand that access control mumbo-jumbo you are talking about." A week later, "Hey, you know we just got bitlockered. Bob's laptop got virused, and because we gave him admin rights his machine encrypted all the things AND the backup server too."
That's wrong.
Engineers aren't allowed to fail. They can have as many "fuckups" along the way, as long as they manage to compensate for it and maintain safety it's not a failure. Doctors... well, this profession has a group mentality of shielding one another from publicly acknowledging and documenting mistakes.
Truth is ransomware it utterly useless against a simple back-up and recovery scenario. Any competent engineers can and will deliver one if paid to. It's that companies don't see the point and don't care. As long as the insurance premium is less than what it will cost to implement such a thing they will keep getting ransomed.
It absolutely can, it is called Zero Trust, as has been a cyber security model for many many years now
The problem is most organizations have their entire business workflow around the Ring Fence model, and organizations is extremely prone to "we have always done it that way" when it comes to business processes so they expect IT to buy some software, or some widget to bolt on to the ring fence that will protect them
Businesses need to completely rework their internal methodologies, and processes which is something most companies refuse to do
Moreover, engineers also design for resilience to human "attackers" in physical systems all the time.
If anything, cybersecurity is easier because you can leverage the work of tens of thousands of other people in your tooling. You wanna re-write ssh or fail2ban yourself?
Yes they are. You should talk to an actual doctor before making such claims.
this is the asymmetric nature of the threat. The governments of the west need to align and go after the offenders or sanction countries that give them refuge. It's impossible for a single company to defend itself against adversaries that can keep trying over and over again with impunity.
I don't blame you if you don't take a strangers word on this, but I would strongly encourage you to look at this YouTube channel VASAviation (https://www.youtube.com/user/victor981994) or watch the excellent movie, "Charlie Victor Romeo", which cover in extreme detail the routine, regular, and on-going mistakes that are made in aviation. Mistakes happen in aviation ALL THE TIME; they just don't cause death or crashes in every event.
I can't speak for Engineers and Doctors, but for: pilots, ground crews, flight mechanics, and ATC I have a tiny soapbox. Each of these jobs routinely on a daily basis have mistakes that are made. Sometimes the errors are minor. Other times they are major, and aren't discovered for months or years after the fact, which either led to the loss of millions of dollars worth of equipment, or worse.
Sure, CEOs and such are usually held responsible for those mistakes, but the aviation community focuses more on the ideals of, "Every rule, written in blood" mentality. Things are permitted, until they aren't. This is precisely because it caused a disaster. Things are safe(r) now because they were significantly less safe just a few years ago. The aviation community is focused on preventative actions as a result of the ongoing looming threat of failures. There is no way to avoid them entirely. There is an understanding that when you are in the air, there is NEVER certainty. You are NEVER safe. You should constantly be concerned about what can (and will) eventually go wrong. It's only a matter of how bad the failure will be.
TLDR
Aviation isn't safe because they "don't allow mistakes" but because they have a rigid and foundational system around how to deal with failure.
What a beautiful dream.
Yes they are. When estimating the forces you need to handle you can round up. This means that if you made a small mistake it like won't matter since you planned to handle much more than necessary.
>in the OR or when calculating dosages
Dosages don't need to me exact. There are some tolerances and it's easy to imagine a system to eliminate most of human error from this process.
>Pilots aren’t allowed small fuckups when flying planes.
Sure they are. A mistake doesn't mean your plane will crash or fall out of the sky. Planes can already fly themselves.
>Why should we tolerate small fuckups and negligence that only serves a company's shareholders’ interests?
Because hindsight is 20/20.
But engineers aren’t held responsible if a malicious actor drivers a bulldozer into one of the bridge’s columns and brings it down. Same with a doctor. If someone is maliciously messing with the meds, then they are not responsible for over or under dosing.
With these security incidences, we are dealing with malicious actors trying to being down the system. I don’t think you can use analogies from civil engineering or medicine for that.
These are really poor analogies. A better analogy would be engineers who then had to protect those bridges from attack via a number of different modes, including drone and cruise missle strikes; and seemingly legitimate travelers on the bridge who are actually malicious.
Likewise, a better analogy would be doctors who had to contend with nurses who are bad actors, substituting saline for needed drugs or deliberately administering higher than prescribed dosages, etc.
That's BS and you know it. Any system that relies on humans being perfect fails rather quickly.
While major incompetence is generally punished, all of those systems you describe have ways built in to deal with routine human failures.
It s a risk assessment, as we pile crap on top of crap in deeper and deeper systems, security holes grow more numerous to a point it will be profitable to cover them better than the competition.
I feel like you aren’t really responding to the claim that having good security is extremely hard and expensive and that it seems bad for every company to have to spend a lot of money on a problem which is likely far from their core competency.
The thing you’re actually asking for is for a lot of businesses to go bankrupt trying to be perfect at something that is not their traditional domain and then you make some claim that they have the audacity to try to create value for their shareholders instead? Isn’t it outrageous that companies might try to do literally the thing companies are meant to do instead of being experts in computer security.
I suppose you would counter that it is easy or cheap or imperative for them to be secure but, in the world where hacking is relatively easy, cheap, and profitable ($1-10mm sounds like a lot to a person but not to a country or reasonably sized business) and there are myriad ways to make trivial-seeming configuration errors in software that leave you open to attack, I think it is neither easy nor cheap. And frankly I think it is not imperative for businesses either because their purpose is to make money and while cybersecurity is an important risk to worry about, they can only invest in it so long as they are making money from the thing they are actually meant to be good at.
[1] regarding the first graf:
1. Mostly small errors by engineers or doctors or pilots are… small errors. Bridges won’t collapse if the concrete was mixed slightly too wet, and bigger cracks are usually noticed and mitigated in good time. Doctors get dosages wrong all the time (which is why there are attempts to train nurses to question everything the doctors say) and while malpractice is definitely a thing, there are lots of cases where things go wrong at a small scale and are either fixed or unnoticed. If you ever ask a doctor about this in a social setting they will probably have plenty of examples. And if you get something wrong in a plane you are unlikely to immediately crash. Even landings can be aborted quite late. Big planes crash very infrequently and I think it is silly to therefore assume that the pilots are generally infallible. So I claim that your direct statement that these are not allowed is false.
2. I also claim the analogy is bad because the examples you give are not adversarial. The patients are generally not trying to trick their doctor into making the wrong prescription and Athena geography isn’t going to shift and change to try to exploit weaknesses in the bridge built across it. The pilot flies high above the ground with separation from other planes and various safety systems. Even in an unlikely event like an engine stall, there are possibilities to recover. In computer security the threats are fast and numerous so any small error can be quickly magnified.
No it isn't, it's super cheap and dead simple. What's expensive and difficult is fixing security problems super late in the game. Undoing bad security is hard. Doing it right is really simple.
For example, we have 2FA everywhere at my company. It's 10 people, so that's easy, and it always will be. If we were 2,000 people I'd have to go through hoops and it'd be a whole mess to roll out 2FA everywhere. By using U2F 2FA from day one we've more or less eliminated credential theft as a threat. By disabling app execution we've eliminated malware as a threat.
Right off the bat the vast majority of attacks just don't work, and those were trivial to implement. We do way more than that, and it was all dead simple.
> One small fuckup is sufficient to have the whole thing compromised
It definitely shouldn't be.
Companies flat out do not care, in part because there aren't consequences for not caring. Otherwise they'd do something about it. Even if you're on an older network where you've got AD and garbage like that you can do a lot to improve things.
Lots of companies were founded prior to the popularisation of 2FA, and these security standards change over time.
Making new applications secure by modern-day standards might be relatively simple - although you are still exposed to security risks if one of your vendors has a vulnerability (you don't often get to see the codebase of your vendors, so a zero day can hit hard).
Then keeping a legacy infrastructure, older codebase or historical on-premise applications (where the vendor may not even exist anymore) secure is more difficult. And all those solutions were 'secure' by the standards of when they were implemented, just times have changed.
And then on top of that, we are talking about Ransomware hackers which are buying zero-days for host operating systems - and at that point all bets are off. Let's not forget, you just needed 1 machine of the 2,000 to be 2 months out of date on patches and you were susceptible to WannaCry.
Like I said, what's hard is undoing bad security. Still, I know companies that are nearly a century old that have rolled out strong 2FA policies across 10's of thousands of workers across the globe. They do it because they value security.
It's much harder to fix a bad network, but it's still just a matter of effort. It's not like we don't know how to do it, it's a matter of effort.
I'm not an security guy, but occasionally, I get asked by a friend that has a big IT problem for a second opinion, and that has happened with ransomware six times in thee last three years. From what I've seen, ransomware attacks are the result of a conspiracy of neglect and a couple global IT mistakes:
* Backup/restore doesn't work at all, which is really the story on every ransomware attack I've seen. Testing and verifying operation was neglected.
* Services are not partitioned in a way that isolates them from attack. For example, backups are stored on a writable network share leading to the backup being encrypted by the ransomware. This is often because getting access control on a service right takes time, and does make it harder for users when they are doing something out of the ordinary.
* User access is read write on everything and uses shared credentials (for example, using the same database user account, with admin rights on all services that touch a database server). This makes it easy for the IT guy, and makes it very easy for the ransomware to spread through the network and take down stuff it shouldn't be able to touch: like the tables for your billing system.
* There's a false belief that you can isolate an insecure network for the secure one. This may work (in theory) in a data center where there is ironclad control of what is connected to the network, but in an office environment, this is impossible. Companies who run their internal network like an ISP where nothing is trusted do a lot better.
* Out of date server software has massive vulnerabilities and hasn't been updated in years and years. This is often because someone customized something and did not maintain it, and eventually, it became impossible to upgrade. So, the admin leaned in on the belief that they could secure the system at the network level.
Insurance companies will have to insist on a few practices if they want to actually be able to sustainably insure against ransomware:
* Working backup / restore * No shared admin credentials * Zero trust on the LAN
I'm curious how much the "locked door" idea comes into play here. Locking your doors doesn't keep out thieves. Rather, it makes you less of a target because you're more difficult than others that didn't lock their doors. Is it reasonable to assume that, the more you protect your system, the less likely you are to be attacked... even if you don't do a perfect job.
I would guess the answer is a definitive yes.
The biggest step that I think most companies could benefit from without drastically increasing their cybersecurity spending would simply be to attempt to deploy and use less software and fewer services. Each new piece of software is just waiting to be exploited, waiting to be out of date and forgotten. It'll become part of someone's "necessary" workflow, but it will never be important enough that it's simply kept up to date and managed well. If that software were never there in the first place, the business would simply use some other workflow process, and there would be nothing to be forgotten and exploited.
Being explicit about what you want and what you'll give up for it is a nice thing.
Well, they seem to be learning now.
One small fuckup is sufficient to have the whole thing compromised (though cybersecurity people do live to talk about defence in depth) and security doesn’t compose well. That is, security is a property of the system as a whole and security of individual parts needn’t imply security of the whole.
I think analogies to physical security are bad because when you think of physical security you don’t imagine stealthy strangers constantly trying to pick your locks or open your windows or even tailgate people into the office at all times of day or night. But once one has a tool to breach a common system, it can be quickly tried on many systems at little marginal cost. When the marginal cost and risk are low, you may see more attacks. The real difference these days is that cryptocurrencies have made the potential payout high and so there are a lot of incentives to carrying these attacks.