News for hackers. Stuff that matters.
Re:Threat Modeling Manifesto by motohagiography on Sunday February 07, 2021 @09:50AM (#26055379)
I've mostly given up on threat modelling as a solution, even after building a tool from scratch and using it in organizations, and now only do it privately on my own stuff, but the reason might be a fresh perspective.

The basic problem is that the demand for security stems almost exclusively from compliance because compliance transfers risk out of the project and onto the standard/model. The idea of thinking through risks and hypotheticals and leaving a paper trail that you have acknowledged them basically converts theoretical risk into real liability within the project. In an ideal world of individual ownership and goodness, this means the incentives are aligned to create a good product, but in the real world of organizations, you've reduced the flexibility of the business to manage that risk, which means to take the risk and respond to it on the fly as it looks like it becomes realized. By problematizing the risk instead of managing it, you have destroyed potential value. Threat modelling is the exercise of problematizing risk.

This is why the threat model is often the elephant in the room. Almost nobody in business wants to have the "negative" conversation about whether to block law enforcement (foreign or domestic), surveillance on users, protect against data snooping by technical staff (which is often a platform perk and feature), or institutional privacy violations and other obvious threats that a threat modelling exercise addresses. A business PM won't problematize those things because it will demand a solution that gets in their critical path. The best they can do is manage it, which means orienting themselves to risk and being ready to respond.

When people buy security products, they almost exclusively buy ones that provide data to manage risk using surveillance and monitoring, which means generating data that drives conversations and enables the organization to adapt in time. If a security product does not either a) externalize a risk with compliance or b) provide data to flexibly manage risk, it's not a security product in the market today because nobody will have bought it.

I realize this is a 10th man view of something these very smart people have spent a lot of time on, but in the 25 years I have been in the field, I have come to the conclusion that security people are essentially environmentalists and activists trying to make a case for internalizing an exogenous problem and cost, and the only way to get traction for it is to produce either low-level discretionary developer tools, or generate data that supports the conversations and provides that flexibility in a business' attitude to risk. Threat modelling is super powerful, but as a forcing function it can destroy value, and this is why it has encountered so much resistance. All manifestos are quixotic, and that is why I think this one may also be.

I have found that the real value of a threat model is defining the business model for a security product, where the threat model is the business case for a product, but that product needs the above features, and to not be a solution to succeed. The security product provides either compliance or data for the threats you derive from your modelling exercise, and it must either transfer risk to a model or provide data to manage. If it leaves responsibility in the project, like threat modelling, it's going to fail.

Re:Threat Modeling Manifesto by A_No_Name_Mouse on Sunday February 07, 2021 @12:36PM (#26056512)
Your comment resonates with me but I'm not sure I fully understand it: "compliance transfers risk out of the project and onto the standard/model": If I choose to be compliant to a standard, some of the risks are mitigated, not transferred; some risks I have that are not covered by the standard are not mitigated or transferred, but still unknown, because I chose not to look into the threats (but just comply with some standard). So how does it transfer risk out of a project?

"A business PM won't problematize those things because it will demand a solution that gets in their critical path. The best they can do is manage it": If you identify a risk you can still choose to accept it and do nothing to mitigate it. So why does it demand a solution?

"you've reduced the flexibility of the business to manage that risk": Why? Again, risk identification does not automatically mean risk mitigation. If you're aware of a risk you may choose to detect problems instead of prevent them from happening. How else would you be able to manage risk if you are not even aware of it?

Maybe your comment is related to something I have experienced. I often think about a theoretical situation related to the movie the Matrix. Suppose there exists a complete risk analysis that provides insight into every aspect of risk for a company. By taking the red pill responsible management would instantly see all these risks, but it cannot be "unseen", so they would have to act to reduce them to acceptable levels (taking away resources that would otherwise be devoted to business functionality). Would they take the red pill or the blue pill? I think most managers would choose to stay ignorant.

[edit: wording]

Re:Threat Modeling Manifesto by lmkg on Sunday February 07, 2021 @01:55PM (#26057141)
> some risks I have that are not covered by the standard are not mitigated or transferred

If you're compliant with a regulation or a standard, then you're not liable for failures of those standards.

I think there's an implicit assumption that the harm of a failure will fall on users/consumers/some other third party, rather than yourself. In this case, the "risk" is liability for damages, rather than actual damages. If you roll your own risk-management strategy, then you assume responsibility for its failures. If you follow a blessed external standard, then you transfer responsibility to the standard. So the risk of a failure is not fully mitigated, but your risk of being liable for that failure is. Which, to GP's point, disincentivizes exploring novel approaches that mitigate the risk of real failures.

Re:Threat Modeling Manifesto by A_No_Name_Mouse on Sunday February 07, 2021 @02:23PM (#26057398)
Thanks. In this context it makes perfect sense.
Re:Threat Modeling Manifesto by __jf__ on Sunday February 07, 2021 @11:05AM (#26055879)
Most of the compliance paperwork I’ve seen does leave room for custom risk assessments, threat modeling or other wordings that invite a business team to do more. However in their rush to go live or otherwise get it over with this security work is done after all other things. It isn’t integrated, in an SDLC for example.

So minimum standards become maximum standards. It’s hard work convincing teams to do better, but at least the compliance docs give permission to develop your own, often better, understanding if the data classification is high enough. It doesn’t happen often but I havent abandoned all hope yet.

Re:Threat Modeling Manifesto by contingencies on Sunday February 07, 2021 @07:01PM (#26059386)
A threat model is just a cognitive tool whose primary value is in creating an alternative perspective on a design problem to broaden thinking and improve outcomes. Like any tool, there are people who carry the hammer too far and suddenly everything is a nail.

As soon as they start harping on about a "journey of understanding" and "multiple deliveries" your bubbling instinct of disgust is correct and you should run and hide.

I don't trust security people to do sane things. - Linus Torvalds (2017)

Re:Threat Modeling Manifesto by clairity on Sunday February 07, 2021 @11:29AM (#26056033)
while i appreciate the insights you bring to bear on the actual behavior of organizations and teams, your response seems to be resigned to accepting the status quo rather than working to change the incentives so that security risks and damages are internalized appropriately.

this kind of excuse-making is how we slide down various slippery slopes (e.g., how google's "don't be evil" eventually failed), even with eyes wide open. instead, the better response is to advocate legislative and organizational improvements while also working within your perceived constraints. it can be both, not either-or.

Re:Threat Modeling Manifesto by motohagiography on Sunday February 07, 2021 @11:40AM (#26056126)
This comment expresses a misunderstanding. Merely not believing hard enough is not why threat modelling has failed. There is an incentive mechanism and this description of it can help them re-orient their strategy.

The counterfactual premise of threat modelling is that a business wants responsibility for mitigating or remediating risk without direct compensation, instead of a method to manage and transfer it. A technologist is just happy to solve problems, so they don't see this open loop as a source of value.

Re:Threat Modeling Manifesto by nokya on Sunday February 07, 2021 @05:18PM (#26058758)
Very interesting comment, thank you for sharing your thoughts. I particularly enjoyed your depiction of security products.

Years ago, I advocated strongly for threat modelling and bringing the process "to the masses". I was a security consultant, it was easy for me to tell others what they should do.

Then I started working as an "internal" in an company large enough to offer me both many different perspectives and full access to things that needed threat modelling. Architects and developers I only interacted with as my "customers" became my colleagues. And my point of view completely changed.

I agree with you: threat modelling, as it is "evangelized" today faces strong resistance from business as an expensive and negative activity. The infrequent times it is enjoyed, reasons are not those we think: threat modelling is really a "cool" experience when done correctly, participants learn a lot of things and usually enjoy the activity. Then comes the second round. And the third. And that's where you see it hits a wall: either you repeat the same thing, again and again, or participants don't have the time.

Today, I still do threat modelling, and quite actively. On my own, as you said. My colleagues don't even know I use this process to prepare my internal publications. I write threat advisories, which I then send to a little more than 300 architects, developers and project managers worldwide. My recommendations are the output of a threat modelling process, these advisories fly into the field in a "finished" state. Threat modelling leverages its full strength in environments that can benefit from high capitalization of thought, such as when you deal with a large number of silos/teams that are exposed to very similar problems.

There is a very simple trick to catch wannabe threat modellers: look for threat models that include a "validate all input" recommendation or alike. If you see this in a threat model, you know you are dealing with someone who hasn't actually understood what threat modelling is and is simply repeating some tricks learned the evening before. I could also talk about what I think of security vendors that try to sell threat modelling as some sort of "transferable" skill, which can be taught in one or two days to non-security professionals. But I will refrain.

Now about the manifesto. In my opinion, Adam Shostack deserves full credit, not for inventing threat modelling but for actually formalizing the process in its most efficient and final form (i.e. the 4 threat modelling "questions"). All attempts to complete his work (e.g. books, methodologies, articles, etc.) following his release of the 4 questions are nothing more that complicated "techniques" to derive threats from a diagram or a specification.