Charles Wilson's Blog

I’ve only been threat modeling for about a decade. In that time, I’ve rarely seen threat modeling used as intended. Why is that and why do we threat model in the first place?

Lately everyone seems to have jumped on the threat modeling bandwagon. Every new certification wants to see a box ticked for threat modeling, so companies are scrambling to make that happen. But many times this means that companies are threat modeling after they’ve finished development. Does that even make sense? Let’s take a step back and ask the question, “Why are we doing this?”

Every development team understands unit testing. They get static analysis. Conceptually, they appreciate penetration testing, if not what it’s limitations are. But when it comes to threat modeling, you get more hand waving than a cruise ship departure.

There’s no end to the number of misconceptions regarding threat modeling. Some see it as a one-time, rubber stamp on the product’s security. Others believe that it must expose all known issues in order to be considered valid. Still others believe that if they contribute a component, that they are exempt from threat modeling because it will be done at the system level. And so forth. The underlying problem is not that all of these ideas are wrong, it’s that people don’t know why threat modeling exists.

First, threat models are only useful if they evolve with the product’s design. Not the implementation, but the design. Why? Because the point of threat modeling is to evaluate the design. The outcome of the exercise is a set of design deficiencies. Note that this is a very different thing from what a penetration test discovers. You can have a perfect design and still find issues during penetration testing, because it can identify errors in implementation. Since the threat model uses an idealization of the systems for its analysis, that isn’t possible.

Second, threat models focus us on best practices by forcing us to consider the decomposition of the system, the trust boundaries, and data flows. A threat model won’t ask if we’re using 4096-bit keys. It will ask if we have appropriate controls. Controls imply requirements, specifically, non-functional security requirements. That’s a topic for another day.

Third, threat models allow us to systematically and consistently consider the system. These are two very important elements. One of the questions we seldom ask properly is “How do we know we have security coverage?” Since threat modeling deals with a model of the system, the answer is once we’ve iterated over all the data flows in all the workflows of the system. Good luck doing that with penetration testing. The second element is equally important. Threat modeling is only useful if it produces consistent repeatable results. That means how we undertake the exercise matters. In my opinion, any moderately trained individual should be able to get the same results out of a threat model as some of equal training. Will there be deviations, sure. Should they be substantive? No.

I’ve always found it frustrating when people expect threat models to identify all know issues. This is both unhelpful and unnecessary. It’s unhelpful, because it implies an over-tuned analysis process that probably won’t work if things change. It’s unnecessary because as mentioned issues that are known are known. Especially if said issues are identified in testing, since you already have coverage. Also, they tend to be implementation issues.

One of the difficulties in the threat modeling process is that is not only exposes design deficiencies, but also that it exposes a lack of design process. In a hyper-full-stack-focused world, many have forgotten that agile doesn’t mean that you just build until it works and document as implemented. That kind of sloppy engineering doesn’t fly when safety is an issue. You can be agile, but all the forms still apply. An advantage in incorporating threat modeling into an agile design process is that since you’re system is by definition smaller, there’s less to model. Additionally, if you’re using compositional threat modeling, things can go really quickly. It’s elementary network theory.

The final thing I’ll bring up is cost. We threat model to save time and money. There’ve been multiple studies showing that the further along the process is when you identify an issue (especially in the underlying design) the more expensive it is to mitigate. And who doesn’t like to save money. Well, project management, I suppose, they just want to save time.

This post is a bit different from my usual link-laden ones, but I’ve been thinking about big picture stuff lately.

References

The Thinker, photo by Andrew Horne (public domain)
https://commons.wikimedia.org/wiki/File:The_Thinker,_Rodin.jpg

Read Full Post »