When Security Engineering is Neither Security, nor Engineering

SecurityEssay

Jun 11

Welcome, dear reader, to the first – and perhaps last – opinion-style blog post I will ever write.

This post is intended for folks who take an interest in the security industry (which I’ve now been involved in for a decade). Others may find it interesting, but due to my own laziness I will assume some foundational background in the space.

Finally, I will admit this post is written in a manner much less subtle than I’m usually inclined to do. Those who know me would likely attest to the fact that I rarely give definitive answers to broad questions. Still, I hope you’ll forgive me, just this one time, to share what I think is the most damning aspect of the computer security engineering field.

A Question

Let me start by asking a question. Can your security department answer the following question:

> How much money will your company lose in the next 5 years due to security issues?

Think about that hard. (The time-horizon length is completely arbitrary, for what it’s worth.)

If your organization is like every organization I’ve ever worked at or talked with, the answer to this question is a timid, but ultimately clear, “no”.

Ouch.

A claim

Here’s my big claim: the fact that we can't reliably answer that question with any rigor is *the* biggest fault of the security engineering field. More so than FUD-driven security products (although that’s related), more so than GenAI security (although it’s horrifying), and more so than phishing (although that's hard to prevent).

I will spend the rest of this essay discussing why I think you should accept this claim, but in essence it’s for the following reason:

Such well-discussed security problems or gripes with the industry are also ones we’ve made progress on. I remain confident that we will find ways to address those just as we succeeded in, for example, securing the internet (HTTPS was, at one point, a lofty pipe dream).

On the other hand, we’ve made essentially no progress measuring security risk. At best, teams find vulnerabilities and mark them high/medium/low (usually arbitrarily), cross that with their companies’ annual revenue and say they’re measuring “risk”. These simple pseudo-forms of analysis might seem satisfying but are wrought with problems. To learn more about why, I suggest Douglas W. Hubbard and Richard Seierse’s apt How to Measure Anything in Cybersecurity Risk [0].

This hand-wavy risk analysis is, or should be, unacceptable. An industry who talks “risk” more often than Disney pumps out Marvel movies should be able to measure it with more rigor than a middle school math class.

On security

What is "security” anyway? I recently asked this to a group of high school students as part of a career day week at their school. I specifically asked about what is meant to “be secure”. The question was not necessarily linked to cybersecurity. The answers were interesting:

“Being safe”
“Not losing your money”
“Not getting ‘hacked’”

And many more. The answers were quite varied and all approached the “real” answer but never quite hit the mark. The “safety” answer was especially interesting, since the security and safety fields overlap significantly. After some pressing, I revealed *my* preferred colloquial definition:

“Security: assurance that your expected state will not be hampered on by an external entity”

In other words: your car is safe when it doesn’t randomly explode with you in it. Your safety in the car is secure if another car hitting yours doesn’t result in your death. Financial security is not about being rich, but being sure you won’t lose your house if you lose your job (for example).

That seemed to drive home the point and the group of high school students seemed to have “gotten it” and provided plenty of other good examples. I think most of us in the security industry “get this” too.

I then asked a harder question: “how do you know you have financial security? How do you know your well-being in your car is guaranteed from external factors”?

The result was a LOT of silence followed by someone eventually muttering “when it feels right”? They seemed to know this couldn’t be a good answer, and yet, I believe most of the security industry doesn’t have much of a better answer either, and we’re the professionals.

I don’t say this to disparage the industry. I do believe we’ve come a long way and have much to be proud of. In particular, it’s my opinion that we’re good at “thinking like an attacker” (a saying that’s almost a cliche at this point)! We’ve learned a lot and can look at systems and think about how to break them well. Security teams are increasingly good at running internal threat modeling programs and can identify many classes of security issues well. Further, advancements in the program analysis space have been immense, and when combined with well-engineered frameworks, a significant amount of low-hanging fruit to attack is gone.

And yet I don’t find this fact fully comforting. While we can identify vulnerability classes and say things like: “we need to have phishing resistant MFA (e.g. WebAuthN)” we rarely can explain why from a monetary standpoint. In other words, we can’t explain the expected loss before or after a proposed change.

Is this capability a bit pedantic? Why does it matter that we can say: “by using username/passwords and not phishing resistant MFA we should expect to see 5-25% of our users have their accounts compromised, resulting in $2 - 5 million in losses”?

A few reasons:

That is what we’re being paid to do.

Organizations might think they’re hiring security teams to “prevent breaches” but that’s of course a misunderstanding. Instead, organizations hire security teams to help keep expected losses below what they are paying out to these very people.

We should be able to justify our jobs.

I worked at a large company with a huge staff in a unit of the company that sells security products. Things were all fine and dandy until one day they laid off a sizable portion of their *security* team in that business unit. Foolish? Maybe. Even probably. However, it was true that the security team was so focused on “actually addressing problems” that they had no way to show that the several million dollar budget they were using was worth it.

It prevents economic waste at a global scale.

Could it be that a sizable portion of the things the security team recommends or uses (or spends on themselves in payroll) actually costs more than what is ever lost? Is spending hundreds of thousands of dollars on a web application firewall actually cost-effective?
Some sources [1] note that about 10% of an organization’s IT budget is spent on security. If IT spend is about 2% of revenue [2], that means we’re spending over 40 billion dollars annually for just the fortune 500 [3].
Is that the right amount? Few know. And that’s the problem.

All this to say: it’s important we measure risk accurately and are able to *forecast* losses. Otherwise, we’re not doing security (risk management), we’re just having fun breaking stuff.

So let’s keep thinking like an attacker, but let’s *also* start thinking like an actuary. Actuary science is the study of risk and is the reason insurance companies stay profitable (because they’re able to reliably estimate impacts over a time horizon for uncertain events). As security teams, we should be able to do the same.

On engineering

Perhaps you’ve bought in that somebody should be able to measure risk and predict costs over time. However, isn’t that the job of the scientific field? Or perhaps cyber insurance companies? Maybe even risk teams? Why should security engineers care? This doesn’t seem like the job of engineers.

I’ll admit for the first many years of my career in security, this is exactly how I thought. Security engineers' job is to help the org build hardened products and identify issues, maybe with respect to “lessons learned” over time. Such rigorous statistics is a bit outside the domain one would expect security engineers to work, so it’s easy to punt it.

Well, over the last few years, my mindset has shifted, in part because I’ve spent more time learning and working with engineers outside the software domain. When you look at other domains with safety-relevant concerns: civil engineering, mechanical, aerospace, chemical, etc. They certainly *do* learn a lot from lessons over time, but it would be foolish to think that’s sufficient. If you’re building a bridge across a river that millions of people will travel over per year, it’s not sufficient to make structural integrity decisions based on what “feels” right. Instead, it’s expected that such engineers will estimate and model the stress across the bridge, longevity of beams or other support structures, and mechanisms and processes for detecting failures early. Of course, you can never have a 100% safe or secure design, that would be horribly expensive (Engineering Explained has a wonderful video that discusses this very thing [4]).

That last sentence implies why security engineers should care about risk modeling. Engineering is all about tradeoffs (aside: my preferred definition for engineering is “design under constraint”). In security engineering one of the primary tradeoffs we are responsible for managing is cost with respect to security. There must exist a series of “acceptable” risks given the cost and “unacceptable” ones. Our job is to ensure we end up in a positive (if not optimal) state.

F-N curve from the 1975 US Nuclear Regulatory Report WASH-1400. Shows the “Frequency of Fatalities due to Man-Caused Events”. In other words, engineers were able to compare the relative safety of nuclear power plant disasters compared to other well-accepted risks. Can we do the same in security?

We also need to be able to explain why we’re confident with our answers. If you asked your nuclear engineering contracting company why they feel confident in the cooling capabilities of a reactor, and they couldn’t give you a very rigorous answer based on lots of data and modeling of the system you would fire them.

All that to say: while it certainly isn’t fun or glamorous to model systems and perform real risk assessments in security, it’s arguably the most important part, and we must do better.

A cause

Look, I get it, hot takes are fun to write, but I really don’t mean to downplay the quality of work the security engineering field is currently doing. I don’t believe the limitations in our industry, mentioned above, are due to laziness or incompetence. I think there are several real factors at play that we need to help address:

Data

When I model systems I often need to get data to make the statistics vaguely reliable. For example, “What’s the likelihood an arbitrary SaaS product you rely on will experience a compromise resulting in customer data loss”?
Without that, how are you going to determine whether building something in-house is better than paying a vendor to manage it, for example?
And yet, such data basically doesn’t exist. Most of the cybersecurity “data” you can find belong to cybersecurity companies with products to sell, not ones to help you make realistic decisions.
This is disappointing and a huge barrier to doing useful risk assessments.

Organizational incentives

In most organizations, security teams have two primary incentives:

Find issues (to prove you’re doing work)
Ensure the company doesn’t experience breaches (because if that happens someone will be pissed and you will be blamed, regardless of whether you “predicted” it or not)

Neither of these play well with the “you should be able to forecast losses related to security and contrast them with the cost of mitigations” as that:

Implies some of the issues you find will be of low impact or not worth doing anything about
Implies you will have to spend work estimating risk instead of “fixing it”, which feels wasteful
Implies the company will inevitably experience a breach or loss even though you’re being paid

As such, there’s not much incentive to spend the time and energy building reliable risk forecasts

Time

The software industry moves super fast and it’s therefore very hard to keep models and data up-to-date.
In addition, there’s a culture expectation in software to automate or streamline processes, which makes “lessons learned”-type security engineering easy: build program analysis tools, do code review or threat modeling sessions, shrink IAM permissions. Modeling tasks are typically not so simple to automate.

Lack of awareness

Very often security teams come from an IT or software engineering background where risk assessment and statistics is not commonplace. As such, it’s easy to simply not think about the need to forecast.

A path

The good news is that these problems are tractable ones.

Regarding data: academia should invest more in researching the intersection of economics and cybersecurity. This is a challenging intersection, but one that is incredibly valuable for society. To support this, the industry should engage in experimentation and share data (at least with researchers). Hiding security-relevant data does everyone a disservice, although I recognize it will take work to convince your company’s legal team to do otherwise. Finally, organizations need to simply attempt risk forecasting and iterate over time. Take whatever data you have, do some one-time risk analysis and estimate losses over a time horizon (say 2 years). 2 years later, compare notes. Was your model right or not? Share those lessons.

Regarding incentives: if you’re considering hiring a security leader (say a CISO) make it clear that you are evaluating the security department like an insurance company. If they say “we expect to lose 5 - 10% of our ARR over the next 5 years” and that happens because you didn’t make any changes, give the security team a raise, not a lay off. On the flip side, if they say “we will lose 5 - 10% of our ARR over the next 5 years” and the only thing you lost was 10 million dollars in ticking off engineering, it’s firing time.

Regarding time: this is simply a matter in investing is tooling and approaches to measure risk across different hierarchies and combine them. Nothing about this is inherently hard, it just needs to be invested in.

Regarding awareness: this post is one attempt. The industry needs to talk more about risk forecasting. Security vendors need to stop making “risk” synonymous with whatever data they can easily present in their specific tool. Instead, build real experiments and publish the fact that, for example, a “high” risk score means we expect this to be exploited in the next 5 years for 1/10 of our customers. In addition, let’s help train security engineers to understand actuary science or risk engineering. As one example, [5] is a great resource.

By not shying away from forecasting and quantitative risk analysis, we can become what our titles imply: engineers who manage security with rigor and professionalism.