Forget Pre-Incident Cost, How Much Did Your Last Incident Cost?
I just read this great post by Rich Mogull titled FireStarter: The Only Value/Loss Metric That Matters. His basic argument, or at least the idea that I derived from it, is the following (all in my own words).
So-called "risk managers" spend a lot of time imagining they can determine "annualized loss expectancy" by predicting how much an incident will cost. Forget all that nonsense. Before imaging what a future incident will cost, figure out how much your last incident cost.
This is brilliant because it is so simple yet drives straight at the heart of the problem. We work incidents all the time and I can't tell you how much they cost. Think about all the factors to consider:
- Value of professional time of everyone who detected and responded to the incident
- Value of computing resources affected by the incident
- Value of data affected by the incident, whether disclosed, degraded, or denied
- Value of brand, reputation, and other "goodwill" items
- What else can you imagine?
So, think about answering these questions for a really good recent interest before wasting time imagining costs of future incidents. I think what you will find is that this can be a really difficult exercise. However, if you can derive some general guidelines, it's worth it.
Comments
In particular I am thinking about the last three items on your list:
Value of data, Value of brand, and whatever else you can imagine.
While it conceptually makes more sense, I don't quite see how it is going to be easier or better, given the "intangible unknowns" of the equation.
Having done a fair bit of research into some of the bigger breaches - TJX and Heartland, for example - one of the things that becomes clear is the the cost of a breach unfolds over a period of years.
If you look at the SEC 10-K filings from TJX for 2007, 2008, and 2009, you will see that the while actual costs are recorded as such, there is also a reserve created that goes up and down in value as things are settled, estimates of future liability are resolved, etc.
The same seems to be true of Heartland.
I am not sure what the statute of limitations is with regard to a data breach - maybe a lawyer could respond, if one is following this discussion.
Patrick Florer
Dallas, Texas
Something I forgot to mention -
For some breaches, fines and legal settlements can be significant, expecially when PCI contracts apply -
It was just announced in the press that Heartland and MasterCard had reached a tentative settlement for $41M over that breach.
With regard to brand, reputation, competitive advantage, stock price - I would forget about trying to quantify these things - it's too difficult, and such data as there are do not support any particular conclusion.
Patrick Florer
Dallas, Texas
ALL economic decisions are based on estimates of future cashflows, discounted for the time-value of money. If you can't estimate the cash flows, at least within some range, then you can't make rational economic decisions. Period.
Historical costs are only meaningful as input to future cost estimates.
By "rational economic decisions", I mean choices between security spending and other corporate spending, choices between spending now vs. spending later, choices between alternative security practices or policies or investments.
Now, you may be suggesting exactly this -- that information security is beyond the reach of economic rationality, and therefore we should fall back on other methods such as guidelines, heuristics, etc. that we can only hope will lead to better decisions and better results than simple-minded methods (hunch, impluse, fad-following, blind-leading-the-blind, etc.).
Lastly, I'd like to point out that historical costs associated with "brand, reputation, and other 'goodwill' items" play out over time, often many years. Thus the full costs won't be recognized if the incident is recent. This is the challenge that the Ponemon survey faces. They "solve" it by using some rule-of-thumb formulas for number of customers lost, cost to replace a customer, etc. but that is simply a way of forecasting future costs.
For more on this, you might look at the blog post I wrote recently about the general problem of incorporating the time dimension in security metrics: http://newschoolsecurity.com/2010/05/getting-the-time-dimension-right/
I thought it was obvious that the purpose of determining historical cost would be as an input for future decision making? The point I was trying to emphasize is that too many managers start backwards in my opinion, creating fantasies about future costs but never having measured past costs.
I'm a big fan of measuring/estimating TOTAL costs of information security, including costs of prevention, detection, forensics, backup/recovery, incident response, help desk, security management, etc.
But I think this needs to be projected forward. The best way to do this is to look at BUDGETED costs, especially outside the security department. It takes some effort to tease out costs related to security (using cost drivers) but it's possible.
But this only works for the frequent and expected costs. TJX and Heartland do not budget in the future for all the costs that they incurred in their past (infamous) breaches.
There's no way around it -- you need to have some way of roughly estimating low probability, high impact costs, and your "willingness to pay" to avoid them.
Such a method is outlined in this presentation: http://meritology.com/resources/Total%20Cost%20of%20Cyber%20(In)security.ppt
But let me be clear -- this method is not yet ready for mainstream use. There are serious unsolved research problems. But I think they can be solved, at least to an acceptable level.
eDiscovery Solutions Group (www.ediscoverysolutionsgroup.com) actually has an incident management system that can track cost and therefore address this requirement.
My slightly provocative question was aimed at the perception that security incidents will always be identified and have measurable cost.
Lost productivity is the only measure I can imagine which can be reliably tracked.