Are the Questions Sound?
Dan Geer, second of the three wise men, was kind enough to share slides from his Measuring Security USENIX class. If I were not teaching at USENIX I would be in Dan's class.
One of the slides bothered me -- not for what Dan said, but for what was said to him. The slide is reproduced above, and the notes below:
These are precisely the questions that any CFO would want to know and we are not in a good position to answer. The present author was confronted with this list, exactly as it is, by the CISO of a major Wall Street bank with the preface “Are you security people so stupid that you cannot tell me....”
This particular CISO came from management audit and therefore was also saying that were he in any other part of the bank, bond portfolios, dervative pricing, equity trading strategies, etc., he would be able to answer such questions to ﬁve digit accuracy. The questions are sound.
I think Dan is giving the CISO too much credit. I think the questions are "semi-sound," and I think the CISO is the stupid one for using such a negative word to describe one of my Three Wise Men.
I'd like to mention several factors which make comparing the world of finance different from the world of digital security. I am recording these because they are more likely the kernel for future developed ideas, but I think they are legitimate points.
- Business: Digital security is not a line of business. No one practices security to make money. Security is not a productive endeavor; security risk is essentially a tax instantiated by the evil capabilities and intentions of threats. Because security is not a line of business, the performance incentives are not the same as a line of business. Security has no ROI; proper business initiatives do. Only security vendors make money from security.
- Accumulation: Digital security, as defined by preserving the confidentiality, integrity, and availability of information, cannot be accumulated. One cannot tap a reserve of security and later replenish it. Data that is exposed to the public Internet can seldom be quashed; data that has been corrupted at time of critical use cannot be changed later, thereby changing the past; and data that was not available at a critical time cannot be made available later, thereby changing the past.
This is not the same with capital (i.e., money). Financial institutions are regulated and operated according to capitalization standards that dictate certain amounts of money to cover potential adverse events. Therefore, money can be stored as a counter to riskier behavior or decreased when pursuing less risky activities. Money at a single point in time is also homogenous; the first dollar of $100 is equally valuable as the hundreth dollar of $100. Information resources are not homogenous.
- Assumptions: Assumptions make financial "five digit accuracy" possible. Consider the assumptions made by the Black-Scholes model, courtesy of Wikipedia, used to price options:
- The price of the underlying instrument St follows a geometric Brownian motion with constant drift μ and volatility σ:
- It is possible to short sell the underlying stock.
- There are no arbitrage opportunities.
- Trading in the stock is continuous.
- There are no transaction costs or taxes.
- All securities are perfectly divisible (e.g. it is possible to buy 1/100th of a share).
- It is possible to borrow and lend cash at a constant risk-free interest rate.
- The stock does not pay a dividend (see below for extensions to handle dividend payments).
The specifics of this equation are not important for this discussion, although those of you who also studied some economics may find plenty of ways to criticize it. (Remember the authors won the Nobel Prize for this equation and paper!) Consider what you could define if digital security practitioners were able to make such assumptions.
- Accuracy: I just said "assumptions make five digit accuracy possible." This isn't really true. If financial five digit accuracy were possible, no markets could be sustained. Simply put, markets exist because two sides agree to a trade. One side sees the world in one way, and the other sees it differently. (This is why market-makers exist on trading floors. When too many traders see the world the same, market-makers provide liquidity to permit trading.) If trading houses all figure out how to make money with five digit accuracy, their advantage is not going to be sustained because no one will want to trade with anyone else -- they're all want to take the same positions.
These are a few thoughts. It would be nice to hear from people with digital security and financial trading experience to provide commentary. Thank you.
Good points as always, especially as to the semi-similarities between digital security and finance, but the wikipedia links are all broken:
The model does not generate a prediction of return that is worth anything in isolation. It is therefore wrong to apply the term "investment" or "return" as being taken from the balance sheet side.
In summary to the extent that ROI or NPV works, they work equally well for security. To the extent that the ROI and NPV models don't tell you what the return is on these projects, independently, they also don't show return on security investment, nor returns on any other investment.