Are the Questions Sound?

Dan Geer, second of the three wise men, was kind enough to share slides from his Measuring Security USENIX class. If I were not teaching at USENIX I would be in Dan's class.

One of the slides bothered me -- not for what Dan said, but for what was said to him. The slide is reproduced above, and the notes below:

These are precisely the questions that any CFO would want to know and we are not in a good position to answer. The present author was confronted with this list, exactly as it is, by the CISO of a major Wall Street bank with the preface “Are you security people so stupid that you cannot tell me....”

This particular CISO came from management audit and therefore was also saying that were he in any other part of the bank, bond portfolios, dervative pricing, equity trading strategies, etc., he would be able to answer such questions to five digit accuracy. The questions are sound.

I think Dan is giving the CISO too much credit. I think the questions are "semi-sound," and I think the CISO is the stupid one for using such a negative word to describe one of my Three Wise Men.

I'd like to mention several factors which make comparing the world of finance different from the world of digital security. I am recording these because they are more likely the kernel for future developed ideas, but I think they are legitimate points.

  • Business: Digital security is not a line of business. No one practices security to make money. Security is not a productive endeavor; security risk is essentially a tax instantiated by the evil capabilities and intentions of threats. Because security is not a line of business, the performance incentives are not the same as a line of business. Security has no ROI; proper business initiatives do. Only security vendors make money from security.

  • Accumulation: Digital security, as defined by preserving the confidentiality, integrity, and availability of information, cannot be accumulated. One cannot tap a reserve of security and later replenish it. Data that is exposed to the public Internet can seldom be quashed; data that has been corrupted at time of critical use cannot be changed later, thereby changing the past; and data that was not available at a critical time cannot be made available later, thereby changing the past.

    This is not the same with capital (i.e., money). Financial institutions are regulated and operated according to capitalization standards that dictate certain amounts of money to cover potential adverse events. Therefore, money can be stored as a counter to riskier behavior or decreased when pursuing less risky activities. Money at a single point in time is also homogenous; the first dollar of $100 is equally valuable as the hundreth dollar of $100. Information resources are not homogenous.

  • Assumptions: Assumptions make financial "five digit accuracy" possible. Consider the assumptions made by the Black-Scholes model, courtesy of Wikipedia, used to price options:

    dS_t = \mu S_t\,dt + \sigma S_t\,dW_t \,

    • It is possible to short sell the underlying stock.

    • There are no arbitrage opportunities.

    • Trading in the stock is continuous.

    • There are no transaction costs or taxes.

    • All securities are perfectly divisible (e.g. it is possible to buy 1/100th of a share).

    • It is possible to borrow and lend cash at a constant risk-free interest rate.

    • The stock does not pay a dividend (see below for extensions to handle dividend payments).

    The specifics of this equation are not important for this discussion, although those of you who also studied some economics may find plenty of ways to criticize it. (Remember the authors won the Nobel Prize for this equation and paper!) Consider what you could define if digital security practitioners were able to make such assumptions.

  • Accuracy: I just said "assumptions make five digit accuracy possible." This isn't really true. If financial five digit accuracy were possible, no markets could be sustained. Simply put, markets exist because two sides agree to a trade. One side sees the world in one way, and the other sees it differently. (This is why market-makers exist on trading floors. When too many traders see the world the same, market-makers provide liquidity to permit trading.) If trading houses all figure out how to make money with five digit accuracy, their advantage is not going to be sustained because no one will want to trade with anyone else -- they're all want to take the same positions.

These are a few thoughts. It would be nice to hear from people with digital security and financial trading experience to provide commentary. Thank you.


Anonymous said…

Good points as always, especially as to the semi-similarities between digital security and finance, but the wikipedia links are all broken:
jbmoore said…
Security is as old as warfare, and perhaps older. Security allows one to continue living or doing business except in cases when it fails, i.e. assassination. Digital security and money are likely convenient fictions. We all go along with the monetary systems currently in use because it's practical and convenient to do so. If our money wasn't worth the paper it's printed on, we'd find some other way to do business and such. Digital security is in its infancy. A lot of it is security by obscurity. It is made all the more complex by the heterogeneous networks and systems that are kludged together and that were likely developed with no security in mind in the first place. Then, there are people who are always the weakest link in the chain. So, giving a quantitative assessment about the digital security of some place is about as meaningful predicting the weather forecasts from flapping butterflys' wings. Besides, what's the difference between an economist and some guy who reads animal entrails to predict events. Both are likely right or wrong most of the time. The economist just looks more legitimate because he has formulas and numbers to fall back on, but the underlying assumptions those numbers rely on have not necessarily been tested, such as the rational investor. If people were rational investors, then why all the Wall Street speculation bubbles and crashes? Such events would be minimized if people were rational in their investments and monetary dealings.
Anonymous said…
To repeat my earlier comment, the mistake here I believe is that the model generates a number of comparison with other projects, assuming the same assumptions and inputs.

The model does not generate a prediction of return that is worth anything in isolation. It is therefore wrong to apply the term "investment" or "return" as being taken from the balance sheet side.

In summary to the extent that ROI or NPV works, they work equally well for security. To the extent that the ROI and NPV models don't tell you what the return is on these projects, independently, they also don't show return on security investment, nor returns on any other investment.
Anonymous said…
This comment has been removed by a blog administrator.

Popular posts from this blog

Zeek in Action Videos

MITRE ATT&CK Tactics Are Not Tactics

New Book! The Best of TaoSecurity Blog, Volume 4