No ROI? No Problem
I continue to be surprised by the confusion surrounding the term Return on Investment (ROI). The Wikipedia entry for Rate of Return treats ROI as a synonym, so it's a good place to go if you want to understand ROI as anyone who's taken introductory corporate finance understands it.
In its simplest form, ROI is a mechanism used to choose projects. For example, assume you have $1000 in assets to allocate to one of three projects, all of which have the same time period and risk.
Clearly, the business should pursue project 3.
Businesspeople make decisions using this sort of mindset. I am no stranger to this world. Consider this example from my consulting past, where I have to choose which engagement to accept for the next week.
Obviously this is the same example as before, but using a real-world scenario.
The problem the "return on security investment" (ROSI) crowd has is they equate savings with return. The key principle to understand is that wealth preservation (saving) is not the same as wealth creation (return).
Assume I am required to obtain a license to perform consulting. If I buy the license before 1 January it costs $500. If I don't meet that deadline the license costs $1000. Therefore, if I buy the license before 1 January, I have avoided a $500 loss. I have not earned $500 as a result of this "project." I am not $500 richer. I essentially bought the license "on sale" compared to the post-1 January price.
Does this mean buying the license before 1 January is a dumb idea because I am not any richer? Of course not! It's a smart idea to avoid losses when the cost of avoiding that loss is equal to or less than the value of the asset being protected.
For example, what if I had to pay $600 to get a plane ticket from a far-away location to appear in person in my county to buy the license before 1 January? In that case, I should just pay the $1000 license fee later. For a $500 plane ticket, the outcome doesn't matter either way. For a $400 plane ticket, I should fly and appear in person. Again, in none of these situations am I actually richer. No wealth is being created, only preserved. There is no ROI, only potential savings.
What if I chose to avoid paying for a license altogether, hoping no one catches me? I've saved even more money -- $500 compared to the pre-1 January price, and $1000 compared to the post-1 January price. This is where the situation becomes more interesting, and this is where subjectivity usually enters the picture concerning expected outcomes.
Let's get back to ROI. The major problem the ROSI crowd has is they are trying to speak the language of their managers who select projects based on ROI. There is no problem with selecting projects based on ROI, if the project is a wealth creation project and not a wealth preservation project.
Security managers should be unafraid to avoid using the term ROI, and instead say "My project will cost $1,000 but save the company $10,000." Saving money / wealth preservation / loss avoidance is good.
Another problem most security managers will encounter is their inability to definitively say that their project will indeed save a certain amount of money. This is not the case for licensing deals, e.g., "Switching from Vendor X's SSL VPN to Vendor Y's SSL VPN will save $10,000" because the outcome is certain, breach of contract nonwithstanding. Certainty or even approximate probability is a huge hurdle for many security projects because of several factors:
As a result, risk assessment is largely guesswork. Guesswork means the savings can be just about anything the security manager chooses to report.
If you look at my older posts on return on security investment you'll see some more advice on how to make your case for security spending without using the term "ROI".
It should be clear by now that ROSI or security ROI is nothing more than warping a defined business term to get attention during budget meetings. I saw the exact same problem in the Air Force. At one point those who flew combat missions were called "operators." Once Information Operations came into vogue, that community wanted to be called "operators" too. At one point a directive came down that intel folks like me were now "operators," just like combat pilots. That lasted about 10 minutes, because suddenly the combat pilots started using the term "trigger-pullers." "Fine," they thought. "Call yourselves operators. We pull triggers." Back to square one.
The bottom line is that security saves money; it does not create money.
In its simplest form, ROI is a mechanism used to choose projects. For example, assume you have $1000 in assets to allocate to one of three projects, all of which have the same time period and risk.
- Invest $1000. Project yields $900 (-10% ROI)
- Invest $1000. Project yields $1000 (0% ROI)
- Invest $1000. Project yields $1100 (10% ROI)
Clearly, the business should pursue project 3.
Businesspeople make decisions using this sort of mindset. I am no stranger to this world. Consider this example from my consulting past, where I have to choose which engagement to accept for the next week.
- Spend $1000 on travel, meals, and other expenses. Project pays $900 (-10% ROI)
- Spend $1000 on travel, meals, and other expenses. Project pays $1000 (0% ROI)
- Spend $1000 on travel, meals, and other expenses. Project pays $1100 (10% ROI)
Obviously this is the same example as before, but using a real-world scenario.
The problem the "return on security investment" (ROSI) crowd has is they equate savings with return. The key principle to understand is that wealth preservation (saving) is not the same as wealth creation (return).
Assume I am required to obtain a license to perform consulting. If I buy the license before 1 January it costs $500. If I don't meet that deadline the license costs $1000. Therefore, if I buy the license before 1 January, I have avoided a $500 loss. I have not earned $500 as a result of this "project." I am not $500 richer. I essentially bought the license "on sale" compared to the post-1 January price.
Does this mean buying the license before 1 January is a dumb idea because I am not any richer? Of course not! It's a smart idea to avoid losses when the cost of avoiding that loss is equal to or less than the value of the asset being protected.
For example, what if I had to pay $600 to get a plane ticket from a far-away location to appear in person in my county to buy the license before 1 January? In that case, I should just pay the $1000 license fee later. For a $500 plane ticket, the outcome doesn't matter either way. For a $400 plane ticket, I should fly and appear in person. Again, in none of these situations am I actually richer. No wealth is being created, only preserved. There is no ROI, only potential savings.
What if I chose to avoid paying for a license altogether, hoping no one catches me? I've saved even more money -- $500 compared to the pre-1 January price, and $1000 compared to the post-1 January price. This is where the situation becomes more interesting, and this is where subjectivity usually enters the picture concerning expected outcomes.
Let's get back to ROI. The major problem the ROSI crowd has is they are trying to speak the language of their managers who select projects based on ROI. There is no problem with selecting projects based on ROI, if the project is a wealth creation project and not a wealth preservation project.
Security managers should be unafraid to avoid using the term ROI, and instead say "My project will cost $1,000 but save the company $10,000." Saving money / wealth preservation / loss avoidance is good.
Another problem most security managers will encounter is their inability to definitively say that their project will indeed save a certain amount of money. This is not the case for licensing deals, e.g., "Switching from Vendor X's SSL VPN to Vendor Y's SSL VPN will save $10,000" because the outcome is certain, breach of contract nonwithstanding. Certainty or even approximate probability is a huge hurdle for many security projects because of several factors:
- Asset value is often undetermined; in some cases, assets themselves are not even inventoried
- Vulnerabilities in assets are unknown, because new flaws are discovered every day
- The threat cannot be properly assessed, because they are unpredictable and creative
As a result, risk assessment is largely guesswork. Guesswork means the savings can be just about anything the security manager chooses to report.
If you look at my older posts on return on security investment you'll see some more advice on how to make your case for security spending without using the term "ROI".
It should be clear by now that ROSI or security ROI is nothing more than warping a defined business term to get attention during budget meetings. I saw the exact same problem in the Air Force. At one point those who flew combat missions were called "operators." Once Information Operations came into vogue, that community wanted to be called "operators" too. At one point a directive came down that intel folks like me were now "operators," just like combat pilots. That lasted about 10 minutes, because suddenly the combat pilots started using the term "trigger-pullers." "Fine," they thought. "Call yourselves operators. We pull triggers." Back to square one.
The bottom line is that security saves money; it does not create money.
Comments
http://www.bloginfosec.com/2007/07/13/bejtlich-and-business-will-it-blend/#comment-1751
Ken
http://www.bloginfosec.com
Bammkkkk
Interesting post. I understand your point, but I'm not sure I agree. I only have a little bit of training in finance, so I certainly don't speak with authority on the subject. That said, I know of some people who do speak with authority who (by my interpretation) disagree with you. Specifically, Lawrence Gordon and Martin Loeb.
Gordon and Loeb are both professors of accounting and information assurance at the University of Maryland School of Business. They are also the authors of the most comprehensive work on information security and finance, "Managing Cybersecurity Resources." Throughout this work (and many of their papers that have been published in peer-reviewed accounting journals) they extensively discuss information security as something that generates return. Given their PhDs in managerial economics, I doubt they are misusing the term.
Your point about security not putting money in a firm's pocket in most cases is well taken; however, my understanding is that you don't have to generate revenue to have return on investment. Return on investment is generated by any capital expenditure that increases the value of the firm. Return on investment is a total value/net worth concept that is distinct from cash flow. Revenue (which security doesn't produce in most cases) is a cash flow concept.
Ryan Heffernan
Thanks for reminding me about their book. I have had it on my Amazon.com wish list, so I will have to buy it to see what they say. Assuming they are economists I am pretty sure they actually will agree with me, but I will read the book when I get a copy.
Funny, the article referenced in my comment above was authored by Lawrence Gordon and Robert Richardson. In it, they state that ROI can't be applied perfectly to information security. I expect that means they are more likely to agree with Richard.
Bammkkkk
My point is that they are economists and they explicitly don’t agree with you. Gordon’s own home page on the University of Maryland site mentions return on security investments:
"In other words, organizations may derive a higher return on their security activities by investing in cyber/information security activities that are directed at improving the security of information sets with a medium level of vulnerability." - (Gordon’s UMD Home Page, emphasis added)
Bammkkkk,
I’ve read nearly everything Gordon and Loeb have written on the subject (I wrote a paper related to this topic for my masters degree), including the paper you cited above. Gordon’s problem with ROI isn’t that security doesn’t have returns, it’s that ROI is a simplistic calculation that doesn’t account for the time value of money (real financial professionals don't use ROI in general). Gordon recommends net present value (NPV) for weighing security investments, which does account for the time value of money, but also involves a concept of return.
I think Richard has an excellent point to make, but I think he has confused the concepts of return and positive cash flow.
-Ryan Heffernan
Again, thanks for reminding me of their work. However, I'm not confusing anything.
I recommend reading Anton's recent post.
I'll have more to say about this after I read Gordon and Loeb's book.
http://www.bloginfosec.com/2007/07/18/email-from-dr-lawrence-gordon-security-roi-possible-but-not-optimal-use-other-metrics
Ken
http://www.bloginfosec.com
I have literally the same real-world experience in regard to sitting in meetings when someone asks about ROI on a security project and the whole project or idea gets almost immediately canned because it is "not a good project because there is no ROI". I got in several debates with colleagues over this and spent a lot of time trying to convince people a particular procedure or softwaare was good security, and I can tell you what Richard says is TRUE - 100%. It is a really frustrating thins that IT people face on a regular basis - especially in regard to security.
If only I had thought of explaining the difference between wealth preservation and gaining wealth (as Richard did in this post) I would had been successful with my Director CFO, President, etc.. Where was this blog post 4 years ago? :-) Just kidding, but seriously thanks for the great info and persepective Richard!
-Tyrel
www.SitesCollide.com
As a result, risk assessment is largely guesswork. Guesswork means the savings can be just about anything the security manager chooses to report.
Please have a look at the Open Source project ORIMOR (http://www.somap.org/repository/). ORIMOR is the short form for "Open Risk Model Repository". We are working on ORIMOR with the intention to solve some of the problems you mention. Assets need to be inventoried and to be valued. What about helping the security officer and put some generic value onto an asset which can then be used in an assessment? We can do the same with Threats. Why not share threat values and likelihood information amongst different organisations? This could lead to an average value which could be better than wild guesses.
Somebody said once that the numbers we currently work with are bad but that these are the only ones we have. Lame excuse. It is our goal to start to share risk related data so that we have the chance to create better data.
Unfortunately this space is too small for such a big topic. Please drop me a note if you want to discuss further.
Adrian
ROI / NPV quite happily deals with negative numbers like savings and expenses, it's neutral to what the balance sheet says about the resultant money flows. Another way of thinking about it is that there is no return on investment in ROI; there is simply an ability to compare projects based on the same assumptions and inputs.