Ripping Into ROI

In April I wrote Calculating Security ROI Is a Waste of Time. The latest print issue of Information Security magazine features a story by Anne Saita that confirms my judgement:

"If you find executives resisting your security suggestions, try simply removing the term 'ROI' from the conversation.

'ROI is no longer effective terminology to use in most security justifications,' says Paul Proctor, Vp of security and risk strategies for META Group. [Paul is also author of the excellent book Practical Intrusion Detection, where he correctly said 'there is no such thing as a false positive.']

Executives, he says, interpret ROI as 'quantifiable financial return following investment.' Security professionals view it more like an insurance premium. The C-suite is also wary of the numbers security ROI calculators crunch.

'Bottom line is that most executives are frustrated and no longer interested in hearing this type of justification,' Proctor says. Instead, express a technology's or program's business value, cost/benefit analysis and risk assessment."

Amen.

Comments

Anonymous said…
What's lead to this condition? "Bottom line is that most executives are frustrated..." - is this because it's been done wrong for so long?

I see this all the time in my current environment...every month, incredible effort is put into pulling together "metrics" of performance, but most parties involved know that these metrics don't do anything other than put pretty graphs on paper...ROI isn't being demonstrated. GIGO.

As far as "no such thing as a false positive"...that's very interesting. Admittedly, until the past couple of months, I haven't been "in the weeds" in the IDS field, but after having encountered only a few issues, it's clear to me that that's the case.

H. Carvey
"Windows Forensics and Incident Recovery"
http://www.windows-ir.com
http://windowsir.blogspot.com
7:42 AM
Richard Bejtlich said…
Harlan, thanks for your comment. We discussed "security ROI" quite a bit at CanSecWest '04 and I've chatted via email with Thomas Ptacek about this. In my view, 90% or more of security should be treated like insurance. You buy insurance as a risk transference method when faced with the prospect of loss. Very rarely does security ever provide a "return on investment." The 10% of cases I could imagine would involve increasing the trust of a customer and thereby encouraging new business. A second scenario would be implementing security to meet requirements to do business. Notice that these two cases, especially the latter (in the form of regulatory requirements), are driving security programs in many companies.

Business people see an investment as the application of resources to a project. A positive return on investment occurs when the financial gain from that application of resources exceeds that of the resources applied, adjusting for the time value of money. A good return on investment occurs when the financial gain exceeds that of other projects with similar risks.

About Paul Proctor's comments: I recommend reading pages 108-111 of his book "Practical Intrusion Detection." He lists "myth #2" as "the false positive myth." He writes "There is no such thing as a false positive in an operational environment because, unless caused by a poorly written signature, every alarm has value. It's just that every alarm does not require your full attention. Poorly written signatures are not false positives; they are system failures and should be treated as such."

I totally agree with this perspective. If someone writes a signature to generate an alert every time an IDS sees "cmd.exe", it is not the fault of the IDS when it reports what it sees. The fault lies with the signature writer. I agree that IDS vendors who write poor signatures are partly to blame, especially if users cannot tune these signatures. (That's why I like Snort and Sourcefire so much -- I can make the detection engine act as I see fit.) Layer 7 firewalls (aka "intrusion prevention systems") are forced to operate with much smaller numbers of rules because their blocking decisions have much higher consequences compared to passive IDSs.
11:06 AM
Anonymous said…
Hi - I think you can get ROI from security if you are not completely efficient or completely effective in your operations (which nobody is that I am aware of). More at http://spiresecurity.typepad.com/spire_security_viewpoint/2004/12/still_more_on_r.html.

In addition, it seems to be a common refrain these days to suggest that security is about insurance. I think this is incorrect. Security is about reducing risk to a level that you can accept. Insurance is what you to do as a hedge against that residual risk. Your health and your car are two good comparison models in this respect.

Security and insurance are complementary - that is why an environment with (qualitatively) less security pays more in insurance, and vice versa.

Pete
10:40 PM
Post a Comment

Popular posts from this blog

Zeek in Action Videos

Image
This is a quick note to point blog readers to my Zeek in Action YouTube video series for the Zeek network security monitoring project .  Each video addresses a topic that I think might be of interest to people trying to understand their network using Zeek and adjacent tools and approaches, like Suricata, Wireshark, and so on.  I am especially pleased with Video 6 on monitoring wireless networks . It took me several weeks to research material for this video. I had to buy new hardware and experiment with a Linux distro that I had not used before -- Parrot .  Please like and subscribe, and let me know if there is a topic you think might make a good video.
Read more

MITRE ATT&CK Tactics Are Not Tactics

Image
Just what are "tactics"? Introduction MITRE ATT&CK  is a great resource, but something about it has bothered me since I first heard about it several years ago. It's a minor point, but I wanted to document it in case it confuses anyone else. The MITRE ATT&CK Design and Philosophy document from March 2020 says the following: At a high-level, ATT&CK is a behavioral model that consists of the following core components: • Tactics, denoting short-term, tactical adversary goals during an attack; • Techniques, describing the means by which adversaries achieve tactical goals; • Sub-techniques, describing more specific means by which adversaries achieve tactical goals at a lower level than techniques; and • Documented adversary usage of techniques, their procedures, and other metadata. My concern is with MITRE's definition of "tactics" as "short-term, tactical adversary goals during an attack," which is oddly recursive. The key word in the tacti...
Read more

New Book! The Best of TaoSecurity Blog, Volume 4

Image
  I've completed the TaoSecurity Blog book series . The new book is  The Best of TaoSecurity Blog, Volume 4: Beyond the Blog with Articles, Testimony, and Scholarship .  It's available now for Kindle , and I'm working on the print edition.  I'm running a 50% off promo on Volumes 1-3 on Kindle through midnight 20 April. Take advantage before the prices go back up. I described the new title thus: Go beyond TaoSecurity Blog with this new volume from author Richard Bejtlich. In the first three volumes of the series, Mr. Bejtlich selected and republished the very best entries from 18 years of writing and over 18 million blog views, along with commentaries and additional material.  In this title, Mr. Bejtlich collects material that has not been published elsewhere, including articles that are no longer available or are stored in assorted digital or physical archives. Volume 4 offers early white papers that Mr. Bejtlich wrote as a network defender, either for technica...
Read more