A Plea to the Worthies
You may have seen stories like Cybersecurity Experts Collaborate with subtitles like A think tank has tapped several heavyweight security experts to staff a commission that will advise the president. That story continues:The Center for Strategic and International Studies (CSIS) wants the commission to come up with a list of recommendations that the new president who takes office in January 2009 "can pick up and run with right away," said James Lewis, director of the CSIS Technology and Public Policy Program. The commission, made up of 32 cybersecurity experts, plans to finish its work by the end of 2008. I am fairly confident that nothing of value will come from this group, but there is one task which could completely reverse my opinion. Rather than wasting time on recommendations that will probably be ignored, how about taking a step in a direction that will have real impact: security metrics. That's right. Spend the first day (or two, if you are a slow reader or can't sit still for long periods) reading Andy Jaquith's book. Next, and this is the crucial part:
Figure out how to play and score the game before you pretend to think you can improve the score.
What does this mean? Just a few ideas include:
- Propose definitions for security, risk, threat, vulnerability, inside threat, external threat, and all the other words we use yet upon which we never agree. Hold hearings and invite real security people (not just digital security people) to express their views.
- Propose some metrics and see how other operations define success. Hold hearings on the results of that process.
- Apply metrics to some real organizations and gain a baseline set of numbers. Repeat the process at determined time intervals. Try to identify correlations and if possible causations. Be anonymous if necessary, but use a real methodology and not the self-selection applied by CSI/FBI and others.
Do you see where I am going here? At the end of the process we could have a framework for seeing just what is happening. I defy anyone to tell me just how bad or good our digital security situation is right now. Some say the sky is falling, others say we're happy! happy!, others say we're just as secure as we need to be to continue limping along. It is a proper role for a panel of worthies to help figure out how the game is played and then what the score is. It is a waste of time to make recommendations before those basic steps have been taken.
Comments
But then - perhaps paradoxically - what did I mean when I say "we certainly need definitions..." at the start of this comment? Every time I come across a definition, I try to be careful to appreciate the context in which its used, who is it used by and so on.
http://www.fcw.com/online/news/150647-1.html
My reaction was very similar to yours. The probability that meaningful contributions will be manifested from this committee is slim. Optimistic scenario: work that is already done is repeated. Pessimistic scenario: more (potentially conflicting) compliance requirements and legislation. :(
This quote,
"Langevin said. 'I expect the recommendations will be a solid document that we can rely on to better secure our networks.'"
What is does "better" mean? Certainly we know how to "do more security", typically with limited real impact.
I think that we are defined by what we don't know. It is like the "badness meter" that MJR refers to.
We might be bad/good, and we know we want to get "better" until we are "good enough." (Whatever all that means).
Your post, here:
http://taosecurity.blogspot.com/2007/10/are-you-secure-prove-it.html
Makes more sense than is likely to come from this committee.
I was writing a response, but it was so long I made it into a full blog post. =)
http://www.guerilla-ciso.com/archives/288