Brief Thoughts on WEIS 2010
Last month I attended my first Workshop on the Economics of Information Security (WEIS 2010) at Harvard. It was cool to visit and it reminded me that I probably spent too much time playing ice hockey and learning martial arts during graduate school, and not enough time taking advantage of the "Hah-vahd experience." Oh well, as Mr Shaw said, "Youth is wasted on the young."So what about WEIS? I attended because of the "big brains" in the audience. Seriously, how often do you get Dan Geer, Ross Anderson, Whit Diffie, Bruce Schneier, Hal Varian, etc., in the same room? I should have taken a picture. Dumb security groupie.
I'll share a few thoughts.
- Tracey Vispoli from Chubb Insurance spoke about cyber insurance. Wow, what an interesting perspective. She said the industry has "no expected loss data" and "no financial impact data." Put that in your pipe and smoke it, Annualized Loss Expectancy (ALE) fans! So how does Chubb price risk without any data, in order to sell polcies? Easy -- price them high and see what happens. This is what the industry did when legislators started creating laws on employment discrimination. Companies wanted insurance, so the industry made them pay through the nose. Later, to compete, insurers dropped rates -- but too low. When they started losing money they jacked up the rates again. Eventually insurers have some data, but only after years of offering a service in the marketplace. That floored me but it makes sense now.
- Again on insurance, Tracey said the industry insures for incidents whose impact can be concretely and quickly measured. What does that mean? Insurance against economic espionage, national security incidents, and related events is unlikely because you can't really measure the impact, at least in the short term!
- After spending two days with academics, I'd like to add to Allan Schiffman's famous phrase "Amateurs study cryptography; professionals study economics":
Amateurs study cryptography; professionals study economics. Operators work in the real world.
Seriously, I think economics will help mitigate many security problems, but some researchers need to visit living, breathing enterprise environments before publishing papers. I won't name names, but if you're writing a paper that relies on raw IDS alerts to measure "attacks on open source software," you need to spend some time in a SOC or CIRT to see what analysts think of that kind of "evidence." - It seems researchers have a suit of academic tools (math, statistics, functions, models, game theory, simulations, previous research, etc.) and they look for data to which they can apply those tools. They formulate a hypothesis, and at that point the applicability of the approach is probably out the window. Very quickly in several talks I noticed that the topic at hand was implementation of an analytical technique, with the underlying problem somewhere several slides back. This seemed a little weird, but it makes sense in the context of researchers doing what they know how to do -- identify an issue, develop a hypothesis, collect data, etc.
Overall I found the experience very interesting, but I'm not sure if I will try to return next year.
Comments
Yes, Tracey Vispoli's keynote talk did (unintentionally) expose the sorry state of the current insurance industry engagement with information security. Another symptom is that we have almost no regular participation at WEIS from the insurance industry, either in research or attendence.
I believe that their back-assward approach to the market is mostly due to a poor business model, and not due to fundamental problems with risk modeling. Basically, they treat cyber insurances as if it was just another type of liability insurance (e.g. industrial safety, automobile) or corporate insurance ("errors and omissions"). The are also trying to sell traditional mass-market risk transfer products to traditional corporate insurance buyers. It think their whole mindset is mistaken and their approach is doomed to fail, commercially, no matter how much data they get. Corporate risk transfer is not the main value-creating opportunity.
Also, the sorry state of the cyber insurance industry says nothing about prospect for risk management methods in general. Likewise, cyber insurance doesn't depend on simplistic ALE formulas. It would be like reducing all of information security to"anti-virus" and "firewalls". It just ain't so.
At WEIS 2009, there several important presentations and discussions on how to get better engagement between academics and industry. The organizers of WEIS 2010 decided to take a different approach, so they included more time slots for papers, and less time for discussion.
As to why there isn't more collaboration and productive engagement between industry and academia on InfoSec economics research, the reasons are complex and there is plenty of responsibility on both sides. One of the biggest obsticles is the length of time it takes to get such a project approved and started. The process can take months and months. Nobody on either side has the time or money or incentives to invest in this "orchestration phase". Likewise, government sponsors seem unable to find ways to facilitate these collaborative processes.
To be more specific, I have approached many individuals at many companies, and asked them what it would take to get them, personally, and their company to participate in a collaborative research project on InfoSec economics. Nobody has been willing or able to step up and make any commitments. They don't even know who to go to in their company to get resources or approval. Everyone is afraid of legal implications, even before talking to their corporate lawyers.
Likewise, finding the right academic research partners is a non-trival task. Funding is a huge issue, along with funding cycles and career advancement. Many researchers do InfoSec economic research "on the side", because their *main work* has to advance their career. (This is interdisciplinary work, and mostly that does *not* advance your career as an academic.)
Regarding your phrase: "Operators work in the real world" -- who could argue with that. But all the "operators" who've been working in the "real world" since the beginning of time have not solved the fundamental problems of InfoSec economics. The core problem with being only focused on "the real world" is that you only have incentives to solve the immedate problems at hand, and only to an acceptable level. There is no incentive, time, or resources to deal with the general case, and especially the values and needs of all stakeholders, not just the people paying your salary.
I'm all in favor of bringing in "real world" problems, experience, and insights into the research and innovation process. But it would be a serious mistake to rely solely on real-world practitioners.
-------
Finally, as someone who has attended both academic and industry conferences in many fields, I can say that all of them have biases, blind spots, and limitations. What's sad is when some in one of these domains casts aspersions on the others because they are lacking in some way. THEY ARE ALL LACKING in some ways, so let's stop focusing on what's wrong with this conference or that, with this field or that, and instead find ways to build productive bridges so that we are collectively smarter and better off.
Your comment, "The core problem with being only focused on "the real world" is that you only have incentives to solve the immediate problems at hand, and only to an acceptable level. There is no incentive, time, or resources to deal with the general case, and especially the values and needs of all stakeholders, not just the people paying your salary."
Really is at the core of the problem, which is IMHO, fundamentally a research problem which will require interdisciplinary teams of scientists.
There is so much evidence that people, in their personal or professional endeavors, simply don't see the incentive to pay for security. This is the reason why "metrics" and "RoI on security" etc. has(is?) dominated the research funding/attention for so long. As we continue to debate on how to govern cybersapce the research presented at WEIS is increasingly pertinent, and is the only source of it's kind. Glad to have seen you there!
That conference is not by nor for operators. This gap SERIOUSLY affects the scope of some of the research...resulting in mind bending displays of analytics, crunching irrelevant data. I had the same complaint. I'll even take it a step further and raise an alarm of the danger that these "results" be be used to "inform" legislation which can literally change the course of our economy and the furture of the WWW...I hope there are guys like you around who can point out, from an expert practioners perspective, the (ir)relevance of what is presented. Or you can sit back and leave that up to the lobbyists...
Do you want to help build the bridge b/w the ivory tower and the workshop?
There is another venue to do just that.
BTW, I too tweeted about standing in a circle with Geer, Varian, Odlyzko, and Schneier and chatting all at the same time...pretty cool.
I look forward to your response.