Friday, March 19, 2010

Time and Cost to Defend the Town


Recently I guest-blogged on the importance of learning how another person thinks. This week I had a chance to apply this lesson with a new decision maker. I learned that I need to develop a way for this executive to think about our security program. I discussed the situation with my wife and she suggested focusing on cost. I thought about this a little more and realized that was the right way to approach the problem.

Consider the following scenario. You're the mayor of a town. You need to decide how much of your budget to allocate to the fire department. To apply the most simplistic analysis to the problem, consider this scene. As mayor you give the fire chief a simple goal: "protect us from fires!" The fire chief asks you: "Mayor, on average, how fast do you want the fire department to respond to a fire?"

I am not an expert on fighting real fires, but let's think about a range of some possible answers.

  • Option 1. Instantly. Literally as soon as a fire is detected, fire fighters are on site. Assume this level of response produces the maximum level of containment and preservation of property value, on average.

  • Option 2. Within 15 minutes. Assume this level of response produces 75% containment and preservation of property value, on average.

  • Option 3. Within 30 minutes. 50% containment and preservation of property value, on average.

  • Option 4. Within 45 minutes. 25% containment and preservation of property value, on average.

  • Option 5. Within 60 minutes. It's too late. With this timing, the property value is destroyed.


As mayor you're likely to first reach for option 1. After all, you want to preserve property value. However, the fire chief says "maybe you should consider the following data."

  • Option 1 costs $64 million. Fire fighters are deployed at 16 locations.

  • Option 2 costs $32 million. Fire fighters are deployed at 8 locations.

  • Option 3 costs $16 million. Fire fighters are deployed at 4 locations.

  • Option 4 costs $8 million. Fire fighters are deployed at 2 locations.

  • Option 5 costs $4 million. Fire fighters are deployed at 1 location.


At this point you're starting to sweat. There has to be a way out of this situation! You decide that you can't afford option 1, or 2, or probably even 3. The recession is hitting your town hard. You ask the fire chief if there's a way to reduce the number of fires expected to occur, so that a smaller fire fighting force can react more quickly to fewer fires.

The fire chief switches from his fire fighter role to that of fire marshall. He says that is certainly possible, if the mayor wants to pick from one or more of the following options.

  • Rebuild dwellings using fire-resistant materials.

  • Inspect and rewire electrical systems, including aggressive, persistent monitoring for faults.

  • Deploy advanced fire, smoke, and related alarms everywhere.

  • Remove flammable materials from dwellings.

  • Educate citizens on fire hazards.

  • Ensure all citizens know how to contact the fire department, and have the means to do so efficiently and effectively.

  • Plus a dozen more options...


You are probably getting the hang of this scenario. At this point the mayor needs to know the cost of each of the fire resistant methods outlined above. Let's not forget one other element: the fire chief asks the police chief to inform the mayor of the arsonist threat, and describes how dedicating counter-threat activities can deter and detain adversaries who set dwellings ablaze.

At the end of the day, the fire chief is presenting options to the mayor, and it's up to the mayor to decide how fast do we want to be able to respond to the fires that will happen, for how much cost.

(I underline the "fires that will happen" because that is the reality of life. Disasters happen, so you have to plan for them.)

For me, this is the best way to approach this executive. The fire chief doesn't get to decide how much money to spend on the problem. That's the mayor's decision. The mayor needs to make a budget choice, preferably with the fire chief's input, and then let the fire chief make the best resource allocation to meet the time goals requested by the mayor.

For me, time and cost are the best levers we can move in digital security. I can measure detection and response time for the incidents we handle. I can track how much money I am spending to meet those time requirements. If the mayor wants faster response time, the mayor can try to reduce the number of fires via fire marshall programs and/or apply more resources to the fire fighters.

Beyond measuring incident detection and response for real intrusions, you can use red teaming/adversary simulation to create metrics. You can say "for the money currently spent on our security resistance program, it takes a Red Team X number of minutes to accomplish Goal X. Is that acceptable?" If X minutes is unacceptable, you can again present cost-benefit analysis in order to derive a decision.

If you think you've heard this line of reasoning before (outside this blog), please check whether the other advocates have emphasized outcomes as I do here and elsewhere. I'm not saying "spend $10 million to achieve 95% patch compliance." That's an input metric. I'm talking about output metrics against real intrusion activity and adversary simulations.

6 comments:

Anonymous said...

In reality, the mayor would go for Option 5 with a cost of Option 1, advertising they went for Option 2.

investment policy statement said...

i agree with this person talking above..in most cases it is option 5, government leaders say it's option 1...and ask people to pay taxes for the expenses for option 1..and that's how the people's money are being invested

Anonymous said...

There would be another option...Call it Option 6, no fire department at all. People in the town manage their own risk when it comes to fire. This costs zero dollars. If I (Mr. Home or Business owner) am worried about a fire I will buy a sprinkler system or hire 24/7 security to watch over the place.

Anonymous said...

Funny comment above from the first anonymous commenter. However, in my experience the conversation goes:

Mayor: "I want option #1..."

Fire Chief: "Great! We'll get started!"

Mayor:: "...but at the cost of #5 of course."

Fire Chief: "But we've been saying that for years and the fires continue to burn out of control."

Mayor: "Look, I saw an ad for the ThreatSlayer Firewall(TM) when I was reading [Forbes/Business Week/Fortune/The Robb Report] on the way to Singapore last week. They said they take care of this. Are you using this product?"

Fire Chief: "No, look, we really don't need that product, and in fact we have an open source version that is already more capable. We simply need more fire fighters."

Mayor: "Well there are lots of hungry people out there who would jump at the chance to be Fire Marshall for our town. If you're telling me you can't get it done..."

..or something along those lines.

Fortunately, when the Mayor actually sees the fire instead of just smelling the smoke, they tend to get it. And that's the only way I've ever seen this story end differently.

I agree that these options are critical though. At a minimum it forces the Fire Department to understand the tradeoffs, and helps prepare the team for the day the Mayor is ready.

Steve Pinkham said...

I think the analogy is a good one for incident response. However, in taking it further the fire department is different from most security programs in an important way: The primary goal of the fire department is to protect the humans involved. The secondary goal is to stop the fire from spreading to the neighbors' houses. The final goal is to try to salvage what they can from the burning house.

In security, we haven't really yet hit the stage of the public fire department that is concerned with the common good. We're still private enterprises installing fire doors and sprinkler systems to protect our own stuff. Yes, we do have CERT, NIST, and MITRE, but they are more like building code organizations then fire departments.

A large problem in creating a cyber fire department is that the risks are global, and attackers leverage the "neighbors" with the weakest laws and other defenses to launch their attacks.

Matthew Rosenquist said...

The method is viable only if the cost of a control can be combined with its effectiveness. The first set of options show this (ex. 30minustes response control, at 50% containment effectiveness, at a $16million cost)

The latter examples do not provide all pieces of the necessary equation (or they are extremely variable). “Educate citizens on fire hazards” cost how much at what effectiveness? Without those qualitative measures, the controls cannot be compared against other options. This is the primary problem with measuring information security.