Thursday, February 05, 2009

More on Weaknesses of Models

I read the following in the Economist;

Edmund Phelps, who won the Nobel prize for economics in 2006, is highly critical of today’s financial services.

"Risk-assessment and risk-management models were never well founded," he says. "There was a mystique to the idea that market participants knew the price to put on this or that risk.

But it is impossible to imagine that such a complex system could be understood in such detail and with such amazing correctness... the requirements for information... have gone beyond our abilities to gather it."


This is absolutely the problem I mentioned in Are the Questions Sound? and Wall Street Clowns and Their Models. Phelps could easily be describing information security models.


Richard Bejtlich is teaching new classes in DC and Europe in 2009. Register by 1 Jan and 1 Feb, respectively, for the best rates.

5 comments:

Anonymous said...

I kind of like models, especially ones that deal with probability and risk.

But it's important to remember that a decision doesn't come from models - it's made by a person.

In the current crisis, perhaps the models failed, perhaps they did not.

Perhaps the decision makers who used the models failed, perhaps they did not.

Some people I know have advanced the view that those who were in a position to profit, understanding the immense risks at hand, decided to pocket the huge rewards and pass the risks on to others, like you and me.

One of the things I enjoyed the most about Taleb's Black Swan book was his assertion that understanding the past is just as difficult as predicting the future.

The chains of causality that we attribute to historical events may explain what happened, or may be just pleasing narratives, written on wisps of air.

Patrick Florer
Dallas, Texas

Brian Cox said...

Great post. I found it very useful. However, upon trying to get the pkg from BSD i found it hd been removed?!?!

Any guess why?

By the way I have tried multiple time but keep finding that sguil-server installs without the event table in mysql. Can you explain this and or point me to the script that creates the tables and the command syntax to import the table layout?

jbmoore said...

Models and risk management models are tools that are supposed to help inform decision makers about possible trends. Goldman Sachs used their risk management tools and models properly and avoided most of the mess the other investment banks found themsleves in. It was a management failure more than anything, otherwise most of the investment banks wouldn't be insolvent now. The New York Times had a write up about it. When one assumes that one's model is absolutely correct and mirrors reality perfectly, then one is likely in for a shock when reality destroys the model at some point.

Anonymous said...

Richard - I am curious what you would do in the place of using a model?

Pete Lindstrom

Michael said...

As I'm doing some info security modeling myself, I'm curious as to what you mean by "information security models?"