More on Weaknesses of Models

I read the following in the Economist;

Edmund Phelps, who won the Nobel prize for economics in 2006, is highly critical of today’s financial services.

"Risk-assessment and risk-management models were never well founded," he says. "There was a mystique to the idea that market participants knew the price to put on this or that risk.

But it is impossible to imagine that such a complex system could be understood in such detail and with such amazing correctness... the requirements for information... have gone beyond our abilities to gather it."

This is absolutely the problem I mentioned in Are the Questions Sound? and Wall Street Clowns and Their Models. Phelps could easily be describing information security models.

Richard Bejtlich is teaching new classes in DC and Europe in 2009. Register by 1 Jan and 1 Feb, respectively, for the best rates.

Comments

Anonymous said…
I kind of like models, especially ones that deal with probability and risk.

But it's important to remember that a decision doesn't come from models - it's made by a person.

In the current crisis, perhaps the models failed, perhaps they did not.

Perhaps the decision makers who used the models failed, perhaps they did not.

Some people I know have advanced the view that those who were in a position to profit, understanding the immense risks at hand, decided to pocket the huge rewards and pass the risks on to others, like you and me.

One of the things I enjoyed the most about Taleb's Black Swan book was his assertion that understanding the past is just as difficult as predicting the future.

The chains of causality that we attribute to historical events may explain what happened, or may be just pleasing narratives, written on wisps of air.

Patrick Florer
Dallas, Texas
1:01 PM
Brian said…
Great post. I found it very useful. However, upon trying to get the pkg from BSD i found it hd been removed?!?!

Any guess why?

By the way I have tried multiple time but keep finding that sguil-server installs without the event table in mysql. Can you explain this and or point me to the script that creates the tables and the command syntax to import the table layout?
7:39 PM
jbmoore said…
Models and risk management models are tools that are supposed to help inform decision makers about possible trends. Goldman Sachs used their risk management tools and models properly and avoided most of the mess the other investment banks found themsleves in. It was a management failure more than anything, otherwise most of the investment banks wouldn't be insolvent now. The New York Times had a write up about it. When one assumes that one's model is absolutely correct and mirrors reality perfectly, then one is likely in for a shock when reality destroys the model at some point.
10:06 AM
Anonymous said…
Richard - I am curious what you would do in the place of using a model?

Pete Lindstrom
9:51 PM
Anonymous said…
As I'm doing some info security modeling myself, I'm curious as to what you mean by "information security models?"
12:18 PM
Post a Comment

Popular posts from this blog

Zeek in Action Videos

Image
This is a quick note to point blog readers to my Zeek in Action YouTube video series for the Zeek network security monitoring project .  Each video addresses a topic that I think might be of interest to people trying to understand their network using Zeek and adjacent tools and approaches, like Suricata, Wireshark, and so on.  I am especially pleased with Video 6 on monitoring wireless networks . It took me several weeks to research material for this video. I had to buy new hardware and experiment with a Linux distro that I had not used before -- Parrot .  Please like and subscribe, and let me know if there is a topic you think might make a good video.
Read more

MITRE ATT&CK Tactics Are Not Tactics

Image
Just what are "tactics"? Introduction MITRE ATT&CK  is a great resource, but something about it has bothered me since I first heard about it several years ago. It's a minor point, but I wanted to document it in case it confuses anyone else. The MITRE ATT&CK Design and Philosophy document from March 2020 says the following: At a high-level, ATT&CK is a behavioral model that consists of the following core components: • Tactics, denoting short-term, tactical adversary goals during an attack; • Techniques, describing the means by which adversaries achieve tactical goals; • Sub-techniques, describing more specific means by which adversaries achieve tactical goals at a lower level than techniques; and • Documented adversary usage of techniques, their procedures, and other metadata. My concern is with MITRE's definition of "tactics" as "short-term, tactical adversary goals during an attack," which is oddly recursive. The key word in the tacti...
Read more

New Book! The Best of TaoSecurity Blog, Volume 4

Image
  I've completed the TaoSecurity Blog book series . The new book is  The Best of TaoSecurity Blog, Volume 4: Beyond the Blog with Articles, Testimony, and Scholarship .  It's available now for Kindle , and I'm working on the print edition.  I'm running a 50% off promo on Volumes 1-3 on Kindle through midnight 20 April. Take advantage before the prices go back up. I described the new title thus: Go beyond TaoSecurity Blog with this new volume from author Richard Bejtlich. In the first three volumes of the series, Mr. Bejtlich selected and republished the very best entries from 18 years of writing and over 18 million blog views, along with commentaries and additional material.  In this title, Mr. Bejtlich collects material that has not been published elsewhere, including articles that are no longer available or are stored in assorted digital or physical archives. Volume 4 offers early white papers that Mr. Bejtlich wrote as a network defender, either for technica...
Read more