Thor vs Clown
It started with this post by M.D.Mufambisi to the pen-list list:Im designing an SMS baking application but i need to research on the security risks involved first... What are the risks around this application? How are such applications normally subverted? Are there any case studies someone can point me to?
After a few responses, Craig Wright chimed in:
The solution needs to be based on risk.
Where a system uses an SMS response with a separate system (such as a web page), the probability that the banking user is compromised and a fraud is committed, P(Compromise), can be calculated as:
P(Compromise) = P(C.SMS) x P(C.PIN)
Where: P(C.SMS) is the probability of compromising the SMS function and P(C.PIN) is the compromise of the user authentication method
Craig followed up with a blog post:
Many people feel that it is not feasible to model risk quantitatively. This of course is blatantly false. In the past, many of the calculations have been computationally infeasible at worst and economically costly at best. This has changed. The large volumes of computational power that is available coupled with novel stochastic methods has resulted in an efficiently viable means of calculating risk quantitatively with a high degree of accuracy. This can be measured as a function of time (as survival time), finance (or monetary value) or any number of other processes...
Tim Mullen, a guy who I first met in 2002 teaching at Black Hat, responded on full-disclosure:
I'm looping in the FD list because often my replies don't make it to Pen-Test, and this has hit a nerve with me.
I've looked over your post...
Once I was able to get past the overwhelming egoism and self-substantiating claims of your contributions to the industry, I arrived at the conclusion that the only portion of the aforementioned page that is not complete drivel and even laughable to anyone who has actually worked towards ascertaining actual risk in production environments, is where you describe your own words as "ravings..."
I'm fine with you sitting back and gloating about the Security Hero award you got from Northcutt, but when I see that you are actually contributing to ANY level of Critical Infrastructure Protection, it makes me fear for anyone who might be counting on your presumed skillset to actually make intelligent decisions about risk where human safety is at stake.
Your "risk formula" is ridiculous. What number would your formula have yielded 2 weeks before SQL Slammer was released? Where is the variable for unpatched systems? What number do we plug in for malicious employee factorization? More importantly, where is the calculation for self absorbed snake-oil selling academics with no real experience using their calculator to come up with magic numbers that represent the risk of a nuclear power plant being hacked?
Since you are (self-described) as "currently the only GIAC GSE (Compliance) holder globally and the most highly accredited Global Information Security Professional" and thus (presumably, if only in your mind) the greatest security mind in the world, how about accepting a challenge to an open debate on the subject at Defcon? People like you are dangerous and need to be exposed before someone in a position of power actually believes that you know what you are talking about. Bring your abacus.
Craig then responded with some sort of monetary challenge, and Tim and Craig are now debating how to arrange that.
If you want history on why I consider model = clown, please check out the posts on my clown tag.
When I read
"In the past, many of the calculations have been computationally infeasible at worst and economically costly at best. This has changed. The large volumes of computational power that is available coupled with novel stochastic methods has resulted in an efficiently viable means of calculating risk quantitatively with a high degree of accuracy."
it is clear to me Craig is pretty well disconnected from reality. Did we not just suffer a global recession exacerbated by clowns who thought they could model risk "with a high degree of accuracy"?
Comments
Given a large enough set of empirical data I don't think that quantitative measures of risk are impossible to achieve. Insurance companies use quantitative measures of risk related to cybercrime. Are they accurate? I'm skeptical.
Craig Wright is a smart guy and I am interested in hearing how he calculates risk. Skepticism is merited given the lack of successful quantitative risk methodologies that exist. But I don't think the personal venomous attacks are merited until he has a chance to present his case.
1. Those who have never modelled anything critizing models is just as funny as those who have never actually done any real security work thinking models solve problems.
2. Don't blame the models. Your everyday things, like the car you use, bridge you cross, the building you work at, are modelled first then built. The right person with the appropriate knowledge can model things well enough to avoid trial-and-error.
3. The recession wasn't caused by models. It was caused by those ego and greed. They never factored those into the formula. A tool is only as good as it's user. People still make money modeling markets a ton of money. When it comes down to it these are in fact simple trend models of past, present, and future. That's why they need high frequency trading.
I didn't have to be a 1890's aeronautical engineer to recognize that none of the models worked and many of them merited criticism. (http://www.youtube.com/watch?v=kEdtvct6Tf0) So I think criticism of models is fair if models have failed to work effectively in the past.
Maybe the next Wright brothers first name is Craig. We will see.
At this point, the debate is largely about his self-importance and blowing smoke with misleadingly precise calculations, rather than the core concept of quantitative risk estimation. That's a shame, because the questions surrounding that core concept deserve further research.
Funny, I built models in undergrad, during a summer program during undergrad, and in graduate school. We did silly exercises like "how many non-white South Africans need to graduate from school each year in order to change the racial mix of the government's civil service by 2000, 2010, etc." or "how many Marines did it take to overwhelm Japanese positions in the South Pacific," and so on. Those and related exercises taught me the futility of applying those approaches to information security "risk."
I think it's important to point out that the things you reference are modeled against the laws of physics, aerodynamics, etc... Given buildings for example, you could model for weather variances using statistical averages over long periods to allow for things like snow loads. I'm fairly confident that the same model does not make allowances for the risk of someone driving a truck full of explosives into the same building.
I for one would argue that attempting to model risk in search of an absolute, quantitative value is an exercise in futility, bordering on delusional.
Guys, I am sure he meant it in jest. Nobody will suggest such hilarity serious, now, will they?
You should check the tag, maybe his post is also labeled "humor."
If not....well... most asylums would admit him, I am sure. For others, he just might be too insane.
Mostly agree, but I think the blame does not all go to models themselves. Everyone who deals with complexity has a model. The problem as you imply is the assumption that the model contains a "high degree of accuracy", also called "false precision" The idea that the model is reality is the dangerous one and the gaps in assurance and security are what we live with when this is the case, so you get the problems that swamp your whole model, as Charlie Munger says - "when you mix raisins and turds, you still have turds"
Or as Warren Buffett said (before the aforementioned false precision induced financial crisis) - "advocates of efficient market theory assume that the market is always efficient instead of mostly efficient. The difference between these two is night and day."
I feel the information security domain often finds itself falling victim to physics envy, modeling solutions with equations not logically derived from valid and peer-reviewed theories, which obscure underlying subjective measures. For an example of this as applied to economics, as Richard suggests in his post, I recommend you read this Business Week article and consider for yourself the parallels to our field.
I agree with the poster above about style being important. Myself and another friend in the infosec community have run into Craig on mailing lists before. He comes across as someone who likes to chest beat about how amazing he is, and seems more interested in "wowing" people with his amazing experience than genuinely helping people who pose questions.
It's a small community. He doesn't seem to be making many friends.
@Richard - agreed that modeling risk is dangerous. It doesn't take a long look at GRC to cause me to weep softly.
@Raymond - I would argue that modeling risk to precision is stupid, However, I would also argue that ignoring risk modeling is impossible. We're all going to get up in the morning tomorrow, go to our jobs, and model risk. The only question is with what rigor and bias.
For me, the last 5 years has yielded the following - neither "side" of the flame war has a leg to stand. Current IRM models are impractical, but yet suggesting that you *really* hack *real* mobile phones is quite a bore to management who sees no history of incidents and cannot create for themselves an adequate picture of impact. Their little Bayesian-like mental processes fold in upon themselves because they have no reference.
Like the APT, risk is real. Like how Richard (or I) feel when we see people denying the reality of APT, I think people who deny the reality of risk-based decisions to be plain silly. And like much of the noise around APT, the noise around "risk management" detracts us from finding real solutions to the real problem.
Something Wonderful....
1) What's so "advanced" about it. Techniques used weren't all that high tech now, they exploited extremely old software (IE6). Nothing advanced about that considering if you're still using IE6, you seriously deserved to be compromised.
2) Aren't all attacks persistent to a degree.
Shouldn't all "malicious hacker attacks" be viewed as "advanced persistent threats", they certainly don't go away and they're certainly more advanced than the admins of those systems. Otherwise, the systems would have never gotten compromised.
Maybe Richard could re-do Extrusion Detection volume 2 and expound on the use of Squid and Snort to the nth degree.
Most of what I read doesn't make me cringe at the thought of anything uber coming out of China. In fact the opposite, makes me think of the stupidity in "security managers" and their bosses who base risk on insanely kooky models. "Still using IE6? WTH?!" How STUPID are we in the US, not how smart or motivated INSERT_GOV_SPONSORED_HACKERS_ARE_HERE
From all I've seen in writing out of "Operation Aurora" doesn't make me think of anything via way of "advanced" more like "obscure." The attacks and the wording used by certain companies describing the attacks, lead me to also believe something doesn't smell kosher and that something isn't coming from the attackers but from those in the security industry spreading half-truths in order to gain market share. Hence the introduction of "Dynamically Unique Metrics Based Analysis for Secure Systems" (DUMBASS)
http://www.theaeonsolution.com/security/?p=231
Hey whatever works for companies. I still don't see how some managers don't get it when it comes to security. AV * EF = JO/KE (no - seriously). I'm just glad I can fall back on my own metrics. RMBSS - Without a shadow of doubt!
http://infiltrated.net/rmbss.html
Paid attention to... Thor vs. Craig? I believe Craig was spot on so please school... Pay attention to what?
I should be trolling elsewhere, but Verizon blocked 4Chan... ;)
It has been detailed that the attacks seen as performed by APT's often start with lowest hanging fruit, and more basic malware. Until detection - at which point they alter their methods or malware to more 'advanced' ones.
The 'persistance' is not about the attacks, but about the methodolgy used to remain in the compromised network - different to snatch and grab, or a regular trojan.
I think the bigger problem is we ALL have seem to have to give names and methods for everything also making every one think it's the best thing since sliced bread...Models, etc. and things such as APT...
APT, AFAICT, is what was called a "targeted attack" in the 70-80s when hackers would infiltrate a network/system and not immediately "show" their hand. Actually have knowledgeable (no script kiddies/metasploit auto hacks)attack vectors in mind before going in to a network because they (or individual)were skilled and had a clear agenda. Not really sure why someone would have to come up with a new catch phrase like APT (unless they want to create buzz for a class they taught to get more students to sign up ;-) ). But to be fair it happens all the time in IT (and maybe in other fields who knows); Cloud Computing vs Clusters? (yes I understand there are slight differences eg. one uses web services/over internet... but really did we HAVE to have another name for it???)
...just my $.02 No flame intended for anyone :-)
http://en.wikipedia.org/wiki/Stochastic
"In mathematics, specifically in probability theory, the field of stochastic processes has been a major area of research."
The probability of a remote exploit affecting SMS at some time is very close to 1.world,,
In other words, its virtually certain that in the real, a system compromise will occur at some point in a reasonable time period - say 1-5 years.
On this basis, protecting the assets and having compromise mitigation in place reduce the impact of compromise.
We forget that physics is a statistical estimate and not the precise set of equations that we seem to come to from high school. That statistics can provide good models as long as the underlying foundations are sound.
It seems to be overlooked that legislation made it a requirement that banks and others offer loans to those who could not be expected to repay them. Banks of course jumped on this as they had a government guarantee.
http://gse-compliance.blogspot.com/2011/10/preamble-into-aligning-systems.html