Friday, July 25, 2008

DNS and the Cyber TARDIS Problem

It's been 16 days since I responded to public notification of DNS problems in Thoughts on Latest Kaminsky DNS Issue, and 4 days since Halvar Flake's post On Dan's request for "no speculation please". Apparently the tubes are still working, since I presume you're reading this post via the Internet and not carrier pigeon. It's still been a remarkable period, characterized by the acronymn in the title of this post.

I'm not referring to the TARDIS of Doctor Who, although centrality of "Time" is the reason I used the TARDIS theme. I mean Time and Relative Data in Security. Time and Relative Data were the key issues in the DNS issue. Who knew more about the problem, and when? Halvar understood this in his post, when he estimated that a savvy attacker would need 1/4 the time of a normal security person to understand the nature of the DNS problem, given the same starting point.

Since Halvar's speculation, Matasano's confirmation, Metasploit's weaponization, and Dan's elaboration, there's been a flurry of offensive and defensive activity. It reminds me somewhat of Y2k: am I still able to use the Internet because DNS administrators have been patching, or are not enough bad guys trying to bother me? It would be nice to see some academics query whatever data (hint) they can find on recent DNS activity to produce some practical research, rather than trying to decipher five year old worm data or yet another port scan. According to this Arbor Networks Blog post by Jose Nazario, his group might have some data to share soon.

I'd like to highlight some of my favorite thoughts from the past few days. I liked FX's post Perception of Vulnerabilities:

The Kaminsky DNS attack is definitively regarded as the most important vulnerability this year. This, I find highly interesting , as we have seen two other gigantic security failures already in 2008. Debian's NRNG (non-random number generator) is most certainly one of them. But honestly, raise your hands if you have even noticed SNMPv3... SNMPv3 is used to manage routers - the routers that forward all your traffic around the world, including your DNS queries. Managing a router means being able to configure it; a.k.a. super user access. Attackers who can configure a router in your path can redirect everything, without you knowing, not just traffic that relies on name resolution.

The weaponization discussion has been great. On one side are people like Hoff and Rich Mogull, who believe the Metasploit team was wrong to weaponize the exploit. I place myself on the other side. I agree with a lot of Andre Gironda's argument in comments on Rich Mogull's post. I think it's important to be able to test if your DNS implementation is vulnerable, as noted by Ron Gula in But I patched our DNS servers ....

With the growing importance of the cloud, and the customer's increasingly reliance on software he/she doesn't control, are we to be satisifed with promises of applied patches, or even the effectiveness of said patches? If you always believe your vendor (i.e., you're naive), answer yes. If you trust but verify, answer no -- and start testing. Metasploit (exercised via a pre-existing, contractual agreement that permits such customer testing) is one way to see if your vendor really is as safe as it claims to be.

People who care about reality -- facts on the ground -- care about testing. Such people also care about monitoring. Prior to Halvar's speculation, probably the best place to try to figure out how to detect what "might" be coming was the Daily Dave mailing list. Since Halvar's post, there's been a lot of monitoring discussions on Emerging-Threats. Monitoring types have been trying to work around implementation challenges in popular tools like Snort, with alternatives like Bro getting more attention. Some historical articles on DNS intracies have helped people understand DNS better, now that we know exactly what to observe.

I believe the actions of the past week have been for the better. Sure, the bad guys have a tool now, but as Druid noted in the Metasplot blog:

I was personally aware of multiple exploits in various levels of development before, during, and after HD and I wrote ours, so we felt at this point publishing working exploit code was fair game.

Poke around for five minutes and you'll find other implementations of exploit code beyond Metasploit anyway, never mind the private ones.

Public speculation followed by weaponization has elevated the issue for those who had to produce "proof" in order to justify patching, as well as helping level the knowledge field. Those of you who object have got to understand this point: real bad guys always win in the Time and Relative Data arena. Their paid job is to find ways to exploit targets. They have the time and knowledge to identify vulnerabilities in DNS regardless of what Dan Kaminsky says or doesn't say. I know whole teams of people who avoid the most elite public conferences because they don't learn anything new.

Defensive-minded security person -- how do you spend your time? Are you like me, balancing operations, planning, meetings, family, and so on, across thousands of systems, with hundreds of classes of vulnerabilities, and nowhere near enough time or resources to mitigate them? Do you know as much about the latest attacks and defenses as the people who discover and exploit them, for a living? Probably not.

Even assuming such adversaries do not know about the DNS problem prior to Dan's disclosure, as soon as they acquire the scent that problems exist (and especially if patches are released), they point their collective noses at the newest victim and tear into it. Halvar's N/4 estimate was very conservative, although he recognized real bad guys probably work a lot faster than that.

I think Dave Aitel put it best:

The motto of the week is that you can't hint at bugs or people will just find them. Either full disclosure or no-disclosure wins, because there's no point doing anything else.

12 comments:

CG said...

good post Richard

Anonymous said...

> Defensive-minded security person -- how do you spend your time?

Honestly? Pursuing my hobbies. Doing anything else is a waste of time. We're either already owned, in which case I know what to do, or going to be owned, in which case I wait until we are and then research and fix us to become unowned.

All the time in the middle is spent working on my king-side attacks as black.

Oh, and I know enough and speak eloquently enough to basically be on salaried retainer.

Saad Kadhi said...

This is an excellent post Richard. It sums it all pretty much concisely. Thank you.

kurt wismer said...

"Either full disclosure or no-disclosure wins, because there's no point doing anything else."

i wonder if covert disclosure was included in that estimation...

tc said...

being a fan of full-disclosure in the first place, it's easy to point and gasp at the 'circus' but difficult to agree with no disclosure at all.

perhaps kurt says it best; let's not forget about covert disclosure. so, maybe dk had it about half-right. at least give vendors a three second head-start before crying havok and running like hell from the dogs of war...but only three seconds; let's not start coddling.

as always, excellent post brian..

tc said...

<-- egg on face.

i just hit "enter" while sitting here talking to my friend brian.

now he thinks he's in richard's league.
pardon me while i go bang my head against a wall.

excellent post....RICHARD
;)

Vivek Rajan said...

Regarding the SNMPv3 bug.

I guess the lesson for developers is that hardcoding could actually be good practice when it comes to digest lengths.

It is easy to get SNMPv3 wrong by applying the wrong view control policies. I have seen places where a SNMPv3 user could simply create a new SNMPv3 user with better rights because the VACM policy did not lock down the VACM tables correctly for the original user. The vendor manuals do not contain sufficient guidelines on how to do it right.

Really good read. Thanks Richard.

Andre Gironda said...

Wow... a lot of great quotes there. This is a well-put-together post. I really like your work here.

One thing I would like to add is that one possible TARDIS explanation is not only that intelligent adversaries have more time and resources -- it's also that they have less of one very important thing: Ethics.

For example, I can beg Paul Vixie over email, harass him on AIM (assuming he even uses it, which I'm sure he doesn't), etc -- but he's never going to tell me about a DNS security issue.

An unethical adversary could just hold him at gunpoint and make demands.

Dan Weber said...

Either full disclosure or no-disclosure wins, because there's no point doing anything else.

What does he mean by "no-disclosure." Were people supposed to just update their DNS servers because new patches were out?

For better or for worse, Dan Kaminsky got a lot of people to patch DNS who wouldn't have bothered otherwise (despite the efforts of some in the community to stifle him), and he did it without blowing the whole lid off of the tea-kettle right away.

LonerVamp said...

Amen on this post, Richard. Well-written and takes the words right out of my virtual mouth.

Changlinn said...

Dan did a good thing whatever way you look at it. He got a fair chunk of vulnerable servers patched in a timely manner. I still myself think their may have been attacks prior to his release of the info, as I was brought in on a case a few months ago and I could find no fault with the companies DNS, or their forwarders, packet capturing, watching tcp/ip sessions, nothing. Switching their forwarders fixed the problem permenantly.

dghnfgj said...
This comment has been removed by a blog administrator.