Comments on SANS Top 20
You may have seen that the latest SANS Top 20 was released yesterday. You may also notice I am listed as one of several dozen "experts" (cough) who "helped create" the list. Based on last year's list, I thought I might join the development process for the latest Top 20. Maybe instead of complaining once the list was published, I could try to influence the process from inside?
First let me say that project lead Rohit Dhamankar did a good job considering the nature of the task. He even made a last-minute effort to solicit my feedback, and some of my comments altered the categories you now see in the Top 20. I thank him for that.
As far as the nature of the list goes, it's important to realize that it's based on a bunch of people's opinions. There is no analysis of past vulnerability trends or conclusions based on real data, like the Vulnerability Type Distribution I mentioned earlier. At the point where I realized people were just going to write up their thoughts on various problems (Internet Explorer, Mac OS X, etc.) I left the project. Rohit emailed me early this week, but I was formally done in early October.
If you think a bunch of people's opinions is worthwhile, then you may find the Top 20 useful. I think the majority of the Top 20's utility, such as it is, derives from name recognition. If that can help influence your organization's management, then I guess it is helpful.
At the very least, the newest Top 20 is a very informative document with plenty of references. I would expect most security practitioners to understand or at least recognize everything on the list. I don't think the list is as "actionable" as the original Top 10, which listed specific vulnerabilities (e.g., "RDS security hole in IIS," CVE-1999-1011) that you needed to patch now.
The latest Top 20 has hundreds of CVE entries, and as such is more of a meta-description of Internet targets. In that respect I like the fact it's called an "attack targets" document, since there's nothing inherently "vulnerable" about, say, Mac OS X. Instead, Mac OS X is being attacked.
What do you think of the new list?
First let me say that project lead Rohit Dhamankar did a good job considering the nature of the task. He even made a last-minute effort to solicit my feedback, and some of my comments altered the categories you now see in the Top 20. I thank him for that.
As far as the nature of the list goes, it's important to realize that it's based on a bunch of people's opinions. There is no analysis of past vulnerability trends or conclusions based on real data, like the Vulnerability Type Distribution I mentioned earlier. At the point where I realized people were just going to write up their thoughts on various problems (Internet Explorer, Mac OS X, etc.) I left the project. Rohit emailed me early this week, but I was formally done in early October.
If you think a bunch of people's opinions is worthwhile, then you may find the Top 20 useful. I think the majority of the Top 20's utility, such as it is, derives from name recognition. If that can help influence your organization's management, then I guess it is helpful.
At the very least, the newest Top 20 is a very informative document with plenty of references. I would expect most security practitioners to understand or at least recognize everything on the list. I don't think the list is as "actionable" as the original Top 10, which listed specific vulnerabilities (e.g., "RDS security hole in IIS," CVE-1999-1011) that you needed to patch now.
The latest Top 20 has hundreds of CVE entries, and as such is more of a meta-description of Internet targets. In that respect I like the fact it's called an "attack targets" document, since there's nothing inherently "vulnerable" about, say, Mac OS X. Instead, Mac OS X is being attacked.
What do you think of the new list?
Comments
That was useless, I see why you distanced yourself from it. Nothing actionable there, not even well written since there were so many weasel words.
I think this document just has a problem with its identity. Does it list actual distinct vulns like they did in the first one, or do they increasingly group things into buckets like "web applications" (cop out!) or instead change all of it and list targets? Do they do already-broken targets or theoretical ones? This latest one has a feeling that some of this is a prediction. Including Mac OS X is a huge disappointment. Yse, it might be an increasingly enticing target, but it is not a problem right now or in the past year. VoIP Phones/Servers had no place in this, especially when you see the glaring holes of wireless insecurities and/or insecurity of data at rest on laptops (theft).
I expect next year they have such a watered down and general top 20 that it is like, 1. Windows, 2. Unix, 3. Anything dealing with the web. And so on...
I really disliked #19 Users (phishing / spear phishing) and #20 Zero-Day Attacks and Prevention Strategies. Users should have just been replaced with either social engineering or phishing.
I really liked the new groupings. I think in the past couple iterations they have really struggled to match 10 Windows and 10 Unix items. Last year 85% of the content was on the 10 Windows vulns, and the Unix ones got very little space. Maybe that's because those Unix ones have been written about almost every year, and maybe it is just getting old. :)
But again, with any list like this, there will be problems and disagreements, which means the identity and purpose of this needs to be very clear.
Richard's post for TOP-20 2005.
And my opinion, which I posted there, it's still valid here.
Sincerely,
But isn't the point of the Top20 to be a consolidated list, designed mainly for the non-security practitioner? Perhaps it was originally, but as security has gotten to be a specialty and formal profession, the resources are widely known to those people; rather, having worked on the project, I believed it to be focused on issues that normal non-security IT people could use as a quick guide.
Do you think the list was more helpful when it was organized as the top 10 vulnerabilities?
I think the original top 10 list was more or less actionable. The current list is not actionable. So my DNS servers are attack targets. And...? Not much you can do about that.
Edward Ray
To support that claim, maybe I should mention CVSS (the Common Vulnerability Scoring System) being used by NIST (aka the US government's standards body) for their National Vulnerability Database (NVD). CVSS was drawn-up by the NIAC Vulnerability Disclosure Working Group. The NIAC VDWG is made up of all the leading experts for CERT, Cisco, eBay, ISS, Microsoft, Qualys, and Symantec.
Oh yeah, and Tenable (the company behind the most popular security scanner, Nessus) today announced a partnership with NIST NVD to use CVSS in all of their products (both Nessus and their Passive Vulnerabiility Scanner) and provide feedback on vulnerabilities.
What's funny is that CVSS isn't as complete as the threat-models that STRIDE/DREAD (see the Microsoft Press book, Threat-Modeling) and Trike (presented at Toorcon'05) provide.
According to a recent survey at Jeremiah Grossman's (WhiteHat Sec) blog, DREAD is more popular than both CVSS and Trike for web application security professionals.
CVSS is likely to become the dominant standard for vulnerability measurement with regards to public disclosure, while is likely to revolutionize the private disclosure industry if used properly and often.
Speaking of the NVD, there is a tool called provided by Purdue University that allows you to create profiles of vendors/products/keywords that daily (2x) update you via email on new vulnerabilities that match your criteria in both the NVD and Secunia vulnerability databases.
It's also my "opinion" to use the OSVDB project to query for vulnerabilities, as it's the most complete implementation I've seen. Reading their daily RSS should be avoided, as OSVDB wants to document every vulnerability, including ones in the past... so you could see PDP-11 exploits and assume it's current. However, reading their blog is recommended.