Tuesday, November 22, 2005

The Good and the Bad About the New SANS Top 20

Back in January I noted that SANS was not using the terms "threat" and "vulnerability" properly in its call for help on the "twenty most critical Internet security vulnerabilities," represented by the logo at left.

You will remember that a threat is a party with the capabilities and intentions to exploit a vulnerability in an asset. A vulnerability is a weakness in an asset that could lead to exploitation. An intruder (the threat) exploits a hole (the vulnerability) in Microsoft IIS to gain remote control of a Web server. In other words, threats exploit vulnerabilities.

Today, version 6 of the Top 20 was released. I'll start with "the good." I believe the majority of the 2005 content is much better than the 2004 edition. The 2004 list, and previous lists, displayed 10 Windows vulnerabilities and 10 (often dubious) Unix vulnerabilities. The 2005 list, in contrast, displays the following vulnerabilities:

Top Vulnerabilities in Windows Systems

* W1. Windows Services
* W2. Internet Explorer
* W3. Windows Libraries
* W4. Microsoft Office and Outlook Express
* W5. Windows Configuration Weaknesses

Top Vulnerabilities in Cross-Platform Applications

* C1. Backup Software
* C2. Anti-virus Software
* C3. PHP-based Applications
* C4. Database Software
* C5. File Sharing Applications
* C6. DNS Software
* C7. Media Players
* C8. Instant Messaging Applications
* C9. Mozilla and Firefox Browsers
* C10. Other Cross-platform Applications

Top Vulnerabilities in UNIX Systems

* U1. UNIX Configuration Weaknesses
* U2. Mac OS X

Top Vulnerabilities in Networking Products

* N1. Cisco IOS and non-IOS Products
* N2. Juniper, CheckPoint and Symantec Products
* N3. Cisco Devices Configuration Weaknesses

Bravo. I think that is a significant step towards realizing the scope of the problem at hand. To be fair to Microsoft, I believe there could have been "Unix services" and "Unix libraries" sections. I applaud the addition of network products and other applications. Content-wise, this is a great resource.

Now, "the bad." The top of the page has this link: -----Jump To Index of Top 20 Threats -----. For Pete's sake, the title of the document is "The Twenty Most Critical Internet Security Vulnerabilities." These are not threats.

Let's see other terms in use:

In the introduction we see:

"In addition to Windows and UNIX categories, we have also included Cross-Platform Applications and Networking Products. The change reflects the dynamic nature of the evolving threat landscape."

I can accept this use of the term threat, if the intent is to refer to parties who exploit vulnerabilities.


"We will update the list and the instructions as more critical threats and more current or convenient methods of protection are identified, and we welcome your input along the way."

Here, threats should be "vulnerabilities".

Section C2:

"Compromising a gateway could potentially cause a much larger impact since the gateway is the outer layer of protection and the only protection for some threats in many small organizations."

This should either replace "threats" with "vulnerabilities", or "for some" with "from some".

Section C5:

"The main threats arising from P2P software are:"

I think threats should be "risks" here, although the list is a muddle of different issues.

Later in that section:

"The number of threats using P2P, IM, IRC, and CIFS within Symantec's top 50 malicious code reports has increased by 39% over the previous six-month period."

Here, I can accept the use of the term as long as the intent is to describe parties abusing P2P, IM, etc.

Section C8:

"These applications provide an increasing security threat to an organization. The major threats are the following:"

Here's a simple rule of thumb: applications can never be "threats." Again, I suggest replacing the second "threats" here with "risks".

One final note: I am not a lone voice speaking on this subject. The Financial Times, of all people, is linked from the SANS page with a story Hackers pose new threat to desktop software. That's the proper use of the term threat, since a hacker is a "party."

Security will not be taken seriously as a "profession" until its "thought leaders" use basic terms properly.


John Ward said...

I think we need another amusing comic. I think this one should feature Spiderman =)

Anonymous said...

ok is there a proper and authentication definition for threat, vulnerability and risk in terms of network security because i see different definitions from different authors.

Anonymous said...

The problem is worse than Richard depicts it. SANS tagged all of OS X as a "Top Vulnerability in UNIX systems", indicating SANS has their own convoluted definition of "vulnerability" that has nothing to do with weaknesses in assets.

Or perhaps they feel less comfortable saying the top vulnerability in Windows is Windows?

Anonymous said...

For *most* people, does the usage of threat or vulnerability really detract from the value of this document?

John Ward said...

From a business standpoint, yes it does matter. The actual meaning may be derived by context in the article, but while it may seem like nitpicking, the improper use of terms "threat", "vulnerability", and "risk" lend themselves to the communication breakdown between IT staff and business parties. This is especially true in a global environment where all parties speak the same "language" by the loosest of definitions. This not only applies to non-North American English speakers, but managers who can barely communicate their intentions. Example: A manager goes around saying we need the newest and latest greatest IDS/IPS to protect ourselves from the newest threats. While more familiar staff may be able to read in between the lines, inexperience and language barriers will lead to other wondering “Is there a new vulnerability that we need to patched on our systems?” or “Are there new parties with the capabilities and intentions of attempting to attack our systems (definition of threat)?” Proper use of the terms enhances communication and prevents potential breakdown points in the communication process. Organizations like SANS, who in some circles are considered an authority, using the incorrect terms promote improper communication techniques in the security field, adding confusion to an already chaotic scene.

Like I said, we need another amusing comic to clarify these terms just like this one (http://taosecurity.blogspot.com/2003/10/dynamic-duo-discuss-digital-risk-ive.html)

GM said...

You need a word to describe "people-or-programs-that-are-trying-to-break-in-to-our-systems," and that really is different from "things-that-are-broken-in-our-software" when you're trying to educate management about what you're doing to protect assets and why they should pay for it.

If people say "threat" because they prefer a one-syllable word to the more correct "vulnerability," they can always say "bug." ;-)

Anonymous said...

But I bug does not necessarily create a vulnerability.

afshin lamei said...

It is very critical to use the terms "threat and vulnerability" in their own place to avoid misunderstandings.

Anonymous said...

The Top20 has become so generic as to be worthless. Does "W1. Windows Services" say anything to us other than that windows software is the single greatest source of windows vulnerabilities? It is so vague as to be meaningless and illogical.

In this iteration, a certain vendor managed to persuade the moderator to not list a number of critical issues by dismissing them as "insecure management practices" even though the "practices" were in fact critical flaws in the base product.

Richard Bejtlich said...

Far be it for me to defend SANS, but a couple of you seem to take exception with SANS because it lists "W1. Windows Services" and "U2. Mac OS X" as "vulnerabilities." The Windows Services item has plenty of specific vulnerabilities listed.

I agree that the Mac OS X listing, however, leaves much to be desired. Item U2 basically says "check your patches."

Arturo 'Buanzo' Busleiman said...

But, in the end, do you agree, in overall terms, that the top-20 2006 document is something a sysadmin (not a security expert), should read, or take into account when analyzing the security of the company he/she works for?

Come on, this is not for security experts that's pretty obvious, it's for admins that are not security experts or are just starting into in-depth security!

That's the purpose of the document, and for that purpose, it's great. It's not a white paper on OSX issues. Come on, man, at least put yourself into perspective.

Of course, you can say MY opinion shouldn't count, as I worked on sans's top-20 for the last three editions. :)