Tuesday, September 19, 2006

SANS Network IPS Testing Webcast

I'm listening to a SANS Webcast on Trustworthy IPS Testing and Certification. Jack Walsh from the Network Intrusion Prevention section of ICSA Labs spoke for about 45 minutes on his testing system. Jack spent a decent amount of time discussing the Network IPS Corporate Certification Testing Criteria (.pdf) and vulnerabilities set (.xls). The vulnerabilities set was just updated a week ago, after being criticized in July.

At present only three products are ICSA Labs certified, according to the ICSA Web site and this press release. ICSA Lab certification is a pass/fail endeavor; there are no grades.

ICSA does not release the name of the companies whose products fail. Looking at the members of the NIPS Product Developers Consortium, you can make some guesses about who participated.

Vendors pay for testing. They do so by paying for a year-long testing period, during which time they will receive at least one "full battery" of testing. Tests are rerun when the vulnerability set is updated or when then attacks used to exploit vulnerabilities change. Although ICSA Labs publishes the vulnerabilities they test, they do not say specifically how they exploit the vulnerabilities. Jack said they do use Metasploit, Core Impact, and home-grown programs. ICSA Labs relies on running real captured network traffic through a NIPS, during which they inject captured attack traffic.

I found the Webcast informative. I was surprised that Jack was so insistent that NIPS provide "mitigation" for denial of service attacks. I don't consider that an essential element of NIPS activity.

Looking at the vulnerability set, they appear to be dominated by "traditional" vulnerabilities, namely weaknesses in services running on servers. You will not see application-layer vulnerabilities like cross-site scripting, for example.

A competitor to ICSA Labs is NSS, who just announced their NSS Group IPS Testing Methodology V4.0 (060731) (.pdf) and a Certified IPS Products list.

5 comments:

Chris Harrington said...

ICSA does not release the name of the companies whose products fail. Looking at the members of the NIPS Product Developers Consortium, you can make some guesses about who participated.

This doesnt mean that all of the consortium vendors failed except for those two. Not all of the consortium members submitted for testing during the initial round. There was a limit imposed by ICSA as to how many companies could participate in a testing round, I *think* it was 10.

--Chris

Richard Bejtlich said...

Chris, that is true. I did not mean to imply that all of the others failed.

Ronny Vaningh said...

Richard

I do agree with Jack that the IPS should able to defend against rate based/dos attacks.

According to me they are idealy placed in the traffic flow to protect against those attacks.
I don't see a lot of other devices in a typical environment which are able geared towards this.

Andy Salo said...

I am from one of the vendors tested. There was no limit to the number of vendors that could submit their product for the first round. Only 10 chose to do so.

JimmytheGeek said...

The 'no false positives after tuning' criteria seems naive to me. Is the cost increased false negatives? Or is there some magic security dust that devines intent in a malformed image file?