"NIP is not a replacement for firewalls and won't be in the foreseeable future. Why? The fundamental problem is false positives -- the potential to block legitimate traffic. Before you can prevent attacks, you have to detect them, but NIP systems rely on intrusion detection, which is hardly an exact science. A properly configured firewall will allow in only the traffic you want, and you can bet the farm on that. We need to feel this same confidence in IDSs before we can believe in NIP systems, but IDS vendors have employed lots of talented brain cells trying to raise detection accuracy, and they're nowhere close to 100 percent." (emphasis added)
Exactly! How is a firewall doing intrusion detection any better than a non-firewall doing intrusion detection?
Mike continues to raise the clue bar with these insights from NIP Attacks in the Bud:
"Network Associates doesn't let users see what constitutes a signature. When we asked about this, the company said it didn't want to help people develop evasion techniques. The Exploit Alert Detail dialog on the Alert Viewer reveals text matches for a given alert, but that one match could be a subset of all possible matches.
Given time, we could have puzzled out most of the signatures via exhaustive searches, so we think Network Associates is just being difficult. In comparison, NetScreen opens signatures for review and editing--an approach we prefer.
The lack of signature information quickly became frustrating, and it complicated troubleshooting when a match was based on a protocol anomaly because there wasn't enough information to know why a match occurred. We had to send packet traces to Network Associates to determine why an SNMP packet was being detected as a NetBIOS issue. It took a few days, but the company resolved the problem and provided an update to the signatures. Signature updates are automated, but you need to buy a support contract to get them." (emphasis added)
This is exactly the problem with many commercial IDS tools. If an analyst can't independently assess why the IDS generated an alert, she will not trust it and will disregard its warnings. Unfortunately, NWC still gave the NAI product its recommendation.
Incidentally, if you read the article in paper or .pdf check out Mike's new hair-do. Holy flowing mane, Batman! I wish I could manage that. :)