Reader Questions on Network Security Monitoring

A regular blog reader and Network Security Monitoring practitioner sent me these questions last month, so I'd like to answer them here.

1. Are all alert data created equal?

This question originates with my employment at an MSSP where we process many types of alert data from Dragon IDS, Cisco IPS and ISS. Snort and Sourcefire strangely are underrepresented. My question is if Dragon IDS, Cisco IPS, ISS, Snort and Sourcefire all looked at the same full-content data, would they all produce the same results? I think not and would like to empirically verify this theory.

Testing detection systems is a complicated topic. I am not sure what methodology a place like NSS Labs uses. I bet they get varying results depending on the product. If you read A Tool for Offline and Live Testing of Evasion Resilience. in Network Intrusion Detection Systems you will see big differences between Snort and Bro, for example.

2. When is an analyst no longer an analyzer of data but an analyzer of dashboards?

This question also originates with my employment at an MSSP because like I said, we process so many disparate alert types and there is only so much time in a shift that it is challenging for an analyst to really spend quality time with a piece of data and conclusively determine what happened. Therefore the analysts evolve into analyzers of dashboards instead of data in order to promptly assess alerts and determine if there was a legitimate attack or not.

I would say you are an analyzer of dashboards when you cannot do the following:

  • Determine how a product generated an indicator

  • See the underlying activity that produced the indicator, whether it is network traffic or raw log messages

  • Research activity for which there is no indicator, i.e., you can only see indicators and not any activity for which an alert did not fire

3. If all you have is alert data, can you positively confirm that you have been compromised?

I know the answer to this one, but am including it for emphasis of the point that alert data alone does not lend itself to digital situational awareness. Alert data + session data is the bare minimum as far as I am concerned. At least with this combination you can observe the egress sessions, in other words, what did the attacker do next?

You are right. If you only have alert data, you cannot validate a security incident.

Thank you for your questions!

Richard Bejtlich is teaching new classes in DC and Europe in 2009. Register by 1 Jan and 1 Feb, respectively, for the best rates.


Richard, regarding point two, I strongly agree.

For determining how a product generated an indicator, I encourage anyone using commercial solutions to pressure the vendor for enough information that the alert criteria are transparent. For anyone evaluating a product, find out how much they will give you and consider it as part of your evaluation.

To use IDS as an example, some vendors may let you see signatures, some may after you sign a NDA, and some may not even with an NDA.

Seeing the underlying activity or researching when there is no indicator can be an issue with many network monitoring appliances since they have limited disk space for packet capture or do not include full session data. As a Sguil user, I've come to see anything less than full packet capture and session data as a compromise I don't want to make. Some situations may require compromising, but full packet capture and session data are amazingly useful.
Anonymous said…
your blog is very good
i invite you to visit my small blog
it's for security
you can give me your opinion on it
Anonymous said…
What resources do you find most useful for staying on top of network security monitoring trends? We already have Tao Security on the list, but we're seeking more.
Jim Sansing said…
I would enhance point 2 to read, "See sufficient context of the underlying activity, including both the request and the response, that produced the indicator, whether it is network traffic or raw log messages". If your IDS does not currently provide this, check out the Realeyes IDS at I have recently built new downloads and created live demos, which can be found on the Technology page.

Later . . . Jim

Popular posts from this blog

Five Reasons I Want China Running Its Own Software

Cybersecurity Domains Mind Map

A Brief History of the Internet in Northern Virginia