1. Are all alert data created equal?
This question originates with my employment at an MSSP where we process many types of alert data from Dragon IDS, Cisco IPS and ISS. Snort and Sourcefire strangely are underrepresented. My question is if Dragon IDS, Cisco IPS, ISS, Snort and Sourcefire all looked at the same full-content data, would they all produce the same results? I think not and would like to empirically verify this theory.
Testing detection systems is a complicated topic. I am not sure what methodology a place like NSS Labs uses. I bet they get varying results depending on the product. If you read A Tool for Offline and Live Testing of Evasion Resilience. in Network Intrusion Detection Systems you will see big differences between Snort and Bro, for example.
2. When is an analyst no longer an analyzer of data but an analyzer of dashboards?
This question also originates with my employment at an MSSP because like I said, we process so many disparate alert types and there is only so much time in a shift that it is challenging for an analyst to really spend quality time with a piece of data and conclusively determine what happened. Therefore the analysts evolve into analyzers of dashboards instead of data in order to promptly assess alerts and determine if there was a legitimate attack or not.
I would say you are an analyzer of dashboards when you cannot do the following:
- Determine how a product generated an indicator
- See the underlying activity that produced the indicator, whether it is network traffic or raw log messages
- Research activity for which there is no indicator, i.e., you can only see indicators and not any activity for which an alert did not fire
3. If all you have is alert data, can you positively confirm that you have been compromised?
I know the answer to this one, but am including it for emphasis of the point that alert data alone does not lend itself to digital situational awareness. Alert data + session data is the bare minimum as far as I am concerned. At least with this combination you can observe the egress sessions, in other words, what did the attacker do next?
You are right. If you only have alert data, you cannot validate a security incident.
Thank you for your questions!
Richard Bejtlich is teaching new classes in DC and Europe in 2009. Register by 1 Jan and 1 Feb, respectively, for the best rates.