NSM in Products

A blog reader recently asked:

I've been tasked with reevaluating our current NSM / SIEM implementation, and I see that you posted about a NetFlow book you are techediting for Lucas.

My question is this, Outside of Sguil, what do you prefer/recommend in the way of NSM products/solutions?

Our current NSM uses a modified version NetFlow and our Networking team also uses Cisco Netflow elsewhere...

While I find it useful to collect header data, the current implementation lacks payload information. So while we may be able to turn back the clock to look at flows for a given duration, its not always possible to see valuable contents...

Another wall I have hit with NetFlow is that the communication of the protocol takes place in somewhat of a half duplex manner (I.E. it is possible to receive the response flow before you receive the request flow) thus making it difficult to assure a particular direction without some processing...

I have yet to see a blog post covering any consolidated comparisons to solutions regarding NSM.

I do have your NSM book on order from Amazon today if it already has the answers I'm looking for...

As always, thank you for your time Richard, I appreciate it greatly.

Thank you for the question. I don't recommend specific products, but I do recommend NSM data types. That way, you can ask the vendor which NSM data types they support, and then decide if their answer is 1) correct and 2) sufficient. For reference, the six NSM data types are:

  1. Alert: judgment made by a product ("Port scan!" or "Buffer overflow!"); either detect or block

  2. Statistical: high-level description of activity (protocol percentages, trending, etc.)

  3. Session: conversations between hosts ("A talked to B on Friday for 61 seconds sending 1234 bytes")

  4. Full Content: all packets on the wire

  5. Extracted Content: rebuild elements of a session and extract metadata

  6. Transaction: generate logs based on request-reply traffic (DNS, HTTP, etc.)
2018 note: in 2013 I added "metadata" as an NSM data type, such as WHOIS data about IP addresses or domain names observed in other NSM data. Also note that in the "Extracted Content" column below, Bro has supported file extraction since at least 2013 (possibly earlier).

Looking at these six types, I can make the following general assessments of products. This is my opinion based on products I have encountered. If you find a product that performs better than the general categories I describe, excellent!

If you want to learn more about this, I'll be discussing it during my solo presentation at the 2009 Information Security Summit, October 29-30, 2009 at Corporate College East in Warrensville Heights, Ohio.


I've been using passive network sensors for about 4 years now and IMO they're still a developing product that have significant flaws. I've seen their use fall into two primary uses. First is discovery of the loudest and most obnoxious hosts on a network. Almost any product on the market is now useful for use in this category. Basically, this use case if for networks that are barely under control and the security folks are just trying to keep the network from sinking.

The second category is more R&D. Years ago (or still today on exceedingly small networks) a talented network admin could visually spot anomalous traffic with tcpdump/wireshark. Humans have a great capacity for noticing what shouldn't be there. Unfortunately at gigbit speeds, that's not practical. I personally like to write code to look for oddities. But recently, I've gotten into Netwitness. It's primarily meant as a net forensics tool, but I love it for near live analysis. When configured to process pcaps from a continuous network dump (cycles every few days) it's an extremely powerful tool. Basically it lets you browse network traffic, see something odd, and then do simple queries to answer interesting questions. This can be labor intensive, but if you're looking for malicious behavior without signatures and you don't have a development staff -- it's the way to go.

Richard, sorry for dropping product names if you're trying to avoid that. You can filter this. However, I'm not affiliated with Netwitness and it's one of those niche products that can be exceedingly useful but is under-known.
SS said…
Hi Mr. Bejtlich,

Let me introduce myself as a big fan of your ideas/books. I am a brazilian Incident Responder and NSM praticioneer (10 years experience on telecom and banking industries).

I just made a blog post citing your excelent "NSM in products" discussion. Here is the automated english translation to it: http://tinyurl.com/blogss


Sandro Süffert
Anonymous said…
Can you elaborate on what you would define as “Extracted Content” and possibly provide an example of common tool a SOC would use to monitor this data type? (I searched your blog and didn’t see any information on this subject) Thanks.

Popular posts from this blog

Zeek in Action Videos

New Book! The Best of TaoSecurity Blog, Volume 4

MITRE ATT&CK Tactics Are Not Tactics