NSM in Products

A blog reader recently asked:

I've been tasked with reevaluating our current NSM / SIEM implementation, and I see that you posted about a NetFlow book you are techediting for Lucas.

My question is this, Outside of Sguil, what do you prefer/recommend in the way of NSM products/solutions?

Our current NSM uses a modified version NetFlow and our Networking team also uses Cisco Netflow elsewhere...

While I find it useful to collect header data, the current implementation lacks payload information. So while we may be able to turn back the clock to look at flows for a given duration, its not always possible to see valuable contents...

Another wall I have hit with NetFlow is that the communication of the protocol takes place in somewhat of a half duplex manner (I.E. it is possible to receive the response flow before you receive the request flow) thus making it difficult to assure a particular direction without some processing...

I have yet to see a blog post covering any consolidated comparisons to solutions regarding NSM.

I do have your NSM book on order from Amazon today if it already has the answers I'm looking for...

As always, thank you for your time Richard, I appreciate it greatly.

Thank you for the question. I don't recommend specific products, but I do recommend NSM data types. That way, you can ask the vendor which NSM data types they support, and then decide if their answer is 1) correct and 2) sufficient. For reference, the six NSM data types are:

  1. Alert: judgment made by a product ("Port scan!" or "Buffer overflow!"); either detect or block

  2. Statistical: high-level description of activity (protocol percentages, trending, etc.)

  3. Session: conversations between hosts ("A talked to B on Friday for 61 seconds sending 1234 bytes")

  4. Full Content: all packets on the wire

  5. Extracted Content: rebuild elements of a session and extract metadata

  6. Transaction: generate logs based on request-reply traffic (DNS, HTTP, etc.)
2018 note: in 2013 I added "metadata" as an NSM data type, such as WHOIS data about IP addresses or domain names observed in other NSM data. Also note that in the "Extracted Content" column below, Bro has supported file extraction since at least 2013 (possibly earlier).

Looking at these six types, I can make the following general assessments of products. This is my opinion based on products I have encountered. If you find a product that performs better than the general categories I describe, excellent!


If you want to learn more about this, I'll be discussing it during my solo presentation at the 2009 Information Security Summit, October 29-30, 2009 at Corporate College East in Warrensville Heights, Ohio.

Comments

Matthew Wollenweber said…
I've been using passive network sensors for about 4 years now and IMO they're still a developing product that have significant flaws. I've seen their use fall into two primary uses. First is discovery of the loudest and most obnoxious hosts on a network. Almost any product on the market is now useful for use in this category. Basically, this use case if for networks that are barely under control and the security folks are just trying to keep the network from sinking.

The second category is more R&D. Years ago (or still today on exceedingly small networks) a talented network admin could visually spot anomalous traffic with tcpdump/wireshark. Humans have a great capacity for noticing what shouldn't be there. Unfortunately at gigbit speeds, that's not practical. I personally like to write code to look for oddities. But recently, I've gotten into Netwitness. It's primarily meant as a net forensics tool, but I love it for near live analysis. When configured to process pcaps from a continuous network dump (cycles every few days) it's an extremely powerful tool. Basically it lets you browse network traffic, see something odd, and then do simple queries to answer interesting questions. This can be labor intensive, but if you're looking for malicious behavior without signatures and you don't have a development staff -- it's the way to go.

Richard, sorry for dropping product names if you're trying to avoid that. You can filter this. However, I'm not affiliated with Netwitness and it's one of those niche products that can be exceedingly useful but is under-known.
12:26 PM
SS said…
Hi Mr. Bejtlich,

Let me introduce myself as a big fan of your ideas/books. I am a brazilian Incident Responder and NSM praticioneer (10 years experience on telecom and banking industries).

I just made a blog post citing your excelent "NSM in products" discussion. Here is the automated english translation to it: http://tinyurl.com/blogss

Regards,

Sandro Süffert
http://suffert.com
7:54 PM
Anonymous said…
Can you elaborate on what you would define as “Extracted Content” and possibly provide an example of common tool a SOC would use to monitor this data type? (I searched your blog and didn’t see any information on this subject) Thanks.
6:19 PM
Post a Comment

Popular posts from this blog

Zeek in Action Videos

Image
This is a quick note to point blog readers to my Zeek in Action YouTube video series for the Zeek network security monitoring project .  Each video addresses a topic that I think might be of interest to people trying to understand their network using Zeek and adjacent tools and approaches, like Suricata, Wireshark, and so on.  I am especially pleased with Video 6 on monitoring wireless networks . It took me several weeks to research material for this video. I had to buy new hardware and experiment with a Linux distro that I had not used before -- Parrot .  Please like and subscribe, and let me know if there is a topic you think might make a good video.
Read more

MITRE ATT&CK Tactics Are Not Tactics

Image
Just what are "tactics"? Introduction MITRE ATT&CK  is a great resource, but something about it has bothered me since I first heard about it several years ago. It's a minor point, but I wanted to document it in case it confuses anyone else. The MITRE ATT&CK Design and Philosophy document from March 2020 says the following: At a high-level, ATT&CK is a behavioral model that consists of the following core components: • Tactics, denoting short-term, tactical adversary goals during an attack; • Techniques, describing the means by which adversaries achieve tactical goals; • Sub-techniques, describing more specific means by which adversaries achieve tactical goals at a lower level than techniques; and • Documented adversary usage of techniques, their procedures, and other metadata. My concern is with MITRE's definition of "tactics" as "short-term, tactical adversary goals during an attack," which is oddly recursive. The key word in the tacti
Read more

New Book! The Best of TaoSecurity Blog, Volume 4

Image
  I've completed the TaoSecurity Blog book series . The new book is  The Best of TaoSecurity Blog, Volume 4: Beyond the Blog with Articles, Testimony, and Scholarship .  It's available now for Kindle , and I'm working on the print edition.  I'm running a 50% off promo on Volumes 1-3 on Kindle through midnight 20 April. Take advantage before the prices go back up. I described the new title thus: Go beyond TaoSecurity Blog with this new volume from author Richard Bejtlich. In the first three volumes of the series, Mr. Bejtlich selected and republished the very best entries from 18 years of writing and over 18 million blog views, along with commentaries and additional material.  In this title, Mr. Bejtlich collects material that has not been published elsewhere, including articles that are no longer available or are stored in assorted digital or physical archives. Volume 4 offers early white papers that Mr. Bejtlich wrote as a network defender, either for technical or pol
Read more