Thoughts on High Speed Network Monitoring

I've been following an interesting thread on snort-users about collecting alert data on high speed networks. Users are debating how much traffic Snort can handle. One way to at least start answering this questions is to enable the performance monitor. Vjay Larosa's post was helpful, as it pointed me towards perfmon-graph. This Perl script works with Snort performance monitor output and RRDtool output to produce graphs of Snort performance statistics.

Using perfmon-graph requires two steps. First, enable Snort to output the statistics you need to a text file. Add the following line to your snort.conf file:

preprocessor perfmonitor: time 60 file /nsm/snort/perfmon.txt pktcnt 500

This tells Snort to output statistics every 60 seconds to a file called /nsm/snort/perfmon.txt The perfmon-graph README warns us not to set the number following 'pktcnt' too high. For example, if 500 packets are not collected in 60 seconds, then we will not get a statistics output.

After restarting Snort with this new preprocessor, your perfmon.txt file will begin collecting entries like these:

bourque# tail -f /nsm/snort/perfmon.txt 
1085192584,0.000,0.1,0.0,0.0,395,11.67,0.4,0.4,0.8,0.4,30,
30,0.0,0,1,0.0,0.0,0.0,0.0,0,0,0.0,0.0,99.9
1085192678,0.000,0.0,0.0,0.0,213,8.78,0.1,0.1,0.3,0.6,3,31,
0.0,0,3,0.0,0.0,0.0,0.0,0,0,0.0,0.0,100.0
1085192746,0.000,0.0,0.0,0.0,316,51.71,0.1,0.1,0.2,0.2,2,31,
0.0,0,1,0.0,0.0,0.0,0.0,0,0,0.0,0.0,100.0


Next decide where to run perfmon-graph and RRDtool. I chose to deploy both on a system other than the sensor running Snort, called janney. On FreeBSD use the /usr/ports/net/rrdtool port to install RRDtool. When running perfmon-graph I got an error about a missing tie/File.pm file, which I learned ships with /usr/ports/devel/p5-Tie-File.

To set up perfmon-graph itself, download and extract it to a convenient directory. I chose to follow the perfmon-graph author's suggestion to use SSH to periodically copy the perfmon.txt file to the system running perfmon-graph. I made the following entry in my crontab to do this:

0,5,10,15,20,25,30,35,40,45,50,55 * * * * ssh worker@bourque 
 "cat /nsm/snort/perfmon.txt" | /usr/local/bin/perl  
 /usr/local/src/perfmon-graph/perfmon-graph.pl  
 /usr/local/www/data/perfmon-graph/bourque/ -

This setup relies on user worker being able to SSH from janney to my sensor, bourque, using public key authentication. I'm using perfmon-graph.pl on FreeBSD 4.9 STABLE, which had an old version of Perl installed (5.005_03). I installed a newer version using the latest package (5.6.1), and executed 'use.perl port' to make it the default. (I can revert to the base with use.perl system.)

Once this is working, a set of files will appear in /usr/local/www/data/perfmon-graph/bourque. You may recognize this as a likely place where Apache could serve up Web pages, and that is the case here. A visit presents graphs like that above, which shows Snort's perception of bandwidth monitoring in Mbps.

The accuracy and value of these statistics are debatable, but they are at least a start.

In addition to setting up performance monitoring, I learned of a great source of information on high speed monitoring issues through this post. It led me to SCAMPI, "A Scaleable Monitoring Platform for the Internet." Their publications page lists many interesting papers with advice and research regarding monitoring high bandwidth networks. The SCAMPI project has the goal of monitoring traffic at 10 to 100 Gbps.

Finally, Brian Caswell posted details on a massive rule update designed to improve Snort performance. He directs interested readers to the updated section of the Snort manual.

Comments

Post a Comment

Popular posts from this blog

Zeek in Action Videos

Image
This is a quick note to point blog readers to my Zeek in Action YouTube video series for the Zeek network security monitoring project .  Each video addresses a topic that I think might be of interest to people trying to understand their network using Zeek and adjacent tools and approaches, like Suricata, Wireshark, and so on.  I am especially pleased with Video 6 on monitoring wireless networks . It took me several weeks to research material for this video. I had to buy new hardware and experiment with a Linux distro that I had not used before -- Parrot .  Please like and subscribe, and let me know if there is a topic you think might make a good video.
Read more

MITRE ATT&CK Tactics Are Not Tactics

Image
Just what are "tactics"? Introduction MITRE ATT&CK  is a great resource, but something about it has bothered me since I first heard about it several years ago. It's a minor point, but I wanted to document it in case it confuses anyone else. The MITRE ATT&CK Design and Philosophy document from March 2020 says the following: At a high-level, ATT&CK is a behavioral model that consists of the following core components: • Tactics, denoting short-term, tactical adversary goals during an attack; • Techniques, describing the means by which adversaries achieve tactical goals; • Sub-techniques, describing more specific means by which adversaries achieve tactical goals at a lower level than techniques; and • Documented adversary usage of techniques, their procedures, and other metadata. My concern is with MITRE's definition of "tactics" as "short-term, tactical adversary goals during an attack," which is oddly recursive. The key word in the tacti...
Read more

New Book! The Best of TaoSecurity Blog, Volume 4

Image
  I've completed the TaoSecurity Blog book series . The new book is  The Best of TaoSecurity Blog, Volume 4: Beyond the Blog with Articles, Testimony, and Scholarship .  It's available now for Kindle , and I'm working on the print edition.  I'm running a 50% off promo on Volumes 1-3 on Kindle through midnight 20 April. Take advantage before the prices go back up. I described the new title thus: Go beyond TaoSecurity Blog with this new volume from author Richard Bejtlich. In the first three volumes of the series, Mr. Bejtlich selected and republished the very best entries from 18 years of writing and over 18 million blog views, along with commentaries and additional material.  In this title, Mr. Bejtlich collects material that has not been published elsewhere, including articles that are no longer available or are stored in assorted digital or physical archives. Volume 4 offers early white papers that Mr. Bejtlich wrote as a network defender, either for technica...
Read more