Friday, March 07, 2008

Common Interface to Packets

Recently a blog reader asked me an interesting question. He wanted to know if it would be possible to replace the variety of network traffic inspection and analysis products with a single box running multiple applications. He was interested in some sort of common interface to packets that could perform the collection function and make traffic available to other products.

There are several ways to look at this issue. First, one can do that already using a commodity hardware platform. It is possible to run multiple traffic inspection applications against a single interface now, but one has to be careful as the number of applications increases. We use this approach with Sguil, where Snort listens to generate alerts, SANCP listens to create session records, Daemonlogger listens to log full content data, PADS listens to generate host records, and so on.

Second, one could buy a fairly open packet capture box and create virtual interfaces which provide a traffic stream to applications. Options which come to mind include Solera Networks capture appliances and Endace Ninja platforms. These typically run Linux and act as a high-end option for packet capture.

Third, one could think of a network tap (like a Net Optics regeneration tap or a Gigamon GigaVUE as that common interface to packet data. The tap collects traffic and then sends it to multiple products. This is a very common scenario for a simple reason: few vendors are willing to accept the decisions made by another vendor regarding packet capture. Everyone wants to collect data themselves, using their own NICs, or drivers, or libraries. That's perfectly understandable but it makes it tough for users who end up managing so many separate boxes.

What do you think?

8 comments:

Martin Roesch said...

Snort 3 is architected to do exactly this. The Snort 3.0 framework is a common platform for implementing network traffic analysis engines. If you look at my blog post on the Snort 3 architecture you'll see that the area denoted as "analytic modules" can accept arbitrary traffic analysis engines. My intention is to enable just what your reader was asking for, a common platform for traffic inspection. Bonus points: it's designed to be hardware accelerated too!

Wesley Shields said...

I didn't know you were using daemonlogger. Have you ever run into the bug I documented at http://www.atarininja.org/index.py/entries/geek/daemonlogger-deleting-wrong-files.1024px? I'm not sure if Marty ever fixed it or not - I pinged him a few months ago but he's probably still insanely busy. It could very well be a nasty bug if you're using it in scripts on automated boxes.

-- WXS

Richard Bejtlich said...

We don't use Daemonlogger's native rotation capacity -- we wrap it with our own script. However, I didn't see that bug. Thanks!

Scott Burch said...

The Gigamon GigaVUE is fantastic! I love these, plug these into your taps for near limitless flexibility and expandability. Why are these so useful?

1) Scalability: Up to 10 Gigamons can be stacked (for a total of 202 ports (any port can be tool or network) functioning in one logical unit) via 10 gigabit Infiniband connectors. There are 8 gigabit copper connectors in the base unit. The unit is expanded using GigaPORT and GigaTap modules. An additional 3 GigaPORT modules can be added to each chassis (this provides an additional 12 ports (are UTP and SFP)
2) Nearly limitless possibilities when it comes to aggregation (many to any), multicasting (any to many), and filtering (vlans, networks, protocols, etc.).
3) Support for 10 Gigabit Ethernet.

I keep running into the situation where all the existing monitor ports are used up. Having the Gigamon basically eliminates this problem.

Also consider the situation where you have many taps and you need to inspect traffic on particular ports from these taps. You can pass all these taps into a Gigamon, then tell the Gigamon to filter the output (so only the traffic on ports you want to see is sent) and send it to one tool port to inspect with one monitoring box.

I've deployed several recently and deploying a couple more in the near future.

firewalz said...

If we take a step back, what do all these devices have in common, including routers and switches? A packet arrives, a process or logic is applied to that packet and based upon the results is egressed or dropped. Now why cant all these funtions be merged into a commom platform

Ish said...

If you are already running daemonlogger, I wonder if there would be any performance benefit in having additional tools like sancp and pads 'tail' the daemonlogger spool files.. Probably not ideal for Snort to that as you likely want it to be a little more real time, but pads and sancp wouldn't be affected that much.

Note that said tools would probably need modification to do this.

g said...

Marty, have any 3rd party vendors picked up on this architecture to provide add-in modules? Or are there plans to develop such partnerships?

Martin Roesch said...

Hi g,

Several vendors have expressed interest, can't say much more at this time. :)