Wednesday, July 29, 2009

Notes from OISF Meeting in DC

This month I was pleased to attend a public meeting of the Open Information Security Foundation in Washington, DC. I got a chance to meet several people I have known for many years through their work with Snort, such as Matt Jonkman, Will Metcalf, Victor Julien, Frank Knobbe, and two guys from a federal agency that have extended Sguil way beyond what I knew anyone was doing! The group posted DC Brainstorming Meeting Notes, but I wanted to record a few thoughts here.

OISF is a US nonprofit, a 501c(3). Their goal is to produce a new network inspection and filtering engine (IDS/IPS) that will be released under GPLv2. They can not and will not commercialize, sell, patent, copyright, or profit from the engine. Rather, others who participate in the OISF Consortium (listed on their Web site) are donating coders, equipment, and financial support in exchange for the ability to commercialize the engine.

OISF works with the Open Source Software Institute, famous for getting FIPS validation for OpenSSL -- something everybody wanted but no one wanted to fund alone. OISF is part of the DHS Homeland Open Security Technology (HOST) program. OISF has received legal guidance from the Software Freedom Law Center.

OISF has many goals for their engine, outlined in the notes I linked earlier. Most interesting is their goal for a production release by the end of this year. If they are to make this goal, I think the project needs to severely limit the requirements for the first release. I would focus on the following.

  • Developing the rules language.

  • Implementing IPv6.

  • Implementing multi-threading.

Those three tasks are monumental, but they would immediately differentiate OISF from other options. There is talk within the project of semi-Snort compatible output, so you might send OISF data to a file in Snort Unified or Unified2 format to be read by Barnyard or Barnyard2.

If you want to know more about the project, the Mailing Lists are the best option. As it develops I will discuss it here.


VictorJ said...

Thanks Richard, for attending our meeting, your input and for your recommendations!

Your focus recommendation is pretty much what we decided after the meeting with one addition: a first version of the IP reputation ideas. What exactly that version should be able to do is something we're just starting to discuss on one of our working group mailinglists.

The rules language is a difficult topic. We'd like to support the Snort language as used by Sourcefire's VRT and Emerging Threads but recognize that we'd probably diverge from it pretty quickly. For this subject we've started a workgroup mailinglist as well.

IPv6 was coded into the engine from the start and, while it needs some work, is in pretty good shape already. Same goes for the multi-threading.

Outputwise Snort's unified1 is working well already. Unified2 is planned. It's actually fun to see alerts generated by our new engine showing up in Sguil :)

VivekRajan said...

Thanks Richard for covering this.

Really cool stuff backed up by great development team.

To me, IP reputation is the most fascinating and ambitious feature. I hope this gets into the first release as well. A step closer to a believable 'threat score'.

Yet, I wonder how the project will be directed after the core developers are done working their magic. Will there be too many competing commercial interests (some with deep pockets) after the initial honeymoon phase ? Apologies for the skepticism.

I will join the IP Rep mailing list to keep track of the discussions there.

Matt Jonkman said...

I agree with you Vivek that IP Reputation is a huge deal. I've been wanting this for 5 years, so we're just going to have to do it ourselves. I think everyone wanted IP Reputation but there was no commercial impetus for the vendors to try it on their own.

As for what happens to the project after the core developers and staff move on: Very good question. We'll all surely move on before the project is dead. The thing that will protect the engine is it's non-profit status. The board and membership decide what to do, how to do it, and whom to do it with. The laws for how a non-profit runs require that it do NOTHING that isn't in the pursuit of it's stated charter which in our case is to create and maintain a new IDS engine. The foundation cannot by law turn a profit or sell a product. The board must be elected by the community, so as long as the community stays involved any group or vendor that seeks to take control of the foundation would be unable to. In fact if the community decides it doesn't want any vendor on the board they can do that with their votes.

That said, I'm very excited about the vendors we have involved, and the long term support of the engine will be in a large way due to the vendors financial support once the grant money runs out in a few years. We are long past the days where first class software can be made and maintained long term by a group of open source zealots. Code this complex requiring this much QA and testing take money to maintain. Lots of money, lots of hardware, lots of very smart folks. We're lucky enough to have most of those things, and that's due in great part to our initial supporters. We've got nearly a quarter million dollars worth of hardware from several vendors on the way to our QA lab! There's no way that would have been available to us without their help.

And the vendors will get something in return of course. They'll get a license to package the engine into their products. That's a good thing, that ensure that they will be VERY interested in making sure the engine is both high quality and is supported over the long term.

We HAVE to involve the vendors, but the non-profit nature of the organization we believe will keep the critical balance between open development and long term support. But the key there is the community staying involved! You have to vote, you have to take a turn being a board member, you have to keep watching what the foundation does and ask questions, and call us out if we do something inappropriate!

If you stay involved and vote this will remain an open and free engine!

Anonymous said...

Can you say anything about the sguil extensions you mention?

VictorJ said...

@Anonymous: there is nothing Sguil specific in the code. By supporting the unified1 event format events can be processed by Barnyard which feeds them to Sguil.

Anonymous said...

Glad to hear about this project. I have been looking around for "the engine" to begin some contribution. Where is the source hosted? Is the development happening behind the scenes, for now?