Sunday, January 15, 2006

ShmooCon Wrap-Up

As soon as I returned from DoD Cybercrime, I headed to ShmooCon. I attended last year but didn't speak. This year David Bianco and I presented Network Security Monitoring with Sguil. I was very surprised by the number of people who attended our talk. I hope you liked it. I brought about 30 books provided by various publishers over the years, and distributed them in an ad-hoc manner at the end of the talk. If you received a book, I would very much appreciate seeing a review posted to

I started ShmooCon by arriving late to Dan Geer's keynote. Even seeing only half the talk, I was incredibly impressed. Dr. Geer is a biostatistician in search of a computer security hypothesis to test. I cannot do his talk justice, as I was reduced to trying to take notes by writing in the margins of a book excerpt I received in my conference bag. Here are a few highlights:

  • Dr. Geer noted that our field "suffers nothing but ambiguity over who owns what risk." It is "completely the opposite" in banking, thanks to "massive simulations" and explicit assignment of risk.

  • Dr. Geer reported that a "major bank" "will not spend any more time on prevention, only response." When a patch arrives from Microsoft, they simply apply it. If the patch breaks something, they fix it. The bank no longer cares about Mean Time To Failure. All they track is Mean Time to Repair. Dr. Geer said this approach is not unusual and it is more common than you might think.

  • Dr. Geer warned that "we are in danger of being overtaken by people with credentials and process instead of skill and knowledge." This sounds like a warning against auditors and non-technical people.

  • One sixth of security vulnerabilities are found by the owners of the flawed software. That means five sixths are found by others.

  • Dr. Geer uses a disease model for computer security. He said we don't need every system to be patched, only "enough." This is called "herd immunity." Enough members of the community are immune to keep the disease from destroying the group.

After Dr. Geer's talk I listened to Joe Stewart of LURHQ describe his sandnet concept. The sandnet is a research network for analyzing malware. Joe said that malware can be investigated by code review or behavioral inspection. Code review is complete but time-consuming and skill-intensive; behavioral inspection is incomplete but faster and easier. Sandnets assist with behavioral inspection by giving malware a real host and a simulated network in which to operate.

A sandnet is unique because it is a structured, semi-automated way to use real machines for malware analysis. Too much malware that Joe researches is VMware-aware, mostly using a backdoor I/O function call. Since his sandnet runs on real hardware, the malware doesn't realize it is being watched. To simulate the network, Joe has a gateway pretend to be the Internet. If the malware needs to retrieve a certain file, Joe watches for what it requests and then places it on his gateway where the malware expects to find it. Expect to see more details released through LURHQ shortly.

Next I watched acidus (Billy Hoffman from SPI Dynamics) describe Covert Crawling. Essentially he has implemented a means to mirror Web sites in a manner that simulates a human user rather than a simple retrieval of all Web site pages. In some ways his work appeared to be a "solution in search of a problem," because he assumes Web site administrators pay attention to their logs and check who is mirroring or otherwise investigating their sites. On the other hand, I know his work will be of great interest to many parties who want to add another layer of discretion to their Web site surveillance activities.

After acidus I saw Dan Kaminsky's latest "Black Ops of TCP/IP" presentation. I think I first saw Dan speak four years ago, and he always delivers. His latest research demonstrates a way to abuse IP fragment reassembly timers to fool IDS/IPS. He explained that highly complex inline devices are easy to fingerprint, since each device accepts or rejects traffic differently -- especially at layer 7. Dan also presented updated data on his adventures investigating Sony, and introduced Xovi, his streaming graph visualization framework. Dan said you can feed Xovi Tcpdump data, which I would love to try.

I started Saturday by arriving late for Jennifer Granick's keynote. (Hey, I live about an hour away, I need to find parking, etc.) Thankfully she ran about 20 minutes over her allotted time, so I probably listened to her for 50 minutes or so. She spent a good deal of time talking about the implications of the Bush administration's domestic spying program. With privacy in mind, I then turned to a talk on improvements in Tor that frustrate identifying hidden servers. Basically the old version allowed malicious parties to identify hidden servers by joining the Tor network and carefully inspecting traffic.

After hearing about Tor I attended a fascinating talk about Kryptos by Elonka Dunin. Kryptos is a scuplture at CIA HQ with four sections of ciphertext. Three have been decoded, but the fourth remains a mystery. I recommend visiting Elonka's site for more information.

I turned back to computer security issues by attending a BoF on reverse engineering hosted by Pedram Amini and Chris Eagle (author of ida-x86emu and Naval Postgraduate School professor). That was an insane group. Greg Hoglund from sat in the front row and contributed a lot to the discussion of reverse engineering, including his work analyzing Warden. Pedram encouraged people to share what they know at A lot of people chimed in regarding Ilfak Guilfanov (IDA Pro developer). Steve Micallef's IDA Plugin document was brought up, as was (warning: Russian).

I managed to see most of Mike Rash's presentation on single packet authorization (SPA), which was cool. I was nervous because I was speaking next, so it was tough to concentrate. After my talk I participated in a Snort BoF held by Brian Caswell and Lurene Grenier. They made good points on high-performance Snort operation, including using an architecture-specific compiler to get better performance. In other words, avoid GCC and use an Intel compiler on Intel, an AMD Compiler on AMD, and so on. Brian mentioned zero copy as a means for faster packet collection, along with Endace NICs. I was fairly burnt out after that, so I headed home. I didn't return for the talks on Sunday, since I wanted to go to church and spend some time with my family.

Four aspects of ShmooCon stand out.

  1. The Shmoo Group threw tons of manpower at this conference. I saw red shirts everywhere. This was welcome and unlike any other conference I've attended.

  2. The quality of the talks was very good. They were not all stellar, but the value for the money is absolutely unparalleled.

  3. I have not spoken with so many recognized speakers, authors, and researchers anywhere else. I personally shared at least a few words with Eric Cole, Jenifer Granick, Greg Hoglund, Brian Krebs, Dan Langille, Dru Lavigne, Ike Levy, Johnny Long, Mike Poor, Mike Rash, George Rosamond, Marcus Sachs, Ed Skoudis, and Visigoth. Several Sguil users were there, including #snort-gui regulars like Hanashi (with whom I presented), nr, snortboy, and transzorp. Many people were kind enough to say hello, and one even gave me a coin from his three letter .gov agency.

  4. Many of the talks are available for sale in DVD format from Media Archives. I am sure their Web site will be updated to reflect ShmooCon soon, but I already see my talk in their catalog.

Kudos to the Shmoo Group and founder Bruce Potter.

If you didn't attend ShmooCon last year, please consider it for 2007. If you did attend this year, what did you think?

Incidentally, did anyone attend the BoF were SANS certification and teaching schedules were debated? If so, would you mind posting some comments here?


Hugh McArthur said...

Just an all around great con/event! After spending all day Saturday at the Wardman Park Marriott I can tell you that there is not a better security event for your $s anywhere...

Compared to what you get for your money at say a SANS and/or a CSI event, ShmooCon is the clear winner.

Couple of highlights - Fyodor did a great presentation on Nmap.

I also liked kaos.theory and their Anonym.OS LiveCD.

Anonymous said...

hey richard, i happened to attend your talk at shmoo. it was pretty good.. unfortunately when David Bianco took over, it seems like the room died. I'm pretty sure enough people there have the foo to figure out what he was blabbing about. it was not complex or very indepth.

your talk however was excellent, answering questions I had about postgres portability. I agree the snort bof was cool, hearing from the snort guys themselves how to make snort Uber fast. It would be interesting to do some tests.

Thanks for the talk, it was really informative

Anonymous said...

Shmoocon was worth the money no question. Even with the fun of trying to find parking - a nice 1 mile hike for me :)

I really found Jennifer Granick's presentation thought provoking. Her challenge to dc area folks to ensure that democracy was built into technology was interesting.

Fyodor as always was fun, the demonstration of the speed improvments for NMAP were astonishing, as was his "using NMAP to find images" talk.

Richard your talk was very good in contrast to the last poster I felt the audience was a bit unprepared for much of the technical side of it. Many people only see the offensive side of information security and few focus on the defensive nature. SANS courses honestly are a joke as are many of the "experts" who claim to be network security analysts because they graduated. Sguil is a great tool, however I do see much of that functionality in the SIM space today - you have too look deep but it does exist. However, Sguil as a freeware product/project is phenominal and Bamm , Johnny and many others who helped get it to this point deserve a loud "Thanks!" for offering a way to dig deep, efficiently and on a budget!

The kaos theory anonym os live cd is an interesting concept but as many stated during the talk... if the intention is to provide out of the box functionality for your mom, then you better be able to support media (PDA, DVD, Camera) out of the box. The project is well-intentioned and I'm sure they'll make significant progress over the long run, they seem like a sharp bunch.

Tor is a great tool even if it's slow, but it got a lot of good press during the con.

The discussion on RE was enlightening, not being a programmer or RE myself I sat in trying to learn and learn I did. Wow I'm very impressed by Pedram and Chris and the entire participating group out there it was a pleasure.

My only negative statements about the con:
1. Parking
2. Stolen prize (PSP) come on guys, how 7th grade.
3. seating was pretty bad in some of the rooms, beams/poles obscured a high percentage of the seats.

Overall the presenters were top notch, the organization was well thought out. I would recommend this con to all dc area infosec interested people.

Richard Bejtlich said...

Who else thought parking at ShmooCon was crazy? I don't see the attraction of holding a conference in DC itself. Perhaps someplace else in NoVA for 2007 with (1) abundant cheap parking ($23 for > 3 hrs? Please!); (2) non-obstructed views; and (3) more than one entry and exit door per room?

Anonymous said...

Parking may have been bad, but for people from out of town it is probably very convenient to be less than a block from the Metro so you don't have to rent a car to get around town. As a local, I guess I'm lucky that I could easily take the Metro by parking at the station near my house. I found it extremely convenient for that reason.

I heartily agree about the Kryptos talk by Elonka Dunin. It was the least useful for me in practical terms but it may have been the most interesting. You can tell she loves the subject. I happened upon her in the lobby while she was showing her slides to a couple more people that had missed her talk and she was still just as enthusiastic!

I couldn't really get into the reverse-engineering talk. I made the mistake of checking the other two talks first, both of which were not very impressive. By then, the reverse-engineering BoF was too full to gain easy entry. That was the only time I had trouble due to overcrowding, unlike Defcon where there were many problems with overcrowding.

Brian Krebs wrote about Simple Nomad's "Hacking the Friendly Skies" in his blog.

Fyodor's talk was basically the same one I saw at Defcon. I would not see him talk again unless I knew there was substantial new content, but it is definitely worth attending for someone that hasn't heard it yet.

I saw some of "Web Application Vulnerabilities and Exploits" by Matt Fisher. The portion I saw was exclusively about SQL injection, but it was eye-opening. He did a good job demonstrating how SQL attacks are quite easy and can be changed as needed to be effective in many situations.

Anonymous said...

I forgot to mention the things I noticed that were different in Fyodor's talk. He added the functionality to change a couple of things interactively while NMap is running, such as verbosity. He also took a little poke at Nessus without naming it by mentioning how a competing scanner was going closed source because of a lack of community contributions. Then he put up a very large number of names that had contributed to NMap.

Anonymous said...


I enjoyed reading your account of the con -- very thorough and hit upon most of the things I thought were excellent about Shmoocon. I couldn't agree more with your assessment of Dan Geer's speech, which I found highly engaging and provocative. I'm sorry I missed your talk, and that we did not get a chance to talk more.

On Friday, I drove to work and Metro-ed in, which I should have done on Saturday as well b/c when I arrived the tiny lot was full and the valet insisted I give them a room number to park my car (they wouldn't let me pay in advance). Kaminsky ended up giving me his and he crashed before I got a chance to buy him the 5 or 6 drinks it would take to cover that crazy $30 parking tab. I got so thirsty at around 3 a.m. Sunday morning that I almost hit up one of those ubiquitous Amway/Quixstar dreamers for one of their energy drinks.

Anyway, I had the pleasure of chatting with Elonka and looking at her slides until the wee hours, as I missed her talk also. Grannick's keynote was solid, as always. Kaminsky's presentation was a riot, as was Simple Nomad's description of his self-described "lame 0day" which really was neither of those things. Still, he kept everyone laughing their heads off with his deadpan delivery of surfing pr0n stored on his target's machines and then patched laptops of security execs sitting next to him on the plane.

And I was honored that you mentioned my name alongside such a list of security luminaries: I'm sure I don't
deserve it! :)

Anonymous said...

This was my first security conference I had an opportunity to attend and I thoroughly enjoyed it. I thought the speakers were very good for the most part. I thought Johnny Long's presentation on Hacking Hollywood was pretty funny.

I did get to see Richard's talk and thought it was very informative. I had never gotten to see squil in action and I was impressed with it. I'll definitely have to give it a closer look now.

So, as for my first sec conference goes, It rocked. I would definitely go back again.


Anonymous said...

To the DC Metro area locals - why would you drive when you can ride the Metro? Metro parking is free on the weekends. For you I66 corridor folks, I believe the closest Metro with parking is Ballston. I almost paid for my 3 days fare and parking what the hotel charged for 1 day.

I saw some of the anti-SANS BoF. But not enough to make any real analysis. The part I caught was:

1) Use the materials from the Linux documentation project

2) Have someone standup and teach from it and do practical exercises

3) Give a test

There was probably much more discussed that I didn't hear so I won't draw any conclusions. Like Bettle said, I didn't see the booze they were serving.

That being said my wife is a former teacher and I did network support the same K-12 system - teaching classes is more than just getting some documentation and standing in front of group of people. It takes class materials (lesson plans, canned exercises, etc) and one has to be able to communicate effectively. It's one thing to hack and maybe teach one-on-one, it's a whole different ballgame to do that in front of a crowd. Plus, remember you're dealing with a whole bunch of geeks from the get-go, so you'll have 7 of 10 students going off and doing "other" stuff with their boxes.

I went to about 10 minutes of the B!tchslapping wireless IDS and couldn't take anymore of the speaker. It just wasn't going anywhere.

Dan Greer's keynote was great!

Jennifer Granick's keynote was interesting, however, there was no "equal time" to represent the other side. Some of us can appreciate the work that is involved within the IC, some just can't. 'Nuff said since this isn't the forum for that. People need to read both of James Bamford's books about the NSA.

Johnny Long's presentation was funny and entertaining, but no real practical application. The Bruce video was intertaining too.

Probably the best new thing I saw was the Covert Crawler. Kaos Theory's OpenBSD LiveCD using Tor was a close second.

I'd seen Richard present Sguil before, but it was a good talk. I don't know how many times Netflow data has filled in the empty spaces of a picture to tell the whole story.

The most difficult about Shmoocon was trying to be at 2 talks at once, sometimes all 3. I generally select my second choice mainly because of the crowding. Thankfully everything will be available here shortly. I feel sorry for the guy who made the DVDs, I don't think he'll sell many because everyone will download the videos.

I'll attend again next year. Hats off to Bettle, Bruce, Heidi, and the rest of the Shmoo Group.


Anonymous said...

Regarding parking: My home office is a half-block from the hotel, on Calvert and McGill. We have room for some 18 cars parked legally (though blocking each other, so everyone would have to leave at more or less the same time or leave keys or something). If you want free parking, give me a call next year. If I haven't moved you're welcome to park.


hellNbak said...

The - Anti-Sans Starting Your Own "Not for profit" Training Institution - was not about taking the linux documentation project and delivering it.

The linux documentation projet was brought up as an example of how an open source project can work but not as a final idea on what to do.

dghnfgj said...
This comment has been removed by a blog administrator.