TaoSecurity Blog

Some of you who rely on various system and application logs might take exception to my emphasis on interpreting network traffic. You might think I am "anti-log." That is absolutely not true. I will demonstrate a case that shows I appreciate logs in certain situations. Last night I was analyzing alert data collected from one of the customers I monitor. One of the Snort alerts I saw (a bleeding-exploit.rules entry) indicated BLEEDING-EDGE EXPLOIT Possible MSIE VML Exploit . This did not look promising, especially since I was not flooded with these events. In other words, if I had seen 100, I would not be 100 times more worried than if I saw only one alert. The fact that I was investigating a single alert made me think this signature might be deadly accurate. I am not going to walk through the entire investigation for this event. Suffice it to say I wanted to know if the victim system was truly exploited. I eventually found myself looking at transcripts of traffic and ...

Since this blog has a higher readership than the NoVA Sec blog, I want to reiterate: The next NoVA Sec meeting will take place 1900 (7 pm) Monday 29 January 2007 at Getronics Red Siren . Wesley Shields will discuss FreeBSD jails .

At the risk of adding yet more fuel to the fire, I'd like to share a few more thoughts on NSM. Although the title of this post is The Self-Defeating Network (SdN), I don't intend it to be a slam of Cisco's Self-Defending Network (SDN). Rather, the post's title demonstrates a probably lame attempt at branding an otherwise potentially boring issue. Thus far I've tried to explain NSM, and the related concept of Defensible Network Architecture (originated in my Tao book, expanded in Extrusion ), from the view of best practices. I've tried to say here's what you should do , because it gives you the best chance to survive on the mean streets of the Internet. In this post I'll take a different approach by describing the Self-Defeating Network -- what you should not do if you want to have a chance to defend your enterprise. These are the characteristics of the Self-Defeating Network (SdN). The SdN is unknown , meaning no one really understands how ...

Frequently I'm asked about the data sources I cite as being necessary for Network Security Monitoring, namely statistical data, session data, full content data, and alert data. Sometimes people ask me "Is it NSM if I'm not collecting full content?" or "Where's the statistical data in Sguil? Without it, is Sguil a NSM tool?" In this post I'd like to address this point and answer a question posted as a comment Joe left on my post My Investigative Process Using NSM . In 2002 while working for Foundstone, I contributed to the fourth edition of Hacking Exposed , pictured at left. On page 2 I defined NSM as the collection, analysis, and escalation of indications and warning to detect and respond to intrusions . Since then I've considered modifying that definition to emphasize the traffic-centric approach I intended to convey by using the term "network." Whenever I speak or write about NSM I emphasize the four types of network data mos...

I mentioned the power of Wireshark display filters when analyzing 802.11 last year. Now I read Ephemeral Diffie Hellman support - NOT ! by the Unsniff guys and they tell me that they cannot decode SSL traffic which uses the ephemeral Diffie-Hellman cipher suite. I wonder what that looks like in traffic? Thanks to Wireshark display filters, I can find a suitable packet. Here's a matching packet. You could use syntax like this with Tshark: tshark -V -n -r capture -R "ssl.handshake.ciphersuite == 0x39" ...edited... Secure Socket Layer TLSv1 Record Layer: Handshake Protocol: Server Hello Content Type: Handshake (22) Version: TLS 1.0 (0x0301) Length: 74 Handshake Protocol: Server Hello Handshake Type: Server Hello (2) Length: 70 Version: TLS 1.0 (0x0301) Random gmt_unix_time: Jan 26, 2007 19:32:44.000000000 random_bytes: 76744E818415307EA6F7C14FAF4BA640F67834C1263E5065... Session ID Length: 32 Session ID (32 b...

If you've read this blog for a while, or even if you've just been following it the last few months, you might be saying "Fine Bejtlich, we get it. So what do you want?" The answer is simple: I want NSM-centric techniques and tools to be accepted as best practices for digital security. I don't say this to sell products. I say this because it's the best chance we have of figuring out what's happening in our enterprise. NSM means deploying sensors to collect statistical, session, full content, and alert data. NSM means having high-fidelity, actionable data available for proactive inspection when possible, and reactive investigation every other time. NSM means not having to wait to hire a network forensics consultant who brings his own gear to the enterprise, hoping for the intruder to make a return appearance while the victim is instrumented. I'd like to see organizations realize they need to keep track of what's happening in their enterpri...

My Monitor Your Routers post touched on the idea of trust. I'd like to talk about that for a moment, from the perspective of an operational security person. I'm not qualified to address trust in the same way an academic might, especially since trust is one of the core ideas of digital security. Trust can be described in extreme mathematical detail and in some cases even proven. I don't know how to do that. Instead, I'm going to describe how I decide what I trust when performing network incident response . The diagram above shows the level of trust I have in the evidence or operation of various devices. I've broken them down into four categories. My trust decreases as the level of interaction with users increases. This is not a slam on users. Rather, it's a reflection of the idea that the level of exposure increases as one considers a device that is operated at the whim of a human. At the very bottom of the pyramid would be a person on his/her user pl...

I know some of you believe that my Network Security Monitoring (NSM) methodology works and is the best option available for independent, self-reliant, network-centric collection, analysis, and escalation of security events. Some of you think NSM is impossible, a waste of time, irrelevant, whatever. I thought I would offer one introductory case based on live data from my cable line demonstrating my investigative process. Maybe after seeing how I do business the doubters will either think differently (doubtful) or offer their own answer to this problem: how do I know what happened in my enterprise? (Please: I don't want to hear people complain that I'm using data from a cable line with one public target IP address; I'm not at liberty to disclose data from my client networks in order to satisfy the perceived need for bigger targets. The investigative methodology is the same. Ok?) s shown in the figure at left, I'm using Sguil for my analysis. I'm not going to...

I had the opportunity to "hang in the sky" (to use John Denver's phrase) again this week. While flying I read one of the best issues of USENIX ;login: I've seen. The December 2006 issue featured these noteworthy articles, most of which aren't online for everyone. USENIX members have the printed copy or can access the .pdf now. Nonmembers have to wait a year or attend the next USENIX conference, where free copies are provided. My favorite article was The Underground Economy: Priceless by Team Cymru ( .pdf available for free now). The article described the sorts of stolen material one can find circulating in the underground. It's a definite wake-up call for anyone who doesn't pay attention to that issue. Choice quotes include: Entire IRC networks--networks, not just single servers--are dedicated to the underground economy. There are 35 to 40 particularly active servers, all of which are easy to find. Furthermore, IRC isn't the only Internet ...

My second Snort Report has been posted. In this edition I talk about upgrading from an older version to 2.6.1.2, and then I begin discussing the snort.conf file. I recommend reading the first Snort Report so you can follow along with my methodology. In the third article (to be posted next month) I describe the sorts of activity you can detect without using Snort rules or dynamic preprocessors. The idea behind this series of articles is to develop an intuitive understanding of Snort's capabilities, starting with the basics and becoming more complicated.

Today I read this new Cisco advisory containing these words: Cisco routers and switches running Cisco IOS® or Cisco IOS XR software may be vulnerable to a remotely exploitable crafted IP option Denial of Service (DoS) attack. Exploitation of the vulnerability may potentially allow for arbitrary code execution. The vulnerability may be exploited after processing an Internet Control Message Protocol (ICMP) packet, Protocol Independent Multicast version 2 (PIMv2) packet, Pragmatic General Multicast (PGM) packet, or URL Rendezvous Directory (URD) packet containing a specific crafted IP option in the packet's IP header... A crafted packet addressed directly to a vulnerable device running Cisco IOS software may result in the device reloading or may allow execution of arbitrary code. This is the sort of "magic packet" that's an attacker's silver bullet. Send an ICMP echo with the right IP option to a router interface and whammo -- you could 0wn the router. Who would n...

While waiting in the airport, and flying between Ottawa and Washington Dulles, I read a copy of Mike Rothman 's new book The Pragmatic CSO . I was somewhat suspicious of some of the early reviews, since they appeared so quickly after the book was published. You can rest assured that I read the whole book -- and I really liked it. The most important feature of "P-CSO" (as it's called) is that it is a business book . P-CSO teaches readers (assumed to be techies, for the most part) how to think like a businessperson who reports and interacts with other businesspeople. I took business classes in college and graduate school, and I run my own business. Most of the time, however, I'm doing technical work. I usually stay so busy that I don't consciously consider the sorts of business issues Mike describes. Consider the following quote from pages 51-2: The only way to get a seat at the table is by holding yourself to the same standards as everyone else. Operat...

It's been several years since I had operational responsibility for a single organization's network security operations. As a consultant I find myself helping many different customers, but I maintain continuous monitoring operations for only a few. Sometimes I wonder what it would be like to step back into a serious security role at a single organization. Are any of you looking for someone with my background (.pdf)? If yes, please feel free to email taosecurity [at] gmail [dot] com. Thank you.

It's been over a year since my last request for comments on a new laptop . I had a scare using my almost 7-year-old Thinkpad a20p today while teaching a private class . I wanted to run VMware Server using a VM configured to need 192 MB RAM. The laptop has 512 MB of physical RAM. When I started the VM, VMware Server complained it didn't have sufficient free RAM. Puzzled, I checked my Windows hardware properties and saw only 256 MB RAM reported! Oh oh. I guessed that maybe one of the two 256 MB RAM sticks in my laptop had been loosened on the trip to the class site. Using a grounding wrist band thoughtfully provided by my class, I removed my laptop's RAM and reseated it. After booting, I saw all 512 MB again. Whew. T series like a T60p. However, I'm considering a new OS strategy. Currently I dual boot Windows 2000 Professional and FreeBSD 6.x. For my next laptop, I'm thinking of installing an OS fully supported by VMware Server, like Ubuntu , with VMware...

A site hosting news on FreeBSD 7.0 also included several great tips for FreeBSD under VMware . One tip talked about the lnc network interface standard under VMware. You can see lnc0 in this sample VM. Here's dmesg output: lnc0: <PCNet/PCI Ethernet adapter> port 0x1400-0x147f irq 18 at device 17.0 on pci0 lnc0: Attaching PCNet/PCI Ethernet adapter lnc0: [GIANT-LOCKED] lnc0: Ethernet address: 00:0c:29:38:7d:ea lnc0: if_start running deferred for Giant lnc0: PCnet-PCI This is what the interface looks like in VMware: taosecurity:/root# ifconfig lnc0 lnc0: flags=108843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST, NEEDSGIANT> mtu 1500 inet6 fe80::20c:29ff:fe38:7dea%lnc0 prefixlen 64 scopeid 0x1 inet 198.18.153.167 netmask 0xffff0000 broadcast 198.18.255.255 ether 00:0c:29:38:7d:ea The fact that lnc is GIANT-locked is bad for network performance. Furthermore, lnc is deprecated in FreeBSD 7.0, replaced by le . The site included a tip to replace the lnc...

I'd like to mention a few FreeBSD news items. First, FreeBSD 6.2 was released Monday. I am not rushing to install it but I plan to deploy it everywhere. I have a subscription to FreeBSDMall.com , so I don't need to download any .iso's at the moment. I plan to upgrade all existing FreeBSD 6.1 systems using Colin Percival's 6.1 to 6.2 binary upgrade script . I am particularly glad to see that Colin's freebsd-update utility is now part of the base system. Second, FreeSBIE 2.0 , a FreeBSD live CD based on FreeBSD 6.2, was just released . I plan to download and try it out, at least in a VM. I'll probably burn a CD to use for testing FreeBSD support on various hardware. Third, this BSDNews.com story pointed me to a site watching developments in FreeBSD 7.0 , called What's cooking for FreeBSD 7.0? . It provides a quick summary of features expected in the next major version of FreeBSD. Keeping with the development theme, the Oct-Dec 2006 status report...

Marty Roesch was kind enough to respond to my recent posts on NSM. We shared a few thoughts in IRC just now, but I thought I would post a few brief ideas here. My primary concern is this: just because you can't collect full content, session, statistical, and alert data everywhere doesn't mean you should avoid collecting it anywhere . I may not have sensors on the sorts of network Marty describes (high bandwidth, core networks) but I have had (and have) sensors elsewhere that did (and do) support storing decent amounts of NSM data on commodity hardware using open source software. I bet you do too. I'm not advocating you store full content on the link to your storage area network. I don't expect Sony to store full content of 8 Gbps of traffic entering their gaming servers. I don't advocate storing full content in the core. Shoot, I probably wouldn't try storing session data in the core. Rather, you should develop attack models for the sorts of incidents ...

It's been 2 1/2 years since my first book was published, although I've been writing and speaking about Network Security Monitoring (NSM) for at least five years. I'm starting to see other people cite my works, which is neat. It also means people are starting to criticize what I wrote, so I need to elaborate on some ideas. The December 2006 ISSA Journal includes an article by Robert Graham titled Detection Isn’t Optional: Monitoring-in-depth . (No, it's not the Robert Graham of Black Ice/ISS fame. This is a different person.) The implication of this article is that NSM is insufficient because it does not integrate SNMP data, event logs, and other sources. I do not disagree with this assessment. The reason I focus on NSM is that I start from the premise of self-reliance . In many enterprises, the security team does not have access to SNMP data from infrastructure devices. That belongs to the networking team. They also might not have access to event logs, sinc...

The image at left is from the first issue of an Intel marketing magazine called Premier IT . I like it because it shows many of the terms I try to describe in this blog, in relationship to each other. In English, the graphic says something like the following: Threats exploit vulnerabilities , thereby exposing assets to a loss of confidentiality/integrity/availability , causing business impact . I disagree that business impact is mitigated by controls . I think those terms were connected to make a pretty cyclical diagram. I would also say that controls mitigate attacks (exploits) by threats, not the threats themselves. Imprisonment mitigates threats. The next diagram shows Intel emphasizes Policy at the base, followed by Training and Education , then Technology and Testing , and finally Monitoring and Enforcement . I think the Training and Education piece is marginally effective at best, at least for the general user population. It's tough enough for security pros to ...

Recently I posted thoughts on Cisco's Self-Defending Network . Today I spent several hours on a Cisco Monitoring, Analysis and Response System (MARS) trying to make sense of the data for a client. I am disappointed to report that I did not find the experience very productive. This post tries to explain the major deficiencies I see in products like MARS. Note: I call this post Operational Traffic Intelligence System Woes because I want it to apply to detecting and resisting intrusions. As I mentioned earlier , hardly anyone builds real intrusion detection systems . So-called "IDS" are really attack indication systems . I also dislike the term intrusion prevention system ("IPS"), since anything that seeks to resist intrusion could be considered an "IPS." Most available "IPS" are firewalls in the sense that anything that denies activity is a policy enforcement system . I use the term traffic intelligence system (TIS) to describe any...

I read the following in the latest SANS NewsBites (link will work shortly): Does anyone on your staff do an excellent job of cleaning out PCs that have been infected by spyware and other malicious software. We are just starting development of a new certification (and related training) for Certified Malware Removal Experts and we are looking for a council of 30 people who have done a lot of it to help vet the skills an dknowledge required for the certification exam and classes. Email cmre@sans.org if you have a lot of experience. This must be the easiest SANS certification of all! The safest way to remove malware is to reinstall from trusted original media (not backups which could be compromised). That doesn't even account for BIOS or other hardware rootkits, but hardly anyone cares about that problem yet. Hopefully SANS will come to the same conclusion that Microsoft already did and drop this idea.

I didn't exactly "read" Self-Defending Networks: The Next Generation of Network Security by Duane DeCapite. Therefore, I won't review the book at Amazon.com. I definitely didn't read a majority of the text, which is a personal requirement for a book review. However, I'd like to discuss the title here. The book has a ton of screen shots and is essentially a big marketing piece for Cisco's Self-Defending Network gear, which includes: Cisco Traffic Anomaly Detector for DDoS identification Cisco Guard for DDoS mitigation Adaptive Security Appliance for firewalling (including IPS) Incident Control System for malware containment with Trend Micro 802.1X for port-based security; note to Cisco: it's not "802.1x" Network Admission Control (NAC) with NAC Appliance or NAC Framework Cisco Security Agent (CSA) for host protection Cisco Security Manager Cisco Monitoring, Analysis and Response System (MARS) for alert management Why do I mentio...

I read the following in the latest SANS NewsBites : Revised Civil Procedure Rules Mean Companies Need to Retain More Digital Data (4 January 2007) The revised Federal Rules of Civil Procedure , which took effect on December 1, 2006, broaden the types of electronic information that organizations may be asked to produce in court during the discovery phase of a trial. The new types of digital information include voice mail systems, flash drives and IM archives. This will place a burden on organizations to retain the data in the event it is needed in a legal case. Section V, Depositions and Discovery, Rule 34 of the Federal Rules of Civil Procedure reads, in part, "Any party may serve on any other party a request to produce and permit the party making the request, or someone acting on the requestor's behalf, to inspect, copy, test or sample any designated documents or electronically stored information - including writings, drawings, graphs, charts, photographs, sound recordings, ...

My old HP DeskJet 970cxi died, so I decided to finally buy a color laser printer. Owning a color laser printer has been sort of a Holy Grail for me. I owned a black-and-white laser printer in 1994, and I always thought the true day of personal desktop publishing would arrive with reasonably priced color laser printers. I bought a Lexmark C530dn at NewEgg.com for slightly more than $500 (when shipping is included). Since I bought the DeskJet several years ago for around $300, this new $500 printer seems the right price. There are cheaper color laser printers from Lexmark and Dell, but I wanted an integrated duplex unit. (I dislike wasting paper and I prefer to carry fewer sheets when possible.) The printer got outstanding CNet scores and I found the Better Buys for Business (.pdf) praise convincing. After lugging the box upstairs (60+ lbs) it took about 15 minutes to set up the printer, install the software on Windows XPSP2 and attach the proper cables. I'm printing thro...

The second of the three security principles listed in my first book is: Many intruders are unpredictable. I think the new Adobe Acrobat Reader vulnerability demonstrates this perfectly. (I'm not calling Stefano Di Paola an intruder; anyone who uses his technique maliciously is an intruder, though.) Who would have thought to abuse a .pdf viewer in such a manner? Read more about the problem here . This event reminds me of soccer goal security .

My Hawke vs the Machine post elicited many comments, all of which I appreciate. I'd like to single out one set of comments for a formal reply here. These are by "DJB," which I highly doubt is Daniel J. Bernstein since the comment ends with "See you at the next ISSA meeting." (DJB lives in Illinois and I live in Virginia.) DJB writes: The topic is not alert-centric vs. NSM, or even passive vs. reactive. The real issue here is Return on Investment for security and Due Care. The cost and lack of common expertise of NSM is why it has not been fully adopted. Every SOC/NOC I’ve ever been in (over 100) suffers the plight you have identified. Furthermore, I could hire a hundred people with your level of expertise or the same number of Gulas, Ranums and Roeschs to perform NSM. The only problem is that the problem would not go away and I would be out a significant amount of money, even if you have “the right forms of data available.” The volume of traffic that we are ...

I write about risk, threat, and other security definitions fairly regularly. Lo and behold I just read a post by someone else who shares my approach. This is a must read . How did you react to the story? A second brother in risk is Gunnar Peterson , who writes in part: When security teams conflate threats and vulnerabilities, the result is confusion. Instead efforts dealing with threats... and vulnerabilities... should be separately optimized, besides both being part of "security"; they don't have that much in common. Oh bravo, especially the old school link to Dan Geer which I should read again.

I received the following from a student in one of my classes . He is asking for help dealing with security issues. He is trying to perform what he calls an "IDS/IPS policy review," which is a tuning exercise. I will apply some comments inline and some final thoughts at the end. If you recall, I was in one of your NSO classes last year. At the end of the day the only place I am able to use everything I learned is at home. This is an example of a security person knowing what should be done but unable to execute in the real world. This is a lesson for all prevention fanboys. You know the type -- they think 100% prevention is possible. In the real world, "business realities" usually interfere. As you are aware with other corporate environments, our company goes by Gartner rating on products and ends up buying a technology where you don't get any kind of data, but just an alert name. So that is a pain within itself. Here we see a security analyst who has been e...