TaoSecurity Blog

Amazon.com just posted my five star review of Forensic Discovery . I read a pre-publication draft of the book which resulted in my quote on the Addison-Wesley site . From the review: "Farmer and Venema do for digital archaeology what Indiana Jones did for historical archaeology. 'Forensic Discovery' unearths hidden treasures in enlightening and entertaining ways, showing how a time-centric approach to computer forensics reveals even the cleverest intruder. I highly recommend reading this book." In the chapter 7 (available online as a .pdf) Farmer and Venema mention the Veeco Nanotheater . Veeco makes products which can scan the surface of disks at nanotechnology scales. They show the image at right, and describe it as "residuals of overwritten information on the sides of magnetic disk tracks." This demonstrates the difficulty of truly "destroying" digital evidence. Forensic Discovery explains the problem this way: "Although memory ...

Visitors to TaoSecurity.com may notice that the icon appearing in the Web browser address bar has changed from the FreeBSD daemon to the yin-yang "S" pictured at left. I created this icon using the following process, detailed by DHCPDump author Edwin Groothuis . First I used xv to crop the TaoSecurity logo, set the image size at 16x16, and save the image (taosecurity.png) in png format. Next I installed Netpbm using the graphics/netpbm FreeBSD port . Once installed I executed the following command to convert my png file to the desired favicon.ico: orr:/home/richard$ pngtopnm taosecurity.png | ppmtowinicon -output favicon.ico When done I had the icon I needed: orr:/home/richard$ file favicon.ico favicon.ico: ms-windows icon resource - 1 icon I copied the favicon.ico file to my Web root, and then added the following to the HEAD section of my index.html file: <LINK REL="SHORTCUT ICON" HREF="favicon.ico"> I had to clear my Web browser cache to see...

Benjamin D. Thomas posted a very thorough review of The Tao of Network Security Monitoring at LinuxSecurity.com . Benjamin writes: "To be honest, this was one of the best books that I've read on network security. Others books often dive too deeply into technical discussions and fail to provide any relevance to network engineers/administrators working in a corporate environment. Budgets, deadlines, and flexibility are issues that we must all address. The Tao of Network Security Monitoring is presented in such a way that all of these are still relevant. One of the greatest virtues of this book is that is offers real-life technical examples, while backing them up with relevant case studies. Network security engineers, system administrations, and security management will find value in this book. It is a must-read for anyone interested in getting into the field, but would still be useful as a reference for the experienced expert."

Last month I found Meling Mudin's IDS blog , and learned of Jose Nazario 's tool Flowgrep . Flowgrep is written in Python . It is similar to Ngrep , which I addressed in my first book . Ngrep is packet-oriented, meaning the strings for which Ngrep searches must all appear in a single packet. If you search for 'bejtlich', and 'bejt' is in one packet and 'lich' another, then Ngrep won't find anything. Flowgrep, in contrast, is conversation-oriented. Flowgrep assembles TCP sessions, as well as pseudo-sessions for UDP and ICMP. Flowgrep will rebuild a conversation where 'bejt' is in one packet and 'lich' another, and report seeing 'bejtlich'. Flowgrep relies on Mike Schiffman's Libnet and Mike Pomraning's Pynids , a Python wrapper for Rafal Wojtczuk's Libnids . Mike was kind enough to work with me over the last week to get Pynids operational on FreeBSD 5.3. Here's how I ended up with a working Flowgrep ...

In late 2003 I published Dynamic Duo Discuss Digital Risk . This was my light-hearted attempt to reinforce the distinction between a threat and a vulnerability . Specifically, a threat is a party with the capabilities and intentions to exploit a vulnerability in an asset. A vulnerability is a weakness in an asset that could lead to exploitation. An intruder (the threat) exploits a hole (the vulnerability) in Microsoft IIS to gain remote control of a Web server. In other words, threats exploit vulnerabilities. This is a simple concept, yet it is frequently confused by security prophets like Bruce Schneier in Beyond Fear . Now SANS is making the same mistake in the latest Incident Handler's Diary . In a posting to announce work on the upcoming SANS Top 20 List , the Diary calls the new report the "SANS CRITICAL INTERNET THREATS 2005" and says: "SANS Critical Internet Threats research is undertaken annually and provides the basis for the SANS 'Top 20...

I read in the latest SANS NewsBites that UC San Diego suffered another intrusion in November 2004, jeopardizing the personal information of about 3,500 people who had taken courses at UCSD Extension. This incident follows a well-publicized intrusion in April 2004 putting at risk personal data on 380,000 people . In both cases UC appears to have caught unstructured threats, as each intruder used the systems as warez depositories for pirated movies and music. I was shocked by this claim concerning the latest intrusion: "Officials said it took two months to notify those who were affected because officials first needed to determine the extent of the breach." This is exactly why I promote network security monitoring as a means to rapidly scope the extent of intrusions. First, generating indicators and warnings in the form of alert data (usually from IDSs) and statistical data gives security professionals a good chance of identifying an intrusion as it happens or shortly th...

Snort 2.3.0 has been released . There appear to be only bug fixes and documentation updates since RC2 arrived last month. At the moment the online manual still shows 2.2.0, but the .pdf packaged with the tarball is the 2.3.0 version. I have not seen any problems with RC2, so I believe the upgrade process should be smooth. I will get to work on an updated Sguil installation guide shortly, as I've accumulated enough minor fixes to warrant an update. In related news, Sourcefire was named Frost & Sullivan’s 2005 Network Security Infrastructure Protection Company of the Year . From the announcement: "Frost & Sullivan presents this Award to the company that demonstrated excellence in all operations. Sourcefire is recognized for its numerous achievements including unique product strategy, important technological developments, and significant gain in market share." Congratulations Sourcefire!

Those of you running the FreeBSD 4.x tree will be happy to know that FreeBSD 4.11 is now available. The release announcement calls 4.11 "the latest release of the FreeBSD Legacy development branch" and states the following: "FreeBSD 4.11 will become the first 'Errata Branch.' In addition to Security fixes other well-tested fixes to basic functionality will be committed to the RELENG_4_11 branch after the release... This is expected to be the last release from the RELENG_4 branch." I am no longer running any 4.x systems and have migrated everything to 5.3.

Last year when US Senator Ted Kennedy was detained for being on a no-fly list , I discussed his plight in relation to intrusion detection system "false positives." If an IDS is operating correctly, every alert it sees is the result of an action it was programmed to take. In other words, when a functioning IDS sees "cmd.exe", it reports seeing "cmd.exe". It doesn't matter if the appearance of "cmd.exe" on the wire is not part of an actual intrusion; a rule to alert on "cmd.exe" does not cause "false positives" if the IDS reports seeing "cmd.exe". A real false positive involves the IDS reporting "cmd.exe" when no such content passed on the wire. Therefore, there are no such things as false positives. Blame the signature writer or IDS developer, not the IDS. Let's move from the realm of IDS false positives to the land of canine false positives. Yesterday the US Supreme Court issued its opinio...

I highly recommend that FreeBSD enthusiasts read Scott Long 's interview at Onlamp.com . He succinctly explains FreeBSD's new Symmetric MultiProcessing Next Generation (SMPng) code and answers good questions posed by interviewer Federico Biancuzzi. Since I maintain multiple Dell PowerEdge 750 servers with Hyper-Threading Technology (HTT), I found Scott's comments on gains from HTT to be interesting. It seems that HTT will only be useful once the new ULE scheduler is equipped to make use of HTT and ULE replaces the 4BSD scheduler. Scott says: "The other design goal of ULE was to have it map out and understand the CPU topology and make good scheduling choices for features like Hyper-Threading. Unfortunately, to my knowledge this work is not yet complete. [Frederic asks:] Does SMPng improve performance on Intel Hyper-Threading capable CPUs? Scott Long: As of right now, very little. The scheduler really needs to be aware of Hyper-Threading and schedule threads and p...

This morning I read the new @RISK: The Consensus Security Alert from SANS and friends. It begins with this comment: "Prediction: This is the year you will see application level attacks mature and proliferate. As hackers focus more on applications, Oracle may start competing with Microsoft as the vendor delivering software with the most critical vulnerabilities." I hear this focus on "applications" constantly, but this is old news. First look at the problem by separating the operating system (OS) kernel from the OS applications. If we look at vulnerabilities in this respect, "applications" have been under attack for decades. Perusing the CERT Advisories list (transitioned to the US-CERT's Technical Cyber Security Alerts in 2004), we see warnings about application vulnerabilities since 1988. For example, in December 1998 we have CA-1988-01: ftpd Vulnerability . You might say that my separation of OS kernel and OS applications doesn't c...

I just read an article titled Microsoft Turns to External Patch Testers . The goal is "is to provide a small number of dedicated external evaluation teams with access to the [beta] patches to test for application compatibility, stability and reliability in simulated production environments." This article cites a Microsoft rep saying "'This is a very controlled program... We have only invited participants with whom we have a close relationship, where we are sure that confidentiality will be maintained.'" This comment makes me question if Microsoft understands what it is doing: Stephen Toulouse, program manager at the Microsoft Security Response Center, "made it clear that the outside testers had no access to information on the vulnerability addressed by the patch. 'They're evaluating the updates in a private, closed-lab environment. They are required to sign an NDA [nondisclosure agreement] and they don't ever know what the patch is co...

Last night I started working on my next book: Extrusion Detection: Security Monitoring for Internal Intrusions . The goal of this book is to help security architects and engineers control and instrument their networks, and help analysts investigate security events. Extrusion Detection is a sequel to my first book , The Tao of Network Security Monitoring: Beyond Intrusion Detection . Extrusion Detection explains how to engineer an organization's internal network to control and detect intruders launching client-side attacks. Client-side attacks are more insidious than server-side attacks, because the intruder targets a vulnerable application anywhere inside a potentially hardened internal network. A powerful means to detect the compromise of internal systems is to watch for outbound connections from the victim to systems on the Internet operated by the intruder. Here we see the significance of the word "extrusion" in the book's title. In addition to watching c...

I'd like to thank higB of secureme.blogspot.com for reminding me to register for ShmooCon 2005 . This is "an all-new, annual East coast hacker convention hell-bent on offering an interesting and new atmosphere for demonstrating technology exploitation, inventive software & hardware solutions, as well as open discussion of critical information security issues." The program looks great, and you can't beat the $199 price tag (pay before 1 Feb) for a 3 day con (Fri 4 Feb - Sun 6 Feb). If you plan to join me for ShmooCon in DC, reply to this post.

Today in #snort-gui on irc.freenode.net, Marty Roesch of Snort fame explained how Snort handles stream reassembly: roesch: when stream4 is doing it's thing it queues the tcp segments as they come in roesch: in stream4 we actually queue the entire packet and keep a pointer to the payload to management reassembly roesch: "flushing" is what happens when we accumulate a certain number of bytes on a stream that's in excess of the "flush point" for that stream roesch: when we flush, we reassemble the segments into a pseudopacket and run it back thru the preprocessor stack and detection engine roesch: if there's a detect, we ask stream4 to log all the queued *packets* on the stream roesch: the first packet gets identified as the attack packet and the rest of them are tagged off of that event roesch: so if you're detecting on "foobar" and it's been spread across three packets as "fo" "ob" "ar" then you're goin...

When I last wrote about buying a Mac mini , I wondered if it would boot headless. This helpful blog entry explains that the Mac mini will indeed boot headless. I don't plan to buy a Mac mini until Mac OS X Tiger arrives later this year. If I need help installing X on a Mac, I can reference Configuring and Running X11 Applications on Mac OS X . If I need an office suite, I'll use NeoOffice .

The FreeBSD July-December 2004 Status Report is now available. I found the following entries to be important. From Release engineering : " FreeBSD 5.4 release engineering will start in March , and FreeBSD 5.5 release engineering will likely start in June . These releases are expected to be more conservative than previous 5.x releases and will follow the same philosophy as previous -STABLE branches of fixing bugs and adding incremental improvements while maintaining API stability. For the 6-CURRENT development branch as well as all future development and stable branches, we are planning to move to a schedule with fixed timelines that move away from the uncertainty and wild schedule fluctuations of the previous 5.x releases. This means that major branches will happen at 18 month intervals, and releases from those branches will happen at 4 month intervals . There will also be a dedicated period of testing and bug fixing at the beginning of each branch before the first release is ...

The February issue of Sys Admin magazine features a new article I wrote titled More Tools for Network Security Monitoring . It discusses Edwin Groothuis' Dhcpdump , Matt Shelton's Passive Asset Detection System (PADS), and John Curry's Security Analyst Network Connection Profiler (SANCP). On a related note, the January 2005 issue of Information Security magazine mentions this blog, along with two others ( Bruce Schneier and Mikko Hypponen ), as reader favorites. Thank you to the anonymous contributor who said "Richard Bejtlich provides some of the best insights into network security... his blog has given me an ongoing extension to his book ." Some of my favorite blogs include Aaron Higbee and friends , Chris Reining , David Bianco , Anton Chuvakin , Dru Lavigne , and Beau Monday .

Earler this month Slashdot announced the publication of Matt Blaze 's new paper Safecracking for the Computer Scientist ( .pdf , 2.5 MB). Bruce Schneier's blog pointed me to an alt.locksmithing thread where locksmiths debate full disclosure in light of Matt's article. The thread starts with the usual defense of "security through obscurity" one might expect: "As many of you know Matt Blaze a professor at Pennsylvania University has published an article that reveals proprietary techniques of safe penetration. It was featured on well known hacker website recently, and it came to our attention on Saturday. It includes information normally reserved to the trade, for good reasons that need not be discussed here. The article is available to the general public without any restrictions whatsoever. We as professionals in the security field are outraged and concerned with the damage that the spread of this sensitive information will cause to security and to ...

SecurityFocus writer Kevin Poulsen continues to provide some of the best original security reporting in his new story Hacker penetrates T-Mobile systems . This is an absolutely fascinating must-read. From the story: "By August 5th the agents already had a good idea what was going on, when Ethics made a fateful mistake. The hacker asked the Secret Service informant for a proxy server -- a host that would pass through Web connections, making them harder to trace. The informant was happy to oblige. The proxy he provided, of course, was a Secret Service machine specially configured for monitoring, and agents watched as the hacker surfed to "My T-Mobile," and entered a username and password belonging to Peter Cavicchia, a Secret Service cyber crime agent in New York. Cavicchia was the agent who last year spearheaded the investigation of Jason Smathers, a former AOL employee accused of stealing 92 million customer e-mail addresses from the company to sell to a spammer. The...

Amazon.com just published my four star review of Gray Hat Hacking . From the review: "'Gray Hat Hacking' (GHH) is positioned as a next-generation book for so-called ethical hackers, moving beyond the tool-centric discussions of books like 'Hacking Exposed.' The authors leave their definition of 'gray hat' unresolved until ch 3, where they claim that a 'white hat' is a person who 'uncovers a vulnerability and exploits it with authorization;' a 'black hat' is one who 'uncovers a vulnerability and illegally exploits it and/or tells others how to;' and a 'gray hat' is one who 'uncovers a vulnerability, does not illegally exploit it or tell others how to do it, but works with the vendor.' I disagree and prefer SearchSecurity.com 's definitions, where white hats find vulnerabilities and tell vendors without providing public exploit code; black hats find vulnerabilities, code exploits, and maliciously attack v...

I wrote about the Metasploit Framework in April 2004 . The Metasploit Framework is an advanced open-source platform for developing, testing, and using exploit code. This week they released version 2.3, which offers 3 user interfaces, 46 exploits, and 68 payloads. One of the more interesting additions is the Meterpreter ( .pdf ). This is a replacement for calling cmd.exe on Windows after an exploit succeeds. Windows support is currently offered and UNIX (to replace calling /bin/sh) is planned. The Meterpreter is extensible, so you can add features once you gain control of the target. You can browse the exploits and payloads using their Web-based interface.

I think Sun and Apple are doing real innovation in the commercial software and hardware spaces, unlike many of their competitors. I already own an old Sun Ultra 30, and I plan to buy several Sun Ray thin clients at work. I've been looking for an excuse to get an Apple system of some kind ever since Mac OS X was released. Since I run FreeBSD on my Thinkpad a20p laptop, I don't need another desktop or laptop system. I've also vowed to never buy another tower form factor PC again. It's either small form factor, laptops, or rackmounts from here on. That left buying an Apple Xserve , which was more horsepower than I could justify buying. Today, Apple released the Mac mini , pictured at right and in Steve Jobs' hands above. This looks like a great little box. Take a look at the back side below. Although it only has one built-in Ethernet port, those two USB 2.0 ports say "additional NICs" to me, assuming I can get a USB-based NIC to work with OS X. ...

In late 2003 I described how part of my home network looked: cable modem - cisco router - freebsd fw/gw - cisco switch - clients This past weekend I decided to remove the firewall/gateway from the picture. When the router is deployed like this, it's called a "router on a stick." cable modem - cisco router - cisco switch - clients In that late 2003 story I explained how I set up 802.1q on the FreeBSD system to pass traffic between VLANs on the Cisco switch. Without that FreeBSD in place, I needed to configure my Cisco 2651XM router to exchange inter-VLAN traffic. Luckily this Cisco document came to the rescue. The process was fairly simple. I administered the router via console cable, so none of my changes resulted in being locked out of one of the interfaces. I don't recommend letting anyone be able to connect to a Cisco router interface, in any case. (For a great presentation on router security, check out this .pdf of a presentation by Sean Convery an...

Today's Slashdot features Security Holes Draw Linux Developers' Ire . Essentially the GRSecurity Linux security patch developers are upset about the lack of response to their discovery of Linux kernel vulnerabilities. This article by Brad Spengler features the 31337 technique used to find the holes: "Using 'advanced static analysis': cd drivers; grep copy_from_user -r ./* |grep -v sizeof I discovered 4 exploitable vulnerabilities in a matter of 15 minutes. More vulnerabilities were found in 2.6 than in 2.4. It's a pretty sad state of affairs for Linux security when someone can find 4 exploitable vulnerabilities in a matter of minutes." I am disappointed that this is the case. I am not a kernel developer so I won't comment on the difficulties associated with removing these sorts of vulnerabilities. However, some of those that are kernel developers do not seem to be heeding the warnings in books like Building Secure Software , which I reviewed...

Today I moved my local name resolution duties from a FreeBSD 4.x system to a FreeBSD 5.3 system. I found the FreeBSD Handbook sparse reading, but this article gave a few more pointers. Here's what I ended up doing. The first step is to recognize that /etc/namedb is a symlink: janney:/home/richard$ ls -al /etc/namedb lrwxrwxrwx 1 root wheel 23 Nov 8 09:14 /etc/namedb -> ../var/named/etc/namedb This supports the new default of running BIND in a jail. Next I created the localhost-v6.rev and localhost.rev files in /var/named/etc/namedb/master. janney:/home/richard$ cd /var/named/etc/namedb/ janney:/var/named/etc/namedb$ sudo sh make-localhost I altered the serial numbers by adding '01' to the end to allow 99 edits per day. (Using the default '20050109' yields one edit per day, if you want your serial number to be related to the day you change it. This is totally optional but I find it helpful.) janney:/var/named/etc/namedb/master$ cat localhost.rev ; ...

When I worked incident response for Foundstone , my boss Kevin Mandia taught me about "investigative leads." This is a Bureau /law enforcement term for items which are recognized as important in a report but require additional scrutiny. I have several network security monitoring investigative leads which I have not yet had time to follow. I list them here in the event one or more of my readers have checked them out: In November Dave Aitel of Immunity, Inc. posted an http://www.derkeiler.com/Mailing-Lists/securityfocus/focus-ids/2004-11/0067.html">announcement of his company's CANVAS Reference Implementation (CRI). CANVAS is a penetration testing toolkit consisting of private exploits written by Immunity, Inc. The CRI is a subset of CANVAS, available for free under NDA, aimed at those wishing to test IDS and layer 7 firewalls (aka "IPS"). I plan to try this out soon, but don't expect public results due to the NDA. There's an extended fo...

Today is the 2nd birthday of the TaoSecurity blog. Thank you to all of my readers. The primary purpose of this blog is to be a "hard drive for my brain." In other words, I mainly record how I accomplish certain tasks, or I put context around security events and related developments. I hope you find the content useful and relevant.

Several people have asked me to comment on George McGarry's Benchmark Comparison of NetBSD 2.0 and FreeBSD 5.3 . My initial reaction to this article is disappointment. I am not upset because the author says his "results indicate that NetBSD has surpassed FreeBSD in performance on nearly every benchmark and is poised to grab the title of the best operating system for the server environment." I am disappointed that the author has decided to use his "results" in a divisive manner. Rather than seek to learn from each BSD project and potentially compete better with Linux or Windows, George decides to drive a wedge between the NetBSD and FreeBSD communities. Regarding the nature of the delivery of this report, George demonstrates he apparently doesn't know how to use email. Instead of posting his findings himself, he relies on Hubert Feyrer to post the abstract for the report in multiple newsgroups, like freebsd-advocacy , freebsd-performance , netbsd-advoca...

Amazon.com just posted my five star review of Building Secure Software . From the review: "'Building Secure Software' (BSS) is an excellent book. I can't believe it was published in the fall of 2001, and I've only gotten to it now. Negative reviewers should remember that a single book can't address every security topic under the sun. BSS is the first of several titles by authors Viega and McGraw; those looking for additional details can peruse their later books."

Although the FreeBSD Handbook offers a VPN over IPSec section, it doesn't describe the scenario I face when deploying network security monitoring sensors. That document also references commands that no longer exist in FreeBSD 5.3, like 'gifconfig.' My architecture looks like this (all IP addresses are obfuscated): remote sensor 'fedorov' ------------- interface em0 public management IP 18.235.153.37 | Internet | VPN concentrator and NAT gateway 'forsberg' ------------- interface em0 external public IP 78.172.25.27 --- interface em1 internal private IP 192.168.1.1 | switch | monitoring backend ------------------ interface em0 internal private IP 192.168.1.10 I need to encrypt communications from the sensor to the monitoring backend. This can involve multiple individual sockets. I don't like to use OpenSSH port forwarding or Stunnel because I must set...