Tuesday, August 25, 2009

Draft Version of New Keeping FreeBSD Applications Up-To-Date

This is a follow-up to my recent post Draft Version of New Keeping FreeBSD Up-To-Date. I updated the draft Keeping FreeBSD Up-To-Date document at http://www.taosecurity.com/kfbutd7.pdf to include new sections on building a kernel and userland on one system and installing on another, and upgrading from one major version of FreeBSD to another via binary upgrades (e.g., 7.1 to 8.0 BETA3, since that just became available).

I have also published another draft document titled Keeping FreeBSD Applications Up-To-Date at http://www.taosecurity.com/kfbautd7.pdf. That is a follow-up to my 2004 article of the same name that use FreeBSD 5.x for the examples.

The new document includes the following.

FreeBSD Handbook
A Common Linux Experience
Simple Package Installation on FreeBSD
Checking for Vulnerable Packages with Portaudit
FreeBSD Package Repositories
Updating Packages by Deletion and Addition
Introducing the FreeBSD Ports Tree
Updatng the FreeBSD Ports Tree
Installing Portupgrade
Updating Packages Using Portupgrade
Removing Packages
Identifying and Removing Leaf Packages
Preparing to Build and Install Packages Using the Ports Tree
Building and Installing Packages Using the Ports Tree: A Simple Example
Building and Installing Packages Using the Ports Tree: A More Complicated Example
Install Packages Built on One System to Another System
Installing Screen Using a Remote FreeBSD Ports Tree
Reading /usr/ports/UPDATING
My Common Package Update Process

As with the last document, this one reflects my personal system administration habits. For example, I use Portupgrade, although others might prefer Portmaster or Portmanager or something else.

If you'd like to read this draft and provide any comments here, I would appreciate them.

On a related note, I'd like to point to the 2006 article The FreeBSD Ports System by Michel Talon. I found it interesting because it takes a deep look at the ports tree and make comparison to Debian systems.

SANS WhatWorks in Incident Detection Summit 2009 Web Site Active

The Web site for the SANS WhatWorks in Incident Detection Summit 2009 is live. I created a rough agenda to provide an idea of the structure of the two-day event. I am still working on speakers but I will probably have too few slots to accommodate all the people I would like to appear! As I secure speakers for the event I will submit them to SANS so they can update the Web site.

The registration link is also active.

Thanks to those of you who posted thoughts to my earier blog post. I also created a twitter.com/sansincdet account, so feel free to post ideas there too.

Saturday, August 22, 2009

Draft Version of New Keeping FreeBSD Up-To-Date

Four years ago I wrote an article titled Keeping FreeBSD Up-To-Date. The goal was to document various ways that a FreeBSD 5.2 system could be updated and upgraded using tools from that time, in an example-drive way that complemented the FreeBSD Handbook.

I decided to write an updated version that starts with a FreeBSD 7.1 RELEASE system and ends by running FreeBSD 7.2-STABLE. Sections include:

FreeBSD Handbook
The Short Answer
Understanding FreeBSD Versions
Learning About Security Issues
Starting with the Installation
Installing Gnupg and Importing Keys
Installing Source Code
Installing CVSup
Applying Kernel Patches Manually
Applying Userland Patches Manually
Using CVSup to Apply Patches
Using Csup to Apply Patches
FreeBSD Update to Upgrade FreeBSD within Versions
STABLE: The End of the Line for a Single Version
What Comes Next?

Looking at the sections, I noted that it might be good to add a section on using FreeBSD Update to upgrade to 8.0, assuming you're starting with a non-7.2-STABLE system. From what I've read, that isn't possible? (Anyone know for sure?)

It would also be nice to publish the final version once 8.0 is RELEASEd so I could incorporate that.

If you'd like to read the document and provide feedback, I'd appreciate constructive comments. The draft is available as a .pdf at http://www.taosecurity.com/kfbutd7.pdf. Thank you.

Friday, August 21, 2009

Renesys Blog on Routing Vulnerabilities

I've been writing about the routing infrastructure monitoring company Renesys for several years. James Cowie's post Staring Into the Gorge contains some real gems:

Here We Go Again.

Imagine an innocent BGP message, sent from a random small network service provider's border router somewhere in the world. It contains a payload that is unusual, but strictly speaking, conformant to protocol. Most of the routers in the world, when faced with such a message, pass it along. But a few have a bug that makes them drop sessions abruptly and reopen them, flooding their neighbors with full-table session resets every time they hear the offending message. The miracle of global BGP ensures that every vulnerable router on earth gets a peek at the offending message in under 30 seconds. The global routing infrastructure rings like a bell, as BGP update rates spike by orders of magnitude in the blink of an eye. Links congest. Small routing hardware falls over and dies. It takes hours for things to return to normal...

At 17:07:26 UTC on Monday, CNCI (AS9354), a small network service provider in Nagoya, Japan, advertised a handful of BGP updates containing an empty AS4PATH attribute...

This one seems to have bitten Cisco's IOS XR, a relatively newborn "from scratch" rewrite of the venerable IOS, destined to run on big iron, like the CSR-1 or the 12000 series...

The global mesh of BGP-speaking routers that we call the Internet has inherent vulnerabilities that stem from the software quality and policy weaknesses of its weakest participants, and the amplification potential of its best-connected participants. Running sloppy software at the edge of the routing mesh (in enterprises, say) is unlikely to give anyone the ability to propagate large amounts of instability or partition the Internet. But closer to the core, I think we have a serious problem to contemplate.

Remember, if you can get just one provider to listen to you, and not filter your announcements, you can get your message into the ear of just about every BGP-speaking router on the planet within about thirty seconds. And if some subpopulation of those routers can be reset, they act as amplifiers for your instability. Power law outage-size distributions are not a myth — they are a logical consequence of the structure of the Internet, the importance of a few key participants in carrying global traffic, and their reliance for interconnection on technologies that are clearly still in the shaking-out-the-obvious-bugs mode.

So that is really cool. I liked the following too:

I was at USENIX last week, and sat in on a great workshop on security metrics, where Sandy Clark gave a somewhat controversial presentation on the interaction between software quality and the timing of exploit appearances...

[O]ne of the strongest predictors of a significantly large time to the emergence of the first zero-day exploit for a new version of software is the degree to which the release represents a substantial rewrite of the code. Doing a rewrite seems to start a "honeymoon period," during which time the system in question is safer from exploitation than it has been in a long time. In fact, the magnitude of the protective effect is so significant, that you might ask yourself whether a dollar spent in pursuit of higher quality code is actually better spent rewriting the code periodically, to whatever quality standard you can achieve.

That sounds like an argument for diversity to me. Writing new software introduces diversity, and the attackers have to decide if they want to spend resources to understand the new target sufficiently to exploit it.

New Must-Read Blog Series from Mike Cloppert

Mike Cloppert has started a series of posts on security intelligence on the SANS Forensics Blog. Part 1 includes multiple worthwhile definitions, and Part 2 follows with a great, correct explanation of risk and its components. Keep your eyes on his section of the blog for at least three more posts. Awesome work Mike.

Thursday, August 20, 2009

Updating FreeBSD Using CVSup through HTTP Proxy

If you've used CVS before, you know that CVS doesn't play well with HTTP proxies. I was looking for a way to run cvsup on FreeBSD behind a proxy when I found a post on the FreeBSD China mailing list. It described using Proxychains with Desproxy to tunnel CVS over a SOCKS proxy through HTTP.

Here's how I followed the instructions in my lab environment.

First I installed Proxychains from the FreeBSD port. You can see my HTTP proxy is port 3128.

freebsd7# setenv HTTP_PROXY
freebsd7# pkg_add -vr proxychains
extract: Package name is proxychains-3.1
extract: CWD to /usr/local
extract: /usr/local/bin/proxychains
extract: /usr/local/bin/proxyresolv
extract: /usr/local/etc/proxychains.conf
extract: /usr/local/lib/libproxychains.so.3
extract: /usr/local/lib/libproxychains.so
extract: /usr/local/lib/libproxychains.la
extract: /usr/local/lib/libproxychains.a
extract: execute '/sbin/ldconfig -m /usr/local/lib'
extract: CWD to .
Running mtree for proxychains-3.1..
mtree -U -f +MTREE_DIRS -d -e -p /usr/local >/dev/null
Attempting to record package into /var/db/pkg/proxychains-3.1..
Package proxychains-3.1 registered in /var/db/pkg/proxychains-3.1

Next I installed Desproxy from source.

freebsd7# mkdir /usr/local/src
freebsd7# fetch http://downloads.sourceforge.net/project/desproxy/desproxy/desproxy-0.1.0-
desproxy-0.1.0-pre3.tar.gz 100% of 51 kB 96 kBps
freebsd7# mkdir /usr/local/desproxy
freebsd7# tar -xzvf desproxy-0.1.0-pre3.tar.gz
x desproxy-0.1.0-pre3/
x desproxy-0.1.0-pre3/Makefile.in
x desproxy-0.1.0-pre3/AUTHORS
x desproxy-0.1.0-pre3/Changes
x desproxy-0.1.0-pre3/config.h.in
x desproxy-0.1.0-pre3/configure
x desproxy-0.1.0-pre3/configure.in
x desproxy-0.1.0-pre3/install-sh
x desproxy-0.1.0-pre3/doc/
x desproxy-0.1.0-pre3/doc/config-en.html
x desproxy-0.1.0-pre3/doc/manual-en.html
x desproxy-0.1.0-pre3/src/
x desproxy-0.1.0-pre3/src/Makefile.in
x desproxy-0.1.0-pre3/src/desproxy-dns.c
x desproxy-0.1.0-pre3/src/desproxy-inetd.c
x desproxy-0.1.0-pre3/src/util.c
x desproxy-0.1.0-pre3/src/desproxy.c
x desproxy-0.1.0-pre3/src/desproxy.h
x desproxy-0.1.0-pre3/src/socket2socket.c
x desproxy-0.1.0-pre3/src/util.h
x desproxy-0.1.0-pre3/src/desproxy-socksserver.c
x desproxy-0.1.0-pre3/INSTALL
x desproxy-0.1.0-pre3/COPYING
freebsd7# cd desproxy-0.1.0-pre3
freebsd7# ./configure --prefix=/usr/local/desproxy
checking for gcc... gcc
checking for C compiler default output... a.out
freebsd7# make install
Using binary dir: /usr/local/desproxy/bin
Using locale dir: /usr/local/desproxy/share/locale
Making directories...
Copying binaries...
desproxy installed
desproxy-inetd installed
desproxy-dns installed
desproxy-socksserver installed
socket2socket installed

* This version lacks locale support *
* locales won't be installed *

* Installation OK *

Before I could start desproxy-socksserver, I needed to edit my Squid proxy configuration. Here's where it gets tricky. If I can control my proxy, can't I figure another way around it? Stay with me for now. I made two changes. I added a variable for port 5999 for CVS:

acl CVS_ports port 5999

Next I added port 5999 as a "safe port":

acl Safe_ports port 5999 # CVS added by RMB 20 Aug 09

Finally I modified what ports were allowed the CONNECT method. By default only 443 is allowed.

# Deny CONNECT to other than CVS ports
http_access allow CONNECT CVS_ports
http_access deny CONNECT !SSL_ports

I thought you might be able to get CVSup to point to port 443 using the -p option, but if you try that you get an error:

Reserved port 443 not permitted

I wonder if this could be removed from the source code?

Assuming you could get CVSup to talk port 443, you could have it point to a host on the Internet under your control. That host could listen on port 443, then forward what it receives to the CVS server using Netcat. I think this would work. I found the following in cvsup-snap-16.1h/client/src/Main.m3

PROCEDURE CheckPort(port: INTEGER) =
IF port = IP.NullPort
OR NOT (FIRST(IP.Port) <= port AND port <= LAST(IP.Port)) THEN
ErrMsg.Fatal("Invalid port " & Fmt.Int(port));
IF port < 1024 THEN
ErrMsg.Fatal("Reserved port " & Fmt.Int(port) & " not permitted");
END CheckPort;

I think if I removed the second check, for "Reserved port", that would remove my problem. To make things easier I just changed 1024 to 10.

To install the modified cvsup-without-gui, I did a 'make fetch' and 'make extract', modified the source, and then did 'make install'.

On a remote host I can run the following using the redir app:

# redir --laddr=[MY_BOUNCE_BOX] --lport=443 --caddr=[CVS server IP] --cport=5999

Then I set the IP in my supfile to be MY_BOUNCE_BOX.

If you can set up this sort of redirection, you can remove the proxy changes outlined earlier.

In the following case, let's assume you can make the necessary proxy changes so you don't have to bounce through a remote host listening for port 443.

Now start desproxy-socksserver. Basically port 1080 is listening on the localhost, and it will forward what it receives to port 3128 (Squid) on the proxy server.

freebsd7# /usr/local/desproxy/bin/desproxy-socksserver 3128 1080

desproxy-socksserver 0.1.0-pre3

(C) 2003 Miguelanxo Otero Salgueiro

TCP port 1080 Bound & Listening
Press to Quit

Now configure Proxychains. Here is my configuration file.

freebsd7# grep -v \# proxychains.conf


chain_len = 1

tcp_read_time_out 15000
tcp_connect_time_out 8000

socks5 1080

There is another problem here. If you can't resolve DNS inside your environment, this will fail. There is a 'proxy_dns' option in Proxychains, but I got an error when using it:

|DNS-response|: freebsd7.localdomain is not exist
|DNS-request| cvsup7.FreeBSD.org
|DNS-response|: cvsup7.FreeBSD.org is not exist
Unknown host "cvsup7.FreeBSD.org"

One way around this is to replace cvsup7.FreeBSD.org, or whatever you want to use, with the IP address of the CVS server. Start desproxy-socksserver.

freebsd7# /usr/local/desproxy/bin/desproxy-socksserver 3128 1080

desproxy-socksserver 0.1.0-pre3

(C) 2003 Miguelanxo Otero Salgueiro

TCP port 1080 Bound & Listening
Press to Quit

Connection request from, port 63950
Connection #0: bidirectional connection stablished

Connection request from, port 53812
Connection #1: bidirectional connection stablished

Connection #0: end of connection
Connection #1: end of connection

Start proxychains and tell it to run cvsup:

freebsd7# proxychains cvsup -g -L 2 /usr/local/etc/freebsd7-example.supfile
ProxyChains-3.1 (http://proxychains.sf.net)
Parsing supfile "/usr/local/etc/freebsd7-example.supfile"
Connecting to cvsup4.FreeBSD.org
Connected to cvsup4.FreeBSD.org
Rejected by server: Access limit exceeded; try again later
Will retry at 17:45:22

That's not cool. That is actually a CVS server error. Let's try new CVSup host in the supfile.

freebsd7# proxychains cvsup -g -L 2 /usr/local/etc/freebsd7-example.supfile
ProxyChains-3.1 (http://proxychains.sf.net)
Parsing supfile "/usr/local/etc/freebsd7-example.supfile"
Connecting to cvsup7.FreeBSD.org
Connected to cvsup7.FreeBSD.org
Server software version: SNAP_16_1h
Negotiating file attribute support
Exchanging collection information
Establishing multiplexed-mode data connection
Updating collection src-all/cvs

So, it worked. I would be interested in knowing if anyone has other methods to get CVSup to work through a HTTP proxy? Most people seem to use SSH tunnels, but what if that is not an option?


It turns out you do NOT need to use desproxy-socksserver. For example, use the following proxychains.conf:

freebsd7# grep -v \# /usr/local/etc/proxychains.conf


chain_len = 1


tcp_read_time_out 15000
tcp_connect_time_out 10000

http 3128

Notice the use of 'http' here instead of 'socks5'. Also, the IP address here is the IP address of the Squid proxy server, whereas the earlier examples pointed to the desproxy-socksserver.

Again I bounce off an Internet host that will send traffic sent to port 443 (to get through the default CONNECT and port settings on Squid):

# redir --laddr=[MY_BOUNCE_BOX] --lport=443 --caddr=[CVS server IP] --cport=5999

Then you can run proxychains by itself.
freebsd7# proxychains cvsup -p 443 -g -L 2 /usr/local/etc/freebsd7-example.supfile
ProxyChains-3.1 (http://proxychains.sf.net)
Parsing supfile "/usr/local/etc/freebsd7-example.supfile"
|DNS-response|: freebsd7.localdomain is not exist
Connecting to MY_BOUNCE_BOX
Connected to MY_BOUNCE_BOX
Server software version: SNAP_16_1h
Negotiating file attribute support
Exchanging collection information
Establishing multiplexed-mode data connection
Updating collection src-all/cvs
Checkout src/COPYRIGHT
Checkout src/LOCKS

If you look at a few packets you can see the setup of the connection.

14:48:04.461432 IP > P 1:39(38) ack 1 win 65535
0x0000: 4500 004e 0ec9 4000 4006 4b3f ac10 8680 E..N..@.@.K?....
0x0010: ac10 0201 fe49 0c38 f690 a51d 2952 c059 .....I.8....)R.Y
0x0020: 5018 ffff 3942 0000 434f 4e4e 4543 5420 P...9B..CONNECT.
0x0030: 0000 0000 0000 0000 0000 0000 003a 3434 MY_BOUNCE_BOX:44
0x0040: 3320 4854 5450 2f31 2e30 0d0a 0d0a 3.HTTP/1.0....
14:48:04.461991 IP > . ack 39 win 64240
0x0000: 4500 0028 d1a2 0000 8006 888b ac10 0201 E..(............
0x0010: ac10 8680 0c38 fe49 2952 c059 f690 a543 .....8.I)R.Y...C
0x0020: 5010 faf0 443f 0000 P...D?..
14:48:04.535724 IP > P 1:40(39) ack 39 win 64240
0x0000: 4500 004f d1a3 0000 8006 8863 ac10 0201 E..O.......c....
0x0010: ac10 8680 0c38 fe49 2952 c059 f690 a543 .....8.I)R.Y...C
0x0020: 5018 faf0 3a58 0000 4854 5450 2f31 2e30 P...:X..HTTP/1.0
0x0030: 2032 3030 2043 6f6e 6e65 6374 696f 6e20 .200.Connection.
0x0040: 6573 7461 626c 6973 6865 640d 0a0d 0a established....
14:48:04.639664 IP > . ack 40 win 65535
0x0000: 4500 0028 0ece 4000 4006 4b60 ac10 8680 E..(..@.@.K`....
0x0010: ac10 0201 fe49 0c38 f690 a543 2952 c080 .....I.8...C)R..
0x0020: 5010 ffff 3f09 0000 0000 0000 0000 P...?.........
14:48:04.837943 IP > P 40:78(38) ack 39 win 64240
0x0000: 4500 004e d1a7 0000 8006 8860 ac10 0201 E..N.......`....
0x0010: ac10 8680 0c38 fe49 2952 c080 f690 a543 .....8.I)R.....C
0x0020: 5018 faf0 6a4f 0000 4f4b 2031 3720 3020 P...jO..OK.17.0.
0x0030: 534e 4150 5f31 365f 3168 2043 5653 7570 SNAP_16_1h.CVSup
0x0040: 2073 6572 7665 7220 7265 6164 790a .server.ready.
14:48:04.839209 IP > P 39:61(22) ack 78 win 65535
0x0000: 4500 003e 0ecf 4000 4006 4b49 ac10 8680 E..>..@.@.KI....
0x0010: ac10 0201 fe49 0c38 f690 a543 2952 c0a6 .....I.8...C)R..
0x0020: 5018 ffff 4731 0000 5052 4f54 4f20 3137 P...G1..PROTO.17
0x0030: 2030 2053 4e41 505f 3136 5f31 680a .0.SNAP_16_1h.

So, you can tunnel CVS through HTTP using proxychains, as long as you bounce off a remote host that listens on port 443. That assumes you have to get around proxy restrictions that only allow CONNECT to port 443.

Three Free Issues of BSD Magazine in .pdf Format

Karolina at BSD Magazine wanted me to let you know that she has posted three free .pdf issues online. The three cover FreeBSD, OpenBSD, and NetBSD. Apparently BSD Magazine has survived a publishing scare and will continue for the foreseeable future. I may also have an article for FreeBSD out soon.

Tuesday, August 18, 2009

Hakin9 04/2009 Issue

I just received a review copy of the 04/2009 Hakin9 magazine. I am most interested in reading part two of Tyler Hudak's article on automating malware analysis. Cartsen Kohler's article on exploiting Windows via printer drivers looks interesting too. Check it out!

Friday, August 14, 2009

Manga Guide to Statistics vs Statistics in a Nutshell

I took statistics classes twice in undergrad (once during the normal school year, a second time during a summer program at another school), and once during my master's program. That was so long ago that I don't remember a lot of what I had to learn. Recently review copies of two books arrived, namely The Manga Guide to Statistics by Shin Takahashi and Trend-pro Co., Ltd and Statistics in a Nutshell by Sarah Boslaugh and Dr. Paul A. Watters.

Both books claim to be the right book to introduce newbies to statistics. You can guess which one does a better job just by looking at the covers. Here's a hint: consider the words "a desktop quick reference" on the O'Reilly title to be false advertising. Too many of the so-called "Nutshell" books published by O'Reilly today are nothing of the sort. They are not like my beloved Unix in a Nutshell, 3rd Ed that helped me navigate the Solaris 7 command line in 1999. No, these days too many "Nutshell" books are just regular titles crammed into the Nutshell form factor.

I think Statistics in a Nutshell should be called "A Guide to Applying Statistics Properly." That would indicate the book's real goal, which is to guide researchers in the realms of data management, research design, and critiquing statistics -- all subjects given their own chapters.

In contrast, The Manga Guide to Statistics is a great introduction for anyone who wants to learn statistics. I admit, I was a little skeptical at first. I don't read manga and any comic books I still own from the 1980s are sealed in their plastic bags, thank you very much. However, I found the approach in the Manga Guide to be compelling and informative. It seems to make a difference when the teacher in the Manga Guide has to "answer" questions posed by a student, and not just speak to readers in the abstract. The Manga Guide also built all of its examples on real life situations, which made the material easier to understand.

As mentioned in a review I saw, the Manga Guide seems to be the perfect book for students in middle school or higher to learn statistics. As a bonus, the Manga Guide includes exercises using Excel, which is much easier than the applications I had to manipulate in the early 1990s. I congratulate No Starch for bringing such an innovative teaching tool to our bookshelves. Check out their entire Manga series for other titles.

GE Is Hiring in Michigan

In June in this post I linked to a speech that GE's CEO gave in Michigan. We're hiring about 1,200 people over the next few years, and the jobs are already appearing at gecareers.com. One of the jobs posted requests an IT Project Manager - Information Technology (Security). This candidate would work in a sister unit to our GE-CIRT doing Identity and Access Management (IAM). If this job looks interesting, please check it out. As other roles in our Corporate security group appear -- especially those in GE-CIRT -- I will let you know.

Thursday, August 13, 2009

Attack Models in the Physical World

A few weeks ago I parked my Ford Explorer (It's not a clunker!!) in a parking garage. On the way out I walked by the pipe shown in the picture at left. It looks like a pipe for carrying a fluid (water maybe?) "protected" by a metal frame.

I think the purpose of the cage is pretty clear. It's deployed to prevent drivers from inadvertently ramming the pipe with their front or rear car bumpers. However, think of all the "attacks" for which it is completely unsuited. Here are the first five I could imagine.

  • Defacement, like painting obscenities on the pipe

  • Cutting the pipe with a saw

  • Melting the pipe with a flame

  • Cracking the pipe with a hammer

  • Stealing water by creating a hole and tube to fill a container

So what if any of these attacks were to happen? Detection and response are my first answers. There's likely a camera somewhere that could see me, my car, and the pipe. Cameras or bystanders are likely to record some detail that would cause the intruder to be identified and later apprehended. Other people in the parking garage are likely to tell someone in authority, or better still, take video or a photo of the intruder in action and then provide that to someone in authority.

So, we can all laugh at the metal cage around this pipe, but it's probably doing just what it needs to do, given the amount of resources available for "defense" and the detection and response "controls" available.

If the defensive posture changed, it would probably not be the result of a security person imagining different attack models against plastic pipes. In other words, it wouldn't be only "decide -> act". Rather, changes would be prompted by observed attacks against real infrastructure. We'd have the full "observe -> orient -> decide -> act" OODA loop. For example, some joker would be seen cutting the pipe using a saw, so patrols and cameras would be enhanced, and possibly wire mesh or plating would be added to the cage to slow down the attacker in time for responders to arrive.

Review of The Myths of Security Posted

Amazon.com just posted my three star review of The Myths of Security by John Viega. From the review:

Let me start by saying I usually like John Viega's books. I rated Building Secure Software 5 stars back in 2005 and 19 Deadly Sins of Software Security 4 stars in 2006. However, I must not be the target audience for this book, and I can't imagine who really would be. The book mainly addresses consumer concerns and largely avoids the enterprise. However, if most consumers think "antivirus" when they think "security," why would they bother reading The Myths of Security (TMOS)?

Incident Detection Mindset

Often you will read or hear about a "security mindset," but this is frequently an "offensive security mindset." This attitude is also called a "breaker" mindset, described in my old post On Breakership. The offensive security mindset means looking at features of the physical or digital worlds and reflexively figuring out ways to circumvent their security or lack of security. Johnny Long is one example of a person with this mindset -- pretty much every place he looks he is figuring out a way to profile or subvert what he sees! To a certain extent this mindset can be taught, although one could argue that truly exceptional offensive security pros have this mindset embedded in their DNA.

It occurred to me today, after writing Build Visibility In, that I have a different mindset. I have an incident detection mindset. Often when I interact with the physical or digital worlds, I reflexively wonder how can I tell if this feature is trustworthy? For example, when I first received my Corporate laptop, I wondered "how can I tell if this box is owned?" When I received my Blackberry, I wondered "how can I tell when this device is owned?" In other words, if the device is compromised, it is not trustworthy. How can I tell?

The prevailing security mindset is a "defensive security mindset," where security people are taught to plan for and resist incidents. This attitude is necessary but not sufficient. We need people who plan for and resist incidents, people who can detect and respond to incidents, and people who can think offensively to assist those who work defensively.

I believe all three of these mindsets can be taught, but of the three I think the incident detection mindset is the rarest. Working to develop an incident detection mindset is one of the goals of this blog, and of posts like this one and the last.

Build Visibility In

Visibility has been a constant theme for this blog. Elsewhere I've used the phrase build visibility in to emphasize the need to integrate visbility requirements into the build and design phases of any technology project. Visibility should not be left as an afterthought. Building security in is required as well, but how can you determine how security is working if you have no visibility?

Based on my experiences with technology deployments since the late 1990s, I've realized that the following cycle defines just about every project I've ever seen.

The cycle is Feature -> Management -> "Security" -> Visibility.

I am seeing this cycle at work in the mobile device space right now. Hardly anyone is thinking about how to determine if a mobile device (Blackberry, etc.) is compromised. The best we can do is imagine the sorts of attacks that might be happening to our mobile infrastructure, without visibility regarding how those devices might already be under attack.

I call this operating only within the Decide -> Act part of the OODA loop (Observe -> Orient -> Decide -> Act). We do it all the time in digital security. I called it Soccer Goal Security in 2005.

Does this cycle resonate with anyone?

Wednesday, August 12, 2009

Question on NSM Scaling

A long-time TaoSecurity Blog reader sent me the following question:

I have a question about scaling NSM in regards to large, complex enterprises that transmit countless gigabytes of data per day.

Last month I interviewed for a position with a large wireless company and the hiring manager was familiar with your work, so as I attempted to extol the value of NSM and explain how I thought that NSM could benefit this organization, I was told by the hiring manager that he felt that NSM worked with small organizations, but did not scale well with organizations of a certain size.

I am curious if you have ever had to counter this type of argument and how you addressed it.

This is a common question. I'll need to address it concisely and precisely in an updated edition of Tao. A few recent posts come to mind, like Requirements for Defensible Network Architecture: Monitored, NSM vs Encrypted Traffic, Plus Virtualization, and Network Security Monitoring Lives. A few principles come to mind.

  • Concentrate on infrastructure you own, not necessarily infrastructure you support. In other words, I don't advocate full NSM for ISPs watching customer links. That may be the issue mentioned in the question, i.e., a wireless company might think NSM is inappropriate for watching customer traffic. I would probably agree with that.

  • Monitor at trust boundaries. The places where the infrastructure you own touches infrastructure you do not own is likely to be the place where you need additional visibility.

  • Monitor what you can, given your technical, political, and legal constraints. You may not be able to continuously capture full content data on 10 Gbps links with commodity hardware, or even specialized hardware. If that is too expensive, then don't do it. However, deploy the capability to capture at those locations when necessary. Better to be prepared than to struggle with workarounds or emergency deployments in a crisis.

  • Solutions can be engineered for almost any environment. I guarantee organizations larger than those in the question are already doing intense monitoring. If you don't believe me, look at the history of wiretapping during the last administration. Outside of that case, organizations like mine are deploying hundreds of sensors around the world. NSM can scale if you engineer it properly and hire people who know what they are doing!

  • Don't make NSM a hammer and every security problem a nail. NSM is one way to gain visibility and situational awareness. It may be worthless to deploy sensors doing traditional NSM on a link that only sees SSL-encrypted traffic between two point systems, or between the Internet and a SSL-only system. In cases like that, the first option might be to deploy host-centric monitoring and logging on the asset in question.

Thank you for questions like these -- please keep sending them. They make good sections for a new book.

Thoughts on Security Careers

Several recent blog posts have discussed security careers. I'll start with Anton Chuvakin's post A Myth of an Expert Generalist:

Lately I’ve run into too many people who [claim to] “know security” or are [claim to be] “security experts.” Now, as some of you recall, I used to do theoretical particle physics before I came to information security. In my physics days, I’d be pretty shocked if I were to meet a colleague in the hallways of the C.N. Yang Institute for Theoretical Physics who would self-identify as “a scientist” or, for that matter, even as “a physicist.” It is overwhelmingly more likely that he would say “quantum chromodynamics” or “lepton number violation in electroweak gauge theories” or “self-ionization of the vacuum” or some such fun thing...

I think this has a lot to do with the fact that the area of security is too new and too fuzzy. However, my point here is that a little common sense goes a long way even at this stage of our industry development. In light of this, next time you meet “a security expert,” ask him what is his area of expertise. If the answer is “security”, run!

Finally, career advice for those new to information security: don’t be a generalist. If you have to be a security generalist, be a “generalist specialist;” namely, know a bit about everything PLUS know a lot about something OR know a lot about “several somethings.” If you ONLY know “a bit about everything,” you’d probably die hungry...

Those are interesting insights. I agree with Anton's characterization of the field as being "too new." Theoretical physics is well over a hundred years old, while digital security is about forty years old.

Jeff Snyder's Security Recruiter Blog posted two good stories recently. The first is Hiring: Why Some Security Jobs Go Unfilled:

I started thinking about why some jobs are open for so long or go unfilled entirely...

A company recently sent a Security Analyst / Security Engineer job description to me for my review. They’ve had the job posted to major job boards for months but can’t seem to find the right person. As I studied the description, I quickly recognized that they were looking for at least two and possibly three different skill sets that typically don’t fit together in one person’s resume.

I pondered why they would create such a difficult expectation that essentially set them up to fail in their quest to find the right security job candidate... [C]ompanies across the nation is a significant squeezing of the belt. CISOs are pressured to deliver more results with less resources. Security professionals have to wear more hats than ever before and they have to be great at nearly everything they do in order to capture the most appealing jobs...

Recruiters don’t create candidates, we find those who already exist. If the person a company wants to hire doesn’t exist or doesn’t exist very often, I may be staring at a search that is set up to fail.

I agree with that statement too, but this idea of wearing so many "hats" is a recipe for failure. Most security people can't keep up with one aspect of the industry, let alone multiple aspects. I wrote about this issue several years ago in More Unrealistic Expectations from CIOs when I raged against the idea of a "multitalented specialist."

My third post again comes from Jeff Snyder, in Conversation: With a CIO regarding his Security Staffing:

The CISO was explaining his company’s need to cut back on staffing levels... [S]omeone came up with the idea that this CIO's company could live with one less information security professional.

As of now, they have one security professional who does security analysis and project management work but not a lot of what he does is considered deeply hands-on technical work.

The other security professional on this CIO's staff is a hands-on technical professional who has very deep technical skills but he is not strong with regulatory compliance, risk management work or work that requires strong interpersonal skills...

My recruiting partner and the CIO came to the conclusion that both security professionals might have to go in order to hire someone who had a broader skill set that included both the business / risk / interpersonal skills and the deeply technical components all wrapped up in one person’s security / technology risk management skill set...

Security professionals in both the present and the future need to bring broad skill sets to prospective employers in order to satisfy the growing demands found in hiring manager’s job descriptions.

Wow. That is a recipe for disaster. Lay off two people who already understand the business in order to replace them with one newbie who is expected to do both jobs? Isn't that the unrealistic expectations problem cited in Jeff's first post?

Tuesday, August 11, 2009

2009 CDX Data Sets Posted

Earlier this year I posted Thoughts on 2009 CDX. Greg Conti just sent me a notice that the West Point Information Technology and Operations Center just published, for free, their Intrusion Detection Labeled Data Sets. They include packet captures generated by NSA Red Team activity, packet captures from West Point defenders, and Snort, DNS, Web server, and host logs. This is great data. Stop using the 1999 DARPA data sets. Please.

Friday, August 07, 2009

SANS Incident Detection Summit in DC in December

Last month I blogged about the SANS Forensics and Incident Response 2009 Summit Round-Up. I am pleased to announce that I will be working with SANS to organize a two day SANS Incident Detection Summit in DC in December. I am working on a preliminary agenda that includes two major themes: network-centric detection and host-centric detection. The Summit will include keynotes, practitioner briefings, tool briefings, vendor briefings, and panels.

As we develop the content I will report it here. I am excited about this event and look forward to seeing you in December. My goal is to "bring detection back", since we all know that detection never really died!

If there are topics you'd like to see at the Summit, feel free to share them here. Thank you.

Update: 9-10 December are the days for the Summit.

Review of IPv6 Security Posted

Amazon.com just posted my five-star review of IPv6 Security by Scott Hogg and Eric Vyncke. From the review:

I've read and reviewed three other books on IPv6 in the last four years: IPv6 Essentials, 2nd Ed (IE2E) in September 2006, Running IPv6 (RI) in January 2006, and IPv6 Network Administration (INA) in August 2005. All three were five-star books, but they lacked the sort of attention to security that I hoped would be covered one day. IPv6 Security by Scott Hogg and Eric Vyncke is the book for which we have been waiting. Although some of the early "philosophical" security discussions (what's a threat, where are they) are lacking, the overwhelming amount of thorough and actionable content makes this book a winner.

Wednesday, August 05, 2009

Blast from the Past

So why a picture of me in uniform from 2000? The answer lies in this article published last month titled Air Force Network Operations begins migration to centralized e-mail, network services:

The Air Force Chief of Staff Gen. Norton Schwartz signed a directive memorandum here recently granting the Air Force Network Operations commander centralized order-issue authority over the operation, defense, maintenance and control of Air Force networks.

As part of an ongoing service-wide cyber operations transformation, the Air Force will establish a centralized user directory and e-mail service known as ADX that will service all Air Force network users.

The changes will be relatively transparent to most network users, but this migration to centralized services will significantly improve security and efficiency on the Air Force Global Information Grid, officials said.

"Major commands and subordinate commanders will no longer 'own' networks, but will be responsible for their portion of the larger AF-GIG," General Schwartz stated in a mass e-mail to Airmen.

I find this fascinating because I was part of a group assembled in late 2000, a few months before I left the Air Force (early 2001), tasked with the same mission. Among other tasks, we were told to centralize Air Force email by June 2001. I think many people in the group photo are laughing hysterically because we knew it was impossible to accomplish anything in the time allotted.

The centralization process has apparently already started:

Keesler Air Force Base, Miss., is the first base to undergo the transformation with 1,800 out of 5,800 users already transferred. Over the next 18 months, the complete migration will include approximately 750,000 users at more than 240 locations around the world.

So, about 11 years after being told to accomplish the same task, the effort will be done! I think there are lessons here for anyone with a similarly large, bureaucratic, turf-centric, distributed, decentralized, global organization.