Wednesday, December 31, 2003

Administering Servers with Webmin

I've been trying Webmin on FreeBSD, Solaris 8, and HP-UX 11i today. Once I repatriate my AIX box from my employer, I intend to install Webmin on it as well. For FreeBSD, I installed the package provided by the FreeBSD project. For Solaris, I used the package provided by Webmin. For HP-UX, I downloaded the tarball and installed from source. I tried the version packaged by HP with their port of Apache and Tomcat, but couldn't get it to install on its own. Webmin uses its own "miniserv.pl" so Apache is not needed.

Webmin is a Web-based, cross-platform system administration tool. Although I'm not a huge fan of Web-based interfaces, Webmin is slick. I recommend installing the Swelltech theme, which is cleaner than the default.

Using Webmin I successfully installed the lsof package for FreeBSD and Solaris. I didn't have any luck with the same package for HP-UX. Webmin allows you to run commands through the Web browser, so once lsof was installed on FreeBSD and Solairs I ran it and viewed the output in the browser.

One of my favorite aspects of Webmin is its package browsing features. You can peruse the installed software very easily. Webmin also makes browsing logfiles a snap.

Webmin's OS support is vast, since it's written in Perl. I noticed however that not all actions work as well as I'd like. For example, when trying to edit the FreeBSD bootup scripts, I was only given the option of editing /etc/rc.local. I expected to edit /etc/rc.conf or scripts in /usr/local/etc/rc.d. I may have been looking in the wrong place, though.

The two books written about Webmin are both available on the Web for free. The book written by Webmin's author is part of the Bruce Perens Open Source Series and can be downloaded in zipped .pdf. Swelltech's CEO wrote a book and provides it here.

Besides Webmin, there's also Usermin, designed for end-user work. It's good for reading mail. Virtualmin is a virtual hosting management system developed by the Webmin author under contract with Swelltech.

Monday, December 29, 2003

Security 101 Book

Today I was asked for my recommendation for a "security 101" book. I hadn't given the subject much thought, although I think Ed Skoudis' Counter-Hack is a great place to start. I looked around my office and found a book Addison-Wesley sent me last year: Internet Site Security by Erik Schetina, Ken Green, Jacob Carlson. After thumbing through the book, I've decided it's excellent. I won't review it on Amazon.com, since my policy is to only review books I've read. Still, a mention here is worthwhile.

This book is so solid I adopted its "assess -> protect -> detect -> respond" security process model to replace the "plan -> prevent -> detect -> respond" version in my own book, just to avoid reinventing the wheel. They also correct state the risk equation as "risk = threat X vulnerability X asset value." If you're looking to get your feet wet in security, or if you're a manager who needs to learn the fundamentals, Internet Site Security is a fine starting point.

Using Sysmon to Detect Faulty Hardware

No sooner had I posted the entry on Sysmon than it detected a network problem. Two of my systems were unreachable. They both sat of a DMZ leg of my gateway. After troubleshooting at various layers I narrowed the issue down to a faulty NIC in the gateway. How often does that happen? Unfortunately the bad NIC is a Intel PRO/100+ Dual Port Server Adapter (PILA8472). When trying to ping out from the NIC to the DMZ, here's the sort of traffic the NIC generated:

00:39:18.628691 192.168.60.1 > 192.168.60.3: icmp: echo request
00:39:19.638731 0:0:0:0:0:0 > 0:0:0:0:0:0 sap 00 I (s=0,r=0,C) len=80
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
00:39:20.648696 0:0:0:0:0:0 > 0:0:0:0:0:0 sap 00 I (s=0,r=0,C) len=80
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
00:39:21.658706 192.168.60.1 > 192.168.60.3: icmp: echo request
00:39:22.668711 192.168.60.1 > 192.168.60.3: icmp: echo request
00:39:23.678706 192.168.60.1 > 192.168.60.3: icmp: echo request
00:39:24.688706 192.168.60.1 > 192.168.60.3: icmp: echo request
00:39:25.698714 0:0:0:0:ff:54 0:0:0:0:0:0 2410 98:
8d44 2414 50f7 4054 0000 0200 7503 8e68
14b8 5801 0000 50cd 80eb fe90 ff54 2410
8d44 2414 50f7 4018 0000 0200 7503 8e68
44c7 404c 16d5 0100 b858 0100 0050 cd80
ebfe 89f6 2c8f 1128 0100 0000 0cfe bfbf
0100 0000

That is truly bizarre. I replaced the NIC with an Adaptec ANA-62044 PCI quad NIC.

FreeBSD on Laptops

I thought the best I could do for help running FreeBSD on laptops was Linux on Laptops, until I learned of the FreeBSD Laptop Compatibility List. This site even had an entry for my Thinkpad a20p. There's an article at Freebsd.org and another database of information also available.

I've had various versions of FreeBSD running on this laptop since I tried installing FreeBSD 4.1.1. I plan to install FreeBSD 5.2 REL once issues in the todo list are solved.

Saturday, December 27, 2003

Understanding Snort DNS TTL Alerts

While reading a recent Network Computing magazine article, I noticed an interesting discussion of "DNS-based route optimizers." These sounded like the products which confused IDS operators four years ago. I read about it in an earlier NWC article.

This December 2003 article states:

"Handling external Web requests... is accomplished by advertising a low DNS TTL of about 10 seconds. This forces the end user's DNS server to request an updated IP address every 10 seconds... the device will provide the external IP address that will provide the best path through the network."

The whole idea with DNS-based systems is to trick clients into visiting the server that's "closest" to them in Internet space. By "closest" I mean the one offering the lowest latency. It's a performance enhancement for visitors.

NWC's graphic does a nice job explaining the system.

Notice the mention of "probes." The review mentions ICMP and TCP-based "probes" in its product feature list. NSM analysts might see these probes and consider them suspicious, when they're normal and harmless.

This made me think of recent alerts I'd seen in Sguil, based on this Snort rule:

alert udp $EXTERNAL_NET 53 -> $HOME_NET any
(msg:"DNS SPOOF query response with TTL of 1 min. and no authority";
content:"|81 80 00 01 00 01 00 00 00 00|";
content:"|c0 0c 00 01 00 01 00 00 00 3c 00 04|";
classtype:bad-unknown; sid:254; rev:3;)

This rule was written to detect intruders responding to a victim's DNS query before the legitimate DNS server does. Whether this is worthwhile is debatable, especially since the rule uses the DNS TTL of exactly one minute. An intruder who uses 59 or 61 seconds evades this rule.

Here is Ethereal's look for one of the DNS responses which triggered this Snort rule. I queried for www.orbitz.com, and the DNS record it returned had a TTL of 1 minute and no authority records. Because the DNS TTL is so low, www.orbitz.com might be using a DNS optimization scheme like that mentioned in NWC.
Compare that to a response for images.amazon.com, where the TTL for the first record is 0, the second has is 5 hours 32 minutes, and the third is 20 seconds. Amazon.com appears to use Akamai extensively:

Finally, keep those responses in mind when looking at the DNS info for a smaller company like Sourcefire that doesn't need to play DNS tricks:

This mini-case study shows how keeping current on Internet infrastructure products helps NSM analysts understand their alerts.

Ways to Install FreeBSD

While perusing the newgroups at unix.derkeiler.com, I learned a new way to get FreeBSD. The FreeBSD Project publishes .iso images of its release software, like 4.9 REL or 5.1 REL. Easy enough. Mirrors for these distributions are available at FreeBSD mirrors.

However, I discovered the FreeBSD Snapshots site offers .iso images of the latest version of each tree, e.g., 4-stable and 5-current. You can download the .iso and finish with a system running the newest FreeBSD, assuming they work on your hardware. If you're more conservative, they maintain a "security release" of the 4.8 distribution as well, which right now is 4.8-RELEASE-p13.

You can even finger their server to learn the newest builds available there:

finger @snapshots.jp.freebsd.org

[snapshots.jp.freebsd.org]
FreeBSD/i386:
The latest version of FreeBSD -CURRENT is: 5.2-CURRENT-20031227-JPSNAP
The latest version of FreeBSD 4-STABLE is: 4.9-STABLE-20031202-JPSNAP
FreeBSD/alpha:
The latest version of FreeBSD -CURRENT is: 5.2-CURRENT-20031227-JPSNAP

More information: finger info@snapshots.jp.FreeBSD.org
Service index: finger help@snapshots.jp.FreeBSD.org

I also learned of two new projects to create CD-ROM based FreeBSD systems, like Knoppix: FreeSBIE and the FreeBSD live-FS project. The original live CD project is LiveCD, which exists in the ports tree.

In miscellaneous news, I tried two new MBLA3300 Intel PRO/100 CardBus II PCMCIA NICs on my Thinkpad a20p running FreeBSD 5.1 REL. I intend to use them for a mobile NSM platform. I found that they were recognizes as fxp0 and fxp1, but only if I booted with one NIC in the top PCMCIA slot or both in the two slots. One NIC in the bottom slot didn't work!

I'm going to put 5.2 REL on the laptop when the new release is published next month. I may followed this thread's guidance on disabling ACPI and enabling APM. I learned how to create a restore partition here. If my XFree86 fonts are ugly, I'll try the guidance in the XFree86 Font De-uglification HOWTO. At some point I may buy a new laptop with no OS installed, like this.

Friday, December 26, 2003

Adding a New Disk in NetBSD

People complain about FreeBSD's '/stand/sysinstall' program, but I wish I could have used it yesterday when adding an 8 GB HDD to my NetBSD box. I loosely followed the official documentation but laughed when I read "Now we create some disklabel partitions, editing the tempfile as already explained. The result is...", followed by a disklabel output created from scratch! This reminded me of the "intuitively obvious" phrase from my college calculus books.

Here's how I did it. This is what the disk looked in dmesg output:

wd1 at pciide0 channel 0 drive 1:
wd1: drive supports 16-sector PIO transfers, LBA addressing
wd1: 8063 MB, 16383 cyl, 16 head, 63 sec, 512 bytes/sect x 16514064 sectors
wd1: 32-bit data port
wd1: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 2 (Ultra/33)
wd1(pciide0:0:1): using PIO mode 4, Ultra-DMA mode 2 (Ultra/33) (using DMA data
transfers)

First I ran fdisk. Notice the BIOS never seems to get the HDD geometry correct. Luckily the OS makes good guesses and I remembered the geometry when I saw it on the HDD label:

bash-2.05b# fdisk -u wd1
Disk: /dev/rwd1d
NetBSD disklabel disk geometry:
cylinders: 16383 heads: 16 sectors/track: 63 (1008 sectors/cylinder)

BIOS disk geometry:
cylinders: 1023 heads: 255 sectors/track: 63 (16065 sectors/cylinder)

Do you want to change our idea of what BIOS thinks? [n] y
BIOS's idea of #cylinders: [1023] 16383
BIOS's idea of #heads: [255] 16
BIOS's idea of #sectors: [63] 63
Disk: /dev/rwd1d
NetBSD disklabel disk geometry:
cylinders: 16383 heads: 16 sectors/track: 63 (1008 sectors/cylinder)

BIOS disk geometry:
cylinders: 16383 heads: 16 sectors/track: 63 (1008 sectors/cylinder)

Are you happy with this choice? [n] y
Partition table:
The data for partition 0 is:
sysid 11 (Primary DOS with 32 bit FAT)
start 63, size 5445972 (2659 MB), flag 0x80
beg: cylinder 0, head 1, sector 1
end: cylinder 338, head 254, sector 63
Do you want to change it? [n] y
sysid: [11] 169
start: [63]
size: [5445972] 8000M
8000M is not a valid decimal number.
size: [5445972] 16384000
Explicitly specify beg/end address? [n] n
sysid 169 (NetBSD)
start 63, size 16384000 (8000 MB), flag 0x80
beg: cylinder 0, head 1, sector 1
end: cylinder 894, head 0, sector 31
Is this entry okay? [n] y
The data for partition 1 is:
sysid 15 (Ext. partition - LBA)
start 5446035, size 11020590 (5381 MB), flag 0x0
beg: cylinder 339, head 0, sector 1
end: cylinder 1022, head 254, sector 63
Extended partition table:
0: sysid 11 (Primary DOS with 32 bit FAT)
start 5446098, size 5510232 (2690 MB), flag 0x0
beg: cylinder 339, head 1, sector 1
end: cylinder 681, head 254, sector 63
1: sysid 5 (Extended partition)
start 10956330, size 5510295 (2690 MB), flag 0x0
beg: cylinder 682, head 0, sector 1
end: cylinder 0, head 254, sector 63
Extended partition table:
0: sysid 11 (Primary DOS with 32 bit FAT)
start 10956393, size 5510232 (2690 MB), flag 0x0
beg: cylinder 682, head 1, sector 1
end: cylinder 0, head 254, sector 63
1:
2:
3:
2:
3:
Do you want to change it? [n] y
sysid: [15] 0
start: [5446035] 0
size: [11020590] 0
Explicitly specify beg/end address? [n]

Is this entry okay? [n] y
The data for partition 2 is:

Do you want to change it? [n] n
The data for partition 3 is:

Do you want to change it? [n] n

We haven't written the MBR back to disk yet. This is your last chance.
Disk: /dev/rwd1d
NetBSD disklabel disk geometry:
cylinders: 16383 heads: 16 sectors/track: 63 (1008 sectors/cylinder)

BIOS disk geometry:
cylinders: 16383 heads: 16 sectors/track: 63 (1008 sectors/cylinder)

Partition table:
0: sysid 169 (NetBSD)
start 63, size 16384000 (8000 MB), flag 0x80
beg: cylinder 0, head 1, sector 1
end: cylinder 894, head 0, sector 31
1:
2:
3:
Should we write new partition table? [n] y

Next I created a disklabel:

bash-2.05b# disklabel -i -I wd1

Basically I messed around with disklabel until something that looked ok was produced. I'm not sure how to get rid of the 'd' and 'e' entries or if it's possible. My understanding is that a is the 'root' partition, 'b' is usually the swap file, 'c' is the entire disk, and 'd' and on are other paritions. This is a second non-booting disk:

8 partitions:
# size offset fstype [fsize bsize cpg/sgs]
a: 16514001 63 4.2BSD 0 0 0 # (Cyl. 0*- 16382)
c: 16514064 0 unused 0 0 # (Cyl. 0 - 16382)
d: 16514064 0 unused 0 0 # (Cyl. 0 - 16382)
e: 16384000 63 unused 0 0 # (Cyl. 0*- 16254*)

Then I did 'newfs':

bash-2.05b# newfs /dev/wd1a

Warning: 64 sector(s) in last cylinder unallocated
/dev/wd1a: 16514000 sectors in 16383 cylinders of 16 tracks, 63 sectors
8063.5MB in 50 cyl groups (328 c/g, 161.44MB/g, 20352 i/g)
super-block backups (for fsck -b #) at:
32, 330720, 661408, 992096, 1322784, 1653472, 1984160,
2314848, 2645536, 2976224, 3306912, 3637600, 3968288, 4298976,
4629664, 4960352, 5290016, 5620704, 5951392, 6282080, 6612768,
6943456, 7274144, 7604832, 7935520, 8266208, 8596896, 8927584,
9258272, 9588960, 9919648, 10250336, 10580000, 10910688, 11241376,
11572064, 11902752, 12233440, 12564128, 12894816, 13225504, 13556192,
13886880, 14217568, 14548256, 14878944, 15209632, 15540320, 15869984,
16200672,

When done I could mount the new drive on a new directory called '/var/extra':

bash-2.05b# mount /dev/wd1a /var/extra

I added the last entry to /etc/fstab:

/dev/wd0a / ffs rw 1 1
/dev/wd0b none swap sw 0 0
/dev/wd0e /var ffs rw 1 2
/dev/wd0f /home ffs rw 1 2
/dev/wd0g /tmp ffs rw 1 2
/dev/wd1a /var/extra ffs rw 1 2

Installing Packages on NetBSD and OpenBSD

Last month I wrote about installing packages on FreeBSD. This entry covers my NetBSD and OpenBSD experiences.

First, a few differences between NetBSD and OpenBSD. Root's default shell in NetBSD is /bin/sh, while OpenBSD uses /bin/csh. This means environment variables can be set in .profile for NetBSD and .cshrc for OpenBSD.

FreeBSD gives users the chance to automatically retrieve packages and dependencies remotely, e.g., 'pkg_add -r mtr'. FreeBSD makes its remote retrieval decisions based on the installed OS. NetBSD and OpenBSD allow the same, but you must specify the OS in an environment variable.

For NetBSD, add the following to .profile:

export PKG_PATH=ftp://ftp.NetBSD.org/pub/NetBSD/packages/1.6.1/i386/All

For OpenBSD, add this to .cshrc:

setenv PKG_PATH ftp://ftp.openbsd.org/pub/OpenBSD/3.4/packages/i386/

These additions make automatic package retrieval easier. I'll install the GTK version of MTR to demonstrate the process on each OS.

For NetBSD, you can browse or search pkgsrc.netbsd.se to find packages. A visit there shows two versions of MTR: mtr and mtr-gtk (both with version 0.54nb1).

Install the newest package with this syntax:

pkg_add -v mtr-gtk

Watch pkg_add find the newest package and install it:

parsing: ftp://ftp.NetBSD.org/pub/NetBSD/packages/1.6.1/i386/All
path: ftp://ftp.NetBSD.org/pub/NetBSD/packages/1.6.1/i386/All
increasing RLIMIT_NOFILE to max. 1772 open files
trying PKG_PATH ftp://ftp.NetBSD.org/pub/NetBSD/packages/1.6.1/i386/All
Spawning FTP coprocess
ftp -detv ftp://ftp.NetBSD.org/pub/NetBSD/packages/1.6.1/i386/All/

Eventually it finds what it needs:

nlist mtr-gtk-*.t[bg]z /var/tmp/pkg.02535d
ftp> cd .
best match: 'ftp://ftp.NetBSD.org/pub/NetBSD/packages/1.6.1/i386/All/mtr-gtk-0.52.tgz'
'ftp://ftp.NetBSD.org/pub/NetBSD/packages/1.6.1/i386/All/mtr-gtk-[0-9]*.t[bg]z'
expanded to 'ftp://ftp.NetBSD.org/pub/NetBSD/packages/1.6.1/i386/All/mtr-gtk-0.5
2.tgz'
Trying to fetch ftp://ftp.NetBSD.org/pub/NetBSD/packages/1.6.1/i386/All/mtr-gtk-
0.52.tgz.
...and so on...

When done, MTR version 0.52 and dependencies are installed:
# pkg_info

pkg_info: disabling PKG_PATH when operating on all packages.
pth-1.4.1nb7 GNU Portable Thread library
bash-2.05.2.7nb1 The GNU Bourne Again Shell
glib-1.2.10nb3 Some useful routines for C programming
gtk+-1.2.10nb3 Gimp toolkit. Libraries for building X11 user interfaces
mtr-gtk-0.52 Traceroute and ping in a single graphical network diagnostic

For OpenBSD, you can browse or search ports.puffy.nu. A search for MTR finds "mtr 0.49" and "mtr 0.49-no_x11". To know the exact name, I prefer searching for mtr at BSDcoders. It shows "mtr-0.49
" and "mtr-0.49-no_x11". (One could assume the hyphens were needed earlier, but it's good to be sure.)

To install the package, you must specify the version, unlike FreeBSD or NetBSD:

pkg_add -v mtr-0.49
Trying to fetch ftp://ftp.openbsd.org/pub/OpenBSD/3.4/packages/i386//mtr-0.49.tg
z.
>>> ftp -o - ftp://ftp.openbsd.org/pub/OpenBSD/3.4/packages/i386//mtr-0.49.tgz
Extracting from FTP connection into /var/tmp/instmp.Zouhf13798
Unknown command.
+CONTENTS
+COMMENT
+DESC
sbin/mtr
man/man8/mtr.8
tar command returns 0 status
pkg: Handling dependencies for mtr-0.49
checking gtk+-* (gtk+-1.2.10p1) -> Not found
checking gtk.1.2 not found
pkg: Handling dependencies for gtk+-1.2.10p1
checking glib-* (glib-1.2.10) -> Not found
checking glib.1.2 not found
checking libiconv-* (libiconv-1.8) -> Not found
checking iconv.3.0 not found
checking gettext->=0.10.38 (gettext-0.10.40p1) -> Not found
checking intl.1.1 not found
...and so on...

When done, mtr-0.49 and its dependencies are installed:

lemelin# pkg_info
bash-2.05b-static GNU Bourne Again Shell
libiconv-1.8 character set conversion library
gettext-0.10.40p1 GNU gettext
glib-1.2.10 useful routines for C programming
gtk+-1.2.10p1 General Toolkit for X11 GUI
mtr-0.49 Matt's traceroute - network diagnostic tool

The best bet for installing newer versions of any software on BSD is to use the ports tree. The absolute newest version can sometimes be only found in source code from the developer, before the ports tree is modified. Still, to quickly install an app, the package system can't be beat -- especially for large apps with numerous dependencies.

Review of Open Source Network Administration Posted

Amazon.com just posted my four star review of Open Source Network Administration. It's been nearly two months since my last review. I've been extremely busy writing The Tao of Network Security Monitoring, so reading has taken a back seat. From the review:
"Open source is the wave of the future, and James Kretchmar's Open Source Network Administration (OSNA) catches that wave in fine form. Although the book is only 238 pages, it contains several gems. I read the book specifically for its coverage of the Multi Router Traffic Grapher (MRTG), OSU's Flow Tools, and Sysmon. By following Kretchmar's instructions, I easily installed these three applications."

Simple Network Health Performance Monitoring with Sysmon

Do you need a simple Web-based application to check if your systems and/or applications are alive? I learned about Sysmon when reading Open Source Network Administration.

Once I wrote my own configuration file to watch my systems, I followed the book's instructions to complete the Sysmon installation. The result is the small screen shot at left. Since you don't need to spend a lot of time checking out the details of my network, you can get the idea by reviewing this image. The green systems are all up. The yellow ones just became unreachable. I forgot to return on one of them to service after working on it. Since it's a bridging firewall for the second box, both are listed in yellow. The orange/reddish entry is a missing box. I lent it to my office for a case, and when it returns the record will be green again!

Sysmon is a good alternative to Nagios or the Network Management Information System (NMIS) if you only need to do simple monitoring of 10-100 systems. Sysmon can check availability by reading HTTP replies and connecting to arbitrary TCP ports, as well as pinging remote systems.

Wednesday, December 24, 2003

Cisco Icons Online

Do you need networking icons for presentations or papers? Check out the completely free, non-copyrighted Cisco collection. I learned about it after reading a tip in Cisco's Packet magazine.

Besides having the icons for use in OpenOffice, the zipped .pdf of conceptual icons is handy. It helps you decode Cisco diagrams by recognizing what certain symbols mean.

Thursday, December 18, 2003

Learning To Install Open Source Software on Solaris and HP-UX

This summer I bought an Ultra 30 workstation and an HP Visualize B2000 workstation to learn Solaris on SPARC and HP-UX on PA-RISC, respectively. Today I worked on installing open source software on each. Starting with the Sun box running Solaris 8 on SPARC, I visited Sun Freeware, an absolutely incredible site providing free compiled binary packages of key open source software. Here's a sample installation:

1. FTP to the Sunfreeware site to retrieve the package for bash
2. Unzip the package with 'gzip -d'
3. Install the package with 'pkgadd -d'

I later installed wget, which made step one much easier. I was even able to install OpenSSH using the site's instructions, which outlined step-by-step the actions needed to install a necessary Solaris patch, then grab the required packages, create keys, and so forth. I made my own startup script using ideas from the instructions:

#!/bin/sh
#
# Simple OpenSSH start script by Richard Bejtlich

SSHCONF=/usr/local/etc/openssh/sshd_config

case "$1" in
start)
echo "Starting sshd using $SSHCONF."
/usr/local/sbin/sshd -f $SSHCONF
echo "Done."
;;
stop)
echo "Killing sshd."
kill `ps -elf | grep /usr/local/sbin/sshd | grep -v grep | awk '{print $4}'`
echo "Done."
;;
restart)
$0 stop
$0 start
;;
esac

For my HP-UX needs, I visited another awesome site -- the Software Porting And Archive Centre for HP-UX. They offered all the packages I needed, but before installing them I needed to get my HP-UX box to finish its boot process. It was hanging while looking for an NFS server, so I followed the directions here to fix it:

1. Enter single user mode by booting and hit 'ESC' to interrupt the boot process.
2. Enter 'boot pri isl'
3. Say 'y' to interact with ISL
4. At ISL prompt enter 'hpux -is boot'
5. In single user mode, mount the following filesystems: /usr and /var
6. Edit with 'vi' /etc/rc.config.d/nfsconf and say 'NFS_CLIENT=0'
7. 'reboot'

At the HP-UX site, I downloaded HP-UX's version of packages, called "depots". For example:

1. FTP to the HP-UX site to retrieve the package.
2. Unzip the package with 'gzip -d' into /usr/local/depot.
3. Run 'swinstall -s /usr/local/depot/<.depot file>'

swinstall when used through a Telnet session is a curses-like package installer. Eventually I installed OpenSSH and used the model for Solaris to get it to work on HP-UX.

Thank goodness for these free repositories of open source software! Would you believe there isn't a "top" command included with Solaris 8? I was able to install it from Sunfreeware, though.

I considered the day a success when I had GCC, wget, bash, and OpenSSH on both boxes, along with all of their dependencies. When I retrieve my AIX box from work I plan to use Bull Freeware to install OpenSSH and other programs. I also found AIX freeware at Darren Tucker's OpenSSH page and the UCLA Public Domain Software Library for AIX. The Encap Archive offers its own package format but lots of software.

Wednesday, December 17, 2003

Verisign Acquires Guardent for $140 Million

Big news from the managed security services space. Consolidation continues, as big companies looking for growth opportunities acquire the small fries. Today Verisign announced it bought Guardent (yes, the Guardent URL spells the company's name incorrectly) for $140 million in stock and cash. Guardent currently employees about 150 people.

Update: Here's eWeek's analysis.

Tuesday, December 16, 2003

Getting Your FreeBSD Box to Speak 802.1q Trunks with a Cisco Switch

I have the following setup on my home LAN:

cable modem - cisco router - freebsd fw/gw - cisco switch - clients

< The client boxes are in two separate VLANs with different address spaces. I needed a way for them to be able to talk to the FreeBSD 4.9 REL firewall/gateway without wasting two interfaces on the fw/gw. Here's how I set this up. I'm no Cisco guru so excuse my lack of shortcuts. I got some help from this how-to, this thread, and this Cisco guide. First, on the switch, I created my VLANs:

gruden#conf term
Enter configuration commands, one per line. End with CNTL/Z.
gruden(config)#vlan 20
gruden(config-vlan)#name green
gruden(config-vlan)#end

gruden#conf term
Enter configuration commands, one per line. End with CNTL/Z.
gruden(config)#vlan 10
gruden(config-vlan)#name yellow
gruden(config-vlan)#end

Next I created my trunk port to speak to the FreeBSD box:

gruden#conf term
Enter configuration commands, one per line. End with CNTL/Z.
gruden(config)#int fa0/24
gruden(config-if)#switchport mode trunk
gruden(config-if)#end

gruden#sh int fa0/24 switchport
Name: Fa0/24
Switchport: Enabled
Administrative Mode: trunk
Operational Mode: trunk
Administrative Trunking Encapsulation: dot1q
Operational Trunking Encapsulation: dot1q
Negotiation of Trunking: On
Access Mode VLAN: 1 (default)
Trunking Native Mode VLAN: 1 (default)
Voice VLAN: none
Administrative private-vlan host-association: none
Administrative private-vlan mapping: none
Operational private-vlan: none
Trunking VLANs Enabled: ALL
Pruning VLANs Enabled: 2-1001
Capture Mode Disabled
Capture VLANs Allowed: ALL

Protected: false

Voice VLAN: none (Inactive)
Appliance trust: none

Then I added each switch port to the appropriate VLAN. This is what adding a single port looks like:

gruden#conf term
Enter configuration commands, one per line. End with CNTL/Z.
gruden(config)#int fa0/1
gruden(config-if)#switchport mode access
gruden(config-if)#switchport access vlan 10
gruden(config-if)#end

gruden#sh int fa0/1 switchport
Name: Fa0/1
Switchport: Enabled
Administrative Mode: static access
Operational Mode: down
Administrative Trunking Encapsulation: dot1q
Negotiation of Trunking: Off
Access Mode VLAN: 10 (yellow)
Trunking Native Mode VLAN: 1 (default)
Voice VLAN: none
Administrative private-vlan host-association: none
Administrative private-vlan mapping: none
Operational private-vlan: none
Trunking VLANs Enabled: ALL
Pruning VLANs Enabled: 2-1001
Capture Mode Disabled
Capture VLANs Allowed: ALL

Protected: false

Voice VLAN: none (Inactive)
Appliance trust: none

On to the FreeBSD box! I used the following commands to set it up. Note that fxp2 is the single physical interface connected to interface 0/24 on the Cisco switch:

ifconfig vlan0 create
ifconfig vlan1 create

ifconfig vlan0 vlan 10 vlandev fxp2
ifconfig vlan1 vlan 20 vlandev fxp2

ifconfig vlan0 inet 10.100.100.1 netmask 255.255.255.0
ifconfig vlan1 inet 172.207.200.1 netmask 255.255.255.0

ifconfig fxp2 up

When done, the interfaces on the FreeBSD box look like this:

moog# ifconfig vlan0
vlan0: flags=8843 mtu 1500
inet 10.100.100.1 netmask 0xffffff00 broadcast 10.100.100.255
inet6 fe80::2d0:b7ff:fe61:3234%vlan0 prefixlen 64 scopeid 0xa
ether 00:02:b3:0a:cd:5b
media: Ethernet autoselect (100baseTX )
status: active
vlan: 10 parent interface: fxp2
moog# ifconfig vlan1
vlan1: flags=8843 mtu 1500
inet 172.207.200.1 netmask 0xffffff00 broadcast 172.207.200.255
inet6 fe80::2d0:b7ff:fe61:3234%vlan1 prefixlen 64 scopeid 0xb
ether 00:02:b3:0a:cd:5b
media: Ethernet autoselect (100baseTX )
status: active
vlan: 20 parent interface: fxp2
fxp2: flags=8843 mtu 1500
inet6 fe80::202:b3ff:fe0a:cd5b%fxp2 prefixlen 64 scopeid 0x3
ether 00:02:b3:0a:cd:5b
media: Ethernet autoselect (100baseTX )
status: active

To make this automatic, add these entries to /etc/rc.conf:

cloned_interfaces="vlan0 vlan1"
ifconfig_vlan0="inet 10.100.100.1 netmask 255.255.255.0 vlan 10 vlandev fxp2"
ifconfig_vlan1="inet 172.207.200.1 netmask 255.255.255.0 vlan 20 vlandev fxp2"
ifconfig_fxp2="up"

When done, the 10.100.100.0/24 and 172.207.200.0/24 networks will be able to talk to each other through the FreeBSD box.

MRTG with FreeBSD and a Cisco Router

It doesn't get much easier than this. I wanted to add the Multi Router Traffic Grapher (MRTG) to my NSM tool collection. Based on the instructions provided by Open Source Network Administration and Cisco, here's how I did it. bourque is the name of my FreeBSD 4.9 REL NSM sensor and gill.taosecurity.com is my Cisco router.

First I enabled the SNMP server on the router. Replace 'public' and 'private' with other community strings, like I did. (These are examples.)

gill(config)#snmp-server community public RO
gill(config)#snmp-server community private RW

Make sure you set up an access list on interfaces where you don't want people accessing the SNMP service on your router:

access-list 101 deny udp any any eq snmp log

Next install an Apache Web server on the system which will hold MRTG's output:

bourque# pkg_add -r ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/
packages-4-stable/All/apache+mod_ssl-1.3.29+2.8.16.tgz
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/
packages-4-stable/All/apache+mod_ssl-1.3.29+2.8.16.tgz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/
packages-4-stable/All/mm-1.3.0.tgz... Done.
bourque# apachectl start

Next install MRTG:

bourque# pkg_add -r ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/
packages-4.9-release/All/mrtg-2.9.29_3,1.tgz
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/
packages-4.9-release/All/mrtg-2.9.29_3,1.tgz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/
packages-4.9-release/All/p5-SNMP_Session-0.95.tgz... Done.

Now configure MRTG:

bourque# mkdir /usr/local/www/data/mrtg
bourque# cfgmaker --global 'WorkDir: /usr/local/www/data/mrtg'
--global 'Options[_]: bits' --global 'IconDir: icons'
--snmp-options=:::::2 --subdirs=HOSTNAME --ifref=ip
--ifdesc=alias --output /usr/local/etc/mrtg/mrtg.cfg
public@gill.taosecurity.com

--base: Get Device Info on public@gill.taosecurity.com:::::2
--base: Vendor Id: cisco
--base: Populating confcache
...edited output...

mkdir /usr/local/www/data/mrtg/icons
cp /usr/local/share/mrtg/* /usr/local/www/data/mrtg/icons/

Now start MRTG:

bourque# mrtg /usr/local/etc/mrtg/mrtg.cfg
WARNING: /usr/local/www/data/mrtg/gill.taosecurity.com/
did not exist I will create it now
...ignore the warnings; these are normal for initial start-up...

Create an index page for the Web server and add an entry in cron to periodically collect MRTG data:

bourque# indexmaker --output /usr/local/www/data/mrtg/index.html
--columns=1 /usr/local/etc/mrtg/mrtg.cfg

bourque# crontab -l
*/5 * * * * /usr/local/bin/mrtg /usr/local/etc/mrtg/mrtg.cfg
--logging /var/log/mrtg.log

You'll want to add the following link for each router name so MRTG can find its icons:

ln -s /usr/local/www/data/mrtg/icons/
/usr/local/www/data/mrtg/gill.taosecurity.com/icons

When you're done you'll see graphs like this when you visit http://sensor/mrtg/index.html. Notice there's only a little bit of data at the far left side, as the system's only been awake for a few minutes.

That's all you need for a basic install. Notice I'm accessing the sensor using HTTP. I could enable HTTPS and access the sensor using that method. Also, be careful running a Web server on your NSM appliance. Lock down who can access it.

Friday, December 12, 2003

CAIDA to the Rescue

Kudos to CAIDA for applying real research to the issue of whether the SCO Web site was hit by a DoS attack or not. CAIDA used its Network Telescopes to watch backscatter from SCO servers and confirmed SCO Web and FTP servers were indeed flooded:

"At 3:20 AM PST on Wednesday, December 10, 2003, the UCSD Network Telescope began to receive backscatter traffic indicating a distributed denial-of-service attack against the SCO Group. Early in the attack, unknown perpetrators targeted SCO's web servers with a SYN flood of approximately 34,000 packets per second...

Around 2:50 AM PST Thursday morning, December 11, the attacker(s) began to attack SCO's ftp (file transfer protocol) servers in addition to continuing the web server attack. Together www.sco.com and ftp.sco.com experienced a SYN flood of over 50,000 packet-per-second early Thursday morning. By mid-morning Thursday (9 AM PST), the attack rate had reduced considerably to around 3,700 packets per second. Throughout Thursday morning, the ftp server received the brunt of the attack, although the high-intensity attack on the ftp server lasted for a considerably shorter duration than the web server attack. At 10:40 AM PST, SCO removed their web servers from the Internet and stopped responding to the incoming attack traffic. Their Internet Service Provider (ISP) appears to have filtered all traffic destined for the web and ftp servers until they came back online at 5 PM PST.

In spite of rumors that SCO has faked the denial-of-service attack to implicate Linux users and garner sympathy from its critics, UCSD's Network Telescope received more than 2.8 million response packets from SCO servers, indicating that SCO responded to more than 700 million attack packets over 32 hours."

Wednesday, December 10, 2003

Creating Fake Interfaces and Bonding Them

In June I posted a way to bond two FreeBSD interfaces to a third unused interface for purposes of combining tap outputs and sniffing the result. This method used ng_one2many and was based on advice from Andrew Fleming. In July I corresponded with John Bradberry who shared his method of using ng_fec, the man-page-less Fast Ether Channel netgraph(4) module.

Two months ago John posted his method, which looks like the email he sent me in July. I finally got a chance to try it, and can report that it works. Here's what's required on a stock FreeBSD 4.9 REL system where sf2 and sf3 see the tap outputs. First, build the ng_fec kernel module:

cd /usr/src/sys/modules/netgraph/fec
make
make install

Next, create the virtual fec0 interface which will see traffic from sf2 and sf3 simultaneously. Note the use of single quote and double quote around the interface names.

ngctl mkpeer fec dummy fec
ngctl msg fec0: add_iface '"sf2"'
ngctl msg fec0: add_iface '"sf3"'
ngctl msg fec0: set_mode_inet
ifconfig sf2 promisc -arp up
ifconfig sf3 promisc -arp up
ifconfig fec0 -arp up

That's it! When sniffing the fec0 interface, you'll see all traffic that sf2 and sf3 see. You can see the results of this process using a few commands. kldstat shows the KLDs that were loaded automatically:

bourque# kldstat
Id Refs Address Size Name
1 4 0xc0100000 43d388 kernel
2 1 0xc211b000 3000 ng_socket.ko
3 2 0xc211f000 9000 netgraph.ko
4 1 0xc212d000 3000 ng_fec.ko

ifconfig shows the new interface:

bourque# ifconfig fec0
fec0: flags=8943 mtu 1500
inet6 fe80::200:d1ff:feed:34df%fec0 prefixlen 64 scopeid 0xe
ether 00:00:d1:ed:34:df
media: Ethernet none
status: active

/var/log/messages shows the fec0 interface in action:

Dec 10 18:27:10 bourque /kernel: fec0: port sf2 in bundle is down
Dec 10 18:27:10 bourque /kernel: fec0: port sf3 in bundle is down
Dec 10 18:27:12 bourque /kernel: fec0: port sf2 in bundle is up
Dec 10 18:27:12 bourque /kernel: fec0: port sf3 in bundle is up

Bruce Schneier on Northeast Blackout

Bruce Schneier wrote about the possible role of MSBlaster in the 14 August 2003 northeast electrical blackout. He reports on the November interim report (.pdf) by a joint US-Canadian taskforce:

"The coincidence is too obvious to ignore. At 2:14 p.m. EDT, the MSBlast worm was dropping systems all across North America. The report doesn't explain why so many computers--both primary and backup systems--at FirstEnergy were failing at around the same time. But MSBlast is certainly a reasonable suspect.

Unfortunately, the report doesn't directly address the MSBlast worm and its effects on FirstEnergy's computers. The closest I could find is this paragraph, on page 99: "Although there were a number of worms and viruses impacting the Internet and Internet connected systems and networks in North America before and during the outage, the SWG's preliminary analysis provides no indication that worm/virus activity had a significant effect on the power generation and delivery systems. Further SWG analysis will test this finding."'

Bruce's article makes valid points. Until the panel explains why the electricity monitoring systems failed, MSBlaster will remain as likely a suspect as any.

I found the report's goals interesting:

"Phase I: Investigate the outage to determine its causes and why it was not contained.

Phase II: Develop recommendations to reduce the possibility of future outages and minimize the scope of any that occur."

This sounds exactly like an incident response plan I use at client sites. In recognizes that determining what happened is important, but that total prevention of future incidents is impossible.

US Government Security Report Card

Yesterday Congressman Putnam of the US House Committee on Government Reform announced the federal government's computer security report card (.pdf). FCW summarized the results. For the first time two agencies scored above 90%: the Nuclear Regulatory Commission earned top honors with an A, and the National Science Foundation received an A-. The grades were based for the first time in the four-year program on the Federal Information Security Management Act (.pdf) reportedly an improvement over the Government Information Security Reform Act (GISRA) (.pdf).

I found it amusing that after the press NASA received for working with SANS to patch systems in 2001 (.pdf), NASA's score has consistently dropped. NASA scored a C- in 2001, a D+ in 2002 and now a D- in 2003. In 2001 NASA was lauded as a "poster child" for their "vulnerability-focused approach to eliminate security problems."

Apparently SANS no longer thinks addressing vulnerabilities is the answer. In their latest NewsBites, SANS reports on a new security survey of "IT professionals." The survey reports "eighty-seven percent [of respondents] said software patches for known vulnerabilities are up to date at their companies." SANS' comment on this statistic is telling: "The saddest part of this study is that it reinforces one of the greatest lies of security - that organizations that keep their systems patched but do not harden operating systems are keeping their systems safe... It's time to stop pretending, and start making sure every system administrator can prove he/she knows how to safely configure a system before being given root or administrator privileges."

My approach has always been simple: prevention fails. Period. Security staff must take steps to ensure they collect the right sorts of information to efficiently scope the extent of compromise and guide recovery. That's why network security monitoring (like implemented by Sguil) is required.

Sunday, December 07, 2003

Creative Commons

After reading why Microsoft Word is not a document exchange format, I found myself at the Creative Common Web site. Their goal is "to build a layer of reasonable, flexible copyright in the face of increasingly restrictive default rules." I found their license builder interesting.

Saturday, December 06, 2003

Spammers Target Cambridgeshire Police Force

I learned of this scam via this Sophos report. Spammers are sending messages which appear to be receipts for £399.99 Apple iPods. The message lists the phone number of the Cambridgeshire Police Force in the UK as the point of contact for complaints. This sounds more like a prank than a structured attack, but the concept is sound. I imagine we'll see more of this in the future. Here's what the email looks like:

Subject: Transaction Receipt (UKCards)
From: "UKCards"
------------------
Please note: All charges to your statement
will appear in the name "UKCARDS LIMITED".

Order Information
Amount: £399.95
Currency: GBP
Merchant Name: HUNTINGDON MAIL ORDER
Description: iPod Music Player 40GB

Customer Service
Telephone: 01480 456111
Email: N/A

Delivery Address
47 Silver Street, London, NW1 5TR

If you have any questions on the delivery
of this order or product details please contact
the merchant directly using the above details.

Friday, December 05, 2003

I commend the Debian project for detailing the exact timetable and methodology associated with their recent compromise. They posted a detailed report on the incident Tuesday. I found several points noteworthy. First, notice how they detected the intrusion. Sharp admins knew something was amiss, and a host-based IDS detected file changes:

"On the evening (GMT) of Thursday, November 20th, the admin team noticed several kernel oopses on master. Since that system was running without problems for a long time, the system was about to be taken into maintenance for deeper investigation of potential hardware problems. However, at the same time, a second machine, murphy, was experiencing exactly the same problems, which made the admins suspicious.

Also, klecker, murphy and gluck have Advanced Intrusion Detection Environment (package aide) installed to monitor filesystem changes and at around the same time it started warning that /sbin/init had been replaced and that the mtime and ctime values for /usr/lib/locale/en_US had changed."

Notice the intruder's actions:

"On Wednesday, November 19th, at approximately 5pm GMT, a sniffed password was used to log into an unprivileged developer account on the host klecker (.debian.org). The attacker then retrieved the source code through HTTP for an (at that time) unknown local kernel exploit and gained root permissions via this exploit. Afterwards, the SucKIT root-kit was installed.

The same account and password data were then used to log into the machine master, to gain root permissions with the same exploit and also to install the SucKIT root-kit...

On the next day the attacker used a password sniffed on master to log into gluck, get root there and also install the SucKIT root-kit."

How was all this password sniffing done? Were clear text protocols involved?

Thursday, December 04, 2003

1500 Helpful Review Votes at Amazon.com

I'd like to thank everyone who's voted my Amazon.com reviews to be "helpful" over the last 3+ years. I started seriously reviewing books with Radia Perlman's Interconnections, 2nd Ed in May 2000. Since then I've written reviews of 116 books on security and computing topics. Some reviews of poor books caused quite a stir. Several were pulled only to have me resubmit "just the facts" in the form of direct quotes. Others caused me to argue with certain members of the community who praised books they should not have. Either they didn't read the book or they were too out of touch to realize printing 250 pages of C code did nothing to teach the reader about "hacking."

Every month I received many books to review. I only read those on my reading list or those that surprise me with their originality. I don't review books I've skimmed, unless the reason I've stopped reading was disappointment with content. As a result, many of my newer reviews are positive. Why waste time reading and reviewing a poor book? I don't get paid for reviews, but I do work on the side evaluating proposals and manuscripts.

I hope you find future reviews helpful!

Tuesday, December 02, 2003

Exploiting Cisco Routers Article

SecurityFocus published the sequel to an article on exploiting Cisco routers. This has been happening for a while but this article spells out the details.

Monday, December 01, 2003

Quirks of NetBSD

Exactly two months ago I reported installing FreeBSD, NetBSD, OpenBSD, and Debian on my laptop for test purposes. Yesterday I tried to upgrade a different box from FreeBSD 5.1 RELEASE to FreeBSD 5.1 CURRENT. I've had no luck getting my SMC 2632W or 2602W wireless NICs, or an Orinoco Gold wireless NIC, to work in any modern version of FreeBSD. (I had the 2632W working with FreeBSD 4.5 and earlier using this hack.) I thought trying CURRENT might be a good idea but the install failed. Wireless support is my biggest grievance with FreeBSD. I used a wireless NIC on OpenBSD 3.3 fine yesterday.

So, I decided to replace the failed FreeBSD install with NetBSD 1.6.1. I got the OS installed but found it to be quirky. It works fine, but it's got its own peculiarities that separates it from FreeBSD. For example, I installed "everything," but OpenSSH was not installed by default. In fact, nothing was listening by default. What is this, OpenBSD? Not even OpenBSD is so draconian. The install process also doesn't give an easy way to configure networking. I added the appropriate entries to /etc/rc.conf because it uses the same format I know from FreeBSD.

Thanks to the official documentation, I was able to get going. If it weren't for Google, I wouldn't know I have to use CTRL-ALT-F4 to switch to terminal 4, rather than ALT-4 like most UNIX OS'. I also found the NetBSD package system installs software into places like /usr/pkg/bin/ rather than /usr/local/bin. NetBSD looks to be picky about licenses too. Here's what I saw when trying to install OpenSSH from /usr/pkgsrc/security/openssh:

===> openssl-0.9.6l has unacceptable license: fee-based-commercial-use.
===> To build this package, add this line to your /etc/mk.conf:
===> ACCEPTABLE_LICENSES+=fee-based-commercial-use

I made the change and OpenSSH installed fine. I had trouble figuring out why the /usr/pkg/etc/rc.d/sshd script wouldn't work properly, so I ended up enabling sshd via a 'sshd="YES"' line rc.conf. I then removed the unneeded OpenSSH, OpenSSL, and Perl packages. I was sad to see though that my wireless NIC still didn't work. In any case, every BSD is a little different, so that's why I'm trying NetBSD.

Friday, November 28, 2003

Snort Add-Ons

Over the last few days I've reviewed several add-ons for Snort. First, everyone using Snort knows about Barnyard. Barnyard processes the output from the spo_unified plugin, which Marty first described in June 2001. spo-unified creates two log files.

To paraphrase Marty, the alert file contains event data (generator, sid, rev, classification, priority, event_reference), timestamp, source IP, destination IP, source port, destination port, protocol, and TCP flags (if applicable). The log file contains the event data, flags that indicate the nature of the stored packet (reassembled fragment, etc.) and the raw binary packet.

Barnyard reads unified output and sends the results to other plugins. In most cases those are database plugins.

MudPit is an alternative to Barnyard. Mudpit was written to overcome the fact that receiving either alert or log data can be insufficient to validate an event, but receiving both simultaneously is wasteful.

At the Sguil project we use our own version of unified output with a modified Barnyard process and new database schema.

Dragos Ruiu wrote Cerebus to read unified output and displays the results in a text-based GUI.

I learned today via this post of the Fast LOgging Project for Snort (Freshmeat site). FLoP doesn't use unified logging at all. It sends Snort output via UNIX domain sockets.

On a slightly different note, I've noticed a few more ambitious projects. For several years CERT has maintained the AirCERT project as a means to share alert data among sensors. I read about the Open Source Security Information Management (OSSIM), Monitoring, Intrusion Detection, Administration System (MIDAS), and Crusoe IDS (announced here) projects, which each bring together data from multiple tools to improve event detection. This is different from Sguil's approach, where we let Snort provide alert data and we provide context using sessions, full content, and eventually statistical data. I know of at least one vendor, Endace, who sells a product featuring data from multiple open source tools.

Voice-Based Fraud Detection

The Register reports on the latest in fraud detection:

"Online insurer Esure is to use technology that recognises when a speaker is under stress in a bid to detect fraud. The company hopes using voice risk analysis (VRA) technology will speed genuine claims, cut fraud and make its claims process more efficient... VRA - which identifies micro changes in the voice that can occur when a speaker is showing higher levels of stress - will be used by esure from 4 December.

The company is keen to emphasise that the technology is a 'stress detector' not a lie detector. When a speaker experiences stress when answering a question or recounting an exaggerated or false statement, the frequency of their voice changes, according to studies originally conducted in Israel. It is this factor that VRA registers and assesses. The system compares responses to particular questions with baseline responses, answers to simple questions that can only be answered truthfully."

Let's consider this for a moment. Say I suffer a car accident. I am going to be very stressed. If I were calling to report the accident, will I trip "VRA"? Of course my voice pattern will differ from earlier calls made to check my deductible or get a new statement mailed. Here's a case where false positives will result in Esure losing business. Still, fraud reduction is a worthy goal.

Thursday, November 27, 2003

According to Reuters, a 38 year old Home Depot worker was arrested for stealing laptops from Wells Fargo. From the article:

"Police recovered the equipment at Krastof's home, along with equipment used for scanning identity cards and checks, he said. 'He is a low-level ID theft kind of guy,' White said of Krastof. Krastof told police that he did not know that sensitive data was on the computer, according to [policeman] White.

Wells Fargo will be able to keep the $100,000 reward it had offered in the case, since the arrest was made from regular police work and not a tip, White said.

Investigators traced the computer to Krastof when he logged onto his own America Online account at home through one of the stolen computers, White said. That enabled authorities to connect the computer's Internet Protocol address, a number that identifies a computer on the Internet, to Krastof's home address through his AOL account, White said."

The article glosses over an important point: how was the stolen computer identified? Wells Fargo may have deployed one of the "Lojack for laptops" solution discussed recently by SC Magazine that send a beacon announcing their presence. Also, why would Krastof say "he did not know that sensitive data was on the computer" when he's an identity thief?

Update: This Slashdot thread alerted me to a second story whose AOL usage description is different:

"A break in the case came in recent days when Krastof plugged one of the computers into a wall socket and turned it on. 'He logged onto an (America Online) account that was registered on that computer and we traced it back to his phone number and address,'' White said."

Ah ha. The original article said "Krastof... logged onto his own America Online account." Obviously if he logged in using the AOL account native to the laptop, AOL could watch for those logins and report them to law enforcement. Case closed. I gave the "analyst" too much credit for thinking "phone home" software might have been used.

Wednesday, November 26, 2003

Ron Gula Replies to Information Security Review of NeVO

You may have read the fairly critical Information Security review of NeVO by Tenable Security. CTO Ron Gula posted a response to the focus-ids group which makes for good reading:

"Since NeVO is on 'all' of the time and it matches for specific vulnerabilities, that means that the vulnerability and IDS correlation which occurs at the Lightning Console is that much more accurate. Our concern at Tenable is that doing correlation based on 'old' vulnerability data (like on a month old Nessus scan) or 'relavent' vulnerability data (like all of the IIS security holes) can produce false correlations."

Monday, November 24, 2003

Tepatche - Automatic OpenBSD System Patcher

I continue to watch for tools to keep BSD systems up-to-date. I learned of a new application for OpenBSD called Tepatche. The author wrote this article for next month's Sys Admin magazine. He also mentions the openbechede package management project.

Incidentally, Colin Percival reports he's attained the $1000 mark needed to buy a new box to provide freebsd-update binary updates. Hopefully we'll see them available for the 5.X tree soon.

While poking around Amazon.com I found a new BSD book will be published in the spring: FreeBSD and OpenBSD Security Solutions.
Wells Fargo Offers $100,000 For Info Leading to Conviction of Laptop Thief

ZDNet reports the following:

"Wells Fargo said on Friday it had offered a $100,000 reward for information leading to the arrest and conviction of the burglar who stole a bank consultant's computer that had sensitive customer information on it. The computer was one of several stolen earlier this month from the office of an analyst for the bank in Concord, California, the bank said. The stolen PC contained names, addresses, bank account numbers and social security numbers for customers who had taken out personal lines of credit that are used for consumer loans and overdraft protection, according to Wells Fargo. No passwords or personal identification numbers were among the stolen data and no other Wells Fargo customers were affected, the bank said... The bank alerted affected customers this week [and] was also monitoring customer accounts, changing account numbers and paying for a year's subscription to a credit monitoring service."

This tells me that the "bank consultant's computer" didn't employ encryption to protect the customer data. It may have taken the physical theft of a computer to stress the importance of California's new privacy laws.

Finding the Name of FreeBSD Packages to Install

I usually install FreeBSD applications using the ports system, but I wanted to know how to use the package collection as well. I wondered how to quickly locate the name and URL of a package so I could pass them as a parameter to pkg_add -r. Using this command FreeBSD fetches the package specified and installs any dependencies automatically.

I found the answer at the FreeBSD Ports Changes page. Here you can query for a package (or port) by name, and more importantly, specify which distribution you want. For example, if you wanted to install Nessus, you could choose from:

  • FreeBSD 4.9 RELEASE: packages created when 4.9 REL was announced

  • FreeBSD 4.x STABLE: the most up-to-date packages built for FreeBSD 4-stable

  • FreeBSD 5.1 RELEASE: packages created when 5.1 REL was announced

  • FreeBSD 5.x CURRENT: the most up-to-date packages built for FreeBSD 5-current


Let's say I wanted to install Nessus. What do these packages look like for the i386 architecture?

Essentially you have to recognize what version of FreeBSD you're running, and then select the appropriate package. Let's say I'm maintaining a FreeBSD 4.9 RELEASE system and want to install Nessus as if the package was installed from the 4.9 REL CD-ROM. In that case I'd choose the nessus-2.0.7.tgz package for FreeBSD 4.9 RELEASE. I use this option when I create installation instructions for Sguil, to simulate building a server using only the CD-ROM packages.

Most people want to run the newest edition of any application. If I wanted the latest and greatest version of Nessus on my 4.9 RELEASE system 'janney', I'd install nessus-2.0.8a.tgz. The second option looks like this:


janney# pkg_add -r ftp://ftp6.freebsd.org/pub/FreeBSD/ports/i386/
packages-4-stable/All/nessus-2.0.8a.tgz
Fetching ftp://ftp6.freebsd.org/pub/FreeBSD/ports/i386/
packages-4-stable/All/nessus-2.0.8a.tgz... Done.
Fetching ftp://ftp6.freebsd.org/pub/FreeBSD/ports/i386/
packages-4-stable/All/nessus-libraries-2.0.8a.tgz... Done.
Fetching ftp://ftp6.freebsd.org/pub/FreeBSD/ports/i386/
packages-4-stable/All/nessus-libnasl-2.0.8a.tgz... Done.

To simulate installing Nessus as a package on a FreeBSD 5.1 RELEASE system 'moog' from a 5.1 REL CD-ROM, I'd add the old nessus-2.0.5_1.tbz package. To get the very latest version of Nessus on a FreeBSD 5.1 system I'd add nessus-2.0.8a.tbz. The second option looks like this:

moog# pkg_add -r ftp://ftp6.freebsd.org/pub/FreeBSD/ports/
i386/packages-current/All/nessus-devel-2.0.8a.tbz
Fetching ftp://ftp6.freebsd.org/pub/FreeBSD/ports/i386/
packages-current/All/nessus-devel-2.0.8a.tbz... Done.
Fetching ftp://ftp6.freebsd.org/pub/FreeBSD/ports/i386/
packages-current/All/nessus-libraries-devel-2.0.8a.tbz... Done.
Fetching ftp://ftp6.freebsd.org/pub/FreeBSD/ports/i386/
packages-current/All/nessus-libnasl-devel-2.0.8a.tbz... Done.

Notice the different package compression schemes -- .tgz for the 4.9 system and .tbz for the 5.1 system. Also see that I passed the entire package URL as an argument to 'pkg_add -r'. The '-r' switch tells pkg_add the package is "remote." For each installation I chose "ftp6.freebsd.org" rather than simply "ftp.freebsd.org". This made pkg_add use one of the mirror sites rather than pound the main FTP server.

In each case I got the appropriate URL from the FreeBSD Ports Changes page. What if I didn't specify the URL, but only the package name?

On a FreeBSD 4.9 system, this happens:

bourque# pkg_add -r nessus
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/
packages-4.9-release/Latest/nessus.tgz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/
packages-4.9-release/All/nessus-libraries-2.0.7.tgz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/
packages-4.9-release/All/nessus-libnasl-2.0.7.tgz... Done.

On a FreeBSD 5.1 system, this happens:

moog# pkg_add -r nessus
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/
packages-5.1-release/Latest/nessus.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/
packages-5.1-release/All/nessus-libraries-2.0.5_1.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/
packages-5.1-release/All/nessus-libnasl-2.0.5_1.tbz... Done.

Notice in each case the packages built for the RELEASE are installed by default. There is probably a way to change this behavior to automatically retrieve the latest edition, but I haven't found it yet.

Here's another example. I chose this one so you could see how the package installation process solves dependency issues. Here I install darkstat, a lightweight alternative to ntop.

bourque# pkg_add -r ftp://ftp.freebsd.org/pub/FreeBSD/ports/
i386/packages-4-stable/All/darkstat-2.6.tgz
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/
packages-4-stable/All/darkstat-2.6.tgz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/
packages-4-stable/All/libiconv-1.9.1_3.tgz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/
packages-4-stable/All/gettext-0.12.1.tgz... Done.


Keep in mind that package installation bypasses all of the benefits of installing the source code through the ports system.

If you want to install a slew of packages at the same time, check out POPS: "Package Of the PackageS" for FreeBSD. This is a ~600 MB CD-ROM .iso with various packages selected by the POPS creator. He likens POPS to a "Linux distribution" in the sense that most FreeBSD installations are fairly minimal.

On a related note, the other king of software installation, Debian, released 3.0r2 last week. The Slashdot thread was informative. I learned of useful sites like apt-get.org and backports.org, as well as tools like apt-secure and aptitude.

Friday, November 21, 2003

Tim O'Reilly on Computer Books

Tim O'Reilly of O'Reilly publishing answered questions on the economics of writing on computer topics. I found this excerpt interesting:

"Your choice of publisher helps [a book be successful]. The clearest lesson from Bookscan (to refer to the data that started this thread) is that the market is consolidating. Fully 80 percent of the market shown by Bookscan (about 65-70 percent of U.S. domestic retail sales, including online accounts) is owned by Pearson, Wiley, O'Reilly, and Microsoft Press, in that order. If you add Osborne and Sybex, you get to 90 percent. (Pearson is a conglomerate owning many individual imprints--AW, PH, Peachpit, Sams, New Riders, Brady, Cisco Press, Adobe Press, Macromedia Press, etc.--so the market looks more diverse than it actually is.) Having been a small publisher who worked my way up over many years, I won't say it [success of a book sold by a small publisher] can't be done. But I think it's a lot harder than it was in the '80s and '90s. In short, being part of an established series at an established publishing house is, unfortunately, very important."

I'm glad my publisher is Pearson's Addison-Wesley!

Wednesday, November 19, 2003

Other Tidbits on SSH, IRC, and other Topics

I needed to bounce through a couple systems while working on a hostile classroom network this week. I found this book excerpt which explains how to chain SSH connections.

I started using the EPIC IRC client on FreeBSD and I wanted to use a customization script. I remembered using Splitfire and found it to be useful. In #snort-gui we've been using Pastebot to provide chunks of text via HTTP rather than IRC on homefries.

Rob Lee's incident-response.org domain registration apparently expired and was scooped by someone else. You can access Rob's site via IP address at http://66.96.178.49/.

Anyone using Secure Instant Messaging Protocol?

Jamil Farshchi published this article on wireless IDS.

PostgreSQL 7.4 Released. Watch Out For MySQL "Gotchas"

PostgreSQL 7.4 was released this week. We use MySQL in the Sguil project but we used PostgreSQL with older NSM tools. I learned about this MySQL "gotchas" site showing odd MySQL behavior. This could prompt a war between the MySQL and PostgreSQL communities.

Speaking of wars, I ran across a site which claims to benchmark various UNIX operating systems. The results caused a crazy thread among OpenBSD users.

What Makes For Credible Certifications?

Peter Stephenson contributed to a SC Magazine article that featured criteria for credible certifications. I found his comments worthwhile:

"The major question to be asked about certifications and their value is: 'Where does the cert come from and what are its objectives?'

A good industry certification will have several recognizable components if it is to be credible:

  • It is based upon an accepted common body of knowledge that is well understood, published and consistent with the objectives of the community applying it.

  • It requires ongoing training and updating on new developments in the field.

  • There is an an examination (the exception is grandfathering, where extensive experience may be substituted).

  • Experience is required.

  • Grandfathering is limited to a brief period at the time of the founding of the certification.

  • It is recognised in the applicable field.

  • It is provided by an organization or association operating in the interests of the community, usually non-profit, not a training company open to independent peer review.



There are credible certifications that are not money-grabs. However, as with anything that promises to improve the acquirer’s status, it is always a case of 'buyer beware.'"

Peter Stephenson is the executive director of the International Institute for Digital Forensic Studies. His organization's new Certified Information Forensics Investigator Certification (CIFI) follows these guidelines.

On a related note, Peter Denning wrote an article (.pdf) two years ago where he defined a profession as having four components:

  1. A durable domain of human concerns

  2. A codified body of principles (conceptual knowledge)

  3. A codified body of practices (embodied knowledge including competence)

  4. Standards for competence, ethics and practice

Tuesday, November 18, 2003

Network Security Monitoring Saves My Bacon

Long-time readers of this blog know I subscribe to a security theory called network security monitoring. Two of NSM's principles are "some intruders are smarter than you" and "intruders are unpredictable." Believing these principles changes the way defenders look at watching their networks. If you assume a smart, unpredictable enemy, you have to take as many defensive actions as possible in the remote hope of catching a bad guy.

This morning I tested these principles not against an intruder, but against a piece of software that took an unexpected action. I was looking for an IRC proxy and found the Night-light IRC proxy. I installed it through the FreeBSD ports system without a problem. I then checked my sockstat output to see what was listening. I found the following unexpected entry:

USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS
root getty 534 0 tcp4 censored:50396 213.145.164.10:25

This looks like my system just connected to 213.145.164.10 on port 25 TCP. I did an nslookup on the destination IP and got these results:

moog# nslookup 213.145.164.10
Server: ns01.rtchrd01.md.comcast.net
Address: 68.48.0.5

Name: brokenarrow.night-light.net
Address: 213.145.164.10

So apparently my box spoke to brokenarrow.night-light.net. I assumed this was a mail server for the night-light.net domain, but I checked this with nslookup:

-bash-2.05b$ nslookup
Default Server: ns01.rtchrd01.md.comcast.net
Address: 68.48.0.5
> set type=mx
> night-light.net
Server: ns01.rtchrd01.md.comcast.net
Address: 68.48.0.5
Non-authoritative answer:
night-light.net preference = 5, mail exchanger = brokenarrow.night-light.net
Authoritative answers can be found from:
night-light.net nameserver = brokenarrow.night-light.net
night-light.net nameserver = jonas.night-light.net
night-light.net nameserver = emptyglass.night-light.net
brokenarrow.night-light.net internet address = 213.145.164.10
jonas.night-light.net internet address = 217.118.34.42
emptyglass.night-light.net internet address = 217.118.34.41

Note I could have connected to port 25 on brokenarrow.night-light.net directly. However, one of the NSM principles is to never touch the source of suspicious activity, to avoid notifying the intruder of your investigation.

I also looked at the output of the installation and saw this:

Sending compilation report to ircproxy-report@night-light.net.

The ircproxy has compiled successfully. To install it type 'make install',
if you choose the root option, remember to 'su root' first.


So, the question is "Now what?" The event didn't trigger any Snort alerts. After all, this is probably just my system sending email. But what did the email contain? Did this new application mail the contents of my password file to the developer? Can I trust this developer?

There's two ways to proceed. A host-based approach involves checking the system hosting the new application for odd activity. This includes checking the source code of the application for the routines that created the socket with brokenarrow.night-light.net.

A network-based -- or NSM -- approach involves checking alert, session, full content, and statistical data for clues. Luckily I had tcpdump data available, so I rebuilt the session and found the following:

moog# tcpflow -c -r snoop.lpc
220 brokenarrow.night-light.net ESMTP Sendmail 8.12.6/8.12.6;
Tue, 18 Nov 2003 16:43:36 +0100 (CET)
EHLO moog.manass01.va.comcast.net
250-brokenarrow.night-light.net
Hello censored.manass01.va.comcast.net [censored], pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-EXPN
250-VERB
250-8BITMIME
250-SIZE
250-DSN
250-ETRN
250-DELIVERBY
250 HELP
MAIL From: SIZE=11475
553 5.1.8 ...
Domain of sender address richard@moog.manass01.va.comcast.net
does not exist
QUIT
221 2.0.0 brokenarrow.night-light.net closing connection

The email was never sent. brokenarrow.night-light.net rejected the attempt because it didn't recognize the sender. While this doesn't tell me exactly what the email would have contained, I know I did not leak any data as a result of this incident.

Saturday, November 15, 2003

TruSecure: "k3wl ," Like "Hackweiser and G-force Pakistan"

The BBC wrote an article about the threat intelligence group, "codename IS/Recon (Information Security Reconnaissance)." They're TruSecure's "moles" -- people who befriend the "underground" and acquire information on their intentions and capabilities. The national intelligence community calls that "human intelligence," or HUMINT.

The article claims TruSecure "currently tracks more than 11,000 individuals in about 900 different hacking groups and gangs." It also states they collect "200 gigabytes of information a day," which "has enabled the team to help out with 54 investigations by law enforcement agencies. IS/Recon gave the FBI over 200 documents about the Melissa virus author after they were asked to get closer to suspects."

Friday, November 14, 2003

Mapping the Internet on a Dare

Slashdot reported on the Opte Project. It's a single guy who's mapping the Internet using code he wrote. Commercial companies like Lumeta provide much more enhanced functionality, but this is still a cool hack.

The Slashdot thread features commentary by Hal Burch and Fyodor, and a useful summary of similar projects. The image at left is supposedly "1/5 of the Internet," but as one Slashdot reader mentioned, it looks a lot like a brain! Given Google has replaced the brain of many people, I imagine this image is appropriate. :)

Trying Fedora Core 1

Today I installed Fedora Core Release 1 in a VMWare session on my laptop. I was unable to using the CD-ROMs I burned and got the same error as described in this thread. I ended up installing the OS using the three .iso files on my laptop hard drive. I installed a default desktop into a 4 GB partition. Here are the daemons listening, the filesystem stats, and the uname output:
[root@localhost root]#netstat -natup

Active Internet connections (servers and established)

Proto Recv-Q Send-Q Local Address   Foreign Address  State      PID/Program name

tcp        0      0 0.0.0.0:1026    0.0.0.0:*        LISTEN     1665/rpc.statd

tcp        0      0 127.0.0.1:1027  0.0.0.0:*        LISTEN     1830/xinetd

tcp        0      0 0.0.0.0:111     0.0.0.0:*        LISTEN     1645/portmap

tcp        0      0 0.0.0.0:22      0.0.0.0:*        LISTEN     1814/sshd

tcp        0      0 127.0.0.1:631   0.0.0.0:*        LISTEN     1777/cupsd

tcp        0      0 127.0.0.1:25    0.0.0.0:*        LISTEN     1851/sendmail: acce

tcp        1      0 127.0.0.1:1034  127.0.0.1:631    CLOSE_WAIT 2103/eggcups

udp        0      0 0.0.0.0:1024    0.0.0.0:*                   1665/rpc.statd

udp        0      0 0.0.0.0:993     0.0.0.0:*                   1665/rpc.statd

udp        0      0 0.0.0.0:111     0.0.0.0:*                   1645/portmap

udp        0      0 0.0.0.0:631     0.0.0.0:*                   1777/cupsd

[root@localhost root]#df -h

Filesystem            Size  Used Avail Use% Mounted on

/dev/sda2             3.6G  1.9G  1.6G  54% /

/dev/sda1              99M  6.3M   88M   7% /boot

none                   62M     0   62M   0% /dev/shm

[root@localhost root]#uname -a

Linux localhost.localdomain 2.4.22-1.2115.nptl #1

 Wed Oct 29 15:42:51 EST 2003 i686 i686 i386 GNU/Linux

The coolest thing in my opinion was trying the yum (Yellow dog Updater, Modified) program. When fedora.redhat.com was down this afternoon I followed these instructions to add a backup source for yum. I then used yum to add nmap. It worked like a charm:
[root@localhost root]# yum install nmap

Gathering header information file(s) from server(s)

Server: Fedora Core 1 - i386 - Base

Server: Fedora Core 1 -- Fedora US mirror

Server: Fedroa Linux (stable) for Fedora Core 1 -- Fedora US mirror

Server: Fedora Core 1 updates -- Fedora US mirror

Server: Fedora Core 1 - i386 - Released Updates

Finding updated packages

Downloading needed headers

Resolving dependencies

Dependencies resolved

I will do the following:

[install: nmap 2:3.48-1.i386]

Is this ok [y/N]: y

Getting nmap-3.48-1.i386.rpm

nmap-3.48-1.i386.rpm      100% |=========================| 368 kB    00:02

Running test transaction:

Test transaction complete, Success!

nmap 100 % done 1/1

Installed:  nmap 2:3.48-1.i386

Transaction(s) Complete

Unfortunately, since no package of ettercap was available, I couldn't try adding it. I then gave up2date a try. I used it too update packages on the system.
[root@localhost root]# up2date-nox -u

Fetching package list for channel: fedora-core-1...

Fetching http://fedora.redhat.com/releases/fedora-core-1/headers/header.info...

########################################

Fetching package list for channel: updates-released...

Fetching http://fedora.redhat.com/updates/released/fedora-core-1/headers/header.info...

########################################

Fetching Obsoletes list for channel: fedora-core-1...

Fetching Obsoletes list for channel: updates-released...

Fetching rpm headers...

########################################

Name                                    Version        Rel

----------------------------------------------------------

glibc                                   2.3.2          101.1               i686

glibc-common                            2.3.2          101.1               i386

nscd                                    2.3.2          101.1               i386

Testing package set / solving RPM inter-dependencies...

########################################

glibc-2.3.2-101.1.i686.rpm: ########################## Done.

glibc-common-2.3.2-101.1.i3 ########################## Done.

nscd-2.3.2-101.1.i386.rpm:  ########################## Done.

Preparing              ########################################### [100%]

Installing...

   1:glibc-common           ########################################### [100%]

   2:glibc                  ########################################### [100%]

Stopping sshd:[  OK  ]

Starting sshd:[  OK  ]

   3:nscd                   ########################################### [100%]

[root@localhost root]#

up2date worked well too. I think I could like this distro.

I got an email from Red Hat explaining the new status of their products. From the email:

"Get Enterprise Linux in three ways:

--> Enterprise Linux WS: for desktop/client systems.
Starting at $179
--> Enterprise Linux ES: for small/mid-range servers.
Starting at $349
--> Enterprise Linux AS: for high-end and mission-critical systems.
Starting at $1499

>>compare all three:

http://info.redhat.com/a/tA-s$LDAJPSNNAOmLYvAK7ybs-q/utbn2"

Wow, those prices are amazing! I'll be interested to see who adopts this product.