Friday, January 06, 2006

New Sguil VM with Client

Hot on the heels of last week's news about the first Sguil VM, I am pleased to announce the release of a new Sguil VM. This new image is a complete self-contained Sguil deployment, with sensor, server, database, and client. The screenshot above shows the Sguil client and Ethereal. Again, you need something like VMware Player or better, and a program to unzip the archive.

The new file is being shared on the Sourceforge mirrors as sguil0-6-0p1_freebsd6-0_1280_06jan06.zip. I noticed this OSDN mirror already has it. The new .zip is 218 MB, and it expands to about 700 MB. The VM disk is 1280 MB (1.25 GB) and it is built with 128 MB RAM.

The new VM is nearly identical to the previous VM. Use the same user accounts, network settings, etc., as previously described. There are two exceptions:

  1. I have added of the Sguil client components. This means you can either connect to the server using your own Sguil client, or log into the new VM as user analyst, run 'startx', and find yourself in a graphical Fluxbox environment.

  2. I have added tools used in my Network Security Operations class, mentioned in this post.


I built the VM using my new installation script described here.

For those who wish to build their own VM, I made the following additions beyond what the script does.

When I first boot the machine, I enter single user mode and create a /boot/loader.conf file with the line 'hint.apic.0.disabled=1'. I still seem to have troubles with time in the VM, although this post seems to indicate the latest VMware combined with 6-STABLE might improve the situation.

I next install Vmware Tools for FreeBSD. This allows a large display at 1024x768. Inside VMware, I follow VM -> Install VMware Tools -> Install. Next, on a local console as root:

mount /cdrom
cd /tmp
tar -xzvfp /cdrom/vmware-freebsd-tools.tar.gz
cd vmware-tools-distrib
./vmware-install.pl

If I need to re-run the configuration, I can try /usr/local/bin/vmware-config-tools.pl.

I also edit /etc/motd so the users see the following at login.

Welcome to the Sguil Virtual Machine!

Richard Bejtlich (richard@taosecurity.com) created this VM to
help those new to Sguil (www.sguil.net) become familiar with
Sguil components and operation.

To start Sguil server components, do the following.

As user sguil, execute these scripts:

/home/sguil/sguild_start.sh
/home/sguil/sensor_agent_start.sh
/home/sguil/barnyard_start.sh

As user root, execute these scripts:

/root/sancp_start.sh
/root/snort_start.sh
/usr/local/bin/log_packets.sh restart

To start the Sguil client, do the following.

Log in as user analyst. Run startx to launch Fluxbox.
Launch a xterm, then run /home/analyst/sguil_client_start.sh.

Note: Thanks to transltr for pointing out that the motd as installed in the VM says to run /root/start_sancp.sh and /root/strt_snort.sh. That will be fixed in the next release.

The VM as provided uses space as follows:

$ df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/ad0s1a 124M 56M 58M 49% /
devfs 1.0K 1.0K 0B 100% /dev
/dev/ad0s1g 62M 76K 57M 0% /home
/dev/ad0s1f 124M 16M 98M 14% /nsm
/dev/ad0s1h 61M 17M 40M 30% /tmp
/dev/ad0s1d 496M 441M 15M 97% /usr
/dev/ad0s1e 124M 24M 90M 21% /var
/dev/acd0 6.9M 6.9M 0B 100% /cdrom

Notice /usr is pretty tight. /nsm is small too. This is a demonstration VM, not a production version. Following my scripts you can easily create your own VM though,

These packages are installed:

$ pkg_info
adns-1.1 Easy to use, asynchronous-capable DNS client library and ut
argus-2.0.6 A generic IP network transaction auditing tool
argus-clients-2.0.6 Client programs for the argus IP network transaction auditi
atk-1.10.3 A GNOME accessibility toolkit (ATK)
barnyard-0.2.0 An output system for Snort
bitstream-vera-1.10_2 Bitstream Vera TrueType font collection
cairo-1.0.2_1 Vector graphics library with cross-device output support
ethereal-0.10.13_3 A powerful network analyzer/capture tool
expat-1.95.8_3 XML 1.0 parser written in C
flow-tools-0.68_1 Suite of tools and library to work with netflow data
flowgrep-0.8a TCP stream/UDP/IP payload 'grep' utility
fluxbox-devel-0.9.14 A small and fast window manager based on BlackBox
fontconfig-2.3.2,1 An XML-based font configuration API for X Windows
fprobe-1.1 Tool that collects network traffic data
freetype2-2.1.10_2 A free and portable TrueType font rendering engine
gettext-0.14.5 GNU gettext package
glib-1.2.10_11 Some useful routines of C programming (previous stable vers
glib-2.8.4 Some useful routines of C programming (current stable versi
gtk-1.2.10_13 Gimp Toolkit for X11 GUI (previous stable version)
gtk-2.8.9 Gimp Toolkit for X11 GUI (current stable version)
hicolor-icon-theme-0.5 A high-color icon theme shell from the FreeDesktop project
ipcad-3.7 IP accounting daemon with Cisco-like RSH and NetFlow export
itcl-3.2.1_1 [incr Tcl] (A.K.A. "itcl")
itk-3.2.1_1 [incr Tk] (A.K.A. "itk")
iwidgets-4.0.1 Iwidgets - [incr Widgets]
jpeg-6b_3 IJG's jpeg compression utilities
libXft-2.1.7 A client-sided font API for X applications
libiconv-1.9.2_1 A character set conversion library
libltdl-1.5.22 System independent dlopen wrapper
libnetdude-0.6 A library for manipulating libpcap/tcpdump trace files
libpcapnav-0.5 A libpcap wrapper library
libxml2-2.6.22 XML parser library for GNOME
mysql-client-5.0.17 Multithreaded SQL database (client)
mysql-server-5.0.17 Multithreaded SQL database (server)
mysqltcl-3.01 TCL module for accessing MySQL databases based on msqltcl
net-snmp-5.2.2 An extendable SNMP implementation
netdude-0.4.5 NETwork DUmp data Displayer and Editor for tcpdump tracefil
ngrep-1.44 Network grep
p0f-2.0.3_1 Passive OS fingerprinting tool
pango-1.10.2 An open-source framework for the layout and rendering of i1
pcre-6.4 Perl Compatible Regular Expressions library
perl-5.8.7 Practical Extraction and Report Language
pkgconfig-0.20 A utility to retrieve information about installed libraries
png-1.2.8_2 Library for manipulating PNG images
py24-pynids-0.5_1 Python interface to libnids
python-2.4.2 An interpreted object-oriented programming language
sancp-1.6.1_1 A network connection profiler
shared-mime-info-0.16_2 A MIME type database from the FreeDesktop project
snort-2.4.3_1 Lightweight network intrusion detection system
tcl-8.4.11,1 Tool Command Language
tclX-8.3.5_2 Extended TCL
tcllib-1.7_1 A collection of utility modules for Tcl
tcltls-1.5.0 SSL extensions for TCL; dynamicly loadable
tcpdstat-0.9 A tool for generating statistics from tcpdump (libpcap) fil
tcpflow-0.21_1 A tool for capturing data transmitted as part of TCP connec
tcpreplay-2.3.5 A tool to replay saved packet capture files
tiff-3.7.4 Tools and library routines for working with TIFF images
tk-8.4.11,2 Graphical toolkit for TCL
trafshow-5.2.1_1,1 Full screen visualization of network traffic
xorg-clients-6.8.2_1 X client programs and related files from X.Org
xorg-fonts-100dpi-6.8.2 X.Org 100dpi bitmap fonts
xorg-fonts-75dpi-6.8.2 X.Org 75dpi bitmap fonts
xorg-fonts-encodings-6.8.2 X.Org font encoding files
xorg-fonts-miscbitmaps-6.8.2 X.Org miscellaneous bitmap fonts
xorg-fonts-truetype-6.8.2 X.Org TrueType fonts
xorg-libraries-6.8.2 X11 libraries and headers from X.Org
xorg-server-6.8.2_7 X.Org X server and related programs
xterm-206_1 Terminal emulator for the X Window System

If you can wait to download the client from Sourceforge, that will make like easier for my hosting company.

If you have comments, please post them here. Thank you!

25 comments:

Anonymous said...

This is VERY cool.

I would like to see some canned data in the db for the true standalone, can't connect to the net demo.

Richard Bejtlich said...

Tcpreplay is on the box. Try retrieving the packet captures from The Tao. Start all the Sguil components, and then run Tcpreplay on the Sguil VM like this as root:

tcpreplay -i lnc1 sf0.lpc

Anonymous said...

I don't have a ID and password for any ID. I must of miss something. Any ideas what they are?

Richard Bejtlich said...

Do you mean a Sguil client user id? The default is sguil with password sguil.

Anonymous said...

wow, dunno what i did, but image kept rebooting in vmware 5.5. So went to single user mode, fsck'd everything, ran login, logged in as root, reinstalled vmware tools, started x and saved a snapshot...turns out sancp_start.sh reboots the image...dunno why I'm not a bsd or sguil guy just thought I'd try it out.

Richard Bejtlich said...

This is sancp_start.sh:

#!/bin/sh
SENSOR=gruden
INTERFACE=lnc1
ifconfig $INTERFACE -arp up
# As a daemon
sancp -D -d /nsm/$SENSOR/sancp/ -i $INTERFACE -u sguil -g sguil -c /usr/local/etc/nsm/sancp.conf > /var/log/sancp.log
# In foreground
#sancp -d /nsm/$SENSOR/sancp/ -i $INTERFACE -u sguil -g sguil -c /usr/local/etc/nsm/sancp.conf > /var/log/sancp.log

I've never heard anything like what you posted. What do you mean, the image keeps rebooting? And why do you blame sancp_start.sh?

Anonymous said...

An MD5 sum for the release being offered would be really nice.

Cheers,

Richard Bejtlich said...

Good idea -- I just posted hashes on the Sguil VM page.

Anonymous said...

I was doing pretty well for a FreeBSD newbie I think. (figured out that I had to remove the proxy server line from the script and uninstalled an old version of the mysql client because it was conflicting). Then I get the error saying the mysqltcl extension does not appear to be installed on this system. Download it at http://www.xdobry.de/mysqltcl/. I downloaded it, but am stuck there. Does anyone have any recommendations about how to install it. I really want to try sguil but just when I think I've got it licked something gets in the way. For any who bother to read this (and heaven forbid, reply), thanks.

Richard Bejtlich said...

Anonymous,

Join us in #snort-gui on irc.freenode.net and ask there, or post to the Sguil users mailing list.

Anonymous said...

I have been looking for a build such as this. I have not yet downloaded it and have one question. Is the virtual disk built as SCSI or IDE? I would like to run it on ESX but ESX does not support IDE.

Richard Bejtlich said...

It's IDE.

It's easy to build the VM though if you use my script, described here.

Anonymous said...

I get "Unable to connect to localhost on port 7734" using sguil:sguil to login (SSL not selected) Any ideas on what I could do?

Richard Bejtlich said...

Drop in to #snort-gui on irc.freenode.net. I can't troubleshoot your problem with that little amount of detail.

Anonymous said...

This image seems to work fine in the workstation environment, at least it started, I haven't played with it enough to figure everything out.

But If I try to bring it up in the ESX environment it won't boot because it needs SCSI drives instead of IDE, or am I missing something simple?

thanks

Richard Bejtlich said...

Apparently ESX won't run images built on virtual IDE drives? If so, try building your own image -- that's the answer I gave an earlier poster. It's not difficult.

dmjamal@gmail.com said...

In the workstation, you can create/add an extra (new) scsi-based vmdk file and then ghost the ide vmdk disk to scsi vmdk and then put/export that scsi vmdk disk to esx server.

Anonymous said...

One of the other problems is that FreeBSD 6.0 wont install on ESX. The latest official version of FreeBSD that ESX supports is 4.10. I am not sure if Richard's script will function on this old of a version.

Fresh Proxy List said...
This comment has been removed by a blog administrator.
ruy_lopez said...

I'm having trouble integrating sancp into sguil. When I query sancp within sguil it returns empty. A quick look at the database with phpMyAdmin shows there is data in the sancp tables, however, there is no src or dest ip address data. The same is true if I look at the realtime files. Here's an example:

4917918058510580844|2006-04-14 19:13:47|2006-04-14 19:13:47|2006-04-14 19:13:47|2054|0|0|0|0|0|0|300|1|0|28|0|0|0|0|0|0|78|0|0|0|N|N|0|0|78|0|0|0|N|N|42|1|42|0|0|Y|Y|Y|0|0|0|0|0|0|0|N|0:11:50:a8:78:b3|ff:ff:ff:ff:ff:ff

I'm using the sancp.conf included with sguil, so it can't be a database field issue(?). Anyone know how sancp gets it's ipaddress data. Might help me isolate the problem.

Also, shouldn't sancp be logging TONS of session data? Instead it's only logging when there's an event. I thought the whole point of session data was to give context to an incident, in a similar manner as whole data capture does. If all sancp does is log the instances of the session that triggered the event, its no different from the session elements of snort's output data.

Richard Bejtlich said...

Hi Ruy,

SANCP has no concept of alerts. It is not tied to Snort in any way.

I recommend posting this question to the snort-users mailing list:

snort-users@lists.sourceforge.net

John Curry, SANCP's author, reads that list.

Anonymous said...

Convert Virtual Machines from IDE to SCSI -- from GSX to ESX and vice versa..

http://www.darksource.org/vmware/

Anonymous said...

Dear Richard,

First of all thanks for the work.

When I run the new vmware version on vmware server 1.0.3 build 44356, I get mounting errors.

##################
/dev/ad0s1g: can't check file system.
/dev/ad0s1g: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.

The same message for /dev/ad0s1f
/dev/ad0s1h
/dev/ad0s1e

For /dev/ad0s1d I get:
/dev/ad0s1d: /dev/ad0s1d: BAD SUPER BLOCK; VALUE IN SUPER BLOCK DISAGREE WITH THOSE IN FIRST ALTERNATE

Then I am dropped to single mode shell.

I hope someone can help.

gunmetalgreen said...

Richard,
I am using the KnoppixNSM hdd install which seems to work and is extremely easy to get up and running. However, the one continuous problem that I have is using the tk client (win or uni) because the tcp 7734 socket becomes unavailable.

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 4882/mysqld
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 4480/sshd
tcp 0 0 127.0.0.1:7735 0.0.0.0:* LISTEN 7667/tclsh
tcp 0 0 127.0.0.1:7735 127.0.0.1:32874 ESTABLISHED7667/tclsh
tcp 0 0 127.0.0.1:32874 127.0.0.1:7735 ESTABLISHED7691/barnyard
tcp 0 0 10.0.0.245:22 10.0.1.244:55209 ESTABLISHED6822/sshd: tmessner
tcp6 0 0 :::3000 :::* LISTEN 7739/ntop
tcp6 0 0 :::443 :::* LISTEN 3535/apache2
udp 0 0 10.0.0.245:32768 192.168.1.1:53 ESTABLISHED3985/tcpdump
udp 0 0 10.0.0.245:32769 192.168.1.1:53 ESTABLISHED4010/tcpdump


I can restart the 'd' server and sometimes it comes back within 10-30min but this is not guaranteed. Often it never comes back resulting the inabilty to monitor events with the client.

This problem is very consistent so I was wondering if anyone else has had similar problems, and if so how were they fixed?

Thanks.

Richard Bejtlich said...

Recommend posting to sguil-users@lists.sourceforge.net.