New Sguil Scripts and VM
I have not been happy with the performance of FreeBSD 6.0 under VMware Workstation or VMware Server Beta. I thought some workarounds helped, but that wasn't really the case.
Also, since releasing my original Sguil installation script, I've wanted to break it into scripts for the Sguil sensor, database, server, and client.
I decided today to kill two birds with one stone. First, I broke the master script into the following smaller scripts.
All of them are available in this archive: sguil_install_scripts.tar.gz.
These are not pretty. There is no error checking. There is no interaction. You will have to make modifications to get them to work flawlessly in your environment.
Important: As written these scripts download packages for FreeBSD 5, not 6. You can modify this.
These will work best "out of the box" if you want to install all Sguil components on a single host. This is the case because I did not make any adjustments to have MySQL listen on a public interface, for example.
So what good are these? Well, you can now see exactly what software is required for each Sguil component. It's possible I may have erred on the side of including one too many packages for a certain component, but I believe this configuration will work. I did some testing to iron out bugs, but I can't guarantee success.
Using these scripts, I created a new Sguil 0.6.1 complete (sensor/database/server/client) VM on FreeBSD 5.4 RELEASE. The following shows how I invoked the scripts, and the adjustments I made to get the patches to work on this VM.
First I downloaded the script collection.
Next I started with the sensor installation.
You'll notice the end of the script suggests making changes to patches to match your environment. Here are the common changes:
Once I made changes I needed, I ran sguil_sensor_install_patch.sh.
The sensor installation provides several start scripts that must also be adjusted for your environment:
At this point I was ready to install the Sguil database. I broke this into two scripts because I needed a delay to ensure MySQL was running before taking certain actions.
Now I start part 2 of the database installation after checking to be sure MySQL is listening on port 3306. Note that the script edits /etc/rc.conf to make MySQL listen on localhost on port 3306. You can also do the following:
To restart the server:
The sensor and database are done. On the the Sguil server. You'll notice I install mysqltcl from the ports tree. I am no longer hosting a package for this. You'll also be prompted to enter a password for the Sguil client. This is proof that mysqltcl and sguild are working.
As the end of the script notes, you should patch /usr/local/etc/nsm/sguild.conf. The /usr/local/etc/nsm/sguild.conf.patch by default should work as is for those with a local installation.
At last we are ready to install the client. It's fairly simple.
I'd like to share a few words on requirements for running these scripts.
Here are the open ports on a system where all components are running. I omit sshd.
I plan to post the new VM when I get a chance.
Also, since releasing my original Sguil installation script, I've wanted to break it into scripts for the Sguil sensor, database, server, and client.
I decided today to kill two birds with one stone. First, I broke the master script into the following smaller scripts.
- sguil_sensor_install.sh
- sguil_sensor_install_patch.sh
- sguil_server_install.sh
- sguil_database_install_pt1.sh
- sguil_database_install_pt2.sh
- sguil_client_install.sh
All of them are available in this archive: sguil_install_scripts.tar.gz.
These are not pretty. There is no error checking. There is no interaction. You will have to make modifications to get them to work flawlessly in your environment.
Important: As written these scripts download packages for FreeBSD 5, not 6. You can modify this.
These will work best "out of the box" if you want to install all Sguil components on a single host. This is the case because I did not make any adjustments to have MySQL listen on a public interface, for example.
So what good are these? Well, you can now see exactly what software is required for each Sguil component. It's possible I may have erred on the side of including one too many packages for a certain component, but I believe this configuration will work. I did some testing to iron out bugs, but I can't guarantee success.
Using these scripts, I created a new Sguil 0.6.1 complete (sensor/database/server/client) VM on FreeBSD 5.4 RELEASE. The following shows how I invoked the scripts, and the adjustments I made to get the patches to work on this VM.
First I downloaded the script collection.
taosecurity:/root# fetch http://www.bejtlich.net/sguil_install_scripts.tar.gz
sguil_install_scripts.tar.gz 100% of 2552 B 1716 kBps
taosecurity:/root# tar -xzvf sguil_install_scripts.tar.gz
x scripts
x scripts/sguil_client_install.sh
x scripts/sguil_sensor_install.sh
x scripts/sguil_server_install.sh
x scripts/sguil_sensor_install_patch.sh
x scripts/sguil_database_install_pt2.sh
x scripts/sguil_database_install_pt1.sh
taosecurity:/root#
taosecurity:/root# cd scripts/
taosecurity:/root/scripts# ls
sguil_client_install.sh sguil_sensor_install.sh
sguil_database_install_pt1.sh sguil_sensor_install_patch.sh
sguil_database_install_pt2.sh sguil_server_install.sh
taosecurity:/root/scripts# chmod +x *.sh
Next I started with the sensor installation.
taosecurity:/root/scripts# ./sguil_sensor_install.sh
Starting Sguil sensor installation.
...edited...
Sguil server installation finished.
You must modify the following patches in /usr/local/etc/nsm
to match your environment.
sensor_agent.conf.patch
snort.conf.patch
barnyard.conf.patch
sancp.conf.patch
log_packets.sh.patch
When done, run sguil_sensor_install_patch.sh
Next, modify the start scripts to match your environment.
These are in the /home/sguil directory:
barnyard_start.sh sensor_agent_start.sh
You'll notice the end of the script suggests making changes to patches to match your environment. Here are the common changes:
- sensor_agent.conf.patch: change hostname from 'gruden' to 'your_hostname'
- snort.conf.patch: change 'var RULE_PATH /nsm/rules/gruden' to 'nsm/rules/your_hostname'
- barnyard.conf.patch: change hostname from 'gruden' to 'your_hostname'; change interface from 'lnc1' to whatever your system uses
- sancp.conf.patch: probably no changes
- log_packets.sh.patch: change hostname from 'gruden' to your_hostname; change interface from 'lnc1' to whatever your system uses
Once I made changes I needed, I ran sguil_sensor_install_patch.sh.
taosecurity:/root/scripts# ./sguil_sensor_install_patch.sh
Patching Sguil sensor configuration and logging scripts.
Hmm... Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|--- sensor_agent.conf Wed Dec 28 14:57:30 2005
|+++ sensor_agent.conf.diff Wed Dec 28 14:58:33 2005
--------------------------
Patching file sensor_agent.conf using Plan A...
Hunk #1 succeeded at 13.
Hunk #2 succeeded at 22.
Hunk #3 succeeded at 55 (offset 6 lines).
done
Hmm... Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|--- snort.conf Wed Dec 28 14:30:42 2005
|+++ snort.conf.diff Wed Dec 28 15:07:23 2005
--------------------------
Patching file snort.conf using Plan A...
Hunk #1 succeeded at 107.
Hunk #2 succeeded at 621.
done
Hmm... Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|--- barnyard.conf Wed Dec 28 14:30:42 2005
|+++ barnyard.conf.diff Wed Dec 28 15:00:38 2005
--------------------------
Patching file barnyard.conf using Plan A...
Hunk #1 succeeded at 23.
Hunk #2 succeeded at 38.
Hunk #3 succeeded at 133.
done
Hmm... Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|--- sancp.conf Wed Dec 28 14:30:42 2005
|+++ sancp.conf.diff Wed Dec 28 15:01:49 2005
--------------------------
Patching file sancp.conf using Plan A...
Hunk #1 succeeded at 45.
done
Hmm... Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|--- log_packets.sh Wed Dec 28 20:11:54 2005
|+++ log_packets.sh.diff Wed Dec 28 20:12:39 2005
--------------------------
Patching file log_packets.sh using Plan A...
Hunk #1 succeeded at 28.
done
The sensor installation provides several start scripts that must also be adjusted for your environment:
- /root/snort_start.sh: change 'gruden' to 'your_hostname' and 'lnc1' to your sniffing interface
- /root/sancp_start.sh: change 'gruden' to 'your_hostname' and 'lnc1' to your sniffing interface
- /home/sguil/barnyard_start.sh: change 'gruden' to 'your_hostname'
At this point I was ready to install the Sguil database. I broke this into two scripts because I needed a delay to ensure MySQL was running before taking certain actions.
taosecurity:/root/scripts# ./sguil_database_install_pt1.sh
Starting Sguil database installation, part 1.
...edited...
The latest information about MySQL is available on the web at
http://www.mysql.com
Support MySQL by buying support/licenses at https://order.mysql.com
Run sguil_database_install_pt2.sh after MySQL is running.
taosecurity:/root/scripts# Starting mysqld daemon with databases from /var/db/mysql
taosecurity:/root/scripts# sockstat -4 | grep 3306
mysql mysqld 1187 10 tcp4 *:3306 *:*
Now I start part 2 of the database installation after checking to be sure MySQL is listening on port 3306. Note that the script edits /etc/rc.conf to make MySQL listen on localhost on port 3306. You can also do the following:
# mysqladmin -p shutdown
Enter password:
To restart the server:
mysqld_safe --bind-address=127.0.0.1 --user=mysql &
taosecurity:/root/scripts# ./sguil_database_install_pt2.sh
Starting Sguil client installation, part 2.
+-------------------+
| Tables_in_sguildb |
+-------------------+
| history |
| nessus |
| nessus_data |
| portscan |
| sensor |
| sessions |
| status |
| user_info |
| version |
+-------------------+
Sguil database installation complete.
The sensor and database are done. On the the Sguil server. You'll notice I install mysqltcl from the ports tree. I am no longer hosting a package for this. You'll also be prompted to enter a password for the Sguil client. This is proof that mysqltcl and sguild are working.
taosecurity:/root/scripts# ./sguil_server_install.sh
Starting Sguil server installation.
...edited...
=> Attempting to fetch from http://www.xdobry.de/mysqltcl/.
mysqltcl-3.01.tar.gz 100% of 164 kB 62 kBps
===> Extracting for mysqltcl-3.01
=> MD5 Checksum OK for mysqltcl-3.01.tar.gz.
===> Patching for mysqltcl-3.01
===> Applying FreeBSD patches for mysqltcl-3.01
===> mysqltcl-3.01 depends on shared library: tcl84 - found
===> mysqltcl-3.01 depends on shared library: mysqlclient.15 - found
===> Configuring for mysqltcl-3.01
===> Building for mysqltcl-3.01
...edited...
Create a Sguil client user password when prompted.
Please enter a passwd for sguil:
Retype passwd:
User 'sguil' added successfully
SGUILD: Exiting...
Sguil server installation finished.
You must modify /usr/local/etc/nsm/sguild.conf.patch
to match your environment.
When done, apply it:
patch -p0 < sguild.conf.patch
As the end of the script notes, you should patch /usr/local/etc/nsm/sguild.conf. The /usr/local/etc/nsm/sguild.conf.patch by default should work as is for those with a local installation.
taosecurity:/root/scripts# cd /usr/local/etc/nsm/
taosecurity:/usr/local/etc/nsm# patch -p0 < sguild.conf.patch
Hmm... Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|--- sguild.conf Wed Dec 28 16:29:21 2005
|+++ sguild.conf.diff Wed Dec 28 16:30:34 2005
--------------------------
Patching file sguild.conf using Plan A...
Hunk #1 succeeded at 1.
Hunk #2 succeeded at 30.
Hunk #3 succeeded at 42.
Hunk #4 succeeded at 71.
done
At last we are ready to install the client. It's fairly simple.
taosecurity:/root/scripts# ./sguil_client_install.sh
Starting Sguil client installation.
...edited...
Sguil client installation finished.
I'd like to share a few words on requirements for running these scripts.
- Make sure you have users sguil and analyst on the system.
- With FreeBSD 5.4, I applied the User installation. I manually extracted the ports tree by downloading it from ftp://ftp.freebsd.org/pub/FreeBSD/ports/ports-current/ports.tar.gz and extracting it in /usr. I did that because the ports tree on the CD is 10 months old.
- If you create a system with DHCP, make sure you have an entry in /etc/hosts for 127.0.0.1 and the name of your sensor, like 'taosecurity taosecurity.taosecurity.com'.
Here are the open ports on a system where all components are running. I omit sshd.
taosecurity:/home/analyst$ sockstat -4 | grep -v sshd
USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS
sguil barnyard 717 12 tcp4 127.0.0.1:51062 127.0.0.1:7735
sguil tclsh8.4 701 3 tcp4 127.0.0.1:53610 127.0.0.1:7736
sguil tclsh8.4 701 4 tcp4 127.0.0.1:7735 *:*
sguil tclsh8.4 701 6 tcp4 127.0.0.1:7735 127.0.0.1:51062
sguil tclsh8.4 672 12 tcp4 *:7734 *:*
sguil tclsh8.4 672 13 tcp4 127.0.0.1:7736 *:*
sguil tclsh8.4 672 14 tcp4 127.0.0.1:7736 127.0.0.1:53610
mysql mysqld 505 10 tcp4 127.0.0.1:3306 *:*
root sendmail 430 3 tcp4 127.0.0.1:25 *:*
root syslogd 315 6 udp4 *:514 *:*
root dhclient 247 4 udp4 *:68 *:*
I plan to post the new VM when I get a chance.
Comments
If used the scripts on both platforms and they seem to work fine.
I recently used your script to help install Sguil on a brand new FreeBSD box. Due to its SATA chipset, I had to use the FreeBSD 6.1BETA4 release. One thing I think you left out of your script/instructions is the creation of the Sguil user.
I found one of your earlier posts regarding the Sguil instructions that helped me get the Sguil user created.
http://taosecurity.blogspot.com/2005/12/rough-sguil-installation-script-my_28.html
Also, unless I missed something, how do you enable Snort to start after a system reboot?
Thanks for the feedback.
Check out /usr/local/etc/rc.d/snort.sh for info on how to use rc.d to start Snort automatically.
portsnap fetch
portsnap update
portupgrade -varRPP
portupgrade -varR
Now the sguil daemon runs, but it doesn't open port 7734 for the sguil client to connect to. I've exhausted my troubleshooting skills to get this working again.
I'm going to try a reinstall as a last resort, but I wanted to post a small comment just in case this is useful to you.
-davis
SENSOR=taosecurity
cd /usr/local/etc/nsm/
barnyard -c barnyard.conf -d /nsm/$SENSOR/ -g gen-msg.map -s sid-msg.map -f snort.log -w /nsm/$SENSOR/waldo.file
It doesn't then look for the gen-msg.map inside the /nsm/taosecurity. It stays in the /usr/local/etc/nsm dir and looks for it. The snort startup script is doing the same. What am I doing wrong here?
I'm using freebsd 5.5
Thanks,
You're in the /usr/local/etc/nsm directory when you tell barnyard to read gen-msg.map and sid-msg.map. The -d flag tells barnyard where to find spool files, not anything else.