Multiple Kernels on FreeBSD

The following is a topic I would enjoy hearing more about. If you have helpful suggestions, please share them as a comment.

Two years ago I described my experiences with building a FreeBSD userland and kernel on one system and installing it on another. I found myself in the same situation recently, where I didn't want to sit around waiting for a couple slow boxes to build themselves custom kernels. I wanted to build the custom kernel on a fast box and use it on the slower boxes. I didn't want to replace the default kernel on any of the boxes. I wanted the new kernel(s) to be additional boot-time options.

This post gave me the answer I needed. Here's how I applied it.

I wanted to build a GENERIC-style kernel, but with security updates applied. First I installed cvsup-without-gui as a package. Next I created this /usr/local/etc/security-supfile file:

*default host=cvsup5.FreeBSD.org
*default base=/usr
*default prefix=/usr
*default release=cvs tag=RELENG_6_1
*default delete use-rel-suffix

*default compress

src-all

This would update my kernel sources and userland to the SECURITY branch effective the time I ran cvsup (next).

kbld# cvsup -g -L 2 /usr/local/etc/security-supfile
Parsing supfile "/usr/local/etc/security-supfile"
Connecting to cvsup5.FreeBSD.org
Connected to cvsup5.FreeBSD.org
Server software version: SNAP_16_1h
Negotiating file attribute support
Exchanging collection information
Establishing multiplexed-mode data connection
Running
Updating collection src-all/cvs
 Edit src/UPDATING
  Add delta 1.416.2.22.2.3 2006.05.31.22.31.41 cperciva
  Add delta 1.416.2.22.2.4 2006.06.14.15.59.27 cperciva
  Add delta 1.416.2.22.2.5 2006.07.07.07.25.21 cperciva
  Add delta 1.416.2.22.2.6 2006.08.23.22.02.25 cperciva
...edited...
Shutting down connection to server
Finished successfully

Next I created the file GENERIC.SECURITY in /usr/src/sys/i386/conf with the following:

include GENERIC

All that does is make GENERIC.SECURITY the same kernel as GENERIC, except with patches applied. At this point you might think I should just update the GENERIC kernel. I could do that, but I'm using this method because later steps show this system works best for my requirements.

Now I can build the kernel.

kbld# cd /usr/src
kbld# make buildkernel KERNCONF=GENERIC.SECURITY INSTKERNNAME=GENERIC.SECURITY
--------------------------------------------------------------
>>> Kernel build for GENERIC.SECURITY started on Wed Sep 20 19:54:46 EDT 2006
--------------------------------------------------------------
===> GENERIC.SECURITY
mkdir -p /usr/obj/usr/src/sys

--------------------------------------------------------------
>>> stage 1: configuring the kernel
--------------------------------------------------------------
...truncated...
--------------------------------------------------------------
>>> Kernel build for GENERIC.SECURITY completed on Wed Sep 20 20:12:42 EDT 2006
--------------------------------------------------------------

Next I installed the kernel.

kbld:/usr/src# make installkernel KERNCONF=GENERIC.SECURITY INSTKERNNAME=GENERIC.SECURITY
--------------------------------------------------------------
>>> Installing kernel
--------------------------------------------------------------
...edited...
kldxref /boot/GENERIC.SECURITY
kbld:/usr/src#

That's it. I make sure host kbld is exporting the appropriate directories via NFS by creating this /etc/exports file:

/usr    -alldirs

That's too loose but this is sufficient for my test network.

Now I move from the kernel builder to a slow system where I would like to make GENERIC.SECURITY available. 192.168.2.103 is kbld, where the new kernel is waiting.

asa633:/root# mount_nfs 192.168.2.103:/usr/src /usr/src
asa633:/root# mount_nfs 192.168.2.103:/usr/obj /usr/obj
asa633:/root# mount
/dev/ad0s1a on / (ufs, local)
devfs on /dev (devfs, local)
/dev/ad0s1f on /home (ufs, local, soft-updates)
/dev/ad1s1d on /nsm (ufs, local, soft-updates)
/dev/ad0s1g on /tmp (ufs, local, soft-updates)
/dev/ad0s1d on /usr (ufs, local, soft-updates)
/dev/ad0s1e on /var (ufs, local, soft-updates)
192.168.2.103:/usr/src on /usr/src (nfs)
192.168.2.103:/usr/obj on /usr/obj (nfs)
asa633:/usr/src# make installkernel KERNCONF=GENERIC.SECURITY INSTKERNNAME=GENERIC.SECURITY
--------------------------------------------------------------
>>> Installing kernel
--------------------------------------------------------------
...edited...
kldxref /boot/GENERIC.SECURITY

How do I get this GENERIC.SECURITY kernel to boot? If I were at the console at boot time, I could say 'boot GENERIC.SECURITY'. Since I am remote, I edit /boot/loader.conf to say

kernel=GENERIC.SECURITY

Now I reboot. After rebooting, I see the new kernel is installed:

asa633:/root# uname -a
FreeBSD asa633.taosecurity.com 6.1-RELEASE-p6 FreeBSD 6.1-RELEASE-p6 #0:
 Wed Sep 20 20:02:56 EDT 2006     
 root@kbld.taosecurity.com:/usr/obj/usr/src/sys/GENERIC.SECURITY  i386

Pretty easy. If I want to boot the default kernel, I remove the entry in /boot/loader.conf.

For example, asa633 is usually running the kernel provided by Colin Percival's FreeBSD-Update code:

asa633:/root# uname -a
FreeBSD asa633.taosecurity.com 6.1-SECURITY FreeBSD 6.1-SECURITY #0:
 Mon Aug 28 05:21:08 UTC 2006     
root@builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  i386

FreeBSD-Update, in fact, very nicely takes care of the latest security problems with Gzip:

asa633:/root# freebsd-update fetch
Fetching updates signature...
Fetching updates...
Fetching hash list signature...
Fetching hash list...
Examining local system...
Fetching updates...
/usr/bin/gunzip...
/usr/bin/gzcat...
/usr/bin/gzip...
/usr/bin/zcat...
Updates fetched

To install these updates, run: '/usr/local/sbin/freebsd-update install'
asa633:/root# freebsd-update install
Backing up /usr/bin/gunzip...
Installing new /usr/bin/gunzip...
Backing up /usr/bin/gzcat...
Recreating hard link from /usr/bin/gunzip to /usr/bin/gzcat...
Backing up /usr/bin/gzip...
Recreating hard link from /usr/bin/gunzip to /usr/bin/gzip...
Backing up /usr/bin/zcat...
Recreating hard link from /usr/bin/gunzip to /usr/bin/zcat...

Easy!

Comments

Anonymous said…
This is helpful to me since I am running on amd64 and freebsd update does not provide updates for !i386. Perhaps I need to donate more money to Colin...?

One question, since you are on i386, why not just use freebsd-update to upate the kernel with a binary update on the slow boxes? I noticed you wanted to keep the default kernel. Any reason for this?
2:05 AM
Richard Bejtlich said…
Hi Joe,

In this example I built GENERIC.SECURITY mainly as an example. When I get a chance to blog again I am going to contrast performance of GENERIC.SECURITY with GENERIC.POLLING. I'll build GENERIC.POLLING using the method in this post.
6:07 AM
Anonymous said…
joe,

FreeBSD Update will be supporting AMD64 soon. I've talked to someone who will be providing the hardware needed -- my understanding is that we're just waiting for someone at the datacentre to add it to the FreeBSD cluster.
11:30 PM
Anonymous said…
Can this method be used in conjunction with freebsd-update?

I'm currently running a custom 6.0 kernel (turned on IPFilter).

## I want to move to freebsd-update to take care of 6.0 > 6.1 and security patches.
First I restore GENERIC kernel:
cd /usr/src
make buildkernel KERNCONF=GENERIC
make installkernel KERNCONF=GENERIC

I have not (that I recall) updated any kernel sources.

Next I use freebsd-update (installed from ports) according to its man page.

## I want to use the above method to extend the GENERIC kernel without modifying it.
Implement GENERIC.FIREWALL to add in IPFilter (later pfSense).

Am I correct in assuming that you only revert to GENERIC when using freebsd-update, then once the binary update is complete, you recompile with GENERIC.SECURITY?

Thanks for any response.

PS I tried to post this comment in Camino 1.0.3 (cookies off), I could not see the word for verification; so I used Safari.
11:58 AM
Richard Bejtlich said…
You can run freebsd-update on a box with a custom kernel. freebsd-update will say the kernel is not GENERIC, but it will then update the userland without a problem.
9:45 AM
Anonymous said…
This comment has been removed by a blog administrator.
3:41 AM
Rick said…
I realize this post is old, but it was able to get me along into the process as well. However, I'd like to know if there can be a menu item added to the beastie when a system boots.

Here is my scenario...I would like to have 2 kernels installed. The default kernel is intended to be the production run-time kernel and the 2nd is a debug kernel with DDB enabled.

If a system kernel panics, it will reboot and boot into the default kernel. What if I want to choose the menu option to boot into this debug kernel in case it panics again relatively soon after rebooting?
1:46 PM
Post a Comment

Popular posts from this blog

Zeek in Action Videos

Image
This is a quick note to point blog readers to my Zeek in Action YouTube video series for the Zeek network security monitoring project .  Each video addresses a topic that I think might be of interest to people trying to understand their network using Zeek and adjacent tools and approaches, like Suricata, Wireshark, and so on.  I am especially pleased with Video 6 on monitoring wireless networks . It took me several weeks to research material for this video. I had to buy new hardware and experiment with a Linux distro that I had not used before -- Parrot .  Please like and subscribe, and let me know if there is a topic you think might make a good video.
Read more

MITRE ATT&CK Tactics Are Not Tactics

Image
Just what are "tactics"? Introduction MITRE ATT&CK  is a great resource, but something about it has bothered me since I first heard about it several years ago. It's a minor point, but I wanted to document it in case it confuses anyone else. The MITRE ATT&CK Design and Philosophy document from March 2020 says the following: At a high-level, ATT&CK is a behavioral model that consists of the following core components: • Tactics, denoting short-term, tactical adversary goals during an attack; • Techniques, describing the means by which adversaries achieve tactical goals; • Sub-techniques, describing more specific means by which adversaries achieve tactical goals at a lower level than techniques; and • Documented adversary usage of techniques, their procedures, and other metadata. My concern is with MITRE's definition of "tactics" as "short-term, tactical adversary goals during an attack," which is oddly recursive. The key word in the tacti...
Read more

New Book! The Best of TaoSecurity Blog, Volume 4

Image
  I've completed the TaoSecurity Blog book series . The new book is  The Best of TaoSecurity Blog, Volume 4: Beyond the Blog with Articles, Testimony, and Scholarship .  It's available now for Kindle , and I'm working on the print edition.  I'm running a 50% off promo on Volumes 1-3 on Kindle through midnight 20 April. Take advantage before the prices go back up. I described the new title thus: Go beyond TaoSecurity Blog with this new volume from author Richard Bejtlich. In the first three volumes of the series, Mr. Bejtlich selected and republished the very best entries from 18 years of writing and over 18 million blog views, along with commentaries and additional material.  In this title, Mr. Bejtlich collects material that has not been published elsewhere, including articles that are no longer available or are stored in assorted digital or physical archives. Volume 4 offers early white papers that Mr. Bejtlich wrote as a network defender, either for technica...
Read more