Multiple Kernels on FreeBSD
The following is a topic I would enjoy hearing more about. If you have helpful suggestions, please share them as a comment.Two years ago I described my experiences with building a FreeBSD userland and kernel on one system and installing it on another. I found myself in the same situation recently, where I didn't want to sit around waiting for a couple slow boxes to build themselves custom kernels. I wanted to build the custom kernel on a fast box and use it on the slower boxes. I didn't want to replace the default kernel on any of the boxes. I wanted the new kernel(s) to be additional boot-time options.
This post gave me the answer I needed. Here's how I applied it.
I wanted to build a GENERIC-style kernel, but with security updates applied. First I installed cvsup-without-gui as a package. Next I created this /usr/local/etc/security-supfile file:
*default host=cvsup5.FreeBSD.org
*default base=/usr
*default prefix=/usr
*default release=cvs tag=RELENG_6_1
*default delete use-rel-suffix
*default compress
src-all
This would update my kernel sources and userland to the SECURITY branch effective the time I ran cvsup (next).
kbld# cvsup -g -L 2 /usr/local/etc/security-supfile
Parsing supfile "/usr/local/etc/security-supfile"
Connecting to cvsup5.FreeBSD.org
Connected to cvsup5.FreeBSD.org
Server software version: SNAP_16_1h
Negotiating file attribute support
Exchanging collection information
Establishing multiplexed-mode data connection
Running
Updating collection src-all/cvs
Edit src/UPDATING
Add delta 1.416.2.22.2.3 2006.05.31.22.31.41 cperciva
Add delta 1.416.2.22.2.4 2006.06.14.15.59.27 cperciva
Add delta 1.416.2.22.2.5 2006.07.07.07.25.21 cperciva
Add delta 1.416.2.22.2.6 2006.08.23.22.02.25 cperciva
...edited...
Shutting down connection to server
Finished successfully
Next I created the file GENERIC.SECURITY in /usr/src/sys/i386/conf with the following:
include GENERIC
All that does is make GENERIC.SECURITY the same kernel as GENERIC, except with patches applied. At this point you might think I should just update the GENERIC kernel. I could do that, but I'm using this method because later steps show this system works best for my requirements.
Now I can build the kernel.
kbld# cd /usr/src
kbld# make buildkernel KERNCONF=GENERIC.SECURITY INSTKERNNAME=GENERIC.SECURITY
--------------------------------------------------------------
>>> Kernel build for GENERIC.SECURITY started on Wed Sep 20 19:54:46 EDT 2006
--------------------------------------------------------------
===> GENERIC.SECURITY
mkdir -p /usr/obj/usr/src/sys
--------------------------------------------------------------
>>> stage 1: configuring the kernel
--------------------------------------------------------------
...truncated...
--------------------------------------------------------------
>>> Kernel build for GENERIC.SECURITY completed on Wed Sep 20 20:12:42 EDT 2006
--------------------------------------------------------------
Next I installed the kernel.
kbld:/usr/src# make installkernel KERNCONF=GENERIC.SECURITY INSTKERNNAME=GENERIC.SECURITY
--------------------------------------------------------------
>>> Installing kernel
--------------------------------------------------------------
...edited...
kldxref /boot/GENERIC.SECURITY
kbld:/usr/src#
That's it. I make sure host kbld is exporting the appropriate directories via NFS by creating this /etc/exports file:
/usr -alldirs
That's too loose but this is sufficient for my test network.
Now I move from the kernel builder to a slow system where I would like to make GENERIC.SECURITY available. 192.168.2.103 is kbld, where the new kernel is waiting.
asa633:/root# mount_nfs 192.168.2.103:/usr/src /usr/src
asa633:/root# mount_nfs 192.168.2.103:/usr/obj /usr/obj
asa633:/root# mount
/dev/ad0s1a on / (ufs, local)
devfs on /dev (devfs, local)
/dev/ad0s1f on /home (ufs, local, soft-updates)
/dev/ad1s1d on /nsm (ufs, local, soft-updates)
/dev/ad0s1g on /tmp (ufs, local, soft-updates)
/dev/ad0s1d on /usr (ufs, local, soft-updates)
/dev/ad0s1e on /var (ufs, local, soft-updates)
192.168.2.103:/usr/src on /usr/src (nfs)
192.168.2.103:/usr/obj on /usr/obj (nfs)
asa633:/usr/src# make installkernel KERNCONF=GENERIC.SECURITY INSTKERNNAME=GENERIC.SECURITY
--------------------------------------------------------------
>>> Installing kernel
--------------------------------------------------------------
...edited...
kldxref /boot/GENERIC.SECURITY
How do I get this GENERIC.SECURITY kernel to boot? If I were at the console at boot time, I could say 'boot GENERIC.SECURITY'. Since I am remote, I edit /boot/loader.conf to say
kernel=GENERIC.SECURITY
Now I reboot. After rebooting, I see the new kernel is installed:
asa633:/root# uname -a
FreeBSD asa633.taosecurity.com 6.1-RELEASE-p6 FreeBSD 6.1-RELEASE-p6 #0:
Wed Sep 20 20:02:56 EDT 2006
root@kbld.taosecurity.com:/usr/obj/usr/src/sys/GENERIC.SECURITY i386
Pretty easy. If I want to boot the default kernel, I remove the entry in /boot/loader.conf.
For example, asa633 is usually running the kernel provided by Colin Percival's FreeBSD-Update code:
asa633:/root# uname -a
FreeBSD asa633.taosecurity.com 6.1-SECURITY FreeBSD 6.1-SECURITY #0:
Mon Aug 28 05:21:08 UTC 2006
root@builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC i386
FreeBSD-Update, in fact, very nicely takes care of the latest security problems with Gzip:
asa633:/root# freebsd-update fetch
Fetching updates signature...
Fetching updates...
Fetching hash list signature...
Fetching hash list...
Examining local system...
Fetching updates...
/usr/bin/gunzip...
/usr/bin/gzcat...
/usr/bin/gzip...
/usr/bin/zcat...
Updates fetched
To install these updates, run: '/usr/local/sbin/freebsd-update install'
asa633:/root# freebsd-update install
Backing up /usr/bin/gunzip...
Installing new /usr/bin/gunzip...
Backing up /usr/bin/gzcat...
Recreating hard link from /usr/bin/gunzip to /usr/bin/gzcat...
Backing up /usr/bin/gzip...
Recreating hard link from /usr/bin/gunzip to /usr/bin/gzip...
Backing up /usr/bin/zcat...
Recreating hard link from /usr/bin/gunzip to /usr/bin/zcat...
Easy!
Comments
One question, since you are on i386, why not just use freebsd-update to upate the kernel with a binary update on the slow boxes? I noticed you wanted to keep the default kernel. Any reason for this?
In this example I built GENERIC.SECURITY mainly as an example. When I get a chance to blog again I am going to contrast performance of GENERIC.SECURITY with GENERIC.POLLING. I'll build GENERIC.POLLING using the method in this post.
FreeBSD Update will be supporting AMD64 soon. I've talked to someone who will be providing the hardware needed -- my understanding is that we're just waiting for someone at the datacentre to add it to the FreeBSD cluster.
I'm currently running a custom 6.0 kernel (turned on IPFilter).
## I want to move to freebsd-update to take care of 6.0 > 6.1 and security patches.
First I restore GENERIC kernel:
cd /usr/src
make buildkernel KERNCONF=GENERIC
make installkernel KERNCONF=GENERIC
I have not (that I recall) updated any kernel sources.
Next I use freebsd-update (installed from ports) according to its man page.
## I want to use the above method to extend the GENERIC kernel without modifying it.
Implement GENERIC.FIREWALL to add in IPFilter (later pfSense).
Am I correct in assuming that you only revert to GENERIC when using freebsd-update, then once the binary update is complete, you recompile with GENERIC.SECURITY?
Thanks for any response.
PS I tried to post this comment in Camino 1.0.3 (cookies off), I could not see the word for verification; so I used Safari.
Here is my scenario...I would like to have 2 kernels installed. The default kernel is intended to be the production run-time kernel and the 2nd is a debug kernel with DDB enabled.
If a system kernel panics, it will reboot and boot into the default kernel. What if I want to choose the menu option to boot into this debug kernel in case it panics again relatively soon after rebooting?