Building Kernel and World on One System, Installing on Another

I'd read Tracking for Multiple Machines in the FreeBSD Handbook, which gives hints on building the FreeBSD userland, or "world," and kernel on one system and installing them on another system. You might do this because the target system is slow and your build machine is fast, or because you prefer to let production machines serve users rather than use CPU cycles rebuilding the world and kernel.

Inspired by this post, I decided to try building the world and kernel on my package builder, "neely," and installing them on target systems. I chose my sensor platform "bourque," as a test system.

First I made sure that neely's world was up to date. I'm tracking the security release of FreeBSD 5.2 (which takes care of 5.2.1). My /usr/local/etc/release-supfile looks like this, with all of the commented lines removed:

*default host=cvsup16.FreeBSD.org
*default base=/usr
*default prefix=/usr
*default release=cvs tag=RELENG_5_2
*default delete use-rel-suffix

*default compress

src-all

I ran this script, update_source_release.sh, which takes care of neely's world and kernel:

#!/bin/sh
# Source updated by Richard Bejtlich
# 1215 07 Nov 03

SERVER=`fastest_cvsup -q -c us`

HOSTNAME=`/bin/hostname -s`
echo "Beginning to update $HOSTNAME."

echo "cvsup -g -L 2 -h $SERVER /usr/local/etc/release-supfile"
cvsup -g -L 2 /usr/local/etc/release-supfile

echo "cd /usr/src/"
cd /usr/src/
echo "/usr/src/usr.sbin/mergemaster/mergemaster.sh -p"
/usr/src/usr.sbin/mergemaster/mergemaster.sh -p

echo "cd /usr/obj"
cd /usr/obj
echo "chflags -R noschg *"
chflags -R noschg *
echo "rm -rf *"
rm -rf *
echo "cd /usr/src"
cd /usr/src
echo "make cleandir"
make cleandir
echo "make cleandir"
make cleandir
echo "make buildworld"
make buildworld

echo "cd /usr/src"
cd /usr/src
echo "make buildkernel KERNCONF=$HOSTNAME"
make buildkernel KERNCONF=$HOSTNAME
echo "make installkernel KERNCONF=$HOSTNAME"
make installkernel KERNCONF=$HOSTNAME

echo "make installworld"
make installworld
echo "/usr/src/usr.sbin/mergemaster/mergemaster.sh"
/usr/src/usr.sbin/mergemaster/mergemaster.sh

echo "Done updating $HOSTNAME"

Now that neely's world and kernel were up to date, I decided to rebuild the kernel for other machines.

I created a separate kernel configuration file for bourque in neely's /usr/src/sys/i386/conf directory. I named that file "bourque" and edited it to suit the bourque system's kernel needs. Next I created the following script to build a kernel for bourque:

neely:/usr/local/bin$ cat rebuild_kernel_bourque.sh 
#!/bin/sh
# Kernel rebuilder by Richard Bejtlich
# 1550 26 May 04

HOSTNAME=bourque
echo "Beginning to rebuild $HOSTNAME kernel."

echo "cd /usr/src"
cd /usr/src
echo "make buildkernel KERNCONF=$HOSTNAME"
make buildkernel KERNCONF=$HOSTNAME

echo "Done rebuilding $HOSTNAME kernel."

When I ran the script, here is some of what I saw:

neely# rebuild_kernel_bourque.sh

Beginning to rebuild bourque kernel.
cd /usr/src
make buildkernel KERNCONF=bourque

--------------------------------------------------------------
>>> Kernel build for bourque started on Wed May 26 16:42:32 EDT 2004
--------------------------------------------------------------
===> bourque
mkdir -p /usr/obj/usr/src/sys
cd /usr/src/sys/i386/conf;  PATH=/usr/obj/usr/src/i386/legacy/usr/sbin:/usr/obj/
usr/src/i386/legacy/usr/bin:/usr/obj/usr/src/i386/legacy/usr/games:/usr/obj/usr/
src/i386/usr/sbin:/usr/obj/usr/src/i386/usr/bin:/usr/obj/usr/src/i386/usr/games:
/sbin:/bin:/usr/sbin:/usr/bin  config  -d /usr/obj/usr/src/sys/bourque 
 /usr/src/sys/i386/conf/bourque
Kernel build directory is /usr/obj/usr/src/sys/bourque
Don't forget to do a ``make depend''
cd /usr/obj/usr/src/sys/bourque;  MAKEOBJDIRPREFIX=/usr/obj 
 MACHINE_ARCH=i386  MACHINE=i386  CPUTYPE= 
 GROFF_BIN_PATH=/usr/obj/usr/src/i386/legacy/usr/bin  
...truncated...

I ensured that neely was exporting its /usr directory via NFS with this /etc/exports file:

/usr -alldirs

Now I moved to bourque, where I created and ran this script:

#!/bin/sh
# Source updater via master builder by Richard Bejtlich
# 1614 26 May 04 

HOSTNAME=`/bin/hostname -s`
echo "Beginning to update $HOSTNAME."

echo "Mounting /usr/src and /usr/obj on neely"
mount_nfs -T neely:/usr/src /usr/src
mount_nfs -T neely:/usr/obj /usr/obj
mount

echo "cd /usr/src"
cd /usr/src
echo "make installkernel KERNCONF=$HOSTNAME"
make installkernel KERNCONF=$HOSTNAME

echo "make installworld"
make installworld
echo "/usr/src/usr.sbin/mergemaster/mergemaster.sh"
/usr/src/usr.sbin/mergemaster/mergemaster.sh

echo "Done updating $HOSTNAME"

Here is some of what I saw when I ran the script:

bourque# update_source_release.sh ^M
Beginning to update bourque.
Mounting /usr/src and /usr/obj on neely
/dev/ad0s1a on / (ufs, local)
devfs on /dev (devfs, local)
/dev/ad0s1f on /home (ufs, local, soft-updates)
/dev/ad1s1d on /nsm (ufs, local, soft-updates)
/dev/ad0s1g on /tmp (ufs, local, soft-updates)
/dev/ad0s1d on /usr (ufs, local, soft-updates)
/dev/ad0s1e on /var (ufs, local, soft-updates)
/dev/ad3s1d on /var/db (ufs, local, soft-updates)
neely:/usr/src on /usr/src (nfs)
neely:/usr/obj on /usr/obj (nfs)
cd /usr/src
make installkernel KERNCONF=bourque
cd /usr/obj/usr/src/sys/bourque;  MAKEOBJDIRPREFIX=/usr/obj  MACHINE_ARCH=i386  
MACHINE=i386  CPUTYPE=  GROFF_BIN_PATH=/usr/obj/usr/src/i386/legacy/usr/bin  GRO
FF_FONT_PATH=/usr/obj/usr/src/i386/legacy/usr/share/groff_font  GROFF_TMAC_PATH=
/usr/obj/usr/src/i386/legacy/usr/share/tmac PATH=/usr/obj/usr/src/i386/legacy/us
r/sbin:/usr/obj/usr/src/i386/legacy/usr/bin:/usr/obj/usr/src/i386/legacy/usr/gam
es:/usr/obj/usr/src/i386/usr/sbin:/usr/obj/usr/src/i386/usr/bin:/usr/obj/usr/src
/i386/usr/games:/sbin:/bin:/usr/sbin:/usr/bin  make KERNEL=kernel install
...truncated...

When I was done I rebooted bourque, and then checked uname output:

bourque# uname -a
FreeBSD bourque.taosecurity.com 5.2.1-RELEASE-p8 FreeBSD 5.2.1-RELEASE-p8 #1:
 Wed May 26 16:42:32 EDT 2004 root@neely.taosecurity.com:
 /usr/obj/usr/src/sys/bourque  i386

I've started using this method on all of my FreeBSD machines with nonstandard kernels. Systems with GENERIC kernels can still track Colin Percival's freebsd-update.

Update: After speaking with Colin, I have another option that is a hybrid of building your own kernel and using Colin's updates of the userland.

Build the kernel you need for the remote system as shown above with the rebuild_kernel_bourque.sh script. This creates a kernel for the remote machine. On the remote machine, run freebsd-update. If you have a GENERIC kernel on the remote machine, freebsd-update will update the kernel as well as the userland. Now install your kernel using this modified script:

#!/bin/sh
# Source updater via master builder, kernel only by Richard Bejtlich
# 1445 4 Jun 04

HOSTNAME=`/bin/hostname -s`
echo "Beginning to update $HOSTNAME."

echo "Mounting /usr/src and /usr/obj on neely"
mount_nfs -T host_with_kernel:/usr/src /usr/src
mount_nfs -T host_with_kernel:/usr/obj /usr/obj
mount

echo "cd /usr/src"
cd /usr/src
echo "make installkernel KERNCONF=$HOSTNAME"
make installkernel KERNCONF=$HOSTNAME

#No need to installworld when using freebsd-update
#echo "make installworld"
#make installworld
echo "/usr/src/usr.sbin/mergemaster/mergemaster.sh"
/usr/src/usr.sbin/mergemaster/mergemaster.sh

echo "Done updating $HOSTNAME"

Essentially the only change is not running 'make installworld'. The next time you run freebsd-update on the remote machine, freebsd-update will update userland and leave the new kernel alone.

Comments

Post a Comment

Popular posts from this blog

Zeek in Action Videos

Image
This is a quick note to point blog readers to my Zeek in Action YouTube video series for the Zeek network security monitoring project .  Each video addresses a topic that I think might be of interest to people trying to understand their network using Zeek and adjacent tools and approaches, like Suricata, Wireshark, and so on.  I am especially pleased with Video 6 on monitoring wireless networks . It took me several weeks to research material for this video. I had to buy new hardware and experiment with a Linux distro that I had not used before -- Parrot .  Please like and subscribe, and let me know if there is a topic you think might make a good video.
Read more

MITRE ATT&CK Tactics Are Not Tactics

Image
Just what are "tactics"? Introduction MITRE ATT&CK  is a great resource, but something about it has bothered me since I first heard about it several years ago. It's a minor point, but I wanted to document it in case it confuses anyone else. The MITRE ATT&CK Design and Philosophy document from March 2020 says the following: At a high-level, ATT&CK is a behavioral model that consists of the following core components: • Tactics, denoting short-term, tactical adversary goals during an attack; • Techniques, describing the means by which adversaries achieve tactical goals; • Sub-techniques, describing more specific means by which adversaries achieve tactical goals at a lower level than techniques; and • Documented adversary usage of techniques, their procedures, and other metadata. My concern is with MITRE's definition of "tactics" as "short-term, tactical adversary goals during an attack," which is oddly recursive. The key word in the tacti
Read more

New Book! The Best of TaoSecurity Blog, Volume 4

Image
  I've completed the TaoSecurity Blog book series . The new book is  The Best of TaoSecurity Blog, Volume 4: Beyond the Blog with Articles, Testimony, and Scholarship .  It's available now for Kindle , and I'm working on the print edition.  I'm running a 50% off promo on Volumes 1-3 on Kindle through midnight 20 April. Take advantage before the prices go back up. I described the new title thus: Go beyond TaoSecurity Blog with this new volume from author Richard Bejtlich. In the first three volumes of the series, Mr. Bejtlich selected and republished the very best entries from 18 years of writing and over 18 million blog views, along with commentaries and additional material.  In this title, Mr. Bejtlich collects material that has not been published elsewhere, including articles that are no longer available or are stored in assorted digital or physical archives. Volume 4 offers early white papers that Mr. Bejtlich wrote as a network defender, either for technical or pol
Read more