Building and Deploying FreeBSD Packages

FreeBSD documentation is excellent, but I haven't found information on strategies for enterprise system administration duties. For example, what is the best way to deploy and upgrade software on multiple machines? Slashdot recently discussed building from source vs packages, but this topic doesn't get much public discussion. Most documentation talks about installing ports or packages from the perspective of a single machine. There's little or no material aside from newsgroup postings on ways to be more efficient.

It makes more sense to me to designate the most powerful system at hand as a "package builder." Sys admins build their own packages from source on this machine and then deploy them on workstations and other servers. For example, I use my Shuttle SB52G2, named 'neely', as a package builder. It runs FreeBSD 5.2.1, like most of the systems in my lab. Right now I'm building the newest OpenOffice port from source. It was recently updated to version 1.1.1. The FreeBSD OpenOffice.org site still shows packages for OO.org 1.1, and ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-current/All/ doesn't provide OpenOffice packages.

I'm building the package using this method:

cd /usr/ports/editors/openoffice-1.1
make package-recursive install

This tells FreeBSD to create a package for OpenOffice in /usr/ports/packages/All. (Note you need to create this directory if you don't have it already.) Saying "package-recursive" instead of just "package" tells FreeBSD to create packages of all of OpenOffice's dependencies. How do I know this? Look at /usr/ports/Mk/bsd.port.mk, where we find:

# package               - Create a package from an _installed_ port.
# package-recursive - Create a package for a port and _all_ of its dependancies.

I am building OO.org on neely, but it's already installed on my laptop, 'orr'. To share the packages built on neely, I export neely's /usr/ports directory via NFS. This means I only have to upgrade the ports tree on neely. I use the following script to keep neely's ports tree up-to-date. If I want to use a specific CVS server, I comment out the SERVER variable using 'fastest_cvsup' and uncomment the SERVER variable using a named CVS server. If I want to clean up the distfiles and so forth, I uncomment the portsclean command at the end of the script.

#!/bin/sh
# Ports updater by Richard Bejtlich
# 0925 07 Nov 03
# 2100 07 Feb 04 - added portsclean -CDD

SERVER=`fastest_cvsup -q -c us`
START_TIME=`/bin/date`

#SERVER=cvsup16.freebsd.org

echo "Starting ports update at $START_TIME."

echo "cvsup -g -L 2 -h $SERVER /usr/local/etc/ports-supfile"
cvsup -g -L 2 -h $SERVER /usr/local/etc/ports-supfile

# According to portsdb(1), INDEX is updated only once per month or so.
# A look at http://www.freebsd.org/cgi/cvsweb.cgi/ports/#dirlist shows
# INDEX (the ports index for 4) and INDEX-5 (this ports index for 5).
# portsb(1) recommends running portsdb -uU

echo "portsdb -uU"
portsdb -uU

echo "cd /var/db"
cd /var/db

echo "pkgdb -u"
pkgdb -u

echo "portversion -v"
portversion -v

echo "portupgrade -varRp"
portupgrade -varRp

#echo "portsclean -CDD"
#portsclean -CDD

echo "Done updating ports tree at `/bin/date`."

Let's say I want to update the JDK on orr. First I update neely's ports tree and update neely's ports. The "-p" switch given to portupgrade in the previous script ensures portupgrade builds a package of the JDK when it's updated. On orr, I use this command once I've mounted neely's /usr/ports to orr's /usr/ports. The "-R" tells portupgrade to upgrade packages required by the given package. "-PP" tells portupgrade to only update using packages; in other words, it won't start updating by downloading source code. "-v" means be verbose.

orr:/root# portupgrade -RvPP jdk
--->  Session started at: Sun, 04 Apr 2004 11:11:56 -0400
** No need to upgrade 'perl-5.6.1_15' (>= perl-5.6.1_15). (specify -f to force)
--->  Checking the availability of the latest package of 'java/javavmwrapper'
--->  Found a package of 'java/javavmwrapper': /usr/ports/packages/All/javavmwrapper-1.5.tbz
--->  Upgrade of java/javavmwrapper started at: Sun, 04 Apr 2004 11:12:00 -0400
--->  Upgrading 'javavmwrapper-1.4' to 'javavmwrapper-1.5' (java/javavmwrapper) 
using a package
--->  Updating dependency info
--->  Modifying /var/db/pkg/jdk-1.4.2p5/+CONTENTS
--->  Modifying /var/db/pkg/linux-sun-jdk-1.4.2.02/+CONTENTS
--->  Uninstallation of javavmwrapper-1.4 started at: Sun, 04 Apr 2004 11:12:05 -0400
--->  Fixing up dependencies before creating a package
--->  Backing up the old version
--->  Uninstalling the old version
--->  Deinstalling 'javavmwrapper-1.4'
pkg_delete: package 'javavmwrapper-1.4' is required by these other packages 
and may not be deinstalled (but I'll delete it anyway):
jdk-1.4.2p5
linux-sun-jdk-1.4.2.02
[Updating the pkgdb  in /var/db/pkg ... - 250 packages found 
(-1 +0) (...) done]
--->  Uninstallation of javavmwrapper-1.4 ended at: Sun, 04 Apr 2004 11:12:08 -0400
 (consumed 00:00:03)
--->  Installation of javavmwrapper-1.5 started at: Sun, 04 Apr 2004 11:12:08 -0400
--->  Installing the new version via the package
--->  Removing the temporary backup files
--->  Installation of javavmwrapper-1.5 ended at: Sun, 04 Apr 2004 11:12:10 -0400
 (consumed 00:00:01)
--->  Cleaning out obsolete shared libraries
[Updating the pkgdb  in /var/db/pkg ... - 251 packages found 
(-0 +1) . done]
--->  Upgrade of java/javavmwrapper ended at: Sun, 04 Apr 2004 11:12:12 -0400
 (consumed 00:00:11)
** No need to upgrade 'imake-4.3.0_2' (>= imake-4.3.0_2). (specify -f to force)
** No need to upgrade 'expat-1.95.7' (>= expat-1.95.7). (specify -f to force)
** No need to upgrade 'pkgconfig-0.15.0_1' (>= pkgconfig-0.15.0_1). (specify -f to force)
** No need to upgrade 'freetype2-2.1.7_2' (>= freetype2-2.1.7_2). (specify -f to force)
** No need to upgrade 'fontconfig-2.2.2,1' (>= fontconfig-2.2.2,1). (specify -f to force)
** No need to upgrade 'XFree86-libraries-4.3.0_6' (>= XFree86-libraries-4.3.0_6).
 (specify -f to force)
** No need to upgrade 'urwfonts-1.0' (>= urwfonts-1.0). (specify -f to force)
** No need to upgrade 'open-motif-2.2.2_2' (>= open-motif-2.2.2_2). (specify -f to force)
--->  Checking the availability of the latest package of 'java/jdk14'
--->  Found a package of 'java/jdk14': /usr/ports/packages/All/jdk-1.4.2p6_4.tbz
--->  Upgrade of java/jdk14 started at: Sun, 04 Apr 2004 11:14:04 -0400
--->  Upgrading 'jdk-1.4.2p5' to 'jdk-1.4.2p6_4' (java/jdk14) using a package
--->  Updating dependency info
--->  Uninstallation of jdk-1.4.2p5 started at: Sun, 04 Apr 2004 11:14:09 -0400
--->  Fixing up dependencies before creating a package
--->  Backing up the old version
--->  Uninstalling the old version
--->  Deinstalling 'jdk-1.4.2p5'
[Updating the pkgdb  in /var/db/pkg ... - 250 packages found 
(-1 +0) (...) done]
--->  Uninstallation of jdk-1.4.2p5 ended at: Sun, 04 Apr 2004 11:16:31 -0400
 (consumed 00:02:22)
--->  Installation of jdk-1.4.2p6_4 started at: Sun, 04 Apr 2004 11:17:08 -0400
--->  Installing the new version via the package

SUN COMMUNITY SOURCE LICENSE Version 2.3 (Rev.  Date Feb. 
23, 1999)
...edited...
--->  Removing the temporary backup files
--->  Installation of jdk-1.4.2p6_4 ended at: Sun, 04 Apr 2004 11:18:14 -0400 (c
onsumed 00:01:05)
--->  Cleaning out obsolete shared libraries
[Updating the pkgdb  in /var/db/pkg ... - 251 packages found 
(-0 +1) . done]
--->  Upgrade of java/jdk14 ended at: Sun, 04 Apr 2004 11:18:19 -0400
 (consumed 00:04:14)
--->  Reporting the results (+:done / -:ignored / *:skipped / !:failed)
        - lang/perl5 (perl-5.6.1_15)
        + java/javavmwrapper (javavmwrapper-1.4)
        - devel/imake-4 (imake-4.3.0_2)
        - textproc/expat2 (expat-1.95.7)
        - devel/pkgconfig (pkgconfig-0.15.0_1)
        - print/freetype2 (freetype2-2.1.7_2)
        - x11-fonts/fontconfig (fontconfig-2.2.2,1)
        - x11/XFree86-4-libraries (XFree86-libraries-4.3.0_6)
        - x11-fonts/urwfonts (urwfonts-1.0)
        - x11-toolkits/open-motif (open-motif-2.2.2_2)
        + java/jdk14 (jdk-1.4.2p5)
--->  Session ended at: Sun, 04 Apr 2004 11:18:27 -0400 (consumed 00:06:30)

This process allows me to build all of my packages on one machine, and then install them quickly and easily on similar systems. I cannot use this method to share packages built on FreeBSD 5.x with systems running 4.x, however. At the moment I've confined myself to sharing packages between machines running strictly the same OS version.

I like this method because it also ensures I don't waste time on slower machines building packages. Furthermore, it ensures that only packages which are capable of being built are deployed on end systems. Sometimes ports are broken, so by using a dedicated package building system I test all of those issues on one box.

When I finally had the package of OpenOffice 1.1.1 built, I used 'portupgrade -PPRv openoffice' to update orr. OpenOffice installed a new copy of its localization files in ~/OpenOffice.org1.1.1 and I had to tell it to look in /usr/local/jdk1.4.2/jre to find the Java Runtime Environment it needed. I also encountered the same two missing files documented here, and used the same method to fix the issue:

orr:/home/richard/OpenOffice.org1.1.0$ find . -name libmozab*.so
./program/libmozab2.so
./program/libmozabdrv2.so 
orr:/home/richard/OpenOffice.org1.1.0$ cp ./program/libmozab*
 /home/richard/OpenOffice.org1.1.1/program/

Popular posts from this blog

Zeek in Action Videos

Image
This is a quick note to point blog readers to my Zeek in Action YouTube video series for the Zeek network security monitoring project .  Each video addresses a topic that I think might be of interest to people trying to understand their network using Zeek and adjacent tools and approaches, like Suricata, Wireshark, and so on.  I am especially pleased with Video 6 on monitoring wireless networks . It took me several weeks to research material for this video. I had to buy new hardware and experiment with a Linux distro that I had not used before -- Parrot .  Please like and subscribe, and let me know if there is a topic you think might make a good video.
Read more

MITRE ATT&CK Tactics Are Not Tactics

Image
Just what are "tactics"? Introduction MITRE ATT&CK  is a great resource, but something about it has bothered me since I first heard about it several years ago. It's a minor point, but I wanted to document it in case it confuses anyone else. The MITRE ATT&CK Design and Philosophy document from March 2020 says the following: At a high-level, ATT&CK is a behavioral model that consists of the following core components: • Tactics, denoting short-term, tactical adversary goals during an attack; • Techniques, describing the means by which adversaries achieve tactical goals; • Sub-techniques, describing more specific means by which adversaries achieve tactical goals at a lower level than techniques; and • Documented adversary usage of techniques, their procedures, and other metadata. My concern is with MITRE's definition of "tactics" as "short-term, tactical adversary goals during an attack," which is oddly recursive. The key word in the tacti
Read more

New Book! The Best of TaoSecurity Blog, Volume 4

Image
  I've completed the TaoSecurity Blog book series . The new book is  The Best of TaoSecurity Blog, Volume 4: Beyond the Blog with Articles, Testimony, and Scholarship .  It's available now for Kindle , and I'm working on the print edition.  I'm running a 50% off promo on Volumes 1-3 on Kindle through midnight 20 April. Take advantage before the prices go back up. I described the new title thus: Go beyond TaoSecurity Blog with this new volume from author Richard Bejtlich. In the first three volumes of the series, Mr. Bejtlich selected and republished the very best entries from 18 years of writing and over 18 million blog views, along with commentaries and additional material.  In this title, Mr. Bejtlich collects material that has not been published elsewhere, including articles that are no longer available or are stored in assorted digital or physical archives. Volume 4 offers early white papers that Mr. Bejtlich wrote as a network defender, either for technical or pol
Read more