Wednesday, November 24, 2004

The traditional means to update the FreeBSD ports tree involves using CVSup in a manner described well by Dru Lavigne. After the sysutils/cvsup
port is installed, and ports-supfile modified to point to a real CVSup server (e.g. *default host=cvsup8.FreeBSD.org'), you run commands like this:

cvsup -g -L 2 /usr/local/etc/ports-supfile
portsdb -uU

The first command updates the ports tree in /usr/ports. The second command creates or updates the INDEX file (via the 'U') and then creates or updates the INDEX.db file (via the 'u') from the INDEX.

There is now an alternative to CVSup available in the ports tree: Portsnap, written by FreeBSD-update author Colin Percival. Installation via packages is easy:

orr:/root# setenv PACKAGESITE ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-5-stable/Latest/
orr:/root# pkg_add -vr portsnap
...edited...
Package portsnap-0.2_1 registered in /var/db/pkg/portsnap-0.2_1

Before you can use portsnap, you will have to create an update configuration
file specifying the server from which to fetch snapshots and the sha1 hash
of the RSA public key which is trusted to sign the snapshots.

A sample configuration file has been installed in

/usr/local/etc/portsnap.conf.sample

which will fetch snapshots built and signed by the author. If you want to
use these updates, copy that file to

/usr/local/etc/portsnap.conf

otherwise, create that file as appropriate.

orr:/root# cp /usr/local/etc/portsnap.conf.sample /usr/local/etc/portsnap.conf
orr:/root# mkdir /usr/local/portsnap

Once installed, see what options Portsnap offers:

orr:/root# portsnap -h
usage: portsnap [options] command [URL]

Options:
-d workdir -- Store working files in workdir
(default: /usr/local/portsnap/)
-f conffile -- Read configuration options from conffile
(default: /usr/local/etc/portsnap.conf)
-k KEY -- Trust an RSA key with SHA1 hash of KEY.
-p portsdir -- Location of uncompressed ports tree
(default: /usr/ports/)
URL -- Fetch updates from given URL.
Commands:
fetch -- Fetch a compressed snapshot of the ports tree,
or update an existing snapshot.
cron -- Sleep rand(3600) seconds, and then fetch updates.
extract -- Extract snapshot of ports tree, replacing existing
files and directories.
update -- Update ports tree to match current snapshot, replacing
files and directories which have changed.

Here is a sample run:

orr:/root# portsnap fetch
Fetching public key... done.
Fetching snapshot tag... done.
Fetching snapshot generated at Sun Oct 24 14:21:27 EDT 2004:
c2246a9802a7155d099dd37fdfcbbc0f77b5864a.tgz 3% of 31 MB 192 kBps
Extracting snapshot... done.
Verifying snapshot integrity... done.
Fetching updated snapshot tag... done.
Updating from Sun Oct 24 14:21:27 EDT 2004 to Wed Nov 24 10:04:58 EST 2004.
Attempting to generate index via delta compression... success.
Generating list of updates needed... 2401 files or ports need to be updated.
Attempting to fetch 2210 patches... 2209 fetched.
Attempting to apply patches... done.
Attempting to fetch 192 new files or ports... done.

orr:/root# portsnap extract
/usr/ports/.cvsignore
/usr/ports/CHANGES
/usr/ports/LEGAL
/usr/ports/MOVED
/usr/ports/Makefile
/usr/ports/Mk/bsd.autotools.mk
/usr/ports/Mk/bsd.emacs.mk
/usr/ports/Mk/bsd.gnome.mk
...edited...
/usr/ports/x11/yalias/
/usr/ports/x11/yelp/
/usr/ports/x11/zenity/

After the initial 'extract', future updates just use 'update':

orr:/root# portsnap fetch
Fetching updated snapshot tag... done.
Updating from Wed Nov 24 15:36:20 EST 2004 to Wed Nov 24 17:36:13 EST 2004.
Attempting to generate index via delta compression... success.
Generating list of updates needed... 8 files or ports need to be updated.
Attempting to fetch 8 patches... 8 fetched.
Attempting to apply patches... done.
Attempting to fetch 0 new files or ports... done.
orr:/root# portsnap update
Removing old files and directories... done.
Extracting new files:
/usr/ports/chinese/zhcon/
/usr/ports/comms/minicom/
/usr/ports/devel/gettext/
/usr/ports/games/wolfpack/
/usr/ports/math/ndiff/
/usr/ports/science/mmtk/
/usr/ports/textproc/libxml2/
/usr/ports/textproc/py-docutils/

That's it. Now, the INDEX-5 file hasn't been updated. A great way to do that is to use 'make fetchindex'. Here's a subsequent run of Portsnap followed by 'make index':

orr:/root# portsnap fetch
Fetching updated snapshot tag... done.
Updating from Wed Nov 24 17:36:13 EST 2004 to Wed Nov 24 19:36:13 EST 2004.
Attempting to generate index via delta compression... success.
Generating list of updates needed... 20 files or ports need to be updated.
Attempting to fetch 17 patches... 17 fetched.
Attempting to apply patches... done.
Attempting to fetch 3 new files or ports... done.
orr:/root# portsnap update
Removing old files and directories... done.
Extracting new files:
/usr/ports/Tools/portbuild/
/usr/ports/audio/libsidplay2/
/usr/ports/chinese/MT/
/usr/ports/comms/libticables/
/usr/ports/devel/bison/
/usr/ports/devel/libmcve/
/usr/ports/devel/libtifiles/
/usr/ports/java/linux-blackdown-jdk14/
/usr/ports/lang/gcc28/
/usr/ports/lang/nickle/
/usr/ports/net/jit/
/usr/ports/net/libicq2000/
/usr/ports/www/opera/
/usr/ports/x11-themes/Makefile
/usr/ports/x11-themes/kde-icons-cezanne/
/usr/ports/x11-themes/kde-icons-gartoon-blue-svg/
/usr/ports/x11-themes/kde-icons-gartoon-svg/
/usr/ports/x11-themes/kde-icons-noia/
/usr/ports/x11-themes/kde-icons-sparkling/
/usr/ports/x11/xextensions/
orr:/root# ls -al /usr/ports/IND*
-rw-r--r-- 1 root wheel 5089899 Apr 30 2004 /usr/ports/INDEX
-rw-r--r-- 1 root wheel 6018417 Nov 19 21:02 /usr/ports/INDEX-5
-rw-r--r-- 1 root wheel 12334080 Nov 19 22:22 /usr/ports/INDEX.db
orr:/root# cd /usr/ports
orr:/usr/ports# make fetchindex
INDEX-5.bz2 100% of 589 kB 159 kBps
orr:/usr/ports# ls -al /usr/ports/IND*
-rw-r--r-- 1 root wheel 5089899 Apr 30 2004 /usr/ports/INDEX
-rw-r--r-- 1 root wheel 6040505 Nov 24 21:03 /usr/ports/INDEX-5
-rw-r--r-- 1 root wheel 12334080 Nov 19 22:22 /usr/ports/INDEX.db

Notice how Portsnap did not affect INDEX-5, but 'make fetchindex' brings a new INDEX-5. To update INDEX.db, run 'portsdb -u':

orr:/usr/ports# portsdb -u
[Updating the portsdb in /usr/ports ... - 11999 port entries found ...
......1000.........2000.........3000.........4000.........5000
.........6000.........7000.........8000.........9000.........10000.....
....11000......... ..... done]
orr:/usr/ports# ls -al /usr/ports/IND*
-rw-r--r-- 1 root wheel 5089899 Apr 30 2004 /usr/ports/INDEX
-rw-r--r-- 1 root wheel 6040505 Nov 24 21:03 /usr/ports/INDEX-5
-rw-r--r-- 1 root wheel 12364800 Nov 24 21:21 /usr/ports/INDEX.db

Notice that we're only one port away from 12,000!

3 comments:

Anonymous said...

Good stuff! Basically everything works as expected. I'm a newbie when it comes to using FreeBSD on the notebook and its good to get to try non-conventional tools like portsnap.

I also bought your Security book a few months back. My only complain is that it cost too much ! , but I guess that's the xchange rate fault (or mine for not making a six figure income).

-Adli, Kuala Lumpur

erson from Sweden said...

Now that portsnap has been brought into base it has made "make fetchindex" redundant and after a "portsnap fetch update" you can go right to "portsdb -u". You really don't have to do "portsdb -u" unless you want to since it will be executed when using any of the portupgrade utilities. For example portversion.

Serg said...

The new versions portsnap allow to start

# portsnap fetch update

at once