Tuesday, June 22, 2004

Configuring RAID-0 with Vinum

I deployed a test platform as a network security monitoring sensor. It has two 4 GB HDDs. I wanted to create a /nsm partition that would span both drives, meaning it would occupy some of the first drive and all of the second drive. This was a proof of concept operation that could apply to systems with multiple, larger drives.

I decided to use Greg Lehey's Vinum, and thanks to some helpful notes from Bamm Visscher and Dave Wheeler, got it set up in a RAID-0 configuration.

When I installed FreeBSD on the system, I created a 768 MB /nsm1 partition on the first drive ad0 and used the entire second drive (ad1) for /nsm2. Here is what df saw after installation. Notice I use the -m switch to show all values in MB.

bourque:/root# df -m
Filesystem 1M-blocks Used Avail Capacity Mounted on
/dev/ad0s1a 247 35 192 16% /
devfs 0 0 0 100% /dev
/dev/ad0s1e 739 0 679 0% /nsm1
/dev/ad1s1d 3977 0 3659 0% /nsm2
/dev/ad0s1g 263 0 242 0% /tmp
/dev/ad0s1d 1978 302 1517 17% /usr
/dev/ad0s1f 495 0 455 0% /var

Here is what bsdlabel saw for each drive:

bourque:/root# bsdlabel /dev/ad0s1
# /dev/ad0s1:
8 partitions:
# size offset fstype [fsize bsize bps/cpg]
a: 524288 0 4.2BSD 2048 16384 32776
b: 524288 524288 swap
c: 8421777 0 unused 0 0 # "raw" part, don't edit
d: 4194304 1048576 4.2BSD 2048 16384 28552
e: 1572864 5242880 4.2BSD 2048 16384 28552
f: 1048576 6815744 4.2BSD 2048 16384 8
g: 557457 7864320 4.2BSD 2048 16384 34848

bourque:/root# bsdlabel /dev/ad1s1
# /dev/ad1s1:
8 partitions:
# size offset fstype [fsize bsize bps/cpg]
c: 8418753 0 unused 0 0 # "raw" part, don't edit
d: 8418753 0 4.2BSD 2048 16384 28552

Before I could use vinum, I needed to edit the disk labels so /dev/ad0s1e and /dev/ad1s1d changed from fstype 4.2BSD to vinum.

After running 'bsdlabel -e /dev/ad0s1' and 'bsdlabel -e /dev/ad1s1' here is what the two disks looked like to bsdlabel:

bourque:/root# bsdlabel /dev/ad0s1
# /dev/ad0s1:
8 partitions:
# size offset fstype [fsize bsize bps/cpg]
a: 524288 0 4.2BSD 2048 16384 32776
b: 524288 524288 swap
c: 8421777 0 unused 0 0 # "raw" part, don't edit
d: 4194304 1048576 4.2BSD 2048 16384 28552
e: 1572864 5242880 vinum
f: 1048576 6815744 4.2BSD 2048 16384 8
g: 557457 7864320 4.2BSD 2048 16384 34848

bourque:/root# bsdlabel /dev/ad1s1
# /dev/ad1s1:
8 partitions:
# size offset fstype [fsize bsize bps/cpg]
c: 8418753 0 unused 0 0 # "raw" part, don't edit
d: 8418753 0 vinum

Next I created the following /etc/vinum.conf file. Notice for the drive sizes I used the values reported by 'df -m':

drive drive1 device /dev/ad0s1e
drive drive2 device /dev/ad1s1d
volume nsm
plex org concat
sd length 739m drive drive1
sd length 3977m drive drive2

After editing /etc/vinum.conf, I ran vinum to create the RAID-0 system:

bourque:/root# create /etc/vinum.conf
2 drives:
D drive1 State: up /dev/ad0s1e A: 28/768 MB (3%)
D drive2 State: up /dev/ad1s1d A: 133/4110 MB (3%)

1 volumes:
V nsm State: up Plexes: 1 Size: 4716 MB

1 plexes:
P nsm.p0 C State: up Subdisks: 2 Size: 4716 MB

2 subdisks:
S nsm.p0.s0 State: up D: drive1 Size: 739 MB
S nsm.p0.s1 State: up D: drive2 Size: 3977 MB

That step created new devices for the vinum drive, as seen next:

bourque:/root# ls /dev/vinum/*
/dev/vinum/control /dev/vinum/controld /dev/vinum/nsm

/dev/vinum/plex:
nsm.p0

/dev/vinum/sd:
nsm.p0.s0 nsm.p0.s1

Now I needed to initialize the filesystem with newfs. The -U option enables soft updates:

bourque:/root# newfs -U /dev/vinum/nsm
/dev/vinum/nsm: 4716.0MB (9658368 sectors) block size 16384, fragment size 2048
using 26 cylinder groups of 183.77MB, 11761 blks, 23552 inodes.
with soft updates
super-block backups (for fsck -b #) at:
160, 376512, 752864, 1129216, 1505568, 1881920, 2258272, 2634624, 3010976,
3387328, 3763680, 4140032, 4516384, 4892736, 5269088, 5645440, 6021792,
6398144, 6774496, 7150848, 7527200, 7903552, 8279904, 8656256, 9032608,
9408960

When done I created a /nsm directory and mounted it:

bourque:/root# mkdir /nsm
bourque:/root# mount /dev/vinum/nsm
bourque:/root# df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/ad0s1a 248M 36M 192M 16% /
devfs 1.0K 1.0K 0B 100% /dev
/dev/ad0s1e 739M 4.0K 680M 0% /nsm1
/dev/ad1s1d 3.9G 4.0K 3.6G 0% /nsm2
/dev/ad0s1g 263M 6.0K 242M 0% /tmp
/dev/ad0s1d 1.9G 303M 1.5G 17% /usr
/dev/ad0s1f 496M 370K 456M 0% /var
/dev/vinum/nsm 4.5G 4.0K 4.1G 0% /nsm

To enable this automatically, I added this line to /etc/fstab:

/dev/vinum/nsm /nsm ufs rw 2 2

In /boot/loader.conf, I added these entries:

vinum_load="YES"
vinum.autostart="YES"

The older method of adding 'start_vinum="YES"' in /etc/rc.conf is still supported, but the new method is preferred.

No comments: