Ubuntu is a good Linux, but reiserfs let me down this autumn (nothing personal, the other linux filesystems suck too), and I had
a particularly stupid installer destroy ~/.mozilla last month.

I badly wanted a ZFS home directory (not badly enough to go near ZFS-FUSE; I’m not deranged :D ).
Nothing fancy, just a mirrored zpool for redundancy and regular snapshots to protect against pilot error.

(update: FreeBSD 7.0 now has excellent ZFS support which might be helpful if you don’t mind leaving Linux)

To avoid having to migrate fully, I moved my home directory to an NFS share on my Solaris machine (iSCSI wouldn’t help; I’d have to mkfs.crappyfs the LUN on the Linux end anyway. Besides, all the Linux iSCSI initiators I tried stank).

It works a treat.

planb$ pwd
/home/sisred
planb$ md5sum misc/oncall.txt
68db8407afbe4965918e3b425e2d5abd  misc/oncall.txt
planb$ rm misc/oncall.txt
planb$ echo 'shit'
shit
planb$ date
Sun Jan  7 21:16:12 GMT 2007
planb$ ls .zfs/snapshot/
[snip - lots of directories]
zfs-auto-snap-2007-01-05-06:00:00/
zfs-auto-snap-2007-01-05-12:00:00/
zfs-auto-snap-2007-01-05-18:00:00/
zfs-auto-snap-2007-01-06-00:00:00/
zfs-auto-snap-2007-01-06-06:00:00/
zfs-auto-snap-2007-01-06-12:00:00/
zfs-auto-snap-2007-01-06-18:00:00/
zfs-auto-snap-2007-01-07-00:00:00/
[snip - lots of directories]
planb$ cp -p .zfs/snapshot/zfs-auto-snap-2007-01-07-18\:00\:00/misc/oncall.txt misc/oncall.txt
planb: $ md5sum misc/oncall.txt
68db8407afbe4965918e3b425e2d5abd  misc/oncall.txt
planb$ uname -rs
Linux 2.6.17-10-386
planb$

Course, this works just as well with any OS that can mount NFS.
The only difference to running directly on Solaris is speed (due to 100Mbit link) and I have to ssh
over to run zfs(1).

Here are my notes (again, they’re mostly a paste of my notes, so ask if anything is unclear).

setup a dedicated link

This is the only remotely tricky bit, and it’s optional.
Both machines are on my desk, so I hooked them up with a crossover cable to keep NFS off the LAN.

I’m using NICs I found in a bin. Once they’re fitted:

both sides

I’ll choose a network of 10.10.10.0/30 for the link.
Each end gets a hostname of ‘red$host’ (the cable is red. Imagination bypass.).

Add this to /etc/hosts on each end:

10.10.10.1  redsun
10.10.10.2  redlinux

and then disable firewalls on those interfaces (eth2 on Linux and e1000g0 on Solaris in my case).

the linux end

add an entry to /etc/network/interfaces

auto eth2
iface eth2 inet static
  address 10.10.10.2
  netmask 255.255.255.252

then bring up the interface with sudo ifup eth2

By default, Linux tosses a coin to choose the interface number at boot, so now and then eth0 and eth2 will
swap.
I’m sure this is useful to some people, but if you prefer a sane networking setup , you can tie them down in /etc/iftab:

eth0 mac 00:13:20:b2:22:23
eth2 mac 00:90:27:12:4f:11

the solaris end

Before you can use a NIC, you need to plumb it in (this hooks up all the kernel processing streams to it, hence the name).

ifconfig
Bring up the interface with:

sun # ifconfig e1000g0 plumb
sun # ifconfig e1000g0 inet 10.10.10.1/30 up

For this to be done automatically at boot:

sun # echo redsun > /etc/hostname.e1000g0
sun # echo "10.10.10.0    255.255.255.252" >> /etc/inet/netmasks

You should be able to ping across the link now.

create a home filesystem

sun # zfs create tank/home
sun # zfs set compression=on tank/home
sun # zfs create tank/home/sisred
sun # zfs set mountpoint=/export/home/sisred tank/home/sisred

create a user

I gave up trying to persuade linux to do NFSv4, so I’m using v3.
That means the uids on each end need to match.

Since I’m moving an existing homedir, I create a Solaris user with matching UID/GID.
(run ‘id’ on the linux host – in my case, ‘1000’ for both).

sun # groupadd -g 1000 sisred
sun # useradd -c "Dick Davies" -g 1000 -u 1000 \
-d /export/home/sisred -m \
-s /usr/bin/bash sisred
sun # chown -R sisred:sisred ~sisred

No need for a password (I use via key-based SSH).

share it out

sun # zfs set sharenfs=on tank/home/sisred
sun # svcadm enable nfs/server
sun # svcs -xv
sun #

The second command will make sure NFS starts at reboot. ‘svcs -xv’ will give useful output if something didn’t work.

Shares are read/write by default. The ‘sharenfs’ property can also take a string of options to share(1M) –
so zfs set sharenfs=ro tank/home/sisred creates a readonly share, for example.

move into your new $HOME

linux # sudo mkdir /nfshome
linux # sudo chown sisred:sisred /nfshome
linux # sudo apt-get install portmap # for NFS locking
linux # sudo mount redsun:/export/home/sisred /nfshome
linux # rsync -va /home/sisred/ /nfshome/

(You need to run the last command as the user due to NFS ‘root squash’).

If it looks OK, you can make it permanent by editing /etc/fstab and remounting:

redsun:/export/home/sisred /home/sisred  nfs proto=tcp        0       2

now you’re just showing off

If performance is a bit slow on small files, take a look here for tuning.

You need something to automate snapshots up at the Solaris end. Two options are:

  1. Tim Fosters SMF snapshots
    - some good features (automated ’ zfs -i send/recv’ of snapshots to a remote host, cleanup of old snapshots etc).
  2. roll your own

UPDATE: I’ve since switched to NFSv4, which was easy when you know how. See here