Solaris 10update 4 is out, and so is glassfish v2. First we need to get our
OS on.
My test x86 machine is a 3Ghz P4 with 1Gb RAM and twin 40Gb disks.
Disks are a bit pokey, but having 2 makes playing around with RAID and ZFS more fun.
Since ZFS root isn’t here yet, I’ll use Solaris Volume Manager (SVM) to mirror the root
filesystem. Applications, /export/home , etc. will live on a ZFS mirrored pool.
(NB: the procedure to install Solaris Express is almost identical, except you can skip the PCA step)
sunshine in a bag
I got the Solaris 10 Update 4 DVD ISO and burnt it off.
The install is straightforward, with a couple of caveats:
- SVM can only mirror slices on solaris fdisk partitions, so make 1 big solaris primary partition.
- only install onto the first disk (c1d0) – we’ll add the second one later.
- choose ‘custom install’ to choose your disk layout
| slice | file system | size | notes |
|---|---|---|---|
| 0 | / | 6000Mb | |
| 1 | swap | 1100Mb | (must be bigger than RAM to save crashdumps) |
| 3 | /metadb | 10Mb | (this is just to reserve the space for SVM bookkeeping) |
| 7 | /zfs | 32035Mb | (the rest of the disk will be a ZFS storage pool) |
Note I haven’t set up a slice for
Live Upgrade
. I’ll detach one submirror before an upgrade, then I can rollback or keep the upgrade by choosing which way to resync them afterwards.
I chose ‘Entire Distribution’, then went off to find a sandwich and play a bit of Hotel Dusk.
After the reboot, you can login as root,
unmount /metadb (c1d0s3) and /zfs (c1d0s7) , remove them from /etc/vfstab,
and delete the mountpoints (you could just set them up later, but the installer is a bit eaiser to explain than the ‘format’ command).
slice up the second disk
We’ll set the second disk to have 1 Solaris fdisk partition.
Pipe the disklabel from c1d0 onto c2d0 so the slice sizes on both are identical:
fdisk -B /dev/rdsk/c2d0p0 prtvtoc /dev/rdsk/c1d0s2 | fmthard -s - /dev/rdsk/c2d0s2
We also need to install grub, so it’s bootable if the first disk dies:
/sbin/installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c2d0s0
stage1 written to partition 0 sector 0 (abs 16065)
stage2 written to partition 0, 260 sectors starting at 50 (abs 16115)Add an entry for c2d0 in /boot/grub/menu.lst:
# second half of SVM root mirror title alternate root root (hd1,0,a) kernel /platform/i86pc/multiboot module /platform/i86pc/boot_archive title alternate root failsafe root (hd1,0,a) kernel /boot/multiboot kernel/unix -s module /boot/x86.miniroot-safe
setting up the state databases
SVM stores its config on-disk, in
state database replicas .
You need half of them to be online at any given time, which means I
need 2 copies on each disk (each is about 4Mb, hence the 10Mb /metadb slice I set aside):
metadb -a -f -c 2 c1d0s3 c2d0s3which says :
- add some state database replicas (-a)
- it’s ok that there aren’t any existing replicas (-f)
- there’ll be 2 database replicas on each device (-c 2)
- and use the slices we set aside earlier (c1d0s3 c2d0s3)
Check they got created OK:
metadb
flags first blk block count
a u 16 8192 /dev/dsk/c1d0s3
a u 8208 8192 /dev/dsk/c1d0s3
a u 16 8192 /dev/dsk/c2d0s3
a u 8208 8192 /dev/dsk/c2d0s3The ‘u’ flag means the replica is up to date (’metadb -i’ gives a legend).
setting up the root RAID-1 mirror
I’ll use my existing root fs as one submirror, then hook up the second disk.
First we tell SVM about the (existing) root slice:
metainit -f d1 1 1 c1d0s0
d1: Concat/Stripe is setupwhich says:
- make a volume called d1 (d1)
- with one stripe (1)
- with one component per stripe (1)
- out of my existing root slice (c1d0s0)
- oh, and yes, I know it contains a filesystem (-f)
We do the same thing for the second disks root slice (this is empty, so we don’t need ’-f’):
metainit d2 1 1 c2d0s0
d2: Concat/Stripe is setupNow we create a mirror volume made up of the populated submirror, d1:
metainit d0 -m d1
d0: mirror is setupwhich says:
- make a volume called d0 (d0)
- which is a mirror made up of volume d1 ( -m d1)
I’.ll start using this volume as the root fs
before I attach the other submirror (if you’re going to fail, fail early).
The ‘metaroot’ command edits /etc/vfstab and /etc/system for you:
metaroot d0
rebootAnd when it comes back up, we’re running on the logical device:
df -h /
Filesystem size used avail capacity Mounted on
/dev/md/dsk/d0 5.8G 3.1G 2.6G 56% /Last thing to do is attach the other half of the mirror:
metattach d0 d2You can watch the mirror syncing up:
metastat -c
d0 m 5.9GB d1 d2 (resync-15%)
d1 s 5.9GB c1d0s0
d2 s 5.9GB c2d0s0Takes about 5 minutes, and that’s pretty much it.
multi-mirror swap shop
Up to you whether to do this – you can use the second swap device for more VM,
but mirroring should help if a disk dies while you’re running.
The process is very similar to the root slice:
metainit -f d51 1 1 c1d0s1
metainit d52 1 1 c2d0s1
metainit d50 -m d51
metattach d50 d52
swap -d /dev/dsk/c1d0s1
swap -a /dev/md/dsk/d50Update /etc/vfstab to use /dev/md/dsk/d50 instead of /dev/dsk/c0d0s1
Setup the ZFS mirror
I want a ZFS mirror for home directories, apps, etc.
It’s not that I don’t trust SVM (although I don’t know it yet),
but it’s just a volume manager – you still have all the hassles of filesystems
on top of it, and if I wanted that I’d still be on Linux LVM.
zpool create tank mirror c1d0s7 c2d0s7
zpool status
pool: tank
state: ONLINE
scrub: none requested
config:NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
mirror ONLINE 0 0 0
c1d0s7 ONLINE 0 0 0
c2d0s7 ONLINE 0 0 0errors: No known data errorsAnd that’s it.
Well, actually, no.
The next thing to do is pull out disks and check you can still boot
the machine. But this is getting a bit long-winded now, so that’ll be another
post.
no, honestly, you can stop reading now
My post-install checklist includes:
- hooking the machine up for outbound email
echo DSsmarthost.whatever.com >> /etc/mail/sendmail.cf echo 'root: me@whatever.com >> /etc/mail/newaliases svcadm restart sendmail newaliases
- hardcode duplex settings
- hook up pca
- setup a firewall
- setup NTP
echo 'server time.apple.com' > /etc/inet/ntp.conf ntpdate -b time.apple.com svcadm enable ntp
- create a user
zfs create -o mountpoint=/export/home tank/home zfs create tank/home/dick useradd -c 'Dick Davies' -d /export/home/dick -s /usr/bin/zsh dick projadd -c 'Dick Davies' user.dick chown -R dick /export/home/dick passwd dick
- switch on the (zone-friendly) Fair Share Scheduler
dispadmin -d FSS reboot
It would be nice to Jumpstart this, and once we get a decent PXE solution that’ll be exactly what I’ll do. This will help no end.