number 9

Something like Puppet could potentially make my life a lot easier.
Puppet can’t do baremetal provisioning; it needs the base OS to be Jumpstart/Kickstarted on first. Fortunately:

• Cobbler makes running a kickstart server a piece of piss
• CentOS is a free binary compatible RHEL clone
• VMWare Fusion takes up a lot less space than a test LAN.

PXE dust

PXE boots require DHCP options that VMware doesn’t enable out of the box.
So I either tweak VMwares dhcpd or use Cobblers DHCP support.

Either choice is fine (they both use ISCs DHCPd anyway);
if you don’t want the overhead of running your cobbler VM all the time,
it probably makes sense to tweak VMwares dhcpd.conf.

Cobbler can add static DHCP entries for systems you define,
(and manage DNS too) so it makes life easier for me
(IRL I’ll have to bribe the DHCP guys to add some options).

install a CentOS 5 VM

• choose redhat -> RHEL5 as your VM type
• name it (‘shoemaker’ in my case)
• 10Gb disk (I just need headless boxes), advanced -> split into 2gb chunks
• boot your CentoOS 5 iso . Bog standard install takes 4 minutes on my MBP.

install cobbler

EPEL is a collection of decent RPMs (including cobbler and puppet) for RHEL, Fedora and CentOS.
Tell yum about them and install cobbler :

rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-3.noarch.rpm
yum install -y cobbler

hardcode your IP

(You’ll need to do this if you installed CentOS to use DHCP, which is the easiest way on VMware)

First, find out your IP, gateway and netmask.

ifconfig eth0 | grep 'inet addr'; netstat -rn | grep UG
           inet addr:192.168.21.134  Bcast:192.168.21.255  Mask:255.255.255.0
 0.0.0.0         192.168.21.2    0.0.0.0         UG        0 0          0 eth0

So you need to

echo "GATEWAY=192.169.21.2" >> /etc/sysconfig/network

edit /etc/sysconfig/network-scripts/ifcfg-eth0

# take this out
  BOOTPROTO=dhcp
  # add these
  BOOTPROTO=static
  IPADDR=192.168.21.134
  NETMASK=255.255.255.0

Then /etc/init.d/network restart.

break VMwares DHCP server.

(NB: Other VMs won’t get DHCP responses until cobbler takes over (duh)).

comment out around line 570 of /Library/Application Support/VMware Fusion/boot.sh

569 ######
570 # let shoemaker do this
571 ###   # vmnet-dhcpd puts itself in the background (daemon mode)
572 ###   "$LIBDIR/vmnet-dhcpd" -cf "$LIBDIR/vmnet8/dhcpd.conf" \
573 ###      -lf "$LEASEDIR"/vmnet-dhcpd-vmnet8.leases \
574 ###      -pf /var/run/vmnet-dhcpd-vmnet8.pid vmnet8
575 #####

(Hint: vmnet8 is the NATted subnet, vmnet1 is the host-only one. Edit the right one.)

Then sudo boot.sh –restart.

install cobbler PXE bits

• xinetd serves up kernels and initrds over TFTP.
• Apache serves out kickstart files and RPMs.
• reposync mirrors remote repos (see here )
  yum install -y dhcp reposync
  for i in xinetd cobblerd httpd # we’ll do DHCP later
  do
  chkconfig $i on
  /etc/init.d/$i start
  done

basic cobbler setup

Note: docs refer to ’/var/lib/cobbler/settings’, but my RPM put it at /etc/cobbler/settings

‘cobbler check’ tells you what it needs you to edit. Obey.

Setup Cobblers DHCP management

Put this in settings:

manage_dhcp: 1
next_server: 'ip-of-cobbler-box'
server: 'ip-of-cobbler-box'

Finally, edit /etc/cobbler/dhcp.template
(man dhcpd.conf and/or crib from /Library/Application Support/VMware Fusion/vmnet8/dhcpd.conf) – my effort is here .

cobbler sync builds your DHCP config, /etc/init.d/dhcpd start makes it live.
If dhcpd won’t start, you cocked up the template. Check /var/log/messages, tweak, ‘cobbler sync’, rinse, repeat.

create a kickstart target and boot it

We can use the install ISO to build a distro and boot profile

  mount /dev/cdrom /mnt
  # copies the DVD onto disk
  cobbler import --mirror=/mnt --name=centos5
  cobbler sync

Make a new VM.

• choose ‘Linux -> Red Hat Enterprise Linux 5’ (or VMware cocks up the disk image)
• pick a name and disk size
• untick ‘Start virtual machine and install OS now’

Change any VM settings if you like (defaults are ok).
Power up the VM. You should see it:

• get an IP (DHCPd)
• download a kernel and initrd over TFTP (xinetd and TFTPd)
• present a menu (pxelinux.0),

Choose ‘centos5’ and you’re away.

further reading

I’ll do a followup article soon with a few tricks/gotchas I’ve found so far. In the meantime:

• Cobbler docs (or ‘man cobbler’) are pretty good (I might even read them sometime).
• Bootstrapping puppet with Cobbler is my next todo.
• there’s a decent wiki too

We were talking about BrandZ
(linux-flavoured Solaris zones) when I realised I hadn’t got round to trying it out yet.

So I did.
It seems to mostly work like a linux box, except
apps that need direct access to the kernel/proc/devices etc can have problems.

As usual, I’m just putting up my notes. If you need more general (or better written) docs,
I recommend

• the official howto
• Bens writeup for opensolaris
• ’man -k zone’
  

  build the container

First thing to do is add the linux brand packages (so ‘create -t’ works).

pkgadd -d . SUNWlxu SUNWlxr

Configure the zone like any other, but pass a flag to ‘create’:

vera # zonecfg -z lux
lux: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:lux> create -t SUNWlx
zonecfg:lux> set zonepath=/zones/lux
zonecfg:lux> add net
zonecfg:lux:net> set address=10.9.8.7/24
zonecfg:lux:net> set physical=iprb0
zonecfg:lux:net> end
zonecfg:lux> commit
zonecfg:lux> exit

install (and tweak) the linux distro

This isn’t a Solaris zone so you need to install linux into the zonepath.
Sun have a CentOS 3.x
image on their website containing X, gnome, etc.

Bizarrely, Suns tar can’t unpack it ,
so you need to install gtar before zoneadm can use it.
This is still less hassle than burning ISOs in my book.

vera # wget http://dlc.sun.com/osol/brandz/downloads/centos_fs_image.tar.bz2
vera # mount /cdrom && pkgadd -d /cdrom/Solaris_11/Product/ SUNWgtar

Now you install linux into the zone (note /zones is on ZFS here, so we get compression/snapshots/cloning for free). This does, indeed, take several minutes:

   vera # zoneadm -z lux install -d centos_fs_image.tar.bz2
   A ZFS file system has been created for this zone.
   Installing zone 'lux' at root directory '/zones/lux'
   from archive '/zones/centos_fs_image.tar.bz2'

   This process may take several minutes.

   Setting up the initial lx brand environment.
   System configuration modifications complete.

   Installation of zone 'lux' completed successfully.

   Details saved to log file:
       "/zones/lux/root/var/log/lux.install.9440.log" 

   vera # zoneadm list -iv
     ID NAME             STATUS         PATH                           BRAND
      0 global           running        /                              native
      - lux              installed      /zones/lux                     lx        

Setting up the network is easy:

vera # cat > /zones/lux/root/etc/sysconfig/network
NETWORKING="yes"
HOSTNAME=lux.whatever.com
^D
vera # cat > /zones/lux/root/etc/resolv.conf
search whatever.com
nameserver 1.2.3.4
nameserver 1.2.3.5
^D

Since I’m a big fan of RSA authentication, I set that up too:

vera # echo 'PermitRootLogin without-password' >> /zones/lux/etc/ssh/sshd_config
vera # mkdir -p /zones/lux/root/root/.ssh/
vera # chmod 700 /zones/lux/root/root/.ssh/
me@mydesktop $ scp ~/.ssh/id_dsa.pub root@vera:/zones/lux/root/root/.ssh/authorized_keys

I also edit /zones/lux/root/etc/shadow and set roots password field to ‘NP’, since neither zlogin or ssh need it to be set.

boot it

vera # zoneadm -z lux boot

It takes a minute or two for rc to finish running,
then you can ‘zlogin lux’ or ‘ssh root@lux.whatever.com’ into it.

constrainers for your containers

Standard resource controls work as you’d expect. Here I’m setting a maximum numbers of processes (LWPs):

vera # zonecfg -z lux
zonecfg:lux> add rctl
zonecfg:lux:rctl> set max-lwps=60
zonecfg:lux> exit
vera # prctl -s -n zone.max-lwps -v 60 -t priv -e deny -i zone lux

(the ‘prctl’ line avoids us having to reboot the zone)

‘prstat -LZ’ in the global zone shows LWPs per zones nicely. As expected, starting a load of processes
in ‘lux’ gives -bash: fork: Resource temporarily unavailable errors at 60 LWPS.

decide it’s not really your cup of tea (optional)

Personally, I’ll stick to plain Solaris zones.
I’m over the worst of the Solaris learning curve.
I’m looking at Solaris for servers and most of the apps I want to run are open source.
It’s a lot less hassle to build from source (or use blastwave) than to maintain Yet Another
OS Instance just to run the same apps against glibc.

Zones gives you multiple environments without the hassle of multiple OSes to maintain, which most
virtualization solutions seem to overlook.
In addition, Solaris resource management is very fine grained inside a zone (per-project limits can co-exist with per-zone limits). I can’t say
the same for other OSes I’ve tried. This helps to minimize the number of zones you need.

Of course, YMMV. If you have something
linux-only (benr mentions acroread)
that you need this could be really handy. Similarly it might be a lot easier to keep your developers
happy if they can run their apps on linux.

If you want to make the zone start at boot, you’ll need to

vera # zonecfg -z lux
zonecfg:lux> set autoboot=true
zonecfg:lux> commit
zonecfg:lux> exit