I have a 2-node Glassfishv2 cluster running Roller 4 on Solaris 10
and discovered (the hard way) that memcached is the only safe caching option .

If you’re trying to run Roller on a Glassfish cluster, do yourself a favour and skim over
this checklist. You’ll thank me later, honest.

install memcached

(If you’re running Solaris Express or OpenSolaris, you can skip this bit.)

It’s part of Coolstack now (ignore the page banners, it’s for Solaris Intel as well).
They have a boatload of other things too
(Squid, Ruby, lighttpd and – ooh! – nginx), but I’m trying to stay focussed here.

I pkgadded CSKmemcached_1.3RC1_i386.pkg.bz2 on both nodes.
Bizarrely, the package didn’t include SMF bits, so I tweaked the ones from SXCE.
Help yourself:

curl -O http://files.hellooperator.net/solaris/smf/memcached
cp memcached /lib/svc/method
chmod +x memcached
curl -O http://files.hellooperator.net/solaris/smf/memcached.xml
cp memcached.xml /var/svc/manifest/application/database
svccfg import /var/svc/manifest/application/database/memcached.xml

tweak and enable memcached

I’m giving it 1 Gb of RAM for starters, tweak the “setprop” line below if you want to change it.

bash-3.00# svccfg -s memcached
svc:/application/database/memcached> setprop memcached/options=("-u" "nobody" "-m" "1024")
svc:/application/database/memcached> quit
bash-3.00# svcadm refresh memcached
bash-3.00# svcadm enable memcached

hook up Roller

First, add the roller memcached plugin to your WARfile:

wget https://roller.dev.java.net/files/documents/190/88023/roller-memcached-4.0.tar.gz
gzip -dc roller-memcached-4.0.tar.gz | tar xf -
cp roller-memcached/* $ROLLER_DIR/WEB-INF/lib

Then add this to $ROLLER_DIR/WEB-INF/class/roller-custom.properties

# Rollers caches aren't cluster safe, so use memcache instead
cache.defaultFactory=net.java.roller.tools.cache.memcached.MemcachedLRUCacheFactoryImpl
# any whitespace in this line will waste your afternoon
cache.memcached.default.servers=clusternode1:11211,clusternode2:11211
#debugging
log4j.category.net.java.roller.tools.cache.memcached=DEBUG

The other howtos I’ve seen all specify MemcachedLRUCacheFactory, which gave me ClassNotFoundExceptions (probably because it doesn’t exist?) -
the one above works fine.

Then rebuild your WARfile and redeploy.

Requests/sec has shot up from about 25/sec to over 350/sec, and it can stand a much higher concurrency level now (if you need a quick benchmarking tool, Siege builds well on OSX).
I trashed the webapp and rebuilt it yesterday (swapped out our awful DB2 backend for a proper database) and needed to svcadm restart memcached, but other than that it’s fire-n-forget.

(Credits : Dave Johnson is the Roller God, Trond Norbye wrote the SXCE SMF bits)

Finally!

ZFS root has been in Indiana (aka OpenSolaris 2008.05) for a while, but I prefer Solaris Express.
As of build 90, it’s supported by the installer.

I installed it on my crappy P4 test box : 1Gb Ram, twin 40Gb disks. Burn the DVD ISO and boot it if you want to play along.

the secret handshake

Choose ‘Solaris Express’ (not ‘Solaris Express Developer Edition), then ‘3 . Solaris Interactive Text (Desktop Session)‘

Only the ‘Interactive Text’ options have the ZFS root option.
Running ‘Desktop’ not ‘Console’ session lets you start a Terminal
to enable compression on the pool when its created (40Gb disks, remember?).

Enabling ZFS compression won’t convert blocks that have already been written (good explanation here), so you want to do it before you populate the filesystem.

you know the drill

• Choose ‘English’
• [X starts up]
• rightclick the desktop and choose ‘programs -> terminal’
• system id
  • networked: yes
  • DHCP: yes
  • IPv6: no
  • [it'll do a DHCP request]
  • Kerberos: no
  • Name service : None (no need if you’re on DHCP)
  • NFS domain : Use NFSv4 domain derived by the system
  • Time Zone : Europe -> Britain (UK)
  • root password : ‘secret’ (heh)
• F2. Standard install
• Manually eject DVD
• Manually reboot
• Choose Media : CD/DVD
• Accept license
• Geographic regions – leave all blank
• System Locale : POSIX C (C)
• Web Start : None
• Choose Filesystem Type : ZFS
• Select Software : Entire Distribution
• choose both disks (this makes a ZFS mirrored pool)
• select ‘put /var on a separate dataset’ (personal choice, but stops / filling up)
• Skip ‘Mount Remote File Systems’

fingers on buzzers

In the Terminal you opened, create a script called ‘readysetgo.sh’

#! /bin/sh
until [ "`zpool list rpool`" ];
do
:
done
zfs set compression=on rpool
until [ "`zfs list rpool/ROOT`" ];
do
:
done
zfs set compression=on rpool/ROOT

Then just run

sh readysetgo.sh

in the Terminal you opened earlier.
You can now start the install. Once the pool is created, it’ll have compression
enabled automatically.

scrooged

Let’s see how much benefit we got. The ‘Entire Distribution’ took about
5Gb of disk without compression, looks to be about 3 Gb with..

vera:~ $ zfs get -r compressratio rpool
NAME                    PROPERTY       VALUE                   SOURCE
rpool                   compressratio  1.62x                   -
rpool/ROOT              compressratio  1.74x                   -
rpool/ROOT/snv_90       compressratio  1.74x                   -
rpool/ROOT/snv_90/var   compressratio  2.60x                   -
rpool/dump              compressratio  1.00x                   -
rpool/swap              compressratio  1.37x                   -
vera:~ $

(UPDATED - thanks to Glenns blog for the neater script, and Andrew in the comments for tidying it up further)

I did Roller 3 on Glassfish 2 a while back
so thought that’d be the simplest thing to put on Glassfish3. The process has got quite a bit easier.

get roller

curl -O http://www.mirrorservice.org/sites/ftp.apache.org/roller/roller-4/v4.0.0/bin/apache-roller-4.0.zip
unzip apache-roller-4.0.zip

If you want to be able to send mail you’ll need Roller 4.0.1 (a bug in 4.0 breaks JavaMail on Glassfish).

setup an empty database

Roller4 isn’t perfect, but it kicks ass when it comes to auto-generating its
own database tables. It still needs the actual database to exist, though:

asadmin create-jdbc-connection-pool \
--datasourceclassname org.apache.derby.jdbc.EmbeddedDataSource \
--property databaseName=\$\{com.sun.aas.instanceRoot\}/databases/rollerdb:\
connectionAttributes=\;create\\=true rollerpool
asadmin ping-connection-pool rollerpool
asadmin create-jdbc-resource --connectionpoolid=rollerpool jdbc/rollerdb

(those of you playing along at home on an embedded database should look at upping your PermGen space at this point).

tweak roller

Roller 4 uses JNDI to find its DB by default, so there isn’t much to tweak:

cd ~/apache-roller-4.0/webapp/roller/WEB-INF/classes
curl -O http://files.hellooperator.net/glassfish/webapps/roller-custom.properties

You’ll want to change the mailserver to one you can use.

The full bewildering list of possible roller properties is in the Install Guide

You should make these changes to security.xml, too (unless you’re on Debian in which case, why bother?hee hee).

make and deploy a WAR

cd ~/apache-roller-4.0/webapp/roller
jar cvf ~/roller.war *
asadmin deploy ~/roller.war

Browse to http://localhost:8080/roller and try it out.

The database is created when you first connect, and the first user you make is the site admin.

Since writing this, I’ve realised JavaDB is pretty crap. Do yourself a favour and try the decent PostgreSQL installer I found.

Glassfish v3 ships with JavaDB (aka Apache Derby aka Cloudscape).
I’ll be using this for trying out Rails and JRuby, but it’s also handy
for things like authentication via JDBC Realms.

A JavaDB database is essentially a directory that only one process can access at a time (a bit like sqlite). This can be Glassfish itself (an embedded database) or a standalone database process (that serves SQL clients over TCP/IP).

Both have pros and cons. I’ll take you through creating both.

option 1: standalone database server

You sure you want JavaDB? There are better options (sorry, can’t help it).

The main benefit to running a network server process is that it’s the only way for multiple clients to access the database simultaneously

(it’s also the only option that makes sense if you were clustering Glassfish).

If you need to create a schema before you deploy a webapp (with NetBeans or ‘rake migrate’)
you’ll have to stop Glassfish first unless you go down this route.

hypnotoad:databases $ asadmin start-database --dbhost 127.0.0.1
Database started in Network Server mode on host 127.0.0.1 and port 1527.
Could not connect to Derby Network Server on host 127.0.0.1 port 1527.
Starting database in the background.
Log redirected to /Users/dick/Applications/glassfishv3-tp2/glassfish/databases/derby.log.
Command start-database executed successfully.

( –dbhost defaults to ‘0.0.0.0’ but this causes problems if you change IP . Stick to 127.0.0.1).

Next, create the connection pool (and associated database - see later).
It’s simplest to do this on the command line (partly due to bug 4889 ):

hypnotoad:databases $ asadmin create-jdbc-connection-pool \
--datasourceclassname=org.apache.derby.jdbc.ClientConnectionPoolDataSource\
--isconnectvalidatereq=true --validationmethod=meta-data \
--property user=GFv3:password=GFv3:databaseName=railsdb:\
connectionAttributes=\;create\\=true \
railspool
Command create-jdbc-connection-pool executed successfully.

• The ’;create=true’ option tells JavaDB to create the ‘railsdb’ database on demand
• host:port defaults to localhost:1527
• username and password can be anything, but are required

We now ‘ping’ the pool. This checks our network connection is good, and has the side-effect
of creating the ‘railsdb’ database:

hypnotoad:databases $ asadmin ping-connection-pool railspool
Command ping-connection-pool executed successfully.
hypnotoad:databases $ ls
derby.log railsdb

option 2. embed Derby in Glassfish

This is my preferred option for several reasons:

1. it saves having to run 2 JVMs
2. in development, I don’t mind stopping Glassfish
3. for production, I want webapps to create their own schema anyway
4. connection validation and authentication is no longer an issue
5. sorry to bang on about it, but if you want a standalone database there are much better options

One side effect is that Glassfish is going to use more memory (especially if you have several connection pools configured). You might want to tweak your JVM .

There’s no need to ‘start-database’ in this case – just go ahead and make the pool:

hypnotoad:glassfishv3-tp2 $ asadmin create-jdbc-connection-pool \
--datasourceclassname org.apache.derby.jdbc.EmbeddedDataSource \
--property databaseName=\$\{com.sun.aas.instanceRoot\}/databases/railsdb:\
connectionAttributes=\;create\\=true \
railspool
Command create-jdbc-connection-pool executed successfully.

• the different DataSource class is what makes it embedded
• provide a full path in the databaseName attribute to avoid current working directory hell
• since Glassfish is the database server, we can skip username,password and connection validation options

If we ping the pool, we can see Glassfish creates derby.log and the database dir

hypnotoad:glassfishv3-tp2 $ asadmin ping-connection-pool railspool
Command ping-connection-pool executed successfully.
hypnotoad:glassfishv3-tp2 $ tail derby.log
2008-05-15 11:12:48.770 GMT:
Booting Derby version The Apache Software Foundation – Apache Derby – 10.2.2.1 – (538595): instance c013800d-0119-ec48-424c-000001a39158
on database directory /Users/dick/Applications/glassfishv3-tp2/glassfish/domains/domain1/databases/railsdb
Database Class Loader started – derby.database.classpath=’‘
hypnotoad:glassfishv3-tp2 $

Butler Lampsons mamma didn’t raise no fools

Whichever option you choose, the command to give the pool a JNDI name is the same:

hypnotoad:glassfishv3-tp2 $ asadmin create-jdbc-resource --connectionpoolid=railspool jdbc/railspool
Command create-jdbc-resource executed successfully.

And we’re done. Now simply write a webapp to use the damn thing.

I enabled FileVault on my MBP and cancelled the conversion when I saw how long it was going to take. When I tried to re-enable it again today, I got an error saying

could not create filevault disk image

Google wasn’t very helpful, so I’m noting down the fix here, which is:

sudo rm -r /Users/Shared/username.sparsebundle

 Turns out FileVault doesn’t use encrypted .dmgs(because they’re fixed size) but instead has a directory called username.sparsebundle (.sparseimage, pre-Leopard) that does the same job.This is initially created as /Users/shared/username.sparsebundle ,populated and only moved into place once the old home directory is deleted.