number 9

(I have avoided that cliche all year, give me a break)

I’ve run various releases of Roller on various Glassfish versions, and am pretty happy with what I’ve settled on.

Here are some gotchas it’s taken me a while to Google fixes for.

Glassfish v2

Glassfish v3 is going to be awesome when it’s done, but it’s not a production release.
Get the Sun-branded release – it’s the same codebase, but has a pretty installer and the
option of commercial support down the line if you ever need it.

Roller bowler bowler penny a pitch

Roller 3 is obsolete now and clanky to setup. 4.0 is much much easier to deploy (the install guide even has instructions for Glossfish2), but JavaMail only works on 4.0.1. Dave Johnson (snoopdave, Roller developer) made a pre-built Roller 4.0.1 tarball available a while back.

It’s good, you should use it.

friends don’t let friends use DB2

No-one uses DB2 unless they’re forced to (no-one you’d want to talk to, anyway).
Roller hates it even more than I do. Avoid it unless you enjoy pain.

Better options I’ve tried are:

• PostgreSQL works fine with Roller I love it (but it can be tricky to cluster)
• MySQL is what most Roller guys use. I can’t bring myself to recommend it (we have previous) but I’d expect less Roller bugs talking to it.
• we’re currently on an Oracle 10g backend, it works so far. Someone else supports it, so that’s ok with me.
• Derby/JavaDB worked well enough, but it’s CLI is pretty dreadful (cheers IBM) and you might need to shell into it one day. Good within NetBeans, when you’re too lazy to install a real DB (did I mention the PostgreSQl installer yet)?

fustercluck

Glassfish clusters are easy to setup and actually fun to look after. If you want Glassfish to HA your sessions you should add a <distributable /> element to your web.xml. Roller doesn’t use sessions for much, but it’s not a lot of work, so you might as well.  Other than that, it’s pretty straightforword.
Roller clusters are a different kind of fun.

Out of the box, Roller will do horrible things to you if you try to cluster it.
My favourites (so far)

• search is broken
• caching doesn’t work (well, it caches, it jut does it inconsistently on random nodes)
• scheduled tasks get run twice or not at all, often corrupting your schedule as they do it
• file uploads need a shared filestore.

I got round these with the following roller-custom.properties:

# hook in an external search engine instead
search.enabled=false

# these should be mounted from a reliable fileserver
uploads.dir=/glassfish/uploads
planet.aggregator.cache.dir=/glassfish/planet

# memcache instead of file-cache, scales better anyway
#
# see my last post for details
#
cache.defaultFactory=net.java.roller.tools.cache.memcached.MemcachedLRUCacheFactoryImpl
cache.memcached.default.servers=clusternode1:11211,clusternode2:11211
# it's a bit verbose by default
log4j.category.net.java.roller.tools.cache.memcached=INFO

# give each node a unique clientId - see below
config.expandedProperties=uploads.dir,search.index.dir,tasks.clientId
tasks.clientId=${roller.nodename}

wait, what?

tasks.clientId needs to be unique for each node in your Roller cluster.
Since we run the same WARfile on each node, we make that an alias to a JVM system property, ‘roller.nodename’, that’s different on each cluster node.

In Glassfish, you can do this by creating a cluster-wide JVM option which is set to a system property, and give each server instance a different value for system property.

Assuming we have a cluster called ‘rollercluster’ with 2 server instances ‘instance01’ and ‘instance02’, this should do the trick:

asadmin create-jvm-options --target roller-cluster '-Droller.nodename=${NODENAME}'
Command create-jvm-options executed successfully.
asadmin create-system-properties --target instance01 NODENAME=instance01
Command create-system-properties executed successfully.
asadmin create-system-properties --target instance02 NODENAME=instance02
Command create-system-properties executed successfully.

Restart your server instances, and
‘ps’ should show ‘-Droller.nodename=instance02’ on the server running instance02.

(Thanks to Anil Gangolli and Kedar Mhaswade on the roller and glassfish lists for helping figure this out)

I have a 2-node Glassfishv2 cluster running Roller 4 on Solaris 10
and discovered (the hard way) that memcached is the only safe caching option .

If you’re trying to run Roller on a Glassfish cluster, do yourself a favour and skim over
this checklist. You’ll thank me later, honest.

install memcached

(If you’re running Solaris Express or OpenSolaris, you can skip this bit.)

It’s part of Coolstack now (ignore the page banners, it’s for Solaris Intel as well).
They have a boatload of other things too
(Squid, Ruby, lighttpd and – ooh! – nginx), but I’m trying to stay focussed here.

I pkgadded CSKmemcached_1.3RC1_i386.pkg.bz2 on both nodes.
Bizarrely, the package didn’t include SMF bits, so I tweaked the ones from SXCE.
Help yourself:

curl -O http://files.hellooperator.net/solaris/smf/memcached
cp memcached /lib/svc/method
chmod +x memcached
curl -O http://files.hellooperator.net/solaris/smf/memcached.xml
cp memcached.xml /var/svc/manifest/application/database
svccfg import /var/svc/manifest/application/database/memcached.xml

tweak and enable memcached

I’m giving it 1 Gb of RAM for starters, tweak the “setprop” line below if you want to change it.

bash-3.00# svccfg -s memcached
svc:/application/database/memcached> setprop memcached/options=("-u" "nobody" "-m" "1024")
svc:/application/database/memcached> quit
bash-3.00# svcadm refresh memcached
bash-3.00# svcadm enable memcached

hook up Roller

First, add the roller memcached plugin to your WARfile:

wget https://roller.dev.java.net/files/documents/190/88023/roller-memcached-4.0.tar.gz
gzip -dc roller-memcached-4.0.tar.gz | tar xf -
cp roller-memcached/* $ROLLER_DIR/WEB-INF/lib

Then add this to $ROLLER_DIR/WEB-INF/class/roller-custom.properties

# Rollers caches aren't cluster safe, so use memcache instead
cache.defaultFactory=net.java.roller.tools.cache.memcached.MemcachedLRUCacheFactoryImpl
# any whitespace in this line will waste your afternoon
cache.memcached.default.servers=clusternode1:11211,clusternode2:11211
#debugging
log4j.category.net.java.roller.tools.cache.memcached=DEBUG

The other howtos I’ve seen all specify MemcachedLRUCacheFactory, which gave me ClassNotFoundExceptions (probably because it doesn’t exist?) -
the one above works fine.

Then rebuild your WARfile and redeploy.

Requests/sec has shot up from about 25/sec to over 350/sec, and it can stand a much higher concurrency level now (if you need a quick benchmarking tool, Siege builds well on OSX).
I trashed the webapp and rebuilt it yesterday (swapped out our awful DB2 backend for a proper database) and needed to svcadm restart memcached, but other than that it’s fire-n-forget.

 

(Credits : Dave Johnson is the Roller God, Trond Norbye wrote the SXCE SMF bits)

Update: roller 4 is a lot faster to setup than 3; see here.

 

Now my appserver and database are setup, I can install something.
Right on cue, Roller 3.1 shipped . Looks pretty good too.

WARmonger

I want to deploy from a WAR file, but they don’t ship a prebuilt one
(a GPL thing ?).
So I’ll make my own
(this isn’t necessary – glassfish can deploy from a webapp directory easily enough – it just feels neater to me somehow).

planb:/tmp $ wget http://www.apache.org/dist/roller/roller-3/v3.1.0/bin/apache-roller-3.1.tar.gz
planb:/tmp $ wget https://roller.dev.java.net/files/documents/190/51361/required-jars-roller-3.1.tar.gz
planb:/tmp $ tar zxvf apache-roller-3.1.tar.gz
planb:/tmp $ tar zxvf required-jars-roller-3.1.tar.gz
planb:/tmp $ cd apache-roller-3.1/webapp/roller/

Jarring up apache-roller-3.1/webapp/roller will make a WAR. I need to make a couple of tweaks first.

this is how we (configure) Roll(er)

change the salt

First, edit WEB-INF/security.xml and change the ‘salts’ .

(optional) install textile plugin

Textile support is always nice (even without live preview):

planb:roller $ cd WEB-INF/lib
planb:lib $ wget https://roller.dev.java.net/files/documents/190/56103/textile-plugin-3.1.tar.gz
planb:lib $ tar zxvf textile-plugin-3.1.tar.gz
planb:lib $ mv textileplugin/* .
planb:lib $ rm -r textileplugin textile-plugin-3.1.tar.gz
planb:lib $ cd -

To enable the textile plugin, you can tweak roller-custom.properties
or download mine
(which also tells Hibernate that I’m using a postgresql database) to somewhere in your classpath:

planb:roller $ cd WEB-INF/classes
planb:classes $ wget http://files.hellooperator.net/solaris/glassfish/roller-custom.properties
planb:classes $ cd -

(optional) install extra themes

Add some bundled themes if that’s your thing (you might want to prune some out, there are 20+ themes in the bundle):

planb:roller $ cd themes
planb:themes $ wget https://roller.dev.java.net/files/documents/190/56087/opt-themes-roller-3.1.tar.gz
planb:themes $ tar zxvf opt-themes-roller-3.1.tar.gz ; rm opt-themes-roller-3.1.tar.gz
planb:themes $ cd ..

(not really optional) setup mail support

Roller depends on JavaMail for email notifications of comments, inviting people to become
authors, etc.
The relevant part of WEB-INF/web.xml is inexplicably commented out (apparently Tomcat doesn’t need this).
If you want to send any mail, uncomment the following chunk:

   planb:roller $ tail -15 WEB-INF/web.xml
   ....
       <resource-ref>
           <res-ref-name>mail/Session</res-ref-name>
           <res-type>javax.mail.Session</res-type>
           <res-auth>Container</res-auth>
       </resource-ref>
   ....

make the jar

Create the JARfile and copy it up to the appserver:

   planb:roller $ pwd
   /tmp/apache-roller-3.1/webapp/roller
   planb:roller $ jar cf ~/roller31.war .
   planb:roller $ scp ~/roller31.war  root@goldfish:

the little schema

Roller has db creation scripts for most databases (here’s one I made earlier )

planb $ scp  WEB-INF/dbscripts/postgresql/createdb.sql root@goldfish:
planb $ ssh root@goldfish
goldfish # PATH=/usr/postgres/8.2/bin/:$PATH
goldfish # psql -h elephantom.mydomain -U dbuser zonedb < createdb.sql
....creation output snipped....
goldfish #

configure glassfish

asadmin is great if you don’t like web frontends (or taking screnshots..). You can do all the setup from the CLI.

  planb $ ssh root@goldfish
  goldfish #

install a jdbc driver

PostgreSQL ones here .

   goldfish # cd /domains/rollerdisco/lib
   goldfish # wget http://jdbc.postgresql.org/download/postgresql-8.2-505.jdbc3.jar
   goldfish # svcadm restart rollerdisco

NB: I’ve put the JAR into the domain directory so it won’t be lost on server upgrades

create a connection pool

To create a pool called ‘rollerpool’ (’asadmin help create-jdbc-connection-pool’):

   goldfish # /j2ee/bin/asadmin create-jdbc-connection-pool \
   --user admin --passwordfile /domains/rollerdisco/.aspass -s \
   --datasourceclassname org.postgresql.ds.PGSimpleDataSource --restype javax.sql.DataSource \
   --steadypoolsize 4 --maxpoolsize 12 \
   --property portNumber=5432:password=sekrit:user=dbuser:serverName=elephantom:databaseName=zonedb \
   --description "Roller Connection Pool" rollerpool
  Command create-jdbc-connection-pool executed successfully.
  goldfish #

Using any datasources other than PGSimpleDatasource with my glassfish build (b33) didn’t work. I got a lot of

java.lang.Exception: Doh! Couldn't instantiate a roller class

Bug 2779 has all the gory details. There is a workaround in b46, which we’ll be upgrading to in my next gf post.

create jdbc resource

Out of the box (WEB-INF/sun-web.xml), Roller looks
for a pool called ‘jdbc/rollerdb’. So ‘tag’ our connection pool with that name by creating a JDBC datasource and roller will use it:

   goldfish # /j2ee/bin/asadmin create-jdbc-resource \
   --user admin --passwordfile /domains/rollerdisco/.aspass -s \
   --connectionpoolid rollerpool jdbc/rollerdb

create a javamail session

We need a JavaMail session with the name we specified in Rollers web.xml earlier. This one talks to a local MTA:

   goldfish # /j2ee/bin/asadmin create-javamail-resource  \
   --user admin --passwordfile /domains/rollerdisco/.aspass -s  \
   --mailhost localhost --mailuser required_although_i_dont_use_smtp_auth --fromaddress gfadmin@yourdomain.com mail/Session

deploy it

You can autodeploy the WARfile (by copying ‘yourwebapp.war’ to /domains/rollerdisco/autodeploy),
but that’ll run the webapp at /yourwebapp. Instead, I’ll use asadmin deploy :

   goldfish # /j2ee/bin/asadmin deploy \
   --user admin --passwordfile /domains/rollerdisco/.aspass -s \
   --upload --contextroot '/' /root/roller31.war

(’—upload’ physically copies the warfile into the appserver, rather than deploying from it ‘in situ’)

can you hear me now?

If you autodeployed, browse to :

http://yourserver:8080/roller31/

and register an admin user, create blogs, etc.

I setup glassfish running on port 80 so
I can just go to

http://goldfish/

One gotcha: roller creates directories under roller-data/ in the glassfish users home directory.
Just don’t wonder what it is and delete it (bad things happen, trust me).

UPDATE:

I’m an ex-tomcat user and I’ve seen JVMs bleed memory over time, so I put a resource control of around 600Mb RAM on the zone. It seems to be using about half that, and hasn’t grow in the last few weeks.