Transparent Application Deployment with Tomcat and JK

If you've ever wanted to get some kind of software load balancing configuration working with Tomcat using the JK Connector, then this article may be for you. More importantly if you've ever wondered how to make production builds transparent to users in J2E development with Tomcat and you don't have a hardware load-balancing budget available, this may also be for you. I work with 1 production server in my work place that delivers the corporate intranet via 1 web application. Being an intranet, there are often several to many patches per week of small to large magnitude. Unfortuantely we cannot patch single classes like we can JSPs due to the JVM (big shame), so we need to come up with another way to deploy web applications that do not pull down the application from users for too long. Now, I don't really like to pull the intranet down at all to deploy changes as we have international customers. Also because it's inconvenient to me as a developer, it may be there's an urgent fix that I want done silently, or general politics about availability. The issue is more pressing for internet sites too where as close to 100% availability is desirable. With load balancing, the general strategy is to provide a single point of access for inbound requests to the web application and then based on some rules, e.g round-robin or weights, send the request to a particular server to handle the request. Load balancing is usually used for performance and high-availability strategies. You will often find load balancing is done by a piece of hardware that sits in front of 2 logically separate servers. If you can afford it, do it. If you're limited in budget or hardware, then what I am about to talk about will provide a strategy for permanent availability with 1 Tomcat server. It does have caveats with respect to sessions however (unless you setup session clustering).

Install another Tomcat

I already had 1 Tomcat installed and running the web application. I simply copied the apache-tomcat folder to apache-tomcat-2, complete with web application. I don't use the Windows installer, I use the ZIP distribution as it offers more flexibility for my taste. I will refer to each Tomcat as TC1 and TC2 TC2's apache-tomcat/conf/server.xml file needed some changes for ports so as to prevent bind exceptions. I changed the main Server element port to 8006 (TC1 is 8005), the HTTP Connector to 8081 (TC1 is 8080) and the AJP port to 8109 (TC1 has 8009). Note to use JK you need to use the AJP Connector. Both server.xmls for TC1 and TC2 also need a modification to the Engine element. A new attribute is required (thanks to the Tomcat Users list for this information). You need to add the jvmRoute attribute with a name that matches the worker name for the Tomcat in question. The worker name is that which you call the worker in the workers.properties configuration (discussed later so just ensure you come back to this attribute). That is all that is needed in terms of configuration difference between TC1 and TC2. Obviously, I modified my build process so that when I build new code, both Tomcats get the new code.

JK

This is the part that really matters and for which I had trouble finding examples on the web. I am assuming you have configured and have working a normal JK setup? You do right? So, here is my workers.properties file

worker.tomcat1.port=8009
worker.tomcat1.host=localhost
worker.tomcat1.type=ajp13
worker.tomcat1.lbfactor=1

worker.tomcat2.port=8109
worker.tomcat2.host=localhost
worker.tomcat2.type=ajp13
worker.tomcat2.lbfactor=1

worker.loadbalancer.type=lb
worker.loadbalancer.balance_workers=tomcat1,tomcat2
worker.loadbalancer.sticky_session=1

Both Tomcats are defined as tomcat1 and tomcat2. Note that these are the worker names I referred to earlier when talking about the jvmRoute attribute of Engine. Make sure that TC1's Engine's jvmRoute="tomcat1" and TC2's Engine's jvmRoute="tomcat2". Also note that you must not use these workers in the workers.list setting. Instead, you create a worker called loadbalancer (a special type of worker indicated to JK by setting it's type to lb) and add the tomcat1 and tomcat2 workers to this using the balance_workers property, and then the workers.list should be just the loadbalancer worker. Also for the loadbalancer, I have explicitly set the sticky_session property to 1 (which is default anyway). This means that JK will examine a request for a JSESSIONID and try and match it to a known active session on a Tomcat worker. If it's found and the Tomcat is available, the request will be forwarded to that Tomcat. Don't forget the URI mappings! Don't forget to modify your URI mappings to map to the loadbalancer worker now instead of a named worker.

/*.do=loadbalancer
/jsp-examples/=loadbalancer
/servlet-examples/=loadbalancer

Restart the web server (in my case IIS) so the JK settings take effect. Also bring up both Tomcats and verify they are happy to co-exist.

Testing taking a Tomcat down and bringing one up

To make myself happy that load balancing worked, I tested by killing off one of the Tomcats whilst observing the JK logging. JK gracefully noted on the request that the Tomcat it had hoped was available was in actual fact not, and so passed it to the other Tomcat worker instead. Great stuff. Second test was to start up with only 1 Tomcat even though the config declares 2 Tomcats. This caused no issues, the available Tomcat was used. I then brought up the 2nd Tomcat and made some new requests to find that as expected JK started to use it. Where I tested sequential requests using the same browser, sessions were maintained to the same Tomcat.

Load balancing to make production builds transparent

With this new setup I hope to be able to make class patching and other builds easier. I shall run both Tomcats side by side. When I need to patch I can bring TC1 down, do the build and then bring it up again, before doing the same with TC2. If your server is not up to 2 Tomcats running on the same server, you can still achieve the same. In that case, you'd run TC1 all the time and not TC2. When you are ready to build or patch, ensure TC2 is the same state as TC1 and bring it up whilst TC1 is running. As soon as it's up, bring TC1 down for the build. There's a slight overlap admittedly which is necessary for the availability. Because JK by default uses "sticky sessions", e.g once a user has made requests and JK has designated a functioning Tomcat, the user stays with that server if their session ID is the same. Therefore there is little risk during the build of worrying that the user will see crossover functionality. The other thing to be aware about is that this method will kill your user sessions which may or may not be a bad thing depending on your application. To get around that you would need to look into session clustering between TC1 and TC2. If your web app does not rely too heavily on sessions, or sessions are re-populated when one is invalidated, you're in good shape. I am happy that this works, we've used it successfully several times now. I would of course rather be able to deploy class patches and builds out of the container or application server, but this is a good work-around that I will try out for a while.