Introduction

In a previous article, I have shown three ways to reverse proxy apache tomcat.This article goes one step further and shows

  • how to add load balancing capabilities to a cluster of Tomcat web applications,
  • how to monitor the status of the load balancer

The methods described here are:

  • mod_proxy + mod_proxy_balancer : a standard http reverse proxy with the mod_balancing module
  • mod_proxy_ajp + mod_proxy_balancer : a similar configuration that connects to the tomcat ajp port
  • mod_jk : the jakarta module in a load balancing mode

Setup

The examples below consider four tomcat servers hosting the same Spring petclinic application. The Tomcat parameters are:

  • hostname: tomcat-server-inst1.local, ajp port : 8109, http port : 8180, jvmroute : worker1
  • hostname: tomcat-server-inst2.local, ajp port : 8209, http port : 8280, jvmroute : worker2
  • hostname: tomcat-server-inst3.local, ajp port : 8309, http port : 8380, jvmroute : worker3
  • hostname: tomcat-server-inst4.local, ajp port : 8409, http port : 8480, jvmroute : worker4

Fronting these is an Apache httpd server which hostname is: apache-server.local.

The examples below reverse proxy the petclinic web application. These have been tested using a linux server (SuSe Linux) but can be readily transferred to any apache installation simply by updating the paths in the configuration files.

mod_proxy + mod_proxy_balancer

Load balancer configuration

This configuration uses mod_proxy_balancer to define a load balancer named tomcat-cluster. The mod_proxy_http module then defines a standard http proxy that points to the load balancer.

#-- status module : provides server status information
LoadModule status_module 			/usr/lib64/apache2-prefork/mod_status.so

#-- proxy_module and prox_http_module : provides proxy capabilities
LoadModule proxy_module 			/usr/lib64/apache2-prefork/mod_proxy.so
LoadModule proxy_http_module 		/usr/lib64/apache2-prefork/mod_proxy_http.so

#-- proxy_balancer_module : load balancing capabilities
LoadModule proxy_balancer_module	/usr/lib64/apache2-prefork/mod_proxy_balancer.so

#--headers_module : for session stickiness support
LoadModule headers_module 			/usr/lib64/apache2-prefork/mod_headers.so

ExtendedStatus On

<Location /server-status>
	SetHandler server-status		
	Order Deny,Allow
	Allow from all
</Location> 


<VirtualHost *:80>
	ServerAdmin webmaster@dummy-host.example.com
	ServerName apache-server.local

	<Location /balancer-manager>
		SetHandler balancer-manager
		Order Deny,Allow
		Allow from all
	</Location> 

	<Proxy balancer://tomcat-cluster>
		BalancerMember http://tomcat-server-inst1.local:8180 route=worker1
		BalancerMember http://tomcat-server-inst2.local:8280 route=worker2
		BalancerMember http://tomcat-server-inst3.local:8380 route=worker3
		BalancerMember http://tomcat-server-inst4.local:8480 route=worker4
	</Proxy>
	
	ProxyPass			/petclinic	balancer://tomcat-cluster/petclinic stickysession=JSESSIONID|jsessionid scolonpathdelim=On
	ProxyPassReverse 	/petclinic	balancer://tomcat-cluster/petclinic 
 
	ProxyPass			/examples	balancer://tomcat-cluster/examples	stickysession=JSESSIONID|jsessionid scolonpathdelim=On
	ProxyPassReverse	/examples	balancer://tomcat-cluster/examples	
 

</VirtualHost>

Monitoring the load balancer

The above configuration declares two monitoring mecanisms:

mod_proxy_ajp + mod_proxy_balancer

Load balancer configuration

This configuration is very similar to the one above but uses the tomcat ajp ports instead of the http ports. The load balancer configuration is the same, using mod_proxy_balancer to define a load balancer named tomcat-cluster. The mod_proxy_ajp module then defines a standard ajp proxy that points to the load balancer.

#-- status module : provides server status information
LoadModule status_module 			/usr/lib64/apache2-prefork/mod_status.so

#-- proxy_module and prox_ajp_module : provides proxy capabilities
LoadModule proxy_module 			/usr/lib64/apache2-prefork/mod_proxy.so
LoadModule proxy_ajp_module			/usr/lib64/apache2-prefork/mod_proxy_ajp.so

#-- proxy_balancer_module : load balancing capabilities
LoadModule proxy_balancer_module	/usr/lib64/apache2-prefork/mod_proxy_balancer.so

#--headers_module : for session stickiness support
LoadModule headers_module 			/usr/lib64/apache2-prefork/mod_headers.so


ExtendedStatus On

<Location /server-status>
	SetHandler server-status		
	Order Deny,Allow
	Allow from all
</Location> 


<VirtualHost *:80>
	ServerAdmin webmaster@dummy-host.example.com
	ServerName apache-server.local

	<Location /balancer-manager>
		SetHandler balancer-manager
		Order Deny,Allow
		Allow from all
	</Location> 

	<Proxy balancer://tomcat-cluster>
		BalancerMember ajp://tomcat-server-inst1.local:8109 route=worker1
		BalancerMember ajp://tomcat-server-inst2.local:8209 route=worker2
		BalancerMember ajp://tomcat-server-inst3.local:8309 route=worker3
		BalancerMember ajp://tomcat-server-inst4.local:8409 route=worker4
	</Proxy>
	
	ProxyPass			/petclinic	balancer://tomcat-cluster/petclinic stickysession=JSESSIONID|jsessionid scolonpathdelim=On
	ProxyPassReverse 	/petclinic	balancer://tomcat-cluster/petclinic 
 
	ProxyPass			/examples	balancer://tomcat-cluster/examples	stickysession=JSESSIONID|jsessionid scolonpathdelim=On
	ProxyPassReverse	/examples	balancer://tomcat-cluster/examples	
 

</VirtualHost>

Monitoring the load balancer

The monitoring configuration is exactly the same as the one above:

mod_jk

Load balancer configuration

This configuration uses the jakarta module in its load balancer configuration. Every tomcat server is defined as a worker in the workers-lb.properties file; the loadbalancer worker is a special worker that acts as a load balancer to all the workers.
In the apache VirtualHost configuration, the JkMount points to the loadbalancer worker that in turns dispatches the requests to the individual workers.

Important note: for sticky sessions to work make sure that every tomcat installation defines in its conf/server.xml a unique value for the jvmroute parameter

/etc/apache2/vhosts.d/jk.conf

# load mod_jk
LoadModule jk_module  /usr/lib64/apache2/mod_jk.so
 
# Where to find workers.properties
JkWorkersFile /etc/apache2/conf.d/workers-lb.properties
 
# Where to put jk shared memory
JkShmFile     /var/log/apache2/mod_jk.shm
 
# Where to put jk logs
JkLogFile     /var/log/apache2/mod_jk.log
 
# Set the jk log level [debug/error/info]
JkLogLevel    info
 
# Select the timestamp log format
JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "

/etc/apache2/conf.d/workers-lb.properties

ps=/
worker.list=loadbalancer,status

worker.worker1.port=8109
worker.worker1.type=ajp13
worker.worker1.host=tomcat-server-inst1.local
worker.worker1.route=worker1

worker.worker2.port=8209
worker.worker2.type=ajp13
worker.worker2.host=tomcat-server-inst2.local
worker.worker2.route=worker2

worker.worker3.port=8309
worker.worker3.type=ajp13
worker.worker3.host=tomcat-server-inst3.local
worker.worker3.route=worker3

worker.worker4.port=8409
worker.worker4.type=ajp13
worker.worker4.host=tomcat-server-inst4.local
worker.worker4.route=worker4

worker.loadbalancer.type=lb
worker.loadbalancer.balance_workers=worker1,worker2,worker3,worker4
worker.loadbalancer.sticky_session=True

worker.status.type=status

/etc/apache2/vhosts.d/petclinic.conf

<VirtualHost *:80>
	ServerAdmin webmaster@dummy-host.example.com
	ServerName apache-server.local
	
	JkMount /petclinic   loadbalancer
	JkMount /petclinic/* loadbalancer

	JkMount /examples/	loadbalancer
	JkMount /examples/*	loadbalancer

	JkMount /status   status
	JkMount /status/*   status
	
</VirtualHost>

Monitoring the load balancer

Monitoring the status of the load balancer is done via the status worker.
The status worker is a special worker that allows the monitoring and real time modification of the loadbalancer

Advertisements