Before proceeding be sure you have read the general introduction to clustering.
Configuring IdP Clustering
The recommended method for clustering the IdP is to use Terracotta to replicate the IdP's in-memory state between nodes. Terracotta is composed of two parts. Terracotta servers that are responsible for lock management and moving data between clients and Terracotta clients which run in the IdP software and publish/receive changed data to/from the server.
The Shibboleth team recommends running a Terracotta server on each IdP node. In this model one server will run in an active mode while the others run in passive mode. If the active server fails the passive servers elect a new active node and the clients reconnect to it.
In this documentation $TC_HOME and $IDP_HOME are used to denote the directory where Terracotta and the IdP, respectively, have been installed.
Step 1: Adjusting your Firewall
Terracotta uses three ports. Two ports, client-to-server port (default: 9510) and a server-to-server port (default: 9530), are used for inter-node communications. Neither of these ports should be reachable from outside the cluster. A third port (default: 9520) is available and used for JMX. This port should be closed off totally if you are not using JMX or should be expressly limited to particular, trusted hosts. Access to this port allows access to IdP run time information which may include passwords and user information.
Step 2: Edit the Terracotta Configuration file
- Download tc-config.xml
- Edit the version numbers for the terracotta integration modules (tim-vector, tim-tomcat-6.0) if the modules are updated.
- Edit the configuration file
$IDP_HOME/conf/tc-config.xml. At the start of the file you will see an example server definition. You will need to add a server definition for each Terracotta server. We recommend just copying and pasting the example
Serverdefinition and changing the following attributes:
- name - a unique name for this server (e.g. idpNode1)
- host - the hostname or IP address of the server (this information is used by clients and servers when they contact a server).
- bind - (optional attribute) gives the IP address/interface to which the server should bind (defaults to 0.0.0.0)
All other information in the example should be fine. Once you've made the changes distribute this file to each IdP node in your cluster.
Step 3: Install Terracotta
- Download Terracotta
- Install Terracotta
- Unix users - expand the downloaded archive (/opt is the suggested location)
- Windows users - run the downloaded
- Set the environment variable
- Run the command:
$TC_HOME/bin/tim-get.sh install tim-vector 2.6.3#* This pulls down and installs a Terracotta Integration Module (TIM) used by the IdP
- Run the command:
$TC_HOME/bin/tim-get.sh install tim-tomcat-6.0 2.1.3#* This pulls down and installs the TIM for Tomcat 6.0
- Run the command:
$TC_HOME/bin/make-boot-jar.sh -f $IDP_HOME/conf/tc-config.xml#* This command will output the location of the created JAR. Write this down as you will need it when start the IdP. This JAR is specific to your JVM make and version.
The instructions above apply to Terracotta version 3.2.1
For Terracotta version 2.7.3, the Terracotta Integration Module for Tomcat was not needed. If you want to use this version, you may find a copy on the public FTP server of Vienna University. Questions regarding this should be directed at the Shibboleth-Users mailing list or directly at Peter Schober.
Step 4. Adjust IdP Configuration (Optional)
If you are supporting artifact resolution you will also need to make a small change to your IdP's
internal.xml file. Locate the bean definition that starts with
<bean id="shibboleth.ArtifactMap" and replace it with the following definition.
Starting Terracotta Servers
Terracotta servers are started with the
start-tc-server script located in the
bin directory of your Terracotta install. This command takes two options:
- -n - the name of the server as given in the Terracotta configuration file
- -f - the fully qualified filesystem path or URL to the terracotta configuration file
Start your Terracotta server before the IdP otherwise the clients will have no server to which to connect.
This page has a sample script for RHEL/CentOS to start Terracotta before Tomcat. The script can be placed into the /etc/init.d directory. Using chkconfig to add it properly to the system startup. You must edit this script to reflect your environment and setup directories.
Starting the IdP & Terracotta Client
In order start the Terracotta client within the JVM used by the IdP you must pass in three JVM arguments. How exactly you do this depends on your Servlet container.
For jdk-1.6.0_18 and above, you must add to JAVA_OPTS the flag -XX:+MustCallLoadClassInternal to avoid startup issues with Tomcat due to a change in classloading made in the JDK. Also be sure to review the streamlining instructions described below for additional Tomcat and Terracotta options which you will need to tune to your environment for a truly robust installation.
This page has a sample script to start up Tomcat with Terracotta on RHEL/CentOS. The script can be placed into the /etc/init.d directory. Use chkconfig to add it properly to the system startup. You must edit this script to reflect your environment and setup directories.
Instructions for streamlining formerly were listed here. Should you be considering a production deployment, you should read those notes which are now documented here: IdPClusterIssues,