Page tree
Skip to end of metadata
Go to start of metadata

STOP!!

Before proceeding be sure you have read the general introduction to clustering.

Configuring IdP Clustering

The recommended method for clustering the IdP is to use Terracotta to replicate the IdP's in-memory state between nodes. Terracotta is composed of two parts. Terracotta servers that are responsible for lock management and moving data between clients and Terracotta clients which run in the IdP software and publish/receive changed data to/from the server.

The Shibboleth team recommends running a Terracotta server on each IdP node. In this model one server will run in an active mode while the others run in passive mode. If the active server fails the passive servers elect a new active node and the clients reconnect to it.

In this documentation $TC_HOME and $IDP_HOME are used to denote the directory where Terracotta and the IdP, respectively, have been installed.

Step 1: Adjusting your Firewall

Terracotta uses three ports. Two ports, client-to-server port (default: 9510) and a server-to-server port (default: 9530), are used for inter-node communications. Neither of these ports should be reachable from outside the cluster. A third port (default: 9520) is available and used for JMX. This port should be closed off totally if you are not using JMX or should be expressly limited to particular, trusted hosts. Access to this port allows access to IdP run time information which may include passwords and user information.

Step 2: Edit the Terracotta Configuration file

  1. Download tc-config.xml
  2. Edit the version numbers for the terracotta integration modules (tim-vector, tim-tomcat-6.0) if the modules are updated.
  3. Edit the configuration file $IDP_HOME/conf/tc-config.xml. At the start of the file you will see an example server definition. You will need to add a server definition for each Terracotta server. We recommend just copying and pasting the example Server definition and changing the following attributes:
    • name - a unique name for this server (e.g. idpNode1)
    • host - the hostname or IP address of the server (this information is used by clients and servers when they contact a server).
    • bind - (optional attribute) gives the IP address/interface to which the server should bind (defaults to 0.0.0.0)

All other information in the example should be fine. Once you've made the changes distribute this file to each IdP node in your cluster.

Terracotta Server Definition Example
<server name="idpNode1" host="node1.idp.example.org" bind="192.168.1.15">
    <dso>
        <persistence>
            <mode>permanent-store</mode>
        </persistence>
    </dso>

    <logs>$IDP_HOME$/cluster/server/logs</logs>
    <data>$IDP_HOME$/cluster/server/data</data>
    <statistics>$IDP_HOME$/cluster/server/stats</statistics>
</server>

Step 3: Install Terracotta

  1. Download Terracotta
  2. Install Terracotta
    • Unix users - expand the downloaded archive (/opt is the suggested location)
    • Windows users - run the downloaded .exe file
  3. Set the environment variable TC_INSTALL_DIR to $TC_HOME
  4. Run the command: $TC_HOME/bin/tim-get.sh install tim-vector 2.6.3#* This pulls down and installs a Terracotta Integration Module (TIM) used by the IdP
  5. Run the command: $TC_HOME/bin/tim-get.sh install tim-tomcat-6.0 2.1.3#* This pulls down and installs the TIM for Tomcat 6.0
  6. Run the command: $TC_HOME/bin/make-boot-jar.sh -f $IDP_HOME/conf/tc-config.xml#* This command will output the location of the created JAR. Write this down as you will need it when start the IdP. This JAR is specific to your JVM make and version.

The instructions above apply to Terracotta version 3.2.1

For Terracotta version 2.7.3, the Terracotta Integration Module for Tomcat was not needed. If you want to use this version, you may find a copy on the public FTP server of Vienna University. Questions regarding this should be directed at the Shibboleth-Users mailing list or directly at Peter Schober.

Step 4. Adjust IdP Configuration (Optional)

If you are supporting artifact resolution you will also need to make a small change to your IdP's internal.xml file. Locate the bean definition that starts with <bean id="shibboleth.ArtifactMap" and replace it with the following definition.

<bean id="shibboleth.ArtifactMap" class="org.opensaml.common.binding.artifact.BasicSAMLArtifactMap" depends-on="shibboleth.LogbackLogging">
    <constructor-arg>
        <bean class="org.opensaml.common.binding.artifact.BasicSAMLArtifactMapEntryFactory">
            <property name="serializeMessage" value="true" />
        </bean>
    </constructor-arg>
    <constructor-arg ref="shibboleth.StorageService" />
    <constructor-arg type="long" value="300000" />
</bean>

If your IdP is using Javascript ScriptAttributeDefinitions, you may experience out of memory conditions due to the Javascript optimizer creating lots of temporary classes. You can work around this by replacing lib/rhino-V###.jar in the IdP distribution with a copy of the .jar with the org/mozilla/javascript/optimizer directory removed.


Starting Terracotta Servers

Terracotta servers are started with the start-tc-server script located in the bin directory of your Terracotta install. This command takes two options:

  • -n - the name of the server as given in the Terracotta configuration file
  • -f - the fully qualified filesystem path or URL to the terracotta configuration file
Example Terracotta Server Startup Command
start-tc-server.sh -n idpNode1 -f http://www.example.org/idp-cluster/tc-config.xml

Start your Terracotta server before the IdP otherwise the clients will have no server to which to connect.

This page has a sample script for RHEL/CentOS to start Terracotta before Tomcat. The script can be placed into the /etc/init.d directory. Using chkconfig to add it properly to the system startup. You must edit this script to reflect your environment and setup directories.

Starting the IdP & Terracotta Client

In order start the Terracotta client within the JVM used by the IdP you must pass in three JVM arguments. How exactly you do this depends on your Servlet container.

  • -Dtc.install-root=$TC_HOME
  • -Dtc.config=CONFIG_FILE_PATH_OR_URL
  • -Xbootclasspath/p:<PATH_TO_CLIENT_JAR_CREATED_BY_TERRACOTTA>
Starting Tomcat with Terracotta Client Options
JAVA_OPTS="-Dtc.install-root=/opt/terracotta \
           -Dtc.config=http://www.example.org/idp-cluster/tc-config.xml \
           -Xbootclasspath/p:/opt/terracotta/lib/dso-boot/dso-boot-hotspot_osx_160_12.jar" \
/opt/tomcat/bin/startup.sh

For jdk-1.6.0_18 and above, you must add to JAVA_OPTS the flag -XX:+MustCallLoadClassInternal to avoid startup issues with Tomcat due to a change in classloading made in the JDK.  Also be sure to review the streamlining instructions described below for additional Tomcat and Terracotta options which you will need to tune to your environment for a truly robust installation.

This page has a sample script to start up Tomcat with Terracotta on RHEL/CentOS. The script can be placed into the /etc/init.d directory. Use chkconfig to add it properly to the system startup. You must edit this script to reflect your environment and setup directories.

Productionalizing

Instructions for streamlining formerly were listed here.  Should you be considering a production deployment, you should read those notes which are now documented here: IdPClusterIssues,

  • No labels