Page tree
Skip to end of metadata
Go to start of metadata

Upgrades MUST be done in place!

If you are upgrading from V3 or V4.0, PLEASE follow the directions on the Upgrading page and do NOT install V4 separately and then attempt to apply your old configuration files. You MUST upgrade "in place" by using an installation directory that contains a copy of the previously working configuration.

Failure to do so will result in a non-working system in the majority of cases because there are absolutely essential differences between the output of the installer in the two cases to prevent your configuration from being altered in unexpected ways.

This problem is exacerbated by the extent of the internal changes made to V4.1 and the results of combining old and new files without an extreme amount of effort will be unpleasant. It is urgent that people heed this warning and attempt to modernize settings (if at all) only after upgrading and verifying the behavior of the system.

Please review these release notes before upgrading your system. You should review the notes for all the versions subsequent to the one you're running prior to upgrade, including referring back to older V3 notes.

Also be aware of the following issues regarding container or Java compatibility:

Known Bugs

These are known issues in the code that are likely to affect deployers and have yet to be fixed.

  • IDP-1623 - Getting issue details... STATUS IDP-1654 - Getting issue details... STATUS
    • This is a bug that impacts DataConnectors that rely on the new "exporting" feature and then fail to return any results (often, but not always because they were sipressed by an ActivationCondition). The simplest workaround for now is to insure such connectors either fail with errors, or are backed up by a Failover connector that ensures some kind of dummy result is returned.
  • IDP-1625 - Getting issue details... STATUS
    • This issue breaks the Cancel option in the Duo login flow when the CSRF feature is enabled. The fix is documented in the issue and can be applied directly to the duo.vm template. Since it involves a template, the fix will be manual and noted in the next version's release notes.



4.1.0 (Unreleased)

Getting issues...

This is a significant new feature release that includes a larger than usual number of new configuration options, but these are backward-compatible and mostly simplify things for new deployers. Existing systems may be adapted to take advantage of these new features in order to reduce the number of unused configuration files. Specifics can be found among the various topics, but you may wish to review the new material on ModuleConfiguration and consider disabling any modules associated with unused features. This will remove some unchanged files outright and rename others so that you can find and remove them.

Note that while much of the documentation includes sections of material labeled "V4.1+", this doesn't mean the V4.0 material is "non-functional" in an upgraded system. It's just easier to document approaches that are "standard" in each version without constantly noting this.

Changes to Defaults

The following defaults have changed for new installs but will not change for properly upgraded systems (generally these are set through explicitly uncommented properties or beans that have to be present to enable a behavior):

  • The new idp.bindings.inMetadataOrder property defaults to true, but is set to false in new installs. The primary purpose and impact is discussed below under "Restoration of Logout Behavior".

Elimination of system Files

With this release, we have eliminated the requirement for a "system" directory in the installation tree in favor of embedding the resources into the system's libraries. Because of historical references in the deployer-editable web.xml file to files in this directory, it's not possible to remove it entirely from an upgraded system to maintain compatibility, but new installs will not include it. A future release will require modifications to web.xml to allow for complete elimination of the directory.

As part of this change, you may also wish to remove a stale reference to "%{idp.home}/system/messages/messages" from the shibboleth.MessageSourceResources bean in services.xml

New Property Files and Property Lookup

Many common settings that used to be configurable only with XML files and Spring configuration are now exposed as properties (if and only if the original beans don't exist, for compatibility). There are new property files in the various subdirectories (conf/authn/authn.properties, conf/admin/admin.properties, conf/c14n/subject-c14n.properties) that contain examples of these settings. While new installs automatically take advantage of these properties, (properly) upgraded systems will not without other adjustments. The linked topics discuss how to "enable" their use in those cases for those that want to take advantage of them.

Also take note that with this release, the process of "finding" property files has been improved. In older versions, and when the idp.searchForProperties property is false/missing, the set of property files loaded is solely controlled by the idp.additionalProperties setting. New installs by default, or installs that explicitly enable the idp.searchForProperties property, will locate and load all files with the ".properties" extension in the conf tree. The explicit property can still be used for directly loading files from alternate locations.

Restoration of Logout Behavior

Adding SOAP Logout support had an unintentional side effect of causing the SOAP binding to be selected at times instead of the more advisable approach, required by the standard, of favoring front-channel bindings. This problem of binding selection dates back to V2 and was caused by the IdP choosing endpoints in metadata order. This has been addressed with a new property, idp.bindings.inMetadataOrder, that can be set to false to produce a more desirable binding selection order that favors Redirect and POST when supported. It defaults to true for compatibility, but is set to false for new installs.

IPv6 Client Address Sanitization

After waiting for a fix upstream that got reversed in a patch, we have applied a global fix to our code to ensure that IPv6 client addresses are consistently exposed without the use of brackets around the address, which should be confined to "address-as-host" scenarios, not bare addresses. This corrects a number of problems, but may result in changes to log files and could impact scripts that were not coded defensively to handle either format.

New Objects

Renamed Objects

New Properties

There are a lot of new properties because of the aforementioned changes, but a few miscellaneous settings are mentioned here:


4.0.1.1 (July 22, 2020)

This is a service release of the Windows installation package in order to address a security issue in Jetty, which has been updated to version 9.4.30. It is unneeded by anyone maintaining their own Java container, which continues to be strongly recommended.

4.0.1 (June 3, 2020)

Getting issues...

This is a patch release to address bugs identified and to shore up some missing LDAP features that arose due to the loss of JNDI support, such as SASL authentication and referral handling. The LDAP features are an exception to our usual policy on new features in a patch release and configuration options are denoted with the 4.0.1 designation where applicable.

A bug was corrected that was causing metadata resolver crashes on invalid metadata containing empty elements. Previous releases inadvertently accepted and ignored these elements. The patch adds warnings about the invalid content but corrects the crashing bug.

Jetty has been updated for the Windows installer to version 9.4.28 and an issue corrected with the use of wildcard certificates. The full fix for this in the case of back-channel deployment involves an update to the plugin we supply for certificate validation bypass, and this plugin has been renamed to reflect that it is designed solely for use with Jetty 9.4. The Jetty94 page has been updated to reflect this. The older plugin and back-channel XML configuration remains compatible for now, but will fail in the unlikely even a wildcard or multi-SAN certificate were used.

Changes to Defaults

  • The transcoding rule shipped with 4.0.0 for the "telephoneNumber" attribute had the SAML 1 and SAML 2 names swapped, so upgraded systems will carry this mistake until corrected by hand.

Known Issues

  • A bug was introduced into the default web.xml file in the interminable attempts to fix warnings around the default method constraints. The element <authn-constraint/> should have been <auth-constraint/>. This may cause warnings or result in the rule being ignored.

4.0.0 (March 11, 2020)

Getting issues...

This is the first release of the fourth-generation Identity Provider software. The key documentation links are located on the IDP4 space Home page, such as SystemRequirements, Installation, and Upgrading material. Note the new SystemRequirements as they have substantially changed with regard to Java and container versions.

This release should interoperate with all previous releases of Shibboleth and other software that supports the same standards.

As a major upgrade, the list of issues fixed and features added is numerous. Many significant changes are highlighted below, with particularly noteworthy issues highlighted in note or warning boxes.

There are API additions, changes, and removals, and any third-party extensions will need to be checked for code compatibility and recompiled for use with this version, and any custom scripts created by deployers should be carefully tested and reviewed.

Changes to Defaults

The following defaults have changed for both new installs and upgrades and should be reviewed for possible incompatibilities:

  • The idp.cookie.secure property now defaults to true if not set by the deployer. In the unlikely event that HTTP compatibility is a requirement, the property would need to be adjusted. (Note that the container's session cookies are controlled by policy in web.xml and while we have also changed that default, only upgraders without their own web.xml customizations would be exposed to that change.)
  • The default LDAP provider is now UnboundID. See the LDAP section below for more information.
  • Some legacy code that was used to maintain the V2-style formatting of the %T audit field (the timestamp of the record) was removed and the log now outputs all date/time fields using the supplied formatter or a default. The format of the %T field may therefore change for some deployers.
    • Related, the shibboleth.AuditDateTimeFormat bean is interpreted as a pattern by the Java DateTimeFormatter class, rather than the deprecated joda-time version, and there are some differences in those classes that may result in changes to your logs. If you require identical output, review is warranted.
  • A side effect of the attribute-related changes noted below is that SAML 2.0 AttributeEncoders with no friendlyName attribute default to populating FriendlyName with the attribute's ID. At present there is no way to restore the original behavior other than to convert to use of the AttributeRegistryConfiguration to express the rule.

The following defaults have changed for new installs but will not change for properly upgraded systems (generally these are set through explicitly uncommented properties or beans that have to be present to enable a behavior):

  • The default XML Encryption algorithm is now AES128-GCM rather than AES128-CBC. See GCMEncryption for much more on this, and do note that newly installed systems will fail to interoperate with a large number of SAML SPs without modern algorithm support.
  • HTML Local Storage is now enabled by default when the client-side session storage feature is used (which has been and remains the default option as well).
    • In addition the logout-enabling idp.session.trackSPSessions and idp.session.secondaryServiceIndex properties are also enabled by default. They can be disabled by deployers that do not wish to try and support logout propagation.
  • The set of profiles enabled by default no longer includes any SAML 1.1 support, or SAML 2.0 Attribute Query.
  • The default trust engines used at runtime now omit PKIX support. Support for only the Metadata Interoperability Profile is now the default behavior with no attempt at a fallback.
  • A few ReloadableServices that were configured by default to fail-fast have changed to non-fail-fast behavior.
  • Audit output has been changed to include a much more useful default format and various improvements to how specific fields are logged to shrink repetitive and unnecessary constant values.

Removal of Deprecated Features

A number of previously deprecated features, settings, APIs, etc. have been removed. The use of these deprecated bits should have produced a standard warning either at startup or during use in the most recent releases of the software, and most of them are now non-functional. Eliminating warnings before upgrading is strongly advised.

Date/Time APIs

The entire code base has been converted from the joda-time Date/Time API to the new Date/Time API introduced in Java 8. This change is widespread in the code, with most uses of long/Long types to represent timestamps and durations converted into Instant and Duration classes.

The change is mostly invisible to deployers because of the widespread support for ISO Duration notation in the configuration. There are a couple of areas of direct impact that are now deprecated and will be removed from a future version:

It was also observed that V3 may have been improperly interpreting timestamps without the trailing 'Z' indicator as compliant UTC values, which was not a correct assumption. Using such values is a violation of the SAML specification in any event, and V4 does not handle such values and will reject them in most cases as expired or in the future depending on the default time zone of the system relative to UTC.

Predicate/Function APIs

The entire code base has been converted from the Guava Predicate/Function API to the new functional API interfaces introduced in Java 8. This change is widespread in the code, but we have supplied migration aids via our numerous utility/parent beans and a support class that converts between the legacy Guava Predicate.apply() signature and the new standard Predicate.test() signature.

We expect most custom code and scripts based on Guava Predicates will work unmodified, but may produce warnings when the apply() method is called to allow code cleanup and prevent future problems.

One place we missed is near the bottom of the views/login.vm file. If you happen to be using the "not-quite-deprecated" Extended Flows feature, there's a call to "apply" in the template that needs to be manually changed to "test". This also applies to new installs prior to V4.1.0 which will contain the fixed template.

Beware of Guava's "compose" method

Legacy use of Guava Functions may work for the most part, but there is a known issue in Spring's wiring behavior that breaks the use of Guava's "compose" helper. If you have custom bean wiring anywhere that contains a reference like this, you will likely have to update it to avoid problems.

 <bean class="com.google.common.base.Functions" factory-method="compose">

Using that syntax will often result in the two constructor arguments to that bean getting swapped/reversed, which reverses the order of the composition, usually producing bad output at best, and a crash or even more catastrophic outcome at worst.

Unfortunately there's no easy fix for this because of the way Guava accomodates the conversion to Java 8's Function interface. As a result, the best thing you can do is simply scan for this and convert the above syntax to:

<bean parent="shibboleth.Functions.Compose">

That is functionally equivalent and should avoid the re-ordering behavior we're seeing with Spring.

Profile Configuration APIs

A lot of cleanup was performed to eliminate inconsistent method names in the APIs used for configuring and overriding profile behavior in relying-party.xml using overrides. In a small number of advanced cases, people may encounter errors in their configuration due to type incompatibilities, which may require reviewing the appropriate class Javadoc. But as a rule of thumb, all the property names now follow these conventions:

  • p:something takes a literal value such as a boolean, integer, long, String, Duration, etc.
  • p:somethingPredicate takes a Java Predicate when the equivalent literal would be a boolean
  • p:somethingLookupStrategy takes a Java Function when the equivalent literal would be other than a boolean

As an example, the V3 property signAssertions was Predicate-valued, while the V4 version takes a literal boolean and signAssertionsPredicate takes a Predicate. There are a number of such cases but not many with common use.

LDAP Changes

The default LDAP implementation is now the UnboundID library and JNDI is no longer officially supported. There will be no JNDI support at all in a future version and we expect to migrate to a native implementation being built within a future version of Ldaptive at that time.

This release includes configuration updates to facilitate that change, particularly in the LDAPAuthnConfiguration and to a lesser degree in the LDAPConnector. Deployers using this feature should review and update their configurations when possible to modernize them and minimize the need for future changes. In many cases, the ldap-authn-config.xml file can be replaced outright with no impact, along with an additional change noted in the LDAPAuthnConfiguration page under "Note Regarding Upgrades".

Most previously necessary <LDAPProperty> elements can be replaced by various new (or recently added) settings that configure the equivalent behavior in a non-JNDI fashion. For example, specifying binary attributes in the LDAPConnector is now possible directly via the <BinaryAttributes> element added in a V3.4 patch. If you encounter properties that have no obvious replacement, seek assistance.

The ldaptive JAAS module has an incompatible change in its handling of timeout settings. Specifying values in millseconds will result in a DateTime parsing exception. The values must be converted to ISO syntax (e.g., 2000 → PT2S).

In addition, various LDAP pooling properties prefixed by "idp.pool.LDAP." (such as idp.pool.LDAP.validatePeriod) previously defaulted to numeric values expressed in seconds (e.g. 300 == 5 minutes). These are now being interpreted as milliseconds, which causes overly frequent pool validation if the defaults are used with an unmodified V3 ldap-authn-config.xml file. This is dicussed further on the LDAPAuthnConfiguration page.

Password Flow Extension

The Password login flow has undergone significant internal reimplementation to support a much higher degree of flexibility. The redesign has resulted in the elimination of one existing extension mechanism that we don't believe was heavily used, described in the V3 documentation under the heading Custom Back-Ends.

A simpler extension point is now provided for implementation of such custom validators, and converting is not a complex task.

Attribute-Related Changes

A major new addition to V4 is the AttributeRegistry service that replaces, or at least supplements, the AttributeEncoder mechanism for translating attributes. While upgraded systems should operate unchanged, deployers are advised to review this new feature and may achieve substantial simplification of their configuration by taking advantage of it. Many existing encoders may be removeable.

Note that installing from scratch and then applying a legacy configuration will in most cases result in duplicate Attributes appearing in SAML messages due to the overlap between the existing encoders and the new rules. The upgrade process is designed to prevent that by excluding the new rules from the default configuration. Of course, the inverse is also true: if you want the new default rules to apply, you must make that transition after upgrading.

A consequence of this new service is that adjusting AttributeEncoders requires reloading "shibboleth.AttributeRegistryService" and not the resolver service as in previous releases.

In the unlikely event that you have any attribute display name or description metadata present for an attribute definition but do not have an encoder, the compatibility support is not going to attach that display metadata to the resulting attribute. This does not seem like a likely scenario, but it's a change.

The rules for attribute IDs are tightened in this release to preclude whitespace and single quotes. With advance warning this set may be tightened further if warranted. We strongly advise sticking to simple ASCII characters.

The class hierarchy around IdPAttributeValue has been simplified and the use of generics eliminated. Some scripts may be impacted by changes to method names. There is no longer a universal getValue method exposed, but getNativeValue generally returns the equivalent result save for scoped values, where a Pair is returned. This change prevents unintentional "lossy" conversions to String for non-String values.

The removal of the deprecated <SourceAttribute> element from the TemplateAttributeDefinition syntax creates a rare scenario that can break existing behavior in a small set of cases if an InputDataConnector is supplied with the allAttributes option set. Using this syntax requires that all inputs either have the same number of values (or no values). Instead of using the <SourceAttribute> element in conjunction with allAttributes, simply enumerate the attributes to be sourced from the connector.

Metadata Changes

Due to additional "up front" processing of some SAML metadata content, certain invalid SAML constructs that used to be ignored are no longer being tolerated during the initial load of a MetadataProvider, in particular the appearance of empty XML elements such as <md:ServiceName/><md:ServiceDescription/>, and all the child elements within the <md:Organization> element.

Such elements are invalid according to the standard and the IdP's behavior is correct, but the initial release had a bug that results in a NullPointerException instead of ignoring the element. The crash was corrected in V4.0.1. Deployers should not in general rely on the IdP to be tolerant of invalid SAML constructs.

Logout Changes

The Single Logout feature now includes some additional control points, properties, and changes to the default views that allow more control over how logout proceeds, whether users are able to cancel/abort them, and other requested "flow of control" enhancements. These are discussed in the LogoutConfiguration topic.

In addition, SOAP logout for SAML has been implemented for the first time, and unfortunately this change can cause some impact to existing deployments because of a bug related to how bindings are chosen, which has historically been in metadata order. This can cause SOAP to be selected in preference to front-channel bindings, which is an error. Control over this has been added for a future release. This section will be updated with more information as various aspects of controlling the SOAP logout behavior are investigated.

New Deprecations

Due to a Spring bug that causes problems with configuration reloading, we have deprecated the original set of HttpClient parent beans and any inheritence from those beans in custom HttpClient configurations. We have introduced new parent beans without predefined settings that were causing the problems, and adjusted the HttpClientConfiguration documentation in various ways to refer to them.

Internal components that rely on a default HttpClient configuration now rely on a system bean definition intended only for internal use, configured as before via the services.properties file.

If you have beans that are inheriting from the deprecated versions, a warning will be issued. They will be removed from V5. This extends to several properties that were used to configure the caching versions of the clients, which will be removed.

CSRF Prevention

The IdP includes a new Cross-Site Request Forgery (CSRF) Protection feature that is enabled by default for new installs. The idp.csrf.enabled property is a general on/off switch if problems are encountered or upgraders wish to enable it. The feature can also be easily incorporated in custom webflow views with minimal effort.

Extensible Installer

Mostly of interest to packagers, the installer is now built with Java and has explicit extension points. See Programming the V4 Installer.

Noteworthy New Features

New Objects

New Properties



  • No labels