Upgrades MUST be done in place!
If you are upgrading from V3, PLEASE follow the directions on the Upgrading page and do NOT install V4 separately and then attempt to apply your old configuration files. You MUST upgrade "in place" by using an installation directory that contains a copy of the previously working configuration.
Failure to do so will result in a non-working system in the majority of cases because there are absolutely essential differences between the output of the installer in the two cases to prevent your configuration from being altered in unexpected ways.
Please review these release notes before upgrading your system. You should review the notes for all the versions subsequent to the one you're running prior to upgrade, including referring back to older V3 notes.
Also be aware of the following issues regarding container or Java compatibility:
4.0.0 (March 11, 2020)
This is the first release of the fourth-generation Identity Provider software. The key documentation links are located on the IDP4 space Home page, such as SystemRequirements, Installation, and Upgrading material. Note the new SystemRequirements as they have substantially changed with regard to Java and container versions.
This release should interoperate with all previous releases of Shibboleth and other software that supports the same standards.
As a major upgrade, the list of issues fixed and features added is numerous. Many significant changes are highlighted below, with particularly noteworthy issues highlighted in note or warning boxes.
There are API additions, changes, and removals, and any third-party extensions will need to be checked for code compatibility and recompiled for use with this version, and any custom scripts created by deployers should be carefully tested and reviewed.
Changes to Defaults
The following defaults have changed for both new installs and upgrades and should be reviewed for possible incompatibilities:
- The idp.cookie.secure property now defaults to true if not set by the deployer. In the unlikely event that HTTP compatibility is a requirement, the property would need to be adjusted. (Note that the container's session cookies are controlled by policy in web.xml and while we have also changed that default, only upgraders without their own web.xml customizations would be exposed to that change.)
The following defaults have changed for new installs but will not change for properly upgraded systems (generally these are set through explicitly uncommented properties or beans that have to be present to enable a behavior):
- The default XML Encryption algorithm is now AES128-GCM rather than AES128-CBC. See GCMEncryption for much more on this, and do note that newly installed systems will fail to interoperate with a large number of SAML SP without modern algorithm support.
- HTML Local Storage is now enabled by default when the client-side session storage feature is used (which has been and remains the default option as well).
- In addition the logout-enabling idp.session.trackSPSessions and idp.session.secondaryServiceIndex properties are also enabled by default. They can be disabled by deployers that do not wish to try and support logout propagation.
- The set of profiles enabled by default no longer includes any SAML 1.1 support, or SAML 2.0 Attribute Query.
- The default trust engines used at runtime now omit PKIX support. Support for only the Metadata Interoperability Profile is now the default behavior with no attempt at a fallback.
- A few ReloadableServices that were configured by default to fail-fast have changed to non-fail-fast behavior.
- Audit output has been changed to include a much more useful default format and various improvements to how specific fields are logged to shrink repetitive and unnecessary constant values.
Removal of Deprecated Features
A number of previously deprecated features, settings, APIs, etc. have been removed. The use of these deprecated bits should have produced a standard warning either at startup or during use in the most recent releases of the software, and most of them are now non-functional. Eliminating warnings before upgrading is strongly advised.
The entire code base has been converted from the joda-time Date/Time API to the new Date/Time API introduced in Java 8. This change is widespread in the code, with most uses of long/Long types to represent timestamps and durations converted into Instant and Duration classes.
The change is mostly invisible to deployers because of the widespread support for ISO Duration notation in the configuration. There are a couple of areas of direct impact that are now deprecated and will be removed from a future version:
- Passing back a login timestamp as a joda-time object in the External login flow.
- Use of a joda-time DateTimeFormatter object, primarily (but not required) for ExpiringPasswordInterceptConfiguration.
It was also observed that V3 may have been improperly interpreting timestamps without the trailing 'Z' indicator as compliant UTC values, which was not a correct assumption. Using such values is a violation of the SAML specification in any event, and V4 does not handle such values and will reject them in most cases as expired or in the future depending on the default time zone of the system relative to UTC.
The entire code base has been converted from the Guava Predicate/Function API to the new functional API interfaces introduced in Java 8. This change is widespread in the code, but we have supplied migration aids via our numerous utility/parent beans and a support class that converts between the legacy Guava Predicate.apply() signature and the new standard Predicate.test() signature. We expect most custom code and scripts will work unmodified, but may produce warnings when the apply() method is called to allow code cleanup and prevent future problems.
Profile Configuration APIs
A lot of cleanup was performed to eliminate inconsistent method names in the APIs used for configuring and overriding profile behavior in relying-party.xml using overrides. In a small number of advanced cases, people may encounter errors in their configuration due to type incompatibilities, which may require reviewing the appropriate class Javadoc. But as a rule of thumb, all the property names now follow these conventions:
- p:something takes a literal value such as a boolean, integer, long, String, Duration, etc.
- p:somethingPredicate takes a Java Predicate when the equivalent literal would be a boolean
- p:somethingLookupStrategy takes a Java Function when the equivalent literal would be other than a boolean
As an example, the V3 property
signAssertions was Predicate-valued, while the V4 version takes a literal boolean and
signAssertionsPredicate takes a Predicate. There are a number of such cases but not many with common use.
The default LDAP implementation is now the UnboundID library and JNDI is no longer officially supported. There will be no JNDI support at all in a future version and we expect to migrate to a native implementation being built within a future version of Ldaptive at that time.
This release includes configuration updates to facilitate that change, particularly in the LDAPAuthnConfiguration and to a lesser degree in the LDAPConnector. Deployers using this feature should review and update their configurations when possible to modernize them and minimize the need for future changes. In many cases, the ldap-authn-config.xml file can be replaced outright with no impact.
Most previously necessary
<LDAPProperty> elements can be replaced by various new (or recently added) settings that configure the equivalent behavior in a non-JNDI fashion. For example, specifying binary attributes in the LDAPConnector is now possible directly via the
<BinaryAttributes> element added in a V3.4 patch.
The ldaptive JAAS module does have an incompatible change in its handling of timeout settings. Specifying values in millseconds will result in a DateTime parsing exception. The values must be converted to ISO syntax (e.g., 2000 → PT2S).
Password Flow Extension
The Password login flow has undergone significant internal reimplementation to support a much higher degree of flexibility. The redesign has resulted in the elimination of one existing extension mechanism that we don't believe was heavily used, described in the V3 documentation under the heading Custom Back-Ends.
A simpler extension point is now provided for implementation of such custom validators, and converting is not a complex task.
A major new addition to V4 is the AttributeRegistry service that replaces, or at least supplements, the AttributeEncoder mechanism for translating attributes. While upgraded systems should operate unchanged, deployers are advised to review this new feature and may achieve substantial simplification of their configuration by taking advantage of it. Many existing encoders may be removeable.
Note that installing from scratch and then applying a legacy configuration will in most cases result in duplicate Attributes appearing in SAML messages due to the overlap between the existing encoders and the new rules. The upgrade process is designed to prevent that by excluding the new rules from the default configuration.
In the unlikely event that you have any attribute display name or description metadata present for an attribute definition but do not have an encoder, the compatibility support is not going to attach that display metadata to the resulting attribute. This does not seem like a likely scenario, but it's a change.
The rules for attribute IDs are tightened in this release to preclude whitespace and single quotes. With advance warning this set may be tightened further if warranted. We strongly advise sticking to simple ASCII characters.
The class hierarchy around IdPAttributeValue has been simplified and the use of generics eliminated. Some scripts may be impacted by changes to method names. There is no longer a universal getValue method exposed, but getNativeValue generally returns the equivalent result save for scoped values, where a Pair is returned. This change prevents unintentional "lossy" conversions to String for non-String values.
The removal of the deprecated
<SourceAttribute> element from the TemplateAttributeDefinition syntax creates a rare scenario that can break existing behavior in a small set of cases if an InputDataConnector is supplied with the
allAttributes option set. Using this syntax requires that all inputs either have the same number of values (or no values). Instead of using the
<SourceAttribute> element in conjunction with
allAttributes, simply enumerate the attributes to be sourced from the connector.
Due to a Spring bug that causes problems with configuration reloading, we have deprecated the original set of HttpClient parent beans and any inheritence from those beans in custom HttpClient configurations. We have introduced new parent beans without predefined settings that were causing the problems, and adjusted the HttpClientConfiguration documentation in various ways to refer to them.
Internal components that rely on a default HttpClient configuration now rely on a system bean definition intended only for internal use, configured as before via the services.properties file.
If you have beans that are inheriting from the deprecated versions, a warning will be issued. They will be removed from V5. This extends to several properties that were used to configure the caching versions of the clients, which will be removed.
The IdP includes a new Cross-Site Request Forgery (CSRF) Protection feature that is enabled by default for new installs. The idp.csrf.enabled property is a general on/off switch if problems are encountered or upgraders wish to enable it. The feature can also be easily incorporated in custom webflow views with minimal effort.
Mostly of interest to packagers, the installer is now built with Java and has explicit extension points. See Programming the V4 Installer.
Noteworthy New Features
- The aforementioned new AttributeRegistryConfiguration and Cross-Site Request Forgery (CSRF) Protection features
- Producing encodeable attributes directly from DataConnectors with the
- More flexible PasswordAuthnConfiguration allowing back-ends to be combined JAAS-style
- SAML proxy login flow (SAMLAuthnConfiguration) and various related features
- shibboleth.authn.SAML.* objects
- c14n/SAML2ProxyTransform c14n flow
- shibboleth.authn.Password.Validators list and related validators
- Many objects connected to AttributeRegistryConfiguration