Welcome to the third decade of the Shibboleth Project.
Various cancelled meetings, holidays, travel, and just generally getting work done have led to a lack of updates, but as of the new year we have started the process of wrapping up V4 IdP development so we can get a beta released as soon as possible. Much of the smaller items left to do are now done, and most substantive remaining issues have been pushed out to future versions, so what's left is primarily testing, finishing up work on the SAML proxy feature, and beginning to update the OIDC extension.
The installer is essentially complete at this stage, and is substantively the same as before though has been rewritten to make more extensive use of Java code in pace of ant so we can make future enhancements to it more easily by "just" writing code.
The SAML proxy feature is missing most of the necessary security code. This will hopefully start getting committed in the next week or two. It is however functionally done and documented (SAMLAuthnConfiguration) and I don't anticipate significant changes being made for this first iteration.
The biggest remaining task is prepping an upgrade of the OIDC extension for use with V4 so that it can be made available at or near the initial release. As recently announced, we have officially assumed ongoing responsibility for the code base as of last month, and have taken the first step by importing the code into our official repository. Our plan is for any required maintenance on the original V1 branch of the extension for use with IdP V3 will be carried out in the original github repository, while work on a V2 extension for use with IdP V4 has begun on the master branch in our repository.
After some consideration, our intent is to maintain most of the current configuration, XML namespaces and types, and Java packages of the original extension, with the understanding that the implementation released with IdP V5 next year would introduce a number of breaking changes to make a number of final adjustments for the long term sustainability of the code. This version is therefore a transitional release, and deployers of the OIDC extension now and with V4 should expect some additional work to upgrade to V5 later.
As part of this transition, the documentation will likely continue to be maintained in the GitHub project wiki since it will be largely the same for this release, with the updated/integrated documentation moving into our wiki for V5.
It is however, a discussion point as to whether the code will be moved into the IdP core or maintained as a separate extension for organizational purposes. If that is the decision made, we will out of necessity also take on the delivery of some kind of formalized "module" system for the automated install and update of extensions to the core software to minimize the impact of that decision. It will, in other words, not matter all that much to a deployer whether the code is separate or not, but a separation will help us design such a module feature, and potentially reduce the impact of security issues in the relatively less mature OIDC code from the rest of the code base.
Long story short, lots of work was completed over the holidays, there's still a fair bit left to do, and we're working hard to get a beta out the door as soon as possible.
Famous last words as we had to ship yet another IdP V3.4 security patch last month to address a memory leak, which required an unfortunately disruptive change to the classes supporting the External authentication flows. We don't take making breaking changes lightly, and do apologize for not fully recognizing the risk of impact to third party plugins, but it was a good chance to remind people that it's just not safe or supportable to directly copy most of our flow definitions because they include non-public classes that are subject to change. We're happy to work with people on-list to provide any missing extension points needed if they're consistent with our goals.
Work on IdP V4 has continued aggressively of late, and the installation changes are wrapping up, which should position us to ship without any major required work items left holding things up.
A new feature I just completed that's worth highlighting: it will be possible in V4 to split one's metadata configuration such that metadata filtering rules can be separated from core metadata provider configuration, allowing a separation between core federation-delivered metadata service definitions from local deployer-created filtering rules.
A few significant updates on the plans for V4:
- We're definitely not intending to ship the OIDC add-on in this release, but will work to ensure a compatible version of it is available at the same time as the release along with additional support for use of SAML metadata to manage relying parties. The final integration into the code base will be a major V5 work item.
- We are optimistic that we will be able to provide a first iteration of SAML proxy authentication support in V4, as the work has proceeded well to this point and we don't foresee any hold ups. It may not be 100% feature-complete, but will be usable certainly, so we can evaluate whatever else might be necessary.
- We continue to expect a seamless or near-seamless upgrade path to V4 from V3.
- We are hopeful of reaching a very late alpha or hopefully beta release by the TechExchange conference in December, and expect to release by early next year. We could probably make an earlier date, but the extra time will help with testing and documentation.
Some on and off work on the SP has been occurring, mostly in response to a serious crash under load caused by a bug in the newly introduced cookie session recovery feature. The fix is done, but we're probably looking at a V3.1 SP release towards the end of the year simply because of resource constraints. We would like to get a new RPM packaging process in place before doing another release, so that will take some extra time to manage. We do have packages available for CentOS 8 for the time being.
The main highlight for this month is that we're going to be issuing an IdP patch, V3.4.5, probably next week. The bug isn't a major one but is an opportunity to ship a few fixes before V4, the most significant being IDP-1491, which allows binary LDAP attributes to work again on newer Java versions where the UnboundID code has to be used. I think we will also include some optional support for addressing the SameSite cookie issue for those deploying without Apache or other software that can more easily address the problem. We have an in depth testing analysis here.
This should, once again, be the final V3 release barring any additional security issues.
I have completed work, more or less, on the revamped password validation support I described last month along with most of the basic documentation updates. We probably will be opening up the draft V4 documentation soon so people can see better what's coming, but we have more work to do on the release notes before that's going to be too helpful.
Support for a handful of new audit fields have been added, such as logging signed requests, encrypted assertions, and requested authentication context information. The default audit format for V4 is not decided at this point, I think we're open to suggestions there, but obviously it doesn't matter much for people upgrading.
An area of current study right now is to make sure all the "fail-fast" and connector validation behavior across all the various services and attribute resolver are more consistent, both with regard to defaults and to make the settings more consistent. This includes fixing the misleading errors that appear when servcies fail to start properly but are falsely attributed to the metrics support.
Most of the deprecated APIs and features have been removed from the master branches as we get closer to a release. I think the only real area of further revamp right now is probably the installer; everything else left is more around new features that we can more easily include or exclude based on the desired schedule.
We're still waiting on CentOS 8, but we're not going to spend much time on the SP again until that drops.
Work on IdP V4 is generally on track, but we're still probably 4-8 weeks from final scoping depending on what it looks like we can finish before meeting our intended scheduling goals.
Some API changes are continuing throughout the code base, most recently a large physical change with relatively little actual impact, removing a lot of generic type parameters from the API that weren't being used effectively and that made it difficult to fix a lot of compile-time warnings in the code. This will have some nominal impact on third party extensions, but nothing very extensive.
We have a tentative strategy for adding more formal CSRF prevention features to the next version, subject to more testing.
An "almost complete" work item is a redesign of the password validation code to support chaining of both JAAS and non-JAAS authentication options. We get asked a lot about how to support multiple LDAP directories at the same time, and we now have an alternative approach for that (and more general cases) that should reduce or eliminate the need for deployers to be copying or mucking with the Password login flow. Building custom validators will also be possible with simpler, more self-contained Java code that doesn't depend on Spring and doesn't require adhering to the odd interface that JAAS modules impose.
Substantial progress has been made moving CI builds to AWS.
CentOS 8 is still a work in progress, but assuming it emerges in September/October we will need to spend some time on SP backlog in preparation for a new release to coincide with doing that packaging work, particularly given that there might be a bug or two to fix to get that packaging done.
Not a long update this month, just some status updates:
- Minimal feedback to this point regarding our publicly announced Java support decisions, but what little there has been suggests the clarity is meeting immediate needs for planning.
- A few people have started to upgrade to newer Java versions and are hitting the LDAP bug. Crashes are the best case and easy to spot, but non-LDAPS deployments are seeing total pool meltdown only under load. No fun, but obvious now that we know what to look for. The one problem we don't have a fix for yet is that JNDI's support for binary attributes is not something that translates to UnboundID, so deployments with that need are probably limited to Java 8 for now. There are years to go before supported Java 8 options lost support so there's time to address this.
- Work is slowly proceeding on developing a CI capability using AWS, to relieve our current resource constraints.
- We have a number of design proposals and prototypes for adding CSRF prevention to the IdP, which is overdue. Discussion will take place on the dev list as well as a regular meeting in the near future.
- Contrary to what we had been thinking could happen, we don't expect to move off of Velocity as a view template technology in the short run, and are hoping the Velocity project will eventually take over the Spring integration code that was removed from Spring 5.
- Documentation of the new attribute registry/mapping feature is getting underway now that the configuration has mostly solidified. The V4 wiki space is still hidden for the moment to avoid confusion, but I have started to move various material out of the V3 space and into the KB area so that it can be kept independent of specific IdP releases.
- Real work on proxying hasn't really started yet, it's still a lot of vaguely formed ideas at this stage. With vacation time I don't think work on this will really get going until August at the soonest.
A final note: RHEL 8 is out, but CentOS 8 is probably still a few weeks out. I do not expect to have official SP packages for either for a while yet.
Lots of work in progress to highlight this month, with one noteworthy concrete item:
We have formalized a set of Java distributions to support to varying degrees, and an expanded set that we are (or shortly will be) testing as part of our CI processes. The full, gory details are on the Java Distributions page, and it's also forward looking to the next version, but the boiled down information for V3 is on the updated SystemRequirements page, and spells out the Java options we are recommending going forward.
As an aid to transitioning people, we are still supporting Oracle Java 8 right now, but we don't see much of a future for that, though that could change if our members indicate they feel differently. The plan is to steer people toward other options of choice, and then officially end that support with V4; the rest of the options listed are the ones that would survive that transition.
The other thing to note is that we're formalizing the concept of "partial support", which is another differentiation for Consortium members by offering direct support for particular platforms with the understanding that we may have more limited ability to quickly reproduce those environments in the unusual case that it becomes necessary. This, we feel, best reflects our generally ambivalent attitude about platforms that "work" but that we don't have constant access to or familiarity with. Today this mostly revolves around Debian, but this gives us the ability to codify it. We also expect to expand this notion to the SP side of the world shortly.
On the development front, a number of projects continue, the bulk of which center on the design foundations for proxying support that I've discussed previously. As of this month we have completed a new service design for the attribute translation functions that were formerly controlled with the Attribute Resolver, and now also includes offloading the display metadata used during attribute release consent, so that will also be possible to supply with centralized, shareable defaults provided by federations or communities. We revamped much of the system's language handling to be more correct and honor browser settings more consistently.
The final form of the new configuration, which will ultimately be possible in a variety of ways, is not fully determined, but we're converging on it. We have no plans to deprecate the use of the Attribute Resolver's AttributeEncoder, DisplayName, etc. elements either, and they remain supported. There are situations where both old and new approaches will make sense, but the new design simplifies bulk proxying of attributes. We will probably start looking at opening the V4 documentation space so that we can start to document some of this new material and get feedback.
We have prototyped the attribute proxying features, including the use of the Attribute Filter engine in the "SP mode" of filtering data inbound as well as outbound, and this is functional when applied to simple examples like data passed in from the External login flow. Today that's a good catch all for proxying, but the next steps will be to start building more intelligent proxy login flows for the various protocols. I expect this will include CAS as a good starting example, building on the work Unicon has done for years in supporting CAS authentication to the IdP. In practice, features like filtering don't matter as much with CAS, but it's a good test case for the real thing later.
The focus is still more on delivering the foundation in V4, so it's possible much of this will remain a work in progress there, but should be more fully realized by the time V5 ships.
Other work in progress includes some prototyping of new anti-CSRF features, and work on adapting SAML SP metadata for use with OIDC relying parties, much as we have done for CAS. This extends a large range of additional IdP command and control to OIDC without having to reinvent the wheel. As the summer moves along we should have an idea soon about whether the OIDC OP support will be an updated add-on or a part of V4.
There's a grab-bag of smaller topics of general interest this month.
If you missed the IAM Online last month, the slides and a recording are available at https://www.incommon.org/iamonline/
An IdP patch was released of relatively minimal criticality unless you're in the process of or planning to move to Java 11, in which case you will likely be impacted by an LDAP bug that's not been fixed to this point, and for which we have documented a workaround. The new patch simplifies the workaround by including the necessary libraries out of the box.
We've published example configuration files in the form of a downloadable starter kit for using Jetty 9.4, the most current release, designed for use on either Java 8 or 11. Jetty 9.3 is probably going to reach its end of life in the relatively near future so anybody still using it (which included myself until early this year) is certainly advised to look at an upgrade soon. I found the process very painless.
On a further related topic, please stay tuned for some imminent documentation on our plans for supporting OpenJDK 8 and 11 as the expected transition away from Oracle starts to pick up steam. As a preview, I can say with some confidence that we are leaning towards Amazon Coretto as a solution of choice for mult-platform support, particularly on Windows, plus support for Red Hat's version. Our plan is to establish a matrix of versions to continuously test that will include both officially supported versions and important alternatives (Oracle's included). This expanded testing will necessitate a revamping of our CI infrastructure, and Internet2 has made access to AWS available so we can offload this testing from our primary project server.
On the development front now, we have a number of feature branches under active development along with ongoing investigatory and design work into various topics. A new feature for generalized encoding and decoding of attribute information is nearing completion and will at least supplement, if not replace, the use of AttributeEncoders. The new design will support more configuration flexibility as we add more protocols to the system. It should reduce the need for unrelated content in the already-complex AttributeResolver configuration. Most particularly, it will allow for more natural ways to configure inbound processing of data from SAML, OIDC, or CAS proxy authentication even when that data isn't being passed along for outbound use. A side effect should be faster processing of configuration lookup based on SAML metadata EntityAttribute extensions (so-called "tags") by allowing tags to be decoded into internally optimized hashmaps instead of repeated linear searching of XML objects in memory.
Finally, note that Red Hat 8 was announced today. A CentOS 8 release is probably weeks away at best, so don't expect much on the SP packaging front there for a while.
We have an IAM Online on April 10th providing an update on the Consortium and an overview of work completed in 2018 and the roadmap covering roughly 2019 through probably the early part of 2021. This will mainly cover our expected scope of work for IdP V4 and V5 (the former already in development), with each planned for roughly a 12 month development and release cycle, faster than our major upgrades tend to be. This is a reflection of the stability of the V3 design, allowing us to make more targeted changes to the software that may necessitate a major version change but won't make upgrades a complex process. We hope simpler upgrades are the norm now rather than an exception.
Most of the basic outline of the roadmap has been mentioned in the course of previous blog entries:
- V4 will be primarily a release focused on technical debt, moving to Java 11 and Spring 5 and removing a lot of deprecated V2 features.
- It will also include new design work to allow development of new proxying features for at least OIDC and possibly some limited SAML login flows. This will allow some degree of delegation of authentication to other IdPs, much as is supported by a lot of other SAML software today. This will be a first attempt, so is likely to have rough spots, but represents the direction of a lot of future work.
- V4 may or may not end up including an integrated version of the GEANT-funded OIDC extension for V3.4. The first official release of this extension was announced recently. The more testing and feedback they get on that release, the more likely it might be considered for inclusion, but if it slips, there will be an updated version compatible with V4 in any case.
- V5 will be focused on polishing the new work integrated into V4, allowing us to adjust APIs where need be. It will probably include more complete SAML proxying support, and will almost certainly include OIDC OP support natively if it isn't incorporated into V4.
- We also have some backlogged work on improving the installation process, so it's possible that will be slotted into V5 as well.
If you have questions or input to give, please attend the IAM Online or send something to the development list.
We have an IdP V3.4 patch coming this month (hopefully) to address some bugs and add some additional warnings to some features that might be removed from V4 if we decide they're simple enough to remediate for deployers. This release will ship with defaults supporting the UnboundID LDAP provider to work around a Java bug people are just starting to run into as they move past Java 8, and by including the necessary jar files, it will be simpler for existing deployments to switch over to it as required.
I've started actively working on new designs for handling the mapping of data between the IdP, SAML, and OIDC to begin to lay the groundwork for the proxying support. There's not much to evaluate yet but the intent is to factor out the current AttributeEncoder support in the AttributeResolver and create a more general omni-directional mapping facility that can be configured by, as much as possible, "shared" definition files for common attribute mappings in different protocols, while still allowing customization where required. It isn't necessarily a given that the older AttributeEncoder approach will be deprecated, it depends on the maintenance burden of keeping it. But it will definitely be historical at best, or a more convenient way to support less common per-SP attribute encoding rules that may be more convenient to express that way.
There will also be code released in the relatively near future that implements client support for Amazon's RedShift JDBC and ODBC database drivers that works with the ECP support in Shibboleth or other SAML-compliant IdPs. There are some challenges federating database clients, but it does functionally work and Amazon approved the release of the code that references a couple of their Java interfaces, so we will be posting it in some form, though not officially supporting it. If you're interested, stay tuned.
The latest SP advisory leads off this month's update, of course. Not super serious, but deeply frustrating to continue to have issues with uncaught exceptions. This time I did a complete survey of the Xerces code base to locate all the non-inherited exception types and added catches for all of them to the outer stack frame that's supposed to catch them. Hopefully that will hold for longer than a couple of months this time. Worth noting for those running very critical systems and that don't want to implement process monitoring, you can add a
catchAll property to the OutOfProcess element in the configuration to force a trap of all exceptions raised, though that can have destablizing effects if errors occur that leave the system in an unknown state.
Work on the next generation of the IdP continues amongst our recently expanded team, and we'll be firming up our scope for this release in time for an IAM Online webinar next month.
There's still not much in the way of clarity around open Java 11 sourcing in a few months time when Oracle drops out, but we're at least putting the code through its paces now, and I've been running in production at OSU with Red Hat's version for a month or so. It seems likely that their version may be a logical choice for a lot of people, particularly on RHEL/CentOS. I've also been running successfully with UnboundID for LDAP and it's very easy to swap that in to avoid the bug I noted last month.
We have made significant progress on the conversion of Date/Time classes to Java's new (since 8) Time libraries, and this includes revising many of our public APIs to make use of the Instant and Duration Java types to better express intent. This will create some compatibility issues for extensions, but nothing too time consuming to fix in most cases, and we don't expect to see major impact for deployers outside of a couple of features. This is an important modernization step for the code base.
Work continues on the removal of deprecated code and features, which is important to complete before we begin work on new features. Parallel work streams continue on some of the items I highlighted last month, and we expect to see the final release of the GEANT OIDC extension for V3 this month so we can start evaluating the best timeline for incorporating that work.
The next project for me is probably going to be a revamping of the internal handling of EntityAttributes in metadata, to implement more efficient caching and matching data structures. So much of the system is tag-driven that it's important to optimize this and get away from the repeated linear searches we're doing and replace them with hashed/map lookups.
The SP and legacy IdP branch have stablized so V4 development has started in earnest. Deprecated code is being actively removed from all the projects and that work should be mostly complete this month. We've completed two major development tasks: moving the projects up to Spring 5 and Java 11, and replacing the Guava Predicate/Function interfaces throughout the code base with the standard Java 8 functional interfaces.
V4 will be built with and require Java 11, but we're still evaluating exactly what versions of Java 11 to officially support or recommend. There may or may not be reliably "pure" distributions of "free for use" Java 11 after September, and we expect to end up officially testing and certifying a (very) limited number of possible implementations with the rest left to the "probably works" category that OpenJDK used to live in. All of this is subject to change, and there will probably be some additional information available during Spring or Summer. Of course, feedback from members will have to be taken into consideration.
Java 8+ includes a number of new standard features that have replaced, or will replace, various proprietary libraries. The first conversion done was the Predicate/Function mechanism that is heavily used across the code, but also by deployers in various ways to customize behavior. For the most part, existing scripts and plugins will work compatibly after the conversion, with some small issues that should lead to warnings in V4 before causing any breakage in future versions.
We will also be migrating the Date/Time classes and interfaces to Java's standard classes, and this is likely to lead to a small number of incompatibilities, but the details will be clearer once that work is done.
Other open investigations and projects underway:
- Evaluating Velocity alternatives for possible use as a replacement, principally FreeMarker since it remains officially supported by Spring. Velocity will be supported for the time being, but we have to maintain that supporting code so moving away from it is a possibility, though not an urgent need.
- Work continues to clean up an OIDC authentication module, which will lead into more significant design work on proxy features over the summer.
- Discussions about how/when/who to incorporate the V3 OIDC extension. It is not officially targeted for V4 but remains a possibility.
A revised dependency management process is being worked on to avoid the need to privately host all of the dependencies of the codebase by pinning artifacts with PGP keys and/or checksums, and extending a Maven plugin to verify our dependencies at build-time. We continue to seek the best way to keep the build process safe from the attacks that standard Maven projects leave themselves open to and that have caused huge security vulnerabilities in other non-Java platforms, demonstrating our paranoia was well founded.
Finally, a regression was identified in the Java 9+ JNDI LDAP provider that causes connection leaks and/or crashes. So far the regression hasn't appeared in Java 8 but does affect all later versions. The bug has been reported to Oracle but so far the only workaround appears to be switching to an alternate LDAP provider. The ldaptive library has been heavily used by its authors with the UnboundID LDAP client, so we are tenatively planning to move to it as a V4 default, as JNDI has been a frequent source of bugs. We also have documentation prepared describing how to do this with V3. We may, if we decided it's warranted and safe, decide to ship a V3 patch with this change, but it depends on what Oracle does with the bug in the next couple of months.
We were unusually busy in December handling a few different security issues and patches and a few work items.
The SP received a long gestating update to add support for OpenSSL 1.1.1 that turned into a security release at the last minute due to a regression introduced in V3. We haven't done significant testing yet, but these OpenSSL and curl updates theoretically add support for TLS 1.3 for the first time. I've done a bit of investigation into the possibility of HTTP/2 support and it turns out that curl doesn't really support it natively, it's only supported through the addition of another library. I haven't seriously entertained the thought of adding that to the mix; I don't think there's sufficient benefit from doing so at this point. The advantages of HTTP/2 don't really apply to the SP's use cases for outbound HTTP traffic, and technically it isn't even allowed by SAML's SOAP 1.1 binding. The TLS 1.3 support is more relevant long term.
The IdP has continued to receive patches due to a combination of security issues and regressions, most of them due to a late decision to deprecate some syntax we probably should have left alone, but c'est la vie. Another patch is coming this week to fix two more regressions identified by deployers.
I completed most of the planned Logout enhancements, and the changes are largely if not completely backward-compatible with the existing view templates, which was unexpected. Since these are V4 features, we don't have updated documentation yet, but there are essentially just some new messages and a property or two that control the sequence of events and the choices presented to the user during logout, and it's now possible to preempt the entire operation if you want to give the user that option, which was, oddly, requested by a number of people.
We haven't seriously opened up the V4 branches for larger work yet, but that's imminent, probably starting with the move to Java 11 as our development baseline. A feature branch was pushed containing an OIDC login flow and I'll probably merge a version of that pretty soon. I was greatly disappointed, though not surprised, to learn how little actual support for OIDC exists among the major social identity sources but we're not going to get into the business of supporting one-off protocols in our core code. I would anticipate us producing unsupported examples as add-ons to demonstrate the non-standard ones that are of interest.
I don't have a real update regarding the broader OIDC support, but if it's not already been finalized, I expect the add-on will be getting a 1.0 release soon.
Mostly a follow up from November's update...
We're ready to ship a small SP patch as soon as curl ships this month, which should be this week, so expect the patch late this week and I'll be circulating advance notice of that to announce today along with the early announcement that we'll be releasing a second IdP 3.4 patch update probably next week to address a CAS security issue and some miscellaneous cleanup work around warnings and deprecations.
After some discussion amongst the team and the Board, most of the consensus settled around aiming to produce a relatively scaled back V4 release by the end of 2019 rather than start a multi-year effort again. The exact scope of that release will depend on what priorities people bring to the table and on what we can manage to finish in that time frame.
I've started work on finishing as much of the undone logout enhancements as possible by the end of the year, and progress has been pretty good. SOAP-based logout is now working and I've started on the UI enhancements requested to provide more control over the sequence of events during logout. The former took more time than expected and the latter looks like it will probably take less.
We're also working on pulling in a third party extension to support OIDC authentication to the IdP as the very early steps of looking at proxying functionality.
Since last month we've released IdP V3.4.0 and the first patch for it once the initial round of bug reports came in. We have one or two remaining annoyances around deprecation warnings outstanding and the ReleaseNotes are being kept up to date to provide the most accurate information we have on what to watch out for. We can't do patches every week obviously but as these things accumulate there will be additional patches coming to get things cleaned up. Overall the release seems to be solid and I've had it in production for a couple of weeks now.
The GEANT OIDC extensions is now in beta, with an improved installation process, so please check it out and provide feedback if you're interested in that.
In parallel with the occasional maintenance commit, we've branched the source tree so the master branches are open for development on the V4 code base. All commits are now intended to be on those master branches, with only specific authorized commits backported to the 3.4 maintenance branch.
The master branches have been moved to a new parent POM based on Java 8 and Spring 5 as the first "step forward" and all but a couple of tests are passing and the IdP seems to be functional, so that's a good sign that we won't have too much remediation needed. Eventually this will be moved to Java 11, but right now the Maven toolchain isn't stable on Java 11, so we're on Java 8 for now.
We also have a buildable artifact containing a curated and well-organized Jetty 9.4 configuration that works with the IdP with minimal adjustments and will probably make that available in some capacity, with an eye towards including it in V4 to help bootstrap new installs (assuming 9.4 is a current release at that point in time).
What isn't really decided at this stage is what the schedule and scope of V4 will be, with the most likely options being a 12 month development cycle with minimal feature additions vs. a longer cycle of perhaps 24 months to accomodate more new features. Both options are possible and make some sense, and hopefully we'll be able to engage people on this question once we have some initial discussions with the Board about it. Regardless of the decision, we don't expect the upgrade process to V4 to be onerous, though we can definitely promise there won't be a supported path to upgrade from V2 directly to V4.
On the SP front, we are in the process of getting a patch out the door, but the latest curl release (7.62.0) was a big one that has a number of regressions, so right now we're probably waiting another month to include the next version rather than ship this one, but that could change depending on bugs reported and the urgency to get an update done.
As I draft this, IdP V3.4 is tagged and will be announced tomorrow, just shy of two years (!) since the last significant feature release. The last month saw a large number of commits from the whole team to finish up late-arriving features and complete internal testing for the release. Pretty much everything originally planned for this release made it in (along with a lot more) except for a significant redesign of the logout functionality, in keeping with the project's long tradition of never getting logout work done on time.
As I outlined before, the list of new features is extensive. This will be the last feature update before V4 when we make the move to Java 11 and Spring 5. This release provides a good window to freshen and optimize configurations and get rid of legacy options and cruft that will be removed from V4.
Once this release is complete and we take a breath or two, work will begin soon on initial scoping of V4. I would like to work on some opportunities for direct interaction with the Consortium Members on what people would like to see from that work, aside from the long list of issues we already have open. One obvious task for the Board will be to have the necessary discussions on the viability of incorporating the OIDC extension being developed for V3.4, which is mostly a question of long term code ownership and maintenance.
Activity on the SP bug front is still very low, but we have a couple of possible issues to get cleaned up and then we can get a new build that supports OpenSSL 1.1.1 out the door in time for the end of the year. This may be an opportune time, with another Java developer coming on board to help out on the IdP, for me to spend a bit of time looking at the viability of an NGINX SP module.
I'll be at Internet2 Tech Exchange and ACAMP next week in Orlando, so I'll see many of you there.
The last month has been spent ramping IdP development back up and preparing for a release of V3.4 in October. The number of bug fixes and features is quite large (pushing 150+), so this is a pretty big new release that is intended to bridge to V4 as the next big update. There are a couple of things that haven't made the cut, primarily redesigning the logout behavior in response to a lot of feedback. Both time constraints and the need to maintain compatibility made it appealing to delay that work until V4 when we have more freedom to change the UI in ways that might require a small bit of rework for deployers instead of guaranteeing drop-in compatibility.
In the meantime, there should be enough to keep people busy with V3.4 for a while:
- A nice HTTPConnector for web services integrations (I built it so I'm biased but I've been using it for Grouper and AWS integration projects and it's been really, really convenient to use)
- Built-in wiring and convenience functions that make it possible to drive a ton of configuration options using metadata "tags"
- Non-browser Duo AuthAPI support
- Improvements to the deployability of the DynamicHTTPMetadataProvider and LocalDynamicMetadataProvider plugins
- Improvements for embedding scriptlets in a variety of configuration scenarios
- A new "attended startup" mode that prevents on-disk access to an unlocked or trivially encrypted private key
- The ability to provision CAS services using SAML metadata for consistency and to support the new metadata-driven configuration mechanisms
- Added support for SAML proxying constructs to detect/react to proxied SPs and to express upstream proxying to SPs
- Greatly enhanced context-check interceptor that can handle multiple scenarios at the same time
- A new impersonation interceptor that supports advanced debugging or testing scenarios
- Some small features to support the drop-in addition of the under-development GEANT OIDC extension
And more, but that's plenty for now.
We'll be wrapping up work this month, testing over the next couple of weeks, and should be freezing around the end of the month with a release in time for TechExchange in Orlando.
We're also working on a more modular and minimal configuration for Jetty 9.4 that will form the basis of not only the Windows "quick install" package but a maintained/curated Jetty configuration to make deployment simpler on other platforms for people without experience with Jetty. We're not planning to fully "embed" Jetty because that's nearly impossible to maintain over time for mature use as a production web server, but it should be a good starting point for most people.
On the SP front, things have been pretty quiet, so one presumes either upgrades are going reasonably smoothly or people are stuck on software with some known security bugs, but I'll choose to assume the former. There will most likely be a patch update later this year if only because OpenSSL 1.1.1 is now out, and 1.1.0 will be EOL soon.
Both OpenSSL and Jetty are introducing the first iterations of TLS 1.3, which is different enough from TLS 1.2 that it will require some time to digest (pun intended). Hopefully if there are some adjustments needed the less frequent use of the back-channel in the SP should soften the impact.