We have completed migration of most of the project infrastructure to AWS, which will be more cost effective and simpler to manage. All services have been moved and are fully functional except for Jenkins, which is being debugged to get all the jobs working again, after which we're hoping for some improvements to the frequent networking problems we have encountered in the past. Total time spent on this is over 75 hours and will hopefully not need to be repeated at quite this scale again. Core services are running on CentOS 8 now to extend the useful life of the deployment. We should be shutting down the old system within the next week. We have not deployed our services with region or availability zone redundancy, as that would greatly increase costs and not be an apples to apples comparison (we weren't redundant before).
The SP V3.1 release was completed, with no significant issues arising to this point. The next major hurdle is tracking the changes coming in OpenSSL 3.0, which is available in alpha form. OpenSSL has about the worst API documentation I know of (as in, there isn't any), so there's a significant risk of breakage to a lot of important code in the SP and its dependencies, so the work involved remains to be seen. I would expect this is going to be the sort of change that may result in a 4.0 update because of the impact.
I'm frankly expecting we may lose all ability to support use of certificate authentication in the SP's back channel support, but that's speculation at this stage, it's just something to watch. Moving to port 443 and signed messages is the best way to inoculate against that risk, for the few deployments still in need of SOAP support. We'll get a bug created to track this work.
We have made significant progress on drafting requirements and starting preliminary work on a new plugin design for the installer to enable more factoring of the code base. This is a big deal and it's not being undertaken lightly, but we don't see how to accomplish what we need with any off-the-shelf solution that would accomodate the freedom of deployment models we have now (i.e., we could use new tools and then write off anybody who doesn't use those tools, and we just don't believe in that approach for the core software). Spring, and particularly Web Flow, also more or less preclude some of the approaches we might otherwise consider because the architecture doesn't allow for the sort of class-loader isolation that something like OSGi would require.
Note that we do not plan to move to any sort of piecemeal "patch" approach to delivering bug fixes, as that's a totally chaotic and unmanageable deployment model that undermines the careful versioning policy we adhere to. But the model will accomodate detection of compatibility problems when upgrades are done, even retroactively (the compatibility rules themselves are expected to be remotely obtained). Once the basic framework exists, we probably will find new uses for it, but the focus is on delivering more manageable installation of complex add-ons.
We will be initially designing a number of deliverables to rely on this approach:
- Non-trivial storage plugins
- New (and old) scripting alternatives
- Official OIDC support, both inbound and outbound
- Duo integration via OIDC, eventually replacing the original integration
In the future, most new work that is expected to be of more limited adoption or that requires substantive new dependencies will probably take this form. We welcome interested parties to provide feedback on the dev list. Initial work on this will hopefully start showing up in V4.1.
I would also like to call attention to the fact that yet another IdP-of-last-resort for our services, UnitedID, has ceased operating. SUNET has kindly agreed to make their open IdP (eduID) available more widely. If you have existing bugs or pages you're tracking and need to rename a Jira or Confluence account name that was formerly issued by UnitedID just drop a note to our contact address and we can rename it for you.
Project activity can best be described as a bit unfocused right now, as a somewhat indirect fallout of the pandemic, not so much due to the change in work habits but because it's self-evidently slowed any real uptake of the IdP 4.0.0 upgrade. More typically we'd be fielding a decent range of bug reports and working on a patch but with the paucity of testing so far we haven't seen much, so we're in a holding pattern on exactly what the next release needs to be. We do have at least some positive feedback that suggests the release is solid, at least for upgrades.
We'll probably be forced to open up the master branch for new work fairly soon to start work on V4.1. There's a small set of deprecation warnings that need to be added so that at least one release is done with them prior to V5, but the list of deprecated material is much shorter than the V3/4 transition was. We have no ETA for 4.1, so we do not advise waiting for that update before making the jump.
After the IdP release and with the relative breather, work is complete on the SP upgrade and posted beta packages, with testing as usual being minimal to non-existent (which is why we don't bother much with them). After some additional testing on shibboleth.net this week, the release will most likely be done next week. Apparently, there's not much immediate pressure to get any SameSite assistance out the door because Google kindly decided to exercise their total dictatorial control of the web and defer wider dissemination of their Chrome change. In any case, the ReleaseNotes for the upgrade and documentation on the changes are available.
We are also actively engaged in reviewing a lot of older issues and finding small items to close out (fixed or not) before tackling some bigger work. We have also been working to fix issues with the project javadocs caused by a range of bugs in Java, javadoc, and the Maven javadoc plugin. The javadoc search feature works again and most of the expected links between projects and dependencies are working again, with the exception of the Java EE APIs.
Most of the summer will probably be taken up by further work on advanced authentication features, designing a plugin installation and upgrade architecture, and adapting the OIDC extension into its final form. Some of this work will probably ship ahead of V5, but the bulk of it will be targeted for that release next year.
Not a long update this month, as the activity is just an extension of last month's update. Work has been hard and heavy though.
Two beta releases of the IdP were completed, with a few serious bugs found and fixed, enough that we are comfortable with an official release being cranked as I write this, with the announcement due tomorrow. V4 is the product of over 15 months of work primarily focused on technical refresh and maintaining the maximum degree of compatibility possible as we move forward 4! Java versions and a full Spring version. The OIDC extension should be finalized within a few days of the IdP release.
We believe we have achieved the goal of a true major upgrade that feels more like a patch. Just be aware that the less you notice, the more work went into making that possible. The ReleaseNotes are extensive and cover the highlights. We will be focused on feedback and bug fixing over the next couple of months so a patch or two should be expected.
On the SP front, work has continued on a 3.1 release that has accumulated a decent list of fixes and features. Work on SameSite and correlation support is complete and the remaining tasks consist of fairly minor enhancements and some documentation cleanup that should make a late March or early April release feasible.
By early summer, work will be underway on a roadmap for the IdP that will take us through V5 and full incorporation of the OIDC extension into the project. We expect that V5 in 2021 will see minimal changes for deployers except for the transition of the OIDC module from third-party to fully-supported extension.
It's been a busy month for the project primarily to get the expected beta release of the Identity Provider V4.0 done. Testing to this point has not been extensive, but the early returns from within the team have been positive and the final release is expected by early March at the latest. Upgrades should be very easy for the majority of deployers.
The lifecycle of support for V3 is essentially a foregone conclusion at this point because Spring 4 is end-of-life at the end of 2020. We don't have any compelling reason to abandon security support for V3 any sooner than that, so V3 will be supported through the remainder of this calendar year.
I undertook a mini-project to blitz a port of the OIDC extension and we are on track to produce a compatible extension along with the IdP release. After a short discussion, we agreed to forego significant breaking changes to this code for now and leave things largely as they have been for this release.
For clarity, since this is a period of transition:
- The code in GitHub is the V3-compatible extension. Issues with it can continue to be reported via GitHub if desired.
- The V4-compatible extension is largely the same but has been moved into our Git repository and will be distributed via our download site. The documentation remains in the GitHub wiki, but we are using our Jira instance for bug reporting and patch submission.
- The V5 extension when it comes will include a number of breaking changes so that the code packages can be renamed and configuration layout changed, and the documentation will be moved into our wiki. At that point we will be in a position to fully support the extension to minimize breakage in the same way we have done for the rest of the feature set.
In parallel with the beta effort, work has begun on the next SP update, V3.1. This feature upgrade is primarily focused on addressing XSRF in two areas:
- Providing a fix for the SameSite issue in a Safari-compatible way
- Adding request/response correlation along with an option to block unsolicited SSO
The SameSite work is complete and will explicitly designate a subset of SP cookies as SameSite=None, with an optional setting to add a second non-SameSite cookie to support broken browsers for hopefully a temporary period of time. It will also be optionally possible to support cross-site use of SP sessions, when XSRF is a "feature" and not a bug.
The correlation work is underway, but is being carefully implemented to maintain compatibility so it will be an optional feature for security conscious deployers to turn on.
There are a few other odds and ends to complete for the SP release, but we're moving aggressively to complete that work as soon as possible so it's quite likely March will see both IdP and SP releases.
We have a significant queue of work items for the IdP once V4 ships, primarily authentication-related, so the roadmap should be getting cleaned up and clarified once we can take stock of things.
Welcome to the third decade of the Shibboleth Project.
Various cancelled meetings, holidays, travel, and just generally getting work done have led to a lack of updates, but as of the new year we have started the process of wrapping up V4 IdP development so we can get a beta released as soon as possible. Much of the smaller items left to do are now done, and most substantive remaining issues have been pushed out to future versions, so what's left is primarily testing, finishing up work on the SAML proxy feature, and beginning to update the OIDC extension.
The installer is essentially complete at this stage, and is substantively the same as before though has been rewritten to make more extensive use of Java code in pace of ant so we can make future enhancements to it more easily by "just" writing code.
The SAML proxy feature is missing most of the necessary security code. This will hopefully start getting committed in the next week or two. It is however functionally done and documented (SAMLAuthnConfiguration) and I don't anticipate significant changes being made for this first iteration.
The biggest remaining task is prepping an upgrade of the OIDC extension for use with V4 so that it can be made available at or near the initial release. As recently announced, we have officially assumed ongoing responsibility for the code base as of last month, and have taken the first step by importing the code into our official repository. Our plan is for any required maintenance on the original V1 branch of the extension for use with IdP V3 will be carried out in the original github repository, while work on a V2 extension for use with IdP V4 has begun on the master branch in our repository.
After some consideration, our intent is to maintain most of the current configuration, XML namespaces and types, and Java packages of the original extension, with the understanding that the implementation released with IdP V5 next year would introduce a number of breaking changes to make a number of final adjustments for the long term sustainability of the code. This version is therefore a transitional release, and deployers of the OIDC extension now and with V4 should expect some additional work to upgrade to V5 later.
As part of this transition, the documentation will likely continue to be maintained in the GitHub project wiki since it will be largely the same for this release, with the updated/integrated documentation moving into our wiki for V5.
It is however, a discussion point as to whether the code will be moved into the IdP core or maintained as a separate extension for organizational purposes. If that is the decision made, we will out of necessity also take on the delivery of some kind of formalized "module" system for the automated install and update of extensions to the core software to minimize the impact of that decision. It will, in other words, not matter all that much to a deployer whether the code is separate or not, but a separation will help us design such a module feature, and potentially reduce the impact of security issues in the relatively less mature OIDC code from the rest of the code base.
Long story short, lots of work was completed over the holidays, there's still a fair bit left to do, and we're working hard to get a beta out the door as soon as possible.
Famous last words as we had to ship yet another IdP V3.4 security patch last month to address a memory leak, which required an unfortunately disruptive change to the classes supporting the External authentication flows. We don't take making breaking changes lightly, and do apologize for not fully recognizing the risk of impact to third party plugins, but it was a good chance to remind people that it's just not safe or supportable to directly copy most of our flow definitions because they include non-public classes that are subject to change. We're happy to work with people on-list to provide any missing extension points needed if they're consistent with our goals.
Work on IdP V4 has continued aggressively of late, and the installation changes are wrapping up, which should position us to ship without any major required work items left holding things up.
A new feature I just completed that's worth highlighting: it will be possible in V4 to split one's metadata configuration such that metadata filtering rules can be separated from core metadata provider configuration, allowing a separation between core federation-delivered metadata service definitions from local deployer-created filtering rules.
A few significant updates on the plans for V4:
- We're definitely not intending to ship the OIDC add-on in this release, but will work to ensure a compatible version of it is available at the same time as the release along with additional support for use of SAML metadata to manage relying parties. The final integration into the code base will be a major V5 work item.
- We are optimistic that we will be able to provide a first iteration of SAML proxy authentication support in V4, as the work has proceeded well to this point and we don't foresee any hold ups. It may not be 100% feature-complete, but will be usable certainly, so we can evaluate whatever else might be necessary.
- We continue to expect a seamless or near-seamless upgrade path to V4 from V3.
- We are hopeful of reaching a very late alpha or hopefully beta release by the TechExchange conference in December, and expect to release by early next year. We could probably make an earlier date, but the extra time will help with testing and documentation.
Some on and off work on the SP has been occurring, mostly in response to a serious crash under load caused by a bug in the newly introduced cookie session recovery feature. The fix is done, but we're probably looking at a V3.1 SP release towards the end of the year simply because of resource constraints. We would like to get a new RPM packaging process in place before doing another release, so that will take some extra time to manage. We do have packages available for CentOS 8 for the time being.
The main highlight for this month is that we're going to be issuing an IdP patch, V3.4.5, probably next week. The bug isn't a major one but is an opportunity to ship a few fixes before V4, the most significant being IDP-1491, which allows binary LDAP attributes to work again on newer Java versions where the UnboundID code has to be used. I think we will also include some optional support for addressing the SameSite cookie issue for those deploying without Apache or other software that can more easily address the problem. We have an in depth testing analysis here.
This should, once again, be the final V3 release barring any additional security issues.
I have completed work, more or less, on the revamped password validation support I described last month along with most of the basic documentation updates. We probably will be opening up the draft V4 documentation soon so people can see better what's coming, but we have more work to do on the release notes before that's going to be too helpful.
Support for a handful of new audit fields have been added, such as logging signed requests, encrypted assertions, and requested authentication context information. The default audit format for V4 is not decided at this point, I think we're open to suggestions there, but obviously it doesn't matter much for people upgrading.
An area of current study right now is to make sure all the "fail-fast" and connector validation behavior across all the various services and attribute resolver are more consistent, both with regard to defaults and to make the settings more consistent. This includes fixing the misleading errors that appear when servcies fail to start properly but are falsely attributed to the metrics support.
Most of the deprecated APIs and features have been removed from the master branches as we get closer to a release. I think the only real area of further revamp right now is probably the installer; everything else left is more around new features that we can more easily include or exclude based on the desired schedule.
We're still waiting on CentOS 8, but we're not going to spend much time on the SP again until that drops.
Work on IdP V4 is generally on track, but we're still probably 4-8 weeks from final scoping depending on what it looks like we can finish before meeting our intended scheduling goals.
Some API changes are continuing throughout the code base, most recently a large physical change with relatively little actual impact, removing a lot of generic type parameters from the API that weren't being used effectively and that made it difficult to fix a lot of compile-time warnings in the code. This will have some nominal impact on third party extensions, but nothing very extensive.
We have a tentative strategy for adding more formal CSRF prevention features to the next version, subject to more testing.
An "almost complete" work item is a redesign of the password validation code to support chaining of both JAAS and non-JAAS authentication options. We get asked a lot about how to support multiple LDAP directories at the same time, and we now have an alternative approach for that (and more general cases) that should reduce or eliminate the need for deployers to be copying or mucking with the Password login flow. Building custom validators will also be possible with simpler, more self-contained Java code that doesn't depend on Spring and doesn't require adhering to the odd interface that JAAS modules impose.
Substantial progress has been made moving CI builds to AWS.
CentOS 8 is still a work in progress, but assuming it emerges in September/October we will need to spend some time on SP backlog in preparation for a new release to coincide with doing that packaging work, particularly given that there might be a bug or two to fix to get that packaging done.
Not a long update this month, just some status updates:
- Minimal feedback to this point regarding our publicly announced Java support decisions, but what little there has been suggests the clarity is meeting immediate needs for planning.
- A few people have started to upgrade to newer Java versions and are hitting the LDAP bug. Crashes are the best case and easy to spot, but non-LDAPS deployments are seeing total pool meltdown only under load. No fun, but obvious now that we know what to look for. The one problem we don't have a fix for yet is that JNDI's support for binary attributes is not something that translates to UnboundID, so deployments with that need are probably limited to Java 8 for now. There are years to go before supported Java 8 options lost support so there's time to address this.
- Work is slowly proceeding on developing a CI capability using AWS, to relieve our current resource constraints.
- We have a number of design proposals and prototypes for adding CSRF prevention to the IdP, which is overdue. Discussion will take place on the dev list as well as a regular meeting in the near future.
- Contrary to what we had been thinking could happen, we don't expect to move off of Velocity as a view template technology in the short run, and are hoping the Velocity project will eventually take over the Spring integration code that was removed from Spring 5.
- Documentation of the new attribute registry/mapping feature is getting underway now that the configuration has mostly solidified. The V4 wiki space is still hidden for the moment to avoid confusion, but I have started to move various material out of the V3 space and into the KB area so that it can be kept independent of specific IdP releases.
- Real work on proxying hasn't really started yet, it's still a lot of vaguely formed ideas at this stage. With vacation time I don't think work on this will really get going until August at the soonest.
A final note: RHEL 8 is out, but CentOS 8 is probably still a few weeks out. I do not expect to have official SP packages for either for a while yet.
Lots of work in progress to highlight this month, with one noteworthy concrete item:
We have formalized a set of Java distributions to support to varying degrees, and an expanded set that we are (or shortly will be) testing as part of our CI processes. The full, gory details are on the Java Distributions page, and it's also forward looking to the next version, but the boiled down information for V3 is on the updated SystemRequirements page, and spells out the Java options we are recommending going forward.
As an aid to transitioning people, we are still supporting Oracle Java 8 right now, but we don't see much of a future for that, though that could change if our members indicate they feel differently. The plan is to steer people toward other options of choice, and then officially end that support with V4; the rest of the options listed are the ones that would survive that transition.
The other thing to note is that we're formalizing the concept of "partial support", which is another differentiation for Consortium members by offering direct support for particular platforms with the understanding that we may have more limited ability to quickly reproduce those environments in the unusual case that it becomes necessary. This, we feel, best reflects our generally ambivalent attitude about platforms that "work" but that we don't have constant access to or familiarity with. Today this mostly revolves around Debian, but this gives us the ability to codify it. We also expect to expand this notion to the SP side of the world shortly.
On the development front, a number of projects continue, the bulk of which center on the design foundations for proxying support that I've discussed previously. As of this month we have completed a new service design for the attribute translation functions that were formerly controlled with the Attribute Resolver, and now also includes offloading the display metadata used during attribute release consent, so that will also be possible to supply with centralized, shareable defaults provided by federations or communities. We revamped much of the system's language handling to be more correct and honor browser settings more consistently.
The final form of the new configuration, which will ultimately be possible in a variety of ways, is not fully determined, but we're converging on it. We have no plans to deprecate the use of the Attribute Resolver's AttributeEncoder, DisplayName, etc. elements either, and they remain supported. There are situations where both old and new approaches will make sense, but the new design simplifies bulk proxying of attributes. We will probably start looking at opening the V4 documentation space so that we can start to document some of this new material and get feedback.
We have prototyped the attribute proxying features, including the use of the Attribute Filter engine in the "SP mode" of filtering data inbound as well as outbound, and this is functional when applied to simple examples like data passed in from the External login flow. Today that's a good catch all for proxying, but the next steps will be to start building more intelligent proxy login flows for the various protocols. I expect this will include CAS as a good starting example, building on the work Unicon has done for years in supporting CAS authentication to the IdP. In practice, features like filtering don't matter as much with CAS, but it's a good test case for the real thing later.
The focus is still more on delivering the foundation in V4, so it's possible much of this will remain a work in progress there, but should be more fully realized by the time V5 ships.
Other work in progress includes some prototyping of new anti-CSRF features, and work on adapting SAML SP metadata for use with OIDC relying parties, much as we have done for CAS. This extends a large range of additional IdP command and control to OIDC without having to reinvent the wheel. As the summer moves along we should have an idea soon about whether the OIDC OP support will be an updated add-on or a part of V4.
There's a grab-bag of smaller topics of general interest this month.
If you missed the IAM Online last month, the slides and a recording are available at https://www.incommon.org/iamonline/
An IdP patch was released of relatively minimal criticality unless you're in the process of or planning to move to Java 11, in which case you will likely be impacted by an LDAP bug that's not been fixed to this point, and for which we have documented a workaround. The new patch simplifies the workaround by including the necessary libraries out of the box.
We've published example configuration files in the form of a downloadable starter kit for using Jetty 9.4, the most current release, designed for use on either Java 8 or 11. Jetty 9.3 is probably going to reach its end of life in the relatively near future so anybody still using it (which included myself until early this year) is certainly advised to look at an upgrade soon. I found the process very painless.
On a further related topic, please stay tuned for some imminent documentation on our plans for supporting OpenJDK 8 and 11 as the expected transition away from Oracle starts to pick up steam. As a preview, I can say with some confidence that we are leaning towards Amazon Coretto as a solution of choice for mult-platform support, particularly on Windows, plus support for Red Hat's version. Our plan is to establish a matrix of versions to continuously test that will include both officially supported versions and important alternatives (Oracle's included). This expanded testing will necessitate a revamping of our CI infrastructure, and Internet2 has made access to AWS available so we can offload this testing from our primary project server.
On the development front now, we have a number of feature branches under active development along with ongoing investigatory and design work into various topics. A new feature for generalized encoding and decoding of attribute information is nearing completion and will at least supplement, if not replace, the use of AttributeEncoders. The new design will support more configuration flexibility as we add more protocols to the system. It should reduce the need for unrelated content in the already-complex AttributeResolver configuration. Most particularly, it will allow for more natural ways to configure inbound processing of data from SAML, OIDC, or CAS proxy authentication even when that data isn't being passed along for outbound use. A side effect should be faster processing of configuration lookup based on SAML metadata EntityAttribute extensions (so-called "tags") by allowing tags to be decoded into internally optimized hashmaps instead of repeated linear searching of XML objects in memory.
Finally, note that Red Hat 8 was announced today. A CentOS 8 release is probably weeks away at best, so don't expect much on the SP packaging front there for a while.
We have an IAM Online on April 10th providing an update on the Consortium and an overview of work completed in 2018 and the roadmap covering roughly 2019 through probably the early part of 2021. This will mainly cover our expected scope of work for IdP V4 and V5 (the former already in development), with each planned for roughly a 12 month development and release cycle, faster than our major upgrades tend to be. This is a reflection of the stability of the V3 design, allowing us to make more targeted changes to the software that may necessitate a major version change but won't make upgrades a complex process. We hope simpler upgrades are the norm now rather than an exception.
Most of the basic outline of the roadmap has been mentioned in the course of previous blog entries:
- V4 will be primarily a release focused on technical debt, moving to Java 11 and Spring 5 and removing a lot of deprecated V2 features.
- It will also include new design work to allow development of new proxying features for at least OIDC and possibly some limited SAML login flows. This will allow some degree of delegation of authentication to other IdPs, much as is supported by a lot of other SAML software today. This will be a first attempt, so is likely to have rough spots, but represents the direction of a lot of future work.
- V4 may or may not end up including an integrated version of the GEANT-funded OIDC extension for V3.4. The first official release of this extension was announced recently. The more testing and feedback they get on that release, the more likely it might be considered for inclusion, but if it slips, there will be an updated version compatible with V4 in any case.
- V5 will be focused on polishing the new work integrated into V4, allowing us to adjust APIs where need be. It will probably include more complete SAML proxying support, and will almost certainly include OIDC OP support natively if it isn't incorporated into V4.
- We also have some backlogged work on improving the installation process, so it's possible that will be slotted into V5 as well.
If you have questions or input to give, please attend the IAM Online or send something to the development list.
We have an IdP V3.4 patch coming this month (hopefully) to address some bugs and add some additional warnings to some features that might be removed from V4 if we decide they're simple enough to remediate for deployers. This release will ship with defaults supporting the UnboundID LDAP provider to work around a Java bug people are just starting to run into as they move past Java 8, and by including the necessary jar files, it will be simpler for existing deployments to switch over to it as required.
I've started actively working on new designs for handling the mapping of data between the IdP, SAML, and OIDC to begin to lay the groundwork for the proxying support. There's not much to evaluate yet but the intent is to factor out the current AttributeEncoder support in the AttributeResolver and create a more general omni-directional mapping facility that can be configured by, as much as possible, "shared" definition files for common attribute mappings in different protocols, while still allowing customization where required. It isn't necessarily a given that the older AttributeEncoder approach will be deprecated, it depends on the maintenance burden of keeping it. But it will definitely be historical at best, or a more convenient way to support less common per-SP attribute encoding rules that may be more convenient to express that way.
There will also be code released in the relatively near future that implements client support for Amazon's RedShift JDBC and ODBC database drivers that works with the ECP support in Shibboleth or other SAML-compliant IdPs. There are some challenges federating database clients, but it does functionally work and Amazon approved the release of the code that references a couple of their Java interfaces, so we will be posting it in some form, though not officially supporting it. If you're interested, stay tuned.
The latest SP advisory leads off this month's update, of course. Not super serious, but deeply frustrating to continue to have issues with uncaught exceptions. This time I did a complete survey of the Xerces code base to locate all the non-inherited exception types and added catches for all of them to the outer stack frame that's supposed to catch them. Hopefully that will hold for longer than a couple of months this time. Worth noting for those running very critical systems and that don't want to implement process monitoring, you can add a
catchAll property to the OutOfProcess element in the configuration to force a trap of all exceptions raised, though that can have destablizing effects if errors occur that leave the system in an unknown state.
Work on the next generation of the IdP continues amongst our recently expanded team, and we'll be firming up our scope for this release in time for an IAM Online webinar next month.
There's still not much in the way of clarity around open Java 11 sourcing in a few months time when Oracle drops out, but we're at least putting the code through its paces now, and I've been running in production at OSU with Red Hat's version for a month or so. It seems likely that their version may be a logical choice for a lot of people, particularly on RHEL/CentOS. I've also been running successfully with UnboundID for LDAP and it's very easy to swap that in to avoid the bug I noted last month.
We have made significant progress on the conversion of Date/Time classes to Java's new (since 8) Time libraries, and this includes revising many of our public APIs to make use of the Instant and Duration Java types to better express intent. This will create some compatibility issues for extensions, but nothing too time consuming to fix in most cases, and we don't expect to see major impact for deployers outside of a couple of features. This is an important modernization step for the code base.
Work continues on the removal of deprecated code and features, which is important to complete before we begin work on new features. Parallel work streams continue on some of the items I highlighted last month, and we expect to see the final release of the GEANT OIDC extension for V3 this month so we can start evaluating the best timeline for incorporating that work.
The next project for me is probably going to be a revamping of the internal handling of EntityAttributes in metadata, to implement more efficient caching and matching data structures. So much of the system is tag-driven that it's important to optimize this and get away from the repeated linear searches we're doing and replace them with hashed/map lookups.
The SP and legacy IdP branch have stablized so V4 development has started in earnest. Deprecated code is being actively removed from all the projects and that work should be mostly complete this month. We've completed two major development tasks: moving the projects up to Spring 5 and Java 11, and replacing the Guava Predicate/Function interfaces throughout the code base with the standard Java 8 functional interfaces.
V4 will be built with and require Java 11, but we're still evaluating exactly what versions of Java 11 to officially support or recommend. There may or may not be reliably "pure" distributions of "free for use" Java 11 after September, and we expect to end up officially testing and certifying a (very) limited number of possible implementations with the rest left to the "probably works" category that OpenJDK used to live in. All of this is subject to change, and there will probably be some additional information available during Spring or Summer. Of course, feedback from members will have to be taken into consideration.
Java 8+ includes a number of new standard features that have replaced, or will replace, various proprietary libraries. The first conversion done was the Predicate/Function mechanism that is heavily used across the code, but also by deployers in various ways to customize behavior. For the most part, existing scripts and plugins will work compatibly after the conversion, with some small issues that should lead to warnings in V4 before causing any breakage in future versions.
We will also be migrating the Date/Time classes and interfaces to Java's standard classes, and this is likely to lead to a small number of incompatibilities, but the details will be clearer once that work is done.
Other open investigations and projects underway:
- Evaluating Velocity alternatives for possible use as a replacement, principally FreeMarker since it remains officially supported by Spring. Velocity will be supported for the time being, but we have to maintain that supporting code so moving away from it is a possibility, though not an urgent need.
- Work continues to clean up an OIDC authentication module, which will lead into more significant design work on proxy features over the summer.
- Discussions about how/when/who to incorporate the V3 OIDC extension. It is not officially targeted for V4 but remains a possibility.
A revised dependency management process is being worked on to avoid the need to privately host all of the dependencies of the codebase by pinning artifacts with PGP keys and/or checksums, and extending a Maven plugin to verify our dependencies at build-time. We continue to seek the best way to keep the build process safe from the attacks that standard Maven projects leave themselves open to and that have caused huge security vulnerabilities in other non-Java platforms, demonstrating our paranoia was well founded.
Finally, a regression was identified in the Java 9+ JNDI LDAP provider that causes connection leaks and/or crashes. So far the regression hasn't appeared in Java 8 but does affect all later versions. The bug has been reported to Oracle but so far the only workaround appears to be switching to an alternate LDAP provider. The ldaptive library has been heavily used by its authors with the UnboundID LDAP client, so we are tenatively planning to move to it as a V4 default, as JNDI has been a frequent source of bugs. We also have documentation prepared describing how to do this with V3. We may, if we decided it's warranted and safe, decide to ship a V3 patch with this change, but it depends on what Oracle does with the bug in the next couple of months.
We were unusually busy in December handling a few different security issues and patches and a few work items.
The SP received a long gestating update to add support for OpenSSL 1.1.1 that turned into a security release at the last minute due to a regression introduced in V3. We haven't done significant testing yet, but these OpenSSL and curl updates theoretically add support for TLS 1.3 for the first time. I've done a bit of investigation into the possibility of HTTP/2 support and it turns out that curl doesn't really support it natively, it's only supported through the addition of another library. I haven't seriously entertained the thought of adding that to the mix; I don't think there's sufficient benefit from doing so at this point. The advantages of HTTP/2 don't really apply to the SP's use cases for outbound HTTP traffic, and technically it isn't even allowed by SAML's SOAP 1.1 binding. The TLS 1.3 support is more relevant long term.
The IdP has continued to receive patches due to a combination of security issues and regressions, most of them due to a late decision to deprecate some syntax we probably should have left alone, but c'est la vie. Another patch is coming this week to fix two more regressions identified by deployers.
I completed most of the planned Logout enhancements, and the changes are largely if not completely backward-compatible with the existing view templates, which was unexpected. Since these are V4 features, we don't have updated documentation yet, but there are essentially just some new messages and a property or two that control the sequence of events and the choices presented to the user during logout, and it's now possible to preempt the entire operation if you want to give the user that option, which was, oddly, requested by a number of people.
We haven't seriously opened up the V4 branches for larger work yet, but that's imminent, probably starting with the move to Java 11 as our development baseline. A feature branch was pushed containing an OIDC login flow and I'll probably merge a version of that pretty soon. I was greatly disappointed, though not surprised, to learn how little actual support for OIDC exists among the major social identity sources but we're not going to get into the business of supporting one-off protocols in our core code. I would anticipate us producing unsupported examples as add-ons to demonstrate the non-standard ones that are of interest.
I don't have a real update regarding the broader OIDC support, but if it's not already been finalized, I expect the add-on will be getting a 1.0 release soon.