We have continued to work on the new IdP plugin framework and are nearing feature completion on several of the first round of plugins we're building, which I mentioned in last month's update. The new Duo plugin in particular will require some time to polish and complete but the bulk of the new work is done.
The major remaining tasks center around completing the plugin framework itself and testing all of that, plus significant changes to some of the IdP's core configuration approaches, to address issues we've identified that make it harder for plugins to "slot themselves" into the system effectively without requiring lots of manual additions. We want to minimize "boilerplate" sorts of changes such that only changes to defaults will require any additions.
As an example, right now adding support for a new login flow via a plugin would require editing general-authn.xml to add a descriptor bean for the new flow. That's going to be eliminated so that default settings for the flow just get applied. The act of deploying the plugin will suffice to install the defaults.
This is part of a larger effort for V4.1 to refactor and improve the configuration. This is all in the vein of "we didn't have time to make it simpler" until now. In some ways, doing this as part of V4.1 is actually a better choice than V4.0 because it forces us to remain compatible as we make these changes, which in turn minimizes the pain of the V4.0 (and V4.1) upgrade process. Using major upgrades can be a crutch to make breaking changes that may be easier to implement but cause pain to others. I prefer we take the path of creating more work for us and less for everybody else.
The configuration improvements have been happening in a staged fashion:
- First, we took on the task of eliminating the system/ folder. This is a long story but suffice to say it's hard to do, requires copying code out of Spring, and won't be 100% gone until V5, but it is essentially unneeded now.
- Next we need to re-engineer some of the internals as I described above, which will allow plugins to be designed more effectively and reduce the need for new configuration of default settings.
- Then we will be retrofitting some of these changes back into the original code and in the process hopefully removing some of the extraneous configuration files from the shipping defaults. This won't help much for upgrades, but new installs will hopefully include fewer files and the system's apparent, if not actual, complexity will be reduced. This may be too-little, too-late but we'll see.
One of the features we will need to add to make this a reality is the ability to "populate" a new default configuration for a feature so that it can be generated by the deployer using a command rather than having to pre-populate all of the possible settings the way we do now. This is also part of the plan for the plugin framework, modeled on many similar systems such as Jetty's.
Lastly we've added a new team member who will be starting to get familiar with the SP build process and eventually automating the packaging we do for it, which will help reduce the amount of time new releases will take. With OpenSSL 3.0 shipping at the end of this year, we hope to be able to exercise a new process to release a compatible version early next year.
Work over the past month has included:
- Research into some complex bugs and behavior that were in the backlog.
- Progress on some of the more complex work scheduled for V4.1
- Work to cleanup and stablize the MDA API for the eventual 1.0 release of that software.
The recent Comodo CA expiration resulted in the observation that the IdP's behavior in the case of LDAP and HTTPS certificate evaluation was different (an expired trust root worked for LDAP, but failed for HTTPS). This took a significant amount of investigation to understand, but we discovered that Java actually has two different PKIX certificate validation libraries built in, and both are being used. One accepts expired roots and the other doesn't. This isn't ideal, but messing around in those layers is even less ideal so it's likely to continue behaving this way.
On the 4.1 front, work on the new plugin architecture continues and we're building several test cases for it around new features that we would like to keep out of the core. The intent is really to try and start delivering more features outside the core release process so they can be maintained with more agility, and in the process probably move to a more regular cadence for the actual software itself, to focus it more on bug fixes and library maintenance. Success will ultimately be measured by the ability to make the deployment process seamless in the sense that it shouldn't matter a great deal whether a feature is part of the core or in a plugin.
Initial plugin work includes:
- Scripting alternatives to accomodate the removal of Nashorn from Java 15+
- Implementation for the next-generation Duo WebSDK
- A TOTP login flow based on a Java port of Google's OATH library
Work on all of them is in various stages of advancement.
One thing that probably distinguishes plugins from core work is the impact on schemas. When it comes to new functionality added to the metadata resolver, attribute resolver, or filter services, which are essentially the remaining components that rely on "custom" configuration schemas, it will be much simpler to continue to include those components in the core software in order to prevent further pollution of the configuration by other XML namespaces to prevent conflicts. An example of this is a new feature to support decrypting data in the attribute resolver. The TOTP feature requires the ability to potentially decrypt token seeds, so rather than build that into the feature directly, a more generic capability was added to the core, and the plugin can leverage that feature for its specific needs.
Since the last update we completed work on the first IdPv4 patch release to address a small number of bugs and some features that were lost in the transition. We believe the bug count has been somewhat artifically low due to the slow pace of adoption but having been running it in production for several weeks I'm also pretty comfortable that there aren't a lot of big bugs lurking, at least new ones.
One issue we hit involved an internal Jetty 9.4 change with wildcard certificates that we needed to accomodate in our examples and (in rarer cases) in the back-channel module and plugin we've provided, largely unchanged, for a number of years. A new plugin was required, and the documentation was updated to reflect this.
There are definitely other pockets of stale documentation that need updating and we need people who encounter those problems to report them so we can get them fixed.
We've since opened the master branches of all the projects for new feature development on V4.1.0 and have started merging in completed new work. A new Jira state of "Pending" was created to track work committed to development branches that is complete but awaiting merge back to master. By week's end most of this work should be merged in and 4.1 superscripts will be appearing in the documentation.
There are a couple of ways to track the work on the next minor and major versions: issue filters (4.1.0, 5.0.0) as we have used in the past, and some Kanban boards for those who might prefer that kind of view. This is increasingly the most effective way to monitor our "roadmap" and the usually-stale Project Roadmap page has been updated to reflect this.
Right now we're mostly knocking off low-hanging fruit and work that's crucial to the eventual transition to V5 rather than complex new features, but we have significant MFA feature work in the plan for 4.1 and have built a PrivacyIdea testbed in AWS to support this.
Initial work in support of the new plugin design is also starting to get merged in and some simple plugins needed for future releases will be built to exercise that code.
Work on the MDA continues as well to move it closer to a V1.0 release this year.
We have completed migration of most of the project infrastructure to AWS, which will be more cost effective and simpler to manage. All services have been moved and are fully functional except for Jenkins, which is being debugged to get all the jobs working again, after which we're hoping for some improvements to the frequent networking problems we have encountered in the past. Total time spent on this is over 75 hours and will hopefully not need to be repeated at quite this scale again. Core services are running on CentOS 8 now to extend the useful life of the deployment. We should be shutting down the old system within the next week. We have not deployed our services with region or availability zone redundancy, as that would greatly increase costs and not be an apples to apples comparison (we weren't redundant before).
The SP V3.1 release was completed, with no significant issues arising to this point. The next major hurdle is tracking the changes coming in OpenSSL 3.0, which is available in alpha form. OpenSSL has about the worst API documentation I know of (as in, there isn't any), so there's a significant risk of breakage to a lot of important code in the SP and its dependencies, so the work involved remains to be seen. I would expect this is going to be the sort of change that may result in a 4.0 update because of the impact.
I'm frankly expecting we may lose all ability to support use of certificate authentication in the SP's back channel support, but that's speculation at this stage, it's just something to watch. Moving to port 443 and signed messages is the best way to inoculate against that risk, for the few deployments still in need of SOAP support. We'll get a bug created to track this work.
We have made significant progress on drafting requirements and starting preliminary work on a new plugin design for the installer to enable more factoring of the code base. This is a big deal and it's not being undertaken lightly, but we don't see how to accomplish what we need with any off-the-shelf solution that would accomodate the freedom of deployment models we have now (i.e., we could use new tools and then write off anybody who doesn't use those tools, and we just don't believe in that approach for the core software). Spring, and particularly Web Flow, also more or less preclude some of the approaches we might otherwise consider because the architecture doesn't allow for the sort of class-loader isolation that something like OSGi would require.
Note that we do not plan to move to any sort of piecemeal "patch" approach to delivering bug fixes, as that's a totally chaotic and unmanageable deployment model that undermines the careful versioning policy we adhere to. But the model will accomodate detection of compatibility problems when upgrades are done, even retroactively (the compatibility rules themselves are expected to be remotely obtained). Once the basic framework exists, we probably will find new uses for it, but the focus is on delivering more manageable installation of complex add-ons.
We will be initially designing a number of deliverables to rely on this approach:
- Non-trivial storage plugins
- New (and old) scripting alternatives
- Official OIDC support, both inbound and outbound
- Duo integration via OIDC, eventually replacing the original integration
In the future, most new work that is expected to be of more limited adoption or that requires substantive new dependencies will probably take this form. We welcome interested parties to provide feedback on the dev list. Initial work on this will hopefully start showing up in V4.1.
I would also like to call attention to the fact that yet another IdP-of-last-resort for our services, UnitedID, has ceased operating. SUNET has kindly agreed to make their open IdP (eduID) available more widely. If you have existing bugs or pages you're tracking and need to rename a Jira or Confluence account name that was formerly issued by UnitedID just drop a note to our contact address and we can rename it for you.
Project activity can best be described as a bit unfocused right now, as a somewhat indirect fallout of the pandemic, not so much due to the change in work habits but because it's self-evidently slowed any real uptake of the IdP 4.0.0 upgrade. More typically we'd be fielding a decent range of bug reports and working on a patch but with the paucity of testing so far we haven't seen much, so we're in a holding pattern on exactly what the next release needs to be. We do have at least some positive feedback that suggests the release is solid, at least for upgrades.
We'll probably be forced to open up the master branch for new work fairly soon to start work on V4.1. There's a small set of deprecation warnings that need to be added so that at least one release is done with them prior to V5, but the list of deprecated material is much shorter than the V3/4 transition was. We have no ETA for 4.1, so we do not advise waiting for that update before making the jump.
After the IdP release and with the relative breather, work is complete on the SP upgrade and posted beta packages, with testing as usual being minimal to non-existent (which is why we don't bother much with them). After some additional testing on shibboleth.net this week, the release will most likely be done next week. Apparently, there's not much immediate pressure to get any SameSite assistance out the door because Google kindly decided to exercise their total dictatorial control of the web and defer wider dissemination of their Chrome change. In any case, the ReleaseNotes for the upgrade and documentation on the changes are available.
We are also actively engaged in reviewing a lot of older issues and finding small items to close out (fixed or not) before tackling some bigger work. We have also been working to fix issues with the project javadocs caused by a range of bugs in Java, javadoc, and the Maven javadoc plugin. The javadoc search feature works again and most of the expected links between projects and dependencies are working again, with the exception of the Java EE APIs.
Most of the summer will probably be taken up by further work on advanced authentication features, designing a plugin installation and upgrade architecture, and adapting the OIDC extension into its final form. Some of this work will probably ship ahead of V5, but the bulk of it will be targeted for that release next year.
Not a long update this month, as the activity is just an extension of last month's update. Work has been hard and heavy though.
Two beta releases of the IdP were completed, with a few serious bugs found and fixed, enough that we are comfortable with an official release being cranked as I write this, with the announcement due tomorrow. V4 is the product of over 15 months of work primarily focused on technical refresh and maintaining the maximum degree of compatibility possible as we move forward 4! Java versions and a full Spring version. The OIDC extension should be finalized within a few days of the IdP release.
We believe we have achieved the goal of a true major upgrade that feels more like a patch. Just be aware that the less you notice, the more work went into making that possible. The ReleaseNotes are extensive and cover the highlights. We will be focused on feedback and bug fixing over the next couple of months so a patch or two should be expected.
On the SP front, work has continued on a 3.1 release that has accumulated a decent list of fixes and features. Work on SameSite and correlation support is complete and the remaining tasks consist of fairly minor enhancements and some documentation cleanup that should make a late March or early April release feasible.
By early summer, work will be underway on a roadmap for the IdP that will take us through V5 and full incorporation of the OIDC extension into the project. We expect that V5 in 2021 will see minimal changes for deployers except for the transition of the OIDC module from third-party to fully-supported extension.
It's been a busy month for the project primarily to get the expected beta release of the Identity Provider V4.0 done. Testing to this point has not been extensive, but the early returns from within the team have been positive and the final release is expected by early March at the latest. Upgrades should be very easy for the majority of deployers.
The lifecycle of support for V3 is essentially a foregone conclusion at this point because Spring 4 is end-of-life at the end of 2020. We don't have any compelling reason to abandon security support for V3 any sooner than that, so V3 will be supported through the remainder of this calendar year.
I undertook a mini-project to blitz a port of the OIDC extension and we are on track to produce a compatible extension along with the IdP release. After a short discussion, we agreed to forego significant breaking changes to this code for now and leave things largely as they have been for this release.
For clarity, since this is a period of transition:
- The code in GitHub is the V3-compatible extension. Issues with it can continue to be reported via GitHub if desired.
- The V4-compatible extension is largely the same but has been moved into our Git repository and will be distributed via our download site. The documentation remains in the GitHub wiki, but we are using our Jira instance for bug reporting and patch submission.
- The V5 extension when it comes will include a number of breaking changes so that the code packages can be renamed and configuration layout changed, and the documentation will be moved into our wiki. At that point we will be in a position to fully support the extension to minimize breakage in the same way we have done for the rest of the feature set.
In parallel with the beta effort, work has begun on the next SP update, V3.1. This feature upgrade is primarily focused on addressing XSRF in two areas:
- Providing a fix for the SameSite issue in a Safari-compatible way
- Adding request/response correlation along with an option to block unsolicited SSO
The SameSite work is complete and will explicitly designate a subset of SP cookies as SameSite=None, with an optional setting to add a second non-SameSite cookie to support broken browsers for hopefully a temporary period of time. It will also be optionally possible to support cross-site use of SP sessions, when XSRF is a "feature" and not a bug.
The correlation work is underway, but is being carefully implemented to maintain compatibility so it will be an optional feature for security conscious deployers to turn on.
There are a few other odds and ends to complete for the SP release, but we're moving aggressively to complete that work as soon as possible so it's quite likely March will see both IdP and SP releases.
We have a significant queue of work items for the IdP once V4 ships, primarily authentication-related, so the roadmap should be getting cleaned up and clarified once we can take stock of things.
Welcome to the third decade of the Shibboleth Project.
Various cancelled meetings, holidays, travel, and just generally getting work done have led to a lack of updates, but as of the new year we have started the process of wrapping up V4 IdP development so we can get a beta released as soon as possible. Much of the smaller items left to do are now done, and most substantive remaining issues have been pushed out to future versions, so what's left is primarily testing, finishing up work on the SAML proxy feature, and beginning to update the OIDC extension.
The installer is essentially complete at this stage, and is substantively the same as before though has been rewritten to make more extensive use of Java code in pace of ant so we can make future enhancements to it more easily by "just" writing code.
The SAML proxy feature is missing most of the necessary security code. This will hopefully start getting committed in the next week or two. It is however functionally done and documented (SAMLAuthnConfiguration) and I don't anticipate significant changes being made for this first iteration.
The biggest remaining task is prepping an upgrade of the OIDC extension for use with V4 so that it can be made available at or near the initial release. As recently announced, we have officially assumed ongoing responsibility for the code base as of last month, and have taken the first step by importing the code into our official repository. Our plan is for any required maintenance on the original V1 branch of the extension for use with IdP V3 will be carried out in the original github repository, while work on a V2 extension for use with IdP V4 has begun on the master branch in our repository.
After some consideration, our intent is to maintain most of the current configuration, XML namespaces and types, and Java packages of the original extension, with the understanding that the implementation released with IdP V5 next year would introduce a number of breaking changes to make a number of final adjustments for the long term sustainability of the code. This version is therefore a transitional release, and deployers of the OIDC extension now and with V4 should expect some additional work to upgrade to V5 later.
As part of this transition, the documentation will likely continue to be maintained in the GitHub project wiki since it will be largely the same for this release, with the updated/integrated documentation moving into our wiki for V5.
It is however, a discussion point as to whether the code will be moved into the IdP core or maintained as a separate extension for organizational purposes. If that is the decision made, we will out of necessity also take on the delivery of some kind of formalized "module" system for the automated install and update of extensions to the core software to minimize the impact of that decision. It will, in other words, not matter all that much to a deployer whether the code is separate or not, but a separation will help us design such a module feature, and potentially reduce the impact of security issues in the relatively less mature OIDC code from the rest of the code base.
Long story short, lots of work was completed over the holidays, there's still a fair bit left to do, and we're working hard to get a beta out the door as soon as possible.
Famous last words as we had to ship yet another IdP V3.4 security patch last month to address a memory leak, which required an unfortunately disruptive change to the classes supporting the External authentication flows. We don't take making breaking changes lightly, and do apologize for not fully recognizing the risk of impact to third party plugins, but it was a good chance to remind people that it's just not safe or supportable to directly copy most of our flow definitions because they include non-public classes that are subject to change. We're happy to work with people on-list to provide any missing extension points needed if they're consistent with our goals.
Work on IdP V4 has continued aggressively of late, and the installation changes are wrapping up, which should position us to ship without any major required work items left holding things up.
A new feature I just completed that's worth highlighting: it will be possible in V4 to split one's metadata configuration such that metadata filtering rules can be separated from core metadata provider configuration, allowing a separation between core federation-delivered metadata service definitions from local deployer-created filtering rules.
A few significant updates on the plans for V4:
- We're definitely not intending to ship the OIDC add-on in this release, but will work to ensure a compatible version of it is available at the same time as the release along with additional support for use of SAML metadata to manage relying parties. The final integration into the code base will be a major V5 work item.
- We are optimistic that we will be able to provide a first iteration of SAML proxy authentication support in V4, as the work has proceeded well to this point and we don't foresee any hold ups. It may not be 100% feature-complete, but will be usable certainly, so we can evaluate whatever else might be necessary.
- We continue to expect a seamless or near-seamless upgrade path to V4 from V3.
- We are hopeful of reaching a very late alpha or hopefully beta release by the TechExchange conference in December, and expect to release by early next year. We could probably make an earlier date, but the extra time will help with testing and documentation.
Some on and off work on the SP has been occurring, mostly in response to a serious crash under load caused by a bug in the newly introduced cookie session recovery feature. The fix is done, but we're probably looking at a V3.1 SP release towards the end of the year simply because of resource constraints. We would like to get a new RPM packaging process in place before doing another release, so that will take some extra time to manage. We do have packages available for CentOS 8 for the time being.
The main highlight for this month is that we're going to be issuing an IdP patch, V3.4.5, probably next week. The bug isn't a major one but is an opportunity to ship a few fixes before V4, the most significant being IDP-1491, which allows binary LDAP attributes to work again on newer Java versions where the UnboundID code has to be used. I think we will also include some optional support for addressing the SameSite cookie issue for those deploying without Apache or other software that can more easily address the problem. We have an in depth testing analysis here.
This should, once again, be the final V3 release barring any additional security issues.
I have completed work, more or less, on the revamped password validation support I described last month along with most of the basic documentation updates. We probably will be opening up the draft V4 documentation soon so people can see better what's coming, but we have more work to do on the release notes before that's going to be too helpful.
Support for a handful of new audit fields have been added, such as logging signed requests, encrypted assertions, and requested authentication context information. The default audit format for V4 is not decided at this point, I think we're open to suggestions there, but obviously it doesn't matter much for people upgrading.
An area of current study right now is to make sure all the "fail-fast" and connector validation behavior across all the various services and attribute resolver are more consistent, both with regard to defaults and to make the settings more consistent. This includes fixing the misleading errors that appear when servcies fail to start properly but are falsely attributed to the metrics support.
Most of the deprecated APIs and features have been removed from the master branches as we get closer to a release. I think the only real area of further revamp right now is probably the installer; everything else left is more around new features that we can more easily include or exclude based on the desired schedule.
We're still waiting on CentOS 8, but we're not going to spend much time on the SP again until that drops.
Work on IdP V4 is generally on track, but we're still probably 4-8 weeks from final scoping depending on what it looks like we can finish before meeting our intended scheduling goals.
Some API changes are continuing throughout the code base, most recently a large physical change with relatively little actual impact, removing a lot of generic type parameters from the API that weren't being used effectively and that made it difficult to fix a lot of compile-time warnings in the code. This will have some nominal impact on third party extensions, but nothing very extensive.
We have a tentative strategy for adding more formal CSRF prevention features to the next version, subject to more testing.
An "almost complete" work item is a redesign of the password validation code to support chaining of both JAAS and non-JAAS authentication options. We get asked a lot about how to support multiple LDAP directories at the same time, and we now have an alternative approach for that (and more general cases) that should reduce or eliminate the need for deployers to be copying or mucking with the Password login flow. Building custom validators will also be possible with simpler, more self-contained Java code that doesn't depend on Spring and doesn't require adhering to the odd interface that JAAS modules impose.
Substantial progress has been made moving CI builds to AWS.
CentOS 8 is still a work in progress, but assuming it emerges in September/October we will need to spend some time on SP backlog in preparation for a new release to coincide with doing that packaging work, particularly given that there might be a bug or two to fix to get that packaging done.
Not a long update this month, just some status updates:
- Minimal feedback to this point regarding our publicly announced Java support decisions, but what little there has been suggests the clarity is meeting immediate needs for planning.
- A few people have started to upgrade to newer Java versions and are hitting the LDAP bug. Crashes are the best case and easy to spot, but non-LDAPS deployments are seeing total pool meltdown only under load. No fun, but obvious now that we know what to look for. The one problem we don't have a fix for yet is that JNDI's support for binary attributes is not something that translates to UnboundID, so deployments with that need are probably limited to Java 8 for now. There are years to go before supported Java 8 options lost support so there's time to address this.
- Work is slowly proceeding on developing a CI capability using AWS, to relieve our current resource constraints.
- We have a number of design proposals and prototypes for adding CSRF prevention to the IdP, which is overdue. Discussion will take place on the dev list as well as a regular meeting in the near future.
- Contrary to what we had been thinking could happen, we don't expect to move off of Velocity as a view template technology in the short run, and are hoping the Velocity project will eventually take over the Spring integration code that was removed from Spring 5.
- Documentation of the new attribute registry/mapping feature is getting underway now that the configuration has mostly solidified. The V4 wiki space is still hidden for the moment to avoid confusion, but I have started to move various material out of the V3 space and into the KB area so that it can be kept independent of specific IdP releases.
- Real work on proxying hasn't really started yet, it's still a lot of vaguely formed ideas at this stage. With vacation time I don't think work on this will really get going until August at the soonest.
A final note: RHEL 8 is out, but CentOS 8 is probably still a few weeks out. I do not expect to have official SP packages for either for a while yet.
Lots of work in progress to highlight this month, with one noteworthy concrete item:
We have formalized a set of Java distributions to support to varying degrees, and an expanded set that we are (or shortly will be) testing as part of our CI processes. The full, gory details are on the Java Distributions page, and it's also forward looking to the next version, but the boiled down information for V3 is on the updated SystemRequirements page, and spells out the Java options we are recommending going forward.
As an aid to transitioning people, we are still supporting Oracle Java 8 right now, but we don't see much of a future for that, though that could change if our members indicate they feel differently. The plan is to steer people toward other options of choice, and then officially end that support with V4; the rest of the options listed are the ones that would survive that transition.
The other thing to note is that we're formalizing the concept of "partial support", which is another differentiation for Consortium members by offering direct support for particular platforms with the understanding that we may have more limited ability to quickly reproduce those environments in the unusual case that it becomes necessary. This, we feel, best reflects our generally ambivalent attitude about platforms that "work" but that we don't have constant access to or familiarity with. Today this mostly revolves around Debian, but this gives us the ability to codify it. We also expect to expand this notion to the SP side of the world shortly.
On the development front, a number of projects continue, the bulk of which center on the design foundations for proxying support that I've discussed previously. As of this month we have completed a new service design for the attribute translation functions that were formerly controlled with the Attribute Resolver, and now also includes offloading the display metadata used during attribute release consent, so that will also be possible to supply with centralized, shareable defaults provided by federations or communities. We revamped much of the system's language handling to be more correct and honor browser settings more consistently.
The final form of the new configuration, which will ultimately be possible in a variety of ways, is not fully determined, but we're converging on it. We have no plans to deprecate the use of the Attribute Resolver's AttributeEncoder, DisplayName, etc. elements either, and they remain supported. There are situations where both old and new approaches will make sense, but the new design simplifies bulk proxying of attributes. We will probably start looking at opening the V4 documentation space so that we can start to document some of this new material and get feedback.
We have prototyped the attribute proxying features, including the use of the Attribute Filter engine in the "SP mode" of filtering data inbound as well as outbound, and this is functional when applied to simple examples like data passed in from the External login flow. Today that's a good catch all for proxying, but the next steps will be to start building more intelligent proxy login flows for the various protocols. I expect this will include CAS as a good starting example, building on the work Unicon has done for years in supporting CAS authentication to the IdP. In practice, features like filtering don't matter as much with CAS, but it's a good test case for the real thing later.
The focus is still more on delivering the foundation in V4, so it's possible much of this will remain a work in progress there, but should be more fully realized by the time V5 ships.
Other work in progress includes some prototyping of new anti-CSRF features, and work on adapting SAML SP metadata for use with OIDC relying parties, much as we have done for CAS. This extends a large range of additional IdP command and control to OIDC without having to reinvent the wheel. As the summer moves along we should have an idea soon about whether the OIDC OP support will be an updated add-on or a part of V4.
There's a grab-bag of smaller topics of general interest this month.
If you missed the IAM Online last month, the slides and a recording are available at https://www.incommon.org/iamonline/
An IdP patch was released of relatively minimal criticality unless you're in the process of or planning to move to Java 11, in which case you will likely be impacted by an LDAP bug that's not been fixed to this point, and for which we have documented a workaround. The new patch simplifies the workaround by including the necessary libraries out of the box.
We've published example configuration files in the form of a downloadable starter kit for using Jetty 9.4, the most current release, designed for use on either Java 8 or 11. Jetty 9.3 is probably going to reach its end of life in the relatively near future so anybody still using it (which included myself until early this year) is certainly advised to look at an upgrade soon. I found the process very painless.
On a further related topic, please stay tuned for some imminent documentation on our plans for supporting OpenJDK 8 and 11 as the expected transition away from Oracle starts to pick up steam. As a preview, I can say with some confidence that we are leaning towards Amazon Coretto as a solution of choice for mult-platform support, particularly on Windows, plus support for Red Hat's version. Our plan is to establish a matrix of versions to continuously test that will include both officially supported versions and important alternatives (Oracle's included). This expanded testing will necessitate a revamping of our CI infrastructure, and Internet2 has made access to AWS available so we can offload this testing from our primary project server.
On the development front now, we have a number of feature branches under active development along with ongoing investigatory and design work into various topics. A new feature for generalized encoding and decoding of attribute information is nearing completion and will at least supplement, if not replace, the use of AttributeEncoders. The new design will support more configuration flexibility as we add more protocols to the system. It should reduce the need for unrelated content in the already-complex AttributeResolver configuration. Most particularly, it will allow for more natural ways to configure inbound processing of data from SAML, OIDC, or CAS proxy authentication even when that data isn't being passed along for outbound use. A side effect should be faster processing of configuration lookup based on SAML metadata EntityAttribute extensions (so-called "tags") by allowing tags to be decoded into internally optimized hashmaps instead of repeated linear searching of XML objects in memory.
Finally, note that Red Hat 8 was announced today. A CentOS 8 release is probably weeks away at best, so don't expect much on the SP packaging front there for a while.
We have an IAM Online on April 10th providing an update on the Consortium and an overview of work completed in 2018 and the roadmap covering roughly 2019 through probably the early part of 2021. This will mainly cover our expected scope of work for IdP V4 and V5 (the former already in development), with each planned for roughly a 12 month development and release cycle, faster than our major upgrades tend to be. This is a reflection of the stability of the V3 design, allowing us to make more targeted changes to the software that may necessitate a major version change but won't make upgrades a complex process. We hope simpler upgrades are the norm now rather than an exception.
Most of the basic outline of the roadmap has been mentioned in the course of previous blog entries:
- V4 will be primarily a release focused on technical debt, moving to Java 11 and Spring 5 and removing a lot of deprecated V2 features.
- It will also include new design work to allow development of new proxying features for at least OIDC and possibly some limited SAML login flows. This will allow some degree of delegation of authentication to other IdPs, much as is supported by a lot of other SAML software today. This will be a first attempt, so is likely to have rough spots, but represents the direction of a lot of future work.
- V4 may or may not end up including an integrated version of the GEANT-funded OIDC extension for V3.4. The first official release of this extension was announced recently. The more testing and feedback they get on that release, the more likely it might be considered for inclusion, but if it slips, there will be an updated version compatible with V4 in any case.
- V5 will be focused on polishing the new work integrated into V4, allowing us to adjust APIs where need be. It will probably include more complete SAML proxying support, and will almost certainly include OIDC OP support natively if it isn't incorporated into V4.
- We also have some backlogged work on improving the installation process, so it's possible that will be slotted into V5 as well.
If you have questions or input to give, please attend the IAM Online or send something to the development list.
We have an IdP V3.4 patch coming this month (hopefully) to address some bugs and add some additional warnings to some features that might be removed from V4 if we decide they're simple enough to remediate for deployers. This release will ship with defaults supporting the UnboundID LDAP provider to work around a Java bug people are just starting to run into as they move past Java 8, and by including the necessary jar files, it will be simpler for existing deployments to switch over to it as required.
I've started actively working on new designs for handling the mapping of data between the IdP, SAML, and OIDC to begin to lay the groundwork for the proxying support. There's not much to evaluate yet but the intent is to factor out the current AttributeEncoder support in the AttributeResolver and create a more general omni-directional mapping facility that can be configured by, as much as possible, "shared" definition files for common attribute mappings in different protocols, while still allowing customization where required. It isn't necessarily a given that the older AttributeEncoder approach will be deprecated, it depends on the maintenance burden of keeping it. But it will definitely be historical at best, or a more convenient way to support less common per-SP attribute encoding rules that may be more convenient to express that way.
There will also be code released in the relatively near future that implements client support for Amazon's RedShift JDBC and ODBC database drivers that works with the ECP support in Shibboleth or other SAML-compliant IdPs. There are some challenges federating database clients, but it does functionally work and Amazon approved the release of the code that references a couple of their Java interfaces, so we will be posting it in some form, though not officially supporting it. If you're interested, stay tuned.