The last month has been dominated by travel to TechExchange and getting the new support services rolled out for testing by the members. TechExchange was a bit less exhausting than the past couple of years, and was dominated by a lot of good conversation and discussion about OpenID Connect, including a good touch-base with the CSC team working on those features for Shibboleth. I'm hopeful there will be some initial bits to play around with by early next year.
Proxies were another dominant area of discussion, and after an offhand comment/question from a member, I went ahead and implemented some policy support for the
<Scoping> element in SAML that allows the IdP to recognize different downstream services on the other side of a proxying SP.
The support rollout has been relatively smooth, with very little use so far (the next Slack message will be the first). The best part has been the dramatic and ongoing increase in members as a result of this change.
On the more traditional development front, work has been progressing on:
- Re-designing the dynamic metadata support in the SP.
- Finishing the long-delayed SOAP client support in the IdP, though that's more of a "looking ahead to doing proxying support" thing than anything the IdP itself is desperate to include.
- Getting a Java 9-compatible build process working with Maven so we can start testing on it.
Going into the holiday season, I will be working to get an updated release of the XML Security library used by the SP, and probably doing a Xerces bug fix release out of the kindness of my heart, and we'll hopefully get a working set of examples done for Jetty 9.4. IdP V3.4.0 is still on track for early next year but we'll need to get some progress made on the remaining work there during the holidays to stay on track.
Just a brief note to post the short set of slides I presented at the BOF at Internet2 TechExchange last week in San Francisco, primarily a summary of 2017 and an outline of the 2018 project goals.
Highlights of the last month of work:
- the recently-release IdP 3.3.2 IdP patch and security advisory
- some development on IdP 3.4
- ongoing cleanup work with the SP code/build and dependencies
- getting Java 9 support into our Jenkins builds and some reconnaissance on it
- prepping for managing access to the new support mechanisms
The most recent bout with defaulting the handling of TLS verification in the LDAP code has led us to deprecate that approach once and for all going forward, and we will be requiring explicit trust configuration of LDAP connections in 4.0. We're still going to have to study the impact on metadata configuration, but that's much less sensitive to this issue in current practice. We will have to consider the impact moving forward as federations transition to query-on-demand approaches, and we fully expect some federations will decide that signatures aren't what they want to do, whatever our opinions might be. But as a project, we are committed to the idea that security needs to be explicit and not left to the behavior of code we don't control.
The other major addition to the patch was a feature addition, which is not typical, but I felt was warranted because the timeline for 3.4 is unknown and I wanted to make sure the supported version included the ability to generate computed pairwise IDs that are safe for case-insensitive applications, of which there are many. Obviously this is a lost cause for many deployers today, but if we can make life better for the new ones, that's a good thing.
A new feature for 3.4 that was completed this week is something requested by one of the newer Consortium Members, a way to impersonate users for testing purposes that comes up in a lot of deployments. I knew that it could be done both simply and (I hope) safely in a particular way so I felt that it was worth spending a few hours on an example we can include in future releases. The documentation on that will be forthcoming but it's already engendered more discussion on the development list than I've seen in months. I firmly believe testing and test accounts is something that a production IDM environment should deal with directly and not leave up to applications or the IdP, but I also know from local experience that convincing IDM teams of that is a lost cause for many sites. Hopefully this will help.
Java 9 is out, and is a huge change to the Java platform, probably the biggest ever. We're still getting our minds around it but it will be a while before we consider what we might need to do to take advantage of it. More immediately, Java 8 will also be EOL by next September, as expected.
On the support front, we're on track to start rolling out the new options next month and begin to phase in that model through the end of the year. The management of all this is not perfect or where one would like it, but we're sensitive to the amount of time we can afford to spend on managing the way we do this, and we have a lot of existing system behavior with JIRA that limits how much we want to change things. I can definitely say that if JIRA had a way for us to offload the group management through, say, an entitlement attribute, we would explore that, but right now the Support Desk module does not rely on JIRA's own group mechanism for grouping customer accounts. I hope that changes at some point.
One priority of late has been to begin moving our development platform to Java 8 (and eventually 9 once it's out) by reorganizing our Maven parent project structure and establishing a new versioning scheme that will allow us to maintain both the current Java 7 code bases and start moving some of our work forward. The initial target for this work is the Metadata Aggregator, which will be the first product released requiring Java 9. This will necessitate creating maintenance branches for some of our libraries so they can be co-developed for both platforms for the rest of the life of IdP V3.
The other major area of work has been to rescue two of the SP's essentially dead dependencies and produce new releases. Xerces V3.2.0 was released last month at a cost of about 60-70 hours of project time. In addition to some general bug fixing, this release includes a greatly valuable feature allowing us to programmatically disable DTD processing in the parser in a future SP revision, limiting our exposure to a likely unending stream of security bugs in that code. The other major benefit is that we will be able to package this version in a manner that no longer conflicts with packages from Red Hat, allowing us to base the SP packages on a maintained version of the library. Red Hat has left security issues in this library unpatched for over a year so we saw no alternative but to invest the time.
We are currently working to produce a new revision of the Santuario xml-security library, primarily to address small compiler compatibility issues and to allow us to package a more minimal build of the library that excludes large sections of unsupportable code we don't use and don't need, again in the interest of reducing our attack surface.
Once this is complete, work can begin in earnest on the SP and on moving the documentation to a new wiki space. We will also have a simpler, more repeatable build process on Windows and will be moving the code base to the latest Microsoft tools. To some extent this is a race against time, but there's always the chance we'll need to do security patches for the current version in the interim.
We have a bug fix release of the IdP coming in early October which we'll be finishing up during the rest of this month, and we are working to wrap up work on some new modular configuration examples for Jetty 9.4 that will eventually find their way into the IdP.
Our intention is to finalize our plans for the new support changes in the next few days and announce a timeline for establishing the new support mechanisms for our Consortium Members.
Next month a couple of us will be at Tech Exchange, and we should hopefully be able to provide an update on the GÉANT-funded OpenID Connect work.
As we start to change some of how we interact with the user community for our software, it's been a mutual suggestion by the Board and the development team that we try and do a better job of keeping people aware of what we're up to as a project and what our upcoming plans look like. To that end, the updates that have been traditionally provided on a monthly basis to the Board will be done publically using wiki postings. Obviously, we can't share 100% of what we're doing because of, for example, security issues we haven't disclosed yet, but for the most part I'm going to try and limit the amount of sanitization.
We welcome feedback on anything we're doing via either comments or preferably the development list.
Before I cover more recent activity in another entry, it's probably appropriate to review 2017 to date.
This has been a transitional year to some degree because there tends to be a rule of "three", whereby the third version of something tends to be the thing you wished you'd done to begin with if you had had the time. While it's technically the fourth "feature" update, IdP V3.3.0 was really the third substantial one (the first was really a bug fix that necesitated API additions, so it became a minor bump instead of just a patch).
The major goal for this release was to deliver a revised framework for handling multi-factor authentication and for orchestrating more complex login flows to address a lot of the enterprise use cases we've been struggling to support well since the original release. Since then, there's been a lot of time spent documenting and supporting that work to make sure it's working for people and to figure out the best ways of doing various things with it. I personally have probably spent up to 15-20% more time on support than usual in fact.
The rest of our time not spent on the usual day to day has been divided between:
- V3.4 planning and development
- Mapping out a plan for a new SP version
- Completing the transition of all our projects to git, which is now completed
- Consulting on Internet2's TIER packaging efforts
- Transitioning the old web site off our system to relieve the developers of the burden of supporting it
- Summer-long discussions and planning for the support changes announced last month
A few of those are worth some elaboration.
The IdP V3.4 release is really more about V4. It's a transition release intended to warn deployers about any use of deprecated features or settings so that there is ample time to adapt anything before V4 is released and removes most of those bits. There are some features planned, notably new support for metadata-driven configuration, but the main purpose is to suitably sunset the V3 branch to support a hopefully-seamless transition for most deployers to V4. That will be the first major release we've ever done whose goal is not to rewrite the software or deliver radical changes but to actually pare back the code and remove old features, change defaults, etc. We hope to deliver V3.4 no later than Q1 2018, and hopefully sooner.
The SP is long overdue for a major update, but realistically that's not likely to happen given its relative maturity and the bandwidth we have as a project. What is needed however is a documentation cleanup, a lot of dependency care and feeding, and a reassessment of our platform support (i.e., Solaris, this means you). So there will be a V3 SP but it's not likely to be a rewrite or even a significant change really (with the notable exception of a native IIS7 module which is already available to test). Unfortunately that doesn't make it a small amount of work because the bulk of it is work we have to do to remediate issues in our dependencies, many of which are no longer being effectively maintained by the Apache Software Foundation. In a perfect world, we would change them to more supported options, but that's a bit prohibitive without additional developers to pick up slack in other areas.
Those are the major highlights. This month's "regular" update won't be very exciting given that a lot of it is covered above but will be put together for tomorrow's Board meeting, so look for it shortly.