Configuring Linux To Run a Servlet Container as Non-Root
Running a servlet container without Apache that needs to bind to ports < 1024 but still run as a non-root user usually requires special setup (unless using Debian or Ubuntu). Some containers include tools to assist with this, the Linux kernel also has features that can help enable this, or another option is to rely on port mapping.
In this approach the Java servlet container listens on a high port and the local packet filter is used to forward requests from standard ports (80, 443) to those high ports.
One caution regarding this approach is that it will cause your IdP to fail if the port mapping software is stopped. Normally dropping a firewall doesn't prevent existing services from running, but this approach changes that situation. You should take care that any administrative staff are well aware of this change.
One way to deal with this issue is generating the iptables (or firewalld) rules dynamically on each start of the container, which is easy with systemd. See also below (POSIX capabilities) for other examples using this technique.
- Linux kernel that support iptables and nat
- IP address and ports numbers of servlet listeners
For non-Red Hat Linux installations modify /etc/rc.d/rc.local to include the following lines:
For Red Hat Linux installations using iptables (Red Hat 6 and earlier by default) modify the nat section of the /etc/sysconfig/iptables to include the following lines:
Note the changes are only the addition of the DNAT lines in the nat section.
For Red Hat Linux installations using firewalld (Red Hat 7 and later by default, unless you specifically switched back to iptables), issue the following commands as root:
- Add iptables rules to non-Red Hat Linux installations by running the iptables commands by hand.
- Restart iptables on Red Hat with the /etc/init.d/iptables script.
Debian, Ubuntu and derivatives come with the authbind utility and integrate this with their Tomcat packages by default (only needs to be enabled). Others can download and build the software from source code (or try an inofficial RPM spec file), of course, but that also requires changed Tomcat startup scripts. Latest changes to authbind were made in 2012 (as per Aug. 2016), so it's not a particularly fast-moving target (i.e., no need to frequently update and rebuild the software, it's very stable).
Check the IdPLinuxNonRootDebianUbuntu page for configuration details.
Apache Commons daemon JSVC
On systems lacking authbind (e.g. RedHat, CentOS & friends) Apache Commons daemon (jsvc) can be used to manage Tomcat. Instead of the complex layering of shell scripts commonly used in Tomcat distributions (and GNU/Linux distributions of packaged Tomcat), using systemd one can override the packaged service file to use jscv instead (or alternatively create another service/unit file and start that instead of
tomcat). JSVC starts as root – and hence can open ports < 1024 – but runs Tomcat and the JVM as unpriviledged user (here set to "tomcat" in the jsvc command below).
- Apache Commons daemon (jsvc), RHEL/CentOS:
- Systemd (other
inits left as excercise to the reader)
The examples below assume a CentOS 7 system and use of the packaged Tomcat 7 (no Tomcat 8 available for CentOS 7 ) and OpenJDK 8 software. Configuration parameters can be read from a file (here CentOS's default config file is used, but you can also supply your own, with arbitrary NAME=value assignments) and so can be tuned independently from the systemd service file.
The following unit file tries to mirror the default Tomcat startup scripts from CentOS 7's tomcat package as closely as possible, only using
jsvc instead, thereby avoiding the low port issue. Adjust memory ("-Xmx1g" used in the example below) as needed/recommended.
Note that with
Type=forking the systemd journal contains messages like this:
We therefore changed the
Type=simple and add the "
-nodetach" parameter to
jsvc, which is how systemd seems to prefer things.
To address the problem of unpriviledged users not being able to bind to priviledged ports head-on POSIX capabilities can be used to allow just that. The java binary used to run the servlet container needs to be given this capability.
- Linux kernel that supports capabilites (from 2.6.24 on, if compiled in)
- Systemd for automated management of the capabilities on service start/stop
(Alternatives are possible using the Kernel's inotify feature and tools like
incron, to ensure changes are re-applied after each Java upgrade)
The examples below assume a CentOS 7 system and use of the packaged Tomcat 7 (no Tomcat 8 available for CentOS 7 ) and OpenJDK 8 software. First try everything manually on the command line (possibly adjusting the paths, this is on AMD64 using all defaults):
Then set your HTTP connector to port="443" or "80" in /etc/tomcat/server.xml and restart Tomcat. Check the status output, listening processes and the process list that it's the unpriviledged
tomcat user that's running Java here.
Only continue with automating/productionalizing this approach once you have this working.
setcap command needs to be reapplied after every Java upgrade. While that could be done automatically, e.g. using the Kernel's inotify feature, we'll use a slightly amended systemd unit file to make that process simple and reliable, without resorting to additional tools.
Copy the content of the systemd unit file (included below, expand to view) into a new file
/etc/systemd/system/tomcat.service or alternatively run
which spawns an editor with the current unit file. Add the lines between
END to the existing unit file, in the same place (after EnvironmentFile and before ExecStart).
JAVA_HOME should come from one of the referenced
EnvironmentFiles, but e.g. if your system is not of the "amd64" arch you may need to change the paths, both for the
java binary as well as the one to
Save, exit the editor and run:
and verify that everything is still working (Java listening on a low port,
java process being run as Tomcat).
While Tomcat is still running verify that the capabilities are there on the Java binary:
After you stop Tomcat again they should be gone (to be added dynamically again on each start):
Running a seperate server process only to proxy all requests to the servlet container is not ideal. At the very least donig that means that 2 servers/services must be correctly configured, integrated and running in order for 1 web server/service to function (i.e., doubling the potential for service disruptions). None of the approaches introduced above suffer from these issues, additionally proxying may introduce performance or timeout problems, problems with proper virtualization of requests, getting the correct IP addresses logged for clients, security issues with HTTP Request Headers set by an external web server / load balancer merely proxying HTTP to the container (which may be forged), etc.
Apache httpd and Tomcat
The example given here assumes use of AJP between Tomcat and httpd. TLS is offloaded to httpd. Tomcat only listens on port 8009 on the loopback interface (inaccessible to the outside world), httpd listens on port 443 for TLS (and maybe also 8443 for backchannel support).
In Tomcat's server.xml only configure an AJP Connector on the loopback interface and disable all other Connectors:
VirtualHostwith TLS (not covered here) and proxy requests to the
idpcontext to the servlet container (adjust to taste if your IDP runs in a different context):
If you offer support for a backchannel repeat the same on the backchannel 8443 vhost (using the IDP's backchannel key pair, not the webserver's SSL one), also making sure httpd does not get in the way with evaluation of client certificates by the IDP application (SSLVerifyClient optional_no_ca):
Webservers without AJP support
One example for this would be use of plain HTTP between the servlet container and the web server (e.g. Nginx, Lighttpd, Pound, etc.). TLS is offloaded to/terminated at the web server. The web server listens on port 443 and proxies all requests to the container, which is listening on port e.g. 8000 with a plain HTTP connector. The container must not be directly accessible from the outside world (e.g. by only listening on the loopback interface).
For backchannel support httpd may also listen on port 8443, proxying requests to the same container port (e.g. 8000) – but possibly a different port on the container might be needed (e.g. 8080) to tell the ports appart in the IDP application. EIther way, the keypair must differ for the backchannel port (i.e., the IDP's backchannel credentials. not web server TLS keys), as usual.
The container may need additional configuration to correctly virtualise the scheme (https vs. http), port (443 vs. 8000) and host name. E.g. for Apache Tomcat those are the
proxyName attributes on the Connector, respectively.
Load balancer, TLS-offloading appliance
Most of the considerations as in section "Webservers without AJP support" apply here as well. In addition to those:
- Since the load balancer will likely be on a different host/machine/system pay attention to properly virtualize the scheme, port and hostname in the container.
- For backchannel support it may be easier (or necessary) to configure TLS (with the IDP's backchannel credentials) on the container itself (e.g. with a HTTPS Connector on port 8443) and pass through port 8443 from the load balancer unmolested.
- Depending on the technologies involved (HTTP proxying, SNAT, etc.) the container may need additional configuration to get the correct information about the HTTP User Agent from the loadbalancer.
Systemd as included in e.g. RHEL/CentOS 7 (from systemd version 209 on) also includes a proxy server that inherits a socket activated by systemd (e.g. port 443, opened with root privileges), connects to a configured server port (e.g. the container listening on port 8443 with a HTTPS connector for web browser TLS use) and bi-directionally forwards all bits.
- This does not make use of TLS offloading, so the container will perform TLS itself (e.g. on port 9443 for web browser/front channel requests), though listening only on the loopback interface (Tomcat:
- The container needs to be properly virtualized, with the externally visible (logical) scheme and port. For Tomcat e.g.
scheme="https" proxyPort="443"on a
- For backchannel requests you'd forgo use of the systemd proxy and directly expose another HTTPS connector on the container, e.g. on port 8443, listening on all interfaces, with the IDP's backchannel key pair, of course.
- All proxy startup and tear down happens dynamically in the container's systemd unit file, so no additional processes need to be managed and monitored.
- But those external server/proxy processes still need to exist and bits will still need to be passed between 2 servers, which may incur some of the general proxying problems.