RelayState is a term from SAMLTwodotZero that refers to a parameter supported by SAML bindings that pass SAML messages through a web browser. A request can include this information and the responder then returns the information with its response, allowing the requester to reconnect the response to any locally-relevant state.
SAMLOnedotOne doesn't formally have the concept of RelayState, but has a one-way mechanism, the TARGET parameter, used in both BrowserProfiles. It's usually been interpreted/used as being the URL of a resource at the ServiceProvider to access once a session is built.
When Shibboleth developed a custom AuthnRequest message to initiate the BrowserProfiles, a target parameter was defined to match the response parameter (the case is unfortunately different, but they correspond). The initial Shibboleth implementations embedded the URL of a resource in it.
Starting with ShibOnedotThree, a new property in ShibbolethXml (=/SPConfig/Local/@localRelayState=) exists to cache the URL locally in a cookie and send a fixed value to the IdentityProvider. The fixed value is arbitrary (an implementation detail) and is a signal to the ServiceProvider to retrieve the URL from the cookie and redirect to it.
Secondly, a new property
/SPConfig/Applications/@=homeURL is used to establish a default location to redirect the user if nothing better can be found. An IdentityProvider can set a TARGET value of
default to rely on this behavior, but this depends on knowing the ServiceProvider is running a new enough version (so it's application-specific, but suitable for those creating such links).
Going forward, I would expect this to be the model used, and that we would generally not use the RelayState mechanism defined in that specification.