Monitoring an MDQ Server

This document shows how to use a library of bash scripts to monitor a metadata query (MDQ) server.

Getting Started

Skim some related documentation on GitHub: Monitoring an HTTP Resource
Install the bash library (see the bash library project page for details)
Configure the bash environment (see the bash library project page for details)

Testing a Single Entity

Experiment 1: Given an IdP entityID, test a single metadata resource on the MDQ server.

Required info:

An IdP entityID
An MDQ base URL

An MDQ base URL is needed to configure a Shibboleth DynamicHTTPMetadataProvider as well.

The following values are for illustration only:

Initialize parameters

$ entityID=https://webauth.umass.edu/idp/shibboleth
$ mdq_base_url=http://mdq-beta.incommon.org/global

Compute the corresponding MDQ protocol request URL:

Compute the MDQ protocol URL

$ location=$( $BIN_DIR/mdq_url.bash $mdq_base_url $entityID )
$ echo $location
http://mdq-beta.incommon.org/global/entities/https%3A%2F%2Fwebauth.umass.edu%2Fidp%2Fshibboleth

Test the MDQ server by requesting entity metadata:

Run the test

$ $BIN_DIR/http_response_stats.bash -n 1 $location

The previous command outputs the following JSON file on stdout:

Default output

[
 {
   "requestInstant": "2018-04-12T21:20:52Z"
   ,
   "friendlyDate": "April 12, 2018"
   ,
   "curlExitCode": "0"
   ,
   "responseCode": "200"
   ,
   "sizeDownload": 9160
   ,
   "speedDownload": 42670.000
   ,
   "timeTotal": 0.214669
 }
]

Limit the size of the JSON array to one object but output all available timing data:

Run the test again

$ $BIN_DIR/http_response_stats.bash -n 1 -a $location

The previous command outputs the following JSON file on stdout:

All output

[
 {
   "requestInstant": "2018-04-12T21:24:06Z"
   ,
   "friendlyDate": "April 12, 2018"
   ,
   "curlExitCode": "0"
   ,
   "responseCode": "200"
   ,
   "sizeDownload": 9160
   ,
   "speedDownload": 93969.000
   ,
   "timeNamelookup": 0.005276
   ,
   "timeConnect": 0.050664
   ,
   "timeAppconnect": 0.000000
   ,
   "timePretransfer": 0.050761
   ,
   "timeStarttransfer": 0.096721
   ,
   "timeTotal": 0.097478
 }
]

For more information, consult the script’s inline help message:

Display help message

$ $BIN_DIR/http_response_stats.bash -h

Monitoring the MDQ Server

Set up a cron job for the previous test.

Cron job

#!/bin/bash
#######################################################################
#
# This script is intended to be used as a cron job.
#
# Configure the following environment variables:
# (also export TMPDIR if it doesn’t already exist)
#
# export BIN_DIR="/path/to/bin/"
# export LIB_DIR="/path/to/lib/"
# export CACHE_DIR="/path/to/http_cache/"
# export LOG_FILE="/path/to/bash_log.txt"
#
# Configure the following local variables:
#
# entityID=
# mdq_base_url=
# out_dir=
#
#######################################################################

# the name of this script
script_name=${0##*/}

# determine the HTTP location
location=$( $BIN_DIR/mdq_url.bash $mdq_base_url $entityID )
status_code=$?
if [ $status_code -ne 0 ]; then
	echo "ERROR: $script_name: unable to compute location" >&2
	exit 2
fi

# adjust the command line with options -n and -a as desired
$BIN_DIR/http_response_stats.bash -d $out_dir $location
status_code=$?
if [ $status_code -ne 0 ]; then
	echo "ERROR: $script_name: unable to monitor location: $location" >&2
	exit 3
fi

exit 0