Still Aggregating

This morning, I was having a conversation with a colleague about the UK federation’s metadata publication system. We needed to reference the order of operations and I remembered that I had once published an article about this system, along with a diagram illustrating just the point under consideration.

Pulling up the article in question, I was slightly surprised to find that the original article was published exactly ten years ago today.

The original architecture of this system was deliberately flexible, because we didn’t really know for sure the environment we’d be operating in. It’s gratifying, therefore, to see that the basic design has held up well enough that we can still use the article as an informal reference.

In detail, of course, things are different now than we anticipated and the deployed system has aspects which are simpler than a decade ago, as well as others which have required some elaboration. At present, for example, eduGAIN has been successful enough that it is currently the only active source of metadata other than that registered by the UK federation itself. On the other side, we produce a number of additional outputs today that we hadn’t really considered at the time, the most important of those probably being that we now publish per-entity metadata as well as the aggregate metadata illustrated.

Another major change – one that happened in 2013, about a year after the article was published – was that we started publishing the tooling itself in a GitHub repository for the benefit of other organisations working in the same area. I remember the big challenge there being that the original repository (which dated back to 2004) contained customer data; the version we published had to be filtered using git filter-branch for privacy (see the ukf-meta-meta repository for the gory details). We’ve since rebased our development so that our internal repository matches the published one except that we don’t expose development work in progress. I’m really glad not to be reliant on filter-branch any more, and I’m sure any reader who has dipped a toe into those waters will sympathise.

Anyway: Happy birthday, helpful diagram!