This article is co-authored with generative AI. While I have cross-checked facts against official documentation where possible, errors may remain. Please verify primary sources before making important decisions.

TL;DR

  • I shipped per-canvas OCR text as IIIF Presentation 3 annotations with motivation: "supplementing". Annona and other spec-aware viewers rendered them. Mirador 4.0.0 (projectmirador.org/embed) showed nothing in the Annotations side panel.
  • Root cause: in Mirador 4.0.0 release, config.annotations.filteredMotivations defaults to ['oa:commenting', 'oa:tagging', 'sc:painting', 'commenting', 'tagging'] — supplementing is not in the allowlist, so an annotation whose only motivation is supplementing is filtered out of the side panel.
  • Mirador master has since flipped the default to filteredMotivations: [] (no filter), but at the time of writing no release ships that change. The released filter still applies in the wild.
  • Putting motivation as ["commenting", "supplementing"] lets the annotation through Mirador 4.0.0’s filter via commenting, while keeping supplementing so spec-aware viewers / text-overlay plugins still recognise it as a transcription.
  • IIIF Presentation 3 §3.5 “Values for motivation” only defines painting and supplementing; other motivation terms (commenting, tagging, …) come from the Web Annotation Data Model. The basis for allowing arrays is IIIF §4.3 (Properties with Multiple Values) plus Web Annotation §3.3.5.
  • Don’t add painting alongside. Each canvas already has a painting annotation for the image — adding another forces the viewer to pick which one “fills” the canvas, with implementation-dependent results.

What happened

While extending the public API for the Hiraga Yuzuru Digital Archive, I added a per-canvas OCR-text endpoint that returns annotations as a Presentation 3 AnnotationPage, referenced from each canvas’s annotations[]:

// /api/iiif/{id}/manifest.json (excerpt)
{
  "items": [
    {
      "id": ".../canvas/p1",
      "type": "Canvas",
      "items": [ /* the painting annotation for the canvas image */ ],
      "annotations": [
        { "id": ".../canvas/p1/annotations", "type": "AnnotationPage" }
      ]
    }
  ]
}

// /api/iiif/{id}/canvas/p1/annotations (initial version)
{
  "@context": "http://iiif.io/api/presentation/3/context.json",
  "id": ".../canvas/p1/annotations",
  "type": "AnnotationPage",
  "items": [
    {
      "id": ".../canvas/p1/annotations/anno-1",
      "type": "Annotation",
      "motivation": "supplementing",
      "body": {
        "type": "TextualBody",
        "language": "ja",
        "format": "text/plain",
        "value": "製造費ノ減少ニ就テ"
      },
      "target": ".../canvas/p1#xywh=4875,1610,88,1282"
    }
  ]
}

The motivation value is supplementing, which is what IIIF Cookbook recipes 0068 (Newspaper) and 0231 (Transcripts/Captions meta-recipe) recommend for transcriptions / OCR. Annona renders these with no issue.

But in Mirador 4.0.0 (https://projectmirador.org/embed/?iiif-content=...), opening the Annotations side panel shows nothing. The manifest loads, the image renders, IIIF Content Search works — only the Annotations panel is empty.

As a sanity check, the canonical IIIF Cookbook 0068 manifest itself shows the same empty-panel behaviour in Mirador 4.0.0 embed (try collection / issue manifest direct). If a canonical spec example doesn’t render in the panel, the cause has to be inside the library, not in our manifest.

Tracking it down — read the deployed Mirador bundle

My first instinct was to grep Mirador’s source for the panel’s filter logic. The current master branch’s src/config/settings.js reads:

annotations: {
  htmlSanitizationRuleSet: 'iiif',
  // filteredMotivations: if empty, all annotation motivations will be shown.
  filteredMotivations: [],
},

— no filter by default. That contradicted observation, so the next step was to actually read the JavaScript that projectmirador.org/embed is serving:

$ curl -sS https://projectmirador.org/embed/ | grep -oE 'src="[^"]+\.js"'
src="/assets/embed-BxGPuA7M.js"

$ curl -sS https://projectmirador.org/assets/embed-BxGPuA7M.js | grep -oE 'mirador[^"]{0,40}\.js'
mirador.es-Bh8H5vm3.js

$ curl -sS https://projectmirador.org/assets/mirador.es-Bh8H5vm3.js \
   | grep -oE 'filteredMotivations:\[[^]]+\]'
filteredMotivations:["oa:commenting","oa:tagging","sc:painting","commenting","tagging"]

The deployed bundle has filteredMotivations explicitly set to a non-empty list. supplementing is not in it.

Comparing the released settings.js to master makes the picture clear:

mirador@v4.0.0  src/config/settings.js
  filteredMotivations: ['oa:commenting','oa:tagging','sc:painting','commenting','tagging']

mirador@master (unreleased)  src/config/settings.js
  filteredMotivations: []

So Mirador 4.0.0 ships a default that filters out supplementing, and master has switched to “show everything” but hasn’t been released. projectmirador.org/embed runs the released bundle, so the filter is active.

The filtering selector itself is in src/state/selectors/annotations.js and is recognisable in the minified bundle:

const getMotivations = createSelector(
  [getConfig, (state, { motivations }) => motivations],
  (config, motivations) => motivations || config.annotations.filteredMotivations,
);

export const getAnnotationResourcesByMotivationForCanvas = createSelector(
  [getPresentAnnotationsCanvas, getMotivations],
  (annotations, motivations) => {
    const resources = flatten(annotations.map(a => a.resources));
    if (!motivations || motivations.length === 0) return resources;
    return resources.filter(r => r.motivations.some(m => motivations.includes(m)));
  },
);

In Mirador 4.0.0 filteredMotivations is non-empty, so r.motivations.some(m => motivations.includes(m)) runs and an annotation whose only motivation is supplementing gets dropped.

Fix — make motivation an array

motivation accepts multiple values per the IIIF / Web Annotation specs, so we list both: commenting to pass Mirador 4.0.0’s filter, and supplementing to keep spec-aware viewers / text-overlay plugins recognising the annotation as a transcription.

{
  "motivation": ["commenting", "supplementing"],
  "body": {
    "type": "TextualBody",
    "language": "ja",
    "format": "text/plain",
    "value": "製造費ノ減少ニ就テ"
  },
  "target": ".../canvas/p1#xywh=4875,1610,88,1282"
}

The implementation change is just swapping a string literal for an array literal. In TypeScript, remember to update the type annotation from 'supplementing' to a tuple type ['commenting', 'supplementing'] as well.

After the change, the Mirador 4.0.0 panel populates as expected.

Spec basis — where multiple motivation values are sanctioned

IIIF Presentation 3 itself is fairly terse on motivation’s value space and cardinality, so the authoritative answer crosses over to the Web Annotation Data Model:

  • IIIF Presentation 3 §3.5 “Values for motivation” defines only two values: painting and supplementing. The third paragraph delegates other values to the Web Annotation Data Model: “Other motivation values given in the Web Annotation specification SHOULD be used.”
  • IIIF §4.3 “Properties with Multiple Values”: properties that allow multiple values must be expressed as an array even when there is only one value.
  • Web Annotation Data Model §3.3.5 “Motivation and Purpose”: “There SHOULD be exactly 1 motivation for each Annotation, and MAY be 0 or more than 1.” — i.e. zero, one, or multiple motivations are all permitted.

commenting / tagging / bookmarking / etc. live in the Web Annotation Data Model §3.3.5 vocabulary, not in IIIF itself. Mirador 4.0.0’s default filter list ['oa:commenting','oa:tagging','sc:painting','commenting','tagging'] includes both the legacy oa: / sc: (Open Annotation / shared canvas) prefixed names and the unprefixed Web Annotation 1.0 forms.

Why not just add painting?

You might be tempted to slap painting motivation onto the OCR annotations so the text “renders on the canvas”. Don’t — there’s already a painting annotation per canvas for the image:

{
  "id": ".../canvas/p1",
  "type": "Canvas",
  "items": [
    {
      "type": "AnnotationPage",
      "items": [
        {
          "type": "Annotation",
          "motivation": "painting",
          "body": { "type": "Image", "id": ".../full/full/0/default.jpg", ... },
          "target": ".../canvas/p1"
        }
      ]
    }
  ]
}

Adding another painting annotation that targets the same canvas means the viewer has to pick which one “fills” the canvas. The spec permits it, but the rendering result is implementation-dependent:

  • last-write-wins, with the image disappearing under empty text boxes,
  • text painted as empty rectangles that obscure the image,
  • both rendered overlapping and unreadable,

— none of which are what you want. OCR text supplements the image rather than replacing it, so supplementing (with commenting tacked on for Mirador 4.0.0 panel compatibility) is the right framing.

Checklist — Mirador 4.0.0 compatibility for OCR annotations

To make per-canvas OCR text show up in Mirador 4.0.0’s Annotations panel:

  1. Use motivation: ["commenting", "supplementing"] (array)
    • The released Mirador 4.0.0 default for filteredMotivations excludes supplementing, so a single-value supplementing annotation never reaches the panel.
    • master has changed the default to [], so once that ships either form will work — but keep both for backward compatibility.
  2. Don’t add painting (collides with the canvas’s image painting annotation).
  3. The annotation’s target must reference the canvas as it appears in the manifest (matching canvas.id exactly), with #xywh= to scope the region.
  4. Coordinates must be in the same pixel space as the canvas’s width/height in the manifest.
    • If the OCR was run on a downsampled image (e.g. 1500×1065) but the canvas exposes the full IIIF source dimensions (e.g. 6944×4928), scale xywh up before emitting.

This is a mismatch between spec recommendations (supplementing for transcription) and a release-default in the most popular reference viewer (Mirador 4.0.0). Emitting the array form from the start avoids the trap. A future Mirador 4 release that picks up the master change should make the workaround unnecessary.

References