I developed an iOS app called “IIIF AR” that places high-resolution images served via the IIIF Image API at real-world scale on floor surfaces using ARKit, dynamically loading finer detail as the camera moves closer. Development was assisted by Anthropic’s Claude Code.

App Overview

IIIF AR is an augmented reality app that places IIIF Image API-compatible images at real-world scale on floor surfaces detected by the iPhone camera.

Key features include:

  • Real-scale placement: Images are placed at accurate physical dimensions in AR space, calculated from pixel size and actual dimensions (cm)
  • Tile-based deep zoom: As the camera approaches, high-resolution tiles are dynamically fetched using the IIIF Image API’s region parameter, loading only the visible area
  • Placement preview: Red corner markers show the intended placement position before tapping
  • Corner poles: Yellow poles at the four corners help gauge the image size even in cluttered environments

Sample Images

Three historical documents related to the Ryukyu Kingdom are included as samples.

NameCollectionSize
Kaitō ShokokukiHistoriographical Institute, The University of Tokyo32.6×21.2 cm
Ryūkyū-koku no ZuOkinawa Churashima Foundation96.8×47.3 cm
Ryūkyū-koku ZuOkinawa Prefectural Museum & Art Museum87.8×175.8 cm

“Ryūkyū-koku Zu” is a hanging scroll measuring 175.8 cm in height, which is close to a person’s height when placed in AR space.

Tech Stack

  • Language: Swift 5.9 / SwiftUI
  • AR: ARKit + RealityKit
  • Image delivery: IIIF Image API 2.0
  • In-app purchases: StoreKit 2 (Tip Jar)
  • Project management: XcodeGen
  • Development tool: Claude Code (Anthropic)

Architecture

AR Placement Pipeline

Floor detection → Placement preview → Tap to place → Tile loading
  1. Horizontal planes are detected using ARWorldTrackingConfiguration.
  2. Images are anchored to detected ARPlaneAnchor instances for stable tracking.
  3. A low-resolution full image (1024px) is always displayed as the base layer.
  4. Higher-resolution tiles are overlaid on top as the camera distance decreases.

IIIF Tile System

The tile-based deep zoom consists of the following seven files.

Tile/
  TileKey.swift           — Tile identifier (scaleFactor, tileX, tileY)
  TileGrid.swift          — Geometry calculations for IIIF tile coordinates
  ZoomLevelMapper.swift   — Camera distance to scale factor mapping
  FrustumCuller.swift     — Visibility test between camera frustum and tiles
  TileTextureCache.swift  — NSCache-based texture cache
  TileFetcher.swift       — Async tile download (max 4 concurrent)
  TileManager.swift       — Orchestrator

Distance and Scale Factor Mapping

DistanceScale FactorDescription
> 3.0m128Entire image in one tile
1.0–1.5m16Standing viewing distance
0.5–0.7m4Crouching distance
≤ 0.3m1Maximum resolution

IIIF URL Construction Example

For sho.tif (25,167×12,483px) at scale factor 8, tile (0,0):

{baseURL}/0,0,4096,4096/512,512/0/default.jpg

For an edge tile (right edge), the region is clamped:

{baseURL}/24576,0,591,4096/74,512/0/default.jpg

Persistent Base Layer Display

During development, there was an issue where the base image would disappear while tiles were loading. Two causes were identified.

  1. Occlusion by transparent tile placeholders: Tile entities with zero opacity were writing to RealityKit’s depth buffer, occluding the base image beneath. This was resolved by only creating entities for tiles whose textures are ready.

  2. Tile deletion on pole toggle: Code that removed “all child elements except imageEntity” when toggling debug corner poles was inadvertently removing tile entities as well. This was resolved by tracking poleEntities and edgeEntities individually.

Development with Claude Code

Development utilized the “agent” capability of Claude Code.

Parallel Development Team

Up to 20 agents were run simultaneously, with the following division of responsibilities.

Implementation team:

  • App icon generation
  • IIIF image caching, parallelization, and retry logic
  • Placement preview (red corner markers)
  • Haptic feedback
  • Camera permission checks
  • Tip Jar (StoreKit 2)
  • Localization (Japanese/English)

Review team:

  • TileGrid/TileKey accuracy verification
  • FrustumCuller math verification
  • IIIF tile URL live testing
  • ARViewContainer code quality review
  • Project configuration review

Notable Bugs Found by the Review Team

IssueCauseFix
Tiles disappear when covering the screenFrustumCuller only tested the 4 corners, failing to detect when a tile encloses the screenAdded center point test + reverse test
Tiles disappear on pole toggleFull child removal caught tilesIndividual edgeEntities tracking
TileFetcher CPU wasteSpin loop (busy wait)Immediate nil return approach
Transparent tiles occlude base imageDepth buffer writesCreate entities only after texture is ready

Icon Design Trial and Error

Claude Code can generate images programmatically (Pillow, SVG), but it did not seem well-suited for creating design-quality icons. The final icon was generated using Gemini’s image generation. Following Apple HIG guidelines, the following points were considered:

  • Provided as a square (iOS applies rounded corner mask automatically)
  • Background color fills to all four corners (no white gaps)
  • No text
  • Minimal number of elements

Source Code

The source code is available at:

References