IIIF Update: Interoperability for Audio & Video ResourcesBy Sheila Rabun posted
Editor's note: This post continues our series on CLIR Affiliates, which Chuck Henry initiated in his June 22 blog. In coming weeks, we will feature updates on our work with the Digital Library of the Middle East, National Digital Stewardship Alliance, and Jisc.
The International Image Interoperability Framework (IIIF) community recently convened in the Vatican for the 2017 IIIF Conference, where 275 people from 130 institutions and organizations in 18 countries participated in the week’s events. In addition to the general sharing of IIIF use cases, implementations, and knowledge, the conference provided a venue for the IIIF Community Groups and IIIF Technical Specification Groups to meet face to face. Particularly of note is the work of the IIIF A/V Technical Specification Group to extend the many benefits of IIIF to audio and video resources, as the IIIF Community works toward version 3.0 of the IIIF Presentation API.
IIIF provides a technical mechanism for leveraging existing technologies to enable interoperable functionality for digital images and digital image repositories within and across institutions. IIIF allows users to interact with digital images by zooming in deeply, easily panning across an image, comparing multiple images side by side (including images from different repositories), annotating images with notes, transcriptions, related resources, and external links. The IIIF Image API, Presentation API, Content Search API, and Authentication API are designed to provide the framework for transfer of image pixels and image presentation data, using annotations to display digital image content and related resources on a “shared canvas” with a given width and height. The Shared Canvas Data Model outlines the basics of the IIIF canvas concept and use of annotations in the IIIF Presentation API.
As video and audio resources are increasingly digitized for access and preservation, cultural heritage institutions are now looking to IIIF to provide the same advanced user functionality and improved repository maintenance benefits for digital A/V collections as for still images.
Use cases for IIIF A/V have been collected from the across the IIIF community, revealing needs for interoperable A/V functionality such as the ability to:
- annotate and display annotations on a video and/or audio file
- compare multiple A/V resources alongside each other
- play multiple synchronized videos at the same time
- play an audio recording (or multiple audio recordings) of a piece of music, while using hit-highlighting to follow along with images of the corresponding musical score
- highlight or specify a region or point of interest in an A/V file
The IIIF A/V Technical Specification Group, chaired by Tom Crane, technical director at Digirati, and Jon Dunn, assistant dean for library technologies at Indiana University, has modeled these use cases in accordance with the IIIF Presentation API specification, adding a time dimension in order to include A/V resources in the landscape of interoperable digital resources on the web. The group’s charter outlines the scope and timeline of this work. So far, the group has modeled multiple A/V use cases and is now experimenting with implementation in software to test and refine.
The British Library recently received a grant from The Andrew W. Mellon Foundation to lead the effort to establish interoperability for A/V resources via IIIF, as part of the Library’s “Save our Sounds” A/V digitization project. Proof of concept tests have been constructed by Masahide Kanzaki, an expert in Linked Open Data (LOD) and author of several textbooks on LOD in Japan. Notably, the “Fire” example shows an image that stays on the canvas for a duration of time while text, A/V content, and digital images are annotated onto the canvas as different points in time. A different example shows a video with text annotations that appear at various points in time. Experiments have also been conducted with the Avalon Media System, an open source A/V resource management software developed and maintained by Northwestern University and Indiana University libraries, based on Samvera and Fedora technologies. As the Avalon development team works toward version 7.0 of the software, they aim to include IIIF support for A/V resources, with help from a new grant from the US Institute of Museum and Library Services. Updates from the IIIF A/V Technical Specification Group can be found on the IIIF Discuss email list. Independent experiments are encouraged and the group welcomes any and all feedback via the email list.
For a step by step introduction to the core concepts involved in IIIF, I recommend this easy-to-read blog series by Tom Crane:
- An Introduction to IIIF
- But where’s my model? IIIF and your metadata
- Annotations: How IIIF resources get their content
- IIIF Search and Discovery
- A/V and beyond – coming soon
 Videos from the IIIF Showcase prior to the conference are available on the IIIF YouTube channel.
Sheila Rabun is Community and Communications Officer for the International Image Interoperability Framework (IIIF) Consortium.