Whisper

Auto-Generating English Subtitles and Audio for Videos with Azure OpenAI Whisper + Speech Services

I have summarized how to automatically add English subtitles and English audio to Japanese videos. This uses Azure OpenAI Service’s Whisper and Speech Services. Overview The goal this time is to make a Japanese audio video multilingual as follows: Japanese version: Original video (Japanese audio, no subtitles) English version: English audio + English subtitles Services Used Service Purpose Azure OpenAI Service (Whisper) Translation from Japanese audio to English text Azure Speech Services (TTS) Synthesis from English text to English audio FFmpeg Audio extraction and video merging Procedure 1. Environment Setup Required Tools # Install FFmpeg (macOS) brew install ffmpeg # Python libraries pip install python-dotenv requests Azure Configuration (.env) AZURE_OPENAI_ENDPOINT=https://xxxxx.openai.azure.com AZURE_OPENAI_API_KEY=your-api-key AZURE_OPENAI_DEPLOYMENT_NAME=whisper AZURE_OPENAI_API_VERSION=2024-06-01 2. Extract Audio from Video Since the Azure Whisper API has a 25MB file size limit, the audio is compressed and extracted. ...

January 21, 2026 · Updated: January 21, 2026 · 3 min · Nakamura

Adding Images to IIIF Manifest Files for Audio Materials

Overview This is a note on trying out the Audio Presentation with Accompanying Image recipe. https://iiif.io/api/cookbook/recipe/0014-accompanyingcanvas/ The following is an example displayed in Clover, where the configured image appears in the player. https://samvera-labs.github.io/clover-iiif/docs/viewer/demo?iiif-content=https://nakamura196.github.io/ramp_data/demo/3571280/manifest.json Manifest File Description An example is stored at the following location. https://github.com/nakamura196/ramp_data/blob/main/docs/demo/3571280/manifest.json Specifically, it was necessary to add an accompanyingCanvas to the Canvas as follows. { "id": "https://nakamura196.github.io/ramp_data/demo/3571280/canvas", "type": "Canvas", "duration": 156.07999999999998, "accompanyingCanvas": { "id": "https://nakamura196.github.io/ramp_data/demo/3571280/canvas/accompanying", "type": "Canvas", "height": 1024, "width": 1024, "items": [ { "id": "https://nakamura196.github.io/ramp_data/demo/3571280/canvas/accompanying/annotation/page", "type": "AnnotationPage", "items": [ { "id": "https://nakamura196.github.io/ramp_data/demo/3571280/canvas/accompanying/annotation/image", "type": "Annotation", "motivation": "painting", "body": { "id": "https://nakamura196.github.io/ramp_data/demo/3571280/3571280_summary_image.jpg", "type": "Image", "height": 1024, "width": 1024, "format": "image/jpeg" }, "target": "https://nakamura196.github.io/ramp_data/demo/3571280/canvas/accompanying/annotation/page" } ] } ] }, It may be a bit hard to follow, but here is the code example using iiif_prezi3. The accompanyingCanvas is created via create_accompanying_canvas() and associated with the canvas. ...

July 12, 2024 · Updated: July 12, 2024 · 2 min · Nakamura

IIIF Audio/Visual: Describing Multiple VTT Files

Overview This is a note on how to describe multiple VTT files for Audio/Visual materials using IIIF. Here, we describe transcription text in both Japanese and English as shown below. https://ramp.avalonmediasystem.org/?iiif-content=https://nakamura196.github.io/ramp_data/demo/3571280/manifest.json Manifest File Description An example is stored at the following location. https://github.com/nakamura196/ramp_data/blob/main/docs/demo/3571280/manifest.json Please also refer to the following article. Specifically, by describing them as multiple annotations as shown below, they were correctly processed by the Ramp viewer. ...

July 12, 2024 · Updated: July 12, 2024 · 2 min · Nakamura

Displaying Audio Files with Subtitles in an IIIF Viewer

Overview I had the opportunity to display audio files with subtitles in an IIIF viewer, so this is a memo. The target is “Accents and Intonation of the Japanese Language (Part 2)” published in the National Diet Library Historical Sound Archive. OpenAI’s Speech to text was used. Please note that the transcription results may contain errors. The following is a display example in Ramp. https://ramp.avalonmediasystem.org/?iiif-content=https://nakamura196.github.io/ramp_data/demo/3571280/manifest.json The following is a display example in Clover. ...

July 10, 2024 · Updated: July 10, 2024 · 2 min · Nakamura