Agent-assisted captions and speech handoff

FileHop supports a complete agent-assisted caption workflow, without bundling its own speech model. It handles media decoding, small speech-audio preparation, timed-cue validation, subtitle files and supported video output. The connected agent supplies transcription using its own available speech model or service.

Use this instead of building a custom audio-extraction and subtitle-writing pipeline. Discover captions_audio and captions_write on the installed build first. They apply to audio recordings as well as videos.

1. Prepare audio for transcription

Call captions_audio with the original shared media path:

{
  "path": "/Users/me/Shared/interview.mp4",
  "format": "mp3"
}

The tool extracts the first audio track, converts it to 16,000 Hz mono, and writes temporary speech audio. MP3 is the default, encoded at 32 kbps: approximately 14 MB per hour, substantially smaller than uploading a whole video. format: "wav" requests 16-bit PCM for transcribers requiring WAV, with a larger file.

The result includes audioPath, audioId, sample rate, channel count, size, duration when known, and expiresAtMs. Use the returned path and ID; do not guess a temporary filename. Response keys are camelCase; the follow-up request uses audio_id.

This is speech preparation, not a high-fidelity music/mastering export. The 16 kHz setting is specific to this workflow; general audio conversion uses its own rate and codec rules.

2. Have the agent transcribe

Give the actual extracted audio file to the agent’s speech-to-text capability. Ask for timestamps in seconds from the original file’s start, with text for each cue. FileHop does not infer speech, speakers, words or translations on its own. If the agent lacks a transcriber, report that missing capability instead of inventing a transcript.

A local audioPath is not a public download URL and is not automatically readable by a remote service. The agent must have a supported way to read/upload local audio. Uploading to an external transcription provider sends audio outside the device; choose that provider according to the user’s instructions.

The temporary file has a two-hour lifetime for cleanup. Use it promptly; abandoned files become eligible for expiry cleanup. Do not use its path as a permanent project asset or deliverable.

3. Return validated captions

Call captions_write using the original media path, the returned audio ID, and exactly one of cues or subtitles:

{
  "path": "/Users/me/Shared/interview.mp4",
  "audio_id": "<audioId returned by captions_audio>",
  "cues": [
    { "start": 0.5, "end": 2.8, "text": "Welcome to the interview." },
    { "start": 3.1, "end": 5.0, "text": "Today we discuss the project." }
  ]
}

These cue lines illustrate the shape only; use text and timings obtained from the recording. Alternatively, subtitles accepts SRT or WebVTT text. The output is SRT regardless of which accepted input representation was used.

FileHop checks cue timing and text before writing. Negative/non-finite times, an end at or before its start, empty text and materially out-of-duration cues are refused with an actionable cue error. There is a small rounding allowance at the media’s end. Current bounds are 20,000 cues and 2,000 bytes of text per cue. Cues are sorted by start time; this does not prove transcription accuracy or resolve overlapping speech.

A validation failure writes no subtitle file. Fix the reported cue rather than silently dropping it. A successful write creates <media name>.srt beside the original, choosing a unique name if one already exists. Read the returned path, cue count and end time. Check audioRemoved for temporary-audio cleanup and any historyError separately from whether the SRT was saved.

The write needs permission in the media’s parent folder. Providing a writable destination to some other tool does not grant permission to this sidecar location.

4. Choose the caption deliverable

Desired result	Route	Important distinction
Captioned local playback	Keep the returned sidecar SRT beside the media	FileHop’s player/editor can pick up sidecar captions; use the actual returned file when collisions changed its name
Always-visible captions	`burn_subtitles` with the media and SRT	Re-encodes captions into the picture; they cannot be switched off
Editable video captions	`project_video_edit` caption operations in a `videoEditor` project	Caption track edits use the project’s IDs/revision and timeline; `captions_write` does not edit a project track
A different subtitle format	`convert` to SRT/VTT/ASS	Converting subtitle syntax does not translate speech or produce word-level timing
Existing embedded subtitles	`subtitles_extract`	Text subtitle extraction needs no transcription; bitmap subtitle tracks cannot be extracted as recognized text
Fix a consistent offset or drift	`subtitles_retime`	Uses time × scale + shift; not forced alignment or re-transcription

Styling and highlighting depend on the chosen playback/burn/project route. Do not promise karaoke highlighting or speaker-aware styling from plain cue-level SRT alone. Project text changes can discard obsolete word timings; inspect the current project schema when word-level timing matters.

Review and finish

Play or preview the beginning, the end and representative captioned segments, checking text accuracy, language, cue duration, synchronization, legibility and overlap with important visuals. A valid subtitle file is not evidence that the transcript is correct.

For a project handoff, keep captions editable, show the project in the human’s FileHop window and record only the range actually previewed. See live review and sessions. Preserve the original recording and report the returned caption/output paths.