--- title: Agent-assisted captions and speech handoff description: Prepare small speech audio for an agent, transcribe with its model, validate timed subtitles and hand off sidecar or editable captions. order: 10 --- # Agent-assisted captions and speech handoff FileHop supports a complete **agent-assisted caption workflow**, without bundling its own speech model. It handles media decoding, small speech-audio preparation, timed-cue validation, subtitle files and supported video output. The connected agent supplies transcription using its own available speech model or service. Use this instead of building a custom audio-extraction and subtitle-writing pipeline. Discover `captions_audio` and `captions_write` on the installed build first. They apply to audio recordings as well as videos. ## 1. Prepare audio for transcription Call `captions_audio` with the original shared media path: ```json { "path": "/Users/me/Shared/interview.mp4", "format": "mp3" } ``` The tool extracts the **first audio track**, converts it to **16,000 Hz mono**, and writes temporary speech audio. MP3 is the default, encoded at 32 kbps: approximately 14 MB per hour, substantially smaller than uploading a whole video. `format: "wav"` requests 16-bit PCM for transcribers requiring WAV, with a larger file. The result includes `audioPath`, `audioId`, sample rate, channel count, size, duration when known, and `expiresAtMs`. Use the returned path and ID; do not guess a temporary filename. Response keys are camelCase; the follow-up request uses `audio_id`. This is speech preparation, not a high-fidelity music/mastering export. The 16 kHz setting is specific to this workflow; general audio conversion uses its own [rate and codec rules](/docs/media/). ## 2. Have the agent transcribe Give the actual extracted audio file to the agent's speech-to-text capability. Ask for timestamps in **seconds from the original file's start**, with text for each cue. FileHop does not infer speech, speakers, words or translations on its own. If the agent lacks a transcriber, report that missing capability instead of inventing a transcript. A local `audioPath` is not a public download URL and is not automatically readable by a remote service. The agent must have a supported way to read/upload local audio. Uploading to an external transcription provider sends audio outside the device; choose that provider according to the user's instructions. The temporary file has a two-hour lifetime for cleanup. Use it promptly; abandoned files become eligible for expiry cleanup. Do not use its path as a permanent project asset or deliverable. ## 3. Return validated captions Call `captions_write` using the **original media path**, the returned audio ID, and exactly one of `cues` or `subtitles`: ```json { "path": "/Users/me/Shared/interview.mp4", "audio_id": "", "cues": [ { "start": 0.5, "end": 2.8, "text": "Welcome to the interview." }, { "start": 3.1, "end": 5.0, "text": "Today we discuss the project." } ] } ``` These cue lines illustrate the shape only; use text and timings obtained from the recording. Alternatively, `subtitles` accepts SRT or WebVTT text. The output is SRT regardless of which accepted input representation was used. FileHop checks cue timing and text before writing. Negative/non-finite times, an end at or before its start, empty text and materially out-of-duration cues are refused with an actionable cue error. There is a small rounding allowance at the media's end. Current bounds are 20,000 cues and 2,000 bytes of text per cue. Cues are sorted by start time; this does not prove transcription accuracy or resolve overlapping speech. A validation failure writes no subtitle file. Fix the reported cue rather than silently dropping it. A successful write creates `.srt` beside the original, choosing a unique name if one already exists. Read the returned `path`, cue count and end time. Check `audioRemoved` for temporary-audio cleanup and any `historyError` separately from whether the SRT was saved. The write needs permission in the media's parent folder. Providing a writable destination to some other tool does not grant permission to this sidecar location. ## 4. Choose the caption deliverable | Desired result | Route | Important distinction | | --- | --- | --- | | Captioned local playback | Keep the returned sidecar SRT beside the media | FileHop's player/editor can pick up sidecar captions; use the actual returned file when collisions changed its name | | Always-visible captions | `burn_subtitles` with the media and SRT | Re-encodes captions into the picture; they cannot be switched off | | Editable video captions | `project_video_edit` caption operations in a `videoEditor` project | Caption track edits use the project's IDs/revision and timeline; `captions_write` does not edit a project track | | A different subtitle format | `convert` to SRT/VTT/ASS | Converting subtitle syntax does not translate speech or produce word-level timing | | Existing embedded subtitles | `subtitles_extract` | Text subtitle extraction needs no transcription; bitmap subtitle tracks cannot be extracted as recognized text | | Fix a consistent offset or drift | `subtitles_retime` | Uses time × scale + shift; not forced alignment or re-transcription | Styling and highlighting depend on the chosen playback/burn/project route. Do not promise karaoke highlighting or speaker-aware styling from plain cue-level SRT alone. Project text changes can discard obsolete word timings; inspect the current project schema when word-level timing matters. ## Review and finish Play or preview the beginning, the end and representative captioned segments, checking text accuracy, language, cue duration, synchronization, legibility and overlap with important visuals. A valid subtitle file is not evidence that the transcript is correct. For a project handoff, keep captions editable, show the project in the human's FileHop window and record only the range actually previewed. See [live review and sessions](/docs/human-review-sessions/). Preserve the original recording and report the returned caption/output paths.