Simultaneous interpretation
Models and usage
List models available to your key
curl 'https://api.anycap.io/v1/audio/ast/models' -H "Authorization: Bearer $ANYCAP_API_KEY"Choose a resource_id, connect via WebSocket and send PCM. Use s2t for captions or s2s for captions and translated audio.
WebSocket endpoint
API clients connect to wss://api.anycap.io/v1/audio/ast/stream and send Authorization: Bearer <connection key> on the handshake. Public API clients must omit Cookie and Origin headers. List sessions with GET /v1/audio/ast/sessions and read one with GET /v1/audio/ast/sessions/{session_id} using the same key. The console page connects to wss://<console API host>/api/v1/capabilities/audio.ast/stream with its session cookie: a browser socket cannot send request headers, and the server only accepts an allowlisted Origin.
import WebSocket from "ws";import { readFileSync, writeFileSync, appendFileSync } from "node:fs"; const base = process.env.ANYCAP_API_BASE || "https://api.anycap.io";const headers = { Authorization: `Bearer ${process.env.ANYCAP_API_KEY}` };const response = await fetch(`${base}/v1/audio/ast/models`, { headers });if (!response.ok) throw new Error(`Discovery failed: ${response.status}`);const { items } = await response.json();const source = items.find((item) => item.resource_id === process.env.ANYCAP_AST_SOURCE_ID) || items[0];if (!source) throw new Error("No authorized interpretation source"); // Raw 16 kHz mono s16le PCM, without a WAV header. Install: npm install wsconst pcm = readFileSync(process.env.ANYCAP_PCM_FILE || "speech.pcm");const seconds = Math.min(60, source.max_session_seconds);if (!pcm.length || pcm.length % 2 || pcm.length > seconds * 32000) { throw new Error("PCM must contain whole samples within the session limit");}const url = new URL("/v1/audio/ast/stream", base);url.protocol = url.protocol === "https:" ? "wss:" : "ws:";url.search = new URLSearchParams({ resource_id: source.resource_id, mode: "s2s", source: "zh", target: "en", seconds: String(seconds), audio_format: "pcm", audio_rate: "24000",}).toString();const socket = new WebSocket(url, { headers }); // No Cookie or Origin.let timer;let terminal = false;writeFileSync("translated.pcm", Buffer.alloc(0));socket.on("message", (data, binary) => { if (binary) return appendFileSync("translated.pcm", data); const event = JSON.parse(data.toString()); if (event.type === "status" && !timer) { // Start audio only after status, then send at approximately real-time speed. let offset = 0; timer = setInterval(() => { if (offset >= pcm.length) { clearInterval(timer); socket.send(JSON.stringify({ type: "finish" })); return; } socket.send(pcm.subarray(offset, offset + 2560)); offset += 2560; }, 80); } if (event.type === "subtitle") console.log(event.channel, event.phase, event.text || ""); if (event.type === "done" || event.type === "error") { terminal = true; clearInterval(timer); console.log(event); socket.close(); }});socket.on("unexpected-response", (_request, response) => { console.error("Handshake rejected:", response.statusCode); response.setEncoding("utf8"); response.on("data", (body) => console.error(body));});socket.on("error", (error) => console.error(error.message));socket.on("close", () => { clearInterval(timer); if (!terminal) console.error("Connection ended without a terminal event; query the session before retrying.");});Simultaneous interpretation streams speech in over a WebSocket and returns bilingual captions plus translated speech on the same socket. Microphone audio goes up as binary frames; captions arrive as JSON events and translated audio comes back as binary frames.
Query parameters
resource_id
/v1/audio/ast/models. Required.mode
s2s returns translated speech and captions; s2t returns captions only.source
target
speaker_id
zh_female_vv_uranus_bigtts or zh_male_jingqiangkanye_emo_mars_bigtts. Omit to match the source speaker.seconds
audio_format
pcm (default): mono little-endian float32 at 24 kHz, signed int16 at 16 kHz. ogg_opus: Ogg Opus, always 48 kHz.audio_rate
Client to server
- Binary frames carry 16 kHz mono signed 16-bit little-endian microphone audio. 80 ms (2,560 bytes) is the recommended frame size and 64 KB is the maximum.
{"type":"finish"}ends speech input and asks the server to send the remaining captions and speech beforedone.{"type":"cancel"}ends the session immediately.- Wait for
statusbefore sending audio. The limit is measured from uploaded PCM bytes, not wall-clock time. A frame exceeding the allowance stops input and returns terminalerrorwithbudget_exhausted.
Server to client
Text frames are JSON events. Binary frames are translated audio in target_format, so decode them with the rate from status. Zero numeric fields and false booleans may be omitted; treat missing values as 0/false.
| Event | Fields | Meaning |
|---|---|---|
status | state, session_id, mode, max_seconds, reserved_credits, target_format, target_rate | First event after the upgrade; only then send audio. reserved_credits is the hold for this session; decode audio with target_format and target_rate. |
subtitle | channel (source/translation), phase (start/partial/final), text, start_ms, end_ms, speaker_changed, detected_language | start opens a sentence, partial replaces the current sentence text, final settles it and appends the line. |
audio | phase (start/end), start_ms, end_ms | Marks one translated sentence's speech; the audio itself arrives as binary frames. |
muted | muted_ms | Part of the input was silence or was not recognized, so that span was not interpreted. |
done | audio_seconds, charged_credits, optional usage | Terminal event with duration and charge. Token-priced sessions include usage.input_audio_tokens, usage.output_text_tokens and usage.output_audio_tokens as decimal strings. The socket closes afterwards. |
error | code, message | Terminal event. Read code; do not silently retry a session that already started. |
Languages
| Mode | Source | Target |
|---|---|---|
s2s, no speaker_id | zh, en, de, fr, es, id, ja, pt | zh, en, de, fr, es, id, ja, pt |
s2s, with speaker_id | The twenty s2t source languages and the two dialects listed below | zh, en |
s2t | zh, en, de, fr, es, id, ja, pt, ko, tr, ms, nl, ro, pl, cs, ar, th, vi, ru, it, plus yue-CN and sh-CN | The twenty languages, without the two dialects |
Both source and target may be zhen for Chinese/English reverse translation. Otherwise Chinese or English must be on at least one side. A dialect source never counts as that side, so yue-CN or sh-CN needs a Chinese or English target. An unsupported language pair is rejected with invalid_input.
Errors
Token-priced sources charge the reported input_audio_tokens, output_text_tokens and output_audio_tokens. Quantities may be fractional; the exact total is rounded up once to whole credits. audio_seconds reports duration and does not determine the token charge. status.token_allowance, when present, describes the reservation allowance. When no usage response is received, the customer charge is zero and the reservation is released regardless of audio duration or output. Execution failures retain their failure reason. Incomplete, invalid or over-allowance usage also releases the full reservation at zero customer charge. Existing seconds-priced sources keep their original settlement.
After the last audio packet, you must send {"type":"finish"} as a separate WebSocket text frame. Keep reading until done or error. seconds is the audio-byte allowance, not a wall-clock completion timer.
Rejections before the socket upgrade are ordinary HTTP responses with {"error":{"code":...}}. After the upgrade, a terminal event has type=error; state describes session/accounting status, code is machine-readable, and message contains the safe concrete failure reason when known. Codes include budget_exhausted, provider_failed, transport_error, and settlement_error. Historical sessions may also contain missing_usage, invalid_usage, usage_exceeds_limit or unknown_outcome. Session queries use state, error_code and error_message for the same meanings.
{"type":"error","code":"provider_failed","message":"Timed out waiting for the next input audio packet. Send PCM audio continuously and send a finish frame when input ends."}| Code | Status | Meaning |
|---|---|---|
temporarily_unavailable | 503 | The live connection could not be established. |
price_unavailable | 503 | No callable published price. |
not_found | 404 | Source not found. |
insufficient_credits | 402 | The balance cannot cover the hold. |
rate_limited | 429 | Too many concurrent sessions. |
capability_unavailable | 503 | The source is unavailable; retry later. |
forbidden | 403 | The key, account, source or origin is not allowed. |
unauthorized | 401 | The key or console session is missing or expired. |
invalid_input | 400 | A query value, source or language pair is not supported. |