Simultaneous interpretation

Models and usage

List models available to your key

BASH
curl 'https://api.anycap.io/v1/audio/ast/models' -H "Authorization: Bearer $ANYCAP_API_KEY"

Choose a resource_id, connect via WebSocket and send PCM. Use s2t for captions or s2s for captions and translated audio.

WebSocket endpoint

API clients connect to wss://api.anycap.io/v1/audio/ast/stream and send Authorization: Bearer <connection key> on the handshake. Public API clients must omit Cookie and Origin headers. List sessions with GET /v1/audio/ast/sessions and read one with GET /v1/audio/ast/sessions/{session_id} using the same key. The console page connects to wss://<console API host>/api/v1/capabilities/audio.ast/stream with its session cookie: a browser socket cannot send request headers, and the server only accepts an allowlisted Origin.

JS
import WebSocket from "ws";import { readFileSync, writeFileSync, appendFileSync } from "node:fs"; const base = process.env.ANYCAP_API_BASE || "https://api.anycap.io";const headers = { Authorization: `Bearer ${process.env.ANYCAP_API_KEY}` };const response = await fetch(`${base}/v1/audio/ast/models`, { headers });if (!response.ok) throw new Error(`Discovery failed: ${response.status}`);const { items } = await response.json();const source = items.find((item) => item.resource_id === process.env.ANYCAP_AST_SOURCE_ID) || items[0];if (!source) throw new Error("No authorized interpretation source"); // Raw 16 kHz mono s16le PCM, without a WAV header. Install: npm install wsconst pcm = readFileSync(process.env.ANYCAP_PCM_FILE || "speech.pcm");const seconds = Math.min(60, source.max_session_seconds);if (!pcm.length || pcm.length % 2 || pcm.length > seconds * 32000) {  throw new Error("PCM must contain whole samples within the session limit");}const url = new URL("/v1/audio/ast/stream", base);url.protocol = url.protocol === "https:" ? "wss:" : "ws:";url.search = new URLSearchParams({  resource_id: source.resource_id, mode: "s2s", source: "zh", target: "en",  seconds: String(seconds), audio_format: "pcm", audio_rate: "24000",}).toString();const socket = new WebSocket(url, { headers }); // No Cookie or Origin.let timer;let terminal = false;writeFileSync("translated.pcm", Buffer.alloc(0));socket.on("message", (data, binary) => {  if (binary) return appendFileSync("translated.pcm", data);  const event = JSON.parse(data.toString());  if (event.type === "status" && !timer) {    // Start audio only after status, then send at approximately real-time speed.    let offset = 0;    timer = setInterval(() => {      if (offset >= pcm.length) {        clearInterval(timer);        socket.send(JSON.stringify({ type: "finish" }));        return;      }      socket.send(pcm.subarray(offset, offset + 2560));      offset += 2560;    }, 80);  }  if (event.type === "subtitle") console.log(event.channel, event.phase, event.text || "");  if (event.type === "done" || event.type === "error") {    terminal = true;    clearInterval(timer);    console.log(event);    socket.close();  }});socket.on("unexpected-response", (_request, response) => {  console.error("Handshake rejected:", response.statusCode);  response.setEncoding("utf8");  response.on("data", (body) => console.error(body));});socket.on("error", (error) => console.error(error.message));socket.on("close", () => {  clearInterval(timer);  if (!terminal) console.error("Connection ended without a terminal event; query the session before retrying.");});

Simultaneous interpretation streams speech in over a WebSocket and returns bilingual captions plus translated speech on the same socket. Microphone audio goes up as binary frames; captions arrive as JSON events and translated audio comes back as binary frames.

Query parameters

ParameterMeaning
resource_id
Authorized source UUID from GET /v1/audio/ast/models. Required.
mode
s2s returns translated speech and captions; s2t returns captions only.
source
Required source language code; see the language table below.
target
Required target language code; see the language table below.
speaker_id
Optional s2s voice: zh_female_vv_uranus_bigtts or zh_male_jingqiangkanye_emo_mars_bigtts. Omit to match the source speaker.
seconds
Requested input-audio allowance. Omit or pass 0 for the source default; positive values below 10 become 10, and values above the source limit are clamped. Values above 600 or below 0 are rejected. Send finish when input ends.
audio_format
pcm (default): mono little-endian float32 at 24 kHz, signed int16 at 16 kHz. ogg_opus: Ogg Opus, always 48 kHz.
audio_rate
For PCM use 16000 or 24000 (default). Ogg Opus always uses 48000; the supplied rate is ignored.

Client to server

  • Binary frames carry 16 kHz mono signed 16-bit little-endian microphone audio. 80 ms (2,560 bytes) is the recommended frame size and 64 KB is the maximum.
  • {"type":"finish"} ends speech input and asks the server to send the remaining captions and speech before done.
  • {"type":"cancel"} ends the session immediately.
  • Wait for status before sending audio. The limit is measured from uploaded PCM bytes, not wall-clock time. A frame exceeding the allowance stops input and returns terminal error with budget_exhausted.

Server to client

Text frames are JSON events. Binary frames are translated audio in target_format, so decode them with the rate from status. Zero numeric fields and false booleans may be omitted; treat missing values as 0/false.

EventFieldsMeaning
statusstate, session_id, mode, max_seconds, reserved_credits, target_format, target_rateFirst event after the upgrade; only then send audio. reserved_credits is the hold for this session; decode audio with target_format and target_rate.
subtitlechannel (source/translation), phase (start/partial/final), text, start_ms, end_ms, speaker_changed, detected_languagestart opens a sentence, partial replaces the current sentence text, final settles it and appends the line.
audiophase (start/end), start_ms, end_msMarks one translated sentence's speech; the audio itself arrives as binary frames.
mutedmuted_msPart of the input was silence or was not recognized, so that span was not interpreted.
doneaudio_seconds, charged_credits, optional usageTerminal event with duration and charge. Token-priced sessions include usage.input_audio_tokens, usage.output_text_tokens and usage.output_audio_tokens as decimal strings. The socket closes afterwards.
errorcode, messageTerminal event. Read code; do not silently retry a session that already started.

Languages

ModeSourceTarget
s2s, no speaker_idzh, en, de, fr, es, id, ja, ptzh, en, de, fr, es, id, ja, pt
s2s, with speaker_idThe twenty s2t source languages and the two dialects listed belowzh, en
s2tzh, en, de, fr, es, id, ja, pt, ko, tr, ms, nl, ro, pl, cs, ar, th, vi, ru, it, plus yue-CN and sh-CNThe twenty languages, without the two dialects

Both source and target may be zhen for Chinese/English reverse translation. Otherwise Chinese or English must be on at least one side. A dialect source never counts as that side, so yue-CN or sh-CN needs a Chinese or English target. An unsupported language pair is rejected with invalid_input.

Errors

Token-priced sources charge the reported input_audio_tokens, output_text_tokens and output_audio_tokens. Quantities may be fractional; the exact total is rounded up once to whole credits. audio_seconds reports duration and does not determine the token charge. status.token_allowance, when present, describes the reservation allowance. When no usage response is received, the customer charge is zero and the reservation is released regardless of audio duration or output. Execution failures retain their failure reason. Incomplete, invalid or over-allowance usage also releases the full reservation at zero customer charge. Existing seconds-priced sources keep their original settlement.

After the last audio packet, you must send {"type":"finish"} as a separate WebSocket text frame. Keep reading until done or error. seconds is the audio-byte allowance, not a wall-clock completion timer.

Rejections before the socket upgrade are ordinary HTTP responses with {"error":{"code":...}}. After the upgrade, a terminal event has type=error; state describes session/accounting status, code is machine-readable, and message contains the safe concrete failure reason when known. Codes include budget_exhausted, provider_failed, transport_error, and settlement_error. Historical sessions may also contain missing_usage, invalid_usage, usage_exceeds_limit or unknown_outcome. Session queries use state, error_code and error_message for the same meanings.

JSON
{"type":"error","code":"provider_failed","message":"Timed out waiting for the next input audio packet. Send PCM audio continuously and send a finish frame when input ends."}
CodeStatusMeaning
temporarily_unavailable503The live connection could not be established.
price_unavailable503No callable published price.
not_found404Source not found.
insufficient_credits402The balance cannot cover the hold.
rate_limited429Too many concurrent sessions.
capability_unavailable503The source is unavailable; retry later.
forbidden403The key, account, source or origin is not allowed.
unauthorized401The key or console session is missing or expired.
invalid_input400A query value, source or language pair is not supported.