Simultaneous interpretation

Simultaneous interpretation streams speech in over a WebSocket and returns bilingual captions plus translated speech on the same socket. Microphone audio goes up as binary frames; captions arrive as JSON events and translated audio comes back as binary frames.

WebSocket endpoint

API clients connect to wss://api.anycap.io/v1/audio/ast/stream and send Authorization: Bearer <connection key> on the handshake. Public API clients must omit Cookie and Origin headers. List sessions with GET /v1/audio/ast/sessions and read one with GET /v1/audio/ast/sessions/{session_id} using the same key. The console page connects to wss://<console API host>/api/v1/capabilities/audio.ast/stream with its session cookie: a browser socket cannot send request headers, and the server only accepts an allowlisted Origin.

import WebSocket from "ws";import { readFileSync, writeFileSync, appendFileSync } from "node:fs"; const base = process.env.ANYCAP_API_BASE || "https://api.anycap.io";const headers = { Authorization: `Bearer ${process.env.ANYCAP_API_KEY}` };const response = await fetch(`${base}/v1/audio/ast/models`, { headers });if (!response.ok) throw new Error(`Discovery failed: ${response.status}`);const { items } = await response.json();const source = items.find((item) => item.resource_id === process.env.ANYCAP_AST_SOURCE_ID) || items[0];if (!source) throw new Error("No authorized interpretation source"); // Raw 16 kHz mono s16le PCM, without a WAV header. Install: npm install wsconst pcm = readFileSync(process.env.ANYCAP_PCM_FILE || "speech.pcm");const seconds = Math.min(60, source.max_session_seconds);if (!pcm.length || pcm.length % 2 || pcm.length > seconds * 32000) {  throw new Error("PCM must contain whole samples within the session limit");}const url = new URL("/v1/audio/ast/stream", base);url.protocol = url.protocol === "https:" ? "wss:" : "ws:";url.search = new URLSearchParams({  resource_id: source.resource_id, mode: "s2s", source: "zh", target: "en",  seconds: String(seconds), audio_format: "pcm", audio_rate: "24000",}).toString();const socket = new WebSocket(url, { headers }); // No Cookie or Origin.let timer;let terminal = false;writeFileSync("translated.pcm", Buffer.alloc(0));socket.on("message", (data, binary) => {  if (binary) return appendFileSync("translated.pcm", data);  const event = JSON.parse(data.toString());  if (event.type === "status" && !timer) {    // Start audio only after status, then send at approximately real-time speed.    let offset = 0;    timer = setInterval(() => {      if (offset >= pcm.length) {        clearInterval(timer);        socket.send(JSON.stringify({ type: "finish" }));        return;      }      socket.send(pcm.subarray(offset, offset + 2560));      offset += 2560;    }, 80);  }  if (event.type === "subtitle") console.log(event.channel, event.phase, event.text || "");  if (event.type === "done" || event.type === "error") {    terminal = true;    clearInterval(timer);    console.log(event);    socket.close();  }});socket.on("unexpected-response", (_request, response) => {  console.error("Handshake rejected:", response.statusCode);  response.setEncoding("utf8");  response.on("data", (body) => console.error(body));});socket.on("error", (error) => console.error(error.message));socket.on("close", () => {  clearInterval(timer);  if (!terminal) console.error("Connection ended without a terminal event; query the session before retrying.");});

Query parameters

Client to server

• Binary frames carry 16 kHz mono signed 16-bit little-endian microphone audio. 80 ms (2,560 bytes) is the recommended frame size and 64 KB is the maximum.
• {"type":"finish"} ends speech input and asks the server to send the remaining captions and speech before done.
• {"type":"cancel"} ends the session immediately.
• Wait for status before sending audio. The limit is measured from uploaded PCM bytes, not wall-clock time. A frame exceeding the allowance stops input and returns terminal error with budget_exhausted.

Server to client

Text frames are JSON events. Binary frames are translated audio in target_format, so decode them with the rate from status. Zero numeric fields and false booleans may be omitted; treat missing values as 0/false.

Languages

Both source and target may be zhen for Chinese/English reverse translation. Otherwise Chinese or English must be on at least one side. A dialect source never counts as that side, so yue-CN or sh-CN needs a Chinese or English target. An unsupported language pair is rejected with invalid_input.

Errors

Token-priced sources charge the reported input_audio_tokens, output_text_tokens and output_audio_tokens. Quantities may be fractional; the exact total is rounded up once to whole credits. audio_seconds reports duration and does not determine the token charge. status.token_allowance, when present, describes the reservation allowance. Missing, invalid or over-allowance usage returns a pending error for reconciliation; do not resubmit that session. Existing seconds-priced sources keep their original settlement.

Rejections before the socket upgrade are ordinary HTTP responses with {"error":{"code":...}}. After the upgrade, terminal error.code can be budget_exhausted, provider_failed, transport_error, settlement_error, missing_usage, invalid_usage or usage_exceeds_limit.