Realtime speech to text

Stream audio into RealtimeVoiceKIT over a WebSocket and get transcripts back while the speaker is still talking. The service transcribes each turn of speech incrementally: you see a live partial transcript within moments, then a final formatted version when the turn ends.

Every session follows the same two step pattern: your server mints a short-lived session token with your API key, then any client (browser, mobile app, backend worker) opens the WebSocket with that token and streams audio.

1. Your server calls POST /v2/realtime/token with your rtvk_ API key and receives a token plus the WebSocket URL.
2. The client opens wss://api.realtimevoicekit.com/v2/realtime/ws with the token and its audio settings as query parameters.
3. The server sends {"type": "ready"} once the session is live.
4. The client streams raw PCM16 audio as binary WebSocket frames.
5. The server pushes transcript events: partials while a turn is in progress, then a final when end_of_turn is true.
6. The client sends {"type": "stop"}; the server flushes remaining transcripts, sends {"type": "done"}, and closes the socket.

Sessions are metered per second of connection time, so a 90 second session on Realtime Pro costs about $0.0135. Call GET /v2/models for the current catalog and rates.

From your server, exchange your API key for a short-lived session token. The token is safe to hand to a browser: it can only be used to open a realtime session, and it expires after expires_in seconds (15 minutes by default).

curl -X POST https://api.realtimevoicekit.com/v2/realtime/token \
  -H "Authorization: Bearer rtvk_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "whisper-voicekit-realtime-pro", "sample_rate": 16000}'

{
  "token": "eyJhbGciOiJIUzI1NiIs...",
  "websocket_url": "wss://api.realtimevoicekit.com/v2/realtime/ws",
  "expires_at": "2026-07-22T12:15:00Z",
  "model": "whisper-voicekit-realtime-pro",
  "sample_rate": 16000
}

Request fields

Response fields

If your free API credit is used up and you have no active API subscription, this endpoint returns HTTP 402 with a message asking you to add billing.

Connect to the returned websocket_url with the token in the query string. The WebSocket reads its own query parameters, so pass the same model, sample_rate, language_code, and medical_mode you used when minting the token.

// Browser or any WebSocket client, using only the short-lived token.
const params = new URLSearchParams({
  token: session.token,
  sample_rate: "16000",
  model: "whisper-voicekit-realtime-pro",
});
const ws = new WebSocket(session.websocket_url + "?" + params.toString());
ws.binaryType = "arraybuffer";

ws.onmessage = (event) => {
  const msg = JSON.parse(event.data);
  if (msg.type === "ready") {
    startStreaming(ws); // safe to send audio now
  } else if (msg.type === "transcript") {
    handleTranscript(msg);
  } else if (msg.type === "done") {
    ws.close();
  }
};

Wait for the ready event before sending audio. If the token is invalid or expired the socket closes immediately with code 4401; an unknown model closes with 4400.

Send audio as binary WebSocket frames containing raw 16-bit little-endian PCM (pcm_s16le), single channel, at the sample_rate you declared. No JSON wrapper, no base64, no WAV header: just the raw bytes.

• Chunk size: send roughly 50 ms to 200 ms of audio per frame. The RealtimeVoiceKIT web client sends about 85 ms per frame, which is a good starting point.
• Pacing: send chunks as you capture them. When streaming from a file, throttle to real time instead of sending the whole file at once.
• Browser capture: microphones capture at 44.1 kHz or 48 kHz. Downsample to your declared rate and convert float samples to 16-bit integers before sending. See Audio requirements.

// float32Samples: mono Float32Array at your declared sample rate.
function sendChunk(ws, float32Samples) {
  const pcm = new Int16Array(float32Samples.length);
  for (let i = 0; i < float32Samples.length; i++) {
    const s = Math.max(-1, Math.min(1, float32Samples[i]));
    pcm[i] = s < 0 ? s * 0x8000 : s * 0x7fff;
  }
  ws.send(pcm.buffer); // binary frame, raw PCM16 bytes
}

While a turn of speech is in progress you receive a stream of transcript events for it, each one replacing the last. When the speaker pauses, the turn closes and you receive the final version.

{
  "type": "transcript",
  "text": "so the quarterly numbers look",
  "end_of_turn": false,
  "formatted": false,
  "words": [
    { "text": "so", "start": 1200, "end": 1330, "confidence": 0.99 },
    { "text": "the", "start": 1330, "end": 1420, "confidence": 0.98 }
  ]
}

{
  "type": "transcript",
  "text": "So the quarterly numbers look strong.",
  "end_of_turn": true,
  "formatted": true,
  "words": [
    { "text": "So", "start": 1200, "end": 1330, "confidence": 0.99 }
  ]
}

• end_of_turn false: a partial for the turn in progress. Each new partial replaces the previous one in your UI.
• end_of_turn true: the turn is complete. Append it to your final transcript.
• formatted true: a restatement of the finished turn with punctuation and casing applied. Replace the raw final turn you just appended with this version.
• words: one object per word with text, start, end (milliseconds), and confidence between 0 and 1.

async def receive_transcripts(ws):
    final_turns = []
    async for raw in ws:
        msg = json.loads(raw)
        if msg["type"] == "transcript":
            if not msg["end_of_turn"]:
                print("partial:", msg["text"])
            elif msg["formatted"] and final_turns:
                final_turns[-1] = msg["text"]  # formatted restatement
            else:
                final_turns.append(msg["text"])
        elif msg["type"] == "done":
            break
    print(" ".join(final_turns))

When you are done, send {"type": "stop"} as a text frame and keep the socket open. The server flushes any remaining transcript events, sends {"type": "done"}, and closes the connection.

client -> server   {"type": "stop"}        (text frame)
server -> client   ...final transcript events...
server -> client   {"type": "done"}
                   socket closes

If the client simply disconnects, the session still ends and is billed for the connected time; you just lose any transcripts that had not been delivered yet. Sessions are also capped at 55 minutes: at the cap the server sends {"type": "reconnect"} and finalizes the session. Mint a fresh token and open a new session to keep going.