Realtime Voice

Build two-way voice agents with gpt-realtime — the model listens to speech and speaks back in real time (like a phone call), over a WebSocket connection to /v1/realtime. Your existing OpenAI Realtime SDK works unchanged; just point it at our endpoint and use your Synthorai key.

Endpoint & authentication

Open a WebSocket to /v1/realtime with your model in the query string. Authentication runs before the upgrade, so a rejected key never opens a socket.

# WebSocket, server-side (Node / Python / Go — anything that can set headers)
GET wss://synthorai.io/v1/realtime?model=gpt-realtime
Authorization: Bearer $YOUR_KEY
# Send only the Authorization header. Do NOT send OpenAI-Beta: realtime=v1
# (the beta protocol is retired; it makes the upstream reject the session).

• Your sk-syn key never leaves our gateway — the upstream credential is swapped in on our side.
• Built for server-side clients (Node, Python, Go — anything that can set an Authorization header). Browser ephemeral tokens are not supported.
• WebSocket only. SIP (telephony) and WebRTC connect the client directly to the upstream and are not proxied — bridge telephony audio into a WebSocket instead.

Configure the session

After the connection opens you receive session.created. Send a session.update to set the voice, modalities, audio format and turn detection. Use the GA shape (session.type: "realtime", audio under audio.input / audio.output) — the old flat beta shape is retired.

{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "output_modalities": ["audio"],
    "instructions": "You are a concise customer-support agent.",
    "audio": {
      "input":  { "format": { "type": "audio/pcm", "rate": 24000 },
                  "turn_detection": { "type": "server_vad" } },
      "output": { "format": { "type": "audio/pcm", "rate": 24000 } }
    }
  }
}

A minimal conversation turn:

1. Stream microphone audio with input_audio_buffer.append (base64 PCM).
2. Commit the turn with input_audio_buffer.commit, then response.create.
3. Receive audio deltas (response.output_audio.delta) and the text transcript, then response.done with usage.
4. With server VAD enabled the model detects turns for you; the same events flow without manual commits.

Tools, function calling & knowledge

Declare functions in session.update. The model calls them mid-conversation — this is how you connect order lookups, ticketing, or any business system. Return the result and the model continues speaking with it:

// 1. Declare tools in session.update: "tools": [{ "type": "function", ... }]
// 2. The model emits a function_call in response.done.
// 3. Return the result, then ask the model to continue:
{ "type": "conversation.item.create",
  "item": { "type": "function_call_output",
            "call_id": "call_abc",
            "output": "{\"status\":\"shipped\"}" } }
{ "type": "response.create" }

Remote MCP servers are also supported ("type": "mcp") — the upstream connects to the MCP server directly. For a knowledge base, inject facts via instructions, a function tool backed by your RAG, or MCP; the model's built-in knowledge is not a reliable source for business facts.

Billing

Billed per response.done usage at the official list price (no markup) — audio and text, input and output, with a cached input rate. Each session writes one ledger row at the end.

Prices shown are for gpt-realtime / gpt-realtime-2.1; gpt-realtime-2.1-mini is roughly a third. Audio input is ~10 tokens/second, output ~20 tokens/second. Keeping conversation history append-only lets the cached rate ($0.40/1M) absorb most of a long call.

Session length & hot-switch

• For calls that may run long, hot-switch well before the cap (e.g. at 50 minutes): open a fresh session and replay the conversation as text with conversation.item.create (user as input_text, assistant as output_text — assistant audio cannot be replayed).
• There is no session resume: if the WebSocket drops, upstream state is lost. Reconnect uses the same text-replay path, so build reconnection in from day one.
• Context is 128K for gpt-realtime-2.1 / -mini (≈3.5 hours of input audio); the legacy gpt-realtime is 32K — don't use it for long calls.

Access

gpt-realtime is in invited beta. It appears in the catalog and pricing, but using it requires your workspace to be granted access — contact us to enable it.