Prompt Caching

Prompt caching bills a repeated prefix at a fraction of its normal input price. Every provider behind the gateway does it slightly differently - some need you to mark the prefix, others cache automatically - and the discount is passed through at provider rates.

Two ways it gets turned on

Explicit providers cache only the prefix you mark. On this gateway cache_control is forwarded on the Anthropic-compatible path; other providers expose their explicit caching through their own APIs rather than through this endpoint. Automatic providers cache any prefix past a minimum length with no code change. Which applies depends on the model you call.

Marking a prefix explicitly

Put the stable part first and mark its last block. Everything before the mark becomes the cached prefix, so the request must keep that part byte-identical between calls:

curl https://synthorai.io/v1/chat/completions \
  -H "Authorization: Bearer $SYNTHORAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-5",
    "messages": [
      {"role": "system", "content": [
        {"type": "text", "text": "<your long, stable system prompt>",
         "cache_control": {"type": "ephemeral"}}
      ]},
      {"role": "user", "content": "The part that changes every call."}
    ]
  }'

Per-provider behaviour

Minimum prefix, cache lifetime and any write surcharge, per provider:

Transcribed from each provider's own documentation; a provider whose semantics we have not verified against its docs is left out rather than guessed at. The minimum shown is the provider's default - individual models set their own, some higher and some lower (several Claude models require 4,096 tokens, one needs only 512). The model's own page carries the figure that applies to it. Where the mode says explicit, marking a prefix with cache_control works on the Anthropic-compatible path; other providers expose explicit caching through their own APIs instead.

How it shows up on your bill

Cached reads bill at a separate, lower input rate - visible per request in usage, and per model on its price page. Where a provider charges to write the cache, the first call costs more than plain input, so caching pays off from the second identical prefix onward. Models with no separate write rate simply bill the write at normal input price.

讓快取在一整段工作階段裡保持有效

快取項目屬於廠商側的某一把 API key。當一條通道配了多把 key 時,連續呼叫可能落到不同的 key 上,於是又付一次寫入費用。給同一段工作階段的每次呼叫帶上同一個 X-Session-Id,閘道就會把它們固定到同一條通道、同一把 key。在 /v1/chat/completions 與 /v1/responses 上,請求主體裡的 prompt_cache_key 能達到同樣效果,無需加標頭。

完整行為、限制與排障: 請求標頭

Common mistakes

• Putting variable content first. Caching matches a prefix. A timestamp or user id at the top of the prompt invalidates everything after it.
• Prefixes under the minimum. Below the threshold nothing is cached and nothing is reported - it looks like caching silently failed. Check the model page rather than this table: several models require a higher minimum than their provider's default.
• Calls spaced further apart than the lifetime. An idle gap longer than the TTL means the next call pays the write cost again.

Full request body reference: Chat Completions