Prompt Caching

Prompt caching bills a repeated prefix at a fraction of its normal input price. Every provider behind the gateway does it slightly differently — some need you to mark the prefix, others cache automatically — and the discount is passed through at provider rates.

Two ways it gets turned on

Explicit providers cache only the prefix you mark. On this gateway cache_control is forwarded on the Anthropic-compatible path; other providers expose their explicit caching through their own APIs rather than through this endpoint. Automatic providers cache any prefix past a minimum length with no code change. Which applies depends on the model you call.

Marking a prefix explicitly

Put the stable part first and mark its last block. Everything before the mark becomes the cached prefix, so the request must keep that part byte-identical between calls:

curl https://synthorai.io/v1/chat/completions \
  -H "Authorization: Bearer $SYNTHORAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-5",
    "messages": [
      {"role": "system", "content": [
        {"type": "text", "text": "<your long, stable system prompt>",
         "cache_control": {"type": "ephemeral"}}
      ]},
      {"role": "user", "content": "The part that changes every call."}
    ]
  }'

Per-provider behaviour

Minimum prefix, cache lifetime and any write surcharge, per provider:

Transcribed from each provider's own documentation; a provider whose semantics we have not verified against its docs is left out rather than guessed at. The minimum shown is the provider's default — individual models set their own, some higher and some lower (several Claude models require 4,096 tokens, one needs only 512). The model's own page carries the figure that applies to it. Where the mode says explicit, marking a prefix with cache_control works on the Anthropic-compatible path; other providers expose explicit caching through their own APIs instead.

How it shows up on your bill

Cached reads bill at a separate, lower input rate — visible per request in usage, and per model on its price page. Where a provider charges to write the cache, the first call costs more than plain input, so caching pays off from the second identical prefix onward. Models with no separate write rate simply bill the write at normal input price.

Common mistakes

• Putting variable content first. Caching matches a prefix. A timestamp or user id at the top of the prompt invalidates everything after it.
• Prefixes under the minimum. Below the threshold nothing is cached and nothing is reported — it looks like caching silently failed. Check the model page rather than this table: several models require a higher minimum than their provider's default.
• Calls spaced further apart than the lifetime. An idle gap longer than the TTL means the next call pays the write cost again.

Full request body reference: Chat Completions