Engineering blog
Real engineering problems we ran into while building an LLM API gateway.
GLM 5.2 Reasoning Effort: the Setting That Cuts Cost 20x (Measured)
Same coding answer: $0.0031 with reasoning effort set right vs $0.062 on GLM 5.2's unbounded default. 20x cheaper, 30x faster. How to set the dial per task.
Claude Fable 5 Won't Run Under ZDR: 30-Day Retention Is Mandatory
ZDR orgs get a 400 error on claude-fable-5: no opt-out on the Claude API, Bedrock, Vertex or Foundry. What it breaks for HIPAA/COPPA, and the routing fix.
LLM Prompt Caching: The Complete 2026 Guide
A five-part series on LLM prompt caching: KV cache architecture, provider comparison, Python tutorial, model-by-use-case matrix, LangChain integration.
-
Prompt Cache Minimums: The Docs Under-State by 1.4–2.4x
Vendors publish a prompt-cache token minimum. Measured across LLM families, auto-cache needs 1.4–2.4x more than the docs say; Claude's explicit cache is exact.
-
GPT-5.6 Cost Guide: Prompt Caching 90% Off, Reasoning Effort
GPT-5.6's two cost levers, measured: explicit breakpoints bill cached input at 10% of the rate, and not sending reasoning_effort bills 1.5x as much as none.
-
Which LLM Is Cheapest for Your Language? Tokenizer Costs Measured
GPT-5.5 bills fewest tokens for European languages, Kimi for Chinese, DeepSeek for Japanese; Claude Fable 5, Opus 4.8 and Sonnet 5 run 1.2-2.3x. Measured.
-
Claude Fable 5 for Agents: Tool-Call Refusals, Cost vs GLM 5.2
Claude Fable 5 across five agent workloads vs glm-5.2, opus-4-8 and sonnet-5: mid-tool-call refusals, adaptive thinking, and cost that shifts 5-15x by shape.
-
LangChain Prompt Caching: Setups That Actually Hit the Cache
LangChain's friendliest syntax silently disables Claude's prompt cache. Measured fixes: cache_control via content blocks, variable placement, usage fields.
-
Claude Sonnet 5's New Tokenizer: 41% More Tokens per Prompt
Claude Sonnet 5's new tokenizer makes the same text about 41% more tokens than Sonnet 4.6, reshaping cost, budgets, and cache eligibility on the gateway.
-
GLM 5.2 Tool Calls in Agent Loops: What 'OpenAI-Compatible' Hides
GLM 5.2 speaks the OpenAI tool-calling API, but text rides with the tool calls and reasoning shows on the turn. How it lines up against OpenAI and Anthropic.
-
What a Simple Transcription Test Can and Can't Tell You
Seven dedicated transcription models on a simple multilingual test: clean speech is solved everywhere, so judge on cost, coverage, and streaming, not accuracy.
-
What Actually Drives Your Image-Generation Bill
Image-generation cost on an LLM gateway, measured across model, resolution and quality. Quality, not caching, decides per-token vs per-image billing.
-
Open-Weight LLM Caching: Why Yours Is Provider Roulette
For open-weight LLMs, prompt caching is solved in the inference engine and broken by routing. A five-layer map, measured across DeepSeek, Qwen and Kimi.
-
Claude Fable 5 Caching: Same Contract, 2.9x the Bill vs Opus 4.6
Claude Fable 5 is live on Synthorai. Measured prompt caching, TTL, tokenization and cost vs Opus 4.6/4.8: same cache contract, new tokenizer, ~2.9x the bill.
-
Provider Drift: How Default Routing Inflates LLM Cost
On a multi-provider gateway's default routing, identical requests scatter across upstreams with separate caches. Hit rate craters and your bill climbs.
-
Does Your LLM Gateway Lie About Cache? A 5-Min Audit
Gateways can report cache hits while billing full price. One script audits both auto-cache (DeepSeek) and marker-based (Claude) caching in five minutes.
-
Claude Opus 4.8 on Synthorai: Caching & TTL vs 4.7/4.6
Claude Opus 4.8 is live on Synthorai. Measured prompt caching and TTL behavior vs Opus 4.7/4.6 — what carries over, plus the tokenizer shift to re-check.
-
Best LLM for Chat, RAG & Agents: 2026 Model + Cost Decision Matrix
Decision matrix matching LLM workload — chatbots, RAG APIs, AI agents — to the right model and caching strategy. Real 2026 pricing, cost math per scenario.
-
LLM Prompt Caching in Python: A Working Code Tutorial
Measured prompt-cache savings across Claude, GPT-5, Gemini 2.5, DeepSeek-v4 and Qwen3 via Synthorai's OpenAI-compatible gateway. Real usage.cost and TTFT.
-
Prompt Caching Compared: Claude, GPT-5, Gemini, DeepSeek, Qwen (2026)
Anthropic Claude, OpenAI GPT-5, Gemini 2.5, DeepSeek-v4 and Qwen3 expose prompt caching in five different shapes — measured 2026 feature comparison.
-
How LLM Prompt Caching Works: KV Cache & TTL Explained
How LLM prompt caching actually works: Transformer attention math behind K/V reuse, the memory-compute tradeoff that shapes TTL, and why it cuts cost and TTFT.