Engineering blog
Real engineering problems we ran into while building a multimodal AI gateway.
GLM 5.2 Reasoning Effort: the Setting That Cuts Cost 20x (Measured)
Same coding answer: $0.0031 with reasoning effort set right vs $0.062 on GLM 5.2's unbounded default. 20x cheaper, 30x faster. How to set the dial per task.
Claude Fable 5 Won't Run Under ZDR: 30-Day Retention Is Mandatory
ZDR orgs get a 400 error on claude-fable-5: no opt-out on the Claude API, Bedrock, Vertex or Foundry. What it breaks for HIPAA/COPPA, and the routing fix.
LLM Prompt Caching: The Complete 2026 Guide (Cut Input Cost 50-90%)
How prompt caching works across Claude, GPT, Gemini and DeepSeek: cut input cost 50-90% and TTFT 3-10x. Architecture, provider comparison, Python code.
-
Best LLM for Translation: 9 Models, 9 Languages, a 400x Cost Spread
9 LLMs, 9 languages, 8,455 blind verdicts: gpt-5.6-sol leads in 7 of 9, gemini-3.7-flash ties it in Korean and Italian; cost spans $0.26 to $104 per 1M chars.
-
LLM Structured Outputs: 4 of 12 APIs Return Valid but Wrong JSON
Structured outputs measured on 12 LLM APIs: 100% schema-valid JSON, wrong values on 4 models with thinking on, same call billed 30 to 4,959 prompt tokens.
-
Which APIs Mark AI Content? 16 Measured Against 10 Marking Laws
16 generative APIs against 10 marking laws: 7 embed C2PA, 4 embed China's label, none both, audio and video nothing, and no C2PA manifest survives a resize.
-
Voice Agent API Cost: a 10-Minute Call Runs $0.04 to $0.57
Every leg measured: STT at $0.002-0.006/min, TTS at $0.004-0.034 per audio-minute, LLM turns, and GPT Realtime. Cascade vs speech-to-speech, priced per call.
-
DeepSeek V4 Pro GA vs Preview, Measured: 18-62% Less Thinking
deepseek-v4-pro-0813 vs the preview build on identical tasks: 18-62% fewer reasoning tokens, a fixed 8,192-token runaway, and a lost 'I don't know'.
-
Gemini 3.7 Flash API Cost, Measured: Tasks Bill 2.5-8x Less Than 3.6
Day-one probes of gemini-3.7-flash: half-price intro until Dec 31, thinking down 26-77% vs 3.6, off-switch gone, prefill 400s. Task bills drop 2.5-8x.
-
How Many Tokens Is an Image? 15 APIs Measured, Same Icon 6 to 1,298
The same 64px icon bills 6 tokens on GPT-5.6 and 1,298 on ByteDance Seed. Three billing schemes, three broken doc rules; dollars follow the rate, not the image.
-
LLM Thinking Controls: What 13 Models Accept, Ignore, or Enforce
reasoning_effort and thinking_budget across 13 LLM APIs: three vendors enforce budgets to the token, others silently ignore them, a cap of 16 can burn 97.
-
Web Search API and Web Fetch API: How They Work and What $0.01 Buys
How server-side web search and web fetch bill: a $0.01 fee, 1,500-3,100 injected result tokens at your model's rate, and a turn two that can re-search.
-
Qwen-Image 3.0 API, Measured: 9 Scenes for One $0.03 Image
Day-one probes of qwen-image-3.0: $0.03 per image with prompts unbilled even at 5,000 tokens, a real nine-in-one trick, and small print that misspells itself.
-
DeepSeek V4 Flash API Cost: Thinking Mode Corrupts Strict JSON
Day-one 0731 measurements: $0.14/$0.28 list, a 1,024-token cache page, and a defect: thinking plus strict json_schema corrupted integers in 8 of 13 runs.
-
Qwen 3.8 Max API Pricing: 16 Thinking Tokens Beat the Off Switch
Day-one measurements of qwen3.8-max at $2/$6: an effort dial that is really a budget cap, a 1/6 accuracy collapse with thinking off, and a 16-token fix.
-
MCP Tool Overhead, Measured: 26 Tools, $0.03 Every Call
Every MCP tool is re-billed as input tokens on every call: one small tool is 401 tokens on Claude, the GitHub server $0.03/call, and families differ 2.7x.
-
Prompt Cache Write Cost: When Does the 1.25x Premium Pay?
One measured agent suite, one premium: a 6% loss on RAG, an 83% saving on batch. The read:write ratio decides, and you can measure yours before the invoice.
-
Claude Opus 5 vs Opus 4.8, Measured: Same Price, 3x Apart
Opus 5 and Opus 4.8 share a $5/$25 rate card, yet Opus 5 billed 3.1x more on identical tasks out of the box. Where it goes, and the switch that closes it.
-
Speech-to-Text APIs: 14 Models, $0.002 to $0.016 a Minute
14 speech-to-text APIs behind one gateway: per-minute rates, who streams, what token-billed gpt-4o converts to, and the China ASR tier most comparisons skip.
-
Gemini 3.6 Flash: the Thinking Dial That Moves Cost 30x (Measured)
Gemini 3.6 Flash bills reasoning you never see, and one request setting swings the same task's cost up to 30x. Measured across five task types, with the catch.
-
Seedance API Pricing, Measured: the Video-Token Formula, Solved
Seedance bills W×H×(24s+1)/1024 video tokens; we solved it to the token. 720p bills as 1248×704, and 4k's cheaper rate costs 2.1x more per second. Measured.
-
GPT-5.6 Prompting Guide: Two Defaults That Bill 1.5x and 10x More
GPT-5.6 defaults are expensive: omitting reasoning_effort bills 1.5x vs 'none'; unmarked prefixes cost 10x cached reads. Request-shape playbook, measured.
-
Kimi K3 API Pricing, Measured: Turn Off the 'Always-On' Reasoning
Kimi K3's docs say reasoning can't be disabled. reasoning_effort:'none' works and cuts simple queries 6x. Measured: effort dial, cache floor, 9-language rates.
-
GPT Live API Pricing: gpt-realtime Speaking Costs 4x
GPT Live is the ChatGPT feature, not an API; behind it is gpt-realtime-2.1: $0.019/min listening, $0.077/min speaking, silence free, cached replay 1/80th.
-
LLM Token Usage: Why a 4-Token Answer Bills 217 Tokens
Measured across GPT-5.6, Claude Fable 5, Qwen3.7-max and five more families: reasoning dominates the output bill. How to read every usage field, and cap them.
-
Prompt Cache Minimums: The Docs Under-State by 1.4-2.4x
Vendors publish a prompt-cache token minimum. Measured across LLM families, auto-cache needs 1.4-2.4x more than the docs say; Claude's explicit cache is exact.
-
GPT-5.6 Cost Guide: Prompt Caching 90% Off, Reasoning Effort
GPT-5.6's two cost levers, measured: explicit breakpoints bill cached input at 10% of the rate, and not sending reasoning_effort bills 1.5x as much as none.
-
Which LLM Is Cheapest for Your Language? Tokenizer Costs Measured
GPT-5.5 bills fewest tokens for European languages, Kimi for Chinese, DeepSeek for Japanese; Claude Fable 5, Opus 4.8 and Sonnet 5 run 1.2-2.3x. Measured.
-
Claude Fable 5 for Agents: Tool-Call Refusals, Cost vs GLM 5.2
Claude Fable 5 across five agent workloads vs glm-5.2, opus-4-8 and sonnet-5: mid-tool-call refusals, adaptive thinking, and cost that shifts 5-15x by shape.
-
LangChain Prompt Caching: Setups That Actually Hit the Cache
LangChain's friendliest syntax silently disables Claude's prompt cache. Measured fixes: cache_control via content blocks, variable placement, usage fields.
-
Claude Sonnet 5's New Tokenizer: 41% More Tokens per Prompt
Claude Sonnet 5's new tokenizer makes the same text about 41% more tokens than Sonnet 4.6, reshaping cost, budgets, and cache eligibility on the gateway.
-
GLM 5.2 Tool Calls in Agent Loops: What 'OpenAI-Compatible' Hides
GLM 5.2 speaks the OpenAI tool-calling API, but text rides with the tool calls and reasoning shows on the turn. How it lines up against OpenAI and Anthropic.
-
Transcription API Cost: 7 Models on the Same Audio
Seven transcription models, one multilingual audio set, one gateway: per-minute cost spans $0.0020 to $0.0164 and accuracy is not the differentiator.
-
Image Generation API Cost: 5 Models Compared ($0.006-$0.039)
Five image models, same prompts, one gateway: $0.006 to $0.039 per image at defaults, plus a quality knob that swings one model's bill 36x. Measured.
-
Open-Weight LLM Caching: Why Yours Is Provider Roulette
For open-weight LLMs, prompt caching is solved in the inference engine and broken by routing. A five-layer map, measured across DeepSeek, Qwen and Kimi.
-
Claude Fable 5 Caching: Same Contract, 2.9x the Bill vs Opus 4.6
Claude Fable 5 is live on Synthorai. Measured prompt caching, TTL, tokenization and cost vs Opus 4.6/4.8: same cache contract, new tokenizer, ~2.9x the bill.
-
Provider Drift: How Default Routing Inflates LLM Cost
On a multi-provider gateway's default routing, identical requests scatter across upstreams with separate caches. Hit rate craters and your bill climbs.
-
Does Your LLM Gateway Lie About Cache? A 5-Min Audit
Gateways can report cache hits while billing full price. One script audits both auto-cache (DeepSeek) and marker-based (Claude) caching in five minutes.
-
Claude Opus 4.8 on Synthorai: Caching & TTL vs 4.7/4.6
Claude Opus 4.8 is live on Synthorai. Measured prompt caching and TTL behavior vs Opus 4.7/4.6, what carries over, plus the tokenizer shift to re-check.
-
Best LLM by Use Case (2026): Chat, RAG & Agents Cost Matrix
Chat, RAG, or agents? Pick the cheapest model that still performs, with measured cost sketches (a 15-step agent task, 100K-query/day RAG) and a decision matrix.
-
LLM Prompt Caching in Python: A Working Code Tutorial
Measured prompt-cache savings across Claude, GPT-5, Gemini 2.5, DeepSeek-v4 and Qwen3 via Synthorai's OpenAI-compatible gateway. Real usage.cost and TTFT.
-
Which LLM Prompt Cache Is Cheapest? 5 Providers Compared (2026)
Claude, GPT-5.x, Gemini, DeepSeek and Qwen cache in five shapes: explicit vs automatic, 5-min vs 1-hour TTLs, reads from 0.1x to 0.5x. Measured side-by-side.
-
How LLM Prompt Caching Works: KV Cache & TTL Explained
How LLM prompt caching actually works: Transformer attention math behind K/V reuse, the memory-compute tradeoff that shapes TTL, and why it cuts cost and TTFT.