Posts tagged prompt-cache
19 posts about prompt-cache.
-
DeepSeek V4 Flash API Cost: Thinking Mode Corrupts Strict JSON
Day-one 0731 measurements: $0.14/$0.28 list, a 1,024-token cache page, and a defect: thinking plus strict json_schema corrupted integers in 8 of 13 runs.
-
Qwen 3.8 Max API Pricing: 16 Thinking Tokens Beat the Off Switch
Day-one measurements of qwen3.8-max at $2/$6: an effort dial that is really a budget cap, a 1/6 accuracy collapse with thinking off, and a 16-token fix.
-
MCP Tool Overhead, Measured: 26 Tools, $0.03 Every Call
Every MCP tool is re-billed as input tokens on every call: one small tool is 401 tokens on Claude, the GitHub server $0.03/call, and families differ 2.7x.
-
Prompt Cache Write Cost: When Does the 1.25x Premium Pay?
One measured agent suite, one premium: a 6% loss on RAG, an 83% saving on batch. The read:write ratio decides, and you can measure yours before the invoice.
-
Claude Opus 5 vs Opus 4.8, Measured: Same Price, 3x Apart
Opus 5 and Opus 4.8 share a $5/$25 rate card, yet Opus 5 billed 3.1x more on identical tasks out of the box. Where it goes, and the switch that closes it.
-
GPT-5.6 Prompting Guide: Two Defaults That Bill 1.5x and 10x More
GPT-5.6 defaults are expensive: omitting reasoning_effort bills 1.5x vs 'none'; unmarked prefixes cost 10x cached reads. Request-shape playbook, measured.
-
Prompt Cache Minimums: The Docs Under-State by 1.4-2.4x
Vendors publish a prompt-cache token minimum. Measured across LLM families, auto-cache needs 1.4-2.4x more than the docs say; Claude's explicit cache is exact.
-
GPT-5.6 Cost Guide: Prompt Caching 90% Off, Reasoning Effort
GPT-5.6's two cost levers, measured: explicit breakpoints bill cached input at 10% of the rate, and not sending reasoning_effort bills 1.5x as much as none.
-
LangChain Prompt Caching: Setups That Actually Hit the Cache
LangChain's friendliest syntax silently disables Claude's prompt cache. Measured fixes: cache_control via content blocks, variable placement, usage fields.
-
Claude Sonnet 5's New Tokenizer: 41% More Tokens per Prompt
Claude Sonnet 5's new tokenizer makes the same text about 41% more tokens than Sonnet 4.6, reshaping cost, budgets, and cache eligibility on the gateway.
-
Open-Weight LLM Caching: Why Yours Is Provider Roulette
For open-weight LLMs, prompt caching is solved in the inference engine and broken by routing. A five-layer map, measured across DeepSeek, Qwen and Kimi.
-
Claude Fable 5 Caching: Same Contract, 2.9x the Bill vs Opus 4.6
Claude Fable 5 is live on Synthorai. Measured prompt caching, TTL, tokenization and cost vs Opus 4.6/4.8: same cache contract, new tokenizer, ~2.9x the bill.
-
Provider Drift: How Default Routing Inflates LLM Cost
On a multi-provider gateway's default routing, identical requests scatter across upstreams with separate caches. Hit rate craters and your bill climbs.
-
Does Your LLM Gateway Lie About Cache? A 5-Min Audit
Gateways can report cache hits while billing full price. One script audits both auto-cache (DeepSeek) and marker-based (Claude) caching in five minutes.
-
Claude Opus 4.8 on Synthorai: Caching & TTL vs 4.7/4.6
Claude Opus 4.8 is live on Synthorai. Measured prompt caching and TTL behavior vs Opus 4.7/4.6, what carries over, plus the tokenizer shift to re-check.
-
LLM Prompt Caching: The Complete 2026 Guide (Cut Input Cost 50-90%)
How prompt caching works across Claude, GPT, Gemini and DeepSeek: cut input cost 50-90% and TTFT 3-10x. Architecture, provider comparison, Python code.
-
LLM Prompt Caching in Python: A Working Code Tutorial
Measured prompt-cache savings across Claude, GPT-5, Gemini 2.5, DeepSeek-v4 and Qwen3 via Synthorai's OpenAI-compatible gateway. Real usage.cost and TTFT.
-
Which LLM Prompt Cache Is Cheapest? 5 Providers Compared (2026)
Claude, GPT-5.x, Gemini, DeepSeek and Qwen cache in five shapes: explicit vs automatic, 5-min vs 1-hour TTLs, reads from 0.1x to 0.5x. Measured side-by-side.
-
How LLM Prompt Caching Works: KV Cache & TTL Explained
How LLM prompt caching actually works: Transformer attention math behind K/V reuse, the memory-compute tradeoff that shapes TTL, and why it cuts cost and TTFT.