Posts tagged llm-gateway
7 posts about llm-gateway.
-
LLM API Rate Limits: 13 Providers Compared, 2 SDKs Never Retry a 429
Which tokens count toward RPM and TPM on 13 LLM APIs and clouds, which return headers, the four things a 429 can mean, and the two SDKs that never retry one.
-
LLM Data Retention and ZDR: Every Party That Can Read Your Prompts
Vendors delete API logs in about 30 days. Your tracing SaaS, vector memory, and gateway console keep prompts until you delete them. What ZDR actually covers.
-
GLM 5.2 Tool Calls in Agent Loops: What 'OpenAI-Compatible' Hides
GLM 5.2 speaks the OpenAI tool-calling API, but text rides with the tool calls and reasoning shows on the turn. How it lines up against OpenAI and Anthropic.
-
GLM 5.2 Reasoning Effort: the Setting That Cuts Cost 20x (Measured)
Same coding answer: $0.0031 with reasoning effort set right vs $0.062 on GLM 5.2's unbounded default. 20x cheaper, 30x faster. How to set the dial per task.
-
Image Generation API Cost: 5 Models Compared ($0.006-$0.039)
Five image models, same prompts, one gateway: $0.006 to $0.039 per image at defaults, plus a quality knob that swings one model's bill 36x. Measured.
-
Provider Drift: How Default Routing Inflates LLM Cost
On a multi-provider gateway's default routing, identical requests scatter across upstreams with separate caches. Hit rate craters and your bill climbs.
-
Does Your LLM Gateway Lie About Cache? A 5-Min Audit
Gateways can report cache hits while billing full price. One script audits both auto-cache (DeepSeek) and marker-based (Claude) caching in five minutes.