LLM Data Retention and ZDR: Every Party That Can Read Your Prompts
Contents
Your prompt does not go to “the AI company”. On an agent stack it crosses a chain of parties that all handle it in cleartext, and exactly which ones depends on your stack: the agent framework’s telemetry, a tracing platform, a memory store, an analytics tool, an AI gateway, and one of three kinds of inference provider, each with its own retention policy, storage region, and training clause. The party that keeps your prompts longest is usually on your side of the API: model vendors delete standard logs in about 30 days, while a default tracing or memory store keeps them until you delete them. Zero data retention (ZDR) is a real and useful arrangement, but it pins down exactly one of those parties on exactly one of three axes. This post maps the whole chain, who retains what and for how long, and what ZDR does and does not cover.
TL;DR
- Every cleartext hop can collect; agent-side tracing and memory stores keep prompts far longer than any model provider does.
- Providers retain per feature: on the Claude API, caching is ZDR-eligible, batch jobs persist 29 days, Fable 5 requires 30 days.
- Geography follows the server: DeepSeek’s official API stores data in the PRC, no other option.
- Free endpoints are the sharpest edge: training rights are usually the price of free capacity.
- ZDR held up in court: the NYT preservation order exempted only zero-retention API customers.
Who can read your prompt on an agent stack?
Everyone on the path, and the path is longer than most teams draw it:
Walk it left to right. The agent layer is yours, but rarely only yours: frameworks ship telemetry, tracing and observability platforms exist precisely to store full prompts and responses, memory features write conversation content into vector stores that never expire on their own, and a session-replay analytics snippet in your web UI captures the prompt as the user types it, before your backend ever sees it. The gateway is one party if you run it yourself and two if it is SaaS, and SaaS gateways come in two flavors. The first is the multi-provider aggregator: one API that routes each request across the providers behind it, so its data posture is always a sum, its own policy plus the policy of whichever host this particular request landed on. The second comes from security vendors: data-loss-prevention (DLP) and guardrail gateways whose entire product is payload inspection, redaction, and policy enforcement, which means reading every prompt is the feature, and flagged prompts are typically retained as security events by design.
Behind the gateway sit three kinds of inference provider: the model vendor’s own API, a cloud provider hosting the model inside your cloud tenancy, and GPU hosts serving open weights under their own logging policies. The bypass branch, self-hosted inference on a dedicated tenant or your own GPUs, is the only path where no third party handles cleartext, and it comes with a catch we will return to.
The rule that organizes everything else: seeing and keeping are different questions. Every box can technically collect. Which ones do is a matter of defaults, settings, and contracts, and the defaults are wildly uneven across the chain.
Who actually keeps your prompts?
Usually your own tooling, longer than anyone else. The provider retention windows that dominate security questionnaires are days; the agent-side stores that nobody questionnaires are forever.
The agent layer retains by design. A tracing platform’s product is a database of your prompts and outputs; retention there is a project setting, not a policy accident. The defaults vary by tool and are worth knowing: GitHub Copilot’s OpenTelemetry integration exports span structure, timing, and token counts but no prompt content unless you explicitly enable content capture, which is the privacy-conservative template more agent tooling should copy. Cursor’s privacy mode exists because the default mode shares code data. Whatever the model provider deletes after 30 days, your trace store still has on day 300.
The gateway keeps what it chooses to keep. A gateway cannot avoid usage records, since billing requires knowing token counts, models, and timestamps per request. It can entirely avoid storing payloads: prompt and response bodies can pass through memory and never be written at rest. But two popular gateway features quietly cross that line. A request-inspection or observability console is payload storage by definition, exactly like a tracing SaaS one hop earlier. And gateway-side caching stores prompt content at the gateway hop: a semantic cache keeps an embedding of your prompt plus the full cached response, and a response cache keeps both sides verbatim, on whatever disk the gateway runs. When you evaluate a gateway, SaaS or self-hosted, the question is the same split three times over: what is in the request record, what does the observability view persist, and what does the cache write?
Providers retain per feature, not per company. The useful mental model comes straight from Anthropic’s API retention documentation, which discloses eligibility feature by feature: prompt caching is ZDR-eligible because cache state is held in memory only, batch processing stores jobs for 29 days because async jobs need storage, code-execution containers persist up to 30 days, and uploaded files live until you delete them. The honest sentence in that page deserves quoting: using a stateful feature “is a choice to step outside your ZDR arrangement for that specific data.” On the OpenAI side, the data-controls documentation lists which endpoints are ZDR-eligible, keeps abuse-monitoring logs up to 30 days by default, and documents that cached prompts live as encrypted key-value tensors on GPU-local storage with a bounded TTL. Standard commercial deletion on the Claude API is within 30 days.
Where does the data physically live?
Wherever the serving infrastructure lives, and the provider kinds give different answers. A cloud-hosted model is the strong case: on Amazon Bedrock the cloud provider is the data processor and inference stays in the region you selected, which is why regulated buyers reach for that lane first. A model vendor’s own API runs where the vendor runs: OpenAI offers US and EU regional processing for most endpoints; smaller vendors often publish nothing. The unambiguous extreme is DeepSeek, whose privacy policy states that personal data is collected, processed, and stored in the People’s Republic of China, with no US or EU option on the official API. The same open weights served by a US GPU host carry none of that geography, which is the cleanest illustration that where a model’s data goes is a property of who serves it, not of the model.
Two tails extend past the primary copy. Caches and batch files live in the serving region for their own lifetimes, separate from the request log. And every provider sits on subprocessors (its own downstream vendors) and backups: deletion commitments are typically phrased as deletion from active systems with backup propagation windows behind them. If your data-flow diagram ends at the API vendor’s logo, it is missing at least those two boxes.
Who requires retention, and who trains on your prompts?
Three different forces keep data alive past the defaults, and only one of them is in the provider’s marketing.
Policy requirements. Some models mandate retention as a safety condition. Claude Fable 5 and Mythos 5 require 30-day retention even for ZDR customers; the enforcement is fail-loud, since a request from an organization whose retention configuration does not meet the requirement is rejected with a 400 rather than silently accepted. Abuse escalation is the other policy tail: content flagged for usage-policy violations is retained on a different clock entirely (Anthropic’s consumer policy documents up to 2 years for flagged inputs and outputs, and up to 7 years for classifier scores).
Legal holds. The NYT v. OpenAI litigation produced the clearest natural experiment in retention promises. A May 2025 preservation order required OpenAI to preserve output logs it would otherwise have deleted, including chats users had deleted, across consumer tiers; the order was narrowed that September and a court later ordered 20 million chat logs produced in discovery. The detail that matters for API buyers: enterprise and zero-data-retention customers were carved out, because there was nothing retained to preserve. A deletion policy bends to a litigation hold; an architecture that never stored the data does not.
Harvesting middlemen. The middle of the chain does get caught monetizing cleartext, and the largest documented case sat closest to the user. In December 2025, security researchers documented that a “privacy” VPN browser extension with millions of installs had been injecting scripts into AI chat pages, intercepting every prompt and response across ChatGPT, Claude, Gemini and five other assistants, and shipping the conversations to a data-broker affiliate; the harvesting ran whether or not the VPN was on, across roughly 8 million users of the publisher’s extension family. The incident was a client-side interceptor rather than an API gateway, but the lesson transfers to every box on the diagram: any intermediary that handles cleartext can be a collection point, its incentive is monetization, and the interception is invisible from both ends. Trust in a middleman is a claim about its business model, not its feature list.
Training clauses. Business APIs from the major vendors do not train on your data by default; consumer products increasingly do unless you opt out, which is one more reason agent traffic belongs on API keys rather than consumer accounts. And a growing number of products put training on a toggle you are expected to find yourself: consumer plans with training on by default and an opt-out buried in settings, developer tools whose telemetry or code-sharing setting doubles as a training consent, vendor programs that trade data-sharing opt-ins for discounts or free quota, and aggregator dashboards with separate paid-tier and free-tier training switches. Each toggle is per-account, sometimes per-workspace, and a default can change with a terms update, so “we checked once” is not a posture; auditing these switches belongs on the same recurring checklist as key rotation.
Free models, singled out on purpose. Every warning above concentrates in one place: the free tier. A free model launch is usually subsidized by whoever benefits from the traffic, most often the model’s own vendor, so prompts sent to the free variant flow to that party under its policy, with training rights commonly part of the deal, processed wherever that party operates, which for Chinese vendors means China. Aggregator gateways publish per-endpoint data policies and a switch to exclude providers that train precisely because free routes are where those clauses live. The rule of thumb is blunt: treat a free endpoint as a data submission, not an API call. Free inference is paid for somehow, and the currency is usually your prompts.
What does ZDR actually mean?
ZDR pins one axis out of three, for one party in the chain, for eligible features only. That is not a criticism; it is the definition, and knowing the edges is what makes it usable.
- The three axes. Training use, retention duration, and human access are independent. Business APIs already do not train on your data; ZDR moves retention to zero for payloads; human access is governed separately by abuse processes. Conflating them is how ZDR gets oversold.
- What survives under ZDR. Usage metadata and billing records; safety-classifier outputs; and whatever stateful features you opt into. Both major vendors now engineer around the abuse-monitoring exception rather than through it: payload-free safety signals on one side, classifier-results-only retention on the other.
- What ZDR does not reach. Everything left of the provider: your tracing store, your gateway logs, your vector memory. A ZDR agreement with a model vendor while a tracing SaaS holds every prompt indefinitely is the most common retention posture we see, and the least coherent.
- The self-hosting caveat. Running open weights on your own GPUs removes every third party, and then your own infrastructure inherits the whole problem: inference-server request logging, access logs, trace files. Self-hosting relocates the retention surface; only deliberate log hygiene shrinks it.
How Synthorai handles it
The gateway hop is ours, so the commitments are concrete. In zero-retention mode, request and response bodies pass through memory and are not written at rest; what persists is the usage record that billing requires: timestamps, model, token counts, and a content hash that lets a request be matched during a dispute without storing what was said. Upstream, the roster is filtered before routing ever happens: Synthorai only onboards providers that commit to zero data retention for our traffic, with one documented exception, Claude Fable 5, whose 30-day retention is mandated by the model vendor and cannot be contracted away. Provider selection is retention selection, and the gateway’s job is to make that property explicit per route rather than buried in a policy PDF: a Fable 5 route carries its retention requirement as documented metadata, not as a surprise.
FAQ
Do LLM providers train on API data by default?
No, the major vendors’ business APIs do not train on customer data by default; that commitment appears in each provider’s data-controls documentation. The exceptions cluster in the long tail: consumer products with opt-out rather than opt-in, some GPU hosts serving open weights, and free-tier endpoints where training rights are part of the price.
Does ZDR mean the provider stores nothing at all?
No. ZDR means prompt and response payloads are not retained for eligible features. Usage metadata, billing records, and safety-classifier outputs persist, and stateful features like batch jobs or file uploads store data by nature, outside the ZDR envelope. Read the feature-eligibility table, not the headline.
Is it safe to send production data to free models?
Treat a free endpoint as a data submission, not an API call. Free capacity is subsidized by whoever benefits from the traffic, training rights are commonly part of the arrangement, and the data is processed wherever the subsidizing party operates. For throwaway experiments that is a fine trade; for anything containing customer data, code, or credentials it rarely is.
Is self-hosting automatically the most private option?
It removes every third party from the cleartext path, which is real. It does not remove retention: inference servers, reverse proxies, and tracing all log by default, so a self-hosted stack with default logging can retain more prompt data than a ZDR API arrangement. Privacy follows the logging configuration, not the hosting model.
Sources verified 2026-08-29: every provider claim above links to the vendor’s own documentation or to primary court records, and policies in this area changed twice in the month before publication, so treat the links as the live source of truth. This is an engineering reading, not legal advice.
Related: Fable 5’s 30-day retention requirement, how prompt caching works, provider cache comparison.