Context Compaction Cost: Measuring the Turns to Payback
Contents
- Which problem are you trying to solve with compaction?
- Scenario 1: What if the model is running out of context?
- Scenario 2: Should you compact before a higher price tier?
- Scenario 3: How many further turns repay one compaction?
- Does compaction waste the cache-write premium?
- How does a payoff formula compare with measured results?
- What changes with DeepSeek’s implicit cache?
- How does growth affect when to compact?
- What interval does the model suggest for continuing work?
- How should you measure the whole cost?
- What policy should you deploy?
- FAQ
- Does context compaction always reduce API cost?
- Should I compact exactly at a price threshold?
- Can I wait until the context window is full?
- Does faster context growth always make one summary pay off sooner?
- Should implicit caching make me wait until the context ceiling?
Context compaction on Claude Sonnet 5.5 repaid its cost after 4-6 further requests in six warm-cache runs starting near 14k tokens. That measured range is workload-specific. A policy must account for summary generation, the actual cache billing model and enough remaining work to repay the intervention.
TL;DR
- Context-window pressure makes reducing active context a capacity requirement; our tests did not measure a model’s maximum window.
- One query was approximately break-even after a high-tier summary ($0.0560882 versus $0.0560655 retained); two queries clearly repaid the summary in that example.
- A retrospective formula check estimated 4-5 payoff turns against 4-6 measured turns; this was not a prospective prediction test.
- A six-question archive task cost $0.0156941 with compaction versus $0.0059911 without it, at 6/6 accuracy.
Which problem are you trying to solve with compaction?
Choose the reason to reduce context before choosing a token threshold. A hard context limit, a higher pricing band, and an ordinary long conversation require different decisions.
Context compaction replaces earlier conversation content with a shorter representation that later requests use. Our experiments used client-side text summaries: an ordinary model call generated a summary, and the application replaced the earlier history with that text. Native signed or encrypted compaction features have different request and usage contracts.
| Situation | Primary goal | What should trigger action? | Evidence in this article |
|---|---|---|---|
| Approaching the model’s context window | Keep the task executable | Enough headroom for the next input, tool result and output | Documented model limits and an engineering policy; no maximum-window experiment |
| Approaching a higher pricing tier | Reduce complete-workload cost | Expected tariff savings exceed summary and continuation costs | A small uncached price-tier experiment |
| Comfortable window and pricing band | Improve a particular workload | Enough useful work remains to repay the summary without losing required state | Cached state-retention pairs and a short archive workload |
Scenario 1: What if the model is running out of context?
Reduce or reorganize active context before the next request can exceed the supported window. At this point, continuing with the complete transcript may cease to be a viable strategy.
Claude Haiku 5.5 documents a 1M-token window and a 128K maximum output in its model specifications. Those are published specifications. We did not test the maximum accepted context on our gateway route, and the price-tier experiment below does not establish that boundary.
Reserve space for the next input, tool result and service-required output. Trigger while the summary request still fits. Carry essential state through a summary or retrievable source record, and score successful continuation; an over-window rejection is not a cheaper successful baseline.
Scenario 2: Should you compact before a higher price tier?
Compact when the expected remaining work repays the summary at the actual prices. Crossing a pricing threshold can make compaction attractive, but the summary may itself be billed in the expensive band.
Claude Haiku 5.5 changes prices above 100,000 input tokens. The standard input/output rates, checked November 4, 2026, are $0.10/$0.50 per million tokens up to that boundary and $0.50/$2.50 above it.
We sent varied fictional library records through the Synthorai gateway. A request with 94,350 input tokens cost $0.009437; one with 111,999 cost $0.0560095. Both generated four output tokens. The dollar difference includes both input length and price band.
Next, we summarized the larger history. That call processed 112,028 input tokens, with no reported cache hits, and cost $0.0560715 in the higher band. Both paths answered the quantity/source queries correctly.
| Remaining queries | Retain history | Summary plus reduced-context queries | Retain, repriced at the low tariff | Compact, repriced at the low tariff |
|---|---|---|---|---|
| 1 | $0.0560655 | $0.0560882 | $0.0112131 | $0.0112310 |
| 2 | $0.1120760 | $0.0560944 | $0.0224152 | $0.0112372 |
One query was approximately break-even: the paths differed by $0.0000227. Two queries clearly repaid the summary in this example. Repricing every token at the low tariff separates volume effects from the tariff effect.
These uncached requests describe one synthetic task. A warm history differs. Check how far a summary moves below the boundary and when new input will cross again; see long-context pricing tiers.
Scenario 3: How many further turns repay one compaction?
We measured 4-6 further turns for Claude Sonnet 5.5 in six cached runs starting with about 13.5k-14.2k input tokens. A turn here means one subsequent model request. The experiment included summary generation and later cache writes, then found the first crossing within twelve continued requests after the shared warmup. In all six runs, compaction remained no more expensive through request 12 after that crossing. This observation does not cover a later cache expiry, another summary or a changed task.
Does compaction waste the cache-write premium?
Previously paid writes are sunk at the decision point. The incremental question is whether reduced future reads cover the summary request and the new writes. These experiments used explicit five-minute caching; implicit caches have provider-specific pricing and may not charge the same write premium.
One medium-growth run shows the difference between the first intervention and another summary later in the session:
| Intervention | Future read costs avoided | Extra continuation writes | Summary-generation cost | Net result by turn 12 |
|---|---|---|---|---|
| First summary vs retaining history | $0.0159875 | $0.0010175 | $0.0049761 | $0.0099939 cheaper |
| Second summary vs stopping after one | $0.0037283 | $0.0013925 | $0.0065302 | $0.0041844 more expensive |
The second row includes $0.0000100 less in other tokens. Summary charges include their own cache buckets; do not count those twice. Token buckets do not identify which individual entries were never reused.
How does a payoff formula compare with measured results?
A simple warm-cache model estimates the remaining turns required. Let H be the history tokens replaced, S the replacement-summary input tokens, A the complete summary-request cost, r the price per cache-read token, and w the price per cache-write token. Preserve unchanged instructions before the replaced history.
Over n future requests, retained history costs n*r*H. The replacement costs A + w*S + (n-1)*r*S. Their incremental difference is:
compact cost minus retain cost
= A + (w-r)*S - n*r*(H-S)
payback turns
= ceil((A + (w-r)*S) / (r*(H-S)))
The formula assumes H > S, immediate cache eligibility, unchanged prices and equal subsequent additions, outputs and tool work. It does not cover cache expiry, extra retained-prefix rewrites or quality-recovery costs. Use a piecewise cost calculation when those change.
We applied the formula retrospectively after the experiments. The calculation uses warmup and summary receipts rather than subsequent costs as inputs, but the outcomes were already known. It is a backcheck, not held-out or prospective validation. The analysis approximates H with total warmup input and S with summary output tokens; those differ from exactly replaceable history and replacement input because of stable instructions, framing and cache eligibility.
| Incoming growth | Run | Retrospective estimate | Measured payoff |
|---|---|---|---|
| About 96 tokens/turn | 1 | 5 turns | 6 turns |
| About 96 tokens/turn | 2 | 5 turns | 6 turns |
| About 1.4k tokens/turn | 1 | 5 turns | 5 turns |
| About 1.4k tokens/turn | 2 | 4 turns | 4 turns |
| About 5.2k-5.5k tokens/turn | 1 | 5 turns | 5 turns |
| About 5.2k-5.5k tokens/turn | 2 | 4 turns | 5 turns |
Backcheck error was at most one turn in these six runs. A future decision needs estimates of A and S from prior summaries and validation on new tasks; this comparison supplies no general error guarantee.
What changes with DeepSeek’s implicit cache?
No separate cache-write fee does not mean free input after compaction. DeepSeek V4.1 Flash 1017 uses automatic caching and separate hit/miss input prices. Its pricing page lists no separate write charge. A new summary prefix still pays the miss-input rate when it is not cached.
The cache documentation requires matching persisted prefix units. Construction takes seconds and hits are best-effort, so the replacement may miss on more than one request. Track prompt_cache_hit_tokens and prompt_cache_miss_tokens; do not assume exactly one cold call or that an unchanged system prefix always hits.
Let p be the miss-input price per token and m the number of cold replacement-prefix requests within n future requests. If the retained baseline stays cached and later common costs cancel:
compact cost minus retain cost
= A + m*(p-r)*S - n*r*(H-S)
For one cold request, substitute p for w, not zero. If m is unknown, calculate several scenarios or price actual hit/miss buckets, including stable-prefix and summary-call misses.
The following is formula arithmetic, not a DeepSeek experiment. Use official off-peak Flash prices of $0.003/M hit input, $0.15/M miss input and $0.60/M output. Assume a 1k-token summary, a warm original history, one cold replacement request, and summary cost A = r*H + output_price*S. Ignore framing, reasoning and tools.
| Original history | Assumed summary-call cost | Calculated payoff |
|---|---|---|
| 10k tokens | $0.000630 | 29 requests |
| 100k tokens | $0.000900 | 4 requests |
Cheap reads can favor a short cached history, while longer work can repay compaction. DeepSeek quality and payback were not measured. Recalculate for peak rates or mixed-rate sessions.
How does growth affect when to compact?
Growth changes the history at the decision point. If j turns pass before intervention and each adds g tokens, substitute H_j = H_0 + g*j, along with the current summary size and cost, into the payoff formula.
Equal subsequent growth adds common costs to both paths. Under unchanged prices and cache behavior those terms cancel. Faster growth reaches capacity or price constraints sooner, but does not by itself define payoff turns.
In our growth tests, essential state stayed small while archival payloads grew. Repetition cost more than one summary at medium growth and less at high growth in both twelve-turn runs. All policies answered 96/96 state queries correctly, including the extension; that does not test unlimited growth in essential state.
What interval does the model suggest for continuing work?
For a long-running task with approximately constant growth, summary size, prices and cache behavior, let F be the fixed charge per compaction cycle, including summary output and rewriting the replacement. Exclude costs linear in the interval: their amortized per-turn contribution is constant. When the summarizer reads history from cache, estimate F as A - r*H + w*S; other pricing requires its own decomposition. If g is growth per turn and k is the interval between summaries, the interval-dependent average cost is:
average interval-dependent cost = F/k + r*g*(k-1)/2
candidate interval k = sqrt(2*F/(r*g))
candidate history threshold = S + g*k
This requires positive F, r and g, small retained summaries and comparable task quality. With zero growth there is no growth-driven periodic trigger; with free reads this model provides no read-cost payback. With our explicit-cache prices and summary sizes, it gives candidate intervals near 30, 7-8 and 4 turns at low, medium and high growth. Test neighboring integers of at least one request, subject to capacity headroom; no interval sweep established these as optimal. A miss-priced cache needs its own fixed-cost decomposition.
A worked calculation makes the units concrete. One recorded medium-growth run had A = $0.0049761, H = 13,712, S ≈ 341 and g ≈ 1,396 tokens/turn. Using r = $0.10/M and w = $2.50/M gives:
F ≈ 0.0049761 - (0.10/1,000,000)*13,712
+ (2.50/1,000,000)*341
= $0.0044574 per cycle
k ≈ sqrt(2*0.0044574 / ((0.10/1,000,000)*1,396))
≈ 8 requests
This calculates a candidate from recorded values; it does not establish an optimal eight-request policy. The finite-horizon check is separate: if an intervention is estimated to repay after five requests and only three remain, skip that optional intervention even when a periodic timer says it is due.
For a task nearing completion, skip another optional summary if it cannot repay itself over the remaining requests. Capacity, pricing tiers and required information retention override this simplified monetary model.
A short archive workload illustrates the gate. Both strategies answered six quantities correctly, but retaining history cost $0.0059911 versus $0.0156941 including a summary. Six queries did not repay that intervention. The summary retained enough detail; no extra lookup was needed. The payoff horizon depends on the actual summary and workload, not a universal four-to-six-turn rule.
How should you measure the whole cost?
Include the summary, the first rewritten request and every continuation. Prompt-token reduction and cache-hit percentage alone cannot establish savings.
For Claude Sonnet 5.5, our accounting used $2/M fresh input, $2.50/M five-minute cache writes, $0.10/M cache reads and $10/M output. Cache reads are cheap but still cost money. Replacing a warm history can create a fresh write, while a very short summary may not qualify for caching at all. Our explicit content-block breakpoints produced measurable cache writes and reads; some short compacted prefixes initially reported neither.
This function assumes a specified cold-prefix request count. Pass the explicit write price or the implicit miss-input price as cold_input_per_million:
import math
def payback_turns(summary_cost, history_tokens, summary_tokens,
read_per_million=0.10, cold_input_per_million=2.50,
cold_requests=1):
if cold_requests < 0 or int(cold_requests) != cold_requests:
raise ValueError("cold_requests must be a nonnegative integer")
r = read_per_million / 1_000_000
p = cold_input_per_million / 1_000_000
if summary_cost < 0 or p < r:
raise ValueError("Unsupported cost assumptions")
if history_tokens <= summary_tokens or r <= 0:
return None
upfront = summary_cost + cold_requests * (p - r) * summary_tokens
saving_per_turn = r * (history_tokens - summary_tokens)
return max(1, cold_requests, math.ceil(upfront / saving_per_turn))
Estimate summary cost from prior calls for a decision before compaction. Validate against full usage receipts, including failures and recovery work; a missing receipt is not a free request.
Measurements ran on November 4, 2026 through the gateway’s Messages interface, with thinking disabled and no hosted tools. Ordinary replies were capped at 128 output tokens and summaries at 2,048. Costs were reconstructed from returned usage at standard prices and checked against reported gateway costs. They are not a supplier-invoice reconciliation. The ordinary workloads exercised client-side summaries, not a native compaction implementation.
What policy should you deploy?
Apply capacity and tariff constraints first, with input/tool/output headroom. For Claude Sonnet 5.5 explicit-cache workloads, include write premiums and summary charges, and skip optional interventions unlikely to repay before completion. Validate integer intervals near the analytic candidate on identical task arrivals and ending turns.
For DeepSeek V4.1 Flash 1017 implicit caching, preserve a useful prefix when hits are stable and little work remains. A later soft capacity trigger can be reasonable; zero separate write fees do not imply waiting for the hard limit. For longer work, estimate actual hit/miss costs and watch the requests after a rewrite until hits appear.
Recalculate when prices, cache state, model or essential-state size changes. Keep retrievable sources and validate changed values in detail-sensitive tasks. Record the summary and following usage in one agent session, using the correct token buckets.
FAQ
Does context compaction always reduce API cost?
Savings must exceed summary, cache reconstruction and recovery costs. Our six-query archive cost $0.0156941 with a summary versus $0.0059911 retained, both at 6/6 accuracy.
Should I compact exactly at a price threshold?
A price threshold is a point to evaluate the complete remaining workload, not an automatic command to summarize. In our Claude Haiku 5.5 example, the summary itself was billed above 100,000 tokens; one later query was approximately break-even, while two clearly repaid it.
Can I wait until the context window is full?
Leave room for the summary request, next input/tool result and output. An already oversized history may not fit a summarization request. We did not measure the maximum-window failure boundary.
Does faster context growth always make one summary pay off sooner?
Faster shared growth raises both strategies’ costs and does not automatically increase the savings from one summary. Repeated compaction can bound further accumulation, but adds summary and rewrite costs. In our medium-growth runs, repeating compaction cost more than one summary over twelve turns; the high-growth runs favored repetition. Choose a policy using growth, horizon and required retained state.
Should implicit caching make me wait until the context ceiling?
Implicit caching can favor retaining a cheap warm history over a short task. DeepSeek V4.1 Flash 1017 still charges miss input for an uncached summary prefix, and persistence is best-effort. Estimate the remaining benefit, then leave tool/input/output headroom before the supported window. The DeepSeek values above are calculations, not measured thresholds.