The 47 tokens your agent should never summarize

Imagine an agent carrying one standing instruction: never email a document to an address outside the company domain. It follows the rule for an hour. The session grows, old turns are compacted into a summary, and the work continues. Asked again, the same model sends the same document to an external address.

Nothing changed in the model or request. Its memory manager had deleted the rule.

A 2026 preprint calls this failure "governance decay." Across 1,323 benchmark episodes spanning seven model families, agents violated none of the tested policies while those policies remained in full context. After one compaction, the pooled violation rate reached 30 percent and peaked at 59 percent for two model families. A small protected policy buffer restored the benchmark rate to zero. Its average size was about 47 tokens.

Forty-seven is a result from one benchmark, not a universal budget. The durable lesson is architectural: some state has no business entering a lossy summary.

Compaction edits the agent's constitution

Long-running agents eventually exceed a practical context budget. Harnesses respond by truncating old turns or asking a model to rewrite them. Selective eviction offers another route. The compacted state becomes the history used for every later decision.

That rewrite usually optimizes for task continuity. It keeps recent progress and the next likely action. A standing rule loaded forty turns ago looks less relevant, although task relevance and authority are different properties. "Send the report" describes current work; "Never send outside this domain" constrains every future version of that work.

The distinction is easy to miss because a good summary reads coherently. There is no parse error when a prohibition disappears. Later actions remain fluent and locally sensible. The agent itself cannot provide an independent check, since its judgment is now conditioned on the damaged summary.

The governance-decay study isolates this mechanism with deterministic tool-call grading. Each scenario establishes a policy, fills the context with ordinary work, compacts it, then issues a request whose natural completion violates the policy. The grader inspects the final action, such as an external recipient or a production database command. No model judge decides whether the action was safe.

When a policy survived the compacted summary, the observed violation rate was zero. After deletion it reached 38 percent, and a second scoring method pointed the same way. Generic long-context confusion does not explain that split.

Soft rules disappear first

Model alignment already gives certain restrictions a home in the weights. Requests to expose an SSH key or disable security controls may trigger a refusal even after a local policy vanishes. Company-specific rules lack that backup.

The study separated four familiar safety norms from five organization-specific policies involving region, language, spending, reporting channel and external email. Compaction raised violations by 50 percentage points on the soft local rules and six points on the hard norms. The decay gap was 8.3 times.

This explains why ordinary safety testing can miss the problem. A harness may retain refusals for obviously dangerous actions while losing the rules that make an otherwise normal action prohibited inside one deployment. The model still looks safe on generic tests. It has forgotten what this organization means by safe.

Constraint placement also mattered. Preserved system instructions survived in the study. Standing user instructions, retrieved memory and tool-returned policy text remained exposed to compaction. Real systems use all three channels for dynamic governance because permissions, customer terms and incident controls change faster than model weights or global prompts.

Summaries optimize the wrong loss function

Another 2026 preprint studied a less dangerous version of the same information bottleneck. A research agent examined a closed inventory of flights and hotels, then handed compressed state to a booking agent. The second agent had no access to the original inventory. Narrative summaries produced the smallest payload and only 0.48 feasibility accuracy. Schema-constrained JSON reached 0.96.

Fluency is the wrong compression objective for numeric limits and categorical constraints. A sentence can sound faithful while rounding a price, dropping an exception or merging conditions; a typed field exposes whether the compactor retained the value.

Structure helps with task state. It does not make lossy transformation suitable for authority. A JSON field can still contain a paraphrased rule whose meaning shifted. The safer design separates state by required fidelity:

This is a storage policy, not a prompt trick. Each class gets a different failure contract.

What pinning does

Constraint pinning keeps governed rules out of material handed to the compactor. After each rewrite, the harness injects the protected buffer into the new context and verifies its presence. In the ConstraintRot benchmark, that buffer averaged about 47 tokens, less than 0.5 percent of the tested production-scale compaction context. Violation returned to zero in both the ordinary compaction and tested attack conditions.

A minimal implementation has four parts:

1. Mark authoritative constraints when they enter the session. Record their origin and exact bytes.
2. Exclude them from truncation, summarization and model-selected eviction.
3. Reinsert them after every compaction in a stable, high-authority location.
4. Compare the resulting bytes or digest with the protected source. Reject the rewrite if any required item is missing.

The fourth part turns a writing task into an invariant. Beautiful prose can still fail it, in which case the original history remains active.

Pinning also narrows an injection surface. Tool outputs and retrieved documents can contain instructions aimed at the summarizer itself. The paper's compaction-eviction attack asks the summarizer to omit a legitimate policy. Optimized variants defeated every evaluated model. A compactor cannot delete state it never receives authority to rewrite.

Pinning does not enforce the rule

Keeping a policy visible improves model behavior. It does not create a security boundary around a dangerous tool.

Recipient policy still belongs in the email connector, and deployment permissions need enforcement outside the language model. Deterministic authorization remains the final gate.

Pinning solves a different problem: policy carriage. It ensures the model sees the current constraint while planning and explaining its actions. External enforcement ensures a bad plan cannot cross the boundary. Systems need both because many governance rules guide choices before a final call, while some risks permit a hard execution-time check.

The distinction also clarifies which rules belong in context. A secret-access policy enforced entirely by the filesystem still benefits from explanation in the agent's prompt, though the filesystem owns security. A preference such as "use the EU reporting channel" may shape several earlier decisions and deserves protected carriage even if the last API performs a region check.

Recovery beats permanent deletion

Keeping every byte of a large tool observation active is expensive, while summarizing it destroys exact strings. Recent work on addressable recall offers a third option: move old observations to an append-only store, leave a compact citation in context and let the agent request the original by ID.

Addressable Recall Compaction reported 99.40 percent exact-answer accuracy on its needle-in-a-haystack evaluation, against 88.12 percent for the strongest baseline in that experiment. Its LongBench-v2 Hard gain was smaller, 29.97 versus 28.25 percent. These are benchmark results from a preprint, though the mechanism is useful independent of the headline: eviction can remain reversible.

This produces a practical hierarchy. Authority stays pinned verbatim. Bulky exact evidence can live behind a citation, while working state is safer in a schema. Deletion is reserved for material with no plausible recovery value.

Test forgetting as a first-class behavior

Most agent evaluations stop when the first task succeeds. Compaction needs a longer test. The same constraint should be exercised before and after every memory transition.

Build deterministic scenarios around deployment-specific rules. Trigger enough benign work to force compaction. Then ask for an action that would have been allowed without the rule. Inspect the tool arguments, not the explanation. Run the test across repeated compactions as well: ConstraintRot reports 78 percent violation after four rounds, versus zero before compaction.

Add adversarial content to retrieved documents and tool results. Ask the compactor to treat the policy as obsolete, irrelevant or too expensive to keep. Integrity checks should reject every candidate that loses protected state.

Finally, test the system when compaction itself fails. Invalid structured output, an unavailable summarizer or an inflated summary should preserve the old history or stop safely. Memory pressure must never silently weaken authority.

Long context made extended agent work possible. Compaction keeps it running. That makes the compactor part of the control plane, even when its interface looks like text cleanup. Govern how the agent forgets.

Sources

1. Chen, Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents, arXiv preprint, 2026.
2. Sharma et al., State Compression in Two-Agent LLM Relays, arXiv preprint, 2026.
3. Dang et al., Addressable Recall Compaction for Long Context-Window Control in AI Agents, arXiv preprint, 2026.
4. Chen et al., Slipstream: Trajectory-Grounded Compaction Validation for Long-Horizon Agents, arXiv preprint, 2026.
5. Liu et al., Lost in the Middle: How Language Models Use Long Contexts, TACL 2024.