An AI agent can have excellent instructions and still fail because the wrong evidence reaches the model at the wrong moment. A stale tool result outranks a fresh source. A summary drops the constraint that mattered. A sub-agent inherits thousands of irrelevant tokens and mistakes another agent’s actions for its own. These failures look like weak reasoning, yet the model is often reasoning over a bad working set.
Context engineering for AI agents: the design of the complete information environment used for each model call. It covers instructions, tool definitions, selected history, retrieved evidence, durable state, large artifacts, security labels, and the transformations that turn stored data into a temporary working context.
Anthropic describes context as a finite resource with diminishing returns and recommends the smallest high-signal token set that still supports the desired behavior. Its analysis also explains why more tokens can reduce precision: transformer attention creates n-squared pairwise relationships among n tokens, while long-context dependencies remain harder than short ones. Selection happens on every call, after the session has accumulated new evidence and before the model receives its temporary working set.
Browse the Prompt Engineering archive for related implementation guides, security patterns, and production case studies.
This guide builds that idea into an operating model. It gives platform engineers a typed context contract, a four-layer runtime, a restoration rule for compaction, and a weighted release matrix. It also covers tool schemas, provenance, memory lifetimes, cache stability, multi-agent handoffs, injection controls, evaluation, and rollback. The emphasis is production behavior that can be observed and tested.
Compile a working view for every model call
Persistent state and model-visible context need different contracts. Google describes the working context in Agent Development Kit as a compiled view over sessions, memory, and artifacts. The session remains the durable event log; ordered processors select and transform a temporary view for one invocation. That separation lets a team change prompt layout or compaction rules without migrating the underlying record.
The compiler analogy exposes stages that can fail independently. Typed sources enter access filtering before retrieval; version and contradiction resolution run before admission; serialization places stable material ahead of volatile blocks. The output manifest sits beside the model response and records selected block IDs, rejected candidates, policy version, and processor timings. During replay, those timings reveal whether a slow request spent its budget in retrieval, ranking, compaction, or inference.
from dataclasses import dataclass
from enum import StrEnum
from typing import Any
class Trust(StrEnum):
POLICY = "policy"
VERIFIED = "verified"
UNTRUSTED = "untrusted"
@dataclass(frozen=True)
class ContextBlock:
block_id: str
kind: str
trust: Trust
source_uri: str
version: str
token_estimate: int
payload: Any
@dataclass(frozen=True)
class ContextManifest:
request_id: str
policy_version: str
blocks: tuple[ContextBlock, ...]
@property
def estimated_tokens(self) -> int:
return sum(block.token_estimate for block in self.blocks)
Give every block an identity, source, version, trust level, and size estimate. Store the observable manifest: inputs, selected blocks, tool calls, policy decisions, and final output. Hidden chain-of-thought stays outside the trace. This record answers the operational question after a bad response: exactly which evidence and policy version did the model receive?
The four-layer runtime
Policy, session, memory, and artifacts have different lifetimes, owners, and deletion semantics. That distinction is operational, not cosmetic.
Suppose a user requests deletion. Removing one memory row isn’t enough if the same preference survives in a session summary, an artifact copy, or a released prompt template. The control matrix below identifies who owns each copy and which deletion receipt proves the change propagated.
| Layer | Lifetime | Typical contents | Required control |
|---|---|---|---|
| Policy | Release | System rules, tool policy, output contract | Immutable version and approval |
| Session | Workflow | Messages, actions, errors, decisions | Append-only event IDs |
| Memory | Cross-session | Approved preferences and durable facts | Owner, expiry, correction, deletion |
| Artifact | Object policy | Files, logs, reports, large tool output | Named version and content digest |
The current package baseline is Google ADK 2.5.0, released July 16, 2026; PyPI requires Python 3.10 or newer. The framework’s workflow, session, event, memory, and artifact APIs expose the same lifecycle boundaries.
A large report illustrates the payoff. Keep its producing call and digest in the session, place the bytes in the artifact store, and admit only the handle until a later step asks to inspect the payload.
Physical co-location doesn’t erase those contracts. One database can hold every layer, but durable facts still need owners and correction paths, artifacts need versions and digests, and policy records need release approval.
Retention then becomes a typed operation. A worker may expire an event range, preserve the policy record that authorized its actions, and issue a deletion receipt naming the affected IDs. If the same data exists behind an artifact handle, that handle belongs in the receipt too.
Budget tokens by value and restoration cost
The admission problem can be written as a constrained selection problem: maximize total decision value while the sum of admitted token estimates stays below the input budget. Reserve the response allowance first. Policy and current user constraints enter as mandatory blocks; optional evidence competes on relevance, authority, freshness, irreversibility, duplication, and cost. A 5 MB tool payload usually loses to its compact handle unless the current step actually needs the bytes.
Use the following planning rubric as a starting point and tune it against local failure data. Score each candidate block from 0 to 5 on relevance, authority, freshness, and irreversibility. Subtract duplication and token cost on the same scale. A simple priority is 2R + 2A + F + I - D - C. The doubled relevance and authority weights keep a fresh but weak source from displacing the primary evidence for the task.
| Candidate | R/A/F/I/D/C | Priority | Admission decision |
|---|---|---|---|
| Current user constraint | 5/5/5/5/0/1 | 29 | Always include |
| Verified API contract | 5/5/4/4/0/2 | 26 | Include relevant fragment |
| Recent failed action | 4/4/5/5/1/1 | 24 | Keep error and cause |
| Old verbose tool result | 1/3/1/1/4/5 | 1 | Replace with a handle |
| Duplicate summary | 2/2/2/1/5/2 | 4 | Drop |
The values above are worked acceptance inputs for one hypothetical system. A team should tune weights against its own failures. In a legal review workflow, authority may outweigh freshness; during incident response, a recent failed action may outrank an older canonical example. The weights encode that product choice, and changing them can reverse which block wins the final slot.
Retrieval needs provenance and contradiction handling
Retrieval quality has at least five independent axes: tenant scope, source authority, version, freshness, and semantic relevance. Policy version 16 from an index may match a query better than effective version 17 from the policy service. Tenant filtering runs first; authority and freshness then select version 17. When those signals cannot decide, the workflow enters clarification or a restricted mode.
The widely cited Lost in the Middle study tested multi-document question answering and key-value retrieval and found that model performance changes with the position of relevant information. Deliberate ordering follows from that result. Copying every important fact to both ends creates a different failure: duplicated policy can diverge after one copy is updated, leaving two valid-looking instructions in the same request.
A retrieval receipt contains the query, filters, candidate IDs, selected IDs, source versions, and ranking scores. During an incident review, an operator compares it with the effective policy. If version 17 never appears among the candidates, indexing or access filtering failed; if it appears and loses, ranking failed; if it wins and disappears, token admission failed. Each outcome sends the investigation to a different component and owner.
Compaction must preserve a route back to evidence
Consider a migration attempt spanning events 210 through 388. Event 247 carries the deployment error, but a loose summary collapses it to “deployment failed.” Thirty actions later, choosing between a credential refresh and a schema repair depends on the missing error class. The checkpoint has to save a route back, even when it omits the full payload.
That route is the event range plus an immutable artifact handle. Anthropic recommends compaction for long-horizon work while retaining architectural decisions, unresolved bugs, and implementation details and clearing redundant tool results.
- Capture the goal, completed actions, constraints, and unresolved decisions.
- Reference failed actions by stable event ID.
- Keep source versions and artifact handles with integrity digests.
- Retain the raw range under the applicable data policy so later branches can restore decisive evidence.
Here, restoration reloads event 247 only when the later branch needs it. The active context stays compact while the immutable log and trace preserve the exact error evidence for replay.
{
"checkpoint_id": "cp_0198",
"covers_events": {"first": 210, "last": 388},
"goal": "migrate billing callbacks without duplicate charges",
"constraints": ["preserve idempotency keys", "no schema change"],
"open_decisions": ["retry ownership"],
"failed_actions": ["deploy_387"],
"artifact_handles": ["trace://billing/run-42"],
"source_versions": {"billing_policy": "17"},
"restorable": true
}Manus uses a related approach: large observations can leave the active context while their URL or sandbox path remains. Its production account reports an average input-to-output ratio near 100:1 and roughly 50 tool calls for a typical task. Those figures explain why reversible compression and external storage have an outsized effect on agent loops.
Memory writes require stricter review than memory reads
A support agent extracts “contact me only by email” from a verified user setting and proposes it for reuse next month. Before persistence, deterministic code checks the memory type, owner, source, expiry, and correction mechanism. The same write path rejects a preference copied from an untrusted attachment, even if both strings look identical.
User preferences, approved project conventions, and confirmed identifiers may belong in durable memory. Temporary task state belongs in the session. Only verified user settings can create preferences; approved releases own policy. Retrieved documents and tool output remain typed evidence. If a summary proposes either write, the memory validator returns a source-type error and leaves the candidate in the session for review.
Use memory when cross-session reuse has clear value and a reliable invalidation path. Short workflows can keep state in their session, while regulated data requires deletion support and an auditable owner. For project conventions that change through code review, a named repository file provides diffs, version history, and the same correction process engineers already use.
Memory evaluation follows the item through later requests. Record successful recalls, stale recalls, contradicted recalls, access denials, corrections, and the outcomes influenced by each item. During a cleanup review, these fields distinguish a frequently useful preference from an old fact that appeared in twenty candidate sets, entered no final context, and can now expire.
Tool definitions and results are part of context security
Tool descriptions influence model behavior, while tool execution changes the outside world. The Model Context Protocol 2025-11-25 tools specification separates these concerns through named tools, JSON Schema inputs, optional output schemas, and explicit error results. It also says clients must treat tool annotations as untrusted unless they come from trusted servers.
A model may produce a schema-valid refund request for the wrong tenant. MCP’s specification requires servers to validate tool inputs, apply access controls, rate limit calls, and sanitize outputs; clients should validate results before the next inference and request confirmation for sensitive operations. Authorization rejects that refund using account ownership, regardless of the model’s confidence or the clarity of the tool description.
Tool results inherit the trust of their source, even when they arrive through a trusted connector. A browser result, support ticket, or repository file can contain hostile instructions. Context assembly should label those payloads as untrusted data before the next inference call, while authorization and egress controls determine what the agent can do with them.
from dataclasses import dataclass
from typing import Any
@dataclass(frozen=True)
class ToolResult:
tool_name: str
call_id: str
source_trust: Trust
output_schema: str
payload: Any
is_error: bool
def admit_tool_result(result: ToolResult, allowed_schemas: set[str]) -> ContextBlock:
if result.output_schema not in allowed_schemas:
raise ValueError("unapproved tool output schema")
return ContextBlock(
block_id=result.call_id,
kind="tool_error" if result.is_error else "tool_result",
trust=result.source_trust,
source_uri=f"tool://{result.tool_name}",
version=result.output_schema,
token_estimate=0,
payload=result.payload,
)
Manus reports that removing errors deprives the model of evidence needed to avoid repeating a bad action. After a failed call, the next context can retain its error class, redacted arguments, and retry state while moving a huge stack trace to an artifact. The model then sees why the call failed without paying the full token cost on every later step.
Stable prefixes reduce latency and cost
Manus reports an agent input-to-output ratio near 100:1. In the same production account, its Claude Sonnet example prices cached input at $0.30 per million tokens and uncached input at $3 per million, a 10-to-1 gap. Those prices can change, while the workload shape remains: an agent repeatedly sends a large prefix to generate a small action.
Now move one timestamp from the suffix into the system prompt. That early token difference invalidates reuse after its position, even though policy, tool definitions, and canonical examples remain identical. Unstable JSON key ordering can produce the same effect. Keeping volatile request IDs, retrieval results, and recent events behind a deterministic prefix preserves more of the reusable computation.
Include the policy version and tool-schema version in the cache key. On either change, invalidate the affected prefix and graph hit rate with input-token volume for each agent version and workflow stage.
Break the metric down by agent version and workflow stage. A strong site-wide average can still hide one expensive branch that misses every turn.
Scope multi-agent handoffs explicitly
Give a sub-agent a task packet containing its goal, evidence, permissions, and return contract. Google ADK distinguishes agents used as tools from hierarchical agent transfer. A tool-like specialist can receive one focused request and selected artifacts. A transferred agent may need more history, so its handoff scope should be explicit and reviewable.
Role attribution is the subtle failure. Foundation-model APIs usually understand system, user, assistant, and tool roles; they don’t inherently distinguish Assistant A from Assistant B. If a new agent receives the old agent’s messages as its own assistant turns, it may claim actions it never performed. Recast prior work as attributed context and pass execution receipts separately.
- Include the goal, acceptance criteria, permissions, relevant evidence, and unresolved decisions.
- Exclude unrelated conversation, unavailable tools, private data outside the callee’s scope, and hidden assumptions.
- Require the callee to return source handles, confidence limits, and any state mutation it requests.
- Give the parent enough information to verify the result without importing the worker’s full exploration trace.
Anthropic notes that focused sub-agents may consume tens of thousands of tokens internally and return a summary of roughly 1,000 to 2,000 tokens. Evidence handles and unresolved uncertainty determine whether that compressed return packet remains useful to the parent agent.
A release matrix for context quality
Offline task success alone misses expensive and unsafe context failures. Evaluate selection, attribution, security, cost, and recovery as separate dimensions. Define thresholds as product acceptance criteria based on risk; don’t present them as universal benchmarks.
| Gate | Measurement | Suggested release criterion | Failure artifact |
|---|---|---|---|
| Evidence recall | Required blocks admitted per case | 100% for safety-critical cases | Rejected block receipt |
| Contradiction | Conflicts surfaced before action | No silent conflict in critical policy | Source pair and resolver decision |
| Isolation | Unauthorized blocks visible | Zero across the security suite | Access-control trace |
| Restoration | Compacted claims traceable to source | All sampled claims restorable | Checkpoint and event range |
| Efficiency | Input tokens and cache hit rate | No regression beyond team budget | Manifest diff by block |
Run adversarial cases too: stale policy versus fresh policy, poisoned retrieval, expired memory, tool-schema drift, oversized output, conflicting sources, and a handoff that removes one required permission. The security suite should include the access, provenance, and authorization controls in this prompt injection defense plan. It should also reflect risks such as prompt injection, insecure output handling, sensitive information disclosure, and excessive agency identified by the OWASP GenAI Security Project.
When a larger window ships, rerun the same matrix. The efficiency row may improve while a poisoned-retrieval case still reaches a tool, or evidence recall may rise while p95 latency breaks the product budget. A failed security or latency row blocks that release, regardless of the advertised window size.
Roll out with shadow manifests and a kill switch
Start by instrumenting the existing agent without changing its context. Record block IDs, token estimates, source versions, cache segments, and tool schemas. This baseline reveals where cost and failures cluster.
- Define typed source records and trust levels for policy, session events, memory, retrieval, tool output, and artifacts.
- Build the context compiler in shadow mode. Compare its manifest with the current prompt while the old path still serves responses.
- Create golden traces for successful tasks, known failures, security attacks, and restoration after compaction.
- Canary the new compiler on low-risk traffic. Track task success, required-evidence recall, denied access, input tokens, cache hit rate, and user corrections.
- Enable rollback by compiler version. Keep storage schemas compatible so a failed presentation strategy doesn’t trap durable state.
| Failed control | Owner |
|---|---|
| Trust label or high-risk action policy | Security |
| Compilation, cache key, or context trace | Platform engineering |
| Task evidence or acceptance criterion | Product team |
| Retention or deletion | Data owner |
The manifest names the failed processor or policy version, giving the incident router a concrete owner.
Frequently Asked Questions
Should raw chain-of-thought be stored for debugging?
No. Store observable inputs, selected context blocks, tool calls, outputs, policy decisions, and final responses. Those artifacts support replay and diagnosis without retaining private hidden reasoning. If a system needs an explanation, request a concise user-facing rationale tied to source IDs. Apply normal privacy, access, and retention controls to the trace because retrieved evidence and tool arguments may still contain sensitive data.
How often should an agent compact its history?
Trigger compaction from workload signals such as token cost, latency, and retrieval quality. A universal turn count misses phase changes inside a workflow. Trigger evaluation when token cost, latency, or retrieval quality crosses a team-defined budget, then test whether a checkpoint preserves constraints, open decisions, failures, and restoration handles. Compaction frequency can vary by workflow stage. A planning phase may need dense history, while repetitive execution can replace old successful outputs with compact receipts much sooner.
Does RAG solve context engineering?
RAG finds candidate evidence. The surrounding runtime still enforces access control, ranks authority and freshness, resolves contradictions, fits a token budget, labels trust, and orders blocks. Retrieval may also be skipped when the current task is fully covered by verified session state. In a compiler pipeline, those decisions sit before and after the retriever, so changing the vector database leaves the other lifecycle controls intact.
What should happen when the context compiler fails?
Fail into a lower-authority mode. A read-only assistant may answer from verified policy, ask for clarification, or queue work for review. It shouldn’t execute a write with missing provenance, unresolved policy conflict, or an invalid tool schema. Emit a structured failure receipt containing the compiler stage, rejected block IDs, policy version, and correlation ID so operators can diagnose the failure without exposing sensitive payloads.
How do you compare two context strategies fairly?
Replay the same task set, model version, tool environment, source snapshot, and sampling settings. Compare task success alongside evidence recall, contradiction handling, security violations, input tokens, cache hit rate, latency, and restoration. Randomize strategy order when online effects matter. Publish the workload contract with the result; a single average score hides whether one strategy wins easy tasks while failing the rare cases that carry real risk.
Production checklist
- Separate durable state from the temporary working context.
- Give every context block identity, provenance, trust, version, and size.
- Apply access control before retrieval and admission.
- Keep policy and tool schemas stable, versioned, and cache-aware.
- Make compaction restorable through event ranges and artifact handles.
- Validate memory writes and support correction, expiry, and deletion.
- Scope sub-agent handoffs and preserve action attribution.
- Evaluate required-evidence recall, security isolation, restoration, and cost.
The first useful change is small: log a context manifest for one workflow and inspect ten failures. For each failure, compare the required source with the selected block IDs, trust labels, source versions, and serialized order. The resulting defect list points to a specific compiler stage: access filtering, retrieval, contradiction resolution, admission, ordering, or restoration.
Primary references for implementation are Anthropic’s context engineering analysis, Google’s ADK context architecture, the MCP tools specification, and Manus’s production context lessons. Use their mechanisms as inputs, then set budgets and release criteria from your own traces.
