Context Assembly and Management

Foundations defined context engineering as the design of information available to one model inference. This chapter implements that idea at runtime. Its focus is not better prompt phrasing; it is the assembly pipeline that decides what enters a particular decision, in what form, with which authority and provenance, and what is deliberately left out.

Thesis

Context should be assembled as a typed, budgeted, and inspectable view of current task needs—not accumulated as an ever-growing transcript.

The source of truth remains outside the prompt. Each call receives a projection appropriate to one decision. The assembly record should make it possible to answer: which information was considered, where it came from, which transformations occurred, why it was included, and what limits or access rules applied?

Define a context contract per decision

A useful contract names the slots a decision needs:

Slot	Purpose	Example controls
Governing instructions	Defines role, policy, output and decision contract	Version, precedence, protected source
Goal and current subtask	States what this call must accomplish	Bounded scope, success condition
Authoritative state projection	Represents facts currently true in the task or environment	State version, freshness, access control
Selected history	Preserves relevant commitments, corrections, and attempts	Compaction policy, omitted-history marker
Evidence	Supplies retrieved or user-provided support	Provenance, authority, date, citation identifier
Tool capabilities	Describes operations available at this step	Eligibility, schema, risk, permission
Recent outcomes	Reports typed tool or verifier results	Outcome class, action ID, uncertainty
Pending decisions and limits	Exposes approvals, deadlines, budget, and stop conditions	Owner, remaining amount, expiry

Not every call needs every slot. A classifier should not inherit an entire tool catalog; an action-argument call should not receive unrelated private history. Explicit slots make omissions and excess visible.

The assembly pipeline

1. Establish identity, authority, and decision scope

Resolve the active principal, tenant, run, step, and decision contract before retrieval. Access filtering must occur before sensitive content enters model-visible context. A prompt instruction saying “do not reveal this” is not an access-control boundary.

2. Load authoritative state

Read current task facts from their systems of record and project only the fields needed for the decision. Include a version or observation time when staleness matters. Do not ask the model to infer current state from a conversation containing both old and corrected values.

3. Retrieve candidate evidence

Search the collections allowed for this principal and task. Retrieval may combine metadata filters, lexical search, embeddings, graphs, or application-specific queries. The original RAG work established the value of combining model generation with updateable external knowledge (Lewis et al., 2020); it does not make retrieval results authoritative by default.

4. Filter, rank, diversify, and deduplicate

Relevance is only one criterion. Apply authority, freshness, permission, document status, source diversity, and conflict rules. Retain contrary evidence when it changes the decision. Deduplicate repeated passages so copied material does not acquire false weight.

5. Select history and memory

Use structured task state for current facts. Include prior messages only when wording, commitments, corrections, or interaction continuity matter. Retrieved memory is a candidate input with scope and validity, not a command or current truth. The State and Memory chapter owns its lifecycle.

6. Expose only eligible tools

Provide tools available in the current state and authority context. Large overlapping catalogs add tokens and make selection ambiguous. Tool descriptions and schemas are part of context, but permission and action validation remain outside the model.

7. Allocate a budget and compose

Reserve capacity for instructions, state, evidence, tool definitions, history, and output. Prioritize required information rather than truncating the concatenated prompt at an arbitrary boundary. Preserve stable prefixes when provider caching makes that useful, but do not let cache design keep obsolete policy in service.

8. Record the assembly manifest

Retain identifiers and versions for included items, queries and filters, transformations, exclusions that materially affect diagnosis, token counts, and the final template version. Sensitive raw context need not be copied into every trace; references and protected snapshots can support audit with appropriate retention.

Position and representation matter

A large context window is capacity, not proof that every item will be used. Lost in the Middle found positional sensitivity in the models and long-context tasks it evaluated (Liu et al., 2024). Newer models may behave differently, so treat layout as an evaluation variable rather than a permanent rule.

Use representation to reveal structure:

• separate governing instructions from evidence and untrusted content;
• label source, time, authority, and citation identifier where material;
• place the immediate decision and required output contract clearly;
• express current state as structured fields when exactness matters;
• distinguish absence, unknown, redacted, and not applicable; and
• preserve typed tool outcomes rather than paraphrasing all results into prose.

XML, JSON, Markdown headings, message roles, or provider-specific blocks can all work. Delimiters help a model interpret structure; they are not a security boundary.

Compaction is a lossy state transition

Long-running work eventually exceeds a practical input budget. Common responses are:

• remove irrelevant turns;
• replace raw tool output with a referenced artifact;
• summarize completed work;
• maintain a structured ledger of goals, decisions, evidence, effects, and pending work;
• retrieve older detail on demand; or
• isolate a subtask and return a bounded result.

Every compaction should state what must survive: current goal, accepted decisions, user corrections, committed effects, unresolved obligations, evidence identifiers, uncertainty, and resume conditions. A summary generated by a model is a derived view. It should link to the underlying record and be replaceable when found inaccurate.

Anthropic describes context engineering for agents as iterative curation under finite attention and recommends combinations of compaction, structured note-taking, and subagents for long tasks (Anthropic, 2025). These are useful patterns, not substitutes for identifying the information a local task must preserve.

Preserve trust and instruction boundaries

Webpages, documents, emails, repository content, tool output, and memory may contain text that attempts to redirect the model. Google describes indirect prompt injection as a continuing security challenge for systems that consume untrusted web content (Google Security Blog, 2025). No wrapper text guarantees that a model will ignore it.

Use layered controls:

1. label external content as data and retain its origin;
2. exclude secrets and data the current operation does not need;
3. limit the tools and authority available to the step;
4. validate model proposals and outputs in application code;
5. require approval for consequential effects; and
6. test with hostile and conflicting content.

Trust labels should survive extraction and summarization. If a summary of an untrusted page is later stored as “memory” without provenance, the boundary has been erased.

Freshness, invalidation, and caching

Context can become stale between retrieval and action. Define freshness by source and consequence: a policy may remain valid for months; inventory or permissions may need a read immediately before commit. Include “valid as of” information when the model must reason about time.

Cache stable instructions, schemas, and immutable evidence when useful. Key dynamic caches by tenant, permissions, source version, query, transformation, and relevant policy version. Invalidation is part of correctness. A fast stale answer is not a successful optimization.

Evaluate the assembly system

Test the actual pipeline, not only a hand-crafted golden prompt. Cases should include:

• a required fact missing from the top retrieval results;
• stale and current versions of the same source;
• conflicting authoritative and non-authoritative claims;
• duplicated evidence;
• malicious instructions inside an otherwise relevant document;
• an overlarge tool catalog;
• a summary that drops a pending obligation;
• permission changes after a cache entry was created; and
• relevant information near different context positions.

Measure retrieval coverage, unsupported claims, citation correctness, context size and latency, stale-input rate, access violations, action quality, and end-to-end outcomes. A retrieval score alone does not show that the model used evidence correctly.

Failure modes

Append-only context

Every turn and tool result remains forever. Noise, cost, contradictions, and exposure grow. Reassemble from current needs and authoritative state.

Top-k as the context policy

Similarity ranking determines truth and completeness. Add authority, freshness, diversity, permission, and task-specific requirements.

Transcript as state

The model reconstructs facts from prose and acts on superseded values. Project current state explicitly.

Summary as source of truth

A lossy derivative overwrites evidence and unresolved uncertainty. Link summaries to underlying records and make their scope visible.

Flattened trust after transformation

Extracted or summarized untrusted text loses its label and later appears authoritative. Propagate provenance and trust metadata.

Hidden truncation

The provider or application drops content without knowing which requirement disappeared. Budget by slot and record omissions.

Universal context template

Every decision receives the same history, tools, and documents. Define contracts by decision class.

Fieldbook guidance

Review one assembly manifest from a successful case and one from a failure:

decision contract and template version
principal, tenant, run, step, and state version
included instruction, state, evidence, history, memory, and tool identifiers
source authority, trust, freshness, and permission decisions
retrieval query, filters, ranking, deduplication, and transformations
slot budgets, actual tokens, omissions, and truncation
cache keys and source versions
resulting action or artifact and its verifier outcome

If the team cannot reconstruct why a material item was present or absent, the context pipeline is not yet observable enough to improve reliably.

Checklist

• Does each material model decision have a context contract with required and excluded information?
• Are identity, tenant, permission, and decision scope resolved before retrieval?
• Is current state projected from authoritative records instead of inferred from the transcript?
• Do retrieval rules consider authority, freshness, diversity, conflicts, and permission as well as similarity?
• Are untrusted content and its transformations labelled through the pipeline?
• Are only currently eligible tools exposed?
• Are context budgets allocated by purpose, with required information protected from arbitrary truncation?
• Does compaction preserve decisions, effects, obligations, evidence, and uncertainty?
• Can the system reconstruct the inputs, versions, transformations, and omissions behind a decision?
• Do evaluations exercise stale, conflicting, missing, hostile, and overlong context?

References and how they are used

• Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” — foundational primary research for combining generation with external, updateable evidence; not treated as proof that retrieved passages are true.
• Liu et al., “Lost in the Middle: How Language Models Use Long Contexts” — peer-reviewed evidence that inclusion and effective use can differ; findings are scoped to the tested models and tasks.
• Anthropic, “Effective context engineering for AI agents” — Anthropic engineering article used for finite context, just-in-time retrieval, compaction, structured notes, and context isolation.
• Google Security Blog, “Mitigating prompt injection attacks with a layered defense strategy” — Google Security Blog article used to support layered controls for indirect prompt injection rather than reliance on prompt wording.
• NIST AI 600-1, Generative Artificial Intelligence Profile — cross-sector risk-management profile supporting content provenance, data lineage, access controls, and evaluation of GAI information flows.