Context Engineering: Giving the Model What It Needs

A model does not see the product, database, organization, or world in which it operates. At each inference step, it can act only on behavior learned during training and the information the system makes available to that call.

That information is often described casually as “the prompt.” In a real system it may include system instructions, a user request, conversation history, retrieved documents, tool definitions, tool results, examples, permissions, and selected task state. Deciding what belongs there is a system-design problem, not merely a writing exercise.

Thesis

Context engineering is the deliberate design of the information available to a model at each inference step. Its goal is to provide sufficient, relevant, authoritative, current, and permitted information—without assuming that more tokens automatically produce better behavior.

Context is therefore a governed runtime dependency. It needs an owner, a selection policy, provenance, trust boundaries, limits, and evaluation. Prompt engineering remains useful, but it addresses only part of this wider responsibility.

This chapter establishes the mental model. The later Architecture chapter will own retrieval, ranking, compaction, caching, memory, and runtime assembly mechanisms.

What context is—and is not

For this chapter, context means the information represented in the input to one model inference. The exact serialization varies by model and provider, but the engineering distinction is stable.

Context is more than the user message

A model call may be shaped by several sources:

Source	What it contributes	Typical risk
System and developer instructions	Role, task, policies, output requirements	Conflicts, obsolete rules, or unenforced promises
User input	Goals, preferences, supplied facts	Ambiguity, missing information, or hostile instructions
Retrieved evidence	Domain knowledge and current facts	Irrelevance, staleness, weak authority, or injection
Tool definitions	Available operations and expected arguments	Too many similar tools, misleading descriptions, or excessive reach
Tool results	Observations from an external system	Errors, partial results, stale values, or untrusted content
Conversation or task history	Prior decisions, commitments, and intermediate work	Accumulated noise, contradictions, or sensitive data retention
Examples	Concrete demonstrations of desired behavior	Overfitting to examples or copying accidental details
Selected state or memory	Facts carried across steps or sessions	Incorrect retrieval, outdated facts, or hidden assumptions

These sources are not equally trustworthy and should not have equal authority. A policy maintained by the application, a preference stated by the user, and instructions embedded in a retrieved webpage are all text, but they are not the same kind of instruction.

Context is not the model’s entire knowledge

Model parameters encode patterns learned during training, but engineers generally cannot enumerate or update that knowledge with the precision of an application database. Runtime context supplements that capability with task-specific instructions and evidence. It does not reveal exactly which training information will influence a response.

A context window is capacity, not attention guaranteed

The context window limits how much input and generated output a model can process in a call. Fitting information inside that limit does not prove that the model will use every item correctly.

In Lost in the Middle, Liu and colleagues found that models on their tested multi-document question-answering and retrieval tasks often performed better when relevant information appeared near the beginning or end than in the middle of a long input (Liu et al., 2024). The models and tasks will age, so the result is not a universal positional law. The durable lesson is narrower: available information is not necessarily effectively used information. Test the behavior of the actual model and context arrangement.

Prompt, context, state, and memory

These concepts interact but should not be used interchangeably.

Concept	Practical meaning	Key question
Prompt	Instructions or content intentionally supplied to elicit behavior	What are we asking the model to do?
Context	Everything made available to the model for this inference step	What can shape this response or decision now?
State	Facts about the task or environment that persist or change across steps	What is currently true outside the model call?
Memory	A mechanism that retains and later retrieves information across interactions	What might be recovered for future work?

A purchase order can exist as durable state in a database. A memory service can retain the user’s delivery preference. Neither affects a particular inference until the system retrieves and represents the relevant information in context—or uses it directly in deterministic application logic.

This distinction prevents two common mistakes. First, a conversation transcript is not a reliable source of truth merely because it is visible to the model. Second, storing information does not ensure that the right information will be retrieved at the right time.

The context selection problem

Good context is not the largest possible context. It is the smallest set that is sufficient for the intended behavior under the actual model and task, with safety margin for cases the selection process may miss.

Selection balances properties that can conflict:

• Relevance: Does this item help with the current decision?
• Completeness: Is a material fact or constraint missing?
• Authority: Is the source entitled to establish this fact or instruction?
• Freshness: Is the information still valid at the time of use?
• Provenance: Can the system identify where it came from and how it changed?
• Consistency: Does it conflict with another item, and is precedence defined?
• Permission: May this information be disclosed to this model, tool, user, or downstream service?
• Cost: What latency, token use, and operational complexity does it add?

Adding an entire repository may preserve facts that a retriever would miss, but it can increase cost and dilute the relevant signal. Aggressive retrieval may be efficient, but a missed document can make the answer confidently incomplete. There is no context strategy independent of the task and its evaluation.

Treat context as a typed information flow

Plain text can obscure origin and authority. A useful mental model is to treat each context item as if it carried a type:

content:
source:
source type:
authority:
created or observed at:
valid at:
trust domain:
task relevance:
permitted audience:
transformation history:

The runtime representation does not need to contain every field verbatim. The system should retain enough metadata to apply selection and access rules, support citations where required, and diagnose why an item was included.

NIST’s Generative AI Profile recommends documenting data origin, content lineage, transformations, and upstream dependencies, and evaluating data and content flows (NIST AI 600-1, 2024). It does not prescribe this record; the form above is Fieldbook guidance derived from that broader obligation.

Retrieval adds evidence, not truth

Retrieval-augmented generation (RAG) made a durable architectural idea explicit: a model can be given selected information from an external, updateable source instead of relying only on parametric knowledge. The original RAG research also identified provenance and knowledge updates as motivations (Lewis et al., 2020).

Retrieval nevertheless introduces its own chain of claims:

1. The system searched the right collection.
2. The collection contained the required information.
3. Ranking selected the relevant material.
4. The selected source was authoritative and current enough.
5. Transformation or summarization preserved the material meaning.
6. The model used that evidence correctly.
7. The output preserved enough provenance for the use case.

Failure at any step can produce a grounded-looking but unsupported result. Retrieval narrows some knowledge gaps; it does not prove completeness or truth.

Separate instructions from untrusted content

External documents, webpages, messages, code, and tool results can contain text that looks like an instruction. A model may respond to that text even when the application intended it to be treated only as data. OWASP describes this class of risk as indirect prompt injection and recommends separating external content, limiting privileges, validating outputs, and requiring approval for high-risk actions (OWASP LLM01:2025).

No formatting convention or instruction hierarchy guarantees that a model will ignore hostile content. Context engineering can preserve the trust distinction, but security also requires controls outside the model:

• retrieve only information the current principal may access;
• label and isolate untrusted content;
• avoid placing secrets in model-visible instructions;
• give tools the least capability and permission required;
• validate proposed actions independently; and
• require meaningful approval before high-consequence effects.

The Model Context Protocol specification similarly warns clients to treat tool annotations as untrusted unless they come from trusted servers (MCP Tools specification, 2025). That is one protocol-specific example of a general rule: connectivity does not establish trust.

Context across a multi-step task

In a one-call capability, context may be assembled once. In a multi-step workflow or agent, every result creates a new selection problem. Tool output, observations, intermediate artifacts, user corrections, and changed state may all matter to the next decision.

Keeping the full history is simple but eventually expensive and noisy. Summarizing or compacting history saves space but can erase provenance, uncertainty, rejected alternatives, or pending work. Writing durable task state outside the transcript improves continuity but requires a schema and update rules. Delegating a subtask can isolate detail but creates a handoff boundary.

Anthropic describes context engineering as the iterative curation of instructions, tools, external data, and message history within a finite resource (Anthropic, 2025). This is useful vendor evidence, not a universal implementation recipe. The correct mechanism depends on what later steps must know and how the system can verify that knowledge survived.

Design options and trade-offs

Approach	When it helps	Main cost or failure risk
Supply all bounded source material	Small corpora where missing a detail is costly	Token cost, distraction, positional sensitivity, and data exposure
Retrieve task-relevant material	Large or changing knowledge collections	Retrieval misses, ranking errors, and new infrastructure
Maintain a structured task record	Long-running work with explicit milestones and decisions	Schema design and incorrect or incomplete state updates
Compact or summarize history	Work that exceeds a practical context size	Loss of detail, provenance, uncertainty, or earlier constraints
Isolate subtasks and return results	Independent investigations with large intermediate context	Handoff loss, inconsistent assumptions, and coordination cost
Ask the user for missing context	A material preference or interpretation cannot be inferred safely	Interruption and transferred effort

These approaches can be combined. The engineering task is to make the selection rules and losses explicit, then test them on representative work.

Failure modes

Prompt-only thinking

The team repeatedly rewrites instructions while ignoring missing evidence, poor tool descriptions, stale history, or incorrect state. The prompt becomes responsible for failures elsewhere in the system.

Context dumping

Everything available is placed into the call because the window is large enough. Cost grows, conflicting instructions accumulate, and important evidence becomes harder to use.

Retrieval as proof

The output cites a retrieved passage, so the team assumes the conclusion is correct. The source may be irrelevant, outdated, non-authoritative, incomplete, or misinterpreted.

Transcript as database

The system relies on prior conversation text as durable state. Corrections, transactions, and current facts become ambiguous, duplicated, or silently lost during compaction.

Memory without validity

A stored preference or fact is retrieved without scope, timestamp, or confirmation. The system confidently applies information that is no longer true.

Flattened trust

Application policy, user requests, retrieved pages, and tool output are concatenated without preserving their authority. Untrusted content can redirect behavior or expose data.

Invisible assembly

The system cannot reconstruct which context items, versions, or transformations shaped a decision. Failures appear to be mysterious model behavior even when the wrong information was supplied.

Fieldbook guidance

Define a context contract for each material model-mediated decision:

Decision or artifact:
Required instructions:
Required evidence:
Optional supporting information:
Authoritative sources:
Freshness requirements:
Conflict and precedence rules:
Untrusted sources and isolation:
Relevant state:
Memory retrieval conditions:
Exclusions and sensitive data rules:
Token or latency budget:
Evidence retained for diagnosis:
Evaluation cases for missing, stale, conflicting, and hostile context:

This is a review worksheet, not a new protocol. Architecture work may implement it through retrieval, state stores, memory, caching, compaction, or simpler code.

Checklist

• Can we name every class of information that enters the model input?
• Have we separated instructions, evidence, examples, state, memory, and untrusted content?
• Does each material fact have appropriate provenance, authority, and freshness?
• Are conflict and precedence rules defined outside vague prompt wording where possible?
• Do retrieval and compaction preserve required evidence, decisions, and unresolved uncertainty?
• Is durable state maintained outside the transcript when correctness depends on it?
• Does memory have scope, validity, retrieval, correction, and deletion rules?
• Is access filtering applied before sensitive information enters model context?
• Are tool definitions and results treated according to their trust domain?
• Can we reconstruct the context that shaped a failed or consequential decision without logging unnecessary secrets?
• Have we tested missing, irrelevant, stale, conflicting, oversized, and adversarial context?
• Does adding more context measurably improve the intended outcome enough to justify its cost and risk?

References and how they are used

• Anthropic, “Effective context engineering for AI agents” (2025) — Anthropic engineering article defining context engineering beyond prompt wording and discussing finite context, tools, history, compaction, and long-running agents. The chapter uses its practical scope but does not treat its recommendations as a standard.
• Liu et al., “Lost in the Middle: How Language Models Use Long Contexts” (TACL, 2024) — peer-reviewed evidence that information placement affected performance on the tested long-context tasks. It does not establish one universal ordering rule for all current models.
• Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” (NeurIPS, 2020) — foundational RAG research supporting external, updateable knowledge and provenance as motivations. Its specific architecture and results should not be generalized into a guarantee that retrieval is correct.
• NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1 (2024) — voluntary lifecycle risk-management profile supporting documentation and evaluation of data origin, lineage, transformations, and upstream dependencies.
• OWASP, “LLM01:2025 Prompt Injection” — OWASP community risk entry supporting the treatment of external content as untrusted and the use of external privilege and consequence controls. It does not offer a foolproof prevention method.
• Model Context Protocol, “Tools” specification (2025-06-18) — a protocol example showing how tool definitions and results enter model interactions and why annotations from untrusted servers should remain untrusted.