Agent Engineering Principles

Agent engineering is not the practice of adding as much autonomy, memory, tool use, or model reasoning as a system can support. It is the practice of deciding where model-mediated behavior creates value, then building the surrounding software so that behavior has the context, authority, evidence, and recovery paths appropriate to its consequences.

In this Fieldbook, agent engineering means designing, building, evaluating, and operating AI-native software systems that exhibit goal-directed behavior, tool use, state management, feedback loops, and bounded autonomy. It is broader than implementing one agent and different from using an agent to assist software development.

This chapter collects the Foundations series into nine principles. They are Fieldbook editorial guidance, not an industry standard or maturity model. They apply in different proportions to a bounded model capability, a predefined AI workflow, and an agent.

Thesis

An effective agentic system gives model-mediated behavior enough discretion to handle semantic uncertainty and adapt to feedback, but not responsibility that the model cannot reliably hold. The surrounding system remains responsible for context, state, authority, evidence, recovery, and accountability.

The principles are deliberately stated as design tensions, not slogans. “More deterministic” is not always safer or simpler; “more agentic” is not automatically more capable or advanced. The right boundary depends on the task, consequence, reversibility, and observed results.

1. Start with the task and outcome, not the agent label

Define the work before choosing the architecture:

Who needs what outcome?
What makes the task difficult to specify with conventional rules?
Which inputs, judgments, actions, and exceptions matter?
What counts as success, partial success, or an unresolved case?
What consequence follows when the system is wrong?

An “agent” is not a product requirement. A bounded model operation or predefined workflow may solve the task with less cost and a smaller failure surface. Conversely, an open-ended task may need model-directed adaptation because enumerating every path is not economical.

The AI-native chapter provides the essentiality test: if removing the model leaves the core purpose and operating model substantially intact, the system may have an AI-enhanced feature rather than an AI-native design. That is a description, not a criticism.

Tension: Starting from current requirements can understate capabilities users discover only through experimentation. Treat early prototypes as learning instruments, but do not turn an impressive demonstration into an operating claim without explicit outcomes and boundaries.

2. Use the least model discretion that earns its cost

Delegate runtime decisions when flexible interpretation or adaptation improves the target outcome. Keep a transition in deterministic code when its rule is stable, economical to express, and important to reproduce or enforce.

Assess delegation separately across:

path discretion—steps, branches, tools, retries, and stopping;
action authority—recommendation, staging, or committed side effect;
operating horizon—one turn, a bounded run, or long-running work; and
human involvement—operation, collaboration, advice, approval, exception handling, or observation.

Increase one dimension because evidence supports it, not because the model can accept another tool. Anthropic’s distinction between predefined workflows and model-directed agents, and its recommendation to add complexity only where it improves results, is useful engineering evidence (Anthropic, 2024). It remains vendor-authored engineering material, not a universal architecture rule.

Tension: Too little discretion can move complexity into brittle branches and prevent useful recovery from exceptions. “Least” means the lowest level that performs the work well—not the smallest number of model calls.

3. Treat context as a governed dependency

Every model decision depends on the information supplied to that inference step. Own the complete context, not only the wording of a system prompt.

For material decisions, define:

required instructions and evidence;
authoritative and permitted sources;
provenance and freshness;
conflict and precedence rules;
relevant task state and memory retrieval conditions;
trust boundaries around user, retrieved, and tool-provided content; and
context-size, latency, and retention limits.

The context-engineering chapter explains why a larger context window does not guarantee effective use and why retrieval supplies evidence rather than truth. Anthropic’s context-engineering article similarly treats context as finite and iteratively curated across instructions, tools, external data, and history (Anthropic, 2025).

Tension: Aggressive selection can omit an unexpected but material fact. Whole-context input can be the safer choice for bounded work. Evaluate the actual selection strategy rather than applying “less context” as a universal rule.

4. Keep state and authority outside the model’s implied memory or intent

The model can propose a state change or describe what it believes happened. The application must retain the authoritative record of what is actually true.

Use explicit system mechanisms for:

task status, completed and pending work, and ownership;
transaction state and idempotency;
identities, permissions, and delegated authority;
approvals and their scope and expiry;
durable user or organizational preferences; and
externally observable outcomes.

A transcript is useful evidence but a poor substitute for a database, ledger, or task record when correctness depends on current state. Stored memory also needs scope, validity, correction, and deletion rules. It should not silently become authority merely because it was retrieved.

Tension: Formal state models add design and synchronization cost. A short advisory interaction may need only a transcript. Add durable structure where continuity, coordination, reconciliation, or consequence requires it.

5. Make uncertainty and unresolved cases explicit

Separate ambiguity in the request, missing or conflicting evidence, run-to-run variation, environmental change, and system error. They call for different responses.

Define what may vary and what must not. Give the system explicit routes such as:

ask for clarification;
retrieve or validate more evidence;
complete with disclosed assumptions;
abstain because support is insufficient;
retry a safe, isolated operation;
fall back to a narrower capability;
request approval; or
escalate to a responsible person.

A system that only represents success and error encourages unsupported certainty or discards useful partial work. Unresolved states should be visible to product behavior, telemetry, and evaluation.

Tension: Too many internal states can burden users and operators. Product interfaces may combine or translate them, but the underlying engineering distinction should remain available for correct routing and diagnosis.

6. Enforce hard constraints outside model judgment

Instructions can communicate rules to a model; they do not by themselves enforce authorization, privacy, transaction integrity, budget, or irreversible-action boundaries.

Use deterministic mechanisms where violation is unacceptable:

authentication and authorization;
least-privilege tool scopes;
schemas, ranges, allowlists, and policy engines;
sandboxing and resource budgets;
transaction constraints and idempotency;
staged execution and approval tokens; and
output encoding and downstream validation.

OWASP’s prompt-injection and excessive-agency risk entries describe how manipulated content combined with excessive permissions can create damaging actions, and recommend external privilege limits and meaningful approval (OWASP LLM01:2025; OWASP LLM06:2025). These controls reduce consequence; they do not make prompt injection impossible.

Tension: Every hard boundary rejects some behavior. Apply it where the invariant matters more than open-ended flexibility, and provide an exception or escalation path where legitimate work would otherwise be blocked.

7. Evaluate the whole system against representative work

The behavior users experience comes from the model together with context, tools, policies, orchestration, application code, and environment. Evaluate that assembled system on tasks that resemble its real work.

Define success across the relevant levels:

output: the answer or artifact;
process: the data, tools, approvals, and constraints used; and
outcome: the resulting state or effect.

Use typical, edge, adversarial, and consequential cases. Run multiple trials when variability matters. Combine deterministic checks, human judgment, and calibrated model-based evaluation according to the claim. Do not substitute a public benchmark or one convincing run for local evidence.

Anthropic’s agent-evaluation article makes the system boundary explicit by evaluating a model and agent harness together, including traces and environmental outcomes (Anthropic, 2026). Earlier production-ML research likewise shows that model quality is entangled with data and software dependencies (Sculley et al., 2015).

Tension: Formal evaluation can prematurely freeze an exploratory objective. Start with lightweight cases and human review, then increase rigor as the capability and consequence become clearer. Always state which decision the evidence is meant to support.

8. Design for observation, interruption, and recovery

A system that can act across multiple steps must expose enough evidence and control to answer:

What goal and configuration started this run?
What context, tools, state, and approvals shaped each material decision?
What changed in the environment?
Is the work progressing, waiting, blocked, or looping?
Can it be paused, cancelled, resumed, retried, reversed, or safely abandoned?
Which artifacts survive failure, and which effects may already have committed?

Observation is not the same as storing every internal token. Retain the events, versions, decisions, inputs, outputs, and state transitions needed for diagnosis and accountability while respecting privacy, security, and cost.

Recovery should be part of the control design. Define stopping conditions, time and spend budgets, retry boundaries, checkpoints, idempotency, fallback, and escalation before a long-running system fails in production.

Tension: More telemetry and checkpoints increase cost and data exposure. Collect what supports a named operational question, and apply access and retention rules to the evidence itself.

9. Keep human and organizational accountability explicit

A model can recommend or execute an action; it cannot own the organizational consequences.

Name the people or roles responsible for:

defining supported use and unacceptable harm;
approving authority and data access;
reviewing evaluation evidence and releases;
responding to incidents and changing conditions;
handling escalation, appeal, and correction; and
deciding when the system should be restricted or withdrawn.

Human involvement must be meaningful. An approval dialog without time, evidence, competence, or a real ability to refuse creates the appearance of control. Conversely, requiring approval for every reversible low-risk step can produce fatigue and eliminate the value of delegated work.

NIST’s Generative AI Profile treats risk management, evaluation, feedback, human roles, and oversight as lifecycle responsibilities for organizations (NIST AI 600-1, 2024). The appropriate process varies by use case; accountability does not.

Tension: Clear ownership can become bureaucratic ceremony. Tie each role and approval to a decision, consequence, and response path rather than adding generic sign-off.

How the principles work together

The principles constrain and support one another:

The task and outcome determine whether model discretion is justified.
Delegated discretion determines what context, authority, and recovery controls are required.
Context and state determine what the system can know at each step.
Uncertainty determines when it should continue, clarify, abstain, or escalate.
Hard constraints bound what uncertainty may affect.
Evaluation supplies evidence that these choices work together.
Observation and recovery make failures diagnosable and containable.
Accountability ensures that evidence leads to responsible decisions.

Optimizing one in isolation can weaken the system. Richer context without trust boundaries increases exposure. More tools without evaluation expand an unmeasured failure surface. More approvals without usable evidence create delay rather than control. Higher benchmark performance without system-level outcomes can improve the wrong component.

Failure modes

Agent-first architecture

The team selects a framework, memory system, and tool layer before defining the task and outcome. Product requirements are bent around the chosen machinery.

Complexity as progress

More agents, longer plans, and additional reflection steps are treated as improvements without comparative evidence. Latency and failure surfaces grow faster than value.

Prompts as policy enforcement

High-consequence rules exist only in instructions. The model is both interpreter and enforcer of the boundary it might misunderstand or be manipulated to cross.

Hidden state and hidden authority

The transcript implies that an action occurred or permission exists, while the external system has a different truth. Recovery and audit become guesswork.

Evaluation after architecture

Success criteria appear only after the system is built, so evaluation measures what the implementation happens to expose rather than the user outcome.

Trace abundance, operational blindness

Every interaction is logged, but no one can identify the current task state, committed effects, failure cause, or responsible owner.

Ceremonial human control

A person clicks approve without sufficient context or repeatedly overrides a system whose authority and workload were designed incorrectly.

Fieldbook guidance

Use this compact record before an architecture review:

Intended user and outcome:
Why model-mediated behavior is essential:
Deterministic workflow and model-directed decisions:
Delegation profile:
Required context and authoritative state:
Permitted tools and action authority:
Allowed variation and unresolved states:
Hard constraints and where they are enforced:
Evaluation tasks, trials, and outcome evidence:
Observation, stopping, recovery, and rollback:
Human roles, escalation, and accountability:
Evidence that added complexity improves the outcome:

If an answer is unknown, record it as an open design question. Do not let the model, framework, or default tool permission answer it accidentally.

Checklist

References and how they are used

Anthropic, “Building effective agents” (2024) — Anthropic engineering article supporting the workflow–agent control distinction, simple composable patterns, environmental feedback, and adding complexity only when results justify it.
OpenAI, “A practical guide to building agents” — OpenAI product-adjacent agent-building article supporting model-managed execution, tools, guardrails, exit conditions, and human intervention. The publication does not establish a universal agent definition.
Anthropic, “Effective context engineering for AI agents” (2025) — Anthropic engineering article used for treating context as a finite, iteratively curated resource across instructions, tools, data, and history.
Anthropic, “Demystifying evals for AI agents” (2026) — Anthropic engineering article used for evaluating the model and agent harness together across tasks, trials, traces, and outcomes.
Sculley et al., “Hidden Technical Debt in Machine Learning Systems” (2015) — production-ML research supporting system-level thinking about dependencies, feedback loops, and change. It predates generative models and agents.
NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1 (2024) — voluntary cross-sector risk-management profile supporting lifecycle risk management, evaluation, feedback, documentation, human roles, and organizational oversight.
OWASP, “LLM01:2025 Prompt Injection” and “LLM06:2025 Excessive Agency” — OWASP community risk entries supporting trust separation, least privilege, external enforcement, and meaningful control over consequential actions. They do not establish foolproof prevention.