Agent Control Loops

An agent is often drawn as a small circle: observe, think, act, repeat. That sketch captures feedback, but it hides nearly every responsibility that determines whether the system is controllable.

Who decides what the model is allowed to observe? Who checks a proposed action before it changes the world? Which state is authoritative after a tool times out? What distinguishes useful iteration from a loop that is consuming tokens without making progress? Who decides that the task is complete—and what evidence supports that decision?

Those responsibilities do not belong to a prompt. They belong to the runtime surrounding the model. The model may choose a route through available actions, but application code still owns the run lifecycle, accepted transitions, committed state, limits, and consequences.

This chapter defines that surrounding structure. It assumes the workflow–agent boundary and delegation profile established in Foundations. It does not reclassify systems by autonomy, prescribe a planning method, or define tool and memory implementations. Its question is narrower:

Once a model has some control over the course of action, how should the system execute that control one bounded, observable, and recoverable step at a time?

Thesis

An agent control loop should be an explicit state-transition system owned by a deterministic harness—not a model repeatedly prompting itself until it says it is done.

On each step, the harness should:

read the goal, authoritative task state, and current run status;
assemble a bounded observation for the next decision;
receive a proposed action or lifecycle transition;
validate that proposal against schemas, permissions, preconditions, and limits;
execute any accepted action and record the actual result;
update authoritative state and evaluate task-relevant progress; and
continue, wait, escalate, fail, cancel, expire, or succeed through an explicit transition.

The model can propose finish, but the harness decides whether the available evidence satisfies the task’s success condition. The model can suggest retrying, but the harness decides whether another attempt is permitted and whether anything material has changed. A maximum-turn limit can stop resource exhaustion; it cannot turn an incomplete run into a successful one.

This separation preserves what agents are useful for—adapting their route from feedback—without delegating the truth of the environment, authority to act, or definition of completion to a probabilistic component.

Problem

A one-shot model feature has a short lifecycle: construct input, call the model, validate output, and return. An agentic subsystem can instead perform an input-dependent number of model calls and actions. Each result changes what the next decision should see.

The resulting loop crosses several boundaries:

model output becomes a request to application code;
application code invokes a tool or changes an environment;
the environment returns data, an error, or an ambiguous partial outcome;
that result becomes context for another model decision; and
the system eventually claims success, failure, or inability to proceed.

Industry descriptions agree that this loop is central. Anthropic describes agents as models using tools from environmental feedback in a loop (Anthropic, 2024). OpenAI describes a run as a loop that operates until an exit condition is reached (OpenAI, “A practical guide to building agents”). ReAct demonstrated the value of interleaving task reasoning and actions so that later decisions can incorporate information from an external environment (Yao et al., 2023).

But “call tools until there are no more tool calls” is only an API loop. It does not answer the architectural questions:

A tool returned HTTP 200, but did the intended state change?
A write timed out, but did the server commit it?
The model produced a final answer, but was the requested artifact created?
A verifier rejected the result, but is another repair attempt likely to add information?
A child agent stopped, but did it succeed, fail, or merely exhaust its budget?
A user cancelled the run while an action was in flight; what is now true?

If the system cannot represent these differences, its apparent flexibility rests on hidden and often unsafe assumptions.

Mental model: the harness owns the loop

Long before LLM agents, autonomic-computing work separated monitoring, analysis, planning, execution, shared knowledge, sensors, and effectors. IBM’s MAPE-K architecture is a useful precedent for keeping environmental observation and action interfaces distinct from the component deciding what to do (IBM Research, 2004).

An LLM agent is not a classical feedback controller, and this analogy does not provide stability or convergence guarantees. Its value is separation of responsibility:

goal + authoritative state + limits
                 │
                 ▼
 observe → propose → validate → execute → record and update → verify
    ▲                                                        │
    └──── continue / wait / escalate / fail / stop ──────────┘

The harness is the deterministic software that owns this cycle. It may be application code, an agent framework, a workflow engine, or several cooperating services. “Deterministic” here does not mean every downstream service behaves predictably. It means the rules for accepting transitions, committing state, enforcing limits, and recording what happened are implemented outside the model’s interpretation.

Some model runtimes perform internal reasoning or several managed operations inside one API call, so the application may observe a coarser loop. That does not remove the external contract: consequential effects still need an enforceable interface, the call needs available limit and cancellation semantics, and its returned status and evidence must map into the application’s run state. The system should not claim step-level observability for decisions that its model provider does not expose.

The six responsibilities below need not be six processes or functions. They need to be distinguishable in the design and trace.

1. Observe

The loop reads the goal, current task state, previous action result, new user or environmental events, pending approvals, and remaining limits. It then constructs an observation suitable for the next decision.

An observation is a representation, not the environment itself. It may be incomplete, stale, malformed, or adversarial. A tool result saying “updated successfully” is evidence; a subsequent read of the system of record may provide stronger evidence. Context Assembly and Management will own selection, provenance, compaction, and trust policy. The loop’s responsibility is to identify which observation version informed each transition.

2. Propose

The decision component proposes one typed transition. Common proposals include:

invoke a named action with arguments;
ask the user for missing information;
wait for an event or approval;
revise or abandon the current route;
create a bounded child task;
report that the goal is blocked or impossible;
escalate to another responsible actor; or
claim that the run is ready to finish.

This proposal may come from a model or from deterministic code. A fixed validation failure does not need another model call merely to select a known error path. Conversely, a semantically ambiguous tool result may justify returning control to the model.

The trace needs the proposal and the decision-relevant record—not private chain-of-thought. Inputs used, action requested, checks applied, and outcome observed are operational facts. A generated explanation of hidden reasoning is not authoritative internal state.

3. Validate

Before an action or lifecycle transition is accepted, the harness checks what can be checked reliably outside model judgment:

the proposal is structurally valid;
the action exists and is available in the current state;
arguments satisfy the declared contract;
identity and authorization permit the operation;
required approvals or preconditions are present;
resource, time, and step budgets allow it;
the operation is compatible with cancellation and retry policy; and
the proposed transition is legal from the current run state.

Validation can reject, normalize, stage, request approval for, or accept a proposal. The later Capabilities, Authority, and Execution Boundaries chapter owns the execution contracts and authority boundaries. The control loop owns the fact that validation is a transition gate rather than optional prompt advice.

4. Execute

The executor performs the accepted action through a bounded interface. Its result should distinguish at least:

confirmed success with returned evidence;
confirmed rejection or failure;
partial success;
timeout or interruption with an unknown external outcome; and
invalid or untrusted response.

Flattening all of these into a text message discards the information needed for safe continuation. An unknown outcome after a write is not equivalent to failure: blindly retrying may duplicate an external effect. Production Engineering will cover idempotency, compensation, checkpoints, and distributed recovery. At the architecture boundary, the loop must preserve outcome semantics rather than invite the model to guess.

5. Record and update

The harness records the accepted transition, action attempt, timestamps, relevant configuration, result, and resulting run status. It updates authoritative task state only according to the action’s commit semantics.

The conversation transcript is not sufficient. It is an input history optimized for model interaction, not necessarily a current, validated representation of orders changed, files written, approvals granted, budgets consumed, or obligations remaining. State and Memory will define storage and validity. The loop requires only that it can reload the facts needed to continue without asking the model to reconstruct truth from prose.

6. Verify and decide

After an action, the system asks two separate questions:

What changed?
Does that change justify continuing, waiting, changing route, or entering a terminal state?

A successful tool call is not automatically progress, and progress is not automatically completion. Downloading ten documents may add evidence, duplicate evidence, or distract from the task. Running a test suite may reveal useful information even when tests fail. Producing another draft may consume resources while leaving quality unchanged.

The verifier therefore judges a named condition using evidence appropriate to the consequence. The harness then accepts a legal transition. The model can inform both judgments, but should not silently collapse them into “I have completed the task.”

A run is a state machine, not a conversation

A run is one bounded attempt to pursue a named goal under a recorded configuration, environment, and set of limits. It may contain many model calls, deterministic transitions, tool actions, pauses, or child runs.

At minimum, the runtime should distinguish these states or their local equivalents:

State	Meaning	What must be retained
created	The goal exists, but execution has not started.	Goal, requester, initial scope, configuration, and limits.
running	The harness may choose and execute another transition.	Current task state, active step, remaining limits, and cancellation signal.
waiting	Progress depends on an expected event, time, response, or approval.	What is awaited, deadline, subscription or resume condition, and owner.
blocked	The current control envelope cannot make responsible progress.	Blocking reason, evidence gathered, attempted routes, and possible escalation.
succeeded	Task-specific success evidence has been accepted.	Outcome, verifier result, committed effects, and residual caveats.
failed	The run ended without success because of a named unrecoverable condition.	Failure class, last known state, effects already committed, and recovery options.
cancelled	An authorized actor requested termination.	Requester, time, in-flight action status, and cleanup result.
expired	A deadline, step, token, cost, or other hard limit ended the run.	Limit reached, partial outcome, committed effects, and whether resumption is allowed.
escalated	Responsibility moved to a person, workflow, or another bounded subsystem.	Recipient, reason, transferred context and state, and ownership confirmation.

waiting and blocked may be suspended rather than terminal states. The important distinction is semantic. A run waiting for a user document should not appear failed; a run unable to progress should not remain “running”; a run that reached its token limit should not appear successful because it emitted a polished summary.

The exact state names are less important than legal transitions. For example, a cancelled run should not later commit a queued action without a new authorization. A failed child run should not be converted into parent success merely because it returned text. A waiting run should resume from persisted state, not from an improvised summary that omits pending effects.

Completion requires evidence

Many agent APIs stop when the model returns a message without another tool call. That can be a useful protocol exit, but it is not a general definition of task success.

Completion evidence can take several forms:

Deterministic state predicate — a required record exists, fields satisfy constraints, or an expected state transition is confirmed.
Executable check — tests pass, a query returns the expected invariant, or a generated artifact parses and runs.
Independent environmental observation — the system reads back the state affected by the action.
Human acceptance — an authorized person reviews the actual outcome and accepts responsibility for the decision.
Model-based evaluation — a calibrated evaluator judges a semantic condition that cheaper checks cannot express.

These can be combined. Consequential actions usually need stronger evidence than advisory drafts. When no independent verifier is economical, the system can return a bounded result with its evidence, unresolved questions, and explicit need for human judgment. It should not silently substitute fluency for proof.

τ-bench evaluates tool-using agents by comparing final database state with an annotated goal state and reports reliability across repeated trials. Its initial experiments showed that plausible interaction does not guarantee correct or consistent environmental outcomes (Yao et al., 2024). The exact benchmark results are not universal, but the evaluation boundary is instructive: verify what happened in the task environment, not only what the agent said.

Self-critique is not external feedback

Asking the same model “Are you sure?” can sometimes change an answer, but it does not add evidence. ICLR 2024 research found that prompted intrinsic self-correction often failed or reduced accuracy on reasoning tasks without external feedback (Huang et al., 2024). CRITIC, by contrast, demonstrated improvements when correction incorporated feedback from tools and external sources in its evaluated tasks (Gou et al., 2024).

Neither result establishes a universal rule for newer models. Together they support a durable distinction:

Another model turn can revise a proposal. Only new or independently evaluated information strengthens the evidence.

A model evaluator may still be appropriate. It is a measurement instrument with false positives, false negatives, configuration sensitivity, and cost—not a deterministic oracle.

Stopping is part of correctness

A loop needs both semantic stopping conditions and safety bounds.

Semantic conditions explain why the task should stop:

success evidence satisfies the goal;
an unrecoverable failure makes the goal impossible under the current scope;
required information or authority is unavailable;
the next material action requires approval or another actor;
the user cancels or changes the goal; or
continuing has negative expected value under an explicit policy.

Safety bounds prevent uncontrolled execution:

maximum steps or model calls;
token, cost, tool-use, or child-run budgets;
wall-clock deadline or idle timeout;
maximum retries by failure class;
recursion or delegation depth;
consecutive no-progress transitions; and
repeated side-effect attempts.

Reaching a safety bound produces expired, blocked, failed, or escalated according to the contract. It does not produce succeeded.

Detect progress, not mere activity

A no-progress detector should compare task-relevant state or evidence across steps. Repeated tool names alone are too crude: polling, pagination, iterative search, and repeated trials can be intentional. Conversely, a loop can avoid identical calls while circling through several actions without changing the task state.

Useful signals include:

no change in the unmet success conditions;
the same failure class recurring without a changed input or strategy;
repeated observations that add no material information;
a cycle in normalized action and state signatures;
declining verifier score or unchanged test failures;
resource consumption exceeding the expected value of another step; and
child runs repeatedly returning the same unresolved dependency.

These are detectors, not universal proofs. They can trigger replanning, a different tool, a request for information, or escalation before they force termination.

A recent preprint analyzed 6,549 public agent repositories and reported 68 manually confirmed cases of effectively unbounded agentic loops across 47 projects. The work is too new to treat as settled, but it provides concrete evidence that framework-induced and nested feedback paths can repeatedly reach costly or state-changing operations unless bounds are explicit (Hou et al., 2026).

Feedback quality shapes the loop

An agent can adapt only to what its interface makes observable. SWE-agent showed that changing the agent–computer interface materially affected software-engineering performance even with the model held fixed (Yang et al., 2024). The broader lesson is not that every agent needs a special shell. It is that observation and action design are part of the reasoning system.

Good feedback is:

typed enough to distinguish outcome classes;
specific enough to support a changed next decision;
bounded enough not to drown the next context;
linked to the action and state version that produced it;
honest about uncertainty and partial effects; and
separated from untrusted instructions embedded in returned content.

Error messages should tell the loop what kind of response is possible. invalid_argument suggests repairing a proposal. permission_denied suggests changing authority or escalating, not retrying. rate_limited may justify waiting. timeout_unknown_outcome demands reconciliation before another write. A plain paragraph saying “something went wrong” forces the model to infer lifecycle semantics from prose.

Common loop shapes

The same lifecycle can support several control structures.

Direct action loop

The model observes current state, proposes one action, receives the result, and chooses again. This is appropriate when each next step depends materially on fresh feedback and actions are individually bounded.

Its main cost is the number of model turns. If several deterministic operations always occur together, execute them as one code-defined action or workflow rather than ask the model to rediscover the sequence.

Plan-informed loop

The system keeps an explicit plan or task list, but each step still uses current observations and may revise the route. The plan is advisory unless particular items are committed into authoritative state.

This improves legibility and can expose missing work, but stale plans can anchor the model after the environment changes. Planning and Task Decomposition will examine when planning earns its cost.

Verify–repair loop

The system produces an artifact, verifies a named condition, and repairs only when evidence identifies a defect. A fixed evaluator–optimizer workflow may use the same shape without being an agent.

The verifier and repairer can both be noisy. The loop needs a retained best-known result, a repair budget, and a rule preventing a weak verifier from replacing a better artifact merely because another round was available.

Bounded child run

A parent run delegates a scoped goal, tools, state projection, budget, and return contract to a child run. The child returns a structured terminal state and evidence, not merely a message.

Nested loops hide multiplication: five parent steps, each allowed five child runs of ten steps, can authorize 250 child steps before retries. Multi-Agent Systems will own coordination topologies; the control-loop requirement is that budgets, cancellation, and failure propagate across the boundary.

Suspended event-driven loop

The run persists state, waits without repeatedly invoking a model, and resumes when an event, time, user response, or approval arrives. Waiting is not “thinking.” A model should not poll itself to simulate time.

Reliable persistence and resumption belong to Production Engineering. Architecturally, the loop needs a first-class waiting state and a deterministic resume trigger.

Design trade-offs

Fine-grained versus coarse actions

Fine-grained actions expose more checkpoints, evidence, and control. They also add model turns, context growth, latency, and opportunities for inconsistent sequencing. Coarse actions reduce orchestration overhead but can hide intermediate effects and make cancellation or recovery harder.

Choose granularity around meaningful decisions and consequences. A database transaction may be one bounded action; an open-ended research investigation should not be one opaque tool call merely to simplify the diagram.

Eager versus staged execution

Executing every valid proposal immediately minimizes latency. Staging allows validation, preview, approval, batching, and cancellation before commitment. The right choice follows action authority, reversibility, and consequence—not the model’s confidence alone.

Full transcript versus state projection

Passing the entire history makes implementation simple and preserves detail, but mixes stale state, untrusted content, failed attempts, and conversational residue. A state projection is more deliberate and efficient but can omit a fact the next decision needs. Context Assembly and Management will own this trade-off; the loop should preserve the source record either way.

Centralized versus nested control

One loop makes budgets, state, and stopping easier to see. Nested loops can isolate expertise or context and run work concurrently, but multiply lifecycle states, hidden retries, and partial failures. Add a child loop only when its boundary has a clearer contract than keeping the decision local.

Frequent versus consequential verification

Verifying after every step catches errors early but can dominate cost and latency. Verifying only at the end allows an early mistake to shape a long trajectory. Concentrate strong checks before irreversible actions, after uncertain external effects, at phase boundaries, and at claimed completion; use cheaper progress signals elsewhere.

Failure modes

The model owns truth

The system treats a generated summary as the current task state. Earlier errors become facts, and later steps cannot distinguish a claim from a committed effect. Keep authoritative state and environmental evidence outside the model’s prose.

Final response equals success

The model stops calling tools, so the harness marks the run complete. The requested record, artifact, or environmental state may still be missing. Treat finish as a proposal and verify the task-specific outcome.

Transcript as state machine

Lifecycle is inferred from the last few messages. Cancellation, partial writes, pending approvals, and exhausted budgets become ambiguous after compaction or restart. Persist explicit run and step states.

Retry without new information

The same action is repeated after the same failure with unchanged arguments, authority, and environment. Cost grows but the probability of success does not materially improve. Require a changed condition, bounded backoff, another strategy, or escalation.

Maximum steps as a completion rule

When the run reaches its limit, it emits the best available answer and reports success. A safety brake has been confused with evidence. Return an honest expired or partial result with unresolved work.

Tool result flattened into prose

Success, rejection, partial effect, and unknown outcome all become strings in the next prompt. The model guesses whether to retry and may duplicate effects. Preserve typed outcome semantics through the loop.

Self-critique as verification

The model restates or revises its answer and the system treats that as independent validation. Use external checks where possible and calibrate model evaluators as fallible instruments.

Hidden nested loops

Tools, agents, evaluators, and handoffs each contain their own retries, but the parent sees one call. Aggregate budgets and expose child status, cancellation, and committed effects.

Uninterruptible action chains

Cancellation is checked only between long sequences of effects. The interface says “stopped” while work continues. Define cancellation points, in-flight semantics, and reconciliation requirements.

Activity mistaken for progress

New searches, longer traces, and more agents create visible motion without reducing unmet success conditions. Track task-relevant state and evidence, not step count as a positive metric.

Fieldbook guidance

Before implementation, write a control-loop contract for each model-directed subsystem:

Contract field	Question to answer
Goal	What bounded outcome is this run pursuing, and who requested it?
Success evidence	Which predicates, checks, observations, or approvals justify `succeeded`?
Authoritative state	Which system records are treated as current truth, and who may update them?
Observations	What can each decision see, at which version, and with what provenance and trust status?
Proposed transitions	Which actions and lifecycle changes may the model request?
Transition gates	Which schemas, preconditions, permissions, approvals, and budgets are enforced by code?
Outcome semantics	How are success, rejection, partial effect, unknown effect, and invalid response represented?
Progress	What material state or evidence change justifies another step?
Run states	How are running, waiting, blocked, succeeded, failed, cancelled, expired, and escalated distinguished?
Limits	What bounds steps, time, tokens, cost, retries, recursion, child runs, and side effects?
Interruption	Who can pause or cancel, when is that observed, and what happens to in-flight actions?
Escalation	What evidence and ownership transfer when the current envelope cannot proceed?
Record	Which configuration, proposal, checks, action result, state version, and verifier result are retained?

A minimal loop can then be reviewed in this form:

load authoritative run state
apply external cancel, deadline, and resume events

while state permits execution:
    observation = assemble_bounded_observation(state)
    proposal = choose_transition(observation)
    decision = validate(proposal, state, authority, limits)

    if decision requires waiting or escalation:
        persist explicit state and transfer ownership
        stop this execution

    result = execute(decision.accepted_action)
    state = record_and_update(state, decision, result)
    verdict = verify_progress_and_completion(state)
    state = apply_legal_transition(state, verdict)

return terminal or suspended state with evidence

Real systems need concurrency control, durable persistence, security, and recovery beyond this sketch. Its purpose is to expose ownership: the model proposes; the harness validates, records, and transitions; the environment supplies evidence; the verifier judges a named condition.

Checklist

References and how they are used

IBM Research, “An architectural approach to autonomic computing” and IBM report H-0219 — longstanding systems precedent for separating monitoring, decision functions, shared state, sensors, and effectors; used as an analogy, not as proof that LLM agents are stable controllers.
Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models” — primary research supporting interleaved actions and environmental feedback in interactive language-model tasks.
Anthropic, “Building effective agents” — Anthropic engineering article supporting tool use from environmental feedback, explicit stopping, and simple composable structures.
OpenAI, “A practical guide to building agents” — OpenAI product-adjacent agent-building article used for the practical concept of a run loop and exit conditions; product-specific examples are not treated as universal architecture.
Yang et al., “SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering” — primary evidence that environment-interface design can materially affect an agent’s behavior and task performance.
Yao et al., “τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains” — primary research supporting environmental outcome checks and repeated-trial reliability rather than dialogue-only evaluation.
Huang et al., “Large Language Models Cannot Self-Correct Reasoning Yet” and Gou et al., “CRITIC” — contrasting evidence used to separate ungrounded self-critique from correction informed by external feedback.
Hou et al., “When Agents Do Not Stop: Uncovering Infinite Agentic Loops in LLM Agents” — recent preprint providing emerging evidence about effectively unbounded feedback paths; its findings are explicitly treated as provisional.