Skip to content
Agentic Engineering Fieldbook

Model APIs, Tool Calling, and Structured Outputs

Observation date and maintenance

Section titled “Observation date and maintenance”

Observed on 2026-07-23.

Applicable surfaces: OpenAI Responses API and related function-calling, structured-output, conversation-state, and evaluation documentation; Anthropic Messages API, tool-use documentation, and API versioning documentation.

Maintenance state: maintained Field Note. Revisit when a provider changes a major request shape, strict schema behavior, tool-result representation, conversation-state semantics, or API version policy.

Current observation

Section titled “Current observation”

Major model APIs have converged on a recognizable pattern: the application gives the model instructions, context, and a set of declared tools; the model may choose a tool and produce structured arguments; the application executes the operation outside the model; then the result returns to a later model step.

OpenAI’s current API documentation describes function calling and structured outputs as part of the model interface, and its Responses API documentation includes conversation-state mechanisms. Anthropic’s Messages API and tool-use documentation expose a similar broad shape, with explicit versioning controls around API behavior. The terms, request envelopes, role names, result shapes, and strictness guarantees differ. The engineering pattern is shared; the contracts are not interchangeable.

The strongest current implication is modest but important: tool declarations and schemas have become first-class behavior-shaping assets. They should be reviewed, versioned, tested, and observed like code, prompts, policies, and routing rules. Changing a tool description, enum, nested schema, response format, or conversation-state policy can change model behavior even when no application binary changes.

Durable engineering implication

Section titled “Durable engineering implication”

Schema-constrained output improves the handoff between model-mediated behavior and deterministic software. It can reduce parser fragility, make invalid values easier to reject, and allow downstream systems to depend on a narrower shape. It does not prove that the answer is true, authorized, complete, fresh, safe, or appropriate for the user’s goal.

Treat every tool call as a proposed transition. The model may propose an operation and arguments; the execution boundary decides whether the operation is valid, authorized, timely, idempotent, observable, and within budget. A strict schema can say that amount is a number. It cannot decide whether this account may refund that amount under the current policy and evidence.

The same separation applies to structured final outputs. A JSON object can be valid and still be unsupported by evidence. A citation field can be present and still point to the wrong source. A risk classification can match an enum and still be semantically wrong. Use schemas to control shape; use evaluation, validation, source checks, and human review to control meaning and consequence.

What not to generalize

Section titled “What not to generalize”

Do not infer that tool calling is a protocol standard because several providers offer similar concepts. Provider APIs differ in message structure, supported schema subsets, state ownership, streaming events, parallel tool behavior, versioning, and error handling.

Do not infer that provider-managed conversation state is the system’s authoritative state. It may be useful context storage, but the application still owns task state, accepted outcomes, approvals, audit records, and external effects unless a narrower contract says otherwise.

Do not infer that a structured-output feature makes evaluation optional. It moves some failures from parsing time to semantic time. That is progress, but it shifts the question from “can we read it?” to “can we trust it for this decision?”

Operational checklist

Section titled “Operational checklist”
  • Record provider, API version or surface, model identifier, request mode, response format, tool declarations, and schema version in release evidence.
  • Put tool schemas and structured-output schemas under code review and change control.
  • Validate tool arguments outside the model for type, range, tenant, principal, purpose, current state, idempotency, and policy.
  • Test malformed, missing, unknown, stale, duplicated, and over-broad tool arguments with deterministic tests.
  • Keep provider-managed conversation state separate from authoritative application state unless the ownership boundary is explicit.
  • Version prompts and tool descriptions together; descriptions influence model selection and argument formation.
  • Evaluate semantic success on representative tasks even when every output is valid JSON.
  • Capture traces that connect model decision, proposed tool call, validation, execution result, and final accepted outcome.
  • Treat schema strictness, streaming behavior, and tool-call limits as dated dependencies, not universal properties.

References and how they are used

Section titled “References and how they are used”