Production Engineering

This part focuses on the operational disciplines required to make systems with variable model behavior measurable, reliable, secure, governable, and economically viable. It assumes the system mechanisms defined in Architecture and asks whether they continue to work under real workloads, change, and failure.

Chapters

• Evaluation Systems and Quality Gates — operationalizes the evaluation principles from Foundations through versioned assets, maintained suites, repeated trials, uncertainty, validity review, and explicit decision gates.
• Observability and Incident Response — connects task-level evidence, correlated telemetry, outcome-centered detection, prepared containment, and post-incident learning.
• Reliability, Recovery, and Durable Execution — covers failure classification, safe retries, idempotency, unknown outcomes, checkpoints, resumption, cancellation, compensation, and dependency recovery.
• Security, Privacy, and Guardrails — covers whole-system threat modeling, prompt injection, least privilege, credentials, egress, sandboxing, output handling, privacy, layered guardrails, supply-chain risk, and adversarial testing.
• Governance, Risk, and Accountability — covers system inventory, risk classification and acceptance, decision rights, policy and obligation mapping, provider governance, assurance, affected-party challenge, reassessment, and retirement.
• Cost, Latency, and Capacity — connects cost per accepted outcome, end-to-end and tail latency, workload budgets, queueing, capacity headroom, admission, backpressure, overload, caching, asynchronous work, and realistic capacity tests.
• Testing, Change, and Release — covers deterministic tests, integration tests, versioning of models, prompts, tools, and skills, staged rollout, compatibility, and rollback. It verifies change safety rather than redefining product-quality evaluation.