Harness Engineering for AI Agents

For several years, the default way to improve an AI application was to reach for a better model, a better prompt, or better retrieval. Those choices still matter. But once a system can plan, call tools, modify an environment, wait for external events, and continue over many steps, the foundation model is only one part of the engineering problem.

The surrounding execution system determines what the model sees, which actions it may take, how progress survives interruption, when work is accepted, and how failures are contained. That surrounding system is the agent harness.

What is an agent harness?

An agent harness is the model-external system that turns a request into a controlled execution trajectory. It assembles context, exposes tools, chooses orchestration patterns, preserves state, enforces policy, records evidence, and decides whether the task should continue, pause, recover, or stop.

Swipe to explore →

This distinction matters because many failures blamed on “the model” are actually system failures: irrelevant context, ambiguous tool descriptions, missing state, unsafe permissions, weak stopping conditions, or a verifier that never tested the result.

Why harness engineering has become important

A short assistant interaction can often fit inside one context window and end with a draft for a human to review. An operational agent may run for hours, call remote services, create irreversible side effects, delegate work, and recover after infrastructure failures. The engineering center of gravity therefore shifts from response quality to execution quality.

• Long-running work: progress must survive context limits, process restarts, and human delays.
• Large tool catalogs: the system must expose the right capabilities without flooding the model with schemas.
• Consequential actions: identity, authorization, approvals, and argument validation must sit at the tool boundary.
• Variable behavior: plans, state transitions, outputs, and stopping criteria need explicit structure where repeatability matters.
• Operational accountability: teams need trajectories, costs, policy decisions, and evidence—not only the final answer.

Swipe to explore →

The loop explains how autonomy unfolds, but it does not by itself create reliability. Execution quality is an end-to-end property: context, tools, control flow, state, verification, and recovery can each become the limiting factor.

Swipe to explore →

This is a heuristic, not a literal equation. Its value is architectural: a severe weakness in any layer can dominate the outcome even when the model is excellent.

The eight load-bearing layers

1. Context engineering

Context engineering decides what the model should know at this moment. Good context is selective and structured: the current goal, relevant evidence, recent observations, applicable policy, available actions, and a concise representation of progress. More context is not automatically better; irrelevant history competes with the information that actually drives the next decision.

Swipe to explore →

2. Tool engineering

Tools are the agent’s action surface. Their names, descriptions, schemas, error semantics, and permission boundaries directly shape behavior. Strong tool contracts are narrow, typed, observable, and explicit about side effects. With a large catalog, retrieval or routing can expose only the tools relevant to the current task rather than placing every schema in context.

Swipe to explore →

3. Planning

Planning makes intent inspectable. A structured plan can define steps, dependencies, expected evidence, budgets, and completion criteria before tools execute. Recent preprint evidence suggests that schema-validated planning can improve reproducibility in some model-task combinations, while also showing that constraints can reduce performance or increase latency when poorly matched. Planning therefore needs evaluation, not ceremony.

4. Orchestration

Orchestration determines how work is decomposed and coordinated. Useful patterns include prompt chaining, routing, parallel workers, orchestrator-worker systems, and evaluator-optimizer loops. The simplest pattern that meets the requirement is usually the best starting point. Multi-agent designs are justified when specialization, parallelism, or independent judgment creates measurable lift.

Swipe to explore →

Routing selects one specialist for a request. Parallel orchestration solves a different problem: it assigns genuinely independent subproblems to multiple workers, then reconciles their evidence through an explicit synthesis step.

Swipe to explore →

5. State and memory

Working context, workflow state, persistent memory, artifact state, and execution state are different things. Treating chat history as the database for all five creates fragile systems. Compaction can preserve continuity when recent conversational detail still matters; a context reset is safer when accumulated history has become noisy, provided the next session receives a structured handoff. Long-running work benefits from progress artifacts that record the goal, completed work, current state, changed artifacts, known failures, constraints, and next steps.

Swipe to explore →

In either case, continuity should come from an explicit state artifact rather than the hidden memory of a particular conversation. That artifact becomes the contract between sessions: what was completed, what changed, what remains open, and what the next run must verify before continuing.

6. Durable execution

Memory does not prevent a purchase order, email, or deployment from being performed twice after a crash. Durable agents need familiar distributed-systems mechanisms: checkpoints, persisted workflow state, bounded retries, timeouts, idempotency keys, pause and resume, and compensation for partially completed operations. The runtime increasingly resembles a workflow engine plus a model runtime plus a state store.

7. Recovery engineering

A useful recovery policy distinguishes transient failures from semantic failures. A network timeout may justify a bounded retry. An invalid tool choice may require replanning. Conflicting evidence may require clarification. A policy violation should stop or escalate. Every loop needs budgets for steps, retries, elapsed time, and external cost.

The common recoverable paths are retry and replan, as shown below. They are not universal continuation rules: exhausted budgets, unsafe requests, unclear intent, and policy violations must leave the autonomous loop for escalation or termination.

8. Human control

A mature harness knows when autonomy should end. High financial impact, irreversible actions, low confidence, policy exceptions, missing information, and security signals should trigger an exact interruption: preserve the proposed action and execution state, obtain approval or rejection, then resume the same run. The human should approve a specific action—not a vague continuation.

Verification must be independent from generation

Asking one agent to create an artifact and then casually judge its own work often produces lenient evaluation. A stronger architecture separates creation from acceptance. The verifier can be another model tuned to skeptical criteria, but it can also be unit tests, integration tests, schema validators, simulations, security scanners, business rules, or a human reviewer.

Anthropic’s 2026 long-running application experiments illustrate both the upside and the cost. A planner–generator–evaluator harness produced a much more functional application than a solo run in one comparison, but consumed dramatically more time and money. The lesson is not “always use three agents.” It is to separate creation from judgment when the measured improvement justifies the overhead.

Guardrails belong at the action boundary

An agent can propose an action; the harness decides whether that proposal reaches the environment. Tool allowlists, identity checks, scoped credentials, argument validation, approval rules, rate limits, sandboxes, network controls, and data-loss prevention should be enforced where the side effect occurs.

• Before execution: authenticate the actor, authorize the action, validate arguments, and check whether approval is required.
• During execution: enforce time, cost, network, and resource limits.
• After execution: capture the result, update state, record policy and tool versions, and verify the outcome.
• On failure: classify the error and choose retry, replan, compensate, escalate, or stop.

Observability is the feedback system

A final response cannot explain a long-running agent. Teams need the trajectory: model calls, retrieved evidence, plans, tool selections, arguments, results, handoffs, guardrail decisions, retries, approvals, and state transitions. Traces let engineers locate the failing layer instead of rewriting the prompt by instinct.

Engineer the harness empirically

Swipe to explore →

• Establish a baseline: begin with one capable model, minimal tools, and the simplest loop that can complete the task.
• Study realistic failures: inspect trajectories and classify failures by context, planning, tools, state, verification, recovery, or control.
• Add one intervention: introduce a router, structured plan, external state, verifier, approval gate, or durable workflow for a specific failure mode.
• Run the same evaluation: compare success, quality, cost, latency, retries, and operational risk.
• Perform ablations: remove the component and confirm that performance falls. If it does not, delete the component.
• Repeat when models change: a harness designed around one model’s weaknesses may become unnecessary—or actively harmful—after a model upgrade.

Five useful harness families

Adaptive harnesses are an emerging research direction. September 2026 preprints such as JIT-Agent and HarnessDev treat the harness itself as a configurable or generated artifact. The early results are promising but mixed: generated harnesses can compete in some domains, yet mature human-engineered systems remain stronger in others and improvements do not always transfer between models or tasks. Enterprises should treat this as active research, not a replacement for architecture ownership.

Build or buy?

Frameworks increasingly provide agent loops, tool interfaces, sessions, guardrails, approvals, tracing, orchestration, and durable-execution integrations. Those capabilities can accelerate delivery, but a framework is an implementation choice—not the architecture. The design should first name the responsibilities and boundaries, then decide which platform owns each one.

From prompt engineering to agent systems engineering

Prompts, retrieval, tools, and models do not disappear. They become components inside a larger system that must plan, act, remember, recover, verify, and improve. That changes the unit of evaluation as well: serious comparisons should describe the model + harness + tools + environment, not present a model score as though the surrounding system were irrelevant.

The durable advantage may therefore sit less in access to one model checkpoint and more in the organization’s context, knowledge, tool contracts, workflow state, evaluation data, policies, and feedback loops. The model remains essential. The harness is what makes its intelligence operational.

Primary sources and further reading