Ju Lin's AI Weblog

What Is an Agent Harness?

An oversimplified equation: agent harness = ai agent − model.

Overview

AI Agent minus Model equals Harness.

An agent harness is everything in an AI agent besides the model — everything outside the LLM inference call. It’s a piece of software that provides an environment for the LLM to observe and take actions.

  1. The agent harness has a system prompt that defines the LLM’s basic behavior and controls how it puts out tokens.
  2. The agent harness includes a set of tools that the LLM can call.
  3. The agent harness interacts with the LLM to get tokens back.
  4. The agent harness interleaves LLM outputs and tool calls, typically called the agent loop.

The system prompt

Sequence diagram: the harness sends the system prompt, then the user message, and the LLM sends tokens back.

Before the user’s message arrives, the harness sends the model a fixed set of instructions: the system prompt. It sets the model’s baseline behavior and constrains how it generates tokens.

Tools

An LLM connected to four labeled tool boxes: read file, run command, search web, send email.

The harness also defines a set of tools the model can invoke: named actions, each with a description of what it does and how to call it. The model doesn’t execute these actions itself — it only requests them.

The model call

The harness sends a prompt to the LLM and receives tokens back.

The harness communicates with the model through a single interface: it sends a prompt and gets tokens back. That call is the only boundary between the harness and the model.

The agent loop

LLM sends tokens to the harness, the harness runs a tool and sends the result back, or stops with an answer.

The harness inspects the returned tokens for a tool call. If one is present, it executes the tool, appends the result to the context, and calls the model again. If not, it returns the output as the final answer. This send-inspect-act cycle is the agent loop.

Further reading