Extension points

piloop has four extension points. Two need only configuration: a harness and an MCP server. A runtime needs code. The evaluate command is your own program.

Runtimes

A runtime is where an agent process runs. The daemon depends only on the Runtime protocol in piloop/runtime/__init__.py:

Method	What it does
`start(spec)`	Starts the agent from a `WorkerSpec` (the argv, the first prompt, the working directory) and returns a `Handle`.
`wait(handle, timeout)`	Blocks until the agent ends. After `timeout` seconds, kills it and returns `timed_out`.
`status(handle)`	Returns the current state without waiting.
`session_id(handle)`	Returns the harness’s session id, when the runtime can learn it.
`jobs(handle)`	Lists the jobs that the agent started and that still run.
`prompt(handle, text)`	Sends a message to the running agent, or raises `NotSteerable`.
`read(handle, lines)`	Returns the recent output as plain text.
`kill(handle)`	Stops the agent and what it left running.

piloop has two runtimes: subprocess, the default (a background process that cannot receive messages), and herdr (a terminal pane that you can watch and type into). To add one, implement the protocol, add a member to RuntimeKind in piloop/config.py and add one entry to RUNTIMES in piloop/runtime/factory.py. A project chooses its runtime with the runtime key of piloop.toml.

Harnesses

A harness is a coding agent CLI. A [harness.<name>] table in piloop.toml says how to start it: the executable, the argv for an interactive session (interactive, for Herdr) and for a background run (headless), and the startup dialogs to accept. piloop fills in placeholders such as {model}, {session_id}, {cwd} and {prompt}.

harnesses = ["claude", "mycli"]
model = { claude = "claude-opus-5-5", mycli = "my-model" }

[harness.mycli]
executable = "mycli"
headless = ["--model", "{model}", "{prompt}"]

The built-in harnesses pi, claude and codex are definitions of the same kind, so a project can override any key of them. Two optional parts of a harness have their own registries in the code: the usage reader that reads the tokens from the harness’s session files (piloop/usage/), and the mcp renderer that writes the MCP servers in the harness’s own flags (piloop/mcp/). The configuration reference has every key.

MCP servers

An [mcp.<name>] table defines an MCP server once, and [roles.<kind>] mcp gives it to the agents of a role. piloop writes it in the format of each harness, and passes API keys only as environment variables. See MCP servers.

The evaluate command

The evaluate command is the contract between piloop and your project. The daemon runs it in a clean worktree of each commit. The command must:

print one line METRIC <name>=<value> on stdout for each metric in piloop.toml;
exit with status 0, within evaluate_timeout seconds. Otherwise the candidate is a crash.

When the daemon sets PILOOP_UNITS_FILE, the command can also write the result of each unit (each example, record or file) to that path, so that the verdict can use a paired bootstrap. See The evaluate command and per-unit results.

Get started

Concepts

Architecture

Guides

Reference

Extension points

Runtimes

Harnesses

MCP servers

The evaluate command

​Runtimes

​Harnesses

​MCP servers

​The evaluate command

Runtimes

Harnesses

MCP servers

The evaluate command