Labloop
A research harness for coding agents
Labloopprint, in line
- Kind
- Artifacts
- Status
- Released
- Reading
- 1 min
An experimental research harness for running coding agents through a sequence of experiments, evaluating their results, and keeping a record of what happened.
Labloop's plate, shown with its construction. The thin thread represents the loop, the two heavy terminals represent the gate, and the slot between them is the only way out. In the print, the plate is the clear area. Drawn by Plate.
Coding agents can write and run experiments, but this alone is not enough to conduct a research campaign: someone still needs to decide which experiments to try next, enforce resource limits, and ensure that the results are evaluated consistently. Labloop provides the tooling around the agent to handle these tasks.
The process starts with a human-defined configuration: the task, a baseline, an evaluator, the available budgets, and a success threshold. Labloop then runs a loop in which a coding agent creates or modifies an experimental candidate; its output is evaluated, the result is recorded, and the next candidate is selected. Claude Code currently runs the individual agent sessions, while Labloop manages the search around them.
Crucially, a good result during the search is not sufficient to support a research claim. Since these scores are used to decide which candidates to pursue, they are optimistic by construction. At the end of the campaign, the selected candidate is therefore evaluated once on a separate final split, and the result is compared against the threshold fixed before the search began.
Failed and invalid experiments remain in the record, together with their configurations and outcomes. The intention is to preserve what actually happened, rather than only the attempts that worked.
Labloop is still experimental. The README on GitHub describes the setup and operation in full.