Better Harness: Evidence-based work evaluation, paths to improvement

1. Guidelines, an evaluation model, and execution tools released together

Better Harness analyzes and improves how coding agents work. According to Qoder, available work records and report formats varied by the program hosting the agent. At release, the Qoder integration was the most complete reference implementation: example code demonstrating the features.

The release has three layers: issue-specific engineering guidelines, the Agent Work Loop evaluation model, and executable code for analyzing real projects. The repository includes not just guidelines but an implementation for gathering evidence and repeating assessments. It is distributed under the MIT license, which permits modification and redistribution.

According to the company, 100,000 people tried the feature in its first three days in Qoder Desktop, before the public release. The release extends an evaluation framework first used in the desktop product to other coding agents.

2. Evaluating use in tasks, not just the presence of settings

The evaluation unit is a task with one goal and completion criteria. A session is an execution unit containing a continuous work record and evidence of the process. Rather than merely checking whether tests exist, the tool examines whether the tests the agent ran actually checked the changes and key risks.

The model has five dimensions and fifteen checks: task understanding, execution control, change verification, reliable delivery, and accumulated learning. It records whether components exist, are linked to the task path, have actual execution results, or are supported by later comparisons. It also distinguishes what could not be verified from what was actually absent.

For the first four dimensions of the current task, the model document sets score ceilings based on evidence strength: how complete the evaluation evidence is. Stronger evidence permits higher scores, but a dimension cannot exceed 59 if a required check remains unresolved or blocked. For accumulated learning, the agent separately selects an integer in the range 35~100.

Table: Score ceilings for the first four dimensions by evidence level

Conditions: Based on the repository's Agent Work Loop model. Applies to the first four dimensions; accumulated learning follows separate rules.

Unit: points

3. Reports identify problems to fix and ways to verify fixes

The analysis examines what the agent did in the session, how the project supports execution, diagnosis, and verification, and evidence of agent asset configuration and use. Agent assets are the settings and tools the agent uses. It then combines these findings against the evaluation model. Alongside scores, reports offer prioritized improvements, each with impact, evidence, scope of changes, and verification steps.

The repository includes a saved example report from analyzing the Better Harness project itself with Codex. The analysis covered one observed session without editing or delivery. Its scores across the five dimensions are shown below.

Table: Dimension scores in the repository's example report

Conditions: One fixed Codex self-analysis example in the repository. Analysis of one observed session, without editing or delivery.

Unit: points

The example's overall task effectiveness score is 58, and its asset status/change progress score is 0. The report notes that delivery procedures are described, but evidence for accepting the current results and a path to restore the installation state are lacking. The format shows which procedures need improvement alongside the scores.

4. Install and run it for your host, the program environment running the tool

Claude Code and Codex were supported at release, with installation methods differing by program. The public documentation gives the following example for registering the repository in Claude Code and installing the plugin, an add-on that provides features.

/plugin marketplace add QoderAI/better-harness
/plugin install better-harness@better-harness

After installation in Claude Code, open a new session in the project to analyze and run /better-harness.

It is built into Qoder Desktop without a separate installation. Users can also select the beta, a pre-release trial feature option, on the Quest screen.

At release, Cursor was supported by loading the source.

Qoder CLI and the JetBrains plugin were also described as available in environments with the desktop app installed.

5. Evidence still requires review and adjustment

The tool's practical benefit lies less in scores than in making judgments reviewable. Each improvement links observed evidence, a minimal scope of changes, and verification steps. This helps teams fix issues one at a time and check for changes in later tasks of the same kind. However, users must learn host-specific installation and output methods, read the evidence, and verify proposed fixes.

The example report is only one case included in the repository, not evidence of long-term improvement. Qoder also says the model was built around familiar projects and task types, so it may miss issues in other technical environments or team settings.

The public documentation suggests starting with one issue whose evidence and impact are clear. Fix and verify it, then observe a similar task to check whether the change also helped with the next task.

Better Harness is not just about scoring agents, but about connecting evidence observed in their work to the next improvement.