AGENT SKILL

harness-refinement

Run a bounded, evidence-based comparison of alternative agent harness configurations for a difficult or repeatedly failing task. Use when a task's quality or cost could plausibly improve through changes to prompts, guardrails, memory, tool guidance, subagents, skills, scripts, or plugins.

License 许可
MIT
最近更新
2026-11-14
Stars
★ 1
Forks
0

01 前言 Front Matter

name
harness-refinement
description
Run a bounded, evidence-based comparison of alternative agent harness configurations for a difficult or repeatedly failing task. Use when a task's quality or cost could plausibly improve through changes to prompts, guardrails, memory, tool guidance, subagents, skills, scripts, or plugins.

03 SKILL.md 正文

<!-- Generated by Paved plugin paved@2.29.0 from core.testing.harness-refinement@0.2.0; edit Paved Core instead. -->

Harness refinement

When to use

Use for complex tasks with uncertain execution strategy or after a verified failed attempt. Skip routine work where an experiment costs more than the task.

Required context

Read the approved task plan, repository verification profile, and relevant harness instructions. Record the exact base revision and fixed acceptance criteria before comparing variants.

Preconditions

The task must have a bounded outcome, repeatable checks, and an approved implementation plan. If the host cannot create isolated worktrees or report usage, record that limitation and do not claim a measured efficiency gain.

Procedure

  1. Define one baseline and at most two candidates. Each candidate changes one or a small, named set of harness elements and states a rationale.
  2. Put alternatives in one experiment.group in the approved delegation graph. Give each the same dependencies, read scope, write scope, acceptance criteria, and base revision. Candidates are alternatives, not cumulative tasks.
  3. Approve the exact graph. Start every experiment task in an isolated worktree; never run alternatives against a shared mutable workspace.
  4. Run the same verification and independent review process for every alternative. Record criterion-level results, changed paths, host-reported model/effort, usage, and duration. Treat usage as host-reported evidence.
  5. Record the paired measurements in the local harness-evaluation format, including shared checks, repeated runs, review outcomes and host measurements. When diff statistics are available, record changed lines for each run and compare them only across paired runs. Use acceptance criteria to check that required risky behavior has tests; code size is never a substitute for correctness or safety. Use the configured analyzer to validate pairing and get a deterministic recommendation.
  6. Compare quality first. Reject candidates that fail a required criterion or regress a criterion the baseline passed. Compare tokens, cost, and duration only when available on both sides.
  7. Keep the baseline when results tie, evidence is incomplete, or efficiency gains are not measured. Adopt a candidate only through the normal human review and change workflow.
  8. Retain the experiment record and explain limits. Do not turn a single task result into a persistent rule. Repeated findings may be submitted as Gardener evidence for human review.

Tools

Use core.repository.diff to inspect changed files, host-managed Claude Code or Codex sessions, and Paved's delegation command. Use the repository's own verification commands for checks.

Rules

Follow the approved-plan, verification, and evidence requirements in the repository instructions and workflow. For implementation, follow core.development.feature-development. For a regression claim, follow core.testing.regression-testing. Before adopting a candidate, follow core.code-review.change-review.

Failure handling

If isolation, equivalent criteria, or independent review cannot be established, stop the comparison and report it as inconclusive. Never infer improvement from model choice or agent self-report alone.

Verification

Use the repository's configured checks and review procedure for every variant. The experiment comparison supplements those gates; it does not replace them.

Evidence

Record the approved graph and digest, base revision, candidate change identifiers, per-criterion outcomes and evidence references, review outcomes, host-reported usage, and the final human decision.

Completion criteria

  • Every variant has equivalent checks, scopes, and an independent review.
  • The comparison names measured benefits and limitations or is marked inconclusive.
  • No persistent harness policy changes without human review.

Failure modes

Failure Signal Response
Candidates contaminate one another Shared workspace or differing base revisions Mark comparison inconclusive and rerun isolated
Self-reported success lacks evidence Missing check or review references Do not count the result as a pass
Apparent savings lack comparable usage One side has no host measurement Report the metric unavailable

References

The Core documentation describes the harness-refinement concept and paired delegation evaluation protocol.

Paved integration

Use the project-local Paved contract as the source of truth. Discover state with node "<plugin root>/scripts/paved.mjs" status --json, load only relevant .paved/project/ context, and invoke verification with node "<plugin root>/scripts/paved.mjs" verify --json. <plugin root> is the installed Paved plugin directory, two directories above this SKILL.md. Always run the launcher from the repository being worked on, never from the plugin directory. Do not edit generated integration files or bypass Paved lifecycle and ownership checks.

04 同仓库更多 Skills