climb
Keep-or-revert loop on a frozen harness: one hypothesis and one commit per attempt, both arms re-measured every time, kept by a rule written before the first attempt, logged outside the tree, in its own worktree and branch. Use when: 'keep trying changes until this counter drops', 'hill-climb this benchmark', 'iterate on this until the target is met', 'run an optimization loop'. After /performance:snapshot baseline, before /performance:verify. Skip when: the score comes from an eval set for an LLM app (`/claude-api hillclimb`); one before-and-after comparison (`/performance:snapshot`).
- License 许可
- MIT
- 最近更新
- 2026-11-15
- Stars
- ★ 30
- Forks
- 7
01 前言 Front Matter
- metadata
- { "summary": "Keep or revert one measured change at a time until the goal is met", "workflow-stage": "implement" }
- description
- Keep-or-revert loop on a frozen harness: one hypothesis and one commit per attempt, both arms re-measured every time, kept by a rule written before the first attempt, logged outside the tree, in its own worktree and branch. Use when: 'keep trying changes until this counter drops', 'hill-climb this benchmark', 'iterate on this until the target is met', 'run an optimization loop'. After /performance:snapshot baseline, before /performance:verify. Skip when: the score comes from an eval set for an LLM app (`/claude-api hillclimb`); one before-and-after comparison (`/performance:snapshot`).
- argument-hint
- [<goal-slug>]
- user-invocable
- true
- disable-model-invocation
- false
02 资源文件 3
- 参考文档evals/evals.json8.5 KB
- 脚本scripts/climb_log.py5.4 KB
- 脚本scripts/test_climb_log.py13.7 KB
03 SKILL.md 正文
Arguments. [<goal-slug>]. e.g. /performance:climb site-build-template-parses
Purpose
Answers "which of these changes actually moved the number, and how far can it go?"
Measure one change at a time: hypothesis, change, commit, measure both arms, apply the rule, keep or revert, log, until the goal is met. Keep every judgment out of reach of the attempt being judged: the rule is written first, the harness is frozen, and the baseline is re-measured beside each attempt.
Read ${CLAUDE_PLUGIN_ROOT}/reference/harness-integrity.md
before the first attempt.
Inputs, or stop
- A goal from
/performance:goalwith its floor, its Realistic target andmin_attempts. A goal withoutmin_attemptsgoes back to/performance:goal; never pick one here, since goal is human-gated. - A harness
/performance:snapshotfroze: the command, the commit it was frozen at, and its discrimination result. Send the run back to/performance:snapshot baselinewhen no harness is frozen, or when its output carries neither tally. Each harness run must print three numbers: the goal's counter, a tally of failures and a tally of finished work units. - A clean checkout.
git status --porcelainin your checkout prints nothing. Refuse otherwise and name the files: the loop starts from a commit, and uncommitted work is not in it. - Optional: a holdout harness
/performance:snapshotfroze, a second command that prints the same three numbers on inputs the main harness does not use. It needs its ownFrozen:line in the baseline report withdiscriminate: exit 0, worded like item 2; send the run back to/performance:snapshot baselinewhen the line is missing. Absent means off: the run behaves exactly as it does without this item, and nothing below that mentions the holdout applies.
Set up, before the first attempt
-
Memory slice. Use
<memory_dir>/<topic-slug>/climb/<goal-slug>/in your checkout (default<memory_dir>is.work/), by absolute path: the loop runs in another worktree, and the log must outlive it. Done when the directory exists. -
Keep rule. Write
keep-rule.jsonthere with exactly three keys:counter(the goal's counter, by name),direction(lowerorhigher) andgain_floor(the change in the counter that counts as a gain; the host's measured noise for a duration,0for a deterministic count). Show it to the person when one is present. Never edit it during the run: once an attempt has a number, any change to the file would tailor the rule to that number. -
Two worktrees, outside your checkout or in a directory its
.gitignoreignores. Start both at your checkout'sHEAD:- climb's own:
git worktree add -b climb/<goal-slug> <climb-path> <start-sha>. A branch with that name already exists: stop and ask, since it may hold an earlier run. - the baseline arm:
git worktree add --detach <baseline-path> <start-sha>. It always sits at the last kept commit.
Every attempt happens in
<climb-path>. Nothing in this skill edits, commits in or resets your checkout. - climb's own:
-
Starting measurement. Run the harness once in each worktree. The counters agree, the error counts are zero and the work counts match, or the harness is not stable enough to climb on: stop and say which number differed.
-
Holdout, only when item 4 is given. Write
holdout.jsonin the slice, holding the holdout'sFrozen:command and sha; likekeep-rule.jsonit is never edited during the run. It is not a key ofkeep-rule.json, whichdecideaccepts only with its three keys. Then run the holdout once in each worktree, by absolute path, as step 4 does: the counters agree, the error counts are zero and the work counts match, or stop and say which number differed. This is a stability check only; its number is never compared at the end. Run the holdout nowhere else until the run ends.
One attempt
- Read the log:
python3 "${CLAUDE_PLUGIN_ROOT}/skills/climb/scripts/climb_log.py" show --log <slice>/log.tsv. Do not retry a mechanism already reverted unless the new attempt says what differs. - State one hypothesis that names a mechanism: what the code does now, why that costs the counter, and what the change removes. "Templates are parsed once per page; parsing once per build removes N-1 parses" qualifies. "The renderer does too much" does not: it names no code path and predicts no number.
- Make one change in
<climb-path>that tests that hypothesis and nothing else. Never edit the harness files, the holdout harness included. Never read holdout output while forming a hypothesis; holdout numbers never go intolog.tsv, soshowcarries none. - Commit it on
climb/<goal-slug>, staging the changed files by name. - Run the project's tests in
<climb-path>. Recordpassorfail. - Measure both arms in one sitting. Run the snapshot harness
ab.shwith--arunning the harness in<baseline-path>and--brunning it in<climb-path>, at the goal's iterations and percentiles.ab.shdiscards each arm's output, so also run the harness once in each worktree and read its counter, error count and work count. Never compare against a number stored from an earlier attempt; the baseline arm is re-measured every time. Never run the holdout harness here. - Decide:
python3 "${CLAUDE_PLUGIN_ROOT}/skills/climb/scripts/climb_log.py" decide --rule <slice>/keep-rule.json --before <baseline counter> --after <attempt counter>. Settle the verdict in this order, stopping at the first line that applies:- Tests failed, the error count rose, or the work count differs from the baseline's:
reverted, whateverdecideprints. Finishing fewer work units is not a speedup. decideprintskept:kept.decideprintsreverted, the counter is no worse than the baseline's, andgit diff --numstat <last kept sha> <attempt-sha>shows lines removed and none added:kept, notedsimplification. Deleting code without losing ground on the counter is a result in its own right.- Otherwise:
reverted.
- Tests failed, the error count rose, or the work count differs from the baseline's:
- Keep or revert.
- Kept: move the baseline arm to the attempt,
git -C <baseline-path> checkout --detach <attempt-sha>. - Reverted: in
<climb-path>,git reset --keep <last kept sha>, which moves only climb's own branch. If it refuses, stop and report the files; never use another reset form to force it.
- Kept: move the baseline arm to the attempt,
- Log the row:
python3 "${CLAUDE_PLUGIN_ROOT}/skills/climb/scripts/climb_log.py" append --log <slice>/log.tsv. Every row passes all nine flags. The outcome goes in--verdict(keptorreverted) and--tests(passorfail); the measured numbers in--before,--afterand--delta; the text in--id,--hypothesis,--changeand--note. When the verdict isreverted, put the dropped SHA in--note;git showon that SHA still displays the attempt.
- Pointer: for what
--keeprefuses and what it moves, see git-reset--keep.
When the run ends
| Logged attempts | Last kept value | Budget | Untried mechanism | Action |
|---|---|---|---|---|
| any | any | spent | any | finish; report the target as not met when it is short |
at or above min_attempts |
meets the Realistic target | left or not set | any | finish |
| any | short of the target, or min_attempts not reached |
left or not set | at least one | next attempt |
| any | short of the target, or min_attempts not reached |
left or not set | none | finish; report Next idea: none left and which condition was not reached |
A budget is optional. Without one, the last row is what ends a run that never reaches its target.
min_attempts is a floor on evidence: the attempts after the first kept one show whether the
counter has more room. The goal, the keep rule and the budget stay as written until the run ends.
Choosing the next hypothesis
-
Smaller first. Between two hypotheses expected to move the counter by about the same amount, attempt the one with the smaller change first; the number never justifies extra code on its own.
-
From the log. Group the logged rows by the mechanism each one named. Prefer a mechanism no row has tried yet over a variant of a reverted one.
-
After three reverts in a row, run
/performance:targetagainst<climb-path>when it is among the available skills, and take the next hypothesis from its ranking; without it, profile the harness run once and pick the largest cost no row has touched. -
Report what each gain cost. Each kept line in the report carries its
git diff --stattotals, so the reviewer can weigh a small gain against a large change.
Unattended runs
The skill starts no /goal and sets no loop. For a run nobody watches, draft its stop condition
first with /planning:draft-goal-condition when that skill is among the available skills,
naming the goal's target, min_attempts and the budget; without it, write the condition from
those same three items. The person starts the run with that condition.
Finish
- Holdout, only when item 4 is given. Both worktrees sit at the last kept sha. When it equals
the start sha, skip the measurement and report
Holdout: not run (nothing kept). Otherwise rungit -C <baseline-path> checkout --detach <start-sha>, then run the holdout once in each worktree, by absolute path, and read its counter, error count and work count; noab.shrun is needed. Calldecide --rule <slice>/keep-rule.json --before <holdout counter on the start sha> --after <holdout counter on the last kept sha>:keptreportsheld,revertedreportsdid not hold. As in step 7.1, a rose error count or a differing work count reportsdid not holdwhateverdecideprints. A run whose kept attempts are all simplifications (step 7.3) reports whateverdecideprints; adid not holdthere says only that the holdout showed no gain. The holdout informs and never reverts: a gain that did not hold is still the climb's result. Do this before step 1 removes the worktrees. - Remove both worktrees with
git worktree remove <path>. The branchclimb/<goal-slug>stays and holds the kept commits in commit order. Whengit worktree removerefuses, leave that worktree in place, name its path and the filesgit status --porcelainlists in the report, and let the person clean it. Never pass--force. - Report:
Branch: climb/<goal-slug>
Log: <absolute path to log.tsv>
Counter: <counter>, <direction>; Realistic target <Realistic target>
Moved: <start value> -> <last kept value> (<percent change>)
Evidence: min_attempts <n>; logged <n>: kept <k>, reverted <r>
Holdout: <held|did not hold>; <holdout counter on start sha> -> <holdout counter on last kept sha>, or "not run (nothing kept)" (only when a holdout was given; with errors or work counts that differ, name them)
Next idea: <the untried hypothesis most likely to move the counter, or "none left">
Changes, oldest first, one line each:
<sha> <change> (<files> files, +<added> -<removed>)
- Pointer: for which worktrees
removeaccepts, see git-worktreeremove. Ignored-only output was also checked by hand: removal succeeds.
Boundary
-
Does not set the goal or the rule's counter.
/performance:goaldoes, with a person. -
Does not freeze the harness.
/performance:snapshotdoes. -
Does not decide the result is real.
/performance:verifyre-derives it in a fresh context. -
Does not climb an eval score. For an LLM app scored against an eval set, the person types
/claude-api hillclimb; this skill never starts it. -
Pointer: for the
/claude-apisubcommands, see https://code.claude.com/docs/en/skills#work-on-claude-api-projects. As of: 2026-10-04. Recheck trigger: that section drops or renameshillclimb.
Next
/performance:verify on the branch climb/<goal-slug>, with the log path.
Gotchas
- The rule file judges every gain. If an attempt looks good and
decidesaysreverted, the attempt is reverted unless it is a pure deletion (step 7, line 3). Change the goal through/performance:goalbefore the next run, never during this one. - A counter that stops moving can mean the harness stopped running the code. Check the work count before concluding a mechanism had no effect.
- The baseline worktree is detached on purpose. It follows kept commits only; a branch there
would collide with
climb/<goal-slug>. - Run the harness, and the holdout harness, by absolute path in both worktrees. A relative path run from the wrong directory measures the same tree twice.
04 同仓库更多 Skills
track
Backlog reads through the bound tracker: search open and closed items, view one item with its comments and state history. Use when another skill searches or reads tracker items.
reviewer-media
Reviewer media for one pull request: decides whether it needs a narrated walkthrough, builds the storyboard from /review:explain-change's digest, has /explainer-video:produce render it and hands the manifest to /source-control:pull-request media. Use when asked for 'reviewer media', 'a walkthrough video for this PR', 'a produced video of this change', or when pull-request ready or monitor hands a PR over.
route
Model and effort per agent role (orchestrator, worker, verifier, retrieval) for a multi-agent run, resolved through the bundled defaults and the user, team and personal override config layers into fenced JSON with single-agent and fan-out variants and the layer behind each value. Use when asked 'which model for the subagents', 'route the roles' or 'model routing', or before a skill passes a role's model on a spawn.
route
Model and effort per agent role (orchestrator, worker, verifier, retrieval) for a multi-agent run, resolved through the bundled defaults and the user, team and personal override config layers into fenced JSON with single-agent and fan-out variants and the layer behind each value. Use when asked 'which model for the subagents', 'route the roles' or 'model routing', or before a skill passes a role's model on a spawn.
principles
Module-design vocabulary for architecture findings, and the checks built on it. Use when: 'what is a deep module', 'what is a seam', 'deep vs shallow module', 'what is the deletion test', 'leverage and locality', 'module design vocabulary', 'which word do we use for X in architecture findings'. Scanning a codebase for friction is /architecture:improve.
write
Structured five-field bug report (title, steps to reproduce, expected vs actual, justified severity, suggested fix location) from an informal description, read-only. Use when the user names a defect they observed, or asks for one written up ('report a bug', 'file a bug', 'write this up as a bug').