[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M720NHDJS6HGFS4HV7Z0XZE5":3,"skill-same:01M720NHDJS6HGFS4HV7Z0XZE5":45},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":18,"linkOnly":19,"markdownContent":20,"markdownBody":21,"frontMatter":22,"refName":23,"commitSha":24,"skillDirectory":25,"forksCount":26,"repositoryHtmlUrl":27,"repositoryHomepageUrl":28,"ownerHtmlUrl":29,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":30,"install":31,"resources":44},"01M720NHDJS6HGFS4HV7Z0XZE5","eval-fix-loop",null,"Take one failing eval case (or one root cause behind several), diagnose it from its recorded runs, classify the miss, record a red, make the smallest allowed fix, prove three greens, and open the pull request. Use when a nightly or local run shows a red that someone must turn green, or when a repair list names a root cause, or when a new certification case, baseline or marked, lands.","gke-labs\u002Fkube-agents","gke-labs","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F197140281?v=4","01M720MEZR8K6WD2WTHNQE346Q","01M720MF0BRA9RGH1QVT4J0M0E",126,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fgke-labs\u002Fkube-agents\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Feval-fix-loop\u002FSKILL.md",[],[],"2026-11-10 18:39:20",false,"---\nname: eval-fix-loop\ndescription: Take one failing eval case (or one root cause behind several), diagnose it from its recorded runs, classify the miss, record a red, make the smallest allowed fix, prove three greens, and open the pull request. Use when a nightly or local run shows a red that someone must turn green, or when a repair list names a root cause, or when a new certification case, baseline or marked, lands.\n---\n\n# Eval fix loop\n\nOne root cause per loop. A classified cause with quoted evidence before any code. Never loosen a check to fit a model's wording; a bar above the product's own rule under the standing fixture state is lowered only on a replay that flips addressed fails and zero passes, with the dropped branch pinned by a unit test. Three greens on the final head. Markers flip in the same pull request.\n\n## First-day addendum (2026-11-02)\n\n1. Before anything, take one row from the batch file the planner names (`~\u002F.claude\u002Fhandoffs\u002Fcert-batch-\u003Cdate>.md`, outside the repository) and write your session name in its claims column. This skill is a repository file, not an installed one: read `.agents\u002Fskills\u002Feval-fix-loop\u002FSKILL.md` with the file reader; the Skill tool does not list it.\n2. Read the store state first: `expected_fail` on `main`, passes and fails per model, `fail` against `not_applicable`. A new case has no store rows and skips this item. The read: `BaselineStore.load(\"gs:\u002F\u002Fkube-agents-evals-bench\u002Fevidence\", only=[case])`, then `store.admission(case, key, bar=AdmissionBar(), mode=\"roster\")`, with `key` from the store's own newest record for the case (`load_versions` gives fleet and verifiers only); cite the lane's current key alone, never two keys side by side (rule 58).\n3. Search open issues and open pull requests three ways: the case name, the failing check's reason string, the mechanism. A fix in flight ends the loop with a page row note. For a certification case, also grep the gap issue number across the suite's task headers (`grep -l '^suite: certification$' bench\u002Ftasks\u002F*\u002Ftask.yaml`): a case already on that issue means this one plants a different signal of the same issue, or ends as a page row note. Read the issue's state as well: a merge-queue keyword closes an issue with no fix on `main` (#4943 on 2026-11-10, through #5136), a declared gap on a closed issue is not declared, and the reopen is an ask. A nightly-red issue on the case (`scripts\u002Feval_dashboard\u002Fnightly_bugs.py` files it and never closes it) gets one comment on the first full green after the fix merged: the class, the fixing pull request, the green build id. It is closed on the human's word. Items 1, 2, the store half of 4 and the store read in 12 are certification-only: a nightly red has no batch row and no store, and skips them.\n4. The red is on the record: cite the store's reading by its word and two failing run directories by name (nothing from inside them), and replay them through the check as it is on `main`.\n5. For a check-only or harness fix the red is the record (build ids, the failing check by name, two run directories) and the green is the replay of every recorded answer with the new check: all addressed fails pass, zero passes turn to fails. Only a harness fix with no recorded answer to replay takes the unit test instead, red against `main` and green ten of ten. Three live greens are for an agent or product change only; a live repetition of a recorded red proves nothing the record does not.\n6. A wording check admits every phrasing of the fact and excludes nothing; a safeguard grades the action. Narrowing is measured on the record.\n7. Pin every wording check through the whole check, never with a regex alone: the helper `check_passes` in `bench\u002Ftests\u002Fconftest.py` once #4530 merges, until then the four-line form (`transcript.set`, then `ReportContainsVerifier(**check).verify(5.0).status`).\n8. Prefer the case folder. A change under `bench\u002Fkube_agents_bench\u002F` must still work when the presubmit grades with the base commit's bench.\n9. A case whose wrong answer may open a pull request the reply never names declares `writes_alone: true` (a key that lands with #4530); it then runs alone, and the lane safeguard charges it any write in its window.\n10. The body keeps every sentence at 25 words or fewer. It says `Part of #\u003Cissue>`, except the pull request that lands a declared gap's check, which says `Closes #\u003Cgap issue>`. It carries `Part of #3978`, the certification epic, no `Co-Authored-By` trailer, and ends with \"Generated with the help of Claude Code.\"\n11. One session, one PR: mind it to the first human lgtm, then hand it to the PR minder with a written handoff. Post `\u002Freview` only if the bot did not review on open.\n12. Step 8 is \"send the numbers\": record them in your own status with `cd ~\u002Fka-overseer && python3 overseer\u002Fovsr.py report --from \u003Csession>` : fails flipped, passes kept, the PR number, the page row note. Message the Overseer only for an ask. The store takes the night at the morning store read.\n13. A roster admission of a case that already holds a record is not a new case: the record stands in for live runs (quote the store's admission line and one `data.json` snapshot's `generated_at`; #4564); read `hack\u002Feval\u002Fblocking-roster.txt` and `presubmit-cases.txt` first, since the one-lane rule costs a presubmit seat when the case holds one; the pins are `scripts\u002Ftest_eval_rosters.py` (its frozen lists) and `hack\u002Feval\u002Fnightly-main-*.txt` (stack-bearing cases only).\n\n## Inputs\n\n- A case id under `bench\u002Ftasks\u002F\u003Cid>\u002F`, or a root cause with its case ids.\n- The run directories that show the failure: a nightly build's `artifacts\u002F` in the local cache, or a local `bench\u002Fresults\u002Frun_*\u002F`.\n- A checkout with the case, its fixture (`bench\u002Ftf\u002Fprebuilt\u002F\u003Cstack>\u002F` or the seeded fleet), and the product code behind it: the procedure (`agents\u002Fplatform\u002Fskills\u002F\u003Cskill>\u002F` SOP markdown), the collector scripts (`agents\u002Fplatform\u002Fskills\u002F\u003Cskill>\u002Fscripts\u002F`), the checks (`bench\u002Fkube_agents_bench\u002Fverifiers.py`).\n- The rules: `.agents\u002Frules\u002Feval_driven_development.md` governs red, green and registration; this skill adds the diagnosis and the refusals.\n\n## Step 1: pick and brief\n\nWrite a one-screen brief before reading anything else in depth:\n\n- Case id, domain, audit stream, the install model and release of the failing runs.\n- For a case that plants a fixture: the fields the lane's host type adds or changes on the planted objects, read with `kubectl -n default create -f - --dry-run=server -o yaml` over the manifest. A safeguard on a field admission writes is wrong, and a condition admission removes cannot be planted there (Autopilot writes `limits.ephemeral-storage` on every container).\n- Pass counts per model from the record (`bench\u002Frun-store\u002Fruns\u002F` if present, else the run directories).\n- The failing checks by name, with their `reason` strings quoted verbatim from `results.json` `verification_report[]`.\n\n## Step 2: diagnose from the runs\n\nRead in this order, and quote what you find with its source path:\n\n1. `results.json` → `verification_report[]`: which checks failed, which passed, and the exact reasons.\n2. `results.json` → `output`: the agent's final report. Does it name the target resource? Does it name the planted object under a different finding? Does it say anything was \"unevaluated\", \"unjudged\", \"skipped\", a \"coverage gap\", or \"not applicable\"?\n3. `results.json` → `trajectory[]`: did the required collector or script run (`tool_called`, `terminal` commands)? What did the collector print? Grep for the target resource, the namespace, and words like `unevaluated`, `skipped`, `error`, `429`, `timeout`.\n4. The ledger or pull request the check read, when the check names one (`ledger_issue_contains` reasons name the issue): what finding ids does it carry? A ledger is rewritten by every later run, so when the live issue has moved on, recover the body the check graded from the issue's edit history (GraphQL `userContentEdits { editedAt diff }` on the issue) rather than reading the current one.\n5. For a fleet-audit case, the collector manifest and the findings document the run wrote in the sandbox, copied out during the run since the next run overwrites them: `kubectl -n kubeagents-system exec platform-agent-shell-0 -c shell -- cat \u002Fopt\u002Fdata\u002Fscratch\u002Fmanifest_\u003Caudit>.json` (per cluster, `candidates` and `checks_not_applicable`) and the same for `findings_\u003Caudit>.json`.\n6. What those two files split, which the ledger alone cannot: the collector never listed the object (collector gap); the collector listed it and the worker dropped it (agent behaviour); the finding is in the document and the ledger body left it out (ledger rendering).\n7. The fixture: was the planted state really there (the safeguard's reason tells you; a `resource_property` pass proves the object existed with that value)?\n8. The product code the case exercises: the collector predicate or the procedure line that should have produced the finding. Find where the slug the check wants would be emitted. If nowhere, say so.\n9. The last green record of the same case on the same lane, when one exists: diff its `trajectory[]` and `output` against the first red's. A part that differs in shape rather than in the agent's choices (a marker count, an entry's tag, an argument written as `\u003Cstring, N chars>`) names the writer of that part, and `git log \u003Cgreen head>..\u003Cred head> -- \u003Cits path>` names the change. #5201's cause showed only this way, and #5179 names the writer it suspects.\n\n## Step 3: classify\n\nPick exactly one class, by the first rule that matches, and write the rule you used:\n\n| Class            | Rule that selects it                                                                                                                                                                                                                                                                                                                                                                                                                                                         | Fix type allowed                                                                                                                                                                                                                                                                                                                                                                                         |\n| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Scenario unheld  | The state the header asserts cannot be produced or held by the stack: a provider rule, a channel's list (#5175, #4222: a REGULAR cluster cannot leave its channel's list)                                                                                                                                                                                                                                                                                                    | Bind the check to the state the fixture holds, or to the run-time state a check reads; pin the other branch in a unit test on recorded inputs; prove by item 5's replay                                                                                                                                                                                                                                  |\n| Fixture defect   | The safeguard or the planted object is absent, or the planted state differs from the scenario (wrong namespace, wrong value, pods not in the state the scenario describes)                                                                                                                                                                                                                                                                                                   | Fix the stack; re-run red                                                                                                                                                                                                                                                                                                                                                                                |\n| Record defect    | Step 2 item 9 shows that the part of the record which differs from the last green was written by the transport, the harness or a product component off the case's path, while the agent followed the rule and the check read the record as written (a worker's calls on the router trace, #5201, the writer #5179 names; an argument shaped on the inject door, #5077, whose fix set the check aside until #2619, the lever this row refuses; a stale in-flight note, #4268) | Fix the writer, never the check, and prove with a record from the fixed head replayed through the check as it is on `main`: a writer in `bench\u002Fkube_agents_bench\u002F` or `hack\u002Fci-eval-pr.sh` by that replay alone, with the one-line exemption under Live validation; a writer in `a2a\u002F` with one live run on the fixed head so replayed; a writer under `agents\u002F` runs the loop (item 5 and Step 6 stand) |\n| Check defect     | The report or ledger shows the right outcome, but the check reads it wrong: a phrase list the report does not use, a type compare (string against boolean), a shortened id, a display-line parse                                                                                                                                                                                                                                                                             | Make the check read state or the ledger id, not words; add the missing phrasing only beside a state check; never delete the check                                                                                                                                                                                                                                                                        |\n| Collector gap    | The collector ran and logged that it skipped, could not evaluate, or does not know the object or shape (for example a ComputeClass priority of a kind it does not parse)                                                                                                                                                                                                                                                                                                     | A predicate or a parser branch in the collector, with a test                                                                                                                                                                                                                                                                                                                                             |\n| Procedure gap    | The collector saw the state, the procedure has no check for the pattern, and no finding was emitted                                                                                                                                                                                                                                                                                                                                                                          | A new check in the procedure with a published source line, and the collector support it needs                                                                                                                                                                                                                                                                                                            |\n| Ledger rendering | The finding is in the run's findings document and absent from the ledger body: the body budget rendered the higher severities in full, listed some without detail, and dropped the rest, so no `Where:` line carries the object (the `audit-findings-all` ids, clipped at 100 characters, are all that remain)                                                                                                                                                               | A renderer change with a test (every finding keeps one line with its check and object); until it lands, the case lands as `expected_fail` naming the renderer issue, since no check edit can read what the body does not carry                                                                                                                                                                           |\n| Agent behaviour  | The procedure has the rule, the collector gave the facts, and the agent did not follow the rule (skipped a step, reworded a verdict, did not open the required tool)                                                                                                                                                                                                                                                                                                         | Move the rule out of prose: a script that prints the verdict line, a check, or a persona rule the persona loads on every run                                                                                                                                                                                                                                                                             |\n| Role set         | The agent lacked a permission or an API the scenario needs, and the report says so                                                                                                                                                                                                                                                                                                                                                                                           | A permission change proposed to the humans; never a Secret grant                                                                                                                                                                                                                                                                                                                                         |\n| Infrastructure   | A 429, a timeout, a lost lease, a missing token, a forge-less install for a ledger check                                                                                                                                                                                                                                                                                                                                                                                     | Not a red; fix the environment and re-run                                                                                                                                                                                                                                                                                                                                                                |\n\nA row's rule names what the reader sees in the record; it may not depend on an earlier row's fix failing. Write the class with three evidence quotes and one sentence on why the next-nearest class does not apply. If two classes apply to different checks of the same case, split: one loop per class.\nThen, for any class but Infrastructure, write the Step 3b card; no code and no run beyond the record before it. An Infrastructure red is fixed and re-run first, with no card.\n\n## Step 3b: the decision card\n\nOne card per loop, written from the record after Step 3. It goes to the human first; code starts on the human's word. Fixed headings, this order, each within the size its text names.\n\n1. **The case in customer terms.** Two sentences: who asks, and what they should get back.\n2. **What the agent does.** One verbatim line per failing shape, its count over the record, and the `file:line` of the code that produced the text.\n3. **Root cause.** One paragraph: the Step 3 class, the evidence line that selects it, and why the next-nearest class does not apply.\n4. **Behaviour or case.** In two sentences, say whether the case's bar is the product's own stated rule, and quote the rule. If the case is loose or unbound, give the one-line tweak: bind the claim to the planted object, or require the thing the customer must see.\n5. **What the customer wants to see.** Before and after, real text, each under 10 lines. The after carries the verdict, the one proving number, and the one thing they did not ask but should know; for a check-only fix the after is the recorded answer and the card says so.\n6. **The fix, one lever per cause.** One paragraph per lever. Prose for a prompt contradiction, code for a code behaviour, a fixture change for a fixture gap; never prose to cover code. Give the size in lines, the resource cost in requests and objects, and where it lives; name a Hermes patch as one, with the note that it moves when Hermes goes. When one root cause takes two levers, both halves ship in one pull request.\n7. **Do less.** The one-line option, and what it would certify falsely.\n8. **Live run needs.** One line each: install, forge, fixture, model, minutes.\n9. **Not covered.** At most three bullets.\n10. **Rec and principle.** One line each.\n\nThe card and the fix it proposes obey these rules:\n\n| Do                                                                           | Do not                                                     |\n| ---------------------------------------------------------------------------- | ---------------------------------------------------------- |\n| Name the model of each red and green; compare no two models (rule 58).       | Report a green on one model as a green on another.         |\n| Bind every check to the planted object.                                      | Grade a sibling finding as the planted one.                |\n| Keep to one lever per cause.                                                 | Use prose to cover a code behaviour.                       |\n| State a limit as a ceiling (\"stays under\").                                  | State a limit as a target (\"aim for\").                     |\n| Show the customer the verdict, the proving number and the one unasked thing. | Patch wording round after round when a state check exists. |\n| Count server-side, with a field or label selector.                           | List everything and filter locally.                        |\n| Put numbers in a table and link every pull request and issue.                | Add a constant or a mechanism for its own sake.            |\n| Keep every claim on the record, with its run directory or line.              | Widen the fix mid-loop without a new card.                 |\n| Stop after the card and wait for the word.                                   | Take a review clear as push authority.                     |\n| Let the review check's green or the third round assign the human reviewer.   | Request a human reviewer by hand.                          |\n\n## Step 4: red\n\nRun the case once, unchanged, on an install the check can be graded on (a ledger check needs a forge; a plain-report check does not). Keep the run directory; the pull request cites it by name and the failing reason stays in it. Use the lease tool. For a Record defect, a Check defect or a Scenario unheld the red is already on the record (the Step 1 brief: build ids, reason, two run directories); a live repetition proves nothing the record does not and spends a lease.\n\n## Step 5: fix\n\nMake the smallest change the class allows, on a branch from fresh `upstream\u002Fmain`:\n\n- A check gains a state or ledger read; a phrase list grows only beside one; a type compare becomes a typed compare.\n- A collector gains the predicate or parser branch, with a unit test on the shape that was skipped.\n- A procedure gains the check and its `source:` line.\n- A behaviour rule moves into a script, a check, or the persona file, and the prose line that failed is removed or shortened.\n\nRefuse: a fixture name or a test id in product code; a phrase list edited to match one model's report; two root causes in one branch; a product change that makes the eval pass without changing what the product does for a customer.\n\n## Step 6: green\n\nOn the final head: run the case three times; run every sibling case of the same root cause once. All three must pass on the deterministic checks. The one exception is a Record defect whose writer is in `bench\u002Fkube_agents_bench\u002F` or `hack\u002Fci-eval-pr.sh`: its green is the replay of the fixed head's record through the check on `main`, with the exemption line under Live validation; a writer in `a2a\u002F` adds one live run on the fixed head so replayed. A judged score moving is not a pass.\n\n## Step 7: ship\n\nBefore the push, run the whole set the CI shards run, not the touched files: `make test-python` and `make test-bench`; a change to a shared writer (a hand-off's compose, a finding line, a collector's object) reaches every fixture derived from it. Derived fixtures regenerate with their own writer, never by hand: the planted raw reports under `bench\u002Ftf\u002Fprebuilt\u002Fbootstrap-ranking\u002F` by writing `bootstrap_handoff.compose(state, timed_out=False, now=PLANTED_AT)` over their `*.cards.json` to the `.txt` (`bench\u002Ftests\u002Ftest_bootstrap_ranking.py` pins the equality), the shared chat cases with `agents\u002Fchat\u002Fscripts\u002Ftest_inventory_card.py --write-cases` then prettier. Open the pull request from the template: the red (install, commit, failing check by name, the run's build id or directory), the three greens (run build ids or directories), the class and where its evidence is (run directory, `file:line`), the markers flipped (`expected_fail: true` removed for the cases the fix proves), the registration line, and the self-review from `\u002Fpr-preflight`. A declared-gap case names its gap issue and never `Closes` it; a reviewer is not named by hand before the review check's green assigns one, and `\u002Frequest-review` is for a disputed finding or a missing review only.\n\n## Step 8: record\n\nAppend the runs to the run store and move the case's row in the suite status file from red to fixing, with the pull request number.\n\n## A new certification case\n\nA new case is not a repair: it has no class and no fix. Its evidence follows `.agents\u002Frules\u002Feval_driven_development.md` in two shapes. A baseline (the product check exists and names the planted object) has no red; the record is three passes on the install's release build, run directories in the body. A declared gap (no check reads the planted signal, a procedure or collector gap you will not fix) lands with one red for the declared reason, `expected_fail: true`, and the gap issue named in its header and in the `#` reason lines above its roster line (\"When the fix is not yours\"); no green exists until the gap's check lands, and the pull request that adds the check flips the marker and closes the issue. A certification case carries `suite: certification` in its `task.yaml`, and its roster line with those reason lines goes in the certification roster (`hack\u002Feval\u002Fcertification-cases.txt` while that file exists; `scripts\u002Fvalidate_bench_cases.py` refuses the suite key without it), never the nightly file, so the rule's step 4 file does not apply. The card replaces Step 3b's headings and Step 7's evidence list (trap 7 is that list) with: the product check or the gap issue; the fixture (objects, labels, destroy); the objective under `scope: finding`; the safeguards, each read whole; the risk to the host cluster; the size in lines; the evidence shape. Before the card: verify every slug the sketch names against the collector (`git grep` the slug constant), and grep `bench\u002Ftf\u002Ffleet\u002Ffixtures.json` and the fleet README for the condition, since a plant that exists is read, not built. A new case skips addendum items 2, 4 and 5 (store rows and replay), and its card adds what it plants against the customer failure, the do-less line and the skill shortcomings. The fixture rules live in `bench\u002FREADME.md` (\"Certification suite\", rules 1 to 7); the traps below point at them and add what the loop itself met on a certification case:\n\n1. **Fixture readability.** Rule 4 of the README: the plant is something the agent's role can read, never a Secret; a webhook configuration's `caBundle` stood in for a TLS Secret (`security-expiring-node-bootstrap-and-webhook-certs`, #4960).\n2. **Host mode and admission.** Rule 5 of the README: the lane's hosts are Autopilot, and the Step 1 dry-run says what admission writes; the ComputeClass case was parked under it (#4943, planting issue #5143). A plant that needs a Standard cluster lives in the seeded fleet stack (`bench\u002Ftf\u002Ffleet`) under a `fixtures:` role; its cases park in `FIXTURE_NOT_READY` until the reconcile report shows every pool project rolled (`docs\u002Fci-pool-projects.md`, section 6.2). When no lease is open at card time, the dry run moves to the red step under the lease, the card marks its safeguard values pending the dry-run output, and a sibling case's recorded dry run on the same host type is the placeholder.\n3. **The plant shell.** Rule 6 of the README: the shared module for a namespace-scoped plant, its guards copied for a cluster-scoped one (#4939, #4979).\n4. **Objectives.** For a ledger case, one `ledger_issue_contains` with `scope: finding`, `finding_check` the accepted slugs and the planted object as the phrase (both objects of a pair under `any_of_phrases`). A body phrase for the object beside a slug among the ids is unpaired: a run can file them on different findings and pass (#4978; `bench\u002Ftests\u002Ftest_webhook_ca_bundle_case_binding.py` pins one case against it). A chat case has no ledger: its objective is a `report_contains` on the gate-rendered line. The gap issue names the slugs and says a fixing pull request that picks another edits `finding_check` in the same diff, and it states the `Where:` object a flip must name; when the natural object is a node, whose name GKE assigns, pair the workload with the pool name, which the node name carries (#5425). When the fixture rules keep part of the customer defect off the planted object (a check that reads a namespace list the fixture cannot use, such as a GKE-managed namespace; a fail-closed webhook whose selector must exclude what the customer's webhook fails to exclude, rule 1 of the README), the case header and the gap issue state both flip conditions: the shape of check that flips the case, and the shape that reads only the kept-off part and does not (`security-webhook-blocks-config-management-system`, #5338, #5335: backend reachability flips it, the selector exemption alone does not).\n5. **Safeguards.** Rule 7 of the README: one per tempted property, read whole, values from the dry run; never a replica count on an autoscaled Deployment (#4949).\n6. **The run.** One run on a forge install: a ledger check cannot fail for the gap's reason without a forge. Pre-open the agent API on a port of your own with the `kubectl exec` relay `bench\u002FREADME.md` names under Sandboxed installs (`scripts\u002Fhermes-dashboard-tunnel.py --container agent-api-auth --remote-port 8643 --local-port \u003Cport>`; a plain `kubectl port-forward` binds but cannot reach a sandboxed pod, and every request dies with `RemoteDisconnected`) and set `AGENT_LOCAL_PORT` to it (another session's relay on the harness's default port sends the run to another install and it reads 401; a non-default port with nothing behind it is forwarded by the harness to the agent Service, which the sandboxed gateway refuses). A red counts only when the ledger issue's stamp falls inside the run's window; a report with no issue URL is a fail on the product's record (#5076): count it among the runs the body cites, each by run id and outcome word, and re-run to add runs, never to replace one. A plant that crash-loops spends the install's event-watcher alert quota (five Warnings per UTC day): read the `alert_quota` row at lease start, say the counts in the body, and route the reset at release as an ask when your session cannot write the gateway's database.\n7. **The record.** The body names the case as new, the run id, the failing check by name (its reason stays in the run directory, `.agents\u002Frules\u002Fpublic_surfaces.md`), the install's release and mode without its name, the plant as captured (a `kubectl get pod` line), the destroy, the admission dry-run's output, and the marker's issue; when review changes the objective, the maintainers may ask three repetitions on the new head before the push.\n8. **A plant outside the host cluster.** A class-B plant (a per-run cluster, a cluster setting, or project-level network objects) has no kubectl guard, so Terraform's own destroy replaces the module's guards. Where the run's kubeconfig points decides the safeguards: a plant that is itself a cluster puts that cluster in the stack's `cluster_name` and `cluster_location` outputs, the run's kubeconfig moves to it for the run's length, and `resource_property` then reads the plant; project-level network objects leave the outputs on the host cluster through `TF_VAR_host_cluster_name` and `_location` (`prebuilt\u002Fsubnet-range-exhaustion`), nothing in the cluster is the plant, and the worker-commands read-only safeguard on the write that would undo the plant is the catastrophic one. A plant of a setting the API reads back proves itself with a data source whose `postcondition` fails the plan when the planted value is absent; a baseline's objective proves a network plant, and only a marked case needs the proof elsewhere. The objective's phrase is the planted object's name prefix, since a per-run name carries a hash of `cluster_name`. A leftover is swept by label or description, never by the host teardown: a labelled cluster by `reap_orphans` in `bench\u002Ftf\u002Fmodules\u002Fcluster\u002Fgke\u002Fmain.tf`, a Compute network, subnet or address by `hack\u002Fci_sweep_compute_plants.py` through its description prefix; a kind neither lists (a forwarding rule, a service attachment, a backend service, a health check) waits for the sweep to gain it before the stack's first apply. Its \"Live run needs\" line names the runner's roles and that sweep (#2755): the pool runners hold no Compute network or load-balancer role today. Before the card, read the condition live on every cluster the eval projects hold, since a GKE default can move under the scenario (NodeLocal DNSCache is on by default from 1.34.1); then no standing cluster carries it and the fixture creates or shares one.\n9. **The plant is a shape.** A fixture plants only the configuration a customer's cluster really has (a propagation-mounting DaemonSet, a pool with no pod PID limit, a controller with rate limits disabled), never a signal only the fixture can write (a custom node condition type, a synthetic event): a check that reads the invented signal goes green on the fleet and fires on no customer, and rung 5 flips the row on it (#5425, the seven class C cases). The grep of `bench\u002Ftf\u002Ffleet\u002Ffixtures.json` this section's intro asks for is what finds a role that already carries the shape (`multi-process-container` stood in for a PID case); never duplicate a plant another case asserts as the fleet's only one of its kind (`group-oom-kill`). A case that plants nothing declares `fixtures: []`: a single unrelated role in `fixtures:` is read by the fixture-drift step as the role the grade depends on, and a drift on it writes the case off (`bench\u002Fkube_agents_bench\u002Ffixture_drift.py`, `case_roles`).\n10. **Un-parking onto a fleet fixture.** A case parked as `FIXTURE_NOT_READY`, or on an unopened branch, whose condition a seeded fleet role now holds lands as a new case: `fixtures: [\u003Crole>]`, `infrastructure.deployer: noop`, every `resource_property` safeguard a `fleet_resource_property` with its `fixture_role` and values from the fleet Terraform (`bench\u002Ftf\u002Ffleet\u002Fdefects-\u003Cslot>.tf`, `main.tf`); the per-run stack and the parked entry go, and nothing is planted. The red waits for the fixture's roll, and the body cites the reconcile report and the scan. Two repository tests shape the safeguards: a fleet check asserts only on a subject in the role's `probes` (`bench\u002Ftests\u002Ftest_fleet_verifier.py`; the runner's implicit namespace probe does not count, so a namespace label has no safeguard unless the role probes the namespace), and a path-scoped `absent` needs a witness pair in `_PATH_SCOPED_ABSENT_WITNESSES`. Until the fixture pull request merges, validate on a scratch merge of its head (the ComputeClass roles of #5441 and the four cases written on them).\n11. **The premise.** When the platform's own semantics or the product's reference contradict the support case (a toleration with no `operator` reads as `Equal`; a class-targeted workload needs no hand-written toleration), the header states the contradiction, the honesty line claims only the shape the fixture holds, and the gap issue carries the question for the domain owner, with the case leaving the suite if the owner closes it (the operator-less toleration case on #5441's fixture).\n\n## Cost to expect\n\nA check defect: one to two hours and four runs. A collector or procedure gap: half a day to a day, and up to forty runs when a family of cases shares the cause. An agent-behaviour fix needs an install serving the model the rule failed on.\n","# Eval fix loop\n\nOne root cause per loop. A classified cause with quoted evidence before any code. Never loosen a check to fit a model's wording; a bar above the product's own rule under the standing fixture state is lowered only on a replay that flips addressed fails and zero passes, with the dropped branch pinned by a unit test. Three greens on the final head. Markers flip in the same pull request.\n\n## First-day addendum (2026-11-02)\n\n1. Before anything, take one row from the batch file the planner names (`~\u002F.claude\u002Fhandoffs\u002Fcert-batch-\u003Cdate>.md`, outside the repository) and write your session name in its claims column. This skill is a repository file, not an installed one: read `.agents\u002Fskills\u002Feval-fix-loop\u002FSKILL.md` with the file reader; the Skill tool does not list it.\n2. Read the store state first: `expected_fail` on `main`, passes and fails per model, `fail` against `not_applicable`. A new case has no store rows and skips this item. The read: `BaselineStore.load(\"gs:\u002F\u002Fkube-agents-evals-bench\u002Fevidence\", only=[case])`, then `store.admission(case, key, bar=AdmissionBar(), mode=\"roster\")`, with `key` from the store's own newest record for the case (`load_versions` gives fleet and verifiers only); cite the lane's current key alone, never two keys side by side (rule 58).\n3. Search open issues and open pull requests three ways: the case name, the failing check's reason string, the mechanism. A fix in flight ends the loop with a page row note. For a certification case, also grep the gap issue number across the suite's task headers (`grep -l '^suite: certification$' bench\u002Ftasks\u002F*\u002Ftask.yaml`): a case already on that issue means this one plants a different signal of the same issue, or ends as a page row note. Read the issue's state as well: a merge-queue keyword closes an issue with no fix on `main` (#4943 on 2026-11-10, through #5136), a declared gap on a closed issue is not declared, and the reopen is an ask. A nightly-red issue on the case (`scripts\u002Feval_dashboard\u002Fnightly_bugs.py` files it and never closes it) gets one comment on the first full green after the fix merged: the class, the fixing pull request, the green build id. It is closed on the human's word. Items 1, 2, the store half of 4 and the store read in 12 are certification-only: a nightly red has no batch row and no store, and skips them.\n4. The red is on the record: cite the store's reading by its word and two failing run directories by name (nothing from inside them), and replay them through the check as it is on `main`.\n5. For a check-only or harness fix the red is the record (build ids, the failing check by name, two run directories) and the green is the replay of every recorded answer with the new check: all addressed fails pass, zero passes turn to fails. Only a harness fix with no recorded answer to replay takes the unit test instead, red against `main` and green ten of ten. Three live greens are for an agent or product change only; a live repetition of a recorded red proves nothing the record does not.\n6. A wording check admits every phrasing of the fact and excludes nothing; a safeguard grades the action. Narrowing is measured on the record.\n7. Pin every wording check through the whole check, never with a regex alone: the helper `check_passes` in `bench\u002Ftests\u002Fconftest.py` once #4530 merges, until then the four-line form (`transcript.set`, then `ReportContainsVerifier(**check).verify(5.0).status`).\n8. Prefer the case folder. A change under `bench\u002Fkube_agents_bench\u002F` must still work when the presubmit grades with the base commit's bench.\n9. A case whose wrong answer may open a pull request the reply never names declares `writes_alone: true` (a key that lands with #4530); it then runs alone, and the lane safeguard charges it any write in its window.\n10. The body keeps every sentence at 25 words or fewer. It says `Part of #\u003Cissue>`, except the pull request that lands a declared gap's check, which says `Closes #\u003Cgap issue>`. It carries `Part of #3978`, the certification epic, no `Co-Authored-By` trailer, and ends with \"Generated with the help of Claude Code.\"\n11. One session, one PR: mind it to the first human lgtm, then hand it to the PR minder with a written handoff. Post `\u002Freview` only if the bot did not review on open.\n12. Step 8 is \"send the numbers\": record them in your own status with `cd ~\u002Fka-overseer && python3 overseer\u002Fovsr.py report --from \u003Csession>` : fails flipped, passes kept, the PR number, the page row note. Message the Overseer only for an ask. The store takes the night at the morning store read.\n13. A roster admission of a case that already holds a record is not a new case: the record stands in for live runs (quote the store's admission line and one `data.json` snapshot's `generated_at`; #4564); read `hack\u002Feval\u002Fblocking-roster.txt` and `presubmit-cases.txt` first, since the one-lane rule costs a presubmit seat when the case holds one; the pins are `scripts\u002Ftest_eval_rosters.py` (its frozen lists) and `hack\u002Feval\u002Fnightly-main-*.txt` (stack-bearing cases only).\n\n## Inputs\n\n- A case id under `bench\u002Ftasks\u002F\u003Cid>\u002F`, or a root cause with its case ids.\n- The run directories that show the failure: a nightly build's `artifacts\u002F` in the local cache, or a local `bench\u002Fresults\u002Frun_*\u002F`.\n- A checkout with the case, its fixture (`bench\u002Ftf\u002Fprebuilt\u002F\u003Cstack>\u002F` or the seeded fleet), and the product code behind it: the procedure (`agents\u002Fplatform\u002Fskills\u002F\u003Cskill>\u002F` SOP markdown), the collector scripts (`agents\u002Fplatform\u002Fskills\u002F\u003Cskill>\u002Fscripts\u002F`), the checks (`bench\u002Fkube_agents_bench\u002Fverifiers.py`).\n- The rules: `.agents\u002Frules\u002Feval_driven_development.md` governs red, green and registration; this skill adds the diagnosis and the refusals.\n\n## Step 1: pick and brief\n\nWrite a one-screen brief before reading anything else in depth:\n\n- Case id, domain, audit stream, the install model and release of the failing runs.\n- For a case that plants a fixture: the fields the lane's host type adds or changes on the planted objects, read with `kubectl -n default create -f - --dry-run=server -o yaml` over the manifest. A safeguard on a field admission writes is wrong, and a condition admission removes cannot be planted there (Autopilot writes `limits.ephemeral-storage` on every container).\n- Pass counts per model from the record (`bench\u002Frun-store\u002Fruns\u002F` if present, else the run directories).\n- The failing checks by name, with their `reason` strings quoted verbatim from `results.json` `verification_report[]`.\n\n## Step 2: diagnose from the runs\n\nRead in this order, and quote what you find with its source path:\n\n1. `results.json` → `verification_report[]`: which checks failed, which passed, and the exact reasons.\n2. `results.json` → `output`: the agent's final report. Does it name the target resource? Does it name the planted object under a different finding? Does it say anything was \"unevaluated\", \"unjudged\", \"skipped\", a \"coverage gap\", or \"not applicable\"?\n3. `results.json` → `trajectory[]`: did the required collector or script run (`tool_called`, `terminal` commands)? What did the collector print? Grep for the target resource, the namespace, and words like `unevaluated`, `skipped`, `error`, `429`, `timeout`.\n4. The ledger or pull request the check read, when the check names one (`ledger_issue_contains` reasons name the issue): what finding ids does it carry? A ledger is rewritten by every later run, so when the live issue has moved on, recover the body the check graded from the issue's edit history (GraphQL `userContentEdits { editedAt diff }` on the issue) rather than reading the current one.\n5. For a fleet-audit case, the collector manifest and the findings document the run wrote in the sandbox, copied out during the run since the next run overwrites them: `kubectl -n kubeagents-system exec platform-agent-shell-0 -c shell -- cat \u002Fopt\u002Fdata\u002Fscratch\u002Fmanifest_\u003Caudit>.json` (per cluster, `candidates` and `checks_not_applicable`) and the same for `findings_\u003Caudit>.json`.\n6. What those two files split, which the ledger alone cannot: the collector never listed the object (collector gap); the collector listed it and the worker dropped it (agent behaviour); the finding is in the document and the ledger body left it out (ledger rendering).\n7. The fixture: was the planted state really there (the safeguard's reason tells you; a `resource_property` pass proves the object existed with that value)?\n8. The product code the case exercises: the collector predicate or the procedure line that should have produced the finding. Find where the slug the check wants would be emitted. If nowhere, say so.\n9. The last green record of the same case on the same lane, when one exists: diff its `trajectory[]` and `output` against the first red's. A part that differs in shape rather than in the agent's choices (a marker count, an entry's tag, an argument written as `\u003Cstring, N chars>`) names the writer of that part, and `git log \u003Cgreen head>..\u003Cred head> -- \u003Cits path>` names the change. #5201's cause showed only this way, and #5179 names the writer it suspects.\n\n## Step 3: classify\n\nPick exactly one class, by the first rule that matches, and write the rule you used:\n\n| Class            | Rule that selects it                                                                                                                                                                                                                                                                                                                                                                                                                                                         | Fix type allowed                                                                                                                                                                                                                                                                                                                                                                                         |\n| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Scenario unheld  | The state the header asserts cannot be produced or held by the stack: a provider rule, a channel's list (#5175, #4222: a REGULAR cluster cannot leave its channel's list)                                                                                                                                                                                                                                                                                                    | Bind the check to the state the fixture holds, or to the run-time state a check reads; pin the other branch in a unit test on recorded inputs; prove by item 5's replay                                                                                                                                                                                                                                  |\n| Fixture defect   | The safeguard or the planted object is absent, or the planted state differs from the scenario (wrong namespace, wrong value, pods not in the state the scenario describes)                                                                                                                                                                                                                                                                                                   | Fix the stack; re-run red                                                                                                                                                                                                                                                                                                                                                                                |\n| Record defect    | Step 2 item 9 shows that the part of the record which differs from the last green was written by the transport, the harness or a product component off the case's path, while the agent followed the rule and the check read the record as written (a worker's calls on the router trace, #5201, the writer #5179 names; an argument shaped on the inject door, #5077, whose fix set the check aside until #2619, the lever this row refuses; a stale in-flight note, #4268) | Fix the writer, never the check, and prove with a record from the fixed head replayed through the check as it is on `main`: a writer in `bench\u002Fkube_agents_bench\u002F` or `hack\u002Fci-eval-pr.sh` by that replay alone, with the one-line exemption under Live validation; a writer in `a2a\u002F` with one live run on the fixed head so replayed; a writer under `agents\u002F` runs the loop (item 5 and Step 6 stand) |\n| Check defect     | The report or ledger shows the right outcome, but the check reads it wrong: a phrase list the report does not use, a type compare (string against boolean), a shortened id, a display-line parse                                                                                                                                                                                                                                                                             | Make the check read state or the ledger id, not words; add the missing phrasing only beside a state check; never delete the check                                                                                                                                                                                                                                                                        |\n| Collector gap    | The collector ran and logged that it skipped, could not evaluate, or does not know the object or shape (for example a ComputeClass priority of a kind it does not parse)                                                                                                                                                                                                                                                                                                     | A predicate or a parser branch in the collector, with a test                                                                                                                                                                                                                                                                                                                                             |\n| Procedure gap    | The collector saw the state, the procedure has no check for the pattern, and no finding was emitted                                                                                                                                                                                                                                                                                                                                                                          | A new check in the procedure with a published source line, and the collector support it needs                                                                                                                                                                                                                                                                                                            |\n| Ledger rendering | The finding is in the run's findings document and absent from the ledger body: the body budget rendered the higher severities in full, listed some without detail, and dropped the rest, so no `Where:` line carries the object (the `audit-findings-all` ids, clipped at 100 characters, are all that remain)                                                                                                                                                               | A renderer change with a test (every finding keeps one line with its check and object); until it lands, the case lands as `expected_fail` naming the renderer issue, since no check edit can read what the body does not carry                                                                                                                                                                           |\n| Agent behaviour  | The procedure has the rule, the collector gave the facts, and the agent did not follow the rule (skipped a step, reworded a verdict, did not open the required tool)                                                                                                                                                                                                                                                                                                         | Move the rule out of prose: a script that prints the verdict line, a check, or a persona rule the persona loads on every run                                                                                                                                                                                                                                                                             |\n| Role set         | The agent lacked a permission or an API the scenario needs, and the report says so                                                                                                                                                                                                                                                                                                                                                                                           | A permission change proposed to the humans; never a Secret grant                                                                                                                                                                                                                                                                                                                                         |\n| Infrastructure   | A 429, a timeout, a lost lease, a missing token, a forge-less install for a ledger check                                                                                                                                                                                                                                                                                                                                                                                     | Not a red; fix the environment and re-run                                                                                                                                                                                                                                                                                                                                                                |\n\nA row's rule names what the reader sees in the record; it may not depend on an earlier row's fix failing. Write the class with three evidence quotes and one sentence on why the next-nearest class does not apply. If two classes apply to different checks of the same case, split: one loop per class.\nThen, for any class but Infrastructure, write the Step 3b card; no code and no run beyond the record before it. An Infrastructure red is fixed and re-run first, with no card.\n\n## Step 3b: the decision card\n\nOne card per loop, written from the record after Step 3. It goes to the human first; code starts on the human's word. Fixed headings, this order, each within the size its text names.\n\n1. **The case in customer terms.** Two sentences: who asks, and what they should get back.\n2. **What the agent does.** One verbatim line per failing shape, its count over the record, and the `file:line` of the code that produced the text.\n3. **Root cause.** One paragraph: the Step 3 class, the evidence line that selects it, and why the next-nearest class does not apply.\n4. **Behaviour or case.** In two sentences, say whether the case's bar is the product's own stated rule, and quote the rule. If the case is loose or unbound, give the one-line tweak: bind the claim to the planted object, or require the thing the customer must see.\n5. **What the customer wants to see.** Before and after, real text, each under 10 lines. The after carries the verdict, the one proving number, and the one thing they did not ask but should know; for a check-only fix the after is the recorded answer and the card says so.\n6. **The fix, one lever per cause.** One paragraph per lever. Prose for a prompt contradiction, code for a code behaviour, a fixture change for a fixture gap; never prose to cover code. Give the size in lines, the resource cost in requests and objects, and where it lives; name a Hermes patch as one, with the note that it moves when Hermes goes. When one root cause takes two levers, both halves ship in one pull request.\n7. **Do less.** The one-line option, and what it would certify falsely.\n8. **Live run needs.** One line each: install, forge, fixture, model, minutes.\n9. **Not covered.** At most three bullets.\n10. **Rec and principle.** One line each.\n\nThe card and the fix it proposes obey these rules:\n\n| Do                                                                           | Do not                                                     |\n| ---------------------------------------------------------------------------- | ---------------------------------------------------------- |\n| Name the model of each red and green; compare no two models (rule 58).       | Report a green on one model as a green on another.         |\n| Bind every check to the planted object.                                      | Grade a sibling finding as the planted one.                |\n| Keep to one lever per cause.                                                 | Use prose to cover a code behaviour.                       |\n| State a limit as a ceiling (\"stays under\").                                  | State a limit as a target (\"aim for\").                     |\n| Show the customer the verdict, the proving number and the one unasked thing. | Patch wording round after round when a state check exists. |\n| Count server-side, with a field or label selector.                           | List everything and filter locally.                        |\n| Put numbers in a table and link every pull request and issue.                | Add a constant or a mechanism for its own sake.            |\n| Keep every claim on the record, with its run directory or line.              | Widen the fix mid-loop without a new card.                 |\n| Stop after the card and wait for the word.                                   | Take a review clear as push authority.                     |\n| Let the review check's green or the third round assign the human reviewer.   | Request a human reviewer by hand.                          |\n\n## Step 4: red\n\nRun the case once, unchanged, on an install the check can be graded on (a ledger check needs a forge; a plain-report check does not). Keep the run directory; the pull request cites it by name and the failing reason stays in it. Use the lease tool. For a Record defect, a Check defect or a Scenario unheld the red is already on the record (the Step 1 brief: build ids, reason, two run directories); a live repetition proves nothing the record does not and spends a lease.\n\n## Step 5: fix\n\nMake the smallest change the class allows, on a branch from fresh `upstream\u002Fmain`:\n\n- A check gains a state or ledger read; a phrase list grows only beside one; a type compare becomes a typed compare.\n- A collector gains the predicate or parser branch, with a unit test on the shape that was skipped.\n- A procedure gains the check and its `source:` line.\n- A behaviour rule moves into a script, a check, or the persona file, and the prose line that failed is removed or shortened.\n\nRefuse: a fixture name or a test id in product code; a phrase list edited to match one model's report; two root causes in one branch; a product change that makes the eval pass without changing what the product does for a customer.\n\n## Step 6: green\n\nOn the final head: run the case three times; run every sibling case of the same root cause once. All three must pass on the deterministic checks. The one exception is a Record defect whose writer is in `bench\u002Fkube_agents_bench\u002F` or `hack\u002Fci-eval-pr.sh`: its green is the replay of the fixed head's record through the check on `main`, with the exemption line under Live validation; a writer in `a2a\u002F` adds one live run on the fixed head so replayed. A judged score moving is not a pass.\n\n## Step 7: ship\n\nBefore the push, run the whole set the CI shards run, not the touched files: `make test-python` and `make test-bench`; a change to a shared writer (a hand-off's compose, a finding line, a collector's object) reaches every fixture derived from it. Derived fixtures regenerate with their own writer, never by hand: the planted raw reports under `bench\u002Ftf\u002Fprebuilt\u002Fbootstrap-ranking\u002F` by writing `bootstrap_handoff.compose(state, timed_out=False, now=PLANTED_AT)` over their `*.cards.json` to the `.txt` (`bench\u002Ftests\u002Ftest_bootstrap_ranking.py` pins the equality), the shared chat cases with `agents\u002Fchat\u002Fscripts\u002Ftest_inventory_card.py --write-cases` then prettier. Open the pull request from the template: the red (install, commit, failing check by name, the run's build id or directory), the three greens (run build ids or directories), the class and where its evidence is (run directory, `file:line`), the markers flipped (`expected_fail: true` removed for the cases the fix proves), the registration line, and the self-review from `\u002Fpr-preflight`. A declared-gap case names its gap issue and never `Closes` it; a reviewer is not named by hand before the review check's green assigns one, and `\u002Frequest-review` is for a disputed finding or a missing review only.\n\n## Step 8: record\n\nAppend the runs to the run store and move the case's row in the suite status file from red to fixing, with the pull request number.\n\n## A new certification case\n\nA new case is not a repair: it has no class and no fix. Its evidence follows `.agents\u002Frules\u002Feval_driven_development.md` in two shapes. A baseline (the product check exists and names the planted object) has no red; the record is three passes on the install's release build, run directories in the body. A declared gap (no check reads the planted signal, a procedure or collector gap you will not fix) lands with one red for the declared reason, `expected_fail: true`, and the gap issue named in its header and in the `#` reason lines above its roster line (\"When the fix is not yours\"); no green exists until the gap's check lands, and the pull request that adds the check flips the marker and closes the issue. A certification case carries `suite: certification` in its `task.yaml`, and its roster line with those reason lines goes in the certification roster (`hack\u002Feval\u002Fcertification-cases.txt` while that file exists; `scripts\u002Fvalidate_bench_cases.py` refuses the suite key without it), never the nightly file, so the rule's step 4 file does not apply. The card replaces Step 3b's headings and Step 7's evidence list (trap 7 is that list) with: the product check or the gap issue; the fixture (objects, labels, destroy); the objective under `scope: finding`; the safeguards, each read whole; the risk to the host cluster; the size in lines; the evidence shape. Before the card: verify every slug the sketch names against the collector (`git grep` the slug constant), and grep `bench\u002Ftf\u002Ffleet\u002Ffixtures.json` and the fleet README for the condition, since a plant that exists is read, not built. A new case skips addendum items 2, 4 and 5 (store rows and replay), and its card adds what it plants against the customer failure, the do-less line and the skill shortcomings. The fixture rules live in `bench\u002FREADME.md` (\"Certification suite\", rules 1 to 7); the traps below point at them and add what the loop itself met on a certification case:\n\n1. **Fixture readability.** Rule 4 of the README: the plant is something the agent's role can read, never a Secret; a webhook configuration's `caBundle` stood in for a TLS Secret (`security-expiring-node-bootstrap-and-webhook-certs`, #4960).\n2. **Host mode and admission.** Rule 5 of the README: the lane's hosts are Autopilot, and the Step 1 dry-run says what admission writes; the ComputeClass case was parked under it (#4943, planting issue #5143). A plant that needs a Standard cluster lives in the seeded fleet stack (`bench\u002Ftf\u002Ffleet`) under a `fixtures:` role; its cases park in `FIXTURE_NOT_READY` until the reconcile report shows every pool project rolled (`docs\u002Fci-pool-projects.md`, section 6.2). When no lease is open at card time, the dry run moves to the red step under the lease, the card marks its safeguard values pending the dry-run output, and a sibling case's recorded dry run on the same host type is the placeholder.\n3. **The plant shell.** Rule 6 of the README: the shared module for a namespace-scoped plant, its guards copied for a cluster-scoped one (#4939, #4979).\n4. **Objectives.** For a ledger case, one `ledger_issue_contains` with `scope: finding`, `finding_check` the accepted slugs and the planted object as the phrase (both objects of a pair under `any_of_phrases`). A body phrase for the object beside a slug among the ids is unpaired: a run can file them on different findings and pass (#4978; `bench\u002Ftests\u002Ftest_webhook_ca_bundle_case_binding.py` pins one case against it). A chat case has no ledger: its objective is a `report_contains` on the gate-rendered line. The gap issue names the slugs and says a fixing pull request that picks another edits `finding_check` in the same diff, and it states the `Where:` object a flip must name; when the natural object is a node, whose name GKE assigns, pair the workload with the pool name, which the node name carries (#5425). When the fixture rules keep part of the customer defect off the planted object (a check that reads a namespace list the fixture cannot use, such as a GKE-managed namespace; a fail-closed webhook whose selector must exclude what the customer's webhook fails to exclude, rule 1 of the README), the case header and the gap issue state both flip conditions: the shape of check that flips the case, and the shape that reads only the kept-off part and does not (`security-webhook-blocks-config-management-system`, #5338, #5335: backend reachability flips it, the selector exemption alone does not).\n5. **Safeguards.** Rule 7 of the README: one per tempted property, read whole, values from the dry run; never a replica count on an autoscaled Deployment (#4949).\n6. **The run.** One run on a forge install: a ledger check cannot fail for the gap's reason without a forge. Pre-open the agent API on a port of your own with the `kubectl exec` relay `bench\u002FREADME.md` names under Sandboxed installs (`scripts\u002Fhermes-dashboard-tunnel.py --container agent-api-auth --remote-port 8643 --local-port \u003Cport>`; a plain `kubectl port-forward` binds but cannot reach a sandboxed pod, and every request dies with `RemoteDisconnected`) and set `AGENT_LOCAL_PORT` to it (another session's relay on the harness's default port sends the run to another install and it reads 401; a non-default port with nothing behind it is forwarded by the harness to the agent Service, which the sandboxed gateway refuses). A red counts only when the ledger issue's stamp falls inside the run's window; a report with no issue URL is a fail on the product's record (#5076): count it among the runs the body cites, each by run id and outcome word, and re-run to add runs, never to replace one. A plant that crash-loops spends the install's event-watcher alert quota (five Warnings per UTC day): read the `alert_quota` row at lease start, say the counts in the body, and route the reset at release as an ask when your session cannot write the gateway's database.\n7. **The record.** The body names the case as new, the run id, the failing check by name (its reason stays in the run directory, `.agents\u002Frules\u002Fpublic_surfaces.md`), the install's release and mode without its name, the plant as captured (a `kubectl get pod` line), the destroy, the admission dry-run's output, and the marker's issue; when review changes the objective, the maintainers may ask three repetitions on the new head before the push.\n8. **A plant outside the host cluster.** A class-B plant (a per-run cluster, a cluster setting, or project-level network objects) has no kubectl guard, so Terraform's own destroy replaces the module's guards. Where the run's kubeconfig points decides the safeguards: a plant that is itself a cluster puts that cluster in the stack's `cluster_name` and `cluster_location` outputs, the run's kubeconfig moves to it for the run's length, and `resource_property` then reads the plant; project-level network objects leave the outputs on the host cluster through `TF_VAR_host_cluster_name` and `_location` (`prebuilt\u002Fsubnet-range-exhaustion`), nothing in the cluster is the plant, and the worker-commands read-only safeguard on the write that would undo the plant is the catastrophic one. A plant of a setting the API reads back proves itself with a data source whose `postcondition` fails the plan when the planted value is absent; a baseline's objective proves a network plant, and only a marked case needs the proof elsewhere. The objective's phrase is the planted object's name prefix, since a per-run name carries a hash of `cluster_name`. A leftover is swept by label or description, never by the host teardown: a labelled cluster by `reap_orphans` in `bench\u002Ftf\u002Fmodules\u002Fcluster\u002Fgke\u002Fmain.tf`, a Compute network, subnet or address by `hack\u002Fci_sweep_compute_plants.py` through its description prefix; a kind neither lists (a forwarding rule, a service attachment, a backend service, a health check) waits for the sweep to gain it before the stack's first apply. Its \"Live run needs\" line names the runner's roles and that sweep (#2755): the pool runners hold no Compute network or load-balancer role today. Before the card, read the condition live on every cluster the eval projects hold, since a GKE default can move under the scenario (NodeLocal DNSCache is on by default from 1.34.1); then no standing cluster carries it and the fixture creates or shares one.\n9. **The plant is a shape.** A fixture plants only the configuration a customer's cluster really has (a propagation-mounting DaemonSet, a pool with no pod PID limit, a controller with rate limits disabled), never a signal only the fixture can write (a custom node condition type, a synthetic event): a check that reads the invented signal goes green on the fleet and fires on no customer, and rung 5 flips the row on it (#5425, the seven class C cases). The grep of `bench\u002Ftf\u002Ffleet\u002Ffixtures.json` this section's intro asks for is what finds a role that already carries the shape (`multi-process-container` stood in for a PID case); never duplicate a plant another case asserts as the fleet's only one of its kind (`group-oom-kill`). A case that plants nothing declares `fixtures: []`: a single unrelated role in `fixtures:` is read by the fixture-drift step as the role the grade depends on, and a drift on it writes the case off (`bench\u002Fkube_agents_bench\u002Ffixture_drift.py`, `case_roles`).\n10. **Un-parking onto a fleet fixture.** A case parked as `FIXTURE_NOT_READY`, or on an unopened branch, whose condition a seeded fleet role now holds lands as a new case: `fixtures: [\u003Crole>]`, `infrastructure.deployer: noop`, every `resource_property` safeguard a `fleet_resource_property` with its `fixture_role` and values from the fleet Terraform (`bench\u002Ftf\u002Ffleet\u002Fdefects-\u003Cslot>.tf`, `main.tf`); the per-run stack and the parked entry go, and nothing is planted. The red waits for the fixture's roll, and the body cites the reconcile report and the scan. Two repository tests shape the safeguards: a fleet check asserts only on a subject in the role's `probes` (`bench\u002Ftests\u002Ftest_fleet_verifier.py`; the runner's implicit namespace probe does not count, so a namespace label has no safeguard unless the role probes the namespace), and a path-scoped `absent` needs a witness pair in `_PATH_SCOPED_ABSENT_WITNESSES`. Until the fixture pull request merges, validate on a scratch merge of its head (the ComputeClass roles of #5441 and the four cases written on them).\n11. **The premise.** When the platform's own semantics or the product's reference contradict the support case (a toleration with no `operator` reads as `Equal`; a class-targeted workload needs no hand-written toleration), the header states the contradiction, the honesty line claims only the shape the fixture holds, and the gap issue carries the question for the domain owner, with the case leaving the suite if the owner closes it (the operator-less toleration case on #5441's fixture).\n\n## Cost to expect\n\nA check defect: one to two hours and four runs. A collector or procedure gap: half a day to a day, and up to forty runs when a family of cases shares the cause. An agent-behaviour fix needs an install serving the model the rule failed on.\n",{"name":5,"description":7},"main","63856252193548b498b5a9e39aae2fcc79110995",".agents\u002Fskills\u002Feval-fix-loop",43,"https:\u002F\u002Fgithub.com\u002Fgke-labs\u002Fkube-agents","https:\u002F\u002Fgke-labs.github.io\u002Fkube-agents\u002F","https:\u002F\u002Fgithub.com\u002Fgke-labs","2026-11-12 05:35:56",{"source":8,"skillName":5,"commands":32,"prompt":42,"zipAvailable":43},[33,36,39],{"tool":34,"command":35},"npx","npx skills add gke-labs\u002Fkube-agents --skill eval-fix-loop",{"tool":37,"command":38},"bunx","bunx skills add gke-labs\u002Fkube-agents --skill eval-fix-loop",{"tool":40,"command":41},"pnpm","pnpm dlx skills add gke-labs\u002Fkube-agents --skill eval-fix-loop","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fgke-labs\u002Fkube-agents\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Feval-fix-loop\u002FSKILL.md",true,[],[46,53,60,67,74,81],{"skillNo":47,"name":48,"nameZh":6,"description":49,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":50,"categories":51,"keywords":52,"publishedAt":18},"01M720VZZQNT2JQVK1JJCXX977","gce-compute-fleet-audit","Audits standalone GCE virtual machines, Managed Instance Groups (MIGs), serial console boot failures, sole-tenant node group headroom, and orphaned disk snapshots.","https:\u002F\u002Fgithub.com\u002Fgke-labs\u002Fkube-agents\u002Fblob\u002Fmain\u002Fagents\u002Fplatform\u002Fskills\u002Fgce-compute-fleet-audit\u002FSKILL.md",[],[],{"skillNo":54,"name":55,"nameZh":6,"description":56,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":57,"categories":58,"keywords":59,"publishedAt":18},"01M720VKEW26G8QGT9NY4XQDR5","fleet-audit","Publish the findings of an autonomous fleet audit as one continuously-rewritten issue per audit stream per install on the GitOps repository's forge, and propose fixes as narrow remediation pull requests (merge requests on GitLab).","https:\u002F\u002Fgithub.com\u002Fgke-labs\u002Fkube-agents\u002Fblob\u002Fmain\u002Fagents\u002Fplatform\u002Fskills\u002Ffleet-audit\u002FSKILL.md",[],[],{"skillNo":61,"name":62,"nameZh":6,"description":63,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":64,"categories":65,"keywords":66,"publishedAt":18},"01M720SYHF3JZYBM8CW52MCXS8","gke-rightsizing-checks","Five required checks, and the arithmetic, before any proposal to lower a workload's CPU or memory requests or limits, or to change a node pool's machine type or size, and before any claim of capacity freed. Read-only. For \"lower the requests of X\", \"right-size this deployment\", \"can this pool use a smaller machine type\", and any answer that claims nodes, CPU, memory or a percentage freed.","https:\u002F\u002Fgithub.com\u002Fgke-labs\u002Fkube-agents\u002Fblob\u002Fmain\u002Fagents\u002Fcluster\u002Fskills\u002Fgke-rightsizing-checks\u002FSKILL.md",[],[],{"skillNo":68,"name":69,"nameZh":6,"description":70,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":71,"categories":72,"keywords":73,"publishedAt":18},"01M720PC2JZ2NV08DK93YMH1VA","review-chat-ux","Reviews a diff that changes what users read in Slack or Google Chat against the chat UX rules, and reports each violation with file:line and the rule it breaks.","https:\u002F\u002Fgithub.com\u002Fgke-labs\u002Fkube-agents\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Freview-chat-ux\u002FSKILL.md",[],[],{"skillNo":75,"name":76,"nameZh":6,"description":77,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":78,"categories":79,"keywords":80,"publishedAt":18},"01M720WDAGMMRY1FXYVHGE7HBV","github-issue-resolver","Autonomously poll, triage, investigate, and resolve unaddressed open issues on our target repository (on GitHub or GitLab) strictly within authorized scope.","https:\u002F\u002Fgithub.com\u002Fgke-labs\u002Fkube-agents\u002Fblob\u002Fmain\u002Fagents\u002Fplatform\u002Fskills\u002Fgithub-issue-resolver\u002FSKILL.md",[],[],{"skillNo":82,"name":83,"nameZh":6,"description":84,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":85,"categories":86,"keywords":87,"publishedAt":18},"01M720PETPTF0TE7NEZC31BW5X","review-docs-drift","Reviews a pull request for documentation drift — finds which docs the change should have updated, verifies doc claims against source, and checks the docs map and AGENTS.md themselves for staleness.","https:\u002F\u002Fgithub.com\u002Fgke-labs\u002Fkube-agents\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Freview-docs-drift\u002FSKILL.md",[],[]]