[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7AX6X4TAKG1M4899BB6V2AA":3,"skill-same:01M7AX6X4TAKG1M4899BB6V2AA":47},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":24,"commitSha":25,"skillDirectory":26,"forksCount":27,"repositoryHtmlUrl":28,"repositoryHomepageUrl":29,"ownerHtmlUrl":30,"ownerBlogUrl":29,"ownerDescription":31,"updatedAt":32,"install":33,"resources":46},"01M7AX6X4TAKG1M4899BB6V2AA","conformance-debug",null,"Fast, targeted debugging of the Harness Conformance Lab (conformance\u002F) — use whenever a conformance check fails (locally or in CI), a check is recorded as a declared limitation, a harness version drifted, a harness TUI check needs poking at directly, or you're about to guess at a fix instead of reproducing the real failure. Covers reading the failure's shape, checking the vendor's current docs\u002Fchangelog\u002Fissues before touching code, the --sandbox and --experiment fast loops, proving the provider speaks the real wire shape with --discovery and mitmproxy, telling a vendor limitation from a Lab defect, and reading evidence\u002Fverdict.json.","uze-sh\u002Fuze","uze-sh","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F334211959?v=4","01M7AX5KNT81AC56YMRT586MBW","uze","01M7AX5KPDQA2QP7DWGZFMH4PQ",4,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fuze-sh\u002Fuze\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fconformance-debug\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: conformance-debug\ndescription: Fast, targeted debugging of the Harness Conformance Lab (conformance\u002F) — use whenever a conformance check fails (locally or in CI), a check is recorded as a declared limitation, a harness version drifted, a harness TUI check needs poking at directly, or you're about to guess at a fix instead of reproducing the real failure. Covers reading the failure's shape, checking the vendor's current docs\u002Fchangelog\u002Fissues before touching code, the --sandbox and --experiment fast loops, proving the provider speaks the real wire shape with --discovery and mitmproxy, telling a vendor limitation from a Lab defect, and reading evidence\u002Fverdict.json.\n---\n\n# Debugging the Harness Conformance Lab\n\nThe Lab runs the real vendor CLI (Claude Code, Codex, OpenCode, Antigravity)\nagainst a synthetic provider in a disposable Docker container. What every\nharness must prove lives in `conformance\u002Fcontract\u002F` (outcome terms, no\nvendor named); how each harness is driven lives in\n`conformance\u002Fharnesses\u002F\u003Cvendor>\u002Fbindings.py`; what is genuinely unique to a\nvendor stays in its `scenarios.py`. A full gate run\n(`python3 conformance\u002Flab.py --harness \u003Ch>`) takes 6-11 minutes and only\ntells you pass\u002Ffail per check. **Never iterate against that loop.** There\nare seconds-long loops built for exactly this — reach for them first.\n\nThe Lab's whole value is that a green is real. Two things make a green\nfalse, and both have shipped for months before: a check that passes\nbecause nothing happened (a hook that never ran, a tool call the harness\nrejected before executing), and a provider that speaks a wire shape the\nreal API does not, so the harness silently does something else than it\nwould in production. Every step below exists to catch one of those.\n\n## Step 0 — is the world still the one the Lab assumes?\n\nDo this **before** touching code, on any failure, any ADAPTED, and any\n`VERSION DRIFT` line in the run summary. The Lab pins nothing: the image\ninstalls each vendor's channel-latest, so a run that fails \"for no reason\"\nis usually the vendor moving.\n\n- **Read the vendor's current docs and changelog — the ones that shipped\n  with the binary you are testing, not the ones in your memory.** Several\n  ship inside the CLI: Antigravity keeps its customization docs under\n  `~\u002F.gemini\u002Fantigravity-cli\u002Fbuiltin\u002Fskills\u002Fagy-customizations\u002Fdocs\u002F`\n  (`hooks.md`, `plugins.md`, `skills.md`, `json_configs.md`, …) and prints\n  release notes with `agy changelog`; Claude Code's are at\n  `code.claude.com\u002Fdocs` (skills, hooks, plugins); Codex and OpenCode\n  publish theirs on their sites and repos. A one-line `--sandbox -- \"agy\n  changelog | head -80\"` or a `WebFetch` of the docs page is cheaper than\n  any hypothesis.\n- **Search the vendor's GitHub issues** for the exact symptom (the error\n  text the TUI rendered, the log line, the flag name). A known regression\n  with an issue number is evidence to pin a version against; a fixed one\n  tells you the next channel bump resolves it.\n- **Treat every ADAPTED entry in `conformance\u002Fevidence\u002Fexpected.json` as a\n  question, not a fact.** Each says \"this harness could not do X when we\n  last looked\". On every version drift, and whenever you touch the area,\n  re-ask it against the current docs: the control may now exist (Claude\n  gained `user-invocable: false`; the Lab kept declaring it unverifiable\n  because the binding read the wrong surface). An ADAPTED that has become\n  false is a false green in the making — the gate escalates a registered\n  ADAPTED that starts passing precisely so you cannot miss it.\n- **Assume nothing about the wire.** Harness behaviour that depends on the\n  provider's response shape (streaming events, tool-call argument names,\n  which request of a turn is the user's) changes across vendor versions.\n  Claude Code accumulates a tool's input only from `input_json_delta`;\n  Antigravity 1.1.24 validates a scripted call against the tool's declared\n  `parametersJsonSchema` before any hook runs, and makes a side call to a\n  lighter model with no tools before the user's turn. A provider that\n  counts requests or invents argument names produces turns that \"settle\"\n  with nothing having happened. See Step 3.\n\n## Step 1 — read the shape of the failure\n\n- **The same first check fails identically across all four harnesses**\n  (e.g. `tui-reached-prompt` \u002F `tui-started`, with a near-empty first\n  `.raw` capture) → the shared **setup script** died before any harness TUI\n  ever launched. Do not touch TUI-driving\u002Fwait logic — the bug is in the\n  setup steps every vertical shares (`uze market add`, `uze add\n  \u003Cplugin>@uze-lab -m`), or in `uze` itself.\n- **Only one harness fails** → the bug is specific to that harness's\n  `crates\u002Fuze-integrations\u002Fsrc\u002F\u003Cvendor>\u002F` module, its `bindings.py`, its\n  `scenarios.py`, or its provider — or the vendor moved (Step 0).\n- **A late-stage check fails after earlier ones pass** → the TUI\u002Fharness\n  came up fine; the bug is in that specific interaction (skills, MCP,\n  hooks), not in provisioning.\n- **A presence check fails while its absence checks pass** (e.g.\n  `hooks-*-denial-relayed` red, `hooks-*-marker-absent-*` green) → the turn\n  ran with nothing intercepted. The absence checks are gated on the\n  presence check for this reason; read what the harness rendered\n  (`Invalid tool parameters`, `invalid arguments`, a permission prompt)\n  and go to Step 3.\n\nThe setup scripts run with `set -e` and `>\u002Fdev\u002Fnull 2>&1` on purpose (a\nclean gate run must be quiet) — which means a setup failure is **silent**\nin the normal evidence: a 0-byte or garbage-looking first `.raw` file is\nthe signature of \"the container died during setup,\" not a TUI bug. Don't\ndebug the wait loop for a dead process.\n\n## Step 2 — reproduce in seconds, not minutes\n\n`lab.py` has a sandbox mode built for this (`--help` documents it in full):\n\n```bash\n# Runs one command inside the real, fully-provisioned harness container\n# (fixture marketplace + synthetic provider already up), then tears down.\npython3 conformance\u002Flab.py --harness \u003Cclaude|codex|opencode|antigravity> \\\n  --sandbox -- \"\u003Cshell command>\"\n```\n\nUse it to poke at the exact step you suspect, e.g.:\n\n```bash\npython3 conformance\u002Flab.py --harness opencode --sandbox -- \\\n  \"uze market add \u002Fwork\u002Fmarket >&2; uze add mcp-plugin@uze-lab -m; uze doctor\"\n```\n\nThis isolates \"did `uze` do the right thing here\" from \"did the TUI render\nthe right thing,\" in seconds, with real (uncensored) stdout\u002Fstderr.\n\n- The sandbox shell is bare: set `PATH=\u002Fusr\u002Flocal\u002Fbin:\u002Fusr\u002Fbin:\u002Fbin:\u002Fusr\u002Flocal\u002F.local\u002Fbin`\n  and copy the vendor fixtures the way `scenarios.py`'s setup fragment does\n  (Antigravity: `settings.json`, `jetski_state.pbtxt`, `installation_id`\n  into `~\u002F.gemini\u002Fantigravity-cli\u002F`) before running the vendor CLI.\n- `--shell` instead of `-- cmd` for an interactive rootless shell in the\n  container (needs a real TTY — from a non-interactive agent session, use\n  `-- cmd` instead). `--keep` leaves the provider + network alive between\n  rounds.\n- To test a **vendor CLI's own behavior** in isolation (does it parse this\n  path\u002Fargument the way we assume? what does its validator say about this\n  file?) — the sandbox drops you next to the real vendor binary. Never\n  guess from docs or your own priors when a one-line call gets you the\n  real exit code, stderr, or log. `agy plugin validate \u003Cdir>` counted 1\n  hook where UZE meant 3, which is how the `hooks.json` wrapper bug was\n  found; `agy -p \"\u002Fhooks\" --output-format json` lists what the harness\n  actually registered; `--log-file` gives you the vendor's own log.\n- **A hypothesis that needs a real turn is an experiment**, not a sandbox\n  command: `conformance\u002Fexperiments\u002F\u003Cvendor>\u002F\u003Cname>.py` gets the\n  provisioned topology, starts its own provider in the mode it needs\n  (`common.start_provider(cfg, \"toolcall\", {...})`), drives a headless or\n  TUI turn, and records its own checks under the run outdir — outside the\n  canonical suite, versioned in the repo as evidence. Run with\n  `lab.py --harness \u003Ch> --experiment \u003Cvendor>\u002F\u003Cname>`. Switch behaviour\n  with environment variables rather than editing the file between runs.\n  `experiments\u002Fantigravity\u002Fhook-tui.py` is the template: it names its\n  container so the vendor's log can be read mid-session with `docker exec`.\n\n## Step 3 — prove the provider speaks the real wire shape\n\nThe synthetic provider is the Lab's model of the vendor's API. If it drifts\nfrom what the real API sends, the harness is not being tested — a\ndifferent program is. Whenever a turn \"settles\" without the expected\neffect, or a harness rejects a scripted call, read the wire:\n\n```bash\npython3 conformance\u002Flab.py --harness \u003Ch> --discovery      # any mode: sandbox, experiment, vertical\n```\n\n`--discovery` appends every raw request the harness sent to the provider\nto `\u003Coutdir>\u002Fraw-requests.log`, across every per-phase provider restart.\nRead the request that carries the tool result: Antigravity's\n`functionResponse.output` said *\"missing properties 'Cwd',\n'WaitMsBeforeAsync', 'CommandLine', 'toolSummary', 'toolAction' —\nadditional properties 'command' not allowed\"*, which named the fix. Read\nthe request's `tools`\u002F`functionDeclarations` (or Anthropic `tools`) for\nthe schema the harness declared — script calls in **that** shape, never\none you remember. Read which request is the user's turn (roles, declared\ntools, `\u003CUSER_REQUEST>`) and make the provider answer by content, the way\na model would, never by request count.\n\nFor the shape of the **response** side — what the real API streams, which\nthe Lab cannot capture offline — use the observation tooling in\n`conformance\u002Fdiscovery\u002F` (mitmproxy addons that sanitize at capture time,\n`run_agy_obs.sh` for a real authenticated session through the proxy). It\nis observation-only and its raw captures never enter the repository; what\nit teaches becomes the provider's contract, written down in the provider's\nmodule docstring with the vendor version it was observed on (see\n`harnesses\u002Fclaude\u002Fprovider.py`: streaming event sequence, `input_json_delta`,\nno `data: [DONE]` for Gemini).\n\nThe checklist for any provider change:\n\n1. The request the harness sends after a tool call carries the tool's\n   *real* result (proof marker, stdout marker, denial reason) — not an\n   error string. Grep `raw-requests.log` for it.\n2. The provider answers the user's turn, not the harness's side calls\n   (title generation, summaries, a lighter model with no tools).\n3. Streaming events match what the vendor's SDK accumulates.\n4. A structural summary (`struct.json`) records *presence* of markers per\n   request; aggregate with `common.observed_markers` (a union), never\n   last-write-wins.\n\n## Step 4 — vendor limitation, or Lab defect?\n\nOnly declare a limitation when the vendor itself cannot do it **in a way\nyou have measured**. A declaration goes through `declare` (or\n`contract\u002Fdeclared.py`'s `presence`\u002F`absence`), whose `holds` is the\nmeasurement of this run, never a constant: the lint refuses a literal.\nRegister it in `evidence\u002Fexpected.json` with its reason, its measurement\nand the versions it was observed on; `*` is refused. The gate then reads\neach run's measurement: a registered one that still holds passes, one that\nholds on a version not listed passes as a repin and the run writes\n`expected.next.json`, one whose measurement found the control escalates,\nand one whose measurement never ran is unproven. The tools for the\nmeasurement:\n\n- **A control probe in the vendor's own format at the vendor's own path,\n  with no UZE in the loop.** If a vendor-format deny hook at\n  `~\u002F.gemini\u002Fconfig\u002Fhooks.json` is loaded, listed by `\u002Fhooks`, and never\n  executes — not even a `touch` per event — then no UZE delivery can be\n  observed there, and the vertical must measure that gate at run time\n  (`hooks > vendor` in the Antigravity vertical) rather than assume it in\n  a spec. A live precondition is the only declaration that can expire on\n  its own: the gate escalates the day the vendor opens it.\n- **The same probe for a rendering or a prompt.** Whether a harness shows\n  a hook's denial as a decision was settled by a hand-written hook in the\n  user's own `settings.json` (`experiments\u002Fclaude\u002Fdeny-render`): Claude\n  2.1.290 showed the documented JSON decision as \"hook error\" too, so the\n  check became a registered declaration instead of a UZE fix. The image\n  has no Python or jq; an experiment edits vendor JSON with `sed`.\n- **Pin the previous image.** `UZE_LAB_IMAGE=\u003Cimage id> python3\n  conformance\u002Flab.py ...` runs the same scenario against the harness\n  version that last passed. Same result on both → it was never the\n  vendor's regression (and probably never worked: check the old green for\n  vacuity). Different → a regression, pin the version in the registry and\n  file\u002Freference the vendor issue.\n- **Read the vendor's log and binary before concluding \"impossible\".**\n  `--log-file` shows what loaded and what surfaced; `strings` on the binary\n  (copy it out with `docker cp`) shows the flags and proto fields that\n  gate a feature (`CustomizationConfig.enable_json_hooks` was found this\n  way). A gate fed by a server-side feature provider the offline Lab never\n  receives is a real limitation; a gate that is a documented setting is a\n  fixture to set.\n\n## Step 5 — the image is the checkout, or the run does not happen\n\nThe image bakes a release build of `uze` plus the real vendor CLIs. A\nstale image tests your *old* code, and its failures read exactly like a\nregression of the change you are making — this has cost whole sessions.\nThe Lab now refuses that: the image carries a `sh.uze.lab.source` label,\nthe digest of every file its build reads (uncommitted ones included,\n`shared\u002Fimage.py`), and **every `lab.py` run compares it with the checkout\nand rebuilds the image first when they differ** (`lab image: … Rebuilding.`\non stderr). So:\n\n- Run `lab.py` from the checkout whose code you are testing. Never point it\n  at an image someone else built, and never skip the rebuild it starts.\n- Before reading a failure, check the run's `provenance:` line: `uze=` must\n  be the version this checkout builds, and `fixture=` its commit. A\n  mismatch means the run proves nothing about your change — rerun.\n- `UZE_LAB_IMAGE=\u003Cimage>` is the one way past the check, on purpose: to\n  rerun a scenario against an older build (Step 4). It says so on stderr;\n  outside that use, leave it unset.\n- `make lab-image` builds the same labelled image by hand.\n\nRebuilds are fast when only `crates\u002F`\u002F`src\u002F` changed (cached apt\u002Fprovision\nlayers); expect ~30-40s from a clean base, ~5-10s incremental. Python under\n`conformance\u002F` (providers, scenarios, bindings, contract) runs on the host\nand is not in the digest — no rebuild for it. The fixture marketplace is\nbaked into the image and is in the digest, so a fixture added since the\nlast build triggers the rebuild on its own.\n\n## The measured vocabulary\n\nWhat a harness calls its tools is measured, never remembered:\n\n- `capture.declared_tools` records every tool each request declares to the\n  model (`declared-tools.json`); a scripted call naming a tool the request\n  did not declare is refused and recorded (`undeclared-calls.json`).\n- The `hook-rows` fixture's census group (no matcher) records the tool\n  name and input every hook receives (`hooked-tools.json`); each alias has\n  its own row group, and `RowsFixtureTest` fails when one is missing.\n- `harnesses\u002F\u003Ch>\u002Fvocabulary.json` is the Lab's own expectation, written\n  independently of UZE's `ToolBinding` tables; `vocabulary-*` checks hold\n  both against the capture, and Rust's `hooks\u002Fmeasured_tests.rs` holds the\n  tables against the committed snapshot (`evidence\u002Ftools\u002F\u003Ch>.json`).\n- `--record-vocabulary` rewrites the snapshot from the run; the diff is the\n  review of what the vendor changed. An `optional` tool (offered only on\n  opt-in) is judged by the session that scripted it, never by the run's\n  union: an agent that lists a tool in its own definition is offered it.\n\n## Prompts and markers\n\n- Every prompt a person meets is answered on screen through the binding\n  (`approval_prompts`, `prepare`), the way a person answers it. A flag,\n  a seeded fixture answer or a bypass needs a `# decision: \u003Cid>` naming an\n  entry under \"Prompts the Lab answers\" in `conformance\u002FDECISIONS.md`; the\n  lint fails otherwise.\n- A marker is a nonce no harness writes on its own; the lint refuses a\n  common word and a marker inside a scripted call's arguments.\n\n## Reading evidence\n\nEach run's `outdir` (default `\u002Ftmp\u002Fharness-conformance\u002F\u003Charness>\u002Frun\u003CN>\u002F`,\nor `$AGY_OUTDIR`) holds:\n\n- `NN_\u003Ccheckpoint>.raw` \u002F `\u003Cphase>.raw` — plain-text screen snapshot at\n  that checkpoint. Empty\u002Fnear-empty on an early one is the setup-died\n  signature (Step 1). The screen helper returns only the bytes that\n  arrived since the last read; a snapshot is what one wait consumed.\n- `\u003Cphase>.typescript` + `.timing` — full terminal recording (`script(1)`\n  format); replay with `make lab-replay`, or strip ANSI and read the tail\n  to see what the harness rendered right before a turn stalled (a\n  permission prompt, `Invalid tool parameters`, a survey).\n- `\u003Cphase>_struct.json` — the provider's structural summary per request\n  (path, roles, declared tools, marker presence). Which model\u002Fpath each\n  request hit tells you which request was the user's turn.\n- `raw-requests.log` — with `--discovery`, the verbatim requests.\n- `verdict.json` — structured result per check, including `gate.adjudication`\n  (`asserted`, `known_declared`, `repin`, `unregistered`, `wildcard`,\n  `unproven`, `escalated`) and the run manifest\n  (harness\u002Fuze\u002Fimage\u002Ffixture versions — check `version_drift` first if\n  nothing else changed).\n\n## Don't\n\n- Don't trust a run whose `provenance:` names another `uze` than the one\n  this checkout builds, and don't set `UZE_LAB_IMAGE` to dodge a rebuild\n  (Step 5).\n- Don't iterate against the full gate run (`lab.py --harness \u003Ch>` with no\n  `--sandbox`\u002F`--experiment`) while narrowing down a cause — that's the\n  ~10-minute loop, reserved for confirming the fix at the end.\n- Don't assume vendor CLI or API behaviour from memory, or from docs older\n  than the binary under test, when a one-line sandbox call, the vendor's\n  shipped docs, its changelog, or a `--discovery` capture gets you the\n  real answer.\n- Don't treat \"all 4 harnesses broke on the same push\" as 4 separate bugs —\n  check the shared setup path first (Step 1).\n- Don't hard-code a declaration in a scenario. It is either a binding's\n  `unsupported` reason (a harness limitation, reviewable) measured through\n  `contract\u002Fdeclared.py`, or a live precondition that measures the vendor\n  each run; a registry entry must exist for it, pinned to the observed\n  versions.\n- Don't answer a prompt for the person with a flag or a seeded file to get\n  a phase green: that proves the capability on a machine nobody has.\n- Don't let a check pass on a turn where nothing happened. Gate every\n  absence on a presence; demand the tool's own stdout, not merely \"a tool\n  result\"; read the wire when in doubt.\n","# Debugging the Harness Conformance Lab\n\nThe Lab runs the real vendor CLI (Claude Code, Codex, OpenCode, Antigravity)\nagainst a synthetic provider in a disposable Docker container. What every\nharness must prove lives in `conformance\u002Fcontract\u002F` (outcome terms, no\nvendor named); how each harness is driven lives in\n`conformance\u002Fharnesses\u002F\u003Cvendor>\u002Fbindings.py`; what is genuinely unique to a\nvendor stays in its `scenarios.py`. A full gate run\n(`python3 conformance\u002Flab.py --harness \u003Ch>`) takes 6-11 minutes and only\ntells you pass\u002Ffail per check. **Never iterate against that loop.** There\nare seconds-long loops built for exactly this — reach for them first.\n\nThe Lab's whole value is that a green is real. Two things make a green\nfalse, and both have shipped for months before: a check that passes\nbecause nothing happened (a hook that never ran, a tool call the harness\nrejected before executing), and a provider that speaks a wire shape the\nreal API does not, so the harness silently does something else than it\nwould in production. Every step below exists to catch one of those.\n\n## Step 0 — is the world still the one the Lab assumes?\n\nDo this **before** touching code, on any failure, any ADAPTED, and any\n`VERSION DRIFT` line in the run summary. The Lab pins nothing: the image\ninstalls each vendor's channel-latest, so a run that fails \"for no reason\"\nis usually the vendor moving.\n\n- **Read the vendor's current docs and changelog — the ones that shipped\n  with the binary you are testing, not the ones in your memory.** Several\n  ship inside the CLI: Antigravity keeps its customization docs under\n  `~\u002F.gemini\u002Fantigravity-cli\u002Fbuiltin\u002Fskills\u002Fagy-customizations\u002Fdocs\u002F`\n  (`hooks.md`, `plugins.md`, `skills.md`, `json_configs.md`, …) and prints\n  release notes with `agy changelog`; Claude Code's are at\n  `code.claude.com\u002Fdocs` (skills, hooks, plugins); Codex and OpenCode\n  publish theirs on their sites and repos. A one-line `--sandbox -- \"agy\n  changelog | head -80\"` or a `WebFetch` of the docs page is cheaper than\n  any hypothesis.\n- **Search the vendor's GitHub issues** for the exact symptom (the error\n  text the TUI rendered, the log line, the flag name). A known regression\n  with an issue number is evidence to pin a version against; a fixed one\n  tells you the next channel bump resolves it.\n- **Treat every ADAPTED entry in `conformance\u002Fevidence\u002Fexpected.json` as a\n  question, not a fact.** Each says \"this harness could not do X when we\n  last looked\". On every version drift, and whenever you touch the area,\n  re-ask it against the current docs: the control may now exist (Claude\n  gained `user-invocable: false`; the Lab kept declaring it unverifiable\n  because the binding read the wrong surface). An ADAPTED that has become\n  false is a false green in the making — the gate escalates a registered\n  ADAPTED that starts passing precisely so you cannot miss it.\n- **Assume nothing about the wire.** Harness behaviour that depends on the\n  provider's response shape (streaming events, tool-call argument names,\n  which request of a turn is the user's) changes across vendor versions.\n  Claude Code accumulates a tool's input only from `input_json_delta`;\n  Antigravity 1.1.24 validates a scripted call against the tool's declared\n  `parametersJsonSchema` before any hook runs, and makes a side call to a\n  lighter model with no tools before the user's turn. A provider that\n  counts requests or invents argument names produces turns that \"settle\"\n  with nothing having happened. See Step 3.\n\n## Step 1 — read the shape of the failure\n\n- **The same first check fails identically across all four harnesses**\n  (e.g. `tui-reached-prompt` \u002F `tui-started`, with a near-empty first\n  `.raw` capture) → the shared **setup script** died before any harness TUI\n  ever launched. Do not touch TUI-driving\u002Fwait logic — the bug is in the\n  setup steps every vertical shares (`uze market add`, `uze add\n  \u003Cplugin>@uze-lab -m`), or in `uze` itself.\n- **Only one harness fails** → the bug is specific to that harness's\n  `crates\u002Fuze-integrations\u002Fsrc\u002F\u003Cvendor>\u002F` module, its `bindings.py`, its\n  `scenarios.py`, or its provider — or the vendor moved (Step 0).\n- **A late-stage check fails after earlier ones pass** → the TUI\u002Fharness\n  came up fine; the bug is in that specific interaction (skills, MCP,\n  hooks), not in provisioning.\n- **A presence check fails while its absence checks pass** (e.g.\n  `hooks-*-denial-relayed` red, `hooks-*-marker-absent-*` green) → the turn\n  ran with nothing intercepted. The absence checks are gated on the\n  presence check for this reason; read what the harness rendered\n  (`Invalid tool parameters`, `invalid arguments`, a permission prompt)\n  and go to Step 3.\n\nThe setup scripts run with `set -e` and `>\u002Fdev\u002Fnull 2>&1` on purpose (a\nclean gate run must be quiet) — which means a setup failure is **silent**\nin the normal evidence: a 0-byte or garbage-looking first `.raw` file is\nthe signature of \"the container died during setup,\" not a TUI bug. Don't\ndebug the wait loop for a dead process.\n\n## Step 2 — reproduce in seconds, not minutes\n\n`lab.py` has a sandbox mode built for this (`--help` documents it in full):\n\n```bash\n# Runs one command inside the real, fully-provisioned harness container\n# (fixture marketplace + synthetic provider already up), then tears down.\npython3 conformance\u002Flab.py --harness \u003Cclaude|codex|opencode|antigravity> \\\n  --sandbox -- \"\u003Cshell command>\"\n```\n\nUse it to poke at the exact step you suspect, e.g.:\n\n```bash\npython3 conformance\u002Flab.py --harness opencode --sandbox -- \\\n  \"uze market add \u002Fwork\u002Fmarket >&2; uze add mcp-plugin@uze-lab -m; uze doctor\"\n```\n\nThis isolates \"did `uze` do the right thing here\" from \"did the TUI render\nthe right thing,\" in seconds, with real (uncensored) stdout\u002Fstderr.\n\n- The sandbox shell is bare: set `PATH=\u002Fusr\u002Flocal\u002Fbin:\u002Fusr\u002Fbin:\u002Fbin:\u002Fusr\u002Flocal\u002F.local\u002Fbin`\n  and copy the vendor fixtures the way `scenarios.py`'s setup fragment does\n  (Antigravity: `settings.json`, `jetski_state.pbtxt`, `installation_id`\n  into `~\u002F.gemini\u002Fantigravity-cli\u002F`) before running the vendor CLI.\n- `--shell` instead of `-- cmd` for an interactive rootless shell in the\n  container (needs a real TTY — from a non-interactive agent session, use\n  `-- cmd` instead). `--keep` leaves the provider + network alive between\n  rounds.\n- To test a **vendor CLI's own behavior** in isolation (does it parse this\n  path\u002Fargument the way we assume? what does its validator say about this\n  file?) — the sandbox drops you next to the real vendor binary. Never\n  guess from docs or your own priors when a one-line call gets you the\n  real exit code, stderr, or log. `agy plugin validate \u003Cdir>` counted 1\n  hook where UZE meant 3, which is how the `hooks.json` wrapper bug was\n  found; `agy -p \"\u002Fhooks\" --output-format json` lists what the harness\n  actually registered; `--log-file` gives you the vendor's own log.\n- **A hypothesis that needs a real turn is an experiment**, not a sandbox\n  command: `conformance\u002Fexperiments\u002F\u003Cvendor>\u002F\u003Cname>.py` gets the\n  provisioned topology, starts its own provider in the mode it needs\n  (`common.start_provider(cfg, \"toolcall\", {...})`), drives a headless or\n  TUI turn, and records its own checks under the run outdir — outside the\n  canonical suite, versioned in the repo as evidence. Run with\n  `lab.py --harness \u003Ch> --experiment \u003Cvendor>\u002F\u003Cname>`. Switch behaviour\n  with environment variables rather than editing the file between runs.\n  `experiments\u002Fantigravity\u002Fhook-tui.py` is the template: it names its\n  container so the vendor's log can be read mid-session with `docker exec`.\n\n## Step 3 — prove the provider speaks the real wire shape\n\nThe synthetic provider is the Lab's model of the vendor's API. If it drifts\nfrom what the real API sends, the harness is not being tested — a\ndifferent program is. Whenever a turn \"settles\" without the expected\neffect, or a harness rejects a scripted call, read the wire:\n\n```bash\npython3 conformance\u002Flab.py --harness \u003Ch> --discovery      # any mode: sandbox, experiment, vertical\n```\n\n`--discovery` appends every raw request the harness sent to the provider\nto `\u003Coutdir>\u002Fraw-requests.log`, across every per-phase provider restart.\nRead the request that carries the tool result: Antigravity's\n`functionResponse.output` said *\"missing properties 'Cwd',\n'WaitMsBeforeAsync', 'CommandLine', 'toolSummary', 'toolAction' —\nadditional properties 'command' not allowed\"*, which named the fix. Read\nthe request's `tools`\u002F`functionDeclarations` (or Anthropic `tools`) for\nthe schema the harness declared — script calls in **that** shape, never\none you remember. Read which request is the user's turn (roles, declared\ntools, `\u003CUSER_REQUEST>`) and make the provider answer by content, the way\na model would, never by request count.\n\nFor the shape of the **response** side — what the real API streams, which\nthe Lab cannot capture offline — use the observation tooling in\n`conformance\u002Fdiscovery\u002F` (mitmproxy addons that sanitize at capture time,\n`run_agy_obs.sh` for a real authenticated session through the proxy). It\nis observation-only and its raw captures never enter the repository; what\nit teaches becomes the provider's contract, written down in the provider's\nmodule docstring with the vendor version it was observed on (see\n`harnesses\u002Fclaude\u002Fprovider.py`: streaming event sequence, `input_json_delta`,\nno `data: [DONE]` for Gemini).\n\nThe checklist for any provider change:\n\n1. The request the harness sends after a tool call carries the tool's\n   *real* result (proof marker, stdout marker, denial reason) — not an\n   error string. Grep `raw-requests.log` for it.\n2. The provider answers the user's turn, not the harness's side calls\n   (title generation, summaries, a lighter model with no tools).\n3. Streaming events match what the vendor's SDK accumulates.\n4. A structural summary (`struct.json`) records *presence* of markers per\n   request; aggregate with `common.observed_markers` (a union), never\n   last-write-wins.\n\n## Step 4 — vendor limitation, or Lab defect?\n\nOnly declare a limitation when the vendor itself cannot do it **in a way\nyou have measured**. A declaration goes through `declare` (or\n`contract\u002Fdeclared.py`'s `presence`\u002F`absence`), whose `holds` is the\nmeasurement of this run, never a constant: the lint refuses a literal.\nRegister it in `evidence\u002Fexpected.json` with its reason, its measurement\nand the versions it was observed on; `*` is refused. The gate then reads\neach run's measurement: a registered one that still holds passes, one that\nholds on a version not listed passes as a repin and the run writes\n`expected.next.json`, one whose measurement found the control escalates,\nand one whose measurement never ran is unproven. The tools for the\nmeasurement:\n\n- **A control probe in the vendor's own format at the vendor's own path,\n  with no UZE in the loop.** If a vendor-format deny hook at\n  `~\u002F.gemini\u002Fconfig\u002Fhooks.json` is loaded, listed by `\u002Fhooks`, and never\n  executes — not even a `touch` per event — then no UZE delivery can be\n  observed there, and the vertical must measure that gate at run time\n  (`hooks > vendor` in the Antigravity vertical) rather than assume it in\n  a spec. A live precondition is the only declaration that can expire on\n  its own: the gate escalates the day the vendor opens it.\n- **The same probe for a rendering or a prompt.** Whether a harness shows\n  a hook's denial as a decision was settled by a hand-written hook in the\n  user's own `settings.json` (`experiments\u002Fclaude\u002Fdeny-render`): Claude\n  2.1.290 showed the documented JSON decision as \"hook error\" too, so the\n  check became a registered declaration instead of a UZE fix. The image\n  has no Python or jq; an experiment edits vendor JSON with `sed`.\n- **Pin the previous image.** `UZE_LAB_IMAGE=\u003Cimage id> python3\n  conformance\u002Flab.py ...` runs the same scenario against the harness\n  version that last passed. Same result on both → it was never the\n  vendor's regression (and probably never worked: check the old green for\n  vacuity). Different → a regression, pin the version in the registry and\n  file\u002Freference the vendor issue.\n- **Read the vendor's log and binary before concluding \"impossible\".**\n  `--log-file` shows what loaded and what surfaced; `strings` on the binary\n  (copy it out with `docker cp`) shows the flags and proto fields that\n  gate a feature (`CustomizationConfig.enable_json_hooks` was found this\n  way). A gate fed by a server-side feature provider the offline Lab never\n  receives is a real limitation; a gate that is a documented setting is a\n  fixture to set.\n\n## Step 5 — the image is the checkout, or the run does not happen\n\nThe image bakes a release build of `uze` plus the real vendor CLIs. A\nstale image tests your *old* code, and its failures read exactly like a\nregression of the change you are making — this has cost whole sessions.\nThe Lab now refuses that: the image carries a `sh.uze.lab.source` label,\nthe digest of every file its build reads (uncommitted ones included,\n`shared\u002Fimage.py`), and **every `lab.py` run compares it with the checkout\nand rebuilds the image first when they differ** (`lab image: … Rebuilding.`\non stderr). So:\n\n- Run `lab.py` from the checkout whose code you are testing. Never point it\n  at an image someone else built, and never skip the rebuild it starts.\n- Before reading a failure, check the run's `provenance:` line: `uze=` must\n  be the version this checkout builds, and `fixture=` its commit. A\n  mismatch means the run proves nothing about your change — rerun.\n- `UZE_LAB_IMAGE=\u003Cimage>` is the one way past the check, on purpose: to\n  rerun a scenario against an older build (Step 4). It says so on stderr;\n  outside that use, leave it unset.\n- `make lab-image` builds the same labelled image by hand.\n\nRebuilds are fast when only `crates\u002F`\u002F`src\u002F` changed (cached apt\u002Fprovision\nlayers); expect ~30-40s from a clean base, ~5-10s incremental. Python under\n`conformance\u002F` (providers, scenarios, bindings, contract) runs on the host\nand is not in the digest — no rebuild for it. The fixture marketplace is\nbaked into the image and is in the digest, so a fixture added since the\nlast build triggers the rebuild on its own.\n\n## The measured vocabulary\n\nWhat a harness calls its tools is measured, never remembered:\n\n- `capture.declared_tools` records every tool each request declares to the\n  model (`declared-tools.json`); a scripted call naming a tool the request\n  did not declare is refused and recorded (`undeclared-calls.json`).\n- The `hook-rows` fixture's census group (no matcher) records the tool\n  name and input every hook receives (`hooked-tools.json`); each alias has\n  its own row group, and `RowsFixtureTest` fails when one is missing.\n- `harnesses\u002F\u003Ch>\u002Fvocabulary.json` is the Lab's own expectation, written\n  independently of UZE's `ToolBinding` tables; `vocabulary-*` checks hold\n  both against the capture, and Rust's `hooks\u002Fmeasured_tests.rs` holds the\n  tables against the committed snapshot (`evidence\u002Ftools\u002F\u003Ch>.json`).\n- `--record-vocabulary` rewrites the snapshot from the run; the diff is the\n  review of what the vendor changed. An `optional` tool (offered only on\n  opt-in) is judged by the session that scripted it, never by the run's\n  union: an agent that lists a tool in its own definition is offered it.\n\n## Prompts and markers\n\n- Every prompt a person meets is answered on screen through the binding\n  (`approval_prompts`, `prepare`), the way a person answers it. A flag,\n  a seeded fixture answer or a bypass needs a `# decision: \u003Cid>` naming an\n  entry under \"Prompts the Lab answers\" in `conformance\u002FDECISIONS.md`; the\n  lint fails otherwise.\n- A marker is a nonce no harness writes on its own; the lint refuses a\n  common word and a marker inside a scripted call's arguments.\n\n## Reading evidence\n\nEach run's `outdir` (default `\u002Ftmp\u002Fharness-conformance\u002F\u003Charness>\u002Frun\u003CN>\u002F`,\nor `$AGY_OUTDIR`) holds:\n\n- `NN_\u003Ccheckpoint>.raw` \u002F `\u003Cphase>.raw` — plain-text screen snapshot at\n  that checkpoint. Empty\u002Fnear-empty on an early one is the setup-died\n  signature (Step 1). The screen helper returns only the bytes that\n  arrived since the last read; a snapshot is what one wait consumed.\n- `\u003Cphase>.typescript` + `.timing` — full terminal recording (`script(1)`\n  format); replay with `make lab-replay`, or strip ANSI and read the tail\n  to see what the harness rendered right before a turn stalled (a\n  permission prompt, `Invalid tool parameters`, a survey).\n- `\u003Cphase>_struct.json` — the provider's structural summary per request\n  (path, roles, declared tools, marker presence). Which model\u002Fpath each\n  request hit tells you which request was the user's turn.\n- `raw-requests.log` — with `--discovery`, the verbatim requests.\n- `verdict.json` — structured result per check, including `gate.adjudication`\n  (`asserted`, `known_declared`, `repin`, `unregistered`, `wildcard`,\n  `unproven`, `escalated`) and the run manifest\n  (harness\u002Fuze\u002Fimage\u002Ffixture versions — check `version_drift` first if\n  nothing else changed).\n\n## Don't\n\n- Don't trust a run whose `provenance:` names another `uze` than the one\n  this checkout builds, and don't set `UZE_LAB_IMAGE` to dodge a rebuild\n  (Step 5).\n- Don't iterate against the full gate run (`lab.py --harness \u003Ch>` with no\n  `--sandbox`\u002F`--experiment`) while narrowing down a cause — that's the\n  ~10-minute loop, reserved for confirming the fix at the end.\n- Don't assume vendor CLI or API behaviour from memory, or from docs older\n  than the binary under test, when a one-line sandbox call, the vendor's\n  shipped docs, its changelog, or a `--discovery` capture gets you the\n  real answer.\n- Don't treat \"all 4 harnesses broke on the same push\" as 4 separate bugs —\n  check the shared setup path first (Step 1).\n- Don't hard-code a declaration in a scenario. It is either a binding's\n  `unsupported` reason (a harness limitation, reviewable) measured through\n  `contract\u002Fdeclared.py`, or a live precondition that measures the vendor\n  each run; a registry entry must exist for it, pinned to the observed\n  versions.\n- Don't answer a prompt for the person with a flag or a seeded file to get\n  a phase green: that proves the capability on a machine nobody has.\n- Don't let a check pass on a turn where nothing happened. Gate every\n  absence on a presence; demand the tool's own stdout, not merely \"a tool\n  result\"; read the wire when in doubt.\n",{"name":5,"description":7},"main","e2cfbb55eec2aa007135f9a2256b73f7170a3b9e",".agents\u002Fskills\u002Fconformance-debug",1,"https:\u002F\u002Fgithub.com\u002Fuze-sh\u002Fuze","https:\u002F\u002Fuze.sh","https:\u002F\u002Fgithub.com\u002Fuze-sh","🧩 Agents come and go. Your work stays.","2026-11-13 05:15:37",{"source":8,"skillName":5,"commands":34,"prompt":44,"zipAvailable":45},[35,38,41],{"tool":36,"command":37},"npx","npx skills add uze-sh\u002Fuze --skill conformance-debug",{"tool":39,"command":40},"bunx","bunx skills add uze-sh\u002Fuze --skill conformance-debug",{"tool":42,"command":43},"pnpm","pnpm dlx skills add uze-sh\u002Fuze --skill conformance-debug","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fuze-sh\u002Fuze\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fconformance-debug\u002FSKILL.md",true,[],[48,55,62,69,76,83],{"skillNo":49,"name":50,"nameZh":6,"description":51,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":52,"categories":53,"keywords":54,"publishedAt":19},"01M7AXDFA81RA77NHGYVZRCH0E","commit","North Star fixture skill — used to prove native\u002Fgenerated delivery across harnesses.","https:\u002F\u002Fgithub.com\u002Fuze-sh\u002Fuze\u002Fblob\u002Fmain\u002Ftests\u002F_fixtures\u002Fgolden\u002Fmarketplace\u002Fplugins\u002Fflow\u002Fskills\u002Fcommit\u002FSKILL.md",[],[],{"skillNo":56,"name":57,"nameZh":6,"description":58,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":59,"categories":60,"keywords":61,"publishedAt":19},"01M7AXCYQ5WMH46JRJVF57FTHA","oc-skill","opencode native","https:\u002F\u002Fgithub.com\u002Fuze-sh\u002Fuze\u002Fblob\u002Fmain\u002Ftests\u002F_fixtures\u002Fforeign\u002Fopencode\u002Fnative-plugin\u002Fskills\u002Foc-skill\u002FSKILL.md",[],[],{"skillNo":63,"name":64,"nameZh":6,"description":65,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":66,"categories":67,"keywords":68,"publishedAt":19},"01M7AXCVSFQ76DR4GXGJ1Q9PBA","uze-plugin-first","Emits the deterministic UZE Plugin First conformance proof.","https:\u002F\u002Fgithub.com\u002Fuze-sh\u002Fuze\u002Fblob\u002Fmain\u002Ftests\u002F_fixtures\u002Fforeign\u002Fcodex\u002Fnative-plugin\u002Fskills\u002Fuze-plugin-first\u002FSKILL.md",[],[],{"skillNo":70,"name":71,"nameZh":6,"description":72,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":73,"categories":74,"keywords":75,"publishedAt":19},"01M7AXCTD57QJ3GEN54W5PREV2","hello","says hello","https:\u002F\u002Fgithub.com\u002Fuze-sh\u002Fuze\u002Fblob\u002Fmain\u002Ftests\u002F_fixtures\u002Fforeign\u002Fclaude\u002Fnative-plugin\u002Fskills\u002Fhello\u002FSKILL.md",[],[],{"skillNo":77,"name":78,"nameZh":6,"description":79,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":80,"categories":81,"keywords":82,"publishedAt":19},"01M7AXCR1HKRHJ7PP39JQS8SQ5","agy-native","antigravity native","https:\u002F\u002Fgithub.com\u002Fuze-sh\u002Fuze\u002Fblob\u002Fmain\u002Ftests\u002F_fixtures\u002Fforeign\u002Fantigravity\u002Fnative-plugin\u002Fskills\u002Fagy-native\u002FSKILL.md",[],[],{"skillNo":84,"name":85,"nameZh":6,"description":86,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":87,"categories":88,"keywords":89,"publishedAt":19},"01M7AXCNTR4WXE4VT89MK4XQST","review","Review code for correctness and missing tests","https:\u002F\u002Fgithub.com\u002Fuze-sh\u002Fuze\u002Fblob\u002Fmain\u002Ftests\u002F_fixtures\u002Fcanonical\u002Fworkflow\u002Fskills\u002Freview\u002FSKILL.md",[],[]]