AGENT SKILL

validate-recommendation

Validate (Recommended) options in AskUserQuestion via N configurable panelists (~/.claude/panel/config.yml). Triggered by the validate-recommendation hook; do not invoke manually.

License 许可
MIT
最近更新
2026-10-13
Stars
★ 4
Forks
1

01 前言 Front Matter

name
validate-recommendation
description
Validate (Recommended) options in AskUserQuestion via N configurable panelists (~/.claude/panel/config.yml). Triggered by the validate-recommendation hook; do not invoke manually.

02 资源文件 53

03 SKILL.md 正文

Validate Recommendation

You were invoked because the validate-recommendation hook fired on an AskUserQuestion call that contained a (Recommended) option marker. Your job: dispatch an N-panelist review (where N comes from ~/.claude/panel/config.yml), aggregate the verdicts into a JSON directive, and act on it.

The panel composition is configurable and ships as three panelists: an adversarial DA, which is told which option was recommended, plus a blinded PE and QA, which are not. Any of them can be turned off, and new ones added, in config.yml. This skill is config-driven; it does NOT assume a fixed panel — always enumerate the enabled set from the config.

Inputs

The hook wrote tool input to a session state file:

STATE_FILE="${TMPDIR:-/tmp}/claude-panel-${CLAUDE_SESSION_ID:-$PPID}.json"

When CLAUDE_SESSION_ID is unset, both hook and skill resolve $PPID to the same value (the Claude Code parent process), so the skill reads exactly the file the hook wrote. Two concurrent CC sessions get different $PPIDs and therefore different state-file paths — the old -unknown sentinel collided across sessions.

Read it via the Read tool. If the file is missing, emit the user-visible message "Panel state file missing; asking the original question." and fall back to issuing the original AskUserQuestion unmodified. Stop here.

The panel config:

CONFIG="${HOME}/.claude/panel/config.yml"

State file keys you'll use:

  • tool_input.questions — array of question objects (1-4 per the AskUserQuestion schema)
  • timeout_seconds — per-panelist budget (default 90)

Setup

export PYTHONPATH="$HOME/.claude${PYTHONPATH:+:$PYTHONPATH}"

This ensures import tool (the shared engine) resolves at runtime. All subsequent python3.12 -m panel … calls inherit this setting.

1. Validate the config

Run via the Bash tool:

cd "${HOME}/.claude/skills/validate-recommendation" && \
    "${CLAUDE_PANEL_PYTHON:-python3.12}" -m panel lint-config --config "$CONFIG"

If exit code is non-zero, fall back: print "Panel disabled: config invalid (see lint-config output)." and re-issue the original AskUserQuestion unmodified. Stop.

2. Read enabled panelists from config

Use jq via Bash to enumerate enabled panelists:

jq -r '.panelists[] | select(.enabled) | "\(.id)|\(.role)|\(.backend)|\(.subagent_type // "")"' \
    <("${CLAUDE_PANEL_PYTHON:-python3.12}" -c "
import yaml, json, sys
print(json.dumps(yaml.safe_load(open('$CONFIG'))))
")

The yaml→json conversion runs once at skill startup. Capture the output into a variable; each line is <id>|<role>|<backend>|<subagent_type> for one enabled panelist.

(If python3.12 -c for yaml→json is awkward in your sandbox, the equivalent is cd "${HOME}/.claude/skills/validate-recommendation" && python3.12 -m panel lint-config --config "$CONFIG", which prints the same data in human-readable form; you can grep its output for the - <id> lines and extract role/backend from them. The cd is required here too: a Bash tool call does not inherit the cwd of step 1's, and without it panel resolves to the ~/.claude/panel config directory.)

2a. Resolve the panel profile

The default panel is all three panelists: an adversarial DA plus a blinded PE and QA. There is no team auto-detect — the ensemble is the standard operating mode, because at N=1 the hard threshold is ceil(1/2) = 1, which makes SOFT-DISSENT unreachable and turns every single OVERTURN into a hard interrupt.

One override, for cost control:

  • CLAUDE_PANEL_PROFILE=solo disables PE and QA for this dispatch, leaving DA only. Read it via Bash: echo "${CLAUDE_PANEL_PROFILE:-}". Use it when the extra two Claude subagent spawns are not worth it.

If solo is requested, write a resolved config with pe and qa set to enabled: false and use it as $CONFIG_RESOLVED for the REST of this run (both the section-2 enumeration AND the §4 aggregator — panel aggregate re-derives its panelist set from its --config, so it would otherwise silently disagree with what was dispatched).

CONFIG_RESOLVED="${TMPDIR:-/tmp}/panel-config-resolved-${CLAUDE_SESSION_ID:-$PPID}.yml"
"${CLAUDE_PANEL_PYTHON:-python3.12}" - "$CONFIG" "$CONFIG_RESOLVED" <<'PY'
import sys, yaml
cfg = yaml.safe_load(open(sys.argv[1]))
for p in cfg.get("panelists", []):
    if p.get("id") in ("pe", "qa"):
        p["enabled"] = False
yaml.safe_dump(cfg, open(sys.argv[2], "w"))
PY

Then re-run the section-2 enumeration against $CONFIG_RESOLVED (it now lists da only) and pass $CONFIG_RESOLVED to the §4 aggregator.

Otherwise use the stock $CONFIG unchanged and skip to step 3.

3. Create the per-session workdir

WORKDIR="${HOME}/.claude/panel/work/${CLAUDE_SESSION_ID:-$PPID}"
mkdir -p "$WORKDIR" && chmod 0700 "$WORKDIR"

This is the directory where per-panelist verdict files land. panel aggregate will read ${WORKDIR}/<id>.verdict for each enabled panelist after fan-out completes.

Per-question dispatch

For EACH question in tool_input.questions that has an option labeled with (Recommended) AND NOT (Recommended; Panel-flagged):

1. Build the per-panelist prompt files

Write the question payload to a JSON file, then let the CLI build each panelist's prompt. Blinding is applied automatically from each panelist's blind flag — do NOT hand-write prompt bodies, and do NOT mention the recommended option to a blinded panelist.

QPAYLOAD="${TMPDIR:-/tmp}/panel-question-${CLAUDE_SESSION_ID:-$PPID}-q<N>.json"
# Write JSON: {"question": ..., "options": [{"label":..., "description":...}],
#              "recommended_label": ..., "reasoning": ...}
# reasoning comes from the recommended option's description; if there is
# none, use "(no reasoning supplied)". NEVER invent hidden reasoning.

# The cd is load-bearing, exactly as in the lint-config, dispatch and
# aggregate blocks. $PYTHONPATH is $HOME/.claude, so from any other cwd
# `python -m panel` resolves `panel` to ~/.claude/panel — the CONFIG
# directory — which has no __init__.py and shadows the real package as a
# namespace package. The call then dies with "No module named
# panel.__main__" and writes no prompt file at all.
cd "${HOME}/.claude/skills/validate-recommendation"

for PID in <each enabled panelist id>; do
  "${CLAUDE_PANEL_PYTHON:-python3.12}" -m panel build-prompt \
    --panelist "$PID" \
    --config "${CONFIG_RESOLVED:-$CONFIG}" \
    --question-file "$QPAYLOAD" \
    --output "${TMPDIR:-/tmp}/panel-prompt-${CLAUDE_SESSION_ID:-$PPID}-q<N>-${PID}.txt"
done

Where <N> is the question index (0-based).

Each panelist is then dispatched with ITS OWN prompt file. A blinded panelist's file has the recommendation removed, the (Recommended) marker stripped, and the options deterministically reordered.

The payload's reasoning field is read only for UNBLINDED panelists; build-prompt substitutes an empty string for a blinded one. That is why the payload may carry the literal (no reasoning supplied) safely. Never call the underlying build_prompt_body yourself with a blinded panelist and a non-empty reasoning — including that literal, which is a non-empty string: it raises ValueError by design. Go through the CLI and the question is moot.

2. Fan out N panelists in ONE message

This is the parallelism point. All panelist dispatches MUST be in a single message so they run concurrently.

For EACH enabled panelist from the config enumeration above:

  • If backend starts with nat- (e.g., nat-nim): include a Bash tool call in your fan-out message. Export CLAUDE_SESSION_ID from the state file's session_id field in the SAME command — panel/trace.py reads it from the environment for verdict-line telemetry, and it is otherwise never a real exported variable (only ever referenced with a ${CLAUDE_SESSION_ID:-$PPID} fallback), which is why trace lines were showing session=unknown:

    export CLAUDE_SESSION_ID="$(jq -r '.session_id' "${TMPDIR:-/tmp}/claude-panel-${CLAUDE_SESSION_ID:-$PPID}.json")" && \
    cd "${HOME}/.claude/skills/validate-recommendation" && \
        "${CLAUDE_PANEL_PYTHON:-python3.12}" -m panel dispatch \
        --panelist "<id>" \
        --config "${CONFIG_RESOLVED:-$CONFIG}" \
        --persona "${HOME}/.claude/skills/validate-recommendation/personas/<role-lowercase>.md" \
        --prompt-file "${TMPDIR:-/tmp}/panel-prompt-${CLAUDE_SESSION_ID:-$PPID}-q<N>-<id>.txt" \
        --output "${WORKDIR}/<id>.verdict"
    

    The --prompt-file is THIS panelist's file from step 1, not a shared one. Passing another panelist's file hands a blinded seat the recommendation.

    dispatch.py writes the verdict file directly (mode 0600). Auth comes from per-backend env vars (nat-nim: $PANEL_DA_API_KEY, falling back to $NVIDIA_API_KEY; nat-anthropic: $ANTHROPIC_API_KEY; nat-openai: $OPENAI_API_KEY).

  • If backend is claude-subagent: include an Agent tool call in your fan-out message:

    • subagent_type: the panelist's subagent_type from config (e.g., principal-engineer, qa-engineer)
    • description: short, e.g., "Panel <role> review"
    • prompt: the concatenation of the persona file's # System prompt section, the # One-shot example section (if present), and the contents of THIS panelist's prompt file from step 1 (…-q<N>-<id>.txt), copied verbatim.

    Read both files via the Read tool BEFORE composing the fan-out message: the persona at ${HOME}/.claude/skills/validate-recommendation/personas/<role-lowercase>.md, and the panelist's own prompt file. Never re-compose the prompt body by hand — for a blinded panelist that is the leak.

All Bash + Agent calls go in ONE message. The framework executes them concurrently.

3. Collect verdict files after fan-out returns

When the fan-out message's tool results come back:

  • nat-* panelists: dispatch.py already wrote the verdict file. Nothing to do.

  • claude-subagent panelists: the Agent tool returned a string. Write the ENTIRE response verbatim to the verdict file via the Write tool:

    ${WORKDIR}/<id>.verdict
    

    Do NOT alter, summarize, or extract from the Agent response. The aggregator does the parsing, and which lines it wants depends on the panelist: an UNBLINDED one emits VERDICT:/RATIONALE:/ALTERNATIVE:, a BLINDED one emits CHOICE:/RATIONALE:. aggregate() maps a blinded CHOICE: onto the same contract — agreement with the recommended label is HOLD, disagreement is OVERTURN naming that option — comparing marker- and case-insensitively. Surrounding prose is ignored, and a blinded reply with no CHOICE: line becomes ERROR for that panelist.

4. Run the aggregator

Same rule as dispatch: export CLAUDE_SESSION_ID from the state file's session_id inside this SAME command, so the aggregator's trace lines carry the real session instead of session=unknown.

DIRECTIVE_JSON=$(export CLAUDE_SESSION_ID="$(jq -r '.session_id' "${TMPDIR:-/tmp}/claude-panel-${CLAUDE_SESSION_ID:-$PPID}.json")" && \
    cd "${HOME}/.claude/skills/validate-recommendation" && \
    "${CLAUDE_PANEL_PYTHON:-python3.12}" -m panel aggregate \
    --config "${CONFIG_RESOLVED:-$CONFIG}" \
    --verdicts-dir "$WORKDIR" \
    --recommended-label "<recommended_label from state for THIS question>")

Capture stdout into $DIRECTIVE_JSON. It's a single-line JSON object.

5. Parse the directive

Extract fields via jq:

VERDICT=$(jq -r '.verdict' <<< "$DIRECTIVE_JSON")
SUMMARY=$(jq -r '.summary' <<< "$DIRECTIVE_JSON")
ESCALATE=$(jq -r '.escalate_to_user // false' <<< "$DIRECTIVE_JSON")
GATE=$(jq -r '.rationale_gate_passed' <<< "$DIRECTIVE_JSON")

Values:

  • VERDICT ∈ {HOLD, SOFT-DISSENT, HARD-DISSENT, ERROR}
  • SUMMARY — one-line user-facing text, already sanitized
  • ESCALATE — true for HARD-DISSENT in Phase 3c, false otherwise
  • GATE — true/false/null per the rationale gate semantics

Acting on the directive

Behavior matrix:

VERDICT == "HOLD"

Behavior depends on ${CLAUDE_PANEL:-on}:

Default (on) — auto-take mode:

The recommendation stands. Take the recommended option without asking the user. Print a brief message:

Panel validated <recommended_label> for "<question excerpt>". <SUMMARY> Proceeding.

Then continue the work as if the user had picked the recommended option. The AskUserQuestion for this question is NEVER issued.

advise — advisory mode (preserves user agency):

Do NOT auto-take. Re-issue AskUserQuestion for this question with the panel's affirmative annotation appended. Augmented payload:

  • Question text: original question + two newlines + **Panel validated:** <SUMMARY> (positive phrasing).
  • Options: identical, EXCEPT the recommended option's label has (Recommended) swapped for (Recommended; Panel-flagged). The hook detects this marker and skips the panel on the re-ask.

VERDICT == "SOFT-DISSENT"

Re-issue AskUserQuestion for this question. Augmented payload:

  • Question text: original + two newlines + <SUMMARY> (the **Panel review:** line). Use it verbatim.
  • Options: identical, EXCEPT the recommended option's label is swapped to (Recommended; Panel-flagged).

VERDICT == "HARD-DISSENT"

In Phase 3c, HARD-DISSENT also escalates to the user (Phase 5 will add re-think cycles). Same payload as SOFT-DISSENT, but include severity in the augmented note:

  • Question text: original + two newlines + Panel HARD-DISSENT: <SUMMARY> (severity-clear prefix).
  • Options: identical, EXCEPT marker swap to (Recommended; Panel-flagged).

When Phase 5 lands, this branch will check re_brainstorm in the directive and emit a re-think markdown directive (no AskUserQuestion) when present. Until then, every HARD-DISSENT escalates the question to the user with full panel feedback.

VERDICT == "ERROR"

Something went wrong (panelist backend down, malformed responses, missing files, aggregator parse error). For this question, re-issue the original AskUserQuestion unmodified — same options, same (Recommended) marker — BUT swap the marker to (Recommended; Panel-flagged) for loop safety.

Print a brief explanation:

Panel evaluation failed for "<question excerpt>" (see hook trace for detail). Asking the question directly.

Cleanup

After processing ALL questions (HOLD + DISSENT + ERROR paths), remove every file this run created — one explicit path per rm -f argument, never a glob (*). Under zsh, an unmatched glob aborts the ENTIRE command line, not just that argument — so a single rm -f a b* c where b* matches nothing removes NEITHER a NOR c either. This is the root cause of the leak this section exists to close: a bare panel-prompt-*-q*.txt glob that matched nothing silently left $STATE_FILE on disk too, and the hook's re-entry guard then bypassed the NEXT, unrelated paneled question it saw.

rm -rf "$WORKDIR"
rm -f "$STATE_FILE"

Then remove each question payload and each per-panelist prompt file you actually wrote, one rm -f per exact path — using the same 0-based question indices you dispatched in the per-question loop above, crossed with the panelist ids you enumerated (you know both sets; do not glob -q*.json or -q*-*.txt):

rm -f "${TMPDIR:-/tmp}/panel-question-${CLAUDE_SESSION_ID:-$PPID}-q0.json"
rm -f "${TMPDIR:-/tmp}/panel-prompt-${CLAUDE_SESSION_ID:-$PPID}-q0-da.txt"
rm -f "${TMPDIR:-/tmp}/panel-prompt-${CLAUDE_SESSION_ID:-$PPID}-q0-pe.txt"
rm -f "${TMPDIR:-/tmp}/panel-prompt-${CLAUDE_SESSION_ID:-$PPID}-q0-qa.txt"
# ... then the same four lines for q1, q2, ... — one payload plus one
# prompt file per enabled panelist, for every question index you actually
# dispatched. Use the ids from YOUR enumeration, not this example's:
# solo profile writes only the da file, and a config with other panelists
# writes theirs. Skip indices that were never a (Recommended) question —
# there is no file for them, and rm -f on a missing path is a no-op.
#
# ${CLAUDE_SESSION_ID:-$PPID} must match the fallback used when the files
# were built. A bare ${CLAUDE_SESSION_ID} resolves to an empty string in a
# session where it is unset, and removes a path nothing ever wrote.

If step 2a resolved a solo profile, $CONFIG_RESOLVED holds the exact resolved-config path already — remove that path directly (it is a single concrete path, not a glob):

if [ -n "${CONFIG_RESOLVED:-}" ]; then
    rm -f "$CONFIG_RESOLVED"
fi

If you crash before cleanup, stale files are harmless PROVIDED CLAUDE_SESSION_ID is unique per session (different session = different path). The hook's re-entry guard on $STATE_FILE is now stale-aware — age >= 600s discards the leftover and replaces it with a fresh trigger instead of bypassing indefinitely — so a leftover $STATE_FILE from a crashed run self-heals within 10 minutes even if this cleanup never runs. panel gc will eventually reap stale workdirs (Phase 6).

Failure modes you must handle gracefully

What goes wrong Behavior
State file missing Print fallback message; re-issue original AskUserQuestion.
panel lint-config fails Print "Panel disabled: config invalid"; re-issue original.
~/.claude/panel/config.yml missing Caught by lint-config; same fallback.
Persona file missing for an ENABLED panelist Caught up front by lint-config's persona/blind cross-check; same fallback as an invalid config. A disabled panelist's persona is never loaded.
panel dispatch crashes (caller-bug exit 1) Verdict file not written. Aggregator coerces to ERROR for that panelist. Severity decides per failure_mode.
panel dispatch exits 0 but writes ERROR verdict Normal path. Severity decides per failure_mode.
Agent tool call errors / returns garbled output Write verbatim to verdict file; aggregator coerces to ERROR for that panelist when its expected line is missing — VERDICT: unblinded, CHOICE: blinded.
panel aggregate crashes (non-zero exit) Fall back. Re-issue original with marker swap.
$WORKDIR unwritable Fall back. Print "Panel infrastructure unavailable".
Missing API keys (e.g., $PANEL_DA_API_KEY) dispatch.py writes ERROR verdict; aggregator emits ERROR directive (at N=1) or degrades (at N>=3 with graceful failure_mode). User-facing message ends up as re-ask original.

The whole panel is best-effort. The user-visible question ALWAYS survives.

Loop safety on fallback

When you fall back to re-issuing the original AskUserQuestion, the original payload still has (Recommended) — meaning the hook will fire AGAIN. The hook has a re-entry guard: if a state file for the current session already exists when the hook runs, and it is fresh (age < 600s — always true for an immediate fallback re-issue), the hook removes it and approves (exit 0) without dispatching the skill again. So your fallback re-issue is mechanically safe.

As a courtesy to future readers, swap the marker to (Recommended; Panel-flagged) in your fallback re-issue (it documents intent even though the re-entry guard makes it a no-op for safety).

Multi-question parallelism (optional optimization)

If the AskUserQuestion call had multiple questions each with a (Recommended) marker, you MAY dispatch panel calls for ALL questions in parallel: fan out the N panelists × M questions in one message. Aggregate runs per question after both panelists return.

For v1, sequential per-question dispatch is fine. Optimize only if multi-question recommendations become common (they aren't currently).

What you must NOT do

  • Do NOT echo $PANEL_DA_API_KEY (or any other API key) in any user-facing text or trace output.
  • Do NOT write API keys to any file.
  • Do NOT modify or summarize the Agent tool's response before writing it to the verdict file. Write verbatim; let the aggregator parse.
  • Do NOT modify the AskUserQuestion call beyond augmenting question text (for dissents) or swapping the marker (for ERROR or fallback).
  • Do NOT re-invoke this skill after auto-taking a HOLD recommendation. The work continues as if the user picked.
  • Do NOT introduce a panelist not in config.yml. New panelists ship via config.yml + a persona file under personas/<role>.md — never ad-hoc in this file.
  • Do NOT call python3.12 -m panel ... for JSON parsing. Use jq. Python invocation is reserved for lint-config, build-prompt, dispatch, and aggregate.
  • Do NOT bypass the hook re-entry guard by clearing the state file yourself. The guard exists to break loops on fallback paths.

04 同仓库更多 Skills

ArangoGutierrez
ArangoGutierrez/claude-toolkit

update-cursor-settings

Modify Cursor/VSCode user settings in settings.json. Use when you want to change editor settings, preferences, configuration, themes, font size, tab size, format on save, auto save, keybindings, or any settings.json values.

★ 4
ArangoGutierrez
ArangoGutierrez/claude-toolkit

update-cli-config

View and modify Cursor CLI configuration settings in ~/.cursor/cli-config.json. Use when the user wants to change CLI settings, configure permissions, switch approval mode, enable vim mode, toggle display options, configure sandbox, or manage any CLI preferences.

★ 4
ArangoGutierrez
ArangoGutierrez/claude-toolkit

statusline

Configure a custom status line in the CLI. Use when the user mentions status line, statusline, statusLine, CLI status bar, prompt footer customization, or wants to add session context above the prompt.

★ 4
ArangoGutierrez
ArangoGutierrez/claude-toolkit

split-to-prs

Split current work into small reviewable PRs. Use when the user asks to split a chat, set of changes, branch, or PR.

★ 4
ArangoGutierrez
ArangoGutierrez/claude-toolkit

shell

Runs the rest of a /shell request as a literal shell command. Use only when the user explicitly invokes /shell and wants the following text executed directly in the terminal.

★ 4
ArangoGutierrez
ArangoGutierrez/claude-toolkit

sdk

Guide users building apps, scripts, CI pipelines, or automations on top of the Cursor SDK - TypeScript (`@cursor/sdk`) or Python (`cursor-sdk` / `cursor_sdk`). Use when the user mentions integrating, installing, or writing code against the Cursor SDK; says `Agent.create`, `Agent.prompt`, `Agent.resume`, `agent.send`, `run.stream`, `run.messages`, `CursorAgentError`, `@cursor/sdk`, `cursor-sdk`, or `cursor_sdk`; asks to run Cursor agents programmatically from a script, CI/CD pipeline, GitHub Action, backend service, or other code outside the Cursor IDE; wants to pick between local and cloud runtime, configure MCP servers for an SDK agent, or handle streaming, cancellation, or errors; or is wiring Cursor into an automation, bot, or REST `/v1/agents` migration. Use eagerly rather than answering from memory; the SDK surface evolves and this skill is the source of truth for the external packages.

★ 4