[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M71JB8Q5PPC92WN16YHAHRN6":3,"skill-same:01M71JB8Q5PPC92WN16YHAHRN6":72},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":44,"linkOnly":45,"markdownContent":46,"markdownBody":47,"frontMatter":48,"refName":49,"commitSha":50,"skillDirectory":51,"forksCount":52,"repositoryHtmlUrl":53,"repositoryHomepageUrl":54,"ownerHtmlUrl":55,"ownerBlogUrl":56,"ownerDescription":6,"updatedAt":57,"install":58,"resources":71},"01M71JB8Q5PPC92WN16YHAHRN6","run-benchmark",null,"Run skill quality benchmarks from specs\u002Fbenchmarks\u002F definitions — N-run with\u002Fwithout-skill delta grading, train\u002Fvalidation split, pass@k + benchmark.json reports. Use before and after evolve-skill to prove quality changes are improvements, not regressions.","danielvm-git\u002Fbigpowers","danielvm-git","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F1020864?v=4","01M71H1N1CTQ529S0KN7KETVG6","Daniel V M","01M71H1N218Y3S8WHCJ58NPRSW",290,"MIT","https:\u002F\u002Fgithub.com\u002Fdanielvm-git\u002Fbigpowers\u002Fblob\u002Fmain\u002F.opencode\u002Fskills\u002Frun-benchmark\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43],"benchmark","run benchmark","run","train","validation","quality","json","changes","with","benchmarks","split","use","prove","delta","before and after","grading","Pass@k","evolve","haiku","reports","2026-11-10 10:12:30",false,"---\nname: run-benchmark\nmodel: haiku\ndescription: \"Run skill quality benchmarks from specs\u002Fbenchmarks\u002F definitions — N-run with\u002Fwithout-skill delta grading, train\u002Fvalidation split, pass@k + benchmark.json reports. Use before and after evolve-skill to prove quality changes are improvements, not regressions.\"\n---\n\n# Run Benchmark\n\n> **HARD GATE** — Do NOT use benchmark scores to declare a skill \"good\" or \"bad\" in isolation. Benchmarks measure relative quality vs. a baseline — they catch regressions, they do not certify correctness.\n\nReads benchmark definitions from `specs\u002Fbenchmarks\u002F`, executes each scenario's grader with and without the skill loaded, and writes a structured `pass@k` report with delta grading that `evolve-skill` consumes.\n\n## With\u002FWithout-Skill Delta Grading\n\nEvery scenario runs N times (default 3) in two modes: **with** the skill loaded and **without** (bare agent with only CLAUDE.md). The delta `Δ = pass@k_with − pass@k_without` isolates the skill's causal contribution. A negative delta is a regression flag.\n\n## Train\u002FValidation Split\n\nBenchmark definitions partition scenarios into two sets:\n\n| Set | Tag | Purpose |\n|-----|-----|---------|\n| **Train** | `split: train` | Development scenarios — used while iterating. Hitting 100% on train is expected. |\n| **Validation** | `split: validation` | Held-out scenarios — the real quality signal. Overfitting train while validation stagnates is a design smell. |\n\n`pass@k` is reported separately for train and validation. Validation score is authoritative; train score is iteration guidance only.\n\n## Usage\n\n```bash\nbash scripts\u002Frun-benchmark.sh \u003Cskill-name>           # benchmark single skill\nbash scripts\u002Frun-benchmark.sh --all                  # benchmark all with definitions\nbash scripts\u002Frun-benchmark.sh \u003Cskill-name> --baseline # pin results as baseline\n```\n\n## Process\n\n1. **Locate definition** — Read `specs\u002Fbenchmarks\u002F\u003Cskill>.yaml`. If absent, stop with message.\n\n2. **Partition scenarios** — Split by `split` field (`train` → iteration, `validation` → authoritative, default: `validation`).\n\n3. **Run each scenario (N-run delta)** — For each scenario, run grader N times (default 3, configurable via `runs:`):\n   - **Without skill:** Agent with only CLAUDE.md\u002FCONVENTIONS.md\n   - **With skill:** Agent with the skill under test active\n   - Code grader: `bash -c \u003Ccommand>`, exit 0 → PASS. Timeout: 15s.\n   - Rubric grader: yes\u002Fno per criterion, ≥ 80% yes → PASS.\n   - Record: `{scenario_id: {with: [P\u002FF,...], without: [P\u002FF,...]}}`\n\n4. **Calculate scores** — Per split (train, validation) and mode (with, without):\n   - `pass@k = sum(weight × pass_rate) \u002F sum(weights)` where `pass_rate = passes\u002Fruns`\n   - `Δ = pass@k_with − pass@k_without` — causal contribution\n   - Round to 2 decimal places\n\n5. **Write benchmark.json** to `specs\u002Fbenchmarks\u002Freports\u002Fbenchmark-\u003Cskill>.json`:\n   ```json\n   {\"skill\":\"survey-context\",\"run_date\":\"2026-06-22\",\"runs_per_scenario\":3,\"train\":{\"with_skill\":0.92,\"without_skill\":0.67,\"delta\":0.25,\"scenarios\":[\"s01\",\"s02\"]},\"validation\":{\"with_skill\":0.83,\"without_skill\":0.60,\"delta\":0.23,\"scenarios\":[\"s03\",\"s04\",\"s05\"]}}\n   ```\n\n6. **Write YAML report** to `specs\u002Fbenchmarks\u002Freports\u002FBENCHMARK-\u003Cskill>-\u003CYYYY-MM-DD>.yaml`:\n   ```yaml\n   skill: survey-context\n   run_date: \"2026-06-22\"\n   runs_per_scenario: 3\n   train:\n     pass_at_k_with: 0.92\n     pass_at_k_without: 0.67\n     delta: 0.25\n   validation:\n     pass_at_k_with: 0.83\n     pass_at_k_without: 0.60\n     delta: 0.23\n   scenarios:\n     - id: s01\n       split: train\n       with_pass_rate: 1.0\n       without_pass_rate: 0.67\n       delta: 0.33\n       weight: 1.0\n   ```\n\n7. **Baseline** (`--baseline`) — Copy to `BASELINE-\u003Cskill>.yaml` + `baseline-\u003Cskill>.json`.\n\n8. **Compare to baseline** — `IMPROVED: Δ 0.17 → 0.25` \u002F `REGRESSION: Δ 0.25 → 0.17 — do NOT ship` \u002F `STABLE`.\n\n9. **Delta threshold gate** — Validation Δ \u003C 0.0 blocks release. Δ \u003C 0.05 warns (marginal). Min meaningful threshold: 0.05.\n","# Run Benchmark\n\n> **HARD GATE** — Do NOT use benchmark scores to declare a skill \"good\" or \"bad\" in isolation. Benchmarks measure relative quality vs. a baseline — they catch regressions, they do not certify correctness.\n\nReads benchmark definitions from `specs\u002Fbenchmarks\u002F`, executes each scenario's grader with and without the skill loaded, and writes a structured `pass@k` report with delta grading that `evolve-skill` consumes.\n\n## With\u002FWithout-Skill Delta Grading\n\nEvery scenario runs N times (default 3) in two modes: **with** the skill loaded and **without** (bare agent with only CLAUDE.md). The delta `Δ = pass@k_with − pass@k_without` isolates the skill's causal contribution. A negative delta is a regression flag.\n\n## Train\u002FValidation Split\n\nBenchmark definitions partition scenarios into two sets:\n\n| Set | Tag | Purpose |\n|-----|-----|---------|\n| **Train** | `split: train` | Development scenarios — used while iterating. Hitting 100% on train is expected. |\n| **Validation** | `split: validation` | Held-out scenarios — the real quality signal. Overfitting train while validation stagnates is a design smell. |\n\n`pass@k` is reported separately for train and validation. Validation score is authoritative; train score is iteration guidance only.\n\n## Usage\n\n```bash\nbash scripts\u002Frun-benchmark.sh \u003Cskill-name>           # benchmark single skill\nbash scripts\u002Frun-benchmark.sh --all                  # benchmark all with definitions\nbash scripts\u002Frun-benchmark.sh \u003Cskill-name> --baseline # pin results as baseline\n```\n\n## Process\n\n1. **Locate definition** — Read `specs\u002Fbenchmarks\u002F\u003Cskill>.yaml`. If absent, stop with message.\n\n2. **Partition scenarios** — Split by `split` field (`train` → iteration, `validation` → authoritative, default: `validation`).\n\n3. **Run each scenario (N-run delta)** — For each scenario, run grader N times (default 3, configurable via `runs:`):\n   - **Without skill:** Agent with only CLAUDE.md\u002FCONVENTIONS.md\n   - **With skill:** Agent with the skill under test active\n   - Code grader: `bash -c \u003Ccommand>`, exit 0 → PASS. Timeout: 15s.\n   - Rubric grader: yes\u002Fno per criterion, ≥ 80% yes → PASS.\n   - Record: `{scenario_id: {with: [P\u002FF,...], without: [P\u002FF,...]}}`\n\n4. **Calculate scores** — Per split (train, validation) and mode (with, without):\n   - `pass@k = sum(weight × pass_rate) \u002F sum(weights)` where `pass_rate = passes\u002Fruns`\n   - `Δ = pass@k_with − pass@k_without` — causal contribution\n   - Round to 2 decimal places\n\n5. **Write benchmark.json** to `specs\u002Fbenchmarks\u002Freports\u002Fbenchmark-\u003Cskill>.json`:\n   ```json\n   {\"skill\":\"survey-context\",\"run_date\":\"2026-06-22\",\"runs_per_scenario\":3,\"train\":{\"with_skill\":0.92,\"without_skill\":0.67,\"delta\":0.25,\"scenarios\":[\"s01\",\"s02\"]},\"validation\":{\"with_skill\":0.83,\"without_skill\":0.60,\"delta\":0.23,\"scenarios\":[\"s03\",\"s04\",\"s05\"]}}\n   ```\n\n6. **Write YAML report** to `specs\u002Fbenchmarks\u002Freports\u002FBENCHMARK-\u003Cskill>-\u003CYYYY-MM-DD>.yaml`:\n   ```yaml\n   skill: survey-context\n   run_date: \"2026-06-22\"\n   runs_per_scenario: 3\n   train:\n     pass_at_k_with: 0.92\n     pass_at_k_without: 0.67\n     delta: 0.25\n   validation:\n     pass_at_k_with: 0.83\n     pass_at_k_without: 0.60\n     delta: 0.23\n   scenarios:\n     - id: s01\n       split: train\n       with_pass_rate: 1.0\n       without_pass_rate: 0.67\n       delta: 0.33\n       weight: 1.0\n   ```\n\n7. **Baseline** (`--baseline`) — Copy to `BASELINE-\u003Cskill>.yaml` + `baseline-\u003Cskill>.json`.\n\n8. **Compare to baseline** — `IMPROVED: Δ 0.17 → 0.25` \u002F `REGRESSION: Δ 0.25 → 0.17 — do NOT ship` \u002F `STABLE`.\n\n9. **Delta threshold gate** — Validation Δ \u003C 0.0 blocks release. Δ \u003C 0.05 warns (marginal). Min meaningful threshold: 0.05.\n",{"name":5,"model":42,"description":7},"main","72f5e4c8b43f60cf86b37078ad8cd7e22daf67aa",".opencode\u002Fskills\u002Frun-benchmark",24,"https:\u002F\u002Fgithub.com\u002Fdanielvm-git\u002Fbigpowers","https:\u002F\u002Fdanielvm-git.github.io\u002Fbigpowers\u002F","https:\u002F\u002Fgithub.com\u002Fdanielvm-git","danielvm.net","2026-11-10 01:18:54",{"source":8,"skillName":5,"commands":59,"prompt":69,"zipAvailable":70},[60,63,66],{"tool":61,"command":62},"npx","npx skills add danielvm-git\u002Fbigpowers --skill run-benchmark",{"tool":64,"command":65},"bunx","bunx skills add danielvm-git\u002Fbigpowers --skill run-benchmark",{"tool":67,"command":68},"pnpm","pnpm dlx skills add danielvm-git\u002Fbigpowers --skill run-benchmark","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fdanielvm-git\u002Fbigpowers\u002Fblob\u002Fmain\u002F.opencode\u002Fskills\u002Frun-benchmark\u002FSKILL.md",true,[],[73,89,96,125,148,167],{"skillNo":74,"name":75,"nameZh":6,"description":76,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":77,"categories":78,"keywords":80,"publishedAt":44},"01M71KBG9F8X644MJB16H4V8M7","skill-verify-fail-open-fixture","Negative-path fixture for run-skill-verify.sh — must never PASS.","https:\u002F\u002Fgithub.com\u002Fdanielvm-git\u002Fbigpowers\u002Fblob\u002Fmain\u002Fspecs\u002Fverifications\u002Ffixtures\u002Fskill-verify-fail-open\u002FSKILL.md",[79],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[81,82,83,84,85,86,26,87,88],"fixture","open","verify","skill verify fail open fixture","skill verify","path","for","never",{"skillNo":90,"name":91,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":92,"categories":93,"keywords":94,"publishedAt":44},"01M71KBE3BKEMKNE0C5DGVDWZH","negative-path","https:\u002F\u002Fgithub.com\u002Fdanielvm-git\u002Fbigpowers\u002Fblob\u002Fmain\u002Fspecs\u002Fverifications\u002Ffixtures\u002Fnegative-path\u002FSKILL.md",[],[95,86],"negative path",{"skillNo":97,"name":98,"nameZh":6,"description":99,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":100,"categories":101,"keywords":105,"publishedAt":44},"01M71KBBGF7Y4EB4AMHBZ8CJFC","write-document","Write, organize, and sync high-integrity technical documents using the BMAD methodology. Ensures every document is Bold, Minimal, Actionable, and Durable. Use when creating architectural docs, technical guides, or organizing the specs\u002F directory.","https:\u002F\u002Fgithub.com\u002Fdanielvm-git\u002Fbigpowers\u002Fblob\u002Fmain\u002Fskills\u002Fwrite-document\u002FSKILL.md",[102],{"categoryNo":103,"categoryName":104,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000002","文档处理",[106,107,108,109,110,111,112,113,114,115,35,116,117,118,119,120,121,122,123,124],"write","write document","document","documents","actionable","technical","methodology","high","bmad","standard","when","minimal","bold","sonnet","integrity","organize","sync","directory","guides",{"skillNo":126,"name":127,"nameZh":6,"description":128,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":129,"categories":130,"keywords":131,"publishedAt":44},"01M71KB8TM089JJD5AY5KAFSDM","wire-observability","Add structured JSON logging, observability commands, and idempotent setup scripts to a project. Use when a project needs production-readiness instrumentation, when user wants structured logging, or as a production-readiness gate at any phase of development.","https:\u002F\u002Fgithub.com\u002Fdanielvm-git\u002Fbigpowers\u002Fblob\u002Fmain\u002Fskills\u002Fwire-observability\u002FSKILL.md",[],[132,133,134,115,135,35,136,137,138,139,119,140,141,142,143,144,145,116,146,147],"observability","wire observability","readiness","commands","development","as","logging","add","instrumentation","at","structured logging","production-readiness","idempotent","phase","project","scripts",{"skillNo":149,"name":150,"nameZh":6,"description":151,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":152,"categories":153,"keywords":154,"publishedAt":44},"01M71KB6CNRANP449DQA3NJZ96","wire-ci","CI pipeline setup with bundled, forge-neutral templates and local validation. Detects the forge from the git remote, generates workflows for supported forges, and skips honestly for the rest. The CI equivalent of wire-observability.","https:\u002F\u002Fgithub.com\u002Fdanielvm-git\u002Fbigpowers\u002Fblob\u002Fmain\u002Fskills\u002Fwire-ci\u002FSKILL.md",[],[155,156,157,158,159,160,28,161,162,163,32,115,164,87,132,119,165,166],"wire ci","ci","remote","local","pipeline","forge","setup","templates","REST","ci pipeline setup","git","workflows",{"skillNo":168,"name":169,"nameZh":6,"description":170,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":171,"categories":172,"keywords":184,"publishedAt":44},"01M71KB2ZV98P03X88QR8MT9F1","visual-dashboard","Start a browser-based dashboard that visualizes architecture, implementation plans, and project status. Persists artifacts in .bigpowers\u002Fdashboard\u002F. Reads specs\u002Fstate.yaml, release-plan.yaml, epics, and planning-status via HTTP API or opencode panel.","https:\u002F\u002Fgithub.com\u002Fdanielvm-git\u002Fbigpowers\u002Fblob\u002Fmain\u002Fskills\u002Fvisual-dashboard\u002FSKILL.md",[173,176,181],{"categoryNo":174,"categoryName":175,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000013","分析与可视化",{"categoryNo":177,"categoryName":178,"taxonomyType":179,"taxonomyTypeDesc":180},"01JBSEEDCAT000000000000030","开发工程师","PROFESSION","适用职业",{"categoryNo":182,"categoryName":183,"taxonomyType":179,"taxonomyTypeDesc":180},"01JBSEEDCAT000000000000033","数据分析师",[185,186,187,188,189,190,191,192,193,119,194,195,196,197,198,199,200,146,201,202],"dashboard","visual","visual dashboard","status","artifacts","http","plans","planning","panel","start","release","opencode","plan","browser","api","architecture","project status","http api"]