[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M71HWS5MKV1EV232RAPCWGND":3,"skill-same:01M71HWS5MKV1EV232RAPCWGND":72},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":44,"linkOnly":45,"markdownContent":46,"markdownBody":47,"frontMatter":48,"refName":49,"commitSha":50,"skillDirectory":51,"forksCount":52,"repositoryHtmlUrl":53,"repositoryHomepageUrl":54,"ownerHtmlUrl":55,"ownerBlogUrl":56,"ownerDescription":6,"updatedAt":57,"install":58,"resources":71},"01M71HWS5MKV1EV232RAPCWGND","run-evals",null,"Eval-Driven Development — define capability and regression evals before building; code graders use verify commands, model graders use explicit rubrics; log pass@k. Use before develop-tdd on new features, or when measuring agent capability over runs.","danielvm-git\u002Fbigpowers","danielvm-git","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F1020864?v=4","01M71H1N1CTQ529S0KN7KETVG6","Daniel V M","01M71H1N218Y3S8WHCJ58NPRSW",290,"MIT","https:\u002F\u002Fgithub.com\u002Fdanielvm-git\u002Fbigpowers\u002Fblob\u002Fmain\u002F.copilot\u002Fskills\u002Frun-evals\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000030","开发工程师","PROFESSION","适用职业",[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43],"run","evals","run evals","develop","use","define","before building","development","capability","sonnet","Pass@k","commands","features","log","code","when","new","eval","regression","model","2026-11-10 10:12:30",false,"---\nname: run-evals\nmodel: sonnet\ndescription: \"Eval-Driven Development — define capability and regression evals before building; code graders use verify commands, model graders use explicit rubrics; log pass@k. Use before develop-tdd on new features, or when measuring agent capability over runs.\"\n---\n\n# Run Evals\n\n> **HARD GATE** — Define evals before implementation. Code graders = runnable `verify:` commands; model graders = explicit rubric with pass\u002Ffail criteria.\n\n## Process\n\n1. Name the capability under test (one sentence).\n2. Write `specs\u002FEVALS-\u003Cfeature>.md` with:\n   - **Capability evals** (does it do the job?)\n   - **Regression evals** (did we break anything?)\n3. Assign grader type per eval: `code` (shell verify) or `model` (rubric).\n4. Assign **strictness tier** per eval (graduated promotion — e45s37):\n\n   | Tier | Meaning | Promotion rule |\n   |------|---------|------------------|\n   | `EXPERIMENTAL` | New eval, may flake | Not gating |\n   | `USUALLY_PASSES` | Stable in dev; ≥2\u002F3 recent runs pass | Blocks BUILD only when combined with ALWAYS_PASSES suite |\n   | `ALWAYS_PASSES` | Zero tolerance; required for release | Any single failure blocks BUILD and merge |\n\n   Promote: `EXPERIMENTAL → USUALLY_PASSES` after 3 consecutive passes; `USUALLY_PASSES → ALWAYS_PASSES` after 5 consecutive passes with zero flakes documented in `specs\u002Fstate.yaml`.\n\n5. Run evals; log results table with pass@k (e.g. 3\u002F3 runs) and tier per eval.\n6. Block BUILD phase until all `ALWAYS_PASSES` evals pass at agreed k. `USUALLY_PASSES` failures warn; `EXPERIMENTAL` failures log only.\n\n## Artefact\n\n`specs\u002Fverifications\u002FeNNsYY-eval-report.md` — see [REFERENCE.md](REFERENCE.md) for template. Eval reports are stored alongside verification evidence in `specs\u002Fverifications\u002F`, keyed by story ID for traceability.\n\n## Verify\n\n→ verify: `test -d specs\u002Fbenchmarks && test -f specs\u002Fbenchmarks\u002FSCHEMA.md`\n\n\n\u003C!-- story: e02s01 -->\n\n---\n\n# Run Evals — Reference\n\n## Strictness tiers (e45s37)\n\nAdd a `tier:` column to each eval row:\n\n| Tier | Gate behaviour |\n|------|----------------|\n| `EXPERIMENTAL` | Log only — does not block |\n| `USUALLY_PASSES` | Warn on failure; blocks only when paired with failing `ALWAYS_PASSES` |\n| `ALWAYS_PASSES` | Hard block on any failure |\n\n## EVALS template\n\n```markdown\n# EVALS: \u003Cfeature>\n\n## Capability\n| ID | Eval | Grader | Tier | verify \u002F rubric |\n|----|------|--------|------|-----------------|\n| C1 | ... | code | ALWAYS_PASSES | `verify: npm test -- \u003Cfile>` |\n| C2 | ... | model | USUALLY_PASSES | Rubric: [ ] criterion A [ ] criterion B |\n\n## Regression\n| ID | Eval | Grader | verify \u002F rubric |\n|----|------|--------|-----------------|\n| R1 | Full suite passes | code | `verify: npm test` |\n\n## Results\n| Run | C1 | C2 | R1 | pass@k |\n|-----|----|----|-----|--------|\n| 1 | PASS | PASS | PASS | 3\u002F3 |\n```\n\n## pass@k\n\nRun capability evals k times (default k=3). Ship when all k pass or document known flake in `specs\u002Fstate.yaml` `handoff.open_decisions`.\n","# Run Evals\n\n> **HARD GATE** — Define evals before implementation. Code graders = runnable `verify:` commands; model graders = explicit rubric with pass\u002Ffail criteria.\n\n## Process\n\n1. Name the capability under test (one sentence).\n2. Write `specs\u002FEVALS-\u003Cfeature>.md` with:\n   - **Capability evals** (does it do the job?)\n   - **Regression evals** (did we break anything?)\n3. Assign grader type per eval: `code` (shell verify) or `model` (rubric).\n4. Assign **strictness tier** per eval (graduated promotion — e45s37):\n\n   | Tier | Meaning | Promotion rule |\n   |------|---------|------------------|\n   | `EXPERIMENTAL` | New eval, may flake | Not gating |\n   | `USUALLY_PASSES` | Stable in dev; ≥2\u002F3 recent runs pass | Blocks BUILD only when combined with ALWAYS_PASSES suite |\n   | `ALWAYS_PASSES` | Zero tolerance; required for release | Any single failure blocks BUILD and merge |\n\n   Promote: `EXPERIMENTAL → USUALLY_PASSES` after 3 consecutive passes; `USUALLY_PASSES → ALWAYS_PASSES` after 5 consecutive passes with zero flakes documented in `specs\u002Fstate.yaml`.\n\n5. Run evals; log results table with pass@k (e.g. 3\u002F3 runs) and tier per eval.\n6. Block BUILD phase until all `ALWAYS_PASSES` evals pass at agreed k. `USUALLY_PASSES` failures warn; `EXPERIMENTAL` failures log only.\n\n## Artefact\n\n`specs\u002Fverifications\u002FeNNsYY-eval-report.md` — see [REFERENCE.md](REFERENCE.md) for template. Eval reports are stored alongside verification evidence in `specs\u002Fverifications\u002F`, keyed by story ID for traceability.\n\n## Verify\n\n→ verify: `test -d specs\u002Fbenchmarks && test -f specs\u002Fbenchmarks\u002FSCHEMA.md`\n\n\n\u003C!-- story: e02s01 -->\n\n---\n\n# Run Evals — Reference\n\n## Strictness tiers (e45s37)\n\nAdd a `tier:` column to each eval row:\n\n| Tier | Gate behaviour |\n|------|----------------|\n| `EXPERIMENTAL` | Log only — does not block |\n| `USUALLY_PASSES` | Warn on failure; blocks only when paired with failing `ALWAYS_PASSES` |\n| `ALWAYS_PASSES` | Hard block on any failure |\n\n## EVALS template\n\n```markdown\n# EVALS: \u003Cfeature>\n\n## Capability\n| ID | Eval | Grader | Tier | verify \u002F rubric |\n|----|------|--------|------|-----------------|\n| C1 | ... | code | ALWAYS_PASSES | `verify: npm test -- \u003Cfile>` |\n| C2 | ... | model | USUALLY_PASSES | Rubric: [ ] criterion A [ ] criterion B |\n\n## Regression\n| ID | Eval | Grader | verify \u002F rubric |\n|----|------|--------|-----------------|\n| R1 | Full suite passes | code | `verify: npm test` |\n\n## Results\n| Run | C1 | C2 | R1 | pass@k |\n|-----|----|----|-----|--------|\n| 1 | PASS | PASS | PASS | 3\u002F3 |\n```\n\n## pass@k\n\nRun capability evals k times (default k=3). Ship when all k pass or document known flake in `specs\u002Fstate.yaml` `handoff.open_decisions`.\n",{"name":5,"model":33,"description":7},"main","72f5e4c8b43f60cf86b37078ad8cd7e22daf67aa",".copilot\u002Fskills\u002Frun-evals",24,"https:\u002F\u002Fgithub.com\u002Fdanielvm-git\u002Fbigpowers","https:\u002F\u002Fdanielvm-git.github.io\u002Fbigpowers\u002F","https:\u002F\u002Fgithub.com\u002Fdanielvm-git","danielvm.net","2026-11-10 01:18:54",{"source":8,"skillName":5,"commands":59,"prompt":69,"zipAvailable":70},[60,63,66],{"tool":61,"command":62},"npx","npx skills add danielvm-git\u002Fbigpowers --skill run-evals",{"tool":64,"command":65},"bunx","bunx skills add danielvm-git\u002Fbigpowers --skill run-evals",{"tool":67,"command":68},"pnpm","pnpm dlx skills add danielvm-git\u002Fbigpowers --skill run-evals","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fdanielvm-git\u002Fbigpowers\u002Fblob\u002Fmain\u002F.copilot\u002Fskills\u002Frun-evals\u002FSKILL.md",true,[],[73,93,100,127,148,169],{"skillNo":74,"name":75,"nameZh":6,"description":76,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":77,"categories":78,"keywords":84,"publishedAt":44},"01M71KBG9F8X644MJB16H4V8M7","skill-verify-fail-open-fixture","Negative-path fixture for run-skill-verify.sh — must never PASS.","https:\u002F\u002Fgithub.com\u002Fdanielvm-git\u002Fbigpowers\u002Fblob\u002Fmain\u002Fspecs\u002Fverifications\u002Ffixtures\u002Fskill-verify-fail-open\u002FSKILL.md",[79],{"categoryNo":80,"categoryName":81,"taxonomyType":82,"taxonomyTypeDesc":83},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[85,86,87,88,89,90,24,91,92],"fixture","open","verify","skill verify fail open fixture","skill verify","path","for","never",{"skillNo":94,"name":95,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":96,"categories":97,"keywords":98,"publishedAt":44},"01M71KBE3BKEMKNE0C5DGVDWZH","negative-path","https:\u002F\u002Fgithub.com\u002Fdanielvm-git\u002Fbigpowers\u002Fblob\u002Fmain\u002Fspecs\u002Fverifications\u002Ffixtures\u002Fnegative-path\u002FSKILL.md",[],[99,90],"negative path",{"skillNo":101,"name":102,"nameZh":6,"description":103,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":104,"categories":105,"keywords":109,"publishedAt":44},"01M71KBBGF7Y4EB4AMHBZ8CJFC","write-document","Write, organize, and sync high-integrity technical documents using the BMAD methodology. Ensures every document is Bold, Minimal, Actionable, and Durable. Use when creating architectural docs, technical guides, or organizing the specs\u002F directory.","https:\u002F\u002Fgithub.com\u002Fdanielvm-git\u002Fbigpowers\u002Fblob\u002Fmain\u002Fskills\u002Fwrite-document\u002FSKILL.md",[106],{"categoryNo":107,"categoryName":108,"taxonomyType":82,"taxonomyTypeDesc":83},"01JBSEEDCAT000000000000002","文档处理",[110,111,112,113,114,115,116,117,118,119,28,39,120,121,33,122,123,124,125,126],"write","write document","document","documents","actionable","technical","methodology","high","bmad","standard","minimal","bold","integrity","organize","sync","directory","guides",{"skillNo":128,"name":129,"nameZh":6,"description":130,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":131,"categories":132,"keywords":133,"publishedAt":44},"01M71KB8TM089JJD5AY5KAFSDM","wire-observability","Add structured JSON logging, observability commands, and idempotent setup scripts to a project. Use when a project needs production-readiness instrumentation, when user wants structured logging, or as a production-readiness gate at any phase of development.","https:\u002F\u002Fgithub.com\u002Fdanielvm-git\u002Fbigpowers\u002Fblob\u002Fmain\u002Fskills\u002Fwire-observability\u002FSKILL.md",[],[134,135,136,119,35,28,31,137,138,139,33,140,141,142,143,144,145,39,146,147],"observability","wire observability","readiness","as","logging","add","instrumentation","at","structured logging","production-readiness","idempotent","phase","project","scripts",{"skillNo":149,"name":150,"nameZh":6,"description":151,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":152,"categories":153,"keywords":154,"publishedAt":44},"01M71KB6CNRANP449DQA3NJZ96","wire-ci","CI pipeline setup with bundled, forge-neutral templates and local validation. Detects the forge from the git remote, generates workflows for supported forges, and skips honestly for the rest. The CI equivalent of wire-observability.","https:\u002F\u002Fgithub.com\u002Fdanielvm-git\u002Fbigpowers\u002Fblob\u002Fmain\u002Fskills\u002Fwire-ci\u002FSKILL.md",[],[155,156,157,158,159,160,161,162,163,164,165,119,166,91,134,33,167,168],"wire ci","ci","remote","local","pipeline","forge","validation","setup","templates","REST","with","ci pipeline setup","git","workflows",{"skillNo":170,"name":171,"nameZh":6,"description":172,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":173,"categories":174,"keywords":182,"publishedAt":44},"01M71KB2ZV98P03X88QR8MT9F1","visual-dashboard","Start a browser-based dashboard that visualizes architecture, implementation plans, and project status. Persists artifacts in .bigpowers\u002Fdashboard\u002F. Reads specs\u002Fstate.yaml, release-plan.yaml, epics, and planning-status via HTTP API or opencode panel.","https:\u002F\u002Fgithub.com\u002Fdanielvm-git\u002Fbigpowers\u002Fblob\u002Fmain\u002Fskills\u002Fvisual-dashboard\u002FSKILL.md",[175,178,179],{"categoryNo":176,"categoryName":177,"taxonomyType":82,"taxonomyTypeDesc":83},"01JBSEEDCAT000000000000013","分析与可视化",{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":180,"categoryName":181,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000033","数据分析师",[183,184,185,186,187,188,189,190,191,33,192,193,194,195,196,197,198,146,199,200],"dashboard","visual","visual dashboard","status","artifacts","http","plans","planning","panel","start","release","opencode","plan","browser","api","architecture","project status","http api"]