[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M730F3ATEKKF3JP6P958FMHN":3,"skill-same:01M730F3ATEKKF3JP6P958FMHN":49},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":26,"commitSha":27,"skillDirectory":28,"forksCount":29,"repositoryHtmlUrl":30,"repositoryHomepageUrl":6,"ownerHtmlUrl":31,"ownerBlogUrl":32,"ownerDescription":33,"updatedAt":34,"install":35,"resources":48},"01M730F3ATEKKF3JP6P958FMHN","eval-harness",null,"Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles","SriramGandhiS\u002FSkills","SriramGandhiS","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F211849986?v=4","01M72WQCW12BYX71C6WN584TTA","Sriram S","01M72WQCWTJ2RD0ET1X31AF1RC",1,"MIT","https:\u002F\u002Fgithub.com\u002FSriramGandhiS\u002FSkills\u002Fblob\u002Fmaster\u002Fskills\u002Feverything-claude-code\u002Fskills\u002Feval-harness\u002FSKILL.md",[],[],"2026-11-11 15:11:14",false,"---\nname: eval-harness\ndescription: Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles\norigin: ECC\ntools: Read, Write, Edit, Bash, Grep, Glob\n---\n\n# Eval Harness Skill\n\nA formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.\n\n## When to Activate\n\n- Setting up eval-driven development (EDD) for AI-assisted workflows\n- Defining pass\u002Ffail criteria for Claude Code task completion\n- Measuring agent reliability with pass@k metrics\n- Creating regression test suites for prompt or agent changes\n- Benchmarking agent performance across model versions\n\n## Philosophy\n\nEval-Driven Development treats evals as the \"unit tests of AI development\":\n- Define expected behavior BEFORE implementation\n- Run evals continuously during development\n- Track regressions with each change\n- Use pass@k metrics for reliability measurement\n\n## Eval Types\n\n### Capability Evals\nTest if Claude can do something it couldn't before:\n```markdown\n[CAPABILITY EVAL: feature-name]\nTask: Description of what Claude should accomplish\nSuccess Criteria:\n  - [ ] Criterion 1\n  - [ ] Criterion 2\n  - [ ] Criterion 3\nExpected Output: Description of expected result\n```\n\n### Regression Evals\nEnsure changes don't break existing functionality:\n```markdown\n[REGRESSION EVAL: feature-name]\nBaseline: SHA or checkpoint name\nTests:\n  - existing-test-1: PASS\u002FFAIL\n  - existing-test-2: PASS\u002FFAIL\n  - existing-test-3: PASS\u002FFAIL\nResult: X\u002FY passed (previously Y\u002FY)\n```\n\n## Grader Types\n\n### 1. Code-Based Grader\nDeterministic checks using code:\n```bash\n# Check if file contains expected pattern\ngrep -q \"export function handleAuth\" src\u002Fauth.ts && echo \"PASS\" || echo \"FAIL\"\n\n# Check if tests pass\nnpm test -- --testPathPattern=\"auth\" && echo \"PASS\" || echo \"FAIL\"\n\n# Check if build succeeds\nnpm run build && echo \"PASS\" || echo \"FAIL\"\n```\n\n### 2. Model-Based Grader\nUse Claude to evaluate open-ended outputs:\n```markdown\n[MODEL GRADER PROMPT]\nEvaluate the following code change:\n1. Does it solve the stated problem?\n2. Is it well-structured?\n3. Are edge cases handled?\n4. Is error handling appropriate?\n\nScore: 1-5 (1=poor, 5=excellent)\nReasoning: [explanation]\n```\n\n### 3. Human Grader\nFlag for manual review:\n```markdown\n[HUMAN REVIEW REQUIRED]\nChange: Description of what changed\nReason: Why human review is needed\nRisk Level: LOW\u002FMEDIUM\u002FHIGH\n```\n\n## Metrics\n\n### pass@k\n\"At least one success in k attempts\"\n- pass@1: First attempt success rate\n- pass@3: Success within 3 attempts\n- Typical target: pass@3 > 90%\n\n### pass^k\n\"All k trials succeed\"\n- Higher bar for reliability\n- pass^3: 3 consecutive successes\n- Use for critical paths\n\n## Eval Workflow\n\n### 1. Define (Before Coding)\n```markdown\n## EVAL DEFINITION: feature-xyz\n\n### Capability Evals\n1. Can create new user account\n2. Can validate email format\n3. Can hash password securely\n\n### Regression Evals\n1. Existing login still works\n2. Session management unchanged\n3. Logout flow intact\n\n### Success Metrics\n- pass@3 > 90% for capability evals\n- pass^3 = 100% for regression evals\n```\n\n### 2. Implement\nWrite code to pass the defined evals.\n\n### 3. Evaluate\n```bash\n# Run capability evals\n[Run each capability eval, record PASS\u002FFAIL]\n\n# Run regression evals\nnpm test -- --testPathPattern=\"existing\"\n\n# Generate report\n```\n\n### 4. Report\n```markdown\nEVAL REPORT: feature-xyz\n========================\n\nCapability Evals:\n  create-user:     PASS (pass@1)\n  validate-email:  PASS (pass@2)\n  hash-password:   PASS (pass@1)\n  Overall:         3\u002F3 passed\n\nRegression Evals:\n  login-flow:      PASS\n  session-mgmt:    PASS\n  logout-flow:     PASS\n  Overall:         3\u002F3 passed\n\nMetrics:\n  pass@1: 67% (2\u002F3)\n  pass@3: 100% (3\u002F3)\n\nStatus: READY FOR REVIEW\n```\n\n## Integration Patterns\n\n### Pre-Implementation\n```\n\u002Feval define feature-name\n```\nCreates eval definition file at `.claude\u002Fevals\u002Ffeature-name.md`\n\n### During Implementation\n```\n\u002Feval check feature-name\n```\nRuns current evals and reports status\n\n### Post-Implementation\n```\n\u002Feval report feature-name\n```\nGenerates full eval report\n\n## Eval Storage\n\nStore evals in project:\n```\n.claude\u002F\n  evals\u002F\n    feature-xyz.md      # Eval definition\n    feature-xyz.log     # Eval run history\n    baseline.json       # Regression baselines\n```\n\n## Best Practices\n\n1. **Define evals BEFORE coding** - Forces clear thinking about success criteria\n2. **Run evals frequently** - Catch regressions early\n3. **Track pass@k over time** - Monitor reliability trends\n4. **Use code graders when possible** - Deterministic > probabilistic\n5. **Human review for security** - Never fully automate security checks\n6. **Keep evals fast** - Slow evals don't get run\n7. **Version evals with code** - Evals are first-class artifacts\n\n## Example: Adding Authentication\n\n```markdown\n## EVAL: add-authentication\n\n### Phase 1: Define (10 min)\nCapability Evals:\n- [ ] User can register with email\u002Fpassword\n- [ ] User can login with valid credentials\n- [ ] Invalid credentials rejected with proper error\n- [ ] Sessions persist across page reloads\n- [ ] Logout clears session\n\nRegression Evals:\n- [ ] Public routes still accessible\n- [ ] API responses unchanged\n- [ ] Database schema compatible\n\n### Phase 2: Implement (varies)\n[Write code]\n\n### Phase 3: Evaluate\nRun: \u002Feval check add-authentication\n\n### Phase 4: Report\nEVAL REPORT: add-authentication\n==============================\nCapability: 5\u002F5 passed (pass@3: 100%)\nRegression: 3\u002F3 passed (pass^3: 100%)\nStatus: SHIP IT\n```\n\n## Product Evals (v1.8)\n\nUse product evals when behavior quality cannot be captured by unit tests alone.\n\n### Grader Types\n\n1. Code grader (deterministic assertions)\n2. Rule grader (regex\u002Fschema constraints)\n3. Model grader (LLM-as-judge rubric)\n4. Human grader (manual adjudication for ambiguous outputs)\n\n### pass@k Guidance\n\n- `pass@1`: direct reliability\n- `pass@3`: practical reliability under controlled retries\n- `pass^3`: stability test (all 3 runs must pass)\n\nRecommended thresholds:\n- Capability evals: pass@3 >= 0.90\n- Regression evals: pass^3 = 1.00 for release-critical paths\n\n### Eval Anti-Patterns\n\n- Overfitting prompts to known eval examples\n- Measuring only happy-path outputs\n- Ignoring cost and latency drift while chasing pass rates\n- Allowing flaky graders in release gates\n\n### Minimal Eval Artifact Layout\n\n- `.claude\u002Fevals\u002F\u003Cfeature>.md` definition\n- `.claude\u002Fevals\u002F\u003Cfeature>.log` run history\n- `docs\u002Freleases\u002F\u003Cversion>\u002Feval-summary.md` release snapshot\n","# Eval Harness Skill\n\nA formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.\n\n## When to Activate\n\n- Setting up eval-driven development (EDD) for AI-assisted workflows\n- Defining pass\u002Ffail criteria for Claude Code task completion\n- Measuring agent reliability with pass@k metrics\n- Creating regression test suites for prompt or agent changes\n- Benchmarking agent performance across model versions\n\n## Philosophy\n\nEval-Driven Development treats evals as the \"unit tests of AI development\":\n- Define expected behavior BEFORE implementation\n- Run evals continuously during development\n- Track regressions with each change\n- Use pass@k metrics for reliability measurement\n\n## Eval Types\n\n### Capability Evals\nTest if Claude can do something it couldn't before:\n```markdown\n[CAPABILITY EVAL: feature-name]\nTask: Description of what Claude should accomplish\nSuccess Criteria:\n  - [ ] Criterion 1\n  - [ ] Criterion 2\n  - [ ] Criterion 3\nExpected Output: Description of expected result\n```\n\n### Regression Evals\nEnsure changes don't break existing functionality:\n```markdown\n[REGRESSION EVAL: feature-name]\nBaseline: SHA or checkpoint name\nTests:\n  - existing-test-1: PASS\u002FFAIL\n  - existing-test-2: PASS\u002FFAIL\n  - existing-test-3: PASS\u002FFAIL\nResult: X\u002FY passed (previously Y\u002FY)\n```\n\n## Grader Types\n\n### 1. Code-Based Grader\nDeterministic checks using code:\n```bash\n# Check if file contains expected pattern\ngrep -q \"export function handleAuth\" src\u002Fauth.ts && echo \"PASS\" || echo \"FAIL\"\n\n# Check if tests pass\nnpm test -- --testPathPattern=\"auth\" && echo \"PASS\" || echo \"FAIL\"\n\n# Check if build succeeds\nnpm run build && echo \"PASS\" || echo \"FAIL\"\n```\n\n### 2. Model-Based Grader\nUse Claude to evaluate open-ended outputs:\n```markdown\n[MODEL GRADER PROMPT]\nEvaluate the following code change:\n1. Does it solve the stated problem?\n2. Is it well-structured?\n3. Are edge cases handled?\n4. Is error handling appropriate?\n\nScore: 1-5 (1=poor, 5=excellent)\nReasoning: [explanation]\n```\n\n### 3. Human Grader\nFlag for manual review:\n```markdown\n[HUMAN REVIEW REQUIRED]\nChange: Description of what changed\nReason: Why human review is needed\nRisk Level: LOW\u002FMEDIUM\u002FHIGH\n```\n\n## Metrics\n\n### pass@k\n\"At least one success in k attempts\"\n- pass@1: First attempt success rate\n- pass@3: Success within 3 attempts\n- Typical target: pass@3 > 90%\n\n### pass^k\n\"All k trials succeed\"\n- Higher bar for reliability\n- pass^3: 3 consecutive successes\n- Use for critical paths\n\n## Eval Workflow\n\n### 1. Define (Before Coding)\n```markdown\n## EVAL DEFINITION: feature-xyz\n\n### Capability Evals\n1. Can create new user account\n2. Can validate email format\n3. Can hash password securely\n\n### Regression Evals\n1. Existing login still works\n2. Session management unchanged\n3. Logout flow intact\n\n### Success Metrics\n- pass@3 > 90% for capability evals\n- pass^3 = 100% for regression evals\n```\n\n### 2. Implement\nWrite code to pass the defined evals.\n\n### 3. Evaluate\n```bash\n# Run capability evals\n[Run each capability eval, record PASS\u002FFAIL]\n\n# Run regression evals\nnpm test -- --testPathPattern=\"existing\"\n\n# Generate report\n```\n\n### 4. Report\n```markdown\nEVAL REPORT: feature-xyz\n========================\n\nCapability Evals:\n  create-user:     PASS (pass@1)\n  validate-email:  PASS (pass@2)\n  hash-password:   PASS (pass@1)\n  Overall:         3\u002F3 passed\n\nRegression Evals:\n  login-flow:      PASS\n  session-mgmt:    PASS\n  logout-flow:     PASS\n  Overall:         3\u002F3 passed\n\nMetrics:\n  pass@1: 67% (2\u002F3)\n  pass@3: 100% (3\u002F3)\n\nStatus: READY FOR REVIEW\n```\n\n## Integration Patterns\n\n### Pre-Implementation\n```\n\u002Feval define feature-name\n```\nCreates eval definition file at `.claude\u002Fevals\u002Ffeature-name.md`\n\n### During Implementation\n```\n\u002Feval check feature-name\n```\nRuns current evals and reports status\n\n### Post-Implementation\n```\n\u002Feval report feature-name\n```\nGenerates full eval report\n\n## Eval Storage\n\nStore evals in project:\n```\n.claude\u002F\n  evals\u002F\n    feature-xyz.md      # Eval definition\n    feature-xyz.log     # Eval run history\n    baseline.json       # Regression baselines\n```\n\n## Best Practices\n\n1. **Define evals BEFORE coding** - Forces clear thinking about success criteria\n2. **Run evals frequently** - Catch regressions early\n3. **Track pass@k over time** - Monitor reliability trends\n4. **Use code graders when possible** - Deterministic > probabilistic\n5. **Human review for security** - Never fully automate security checks\n6. **Keep evals fast** - Slow evals don't get run\n7. **Version evals with code** - Evals are first-class artifacts\n\n## Example: Adding Authentication\n\n```markdown\n## EVAL: add-authentication\n\n### Phase 1: Define (10 min)\nCapability Evals:\n- [ ] User can register with email\u002Fpassword\n- [ ] User can login with valid credentials\n- [ ] Invalid credentials rejected with proper error\n- [ ] Sessions persist across page reloads\n- [ ] Logout clears session\n\nRegression Evals:\n- [ ] Public routes still accessible\n- [ ] API responses unchanged\n- [ ] Database schema compatible\n\n### Phase 2: Implement (varies)\n[Write code]\n\n### Phase 3: Evaluate\nRun: \u002Feval check add-authentication\n\n### Phase 4: Report\nEVAL REPORT: add-authentication\n==============================\nCapability: 5\u002F5 passed (pass@3: 100%)\nRegression: 3\u002F3 passed (pass^3: 100%)\nStatus: SHIP IT\n```\n\n## Product Evals (v1.8)\n\nUse product evals when behavior quality cannot be captured by unit tests alone.\n\n### Grader Types\n\n1. Code grader (deterministic assertions)\n2. Rule grader (regex\u002Fschema constraints)\n3. Model grader (LLM-as-judge rubric)\n4. Human grader (manual adjudication for ambiguous outputs)\n\n### pass@k Guidance\n\n- `pass@1`: direct reliability\n- `pass@3`: practical reliability under controlled retries\n- `pass^3`: stability test (all 3 runs must pass)\n\nRecommended thresholds:\n- Capability evals: pass@3 >= 0.90\n- Regression evals: pass^3 = 1.00 for release-critical paths\n\n### Eval Anti-Patterns\n\n- Overfitting prompts to known eval examples\n- Measuring only happy-path outputs\n- Ignoring cost and latency drift while chasing pass rates\n- Allowing flaky graders in release gates\n\n### Minimal Eval Artifact Layout\n\n- `.claude\u002Fevals\u002F\u003Cfeature>.md` definition\n- `.claude\u002Fevals\u002F\u003Cfeature>.log` run history\n- `docs\u002Freleases\u002F\u003Cversion>\u002Feval-summary.md` release snapshot\n",{"name":5,"tools":24,"origin":25,"description":7},"Read, Write, Edit, Bash, Grep, Glob","ECC","master","d6ca01da0f24a3b0e26e4a8b7b66cbd25d429f5d","skills\u002Feverything-claude-code\u002Fskills\u002Feval-harness",0,"https:\u002F\u002Fgithub.com\u002FSriramGandhiS\u002FSkills","https:\u002F\u002Fgithub.com\u002FSriramGandhiS","https:\u002F\u002Fsriram.website\u002F","Curious guy who keeps tinkering until something works","2026-05-20 16:21:34",{"source":8,"skillName":5,"commands":36,"prompt":46,"zipAvailable":47},[37,40,43],{"tool":38,"command":39},"npx","npx skills add SriramGandhiS\u002FSkills --skill eval-harness",{"tool":41,"command":42},"bunx","bunx skills add SriramGandhiS\u002FSkills --skill eval-harness",{"tool":44,"command":45},"pnpm","pnpm dlx skills add SriramGandhiS\u002FSkills --skill eval-harness","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FSriramGandhiS\u002FSkills\u002Fblob\u002Fmaster\u002Fskills\u002Feverything-claude-code\u002Fskills\u002Feval-harness\u002FSKILL.md",true,[],[50,57,62,68,75,82],{"skillNo":51,"name":52,"nameZh":6,"description":53,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":54,"categories":55,"keywords":56,"publishedAt":19},"01M732609EPJWV13PVTX3GT4TM","webgpu-threejs-tsl","Comprehensive guide for developing WebGPU-enabled Three.js applications using TSL (Three.js Shading Language). Covers WebGPU renderer setup, TSL syntax and node materials, compute shaders, post-processing effects, and WGSL integration. Use this skill when working with Three.js WebGPU, TSL shaders, node materials, or GPU compute in Three.js.","https:\u002F\u002Fgithub.com\u002FSriramGandhiS\u002FSkills\u002Fblob\u002Fmaster\u002Fwebgpu-threejs-tsl\u002Fskills\u002Fwebgpu-threejs-tsl\u002FSKILL.md",[],[],{"skillNo":58,"name":52,"nameZh":6,"description":53,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":59,"categories":60,"keywords":61,"publishedAt":19},"01M7325XM47AGWB6K6C4VXKE6B","https:\u002F\u002Fgithub.com\u002FSriramGandhiS\u002FSkills\u002Fblob\u002Fmaster\u002Fwebgpu-threejs-tsl\u002FSKILL.md",[],[],{"skillNo":63,"name":64,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":65,"categories":66,"keywords":67,"publishedAt":19},"01M7325WP6DES69JKC840VMFEP","verification-loop","https:\u002F\u002Fgithub.com\u002FSriramGandhiS\u002FSkills\u002Fblob\u002Fmaster\u002Fweb-assets\u002Fdocs\u002Fzh-TW\u002Fskills\u002Fverification-loop\u002FSKILL.md",[],[],{"skillNo":69,"name":70,"nameZh":6,"description":71,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":72,"categories":73,"keywords":74,"publishedAt":19},"01M7325TFQF1DZTQ6J3WGQQ1VR","tdd-workflow","Use this skill when writing new features, fixing bugs, or refactoring code. Enforces test-driven development with 80%+ coverage including unit, integration, and E2E tests.","https:\u002F\u002Fgithub.com\u002FSriramGandhiS\u002FSkills\u002Fblob\u002Fmaster\u002Fweb-assets\u002Fdocs\u002Fzh-TW\u002Fskills\u002Ftdd-workflow\u002FSKILL.md",[],[],{"skillNo":76,"name":77,"nameZh":6,"description":78,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":79,"categories":80,"keywords":81,"publishedAt":19},"01M7325RGV9AY9PK92D2V8KKPW","strategic-compact","Suggests manual context compaction at logical intervals to preserve context through task phases rather than arbitrary auto-compaction.","https:\u002F\u002Fgithub.com\u002FSriramGandhiS\u002FSkills\u002Fblob\u002Fmaster\u002Fweb-assets\u002Fdocs\u002Fzh-TW\u002Fskills\u002Fstrategic-compact\u002FSKILL.md",[],[],{"skillNo":83,"name":84,"nameZh":6,"description":85,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":86,"categories":87,"keywords":88,"publishedAt":19},"01M7325MZC28SE86E7NMCNEFS4","security-review","Use this skill when adding authentication, handling user input, working with secrets, creating API endpoints, or implementing payment\u002Fsensitive features. Provides comprehensive security checklist and patterns.","https:\u002F\u002Fgithub.com\u002FSriramGandhiS\u002FSkills\u002Fblob\u002Fmaster\u002Fweb-assets\u002Fdocs\u002Fzh-TW\u002Fskills\u002Fsecurity-review\u002FSKILL.md",[],[]]