[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M799FXA2NC21YK89NE8PPSAX":3,"skill-same:01M799FXA2NC21YK89NE8PPSAX":47},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":27,"commitSha":28,"skillDirectory":29,"forksCount":14,"repositoryHtmlUrl":30,"repositoryHomepageUrl":6,"ownerHtmlUrl":31,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":32,"install":33,"resources":46},"01M799FXA2NC21YK89NE8PPSAX","eval-harness",null,"Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles","pjherron\u002Fhypoc","pjherron","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F800763?v=4","01M799C7XC0XKKW2D6RMYNHCY8","Patrick Herron","01M799C7XZPGZ9QZ7XAQN5MQAN",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fpjherron\u002Fhypoc\u002Fblob\u002Fmain\u002Fhypoc\u002Fskills\u002Feval-harness\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: eval-harness\ndescription: Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles\nmetadata:\n  origin: ECC\ntools: Read, Write, Edit, Bash, Grep, Glob\n---\n\n# Eval Harness Skill\n\nA formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.\n\n## When to Activate\n\n- Setting up eval-driven development (EDD) for AI-assisted workflows\n- Defining pass\u002Ffail criteria for Claude Code task completion\n- Measuring agent reliability with pass@k metrics\n- Creating regression test suites for prompt or agent changes\n- Benchmarking agent performance across model versions\n\n## Philosophy\n\nEval-Driven Development treats evals as the \"unit tests of AI development\":\n- Define expected behavior BEFORE implementation\n- Run evals continuously during development\n- Track regressions with each change\n- Use pass@k metrics for reliability measurement\n\n## Eval Types\n\n### Capability Evals\nTest if Claude can do something it couldn't before:\n```markdown\n[CAPABILITY EVAL: feature-name]\nTask: Description of what Claude should accomplish\nSuccess Criteria:\n  - [ ] Criterion 1\n  - [ ] Criterion 2\n  - [ ] Criterion 3\nExpected Output: Description of expected result\n```\n\n### Regression Evals\nEnsure changes don't break existing functionality:\n```markdown\n[REGRESSION EVAL: feature-name]\nBaseline: SHA or checkpoint name\nTests:\n  - existing-test-1: PASS\u002FFAIL\n  - existing-test-2: PASS\u002FFAIL\n  - existing-test-3: PASS\u002FFAIL\nResult: X\u002FY passed (previously Y\u002FY)\n```\n\n## Grader Types\n\n### 1. Code-Based Grader\nDeterministic checks using code:\n```bash\n# Check if file contains expected pattern\ngrep -q \"export function handleAuth\" src\u002Fauth.ts && echo \"PASS\" || echo \"FAIL\"\n\n# Check if tests pass\nnpm test -- --testPathPattern=\"auth\" && echo \"PASS\" || echo \"FAIL\"\n\n# Check if build succeeds\nnpm run build && echo \"PASS\" || echo \"FAIL\"\n```\n\n### 2. Model-Based Grader\nUse Claude to evaluate open-ended outputs:\n```markdown\n[MODEL GRADER PROMPT]\nEvaluate the following code change:\n1. Does it solve the stated problem?\n2. Is it well-structured?\n3. Are edge cases handled?\n4. Is error handling appropriate?\n\nScore: 1-5 (1=poor, 5=excellent)\nReasoning: [explanation]\n```\n\n### 3. Human Grader\nFlag for manual review:\n```markdown\n[HUMAN REVIEW REQUIRED]\nChange: Description of what changed\nReason: Why human review is needed\nRisk Level: LOW\u002FMEDIUM\u002FHIGH\n```\n\n## Metrics\n\n### pass@k\n\"At least one success in k attempts\"\n- pass@1: First attempt success rate\n- pass@3: Success within 3 attempts\n- Typical target: pass@3 > 90%\n\n### pass^k\n\"All k trials succeed\"\n- Higher bar for reliability\n- pass^3: 3 consecutive successes\n- Use for critical paths\n\n## Eval Workflow\n\n### 1. Define (Before Coding)\n```markdown\n## EVAL DEFINITION: feature-xyz\n\n### Capability Evals\n1. Can create new user account\n2. Can validate email format\n3. Can hash password securely\n\n### Regression Evals\n1. Existing login still works\n2. Session management unchanged\n3. Logout flow intact\n\n### Success Metrics\n- pass@3 > 90% for capability evals\n- pass^3 = 100% for regression evals\n```\n\n### 2. Implement\nWrite code to pass the defined evals.\n\n### 3. Evaluate\n```bash\n# Run capability evals\n[Run each capability eval, record PASS\u002FFAIL]\n\n# Run regression evals\nnpm test -- --testPathPattern=\"existing\"\n\n# Generate report\n```\n\n### 4. Report\n```markdown\nEVAL REPORT: feature-xyz\n========================\n\nCapability Evals:\n  create-user:     PASS (pass@1)\n  validate-email:  PASS (pass@2)\n  hash-password:   PASS (pass@1)\n  Overall:         3\u002F3 passed\n\nRegression Evals:\n  login-flow:      PASS\n  session-mgmt:    PASS\n  logout-flow:     PASS\n  Overall:         3\u002F3 passed\n\nMetrics:\n  pass@1: 67% (2\u002F3)\n  pass@3: 100% (3\u002F3)\n\nStatus: READY FOR REVIEW\n```\n\n## Integration Patterns\n\n### Pre-Implementation\n```\n\u002Feval define feature-name\n```\nCreates eval definition file at `.claude\u002Fevals\u002Ffeature-name.md`\n\n### During Implementation\n```\n\u002Feval check feature-name\n```\nRuns current evals and reports status\n\n### Post-Implementation\n```\n\u002Feval report feature-name\n```\nGenerates full eval report\n\n## Eval Storage\n\nStore evals in project:\n```\n.claude\u002F\n  evals\u002F\n    feature-xyz.md      # Eval definition\n    feature-xyz.log     # Eval run history\n    baseline.json       # Regression baselines\n```\n\n## Best Practices\n\n1. **Define evals BEFORE coding** - Forces clear thinking about success criteria\n2. **Run evals frequently** - Catch regressions early\n3. **Track pass@k over time** - Monitor reliability trends\n4. **Use code graders when possible** - Deterministic > probabilistic\n5. **Human review for security** - Never fully automate security checks\n6. **Keep evals fast** - Slow evals don't get run\n7. **Version evals with code** - Evals are first-class artifacts\n\n## Example: Adding Authentication\n\n```markdown\n## EVAL: add-authentication\n\n### Phase 1: Define (10 min)\nCapability Evals:\n- [ ] User can register with email\u002Fpassword\n- [ ] User can login with valid credentials\n- [ ] Invalid credentials rejected with proper error\n- [ ] Sessions persist across page reloads\n- [ ] Logout clears session\n\nRegression Evals:\n- [ ] Public routes still accessible\n- [ ] API responses unchanged\n- [ ] Database schema compatible\n\n### Phase 2: Implement (varies)\n[Write code]\n\n### Phase 3: Evaluate\nRun: \u002Feval check add-authentication\n\n### Phase 4: Report\nEVAL REPORT: add-authentication\n==============================\nCapability: 5\u002F5 passed (pass@3: 100%)\nRegression: 3\u002F3 passed (pass^3: 100%)\nStatus: SHIP IT\n```\n\n## Product Evals (v1.8)\n\nUse product evals when behavior quality cannot be captured by unit tests alone.\n\n### Grader Types\n\n1. Code grader (deterministic assertions)\n2. Rule grader (regex\u002Fschema constraints)\n3. Model grader (LLM-as-judge rubric)\n4. Human grader (manual adjudication for ambiguous outputs)\n\n### pass@k Guidance\n\n- `pass@1`: direct reliability\n- `pass@3`: practical reliability under controlled retries\n- `pass^3`: stability test (all 3 runs must pass)\n\nRecommended thresholds:\n- Capability evals: pass@3 >= 0.90\n- Regression evals: pass^3 = 1.00 for release-critical paths\n\n### Eval Anti-Patterns\n\n- Overfitting prompts to known eval examples\n- Measuring only happy-path outputs\n- Ignoring cost and latency drift while chasing pass rates\n- Allowing flaky graders in release gates\n\n### Minimal Eval Artifact Layout\n\n- `.claude\u002Fevals\u002F\u003Cfeature>.md` definition\n- `.claude\u002Fevals\u002F\u003Cfeature>.log` run history\n- `docs\u002Freleases\u002F\u003Cversion>\u002Feval-summary.md` release snapshot\n","# Eval Harness Skill\n\nA formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.\n\n## When to Activate\n\n- Setting up eval-driven development (EDD) for AI-assisted workflows\n- Defining pass\u002Ffail criteria for Claude Code task completion\n- Measuring agent reliability with pass@k metrics\n- Creating regression test suites for prompt or agent changes\n- Benchmarking agent performance across model versions\n\n## Philosophy\n\nEval-Driven Development treats evals as the \"unit tests of AI development\":\n- Define expected behavior BEFORE implementation\n- Run evals continuously during development\n- Track regressions with each change\n- Use pass@k metrics for reliability measurement\n\n## Eval Types\n\n### Capability Evals\nTest if Claude can do something it couldn't before:\n```markdown\n[CAPABILITY EVAL: feature-name]\nTask: Description of what Claude should accomplish\nSuccess Criteria:\n  - [ ] Criterion 1\n  - [ ] Criterion 2\n  - [ ] Criterion 3\nExpected Output: Description of expected result\n```\n\n### Regression Evals\nEnsure changes don't break existing functionality:\n```markdown\n[REGRESSION EVAL: feature-name]\nBaseline: SHA or checkpoint name\nTests:\n  - existing-test-1: PASS\u002FFAIL\n  - existing-test-2: PASS\u002FFAIL\n  - existing-test-3: PASS\u002FFAIL\nResult: X\u002FY passed (previously Y\u002FY)\n```\n\n## Grader Types\n\n### 1. Code-Based Grader\nDeterministic checks using code:\n```bash\n# Check if file contains expected pattern\ngrep -q \"export function handleAuth\" src\u002Fauth.ts && echo \"PASS\" || echo \"FAIL\"\n\n# Check if tests pass\nnpm test -- --testPathPattern=\"auth\" && echo \"PASS\" || echo \"FAIL\"\n\n# Check if build succeeds\nnpm run build && echo \"PASS\" || echo \"FAIL\"\n```\n\n### 2. Model-Based Grader\nUse Claude to evaluate open-ended outputs:\n```markdown\n[MODEL GRADER PROMPT]\nEvaluate the following code change:\n1. Does it solve the stated problem?\n2. Is it well-structured?\n3. Are edge cases handled?\n4. Is error handling appropriate?\n\nScore: 1-5 (1=poor, 5=excellent)\nReasoning: [explanation]\n```\n\n### 3. Human Grader\nFlag for manual review:\n```markdown\n[HUMAN REVIEW REQUIRED]\nChange: Description of what changed\nReason: Why human review is needed\nRisk Level: LOW\u002FMEDIUM\u002FHIGH\n```\n\n## Metrics\n\n### pass@k\n\"At least one success in k attempts\"\n- pass@1: First attempt success rate\n- pass@3: Success within 3 attempts\n- Typical target: pass@3 > 90%\n\n### pass^k\n\"All k trials succeed\"\n- Higher bar for reliability\n- pass^3: 3 consecutive successes\n- Use for critical paths\n\n## Eval Workflow\n\n### 1. Define (Before Coding)\n```markdown\n## EVAL DEFINITION: feature-xyz\n\n### Capability Evals\n1. Can create new user account\n2. Can validate email format\n3. Can hash password securely\n\n### Regression Evals\n1. Existing login still works\n2. Session management unchanged\n3. Logout flow intact\n\n### Success Metrics\n- pass@3 > 90% for capability evals\n- pass^3 = 100% for regression evals\n```\n\n### 2. Implement\nWrite code to pass the defined evals.\n\n### 3. Evaluate\n```bash\n# Run capability evals\n[Run each capability eval, record PASS\u002FFAIL]\n\n# Run regression evals\nnpm test -- --testPathPattern=\"existing\"\n\n# Generate report\n```\n\n### 4. Report\n```markdown\nEVAL REPORT: feature-xyz\n========================\n\nCapability Evals:\n  create-user:     PASS (pass@1)\n  validate-email:  PASS (pass@2)\n  hash-password:   PASS (pass@1)\n  Overall:         3\u002F3 passed\n\nRegression Evals:\n  login-flow:      PASS\n  session-mgmt:    PASS\n  logout-flow:     PASS\n  Overall:         3\u002F3 passed\n\nMetrics:\n  pass@1: 67% (2\u002F3)\n  pass@3: 100% (3\u002F3)\n\nStatus: READY FOR REVIEW\n```\n\n## Integration Patterns\n\n### Pre-Implementation\n```\n\u002Feval define feature-name\n```\nCreates eval definition file at `.claude\u002Fevals\u002Ffeature-name.md`\n\n### During Implementation\n```\n\u002Feval check feature-name\n```\nRuns current evals and reports status\n\n### Post-Implementation\n```\n\u002Feval report feature-name\n```\nGenerates full eval report\n\n## Eval Storage\n\nStore evals in project:\n```\n.claude\u002F\n  evals\u002F\n    feature-xyz.md      # Eval definition\n    feature-xyz.log     # Eval run history\n    baseline.json       # Regression baselines\n```\n\n## Best Practices\n\n1. **Define evals BEFORE coding** - Forces clear thinking about success criteria\n2. **Run evals frequently** - Catch regressions early\n3. **Track pass@k over time** - Monitor reliability trends\n4. **Use code graders when possible** - Deterministic > probabilistic\n5. **Human review for security** - Never fully automate security checks\n6. **Keep evals fast** - Slow evals don't get run\n7. **Version evals with code** - Evals are first-class artifacts\n\n## Example: Adding Authentication\n\n```markdown\n## EVAL: add-authentication\n\n### Phase 1: Define (10 min)\nCapability Evals:\n- [ ] User can register with email\u002Fpassword\n- [ ] User can login with valid credentials\n- [ ] Invalid credentials rejected with proper error\n- [ ] Sessions persist across page reloads\n- [ ] Logout clears session\n\nRegression Evals:\n- [ ] Public routes still accessible\n- [ ] API responses unchanged\n- [ ] Database schema compatible\n\n### Phase 2: Implement (varies)\n[Write code]\n\n### Phase 3: Evaluate\nRun: \u002Feval check add-authentication\n\n### Phase 4: Report\nEVAL REPORT: add-authentication\n==============================\nCapability: 5\u002F5 passed (pass@3: 100%)\nRegression: 3\u002F3 passed (pass^3: 100%)\nStatus: SHIP IT\n```\n\n## Product Evals (v1.8)\n\nUse product evals when behavior quality cannot be captured by unit tests alone.\n\n### Grader Types\n\n1. Code grader (deterministic assertions)\n2. Rule grader (regex\u002Fschema constraints)\n3. Model grader (LLM-as-judge rubric)\n4. Human grader (manual adjudication for ambiguous outputs)\n\n### pass@k Guidance\n\n- `pass@1`: direct reliability\n- `pass@3`: practical reliability under controlled retries\n- `pass^3`: stability test (all 3 runs must pass)\n\nRecommended thresholds:\n- Capability evals: pass@3 >= 0.90\n- Regression evals: pass^3 = 1.00 for release-critical paths\n\n### Eval Anti-Patterns\n\n- Overfitting prompts to known eval examples\n- Measuring only happy-path outputs\n- Ignoring cost and latency drift while chasing pass rates\n- Allowing flaky graders in release gates\n\n### Minimal Eval Artifact Layout\n\n- `.claude\u002Fevals\u002F\u003Cfeature>.md` definition\n- `.claude\u002Fevals\u002F\u003Cfeature>.log` run history\n- `docs\u002Freleases\u002F\u003Cversion>\u002Feval-summary.md` release snapshot\n",{"name":5,"tools":24,"metadata":25,"description":7},"Read, Write, Edit, Bash, Grep, Glob",{"origin":26},"ECC","main","eebcd780d62a0710aa03028091dc1e9baa995b83","hypoc\u002Fskills\u002Feval-harness","https:\u002F\u002Fgithub.com\u002Fpjherron\u002Fhypoc","https:\u002F\u002Fgithub.com\u002Fpjherron","2026-09-01 06:45:11",{"source":8,"skillName":5,"commands":34,"prompt":44,"zipAvailable":45},[35,38,41],{"tool":36,"command":37},"npx","npx skills add pjherron\u002Fhypoc --skill eval-harness",{"tool":39,"command":40},"bunx","bunx skills add pjherron\u002Fhypoc --skill eval-harness",{"tool":42,"command":43},"pnpm","pnpm dlx skills add pjherron\u002Fhypoc --skill eval-harness","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fpjherron\u002Fhypoc\u002Fblob\u002Fmain\u002Fhypoc\u002Fskills\u002Feval-harness\u002FSKILL.md",true,[],[48,55,62,69,76,83],{"skillNo":49,"name":50,"nameZh":6,"description":51,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":52,"categories":53,"keywords":54,"publishedAt":19},"01M799MNB4RVHZZECC75ZFGH0Y","workspace-surface-audit","Audit the active repo, MCP servers, plugins, connectors, env surfaces, and harness setup, then recommend the highest-value ECC-native skills, hooks, agents, and operator workflows. Use when the user wants help setting up Claude Code or understanding what capabilities are actually available in their environment.","https:\u002F\u002Fgithub.com\u002Fpjherron\u002Fhypoc\u002Fblob\u002Fmain\u002Fhypoc\u002Fskills\u002Fworkspace-surface-audit\u002FSKILL.md",[],[],{"skillNo":56,"name":57,"nameZh":6,"description":58,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":59,"categories":60,"keywords":61,"publishedAt":19},"01M799MHR27W0YV1VMW26N691S","Verification & Quality Assurance","Comprehensive truth scoring, code quality verification, and automatic rollback system with 0.95 accuracy threshold for ensuring high-quality agent outputs and codebase reliability.","https:\u002F\u002Fgithub.com\u002Fpjherron\u002Fhypoc\u002Fblob\u002Fmain\u002Fhypoc\u002Fskills\u002Fverification-quality\u002FSKILL.md",[],[],{"skillNo":63,"name":64,"nameZh":6,"description":65,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":66,"categories":67,"keywords":68,"publishedAt":19},"01M799MEX0F1D253S3KZ1635PE","verification-loop","A comprehensive verification system for Claude Code sessions.","https:\u002F\u002Fgithub.com\u002Fpjherron\u002Fhypoc\u002Fblob\u002Fmain\u002Fhypoc\u002Fskills\u002Fverification-loop\u002FSKILL.md",[],[],{"skillNo":70,"name":71,"nameZh":6,"description":72,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":73,"categories":74,"keywords":75,"publishedAt":19},"01M799MCFN5N6A9VWM6R2TFJRK","triage","Move issues and external PRs through a state machine of triage roles — categorise, verify, grill if needed, and write agent-ready briefs.","https:\u002F\u002Fgithub.com\u002Fpjherron\u002Fhypoc\u002Fblob\u002Fmain\u002Fhypoc\u002Fskills\u002Ftriage\u002FSKILL.md",[],[],{"skillNo":77,"name":78,"nameZh":6,"description":79,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":80,"categories":81,"keywords":82,"publishedAt":19},"01M799M695F2XJBTP21YHN0R46","token-budget-advisor","Offers the user an informed choice about how much response depth to consume before answering. Use this skill when the user explicitly wants to control response length, depth, or token budget. TRIGGER when: \"token budget\", \"token count\", \"token usage\", \"token limit\", \"response length\", \"answer depth\", \"short version\", \"brief answer\", \"detailed answer\", \"exhaustive answer\", \"respuesta corta vs larga\", \"cuántos tokens\", \"ahorrar tokens\", \"responde al 50%\", \"dame la versión corta\", \"quiero controlar cuánto usas\", or clear variants where the user is explicitly asking to control answer size or depth. DO NOT TRIGGER when: user has already specified a level in the current session (maintain it), the request is clearly a one-word answer, or \"token\" refers to auth\u002Fsession\u002Fpayment tokens rather than response size.","https:\u002F\u002Fgithub.com\u002Fpjherron\u002Fhypoc\u002Fblob\u002Fmain\u002Fhypoc\u002Fskills\u002Ftoken-budget-advisor\u002FSKILL.md",[],[],{"skillNo":84,"name":85,"nameZh":6,"description":86,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":87,"categories":88,"keywords":89,"publishedAt":19},"01M799M387CZZVGC18PN0AZ70S","to-tickets","Break a plan, spec, or the current conversation into a set of tracer-bullet tickets, each declaring its blocking edges, published to the configured tracker — edges as text in a local file, or native blocking links on a real tracker.","https:\u002F\u002Fgithub.com\u002Fpjherron\u002Fhypoc\u002Fblob\u002Fmain\u002Fhypoc\u002Fskills\u002Fto-tickets\u002FSKILL.md",[],[]]