[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M75W7106F5PV7XT3719FGPRB":3,"skill-same:01M75W7106F5PV7XT3719FGPRB":47},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":24,"commitSha":25,"skillDirectory":26,"forksCount":27,"repositoryHtmlUrl":28,"repositoryHomepageUrl":6,"ownerHtmlUrl":29,"ownerBlogUrl":30,"ownerDescription":31,"updatedAt":32,"install":33,"resources":46},"01M75W7106F5PV7XT3719FGPRB","agent-evals",null,"Design evaluation cases and scoring criteria for an AI agent workflow. Includes benchmark analysis and tool-use auditing. Use when asked to create eval suites, benchmark tasks, pass\u002Ffail criteria, quality rubrics, interpret scores, analyze regressions, or audit agent tool usage. Do not use for end-user product analytics.","ItWasYou-SAM\u002FSkill-is-All-you-Need","ItWasYou-SAM","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F255364315?v=4","01M75VMN2DKTF5V9QPG2D5PBY0","Sam","01M75VMN31FCZMM9RZ4W8ZHCR4",6,"MIT","https:\u002F\u002Fgithub.com\u002FItWasYou-SAM\u002FSkill-is-All-you-Need\u002Fblob\u002Fmaster\u002F08-agent-skills\u002Fagent-evals\u002FSKILL.md",[],[],"2026-11-12 13:55:43",false,"---\nname: agent-evals\ndescription: Design evaluation cases and scoring criteria for an AI agent workflow. Includes benchmark analysis and tool-use auditing. Use when asked to create eval suites, benchmark tasks, pass\u002Ffail criteria, quality rubrics, interpret scores, analyze regressions, or audit agent tool usage. Do not use for end-user product analytics.\n---\n\n# Agent Evals\n\n## Goal\nDefine measurable tests for agent quality and reliability.\n\n## When to use\n- Creating evaluation suites for agent workflows\n- Defining pass\u002Ffail or scored criteria\n- Measuring agent reliability over time\n- Benchmarking against baseline performance\n- Identifying failure modes and edge cases\n\n## When not to use\n- End-user product analytics\n- A\u002FB testing for UX features\n- Infrastructure performance monitoring\n- Model fine-tuning decisions\n- Use benchmark-analysis for interpreting existing benchmark results\n\n## Process\n1. Identify agent responsibilities.\n2. Break evaluation into task categories and failure modes.\n3. Create representative test cases.\n4. Define pass\u002Ffail or scored criteria.\n5. Separate functional correctness from style and efficiency.\n6. Recommend baseline and stretch metrics.\n\n## Eval case template\n\n```yaml\neval_id: \"eval_001\"\ntask: \"Summarize this PR\"\ninput:\n  pr_url: \"https:\u002F\u002Fgithub.com\u002Forg\u002Frepo\u002Fpull\u002F123\"\n  context: \"User wants a 3-bullet summary\"\nexpected_output:\n  format: \"bullet list, 3 items\"\n  content_requirements:\n    - \"Mentions the main change\"\n    - \"Notes any breaking changes\"\n    - \"References related issues\"\nscoring:\n  format_compliance: 0-5\n  content_accuracy: 0-5\n  completeness: 0-5\n  conciseness: 0-5\n```\n\n## Success metrics\n\n| Metric | Definition | Target |\n|--------|------------|--------|\n| Task Completion | Did it complete the task? | >90% |\n| Format Compliance | Did it follow output format? | >95% |\n| Content Accuracy | Is the content correct? | >85% |\n| Edge Case Handling | Does it handle edge cases? | >80% |\n| Consistency | Same input → similar output | \u003C10% variance |\n\n## Failure-bucket schema\n\n```yaml\nfailure_buckets:\n  - bucket: \"format_violation\"\n    description: \"Output doesn't match expected format\"\n    examples:\n      - \"Expected JSON, got markdown\"\n      - \"Expected 3 bullets, got 5\"\n    fix: \"Add format enforcement to prompt\"\n\n  - bucket: \"content_hallucination\"\n    description: \"Output contains facts not in context\"\n    examples:\n      - \"Claims PR adds feature X when it doesn't\"\n      - \"Cites issue #456 when PR references #789\"\n    fix: \"Ground output in provided context\"\n\n  - bucket: \"incomplete\"\n    description: \"Output misses required elements\"\n    examples:\n      - \"Only 2 bullets when 3 required\"\n      - \"Doesn't mention breaking changes\"\n    fix: \"Add completeness checklist to prompt\"\n```\n\n## Benchmark analysis\n\nWhen analyzing benchmark or evaluation results:\n\n1. Identify what was measured and why.\n2. Compare baseline vs current results.\n3. Separate signal from noise.\n4. Identify regressions, improvements, and suspicious outliers.\n5. Explain likely causes.\n6. Recommend next experiments or fixes.\n\n### Analysis checklist\n- What was the measurement methodology?\n- Is the sample size sufficient?\n- Are the results statistically significant?\n- What changed between baseline and current?\n- Are there confounding variables?\n\n## Tool-use auditing\n\nWhen auditing agent tool usage:\n\n1. List tools available to the agent.\n2. Compare actual tool use with ideal tool use for the task.\n3. Identify redundant, missing, or unsafe calls.\n4. Check sequencing and fallback behavior.\n5. Recommend better decision rules and guardrails.\n\n### Audit checklist\n- Are all tool calls necessary?\n- Are there redundant calls?\n- Is the sequencing optimal?\n- Are fallbacks properly handled?\n- Are there unsafe actions?\n\n## Output format\n- Scope\n- Eval categories\n- Test cases\n- Scoring rubric\n- Success thresholds\n- Reporting format\n","# Agent Evals\n\n## Goal\nDefine measurable tests for agent quality and reliability.\n\n## When to use\n- Creating evaluation suites for agent workflows\n- Defining pass\u002Ffail or scored criteria\n- Measuring agent reliability over time\n- Benchmarking against baseline performance\n- Identifying failure modes and edge cases\n\n## When not to use\n- End-user product analytics\n- A\u002FB testing for UX features\n- Infrastructure performance monitoring\n- Model fine-tuning decisions\n- Use benchmark-analysis for interpreting existing benchmark results\n\n## Process\n1. Identify agent responsibilities.\n2. Break evaluation into task categories and failure modes.\n3. Create representative test cases.\n4. Define pass\u002Ffail or scored criteria.\n5. Separate functional correctness from style and efficiency.\n6. Recommend baseline and stretch metrics.\n\n## Eval case template\n\n```yaml\neval_id: \"eval_001\"\ntask: \"Summarize this PR\"\ninput:\n  pr_url: \"https:\u002F\u002Fgithub.com\u002Forg\u002Frepo\u002Fpull\u002F123\"\n  context: \"User wants a 3-bullet summary\"\nexpected_output:\n  format: \"bullet list, 3 items\"\n  content_requirements:\n    - \"Mentions the main change\"\n    - \"Notes any breaking changes\"\n    - \"References related issues\"\nscoring:\n  format_compliance: 0-5\n  content_accuracy: 0-5\n  completeness: 0-5\n  conciseness: 0-5\n```\n\n## Success metrics\n\n| Metric | Definition | Target |\n|--------|------------|--------|\n| Task Completion | Did it complete the task? | >90% |\n| Format Compliance | Did it follow output format? | >95% |\n| Content Accuracy | Is the content correct? | >85% |\n| Edge Case Handling | Does it handle edge cases? | >80% |\n| Consistency | Same input → similar output | \u003C10% variance |\n\n## Failure-bucket schema\n\n```yaml\nfailure_buckets:\n  - bucket: \"format_violation\"\n    description: \"Output doesn't match expected format\"\n    examples:\n      - \"Expected JSON, got markdown\"\n      - \"Expected 3 bullets, got 5\"\n    fix: \"Add format enforcement to prompt\"\n\n  - bucket: \"content_hallucination\"\n    description: \"Output contains facts not in context\"\n    examples:\n      - \"Claims PR adds feature X when it doesn't\"\n      - \"Cites issue #456 when PR references #789\"\n    fix: \"Ground output in provided context\"\n\n  - bucket: \"incomplete\"\n    description: \"Output misses required elements\"\n    examples:\n      - \"Only 2 bullets when 3 required\"\n      - \"Doesn't mention breaking changes\"\n    fix: \"Add completeness checklist to prompt\"\n```\n\n## Benchmark analysis\n\nWhen analyzing benchmark or evaluation results:\n\n1. Identify what was measured and why.\n2. Compare baseline vs current results.\n3. Separate signal from noise.\n4. Identify regressions, improvements, and suspicious outliers.\n5. Explain likely causes.\n6. Recommend next experiments or fixes.\n\n### Analysis checklist\n- What was the measurement methodology?\n- Is the sample size sufficient?\n- Are the results statistically significant?\n- What changed between baseline and current?\n- Are there confounding variables?\n\n## Tool-use auditing\n\nWhen auditing agent tool usage:\n\n1. List tools available to the agent.\n2. Compare actual tool use with ideal tool use for the task.\n3. Identify redundant, missing, or unsafe calls.\n4. Check sequencing and fallback behavior.\n5. Recommend better decision rules and guardrails.\n\n### Audit checklist\n- Are all tool calls necessary?\n- Are there redundant calls?\n- Is the sequencing optimal?\n- Are fallbacks properly handled?\n- Are there unsafe actions?\n\n## Output format\n- Scope\n- Eval categories\n- Test cases\n- Scoring rubric\n- Success thresholds\n- Reporting format\n",{"name":5,"description":7},"master","f0b9a39cdf4f3e791e3acb54c8740ca23f704176","08-agent-skills\u002Fagent-evals",0,"https:\u002F\u002Fgithub.com\u002FItWasYou-SAM\u002FSkill-is-All-you-Need","https:\u002F\u002Fgithub.com\u002FItWasYou-SAM","https:\u002F\u002Fit-was-you-sam.vercel.app\u002F","AI Engineer ","2026-11-10 22:32:18",{"source":8,"skillName":5,"commands":34,"prompt":44,"zipAvailable":45},[35,38,41],{"tool":36,"command":37},"npx","npx skills add ItWasYou-SAM\u002FSkill-is-All-you-Need --skill agent-evals",{"tool":39,"command":40},"bunx","bunx skills add ItWasYou-SAM\u002FSkill-is-All-you-Need --skill agent-evals",{"tool":42,"command":43},"pnpm","pnpm dlx skills add ItWasYou-SAM\u002FSkill-is-All-you-Need --skill agent-evals","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FItWasYou-SAM\u002FSkill-is-All-you-Need\u002Fblob\u002Fmaster\u002F08-agent-skills\u002Fagent-evals\u002FSKILL.md",true,[],[48,55,62,69,75,81],{"skillNo":49,"name":50,"nameZh":6,"description":51,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":52,"categories":53,"keywords":54,"publishedAt":19},"01M75WFD9BZ3S2ZDECD86QP3T9","invoice-organizer","Automatically organizes invoices and receipts for tax preparation by reading messy files, extracting key information, renaming them consistently, and sorting them into logical folders. Turns hours of manual bookkeeping into minutes of automated organization.","https:\u002F\u002Fgithub.com\u002FItWasYou-SAM\u002FSkill-is-All-you-Need\u002Fblob\u002Fmaster\u002F12-business-productivity\u002Finvoice-organizer\u002FSKILL.md",[],[],{"skillNo":56,"name":57,"nameZh":6,"description":58,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":59,"categories":60,"keywords":61,"publishedAt":19},"01M75WF96VTQYB6FHW43DK9JZ3","file-organizer","Intelligently organizes your files and folders across your computer by understanding context, finding duplicates, suggesting better structures, and automating cleanup tasks. Reduces cognitive load and keeps your digital workspace tidy without manual effort.","https:\u002F\u002Fgithub.com\u002FItWasYou-SAM\u002FSkill-is-All-you-Need\u002Fblob\u002Fmaster\u002F12-business-productivity\u002Ffile-organizer\u002FSKILL.md",[],[],{"skillNo":63,"name":64,"nameZh":6,"description":65,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":66,"categories":67,"keywords":68,"publishedAt":19},"01M75WF5VR42JXJZ5N6152R4P6","developer-growth-analysis","Analyzes your recent Claude Code chat history to identify coding patterns, development gaps, and areas for improvement, curates relevant learning resources from HackerNews, and automatically sends a personalized growth report to your Slack DMs.","https:\u002F\u002Fgithub.com\u002FItWasYou-SAM\u002FSkill-is-All-you-Need\u002Fblob\u002Fmaster\u002F12-business-productivity\u002Fdeveloper-growth-analysis\u002FSKILL.md",[],[],{"skillNo":70,"name":71,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":72,"categories":73,"keywords":74,"publishedAt":19},"01M75WF4PKRDZMHWSC90FVJM1R","connect","https:\u002F\u002Fgithub.com\u002FItWasYou-SAM\u002FSkill-is-All-you-Need\u002Fblob\u002Fmaster\u002F12-business-productivity\u002Fconnect\u002FSKILL.md",[],[],{"skillNo":76,"name":77,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":78,"categories":79,"keywords":80,"publishedAt":19},"01M75WF1H9N7Y1C5V6TM397ZGJ","twitter-algorithm-optimizer","https:\u002F\u002Fgithub.com\u002FItWasYou-SAM\u002FSkill-is-All-you-Need\u002Fblob\u002Fmaster\u002F11-brand-marketing\u002Ftwitter-algorithm-optimizer\u002FSKILL.md",[],[],{"skillNo":82,"name":83,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":84,"categories":85,"keywords":86,"publishedAt":19},"01M75WEZGF264KM1G13P0CTY4C","tailored-resume-generator","https:\u002F\u002Fgithub.com\u002FItWasYou-SAM\u002FSkill-is-All-you-Need\u002Fblob\u002Fmaster\u002F11-brand-marketing\u002Ftailored-resume-generator\u002FSKILL.md",[],[]]