[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M79C0VYH8C6Z2H3F7KSC1R1F":3,"skill-same:01M79C0VYH8C6Z2H3F7KSC1R1F":48},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":27,"commitSha":28,"skillDirectory":29,"forksCount":30,"repositoryHtmlUrl":31,"repositoryHomepageUrl":6,"ownerHtmlUrl":32,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":33,"install":34,"resources":47},"01M79C0VYH8C6Z2H3F7KSC1R1F","agent-eval",null,"Head-to-head comparison of coding agents (Claude Code, Aider, Codex, etc.) on custom tasks with pass rate, cost, time, and consistency metrics","NguyenPhuocAnhDung\u002Foil_forecast_tail_risk","NguyenPhuocAnhDung","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F167469630?v=4","01M799ZYV3KJAR2SVWBJ6HS2KT","Nguyễn Phước Anh Dũng","01M799ZYVSZ10CRDCF7JS9SA8C",0,"MIT","https:\u002F\u002Fgithub.com\u002FNguyenPhuocAnhDung\u002Foil_forecast_tail_risk\u002Fblob\u002Fmain\u002F.agents\u002Fecc\u002Fskills\u002Fagent-eval\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: agent-eval\ndescription: Head-to-head comparison of coding agents (Claude Code, Aider, Codex, etc.) on custom tasks with pass rate, cost, time, and consistency metrics\nlicense: MIT\nmetadata:\n  origin: ECC\ntools: Read, Write, Edit, Bash, Grep, Glob\n---\n\n# Agent Eval Skill\n\nA lightweight CLI tool for comparing coding agents head-to-head on reproducible tasks. Every \"which coding agent is best?\" comparison runs on vibes — this tool systematizes it.\n\n## When to Activate\n\n- Comparing coding agents (Claude Code, Aider, Codex, etc.) on your own codebase\n- Measuring agent performance before adopting a new tool or model\n- Running regression checks when an agent updates its model or tooling\n- Producing data-backed agent selection decisions for a team\n\n## Installation\n\n> **Note:** Install agent-eval from its repository after reviewing the source.\n\n## Core Concepts\n\n### YAML Task Definitions\n\nDefine tasks declaratively. Each task specifies what to do, which files to touch, and how to judge success:\n\n```yaml\nname: add-retry-logic\ndescription: Add exponential backoff retry to the HTTP client\nrepo: .\u002Fmy-project\nfiles:\n  - src\u002Fhttp_client.py\nprompt: |\n  Add retry logic with exponential backoff to all HTTP requests.\n  Max 3 retries. Initial delay 1s, max delay 30s.\njudge:\n  - type: pytest\n    command: pytest tests\u002Ftest_http_client.py -v\n  - type: grep\n    pattern: \"exponential_backoff|retry\"\n    files: src\u002Fhttp_client.py\ncommit: \"abc1234\"  # pin to specific commit for reproducibility\n```\n\n### Git Worktree Isolation\n\nEach agent run gets its own git worktree — no Docker required. This provides reproducibility isolation so agents cannot interfere with each other or corrupt the base repo.\n\n### Metrics Collected\n\n| Metric | What It Measures |\n|--------|-----------------|\n| Pass rate | Did the agent produce code that passes the judge? |\n| Cost | API spend per task (when available) |\n| Time | Wall-clock seconds to completion |\n| Consistency | Pass rate across repeated runs (e.g., 3\u002F3 = 100%) |\n\n## Workflow\n\n### 1. Define Tasks\n\nCreate a `tasks\u002F` directory with YAML files, one per task:\n\n```bash\nmkdir tasks\n# Write task definitions (see template above)\n```\n\n### 2. Run Agents\n\nExecute agents against your tasks:\n\n```bash\nagent-eval run --task tasks\u002Fadd-retry-logic.yaml --agent claude-code --agent aider --runs 3\n```\n\nEach run:\n1. Creates a fresh git worktree from the specified commit\n2. Hands the prompt to the agent\n3. Runs the judge criteria\n4. Records pass\u002Ffail, cost, and time\n\n### 3. Compare Results\n\nGenerate a comparison report:\n\n```bash\nagent-eval report --format table\n```\n\n```\nTask: add-retry-logic (3 runs each)\n┌──────────────┬───────────┬────────┬────────┬─────────────┐\n│ Agent        │ Pass Rate │ Cost   │ Time   │ Consistency │\n├──────────────┼───────────┼────────┼────────┼─────────────┤\n│ claude-code  │ 3\u002F3       │ $0.12  │ 45s    │ 100%        │\n│ aider        │ 2\u002F3       │ $0.08  │ 38s    │  67%        │\n└──────────────┴───────────┴────────┴────────┴─────────────┘\n```\n\n## Judge Types\n\n### Code-Based (deterministic)\n\n```yaml\njudge:\n  - type: pytest\n    command: pytest tests\u002F -v\n  - type: command\n    command: npm run build\n```\n\n### Pattern-Based\n\n```yaml\njudge:\n  - type: grep\n    pattern: \"class.*Retry\"\n    files: src\u002F**\u002F*.py\n```\n\n### Model-Based (LLM-as-judge)\n\n```yaml\njudge:\n  - type: llm\n    prompt: |\n      Does this implementation correctly handle exponential backoff?\n      Check for: max retries, increasing delays, jitter.\n```\n\n## Best Practices\n\n- **Start with 3-5 tasks** that represent your real workload, not toy examples\n- **Run at least 3 trials** per agent to capture variance — agents are non-deterministic\n- **Pin the commit** in your task YAML so results are reproducible across days\u002Fweeks\n- **Include at least one deterministic judge** (tests, build) per task — LLM judges add noise\n- **Track cost alongside pass rate** — a 95% agent at 10x the cost may not be the right choice\n- **Version your task definitions** — they are test fixtures, treat them as code\n\n## Links\n\n- Repository: [github.com\u002Fjoaquinhuigomez\u002Fagent-eval](https:\u002F\u002Fgithub.com\u002Fjoaquinhuigomez\u002Fagent-eval)\n","# Agent Eval Skill\n\nA lightweight CLI tool for comparing coding agents head-to-head on reproducible tasks. Every \"which coding agent is best?\" comparison runs on vibes — this tool systematizes it.\n\n## When to Activate\n\n- Comparing coding agents (Claude Code, Aider, Codex, etc.) on your own codebase\n- Measuring agent performance before adopting a new tool or model\n- Running regression checks when an agent updates its model or tooling\n- Producing data-backed agent selection decisions for a team\n\n## Installation\n\n> **Note:** Install agent-eval from its repository after reviewing the source.\n\n## Core Concepts\n\n### YAML Task Definitions\n\nDefine tasks declaratively. Each task specifies what to do, which files to touch, and how to judge success:\n\n```yaml\nname: add-retry-logic\ndescription: Add exponential backoff retry to the HTTP client\nrepo: .\u002Fmy-project\nfiles:\n  - src\u002Fhttp_client.py\nprompt: |\n  Add retry logic with exponential backoff to all HTTP requests.\n  Max 3 retries. Initial delay 1s, max delay 30s.\njudge:\n  - type: pytest\n    command: pytest tests\u002Ftest_http_client.py -v\n  - type: grep\n    pattern: \"exponential_backoff|retry\"\n    files: src\u002Fhttp_client.py\ncommit: \"abc1234\"  # pin to specific commit for reproducibility\n```\n\n### Git Worktree Isolation\n\nEach agent run gets its own git worktree — no Docker required. This provides reproducibility isolation so agents cannot interfere with each other or corrupt the base repo.\n\n### Metrics Collected\n\n| Metric | What It Measures |\n|--------|-----------------|\n| Pass rate | Did the agent produce code that passes the judge? |\n| Cost | API spend per task (when available) |\n| Time | Wall-clock seconds to completion |\n| Consistency | Pass rate across repeated runs (e.g., 3\u002F3 = 100%) |\n\n## Workflow\n\n### 1. Define Tasks\n\nCreate a `tasks\u002F` directory with YAML files, one per task:\n\n```bash\nmkdir tasks\n# Write task definitions (see template above)\n```\n\n### 2. Run Agents\n\nExecute agents against your tasks:\n\n```bash\nagent-eval run --task tasks\u002Fadd-retry-logic.yaml --agent claude-code --agent aider --runs 3\n```\n\nEach run:\n1. Creates a fresh git worktree from the specified commit\n2. Hands the prompt to the agent\n3. Runs the judge criteria\n4. Records pass\u002Ffail, cost, and time\n\n### 3. Compare Results\n\nGenerate a comparison report:\n\n```bash\nagent-eval report --format table\n```\n\n```\nTask: add-retry-logic (3 runs each)\n┌──────────────┬───────────┬────────┬────────┬─────────────┐\n│ Agent        │ Pass Rate │ Cost   │ Time   │ Consistency │\n├──────────────┼───────────┼────────┼────────┼─────────────┤\n│ claude-code  │ 3\u002F3       │ $0.12  │ 45s    │ 100%        │\n│ aider        │ 2\u002F3       │ $0.08  │ 38s    │  67%        │\n└──────────────┴───────────┴────────┴────────┴─────────────┘\n```\n\n## Judge Types\n\n### Code-Based (deterministic)\n\n```yaml\njudge:\n  - type: pytest\n    command: pytest tests\u002F -v\n  - type: command\n    command: npm run build\n```\n\n### Pattern-Based\n\n```yaml\njudge:\n  - type: grep\n    pattern: \"class.*Retry\"\n    files: src\u002F**\u002F*.py\n```\n\n### Model-Based (LLM-as-judge)\n\n```yaml\njudge:\n  - type: llm\n    prompt: |\n      Does this implementation correctly handle exponential backoff?\n      Check for: max retries, increasing delays, jitter.\n```\n\n## Best Practices\n\n- **Start with 3-5 tasks** that represent your real workload, not toy examples\n- **Run at least 3 trials** per agent to capture variance — agents are non-deterministic\n- **Pin the commit** in your task YAML so results are reproducible across days\u002Fweeks\n- **Include at least one deterministic judge** (tests, build) per task — LLM judges add noise\n- **Track cost alongside pass rate** — a 95% agent at 10x the cost may not be the right choice\n- **Version your task definitions** — they are test fixtures, treat them as code\n\n## Links\n\n- Repository: [github.com\u002Fjoaquinhuigomez\u002Fagent-eval](https:\u002F\u002Fgithub.com\u002Fjoaquinhuigomez\u002Fagent-eval)\n",{"name":5,"tools":24,"license":15,"metadata":25,"description":7},"Read, Write, Edit, Bash, Grep, Glob",{"origin":26},"ECC","main","89920776f7cbf648024818498d24874504a8d781",".agents\u002Fecc\u002Fskills\u002Fagent-eval",1,"https:\u002F\u002Fgithub.com\u002FNguyenPhuocAnhDung\u002Foil_forecast_tail_risk","https:\u002F\u002Fgithub.com\u002FNguyenPhuocAnhDung","2026-09-24 13:39:29",{"source":8,"skillName":5,"commands":35,"prompt":45,"zipAvailable":46},[36,39,42],{"tool":37,"command":38},"npx","npx skills add NguyenPhuocAnhDung\u002Foil_forecast_tail_risk --skill agent-eval",{"tool":40,"command":41},"bunx","bunx skills add NguyenPhuocAnhDung\u002Foil_forecast_tail_risk --skill agent-eval",{"tool":43,"command":44},"pnpm","pnpm dlx skills add NguyenPhuocAnhDung\u002Foil_forecast_tail_risk --skill agent-eval","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FNguyenPhuocAnhDung\u002Foil_forecast_tail_risk\u002Fblob\u002Fmain\u002F.agents\u002Fecc\u002Fskills\u002Fagent-eval\u002FSKILL.md",true,[],[49,57,64,71,78,85],{"skillNo":50,"name":51,"nameZh":6,"description":52,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":53,"licenseFileUrl":6,"sourceHtmlUrl":54,"categories":55,"keywords":56,"publishedAt":19},"01M79DARA6C9DT0XSPREN3DC2B","ml-best-practices","CRITICAL RULE: You MUST use this skill whenever the task involves any machine learning tasks or data analysis.\nUse this skill if the user's prompt or requirements mention any of the following:\n  * Clustering\n  * Classification\n  * Regression\n  * Time series forecasting\n  * Statistical testing\n  * Model comparison\n  * ML\n  * Data analysis\n\nSQL\u002FBigQuery ML HANDOFF: If the user requires a SQL solution, use this skill to dictate the ANALYSIS STEPS (e.g., markdown analysis cells, visualization logic), but defer to `bigquery` for all SQL syntax.","Apache-2.0","https:\u002F\u002Fgithub.com\u002FNguyenPhuocAnhDung\u002Foil_forecast_tail_risk\u002Fblob\u002Fmain\u002F.agents\u002Fworker_msB_fix\u002Fskills\u002Fml-best-practices\u002FSKILL.md",[],[],{"skillNo":58,"name":59,"nameZh":6,"description":60,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":61,"categories":62,"keywords":63,"publishedAt":19},"01M79D3735P38W4M5PPKY8V7ZN","verification-loop","A comprehensive verification system for Claude Code sessions.","https:\u002F\u002Fgithub.com\u002FNguyenPhuocAnhDung\u002Foil_forecast_tail_risk\u002Fblob\u002Fmain\u002F.agents\u002Fecc\u002Fskills\u002Fverification-loop\u002FSKILL.md",[],[],{"skillNo":65,"name":66,"nameZh":6,"description":67,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":53,"licenseFileUrl":6,"sourceHtmlUrl":68,"categories":69,"keywords":70,"publishedAt":19},"01M79CYA4HEQN5ZFD0TD5YKW6J","returns-reverse-logistics","Codified expertise for returns authorization, receipt and inspection, disposition decisions, refund processing, fraud detection, and warranty claims management. Informed by returns operations managers with 15+ years experience. Includes grading frameworks, disposition economics, fraud pattern recognition, and vendor recovery processes. Use when handling product returns, reverse logistics, refund decisions, return fraud detection, or warranty claims.","https:\u002F\u002Fgithub.com\u002FNguyenPhuocAnhDung\u002Foil_forecast_tail_risk\u002Fblob\u002Fmain\u002F.agents\u002Fecc\u002Fskills\u002Freturns-reverse-logistics\u002FSKILL.md",[],[],{"skillNo":72,"name":73,"nameZh":6,"description":74,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":75,"categories":76,"keywords":77,"publishedAt":19},"01M79CXQ5Y3TGM7WV8B4MR7W8A","recursive-decision-ledger","Use when the user asks for repeated rollouts, marked decision processes, high-dimensional search, stochastic optimization, local-optima exploration, ensemble comparison, or recursive reasoning with a visible evidence trail.","https:\u002F\u002Fgithub.com\u002FNguyenPhuocAnhDung\u002Foil_forecast_tail_risk\u002Fblob\u002Fmain\u002F.agents\u002Fecc\u002Fskills\u002Frecursive-decision-ledger\u002FSKILL.md",[],[],{"skillNo":79,"name":80,"nameZh":6,"description":81,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":53,"licenseFileUrl":6,"sourceHtmlUrl":82,"categories":83,"keywords":84,"publishedAt":19},"01M79CW442W74WMRYBQS3278T9","quality-nonconformance","Codified expertise for quality control, non-conformance investigation, root cause analysis, corrective action, and supplier quality management in regulated manufacturing. Informed by quality engineers with 15+ years experience across FDA, IATF 16949, and AS9100 environments. Includes NCR lifecycle management, CAPA systems, SPC interpretation, and audit methodology. Use when investigating non-conformances, performing root cause analysis, managing CAPAs, interpreting SPC data, or handling supplier quality issues.","https:\u002F\u002Fgithub.com\u002FNguyenPhuocAnhDung\u002Foil_forecast_tail_risk\u002Fblob\u002Fmain\u002F.agents\u002Fecc\u002Fskills\u002Fquality-nonconformance\u002FSKILL.md",[],[],{"skillNo":86,"name":87,"nameZh":6,"description":88,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":53,"licenseFileUrl":6,"sourceHtmlUrl":89,"categories":90,"keywords":91,"publishedAt":19},"01M79CV8N0CTPG6XXJ5WQNC6YE","production-scheduling","Codified expertise for production scheduling, job sequencing, line balancing, changeover optimization, and bottleneck resolution in discrete and batch manufacturing. Informed by production schedulers with 15+ years experience. Includes TOC\u002Fdrum-buffer-rope, SMED, OEE analysis, disruption response frameworks, and ERP\u002FMES interaction patterns. Use when scheduling production, resolving bottlenecks, optimizing changeovers, responding to disruptions, or balancing manufacturing lines.","https:\u002F\u002Fgithub.com\u002FNguyenPhuocAnhDung\u002Foil_forecast_tail_risk\u002Fblob\u002Fmain\u002F.agents\u002Fecc\u002Fskills\u002Fproduction-scheduling\u002FSKILL.md",[],[]]