[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7ED3R66MSPTQZ32BCRRTPF9":3,"skill-same:01M7ED3R66MSPTQZ32BCRRTPF9":49},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":27,"commitSha":28,"skillDirectory":29,"forksCount":30,"repositoryHtmlUrl":31,"repositoryHomepageUrl":6,"ownerHtmlUrl":32,"ownerBlogUrl":6,"ownerDescription":33,"updatedAt":34,"install":35,"resources":48},"01M7ED3R66MSPTQZ32BCRRTPF9","agent-eval",null,"Head-to-head comparison of coding agents (Claude Code, Aider, Codex, etc.) on custom tasks with pass rate, cost, time, and consistency metrics","sanjeevafk\u002Fagent-skills","sanjeevafk","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F230678117?v=4","01M7C540EFY46ETG1NNWKS5X3Q","Sanjeev","01M7C540F366J5C1D587BMKH9Q",0,"MIT","https:\u002F\u002Fgithub.com\u002Fsanjeevafk\u002Fagent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fagent-eval\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: agent-eval\ndescription: Head-to-head comparison of coding agents (Claude Code, Aider, Codex, etc.) on custom tasks with pass rate, cost, time, and consistency metrics\nlicense: MIT\nmetadata:\n  origin: ECC\ntools: Read, Write, Edit, Bash, Grep, Glob\n---\n\n# Agent Eval Skill\n\nA lightweight CLI tool for comparing coding agents head-to-head on reproducible tasks. Every \"which coding agent is best?\" comparison runs on vibes — this tool systematizes it.\n\n## When to Activate\n\n- Comparing coding agents (Claude Code, Aider, Codex, etc.) on your own codebase\n- Measuring agent performance before adopting a new tool or model\n- Running regression checks when an agent updates its model or tooling\n- Producing data-backed agent selection decisions for a team\n\n## Installation\n\n> **Note:** Install agent-eval from its repository after reviewing the source.\n\n## Core Concepts\n\n### YAML Task Definitions\n\nDefine tasks declaratively. Each task specifies what to do, which files to touch, and how to judge success:\n\n```yaml\nname: add-retry-logic\ndescription: Add exponential backoff retry to the HTTP client\nrepo: .\u002Fmy-project\nfiles:\n  - src\u002Fhttp_client.py\nprompt: |\n  Add retry logic with exponential backoff to all HTTP requests.\n  Max 3 retries. Initial delay 1s, max delay 30s.\njudge:\n  - type: pytest\n    command: pytest tests\u002Ftest_http_client.py -v\n  - type: grep\n    pattern: \"exponential_backoff|retry\"\n    files: src\u002Fhttp_client.py\ncommit: \"abc1234\"  # pin to specific commit for reproducibility\n```\n\n### Git Worktree Isolation\n\nEach agent run gets its own git worktree — no Docker required. This provides reproducibility isolation so agents cannot interfere with each other or corrupt the base repo.\n\n### Metrics Collected\n\n| Metric | What It Measures |\n|--------|-----------------|\n| Pass rate | Did the agent produce code that passes the judge? |\n| Cost | API spend per task (when available) |\n| Time | Wall-clock seconds to completion |\n| Consistency | Pass rate across repeated runs (e.g., 3\u002F3 = 100%) |\n\n## Workflow\n\n### 1. Define Tasks\n\nCreate a `tasks\u002F` directory with YAML files, one per task:\n\n```bash\nmkdir tasks\n# Write task definitions (see template above)\n```\n\n### 2. Run Agents\n\nExecute agents against your tasks:\n\n```bash\nagent-eval run --task tasks\u002Fadd-retry-logic.yaml --agent claude-code --agent aider --runs 3\n```\n\nEach run:\n1. Creates a fresh git worktree from the specified commit\n2. Hands the prompt to the agent\n3. Runs the judge criteria\n4. Records pass\u002Ffail, cost, and time\n\n### 3. Compare Results\n\nGenerate a comparison report:\n\n```bash\nagent-eval report --format table\n```\n\n```\nTask: add-retry-logic (3 runs each)\n┌──────────────┬───────────┬────────┬────────┬─────────────┐\n│ Agent        │ Pass Rate │ Cost   │ Time   │ Consistency │\n├──────────────┼───────────┼────────┼────────┼─────────────┤\n│ claude-code  │ 3\u002F3       │ $0.12  │ 45s    │ 100%        │\n│ aider        │ 2\u002F3       │ $0.08  │ 38s    │  67%        │\n└──────────────┴───────────┴────────┴────────┴─────────────┘\n```\n\n## Judge Types\n\n### Code-Based (deterministic)\n\n```yaml\njudge:\n  - type: pytest\n    command: pytest tests\u002F -v\n  - type: command\n    command: npm run build\n```\n\n### Pattern-Based\n\n```yaml\njudge:\n  - type: grep\n    pattern: \"class.*Retry\"\n    files: src\u002F**\u002F*.py\n```\n\n### Model-Based (LLM-as-judge)\n\n```yaml\njudge:\n  - type: llm\n    prompt: |\n      Does this implementation correctly handle exponential backoff?\n      Check for: max retries, increasing delays, jitter.\n```\n\n## Best Practices\n\n- **Start with 3-5 tasks** that represent your real workload, not toy examples\n- **Run at least 3 trials** per agent to capture variance — agents are non-deterministic\n- **Pin the commit** in your task YAML so results are reproducible across days\u002Fweeks\n- **Include at least one deterministic judge** (tests, build) per task — LLM judges add noise\n- **Track cost alongside pass rate** — a 95% agent at 10x the cost may not be the right choice\n- **Version your task definitions** — they are test fixtures, treat them as code\n\n## Links\n\n- Repository: [github.com\u002Fjoaquinhuigomez\u002Fagent-eval](https:\u002F\u002Fgithub.com\u002Fjoaquinhuigomez\u002Fagent-eval)\n","# Agent Eval Skill\n\nA lightweight CLI tool for comparing coding agents head-to-head on reproducible tasks. Every \"which coding agent is best?\" comparison runs on vibes — this tool systematizes it.\n\n## When to Activate\n\n- Comparing coding agents (Claude Code, Aider, Codex, etc.) on your own codebase\n- Measuring agent performance before adopting a new tool or model\n- Running regression checks when an agent updates its model or tooling\n- Producing data-backed agent selection decisions for a team\n\n## Installation\n\n> **Note:** Install agent-eval from its repository after reviewing the source.\n\n## Core Concepts\n\n### YAML Task Definitions\n\nDefine tasks declaratively. Each task specifies what to do, which files to touch, and how to judge success:\n\n```yaml\nname: add-retry-logic\ndescription: Add exponential backoff retry to the HTTP client\nrepo: .\u002Fmy-project\nfiles:\n  - src\u002Fhttp_client.py\nprompt: |\n  Add retry logic with exponential backoff to all HTTP requests.\n  Max 3 retries. Initial delay 1s, max delay 30s.\njudge:\n  - type: pytest\n    command: pytest tests\u002Ftest_http_client.py -v\n  - type: grep\n    pattern: \"exponential_backoff|retry\"\n    files: src\u002Fhttp_client.py\ncommit: \"abc1234\"  # pin to specific commit for reproducibility\n```\n\n### Git Worktree Isolation\n\nEach agent run gets its own git worktree — no Docker required. This provides reproducibility isolation so agents cannot interfere with each other or corrupt the base repo.\n\n### Metrics Collected\n\n| Metric | What It Measures |\n|--------|-----------------|\n| Pass rate | Did the agent produce code that passes the judge? |\n| Cost | API spend per task (when available) |\n| Time | Wall-clock seconds to completion |\n| Consistency | Pass rate across repeated runs (e.g., 3\u002F3 = 100%) |\n\n## Workflow\n\n### 1. Define Tasks\n\nCreate a `tasks\u002F` directory with YAML files, one per task:\n\n```bash\nmkdir tasks\n# Write task definitions (see template above)\n```\n\n### 2. Run Agents\n\nExecute agents against your tasks:\n\n```bash\nagent-eval run --task tasks\u002Fadd-retry-logic.yaml --agent claude-code --agent aider --runs 3\n```\n\nEach run:\n1. Creates a fresh git worktree from the specified commit\n2. Hands the prompt to the agent\n3. Runs the judge criteria\n4. Records pass\u002Ffail, cost, and time\n\n### 3. Compare Results\n\nGenerate a comparison report:\n\n```bash\nagent-eval report --format table\n```\n\n```\nTask: add-retry-logic (3 runs each)\n┌──────────────┬───────────┬────────┬────────┬─────────────┐\n│ Agent        │ Pass Rate │ Cost   │ Time   │ Consistency │\n├──────────────┼───────────┼────────┼────────┼─────────────┤\n│ claude-code  │ 3\u002F3       │ $0.12  │ 45s    │ 100%        │\n│ aider        │ 2\u002F3       │ $0.08  │ 38s    │  67%        │\n└──────────────┴───────────┴────────┴────────┴─────────────┘\n```\n\n## Judge Types\n\n### Code-Based (deterministic)\n\n```yaml\njudge:\n  - type: pytest\n    command: pytest tests\u002F -v\n  - type: command\n    command: npm run build\n```\n\n### Pattern-Based\n\n```yaml\njudge:\n  - type: grep\n    pattern: \"class.*Retry\"\n    files: src\u002F**\u002F*.py\n```\n\n### Model-Based (LLM-as-judge)\n\n```yaml\njudge:\n  - type: llm\n    prompt: |\n      Does this implementation correctly handle exponential backoff?\n      Check for: max retries, increasing delays, jitter.\n```\n\n## Best Practices\n\n- **Start with 3-5 tasks** that represent your real workload, not toy examples\n- **Run at least 3 trials** per agent to capture variance — agents are non-deterministic\n- **Pin the commit** in your task YAML so results are reproducible across days\u002Fweeks\n- **Include at least one deterministic judge** (tests, build) per task — LLM judges add noise\n- **Track cost alongside pass rate** — a 95% agent at 10x the cost may not be the right choice\n- **Version your task definitions** — they are test fixtures, treat them as code\n\n## Links\n\n- Repository: [github.com\u002Fjoaquinhuigomez\u002Fagent-eval](https:\u002F\u002Fgithub.com\u002Fjoaquinhuigomez\u002Fagent-eval)\n",{"name":5,"tools":24,"license":15,"metadata":25,"description":7},"Read, Write, Edit, Bash, Grep, Glob",{"origin":26},"ECC","main","4e084e84eb42fc53e144c89c0f7f48b39c8a4b0f","skills\u002Fagent-eval",1,"https:\u002F\u002Fgithub.com\u002Fsanjeevafk\u002Fagent-skills","https:\u002F\u002Fgithub.com\u002Fsanjeevafk","First make it work. Then make it right. Then make it fast.","2026-11-14 13:36:16",{"source":8,"skillName":5,"commands":36,"prompt":46,"zipAvailable":47},[37,40,43],{"tool":38,"command":39},"npx","npx skills add sanjeevafk\u002Fagent-skills --skill agent-eval",{"tool":41,"command":42},"bunx","bunx skills add sanjeevafk\u002Fagent-skills --skill agent-eval",{"tool":44,"command":45},"pnpm","pnpm dlx skills add sanjeevafk\u002Fagent-skills --skill agent-eval","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fsanjeevafk\u002Fagent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fagent-eval\u002FSKILL.md",true,[],[50,57,65,74,82,89],{"skillNo":51,"name":52,"nameZh":6,"description":53,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":54,"categories":55,"keywords":56,"publishedAt":19},"01M7EFHX67M3KKXT1BXVPA5ADT","zarr-python","Chunked N-D arrays for cloud storage (Zarr-Python 3). Compressed arrays, parallel I\u002FO, S3\u002FGCS via fsspec, NumPy\u002FDask\u002FXarray compatible, for large-scale scientific computing pipelines.","https:\u002F\u002Fgithub.com\u002Fsanjeevafk\u002Fagent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fzarr-python\u002FSKILL.md",[],[],{"skillNo":58,"name":59,"nameZh":6,"description":60,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":61,"sourceHtmlUrl":62,"categories":63,"keywords":64,"publishedAt":19},"01M7EFFQGM51TZBGRENXY50ZYM","whiteboard-diagrams","Create accurate, simple, hand-drawn whiteboard diagrams from a user's process, system, architecture, learning workflow, comparison, or causal explanation. Use when the user asks for a scribbly whiteboard visual, educational workflow graphic, hand-drawn explainer, or diagram like the bundled reference; ask one focused clarification question when the subject or relationships are missing, then generate and visually verify the final image.","https:\u002F\u002Fgithub.com\u002Fsanjeevafk\u002Fagent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fwhiteboard-diagrams\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fsanjeevafk\u002Fagent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fwhiteboard-diagrams\u002FSKILL.md",[],[],{"skillNo":66,"name":67,"nameZh":6,"description":68,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":69,"licenseFileUrl":70,"sourceHtmlUrl":71,"categories":72,"keywords":73,"publishedAt":19},"01M7EFF1CSQNTDMHFK4ZGY6NYK","webapp-testing","Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Fsanjeevafk\u002Fagent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fwebapp-testing\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002Fsanjeevafk\u002Fagent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fwebapp-testing\u002FSKILL.md",[],[],{"skillNo":75,"name":76,"nameZh":6,"description":77,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":69,"licenseFileUrl":78,"sourceHtmlUrl":79,"categories":80,"keywords":81,"publishedAt":19},"01M7EFE8106X6C28FT8PA74427","web-artifacts-builder","Suite of tools for creating elaborate, multi-component claude.ai HTML artifacts using modern frontend web technologies (React, Tailwind CSS, shadcn\u002Fui). Use for complex artifacts requiring state management, routing, or shadcn\u002Fui components - not for simple single-file HTML\u002FJSX artifacts.","https:\u002F\u002Fgithub.com\u002Fsanjeevafk\u002Fagent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fweb-artifacts-builder\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002Fsanjeevafk\u002Fagent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fweb-artifacts-builder\u002FSKILL.md",[],[],{"skillNo":83,"name":84,"nameZh":6,"description":85,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":86,"categories":87,"keywords":88,"publishedAt":19},"01M7EFCSNZM9G7ZHS6F127Q71X","verification-loop","A comprehensive verification system for Claude Code sessions.","https:\u002F\u002Fgithub.com\u002Fsanjeevafk\u002Fagent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fverification-loop\u002FSKILL.md",[],[],{"skillNo":90,"name":91,"nameZh":6,"description":92,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":93,"sourceHtmlUrl":94,"categories":95,"keywords":96,"publishedAt":19},"01M7EFCE7EZMKPRYDDNKBQ9EEH","vercel-deploy","Deploy applications and websites to Vercel. Use when the user requests deployment actions like \"deploy my app\", \"deploy and give me the link\", \"push this live\", or \"create a preview deployment\".","https:\u002F\u002Fgithub.com\u002Fsanjeevafk\u002Fagent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fvercel-deploy\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002Fsanjeevafk\u002Fagent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fvercel-deploy\u002FSKILL.md",[],[]]