pair-benchmark-all-executors
Run one benchmark script that compares pairv2, pair via Claude Teams, and pair via direct Python
- License 许可
- MIT
- 最近更新
- 2026-11-14
- Stars
- ★ 6
- Forks
- 0
01 前言 Front Matter
- name
- pair-benchmark-all-executors
- type
- usage
- scope
- project
- description
- Run one benchmark script that compares pairv2, pair via Claude Teams, and pair via direct Python
03 SKILL.md 正文
Pair Benchmark (All Executors)
Purpose
Use one script to benchmark all pair executors:
pairv2(.claude/pair/pair_execute_v2.py)/pairvia Claude Teams prompt path/pairvia direct Python (.claude/pair/pair_execute.py)
Script
python .claude/pair/benchmark_pair_executors.py (or venv/bin/python when using a virtualenv)
Quick Start
Run all three executors in one benchmark:
python .claude/pair/benchmark_pair_executors.py \
--executor-set all \
--benchmark-iterations 1 \
--timeout-seconds 180 \
--teams-command "claude --print" \
--artifact /tmp/{repo}/{branch}/pair/all_three.json
Run a small, named Amazon-style ecommerce preset:
python .claude/pair/benchmark_pair_executors.py \
--task-preset amazon_clone \
--pairv2-max-cycles 2 \
--timeout-seconds 1200 \
--artifact /tmp/{repo}/{branch}/pair/amazon_clone_benchmark.json
For historical benchmark prompts, including alternate Amazon-style variants, see
testing_llm/pair/benchmark_tasks.json.
Run all three in parallel:
python .claude/pair/benchmark_pair_executors.py \
--executor-set all \
--parallel \
--benchmark-iterations 1 \
--timeout-seconds 180 \
--teams-command "claude --print" \
--artifact /tmp/{repo}/{branch}/pair/all_three_parallel.json
Executor Modes
--executor-set all: pair python + pair teams + pairv2--executor-set legacy-vs-v2: pair python vs pairv2--executor-set teams-vs-v2: pair teams vs pairv2--executor-set pairv2-only: pairv2 only
Key Output Fields
pair_python,pair_python_status_counts,pair_python_rawpair_claude_teams,pair_teams_status_counts,pair_teams_rawpairv2_langgraph,pairv2_status_counts,pairv2_rawspeedup_ratio_pair_python_over_v2speedup_ratio_pair_teams_over_v2speedup_ratio_pair_python_over_teams
Current Defaults (updated Feb 2026)
--executor-set:pairv2-only--coder-cli:minimax--verifier-cli:minimax--task: hello world script + pytest test (default); overridden by--task-presetwhen provided--timeout-seconds:600
Run with no args to use these defaults:
venv/bin/python .claude/pair/benchmark_pair_executors.py
Notes
minimaxis not a CLI binary — it runsclaudewith MiniMax API endpoint.
04 同仓库更多 Skills
disk-reclaim-playbook
Use Disk Magician to diagnose disk growth or qualify cleanup on macOS or Linux.
claudea-bench
Compare `claudea` and normal `claude` on a fair split of comparable work by default; use `--strict` for duplicate runs of the exact same task.
pr
Alias for the canonical pr-lifecycle PR workflow skill (/pr-workflow, /prw).
whisper
OpenAI's general-purpose speech recognition model. Supports 99 languages, transcription, translation to English, and language identification. Six model sizes from tiny (39M params) to large (1550M params). Use for speech-to-text, podcast transcription, or multilingual audio processing. Best for robust, multilingual ASR.
webapp-testing
Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.
pr-report
This skill should be used when the user asks for a PR report, PR audit, PR delta analysis, /pr-report, or wants per-PR summary with delta files/lines. Generates a structured report for one or more open PRs covering purpose, files changed, +/− line counts, /code-standards + /thermo review findings, /ponytail simplification opportunities, file-overlap combination candidates, and /green status per pr-green-definition. Output: machine-readable JSONL + human-readable markdown. Read-only by default; can apply simplifications if user opts in.