[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7E3FN3XD2FDDZRXT2XMHAW1":3,"skill-same:01M7E3FN3XD2FDDZRXT2XMHAW1":59},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":39,"commitSha":40,"skillDirectory":41,"forksCount":14,"repositoryHtmlUrl":42,"repositoryHomepageUrl":42,"ownerHtmlUrl":43,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":44,"install":45,"resources":58},"01M7E3FN3XD2FDDZRXT2XMHAW1","evolve-skill",null,"Evolve Skill: measurement-first skill optimizer. Evaluates SKILL.md files against an anchored 9-dimension rubric, validates that the rubric itself is stable (test-retest), optimizes with a hill-climbing loop that only accepts improvements larger than measurement noise, protects against overfitting with train\u002Fholdout prompt splits, and preserves original function via semantic diff checks. Use when user mentions: optimize skill, skill review, skill audit, improve SKILL.md, skill score, skill rubric, skill quality, make my skill better, rate my skills, evolve skills, skill hill climbing, automated skill improvement.","Mr-Nobody-Anonymous\u002FAll-skills","Mr-Nobody-Anonymous","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F184253119?v=4","01M7AEXFJHKTYQW5R0ZCQQTWFV","anonymous","01M7C00M3GQR4QKP9GZA42GFAK",0,"MIT","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fagent-evaluation\u002Ftemp_clone_p4\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: evolve-skill\ndescription: \"Evolve Skill: measurement-first skill optimizer. Evaluates SKILL.md files against an anchored 9-dimension rubric, validates that the rubric itself is stable (test-retest), optimizes with a hill-climbing loop that only accepts improvements larger than measurement noise, protects against overfitting with train\u002Fholdout prompt splits, and preserves original function via semantic diff checks. Use when user mentions: optimize skill, skill review, skill audit, improve SKILL.md, skill score, skill rubric, skill quality, make my skill better, rate my skills, evolve skills, skill hill climbing, automated skill improvement.\"\ncategory: agent-evaluation\ndomain: agent-evaluation\nsubdomain: general\nversion: 1.0.0\nlicense: MIT\nrisk: low\nsource:\n  repository: \"taneltaluri\u002Fevolve-skill\"\n  commit: \"8d24739a45\"\n  imported_at: \"2026-09-20\"\n  license: \"MIT\"\nplatforms:\n  - claude-code\n  - cursor\n  - codex\n  - gemini\n  - antigravity\n  - copilot\n---\n\n\n# Evolve Skill\n\n> A measurement-first skill optimizer. Inspired by Karpathy's `autoresearch` and the `darwin-skill` project, but rebuilt around one conviction: **if you cannot trust your measurement, you cannot trust your optimization.**\n>\n> Every decision this skill makes is gated by whether the measurement is reliable enough to justify it.\n\n---\n\n## What This Skill Is Not\n\nThis is not a formatting linter. This is not a prompt rewriter. This is not an autonomous agent that will rewrite your skills in the background.\n\nThis is a **disciplined experiment loop** with strict gates around scoring noise, overfitting, and function drift — the three failure modes that make most \"auto-optimize\" tools produce worse skills over time.\n\nIf you want a quick skill tidy-up, use a simpler tool. If you have 20+ skills and need to know which ones are actually improving, this is for you.\n\n---\n\n## Core Philosophy — The Three Gates\n\nEvery optimization cycle passes through three gates. If any gate fails, the cycle stops or rolls back.\n\n### Gate 1 — Measurement Stability (can we trust the score?)\n\nBefore any optimization runs, score the baseline skill **three independent times** using three separate sub-agents. Compute the standard deviation. If `SD > 2.0 points`, the rubric is too loose and no optimization may proceed until the rubric is tightened.\n\nA ratchet mechanism with unreliable measurement is not a ratchet — it is a random walk pretending to be progress.\n\n### Gate 2 — Effect Size (is this improvement real?)\n\nA keep decision requires `Δscore ≥ max(3, 2×SD)` where SD is the measurement noise from Gate 1. A skill that \"improved\" by 1.5 points when noise is ±2 points did not improve.\n\n### Gate 3 — Function Preservation (is it still the same skill?)\n\nBefore and after every change, extract the skill's core functions as a bullet list (via a sub-agent). If function overlap drops below 80%, the change is rejected regardless of score. A higher-scoring skill that silently lost a capability is worse, not better.\n\n---\n\n## The 9-Dimension Anchored Rubric (100 points)\n\nEach dimension is scored 1–10. Every dimension has three anchor points (1, 5, 10) defined in `rubric\u002Fanchored-rubric.md`. Scorers must reference the anchors and cite which anchor their score lies closest to. This is what stabilizes inter-rater agreement.\n\n**Structure (50 points)**\n\n| # | Dimension | Weight | What it measures |\n|---|-----------|--------|------------------|\n| 1 | Frontmatter quality | 8 | name format, description ≤1024 chars, triggers |\n| 2 | Workflow clarity | 12 | numbered steps, I\u002FO per step |\n| 3 | Edge-case coverage | 8 | fallbacks, error recovery |\n| 4 | Checkpoint design | 6 | human-in-loop gates before destructive actions |\n| 5 | Instruction specificity | 12 | concrete params, formats, examples |\n| 6 | Resource integration | 4 | references\u002Fscripts\u002Fassets resolve |\n\n**Effectiveness (40 points)**\n\n| # | Dimension | Weight | What it measures |\n|---|-----------|--------|------------------|\n| 7 | Architectural fit | 10 | non-redundant, matches ecosystem conventions |\n| 8 | Live test performance | 30 | output quality on held-out test prompts |\n\n**Localization (10 points)**\n\n| # | Dimension | Weight | What it measures |\n|---|-----------|--------|------------------|\n| 9 | Localization quality | 10 | triggers + examples in all declared languages, cross-platform paths |\n\n**Total:** `Σ(dimension_score × weight) \u002F 10`, capped at 100.\n\n**The effectiveness dimension (30 points) is the largest single weight** by design. A beautifully written skill that produces bad output is still a bad skill.\n\n---\n\n## Workflow\n\n### Phase 0 — Scope and Setup\n\n1. Determine scope:\n   - `optimize all skills` → scan `.claude\u002Fskills\u002F*\u002FSKILL.md` and skill directories\n   - `optimize \u003Cname>` → single skill\n   - `evaluate only` → Phases 0–2 only, no edits\n2. Create a dated branch: `evolve\u002FYYYYMMDD-HHMM`\n3. Initialize `results.tsv` and `playbook.md` if they do not exist\n4. Read existing `playbook.md` — known patterns will be tried first in Phase 3\n\n### Phase 1 — Test Prompt Design (Train \u002F Holdout Split)\n\nFor each skill, the user — not the agent — writes prompts. The agent may suggest, but suggestions must be confirmed.\n\n1. **Training prompts (3)** — seen during optimization, used to drive keep\u002Frevert decisions\n   - One happy-path prompt\n   - One complex or ambiguous prompt\n   - One edge-case prompt\n2. **Holdout prompts (2)** — never seen during optimization, used only for final validation\n   - Cover different scenarios than the training prompts\n3. Save to `\u003Cskill_dir>\u002Ftest-prompts.json` using the template in `templates\u002Ftest-prompts.template.json`\n4. Show both sets to the user. Confirm before proceeding.\n\n**Why both sets exist:** without holdouts, the optimization overfits to the training prompts and appears to improve while quietly getting worse on unseen inputs.\n\n### Phase 2 — Baseline Evaluation with Stability Check\n\nFor each skill:\n\n1. **Score three times independently** using three separate sub-agents with no shared context\n2. Compute mean and SD across the three runs\n3. If `SD > 2.0 points`:\n   - Flag the skill as `calibration_required`\n   - Skip optimization for this skill\n   - Report to user: \"Rubric is too loose for this skill. Tighten anchors before proceeding.\"\n4. If `SD ≤ 2.0`:\n   - Store `baseline_score = mean`, `baseline_sd = sd`\n   - Proceed to Phase 3\n\nFor the live-test dimension (30 points): run each training prompt through two sub-agents — one with the skill loaded, one without (baseline). Compare output quality against the anchors in the rubric.\n\nIf sub-agents are unavailable, set `confidence=low` and **do not** use the score for keep\u002Frevert decisions. A low-confidence score is diagnostic only.\n\n### Phase 3 — Optimization Loop\n\nBudget allocation follows a multi-armed bandit pattern, not round-robin:\n\n```\ninitial_budget_per_skill = 3 rounds\ntotal_budget = 30 rounds across 10 skills (configurable)\n\nround allocation:\n  if last round Δ ≥ 5 points:      +2 bonus rounds\n  if last round Δ between 0 and 3:  continue at baseline allocation\n  if last round Δ ≤ 0 (reverted):   forfeit remaining budget, move on\n  hot skills (3+ consecutive keeps) can use up to 10 rounds total\n```\n\nEach round:\n\n```\n1. Consult playbook.md:\n   \"Has a known pattern worked for a skill like this before?\"\n   If yes, try the known pattern first.\n\n2. If no known pattern, diagnose:\n   Find the lowest-scoring dimension.\n\n3. Generate exactly one targeted change:\n   - Which lines change\n   - Which rubric dimension this targets\n   - Predicted Δ score\n\n4. Execute on an experiment branch:\n   git checkout -b evolve\u002FYYYYMMDD-HHMM\u002F\u003Cskill>\u002Fexp-\u003CN>\n   edit SKILL.md\n   git commit -m \"exp: \u003Cskill> \u003Cdimension> \u003Csummary>\"\n\n5. Re-score with an INDEPENDENT sub-agent (never the editing agent):\n   - Run training prompts only (holdout stays sealed)\n   - Compute new_score and run Gate 3 (function preservation)\n\n6. Apply gates:\n   Gate 2: if new_score > old_score + max(3, 2×SD) → candidate keep\n   Gate 3: if function_overlap ≥ 80% → candidate keep\n   Otherwise → branch stays for history, move on\n\n7. On candidate keep:\n   git checkout main_branch\n   git merge --no-ff evolve\u002F...\u002Fexp-\u003CN>\n   update results.tsv with status=keep\n\n8. On reject:\n   leave the experiment branch in place (do not delete — it is training data)\n   update results.tsv with status=reject and reason code\n\n9. Log to results.tsv with every column filled\n```\n\nAfter a skill's budget is exhausted, insert a **human checkpoint**:\n- Show diff across all kept experiments\n- Show training-prompt output comparison\n- Show delta summary\n- Wait for explicit user approval\n\n### Phase 4 — Holdout Validation (Overfitting Check)\n\nThis runs once per skill, after Phase 3 completes and the user has approved.\n\n1. Run the **holdout prompts** (which have never been seen during optimization) against both the original and optimized versions\n2. Score only the live-test dimension (30 points)\n3. If `holdout_delta \u003C 0`:\n   - The optimization overfit to the training prompts\n   - Roll back the entire optimization for this skill\n   - Log as `overfit_detected` in results.tsv\n4. If `holdout_delta ≥ 0`:\n   - Optimization is validated\n   - Write any discovered patterns into `playbook.md`\n\n### Phase 5 — Playbook Update and Reporting\n\nFor every keep that produced `Δ ≥ 5 points`, add an entry to `playbook.md`:\n\n```markdown\n## Pattern: \u003Cshort name>\n- **Problem signature:** \u003Cwhat triggers this pattern's applicability>\n- **Intervention:** \u003Cthe specific edit>\n- **Measured impact:** \u003Cskill, Δ, n observations>\n- **Where it worked:** \u003Cskill names>\n- **Where it failed:** \u003Cskill names, if any>\n```\n\nPhase 3 of future runs will check this playbook before diagnosing fresh.\n\nFinal report includes:\n- Per-skill before\u002Fafter with confidence intervals\n- Holdout validation results\n- Playbook additions\n- Branches with failed experiments (for later review)\n\n---\n\n## Constraints\n\n1. **Core function is immutable.** Gate 3 enforces this. If a change removes a capability, it is rejected even if the score went up.\n2. **No new dependencies.** Do not add scripts, references, or external tools the original skill did not declare.\n3. **One dimension per experiment.** Multi-dimension changes make keep\u002Frevert decisions unattributable.\n4. **Size ceiling.** Optimized SKILL.md must be ≤150% of original size.\n5. **Git-first, rollback-safe.** Every change is on a branch. `git revert` is for merge commits only; failed branches are preserved.\n6. **Score independence.** The sub-agent that edits is never the sub-agent that scores. Enforced by session separation.\n7. **Confidence gating.** A `confidence=low` score cannot drive a keep decision under any circumstances.\n8. **Localization is a dimension, not an afterthought.** If a skill declares bilingual support, both languages are tested.\n\n---\n\n## File Layout\n\n```\n.claude\u002Fskills\u002Fevolve-skill\u002F\n├── SKILL.md                           # This file\n├── rubric\u002F\n│   ├── anchored-rubric.md             # Human-readable anchors\n│   └── rubric-v1.json                 # Machine-readable rubric\n├── scripts\u002F\n│   ├── calibrate.py                   # Test-retest variance\n│   ├── score.py                       # Apply rubric with anchors\n│   ├── semantic_diff.py               # Function preservation\n│   ├── ratchet.py                     # Keep\u002Frevert with CI\n│   └── allocate.py                    # Bandit budget allocation\n├── templates\u002F\n│   ├── test-prompts.template.json     # Train\u002Fholdout structure\n│   ├── result-card.html               # Optional visual card\n│   └── playbook.template.md           # Starter playbook\n├── examples\u002F\n│   ├── example-walkthrough.md         # Full optimization run\n│   └── example-results.tsv            # Real log format\n└── results.tsv                        # Created on first run\n```\n\nPer-skill files created during optimization:\n\n```\n\u003Ctarget_skill>\u002F\n├── SKILL.md\n├── test-prompts.json                  # Train + holdout split\n├── baseline-scores.json               # Three-run variance record\n└── optimization-log.md                # Per-skill narrative\n```\n\n---\n\n## results.tsv Schema\n\nTab-separated, append-only. Every row is an event.\n\n```\ntimestamp  branch  skill  old_score  new_score  delta  sd  status  dimension  intervention  function_overlap  holdout_delta  confidence  eval_mode  notes\n```\n\nColumn definitions:\n\n- `status` ∈ {`baseline`, `keep`, `reject_effect_size`, `reject_function_drift`, `reject_noisy`, `overfit_detected`}\n- `confidence` ∈ {`high`, `medium`, `low`} — derived from `eval_mode`\n- `eval_mode` ∈ {`full_test` (sub-agents with live prompts), `partial` (structure only), `simulation` (no sub-agent available)}\n- `function_overlap` — semantic-diff overlap percentage from Gate 3\n- `holdout_delta` — only populated on Phase 4 rows\n\n---\n\n## Intervention Playbook (Priority Order)\n\nWhen diagnosing the lowest-scoring dimension, consult this ranked list of proven interventions.\n\n### P0 — Effectiveness Problems (Live-Test Signals)\n\n- Output misses user intent → search SKILL.md for misleading instructions, prune\n- `with_skill` output worse than `baseline` → skill is over-constraining, simplify\n- Output format inconsistent → add explicit output schema with example\n\n### P1 — Structural Problems\n\n- Frontmatter trigger list too narrow → expand with synonyms and natural phrasings in all declared languages\n- Workflow lacks Phase\u002FStep structure → impose linear numbered flow\n- No human checkpoint before destructive actions → insert confirmation gate\n\n### P2 — Specificity Problems\n\n- Vague steps (\"process the image\") → replace with specific operation + parameters\n- Missing I\u002FO specs → add format, path, and example for each step\n- No error handling → add \"if X fails, then Y\" branches\n\n### P3 — Readability Problems\n\n- Paragraphs too long → break into tables or shorter blocks\n- Repeated descriptions → deduplicate\n- No quick reference → add TL;DR or decision tree\n\nEvery P0 or P1 intervention that produces `Δ ≥ 5 points` must be added to `playbook.md` with a pattern signature.\n\n---\n\n## Usage Patterns\n\n### Full audit + optimization (first run)\n```\nUser: \"optimize all my skills\"\n→ Phases 0–5\n→ Agent suggests starting with 5 lowest baseline scores\n```\n\n### Single skill\n```\nUser: \"optimize my blog-writer skill\"\n→ Phases 0–5 limited to one skill\n```\n\n### Evaluation only (no edits)\n```\nUser: \"rate all my skills\"\n→ Phases 0–2 only\n→ Report includes confidence level per skill\n```\n\n### Calibration check\n```\nUser: \"is my rubric reliable?\"\n→ Phase 2 three-run variance only\n→ Reports SD per skill\n→ Flags which skills cannot be optimized under current rubric\n```\n\n### Playbook review\n```\nUser: \"what patterns work for my skills?\"\n→ Display playbook.md with frequency and average Δ per pattern\n```\n\n### Rollback\n```\nUser: \"revert optimization for \u003Cskill>\"\n→ git checkout main_branch\n→ git revert \u003Cmerge_commit>\n→ Restore from pre-evolution tag\n```\n\n---\n\n## What Makes This Different from Darwin \u002F autoresearch\n\n| Concern | autoresearch | darwin-skill | evolve-skill |\n|---------|-------------|--------------|--------------|\n| Measurement noise | Tolerable (loss is tight) | Unmeasured | **Gate 1 requires SD ≤ 2.0** |\n| Overfitting | Natural (val set) | Unprotected | **Train\u002Fholdout split, Phase 4 validation** |\n| Function drift | Not a concern (code) | Addressed by rule, not measurement | **Gate 3 semantic diff, 80% overlap minimum** |\n| Rubric stability | N\u002FA | \"Score 1–10\" | **Anchored 1\u002F5\u002F10 points per dimension** |\n| Keep threshold | `new_loss \u003C old_loss` | `new > old` | `Δ ≥ max(3, 2×SD)` |\n| Budget allocation | Compute-bound | Fixed 3 rounds | Multi-armed bandit |\n| Failed experiments | Discarded | `git revert` creates noise | **Preserved on branches** |\n| Learning across skills | N\u002FA | None | **Playbook crosstalk** |\n| Human oversight | None | Per-skill pause | Per-skill pause + holdout gate |\n\n---\n\n## Success Criteria for This Skill Itself\n\nThis skill is working correctly if, over a full optimization run:\n\n- Fewer than 20% of candidate changes are rejected by Gate 2 (effect-size too small) → anchors are useful\n- Fewer than 5% of candidate changes are rejected by Gate 3 (function drift) → the editor is disciplined\n- Holdout delta agrees with training delta direction ≥ 90% of the time → no significant overfitting\n- Playbook patterns reused in later runs have positive expected delta → crosstalk works\n\nThese are meta-metrics — tracked in `results.tsv` over time — that tell you whether the optimizer itself is healthy.\n\n---\n\n## Design Inspiration\n\n- **Karpathy, autoresearch** — autonomous experiment loops driven by a measurable objective\n- **darwin-skill (alchaincyf)** — applying the autoresearch pattern to SKILL.md optimization\n- **Inter-rater agreement research** — why anchored rubrics reduce scorer variance\n- **Train\u002Fvalidation\u002Ftest discipline from ML** — why holdouts matter\n\nThe contribution of this skill is not novelty. It is **measurement discipline**: the loss function (rubric) is treated as a first-class object that itself must be validated, not assumed.\n\n---\n\n## License\n\nMIT. Use, fork, modify, ship. Attribution appreciated but not required.\n","# Evolve Skill\n\n> A measurement-first skill optimizer. Inspired by Karpathy's `autoresearch` and the `darwin-skill` project, but rebuilt around one conviction: **if you cannot trust your measurement, you cannot trust your optimization.**\n>\n> Every decision this skill makes is gated by whether the measurement is reliable enough to justify it.\n\n---\n\n## What This Skill Is Not\n\nThis is not a formatting linter. This is not a prompt rewriter. This is not an autonomous agent that will rewrite your skills in the background.\n\nThis is a **disciplined experiment loop** with strict gates around scoring noise, overfitting, and function drift — the three failure modes that make most \"auto-optimize\" tools produce worse skills over time.\n\nIf you want a quick skill tidy-up, use a simpler tool. If you have 20+ skills and need to know which ones are actually improving, this is for you.\n\n---\n\n## Core Philosophy — The Three Gates\n\nEvery optimization cycle passes through three gates. If any gate fails, the cycle stops or rolls back.\n\n### Gate 1 — Measurement Stability (can we trust the score?)\n\nBefore any optimization runs, score the baseline skill **three independent times** using three separate sub-agents. Compute the standard deviation. If `SD > 2.0 points`, the rubric is too loose and no optimization may proceed until the rubric is tightened.\n\nA ratchet mechanism with unreliable measurement is not a ratchet — it is a random walk pretending to be progress.\n\n### Gate 2 — Effect Size (is this improvement real?)\n\nA keep decision requires `Δscore ≥ max(3, 2×SD)` where SD is the measurement noise from Gate 1. A skill that \"improved\" by 1.5 points when noise is ±2 points did not improve.\n\n### Gate 3 — Function Preservation (is it still the same skill?)\n\nBefore and after every change, extract the skill's core functions as a bullet list (via a sub-agent). If function overlap drops below 80%, the change is rejected regardless of score. A higher-scoring skill that silently lost a capability is worse, not better.\n\n---\n\n## The 9-Dimension Anchored Rubric (100 points)\n\nEach dimension is scored 1–10. Every dimension has three anchor points (1, 5, 10) defined in `rubric\u002Fanchored-rubric.md`. Scorers must reference the anchors and cite which anchor their score lies closest to. This is what stabilizes inter-rater agreement.\n\n**Structure (50 points)**\n\n| # | Dimension | Weight | What it measures |\n|---|-----------|--------|------------------|\n| 1 | Frontmatter quality | 8 | name format, description ≤1024 chars, triggers |\n| 2 | Workflow clarity | 12 | numbered steps, I\u002FO per step |\n| 3 | Edge-case coverage | 8 | fallbacks, error recovery |\n| 4 | Checkpoint design | 6 | human-in-loop gates before destructive actions |\n| 5 | Instruction specificity | 12 | concrete params, formats, examples |\n| 6 | Resource integration | 4 | references\u002Fscripts\u002Fassets resolve |\n\n**Effectiveness (40 points)**\n\n| # | Dimension | Weight | What it measures |\n|---|-----------|--------|------------------|\n| 7 | Architectural fit | 10 | non-redundant, matches ecosystem conventions |\n| 8 | Live test performance | 30 | output quality on held-out test prompts |\n\n**Localization (10 points)**\n\n| # | Dimension | Weight | What it measures |\n|---|-----------|--------|------------------|\n| 9 | Localization quality | 10 | triggers + examples in all declared languages, cross-platform paths |\n\n**Total:** `Σ(dimension_score × weight) \u002F 10`, capped at 100.\n\n**The effectiveness dimension (30 points) is the largest single weight** by design. A beautifully written skill that produces bad output is still a bad skill.\n\n---\n\n## Workflow\n\n### Phase 0 — Scope and Setup\n\n1. Determine scope:\n   - `optimize all skills` → scan `.claude\u002Fskills\u002F*\u002FSKILL.md` and skill directories\n   - `optimize \u003Cname>` → single skill\n   - `evaluate only` → Phases 0–2 only, no edits\n2. Create a dated branch: `evolve\u002FYYYYMMDD-HHMM`\n3. Initialize `results.tsv` and `playbook.md` if they do not exist\n4. Read existing `playbook.md` — known patterns will be tried first in Phase 3\n\n### Phase 1 — Test Prompt Design (Train \u002F Holdout Split)\n\nFor each skill, the user — not the agent — writes prompts. The agent may suggest, but suggestions must be confirmed.\n\n1. **Training prompts (3)** — seen during optimization, used to drive keep\u002Frevert decisions\n   - One happy-path prompt\n   - One complex or ambiguous prompt\n   - One edge-case prompt\n2. **Holdout prompts (2)** — never seen during optimization, used only for final validation\n   - Cover different scenarios than the training prompts\n3. Save to `\u003Cskill_dir>\u002Ftest-prompts.json` using the template in `templates\u002Ftest-prompts.template.json`\n4. Show both sets to the user. Confirm before proceeding.\n\n**Why both sets exist:** without holdouts, the optimization overfits to the training prompts and appears to improve while quietly getting worse on unseen inputs.\n\n### Phase 2 — Baseline Evaluation with Stability Check\n\nFor each skill:\n\n1. **Score three times independently** using three separate sub-agents with no shared context\n2. Compute mean and SD across the three runs\n3. If `SD > 2.0 points`:\n   - Flag the skill as `calibration_required`\n   - Skip optimization for this skill\n   - Report to user: \"Rubric is too loose for this skill. Tighten anchors before proceeding.\"\n4. If `SD ≤ 2.0`:\n   - Store `baseline_score = mean`, `baseline_sd = sd`\n   - Proceed to Phase 3\n\nFor the live-test dimension (30 points): run each training prompt through two sub-agents — one with the skill loaded, one without (baseline). Compare output quality against the anchors in the rubric.\n\nIf sub-agents are unavailable, set `confidence=low` and **do not** use the score for keep\u002Frevert decisions. A low-confidence score is diagnostic only.\n\n### Phase 3 — Optimization Loop\n\nBudget allocation follows a multi-armed bandit pattern, not round-robin:\n\n```\ninitial_budget_per_skill = 3 rounds\ntotal_budget = 30 rounds across 10 skills (configurable)\n\nround allocation:\n  if last round Δ ≥ 5 points:      +2 bonus rounds\n  if last round Δ between 0 and 3:  continue at baseline allocation\n  if last round Δ ≤ 0 (reverted):   forfeit remaining budget, move on\n  hot skills (3+ consecutive keeps) can use up to 10 rounds total\n```\n\nEach round:\n\n```\n1. Consult playbook.md:\n   \"Has a known pattern worked for a skill like this before?\"\n   If yes, try the known pattern first.\n\n2. If no known pattern, diagnose:\n   Find the lowest-scoring dimension.\n\n3. Generate exactly one targeted change:\n   - Which lines change\n   - Which rubric dimension this targets\n   - Predicted Δ score\n\n4. Execute on an experiment branch:\n   git checkout -b evolve\u002FYYYYMMDD-HHMM\u002F\u003Cskill>\u002Fexp-\u003CN>\n   edit SKILL.md\n   git commit -m \"exp: \u003Cskill> \u003Cdimension> \u003Csummary>\"\n\n5. Re-score with an INDEPENDENT sub-agent (never the editing agent):\n   - Run training prompts only (holdout stays sealed)\n   - Compute new_score and run Gate 3 (function preservation)\n\n6. Apply gates:\n   Gate 2: if new_score > old_score + max(3, 2×SD) → candidate keep\n   Gate 3: if function_overlap ≥ 80% → candidate keep\n   Otherwise → branch stays for history, move on\n\n7. On candidate keep:\n   git checkout main_branch\n   git merge --no-ff evolve\u002F...\u002Fexp-\u003CN>\n   update results.tsv with status=keep\n\n8. On reject:\n   leave the experiment branch in place (do not delete — it is training data)\n   update results.tsv with status=reject and reason code\n\n9. Log to results.tsv with every column filled\n```\n\nAfter a skill's budget is exhausted, insert a **human checkpoint**:\n- Show diff across all kept experiments\n- Show training-prompt output comparison\n- Show delta summary\n- Wait for explicit user approval\n\n### Phase 4 — Holdout Validation (Overfitting Check)\n\nThis runs once per skill, after Phase 3 completes and the user has approved.\n\n1. Run the **holdout prompts** (which have never been seen during optimization) against both the original and optimized versions\n2. Score only the live-test dimension (30 points)\n3. If `holdout_delta \u003C 0`:\n   - The optimization overfit to the training prompts\n   - Roll back the entire optimization for this skill\n   - Log as `overfit_detected` in results.tsv\n4. If `holdout_delta ≥ 0`:\n   - Optimization is validated\n   - Write any discovered patterns into `playbook.md`\n\n### Phase 5 — Playbook Update and Reporting\n\nFor every keep that produced `Δ ≥ 5 points`, add an entry to `playbook.md`:\n\n```markdown\n## Pattern: \u003Cshort name>\n- **Problem signature:** \u003Cwhat triggers this pattern's applicability>\n- **Intervention:** \u003Cthe specific edit>\n- **Measured impact:** \u003Cskill, Δ, n observations>\n- **Where it worked:** \u003Cskill names>\n- **Where it failed:** \u003Cskill names, if any>\n```\n\nPhase 3 of future runs will check this playbook before diagnosing fresh.\n\nFinal report includes:\n- Per-skill before\u002Fafter with confidence intervals\n- Holdout validation results\n- Playbook additions\n- Branches with failed experiments (for later review)\n\n---\n\n## Constraints\n\n1. **Core function is immutable.** Gate 3 enforces this. If a change removes a capability, it is rejected even if the score went up.\n2. **No new dependencies.** Do not add scripts, references, or external tools the original skill did not declare.\n3. **One dimension per experiment.** Multi-dimension changes make keep\u002Frevert decisions unattributable.\n4. **Size ceiling.** Optimized SKILL.md must be ≤150% of original size.\n5. **Git-first, rollback-safe.** Every change is on a branch. `git revert` is for merge commits only; failed branches are preserved.\n6. **Score independence.** The sub-agent that edits is never the sub-agent that scores. Enforced by session separation.\n7. **Confidence gating.** A `confidence=low` score cannot drive a keep decision under any circumstances.\n8. **Localization is a dimension, not an afterthought.** If a skill declares bilingual support, both languages are tested.\n\n---\n\n## File Layout\n\n```\n.claude\u002Fskills\u002Fevolve-skill\u002F\n├── SKILL.md                           # This file\n├── rubric\u002F\n│   ├── anchored-rubric.md             # Human-readable anchors\n│   └── rubric-v1.json                 # Machine-readable rubric\n├── scripts\u002F\n│   ├── calibrate.py                   # Test-retest variance\n│   ├── score.py                       # Apply rubric with anchors\n│   ├── semantic_diff.py               # Function preservation\n│   ├── ratchet.py                     # Keep\u002Frevert with CI\n│   └── allocate.py                    # Bandit budget allocation\n├── templates\u002F\n│   ├── test-prompts.template.json     # Train\u002Fholdout structure\n│   ├── result-card.html               # Optional visual card\n│   └── playbook.template.md           # Starter playbook\n├── examples\u002F\n│   ├── example-walkthrough.md         # Full optimization run\n│   └── example-results.tsv            # Real log format\n└── results.tsv                        # Created on first run\n```\n\nPer-skill files created during optimization:\n\n```\n\u003Ctarget_skill>\u002F\n├── SKILL.md\n├── test-prompts.json                  # Train + holdout split\n├── baseline-scores.json               # Three-run variance record\n└── optimization-log.md                # Per-skill narrative\n```\n\n---\n\n## results.tsv Schema\n\nTab-separated, append-only. Every row is an event.\n\n```\ntimestamp  branch  skill  old_score  new_score  delta  sd  status  dimension  intervention  function_overlap  holdout_delta  confidence  eval_mode  notes\n```\n\nColumn definitions:\n\n- `status` ∈ {`baseline`, `keep`, `reject_effect_size`, `reject_function_drift`, `reject_noisy`, `overfit_detected`}\n- `confidence` ∈ {`high`, `medium`, `low`} — derived from `eval_mode`\n- `eval_mode` ∈ {`full_test` (sub-agents with live prompts), `partial` (structure only), `simulation` (no sub-agent available)}\n- `function_overlap` — semantic-diff overlap percentage from Gate 3\n- `holdout_delta` — only populated on Phase 4 rows\n\n---\n\n## Intervention Playbook (Priority Order)\n\nWhen diagnosing the lowest-scoring dimension, consult this ranked list of proven interventions.\n\n### P0 — Effectiveness Problems (Live-Test Signals)\n\n- Output misses user intent → search SKILL.md for misleading instructions, prune\n- `with_skill` output worse than `baseline` → skill is over-constraining, simplify\n- Output format inconsistent → add explicit output schema with example\n\n### P1 — Structural Problems\n\n- Frontmatter trigger list too narrow → expand with synonyms and natural phrasings in all declared languages\n- Workflow lacks Phase\u002FStep structure → impose linear numbered flow\n- No human checkpoint before destructive actions → insert confirmation gate\n\n### P2 — Specificity Problems\n\n- Vague steps (\"process the image\") → replace with specific operation + parameters\n- Missing I\u002FO specs → add format, path, and example for each step\n- No error handling → add \"if X fails, then Y\" branches\n\n### P3 — Readability Problems\n\n- Paragraphs too long → break into tables or shorter blocks\n- Repeated descriptions → deduplicate\n- No quick reference → add TL;DR or decision tree\n\nEvery P0 or P1 intervention that produces `Δ ≥ 5 points` must be added to `playbook.md` with a pattern signature.\n\n---\n\n## Usage Patterns\n\n### Full audit + optimization (first run)\n```\nUser: \"optimize all my skills\"\n→ Phases 0–5\n→ Agent suggests starting with 5 lowest baseline scores\n```\n\n### Single skill\n```\nUser: \"optimize my blog-writer skill\"\n→ Phases 0–5 limited to one skill\n```\n\n### Evaluation only (no edits)\n```\nUser: \"rate all my skills\"\n→ Phases 0–2 only\n→ Report includes confidence level per skill\n```\n\n### Calibration check\n```\nUser: \"is my rubric reliable?\"\n→ Phase 2 three-run variance only\n→ Reports SD per skill\n→ Flags which skills cannot be optimized under current rubric\n```\n\n### Playbook review\n```\nUser: \"what patterns work for my skills?\"\n→ Display playbook.md with frequency and average Δ per pattern\n```\n\n### Rollback\n```\nUser: \"revert optimization for \u003Cskill>\"\n→ git checkout main_branch\n→ git revert \u003Cmerge_commit>\n→ Restore from pre-evolution tag\n```\n\n---\n\n## What Makes This Different from Darwin \u002F autoresearch\n\n| Concern | autoresearch | darwin-skill | evolve-skill |\n|---------|-------------|--------------|--------------|\n| Measurement noise | Tolerable (loss is tight) | Unmeasured | **Gate 1 requires SD ≤ 2.0** |\n| Overfitting | Natural (val set) | Unprotected | **Train\u002Fholdout split, Phase 4 validation** |\n| Function drift | Not a concern (code) | Addressed by rule, not measurement | **Gate 3 semantic diff, 80% overlap minimum** |\n| Rubric stability | N\u002FA | \"Score 1–10\" | **Anchored 1\u002F5\u002F10 points per dimension** |\n| Keep threshold | `new_loss \u003C old_loss` | `new > old` | `Δ ≥ max(3, 2×SD)` |\n| Budget allocation | Compute-bound | Fixed 3 rounds | Multi-armed bandit |\n| Failed experiments | Discarded | `git revert` creates noise | **Preserved on branches** |\n| Learning across skills | N\u002FA | None | **Playbook crosstalk** |\n| Human oversight | None | Per-skill pause | Per-skill pause + holdout gate |\n\n---\n\n## Success Criteria for This Skill Itself\n\nThis skill is working correctly if, over a full optimization run:\n\n- Fewer than 20% of candidate changes are rejected by Gate 2 (effect-size too small) → anchors are useful\n- Fewer than 5% of candidate changes are rejected by Gate 3 (function drift) → the editor is disciplined\n- Holdout delta agrees with training delta direction ≥ 90% of the time → no significant overfitting\n- Playbook patterns reused in later runs have positive expected delta → crosstalk works\n\nThese are meta-metrics — tracked in `results.tsv` over time — that tell you whether the optimizer itself is healthy.\n\n---\n\n## Design Inspiration\n\n- **Karpathy, autoresearch** — autonomous experiment loops driven by a measurable objective\n- **darwin-skill (alchaincyf)** — applying the autoresearch pattern to SKILL.md optimization\n- **Inter-rater agreement research** — why anchored rubrics reduce scorer variance\n- **Train\u002Fvalidation\u002Ftest discipline from ML** — why holdouts matter\n\nThe contribution of this skill is not novelty. It is **measurement discipline**: the loss function (rubric) is treated as a first-class object that itself must be validated, not assumed.\n\n---\n\n## License\n\nMIT. Use, fork, modify, ship. Attribution appreciated but not required.\n",{"name":5,"risk":24,"domain":25,"source":26,"license":15,"version":30,"category":25,"platforms":31,"subdomain":38,"description":7},"low","agent-evaluation",{"commit":27,"license":15,"repository":28,"imported_at":29},"8d24739a45","taneltaluri\u002Fevolve-skill","2026-09-20","1.0.0",[32,33,34,35,36,37],"claude-code","cursor","codex","gemini","antigravity","copilot","general","main","2d1750717747097b602acf30bd09a7cbd4b4c6ea","awesome_skills\u002Fagent-evaluation\u002Ftemp_clone_p4","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous","2026-11-09 11:45:36",{"source":8,"skillName":5,"commands":46,"prompt":56,"zipAvailable":57},[47,50,53],{"tool":48,"command":49},"npx","npx skills add Mr-Nobody-Anonymous\u002FAll-skills --skill evolve-skill",{"tool":51,"command":52},"bunx","bunx skills add Mr-Nobody-Anonymous\u002FAll-skills --skill evolve-skill",{"tool":54,"command":55},"pnpm","pnpm dlx skills add Mr-Nobody-Anonymous\u002FAll-skills --skill evolve-skill","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fagent-evaluation\u002Ftemp_clone_p4\u002FSKILL.md",true,[],[60,68,75,82,89,96],{"skillNo":61,"name":62,"nameZh":6,"description":63,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":64,"categories":65,"keywords":66,"publishedAt":67},"01M7FFFZPC2025JQQHPHDX154J","dei-assessment","Assess diversity, equity, and inclusion with metrics. TRIGGERS - Use when user needs help with dei-assessment related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fdei-assessment\u002FSKILL.md",[],[],"2026-11-16 10:05:34",{"skillNo":69,"name":70,"nameZh":6,"description":71,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":72,"categories":73,"keywords":74,"publishedAt":67},"01M7FFFWN8H4WA7A6Y398QKB6J","dei-action-plan","Create DEI action plans with goals and initiatives. TRIGGERS - Use when user needs help with dei-action-plan related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fdei-action-plan\u002FSKILL.md",[],[],{"skillNo":76,"name":77,"nameZh":6,"description":78,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":79,"categories":80,"keywords":81,"publishedAt":67},"01M7FFFSHTM0QZRVZ2NRJTJ0PY","culture-fit-assessment","Design culture add assessment processes. TRIGGERS - Use when user needs help with culture-fit-assessment related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fculture-fit-assessment\u002FSKILL.md",[],[],{"skillNo":83,"name":84,"nameZh":6,"description":85,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":86,"categories":87,"keywords":88,"publishedAt":67},"01M7FFFMA12RKEQA38KHJSXXPS","cowork-to-code-bridge","Use an already-installed, independently verified cowork-to-code bridge to run narrowly approved actions on the user's own macOS, Linux, or WSL2 machine through a local file queue.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fcowork-to-code-bridge\u002FSKILL.md",[],[],{"skillNo":90,"name":91,"nameZh":6,"description":92,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":93,"categories":94,"keywords":95,"publishedAt":67},"01M7FFFH1F6M8YRGW5V487GH1K","counter-offer-response","Draft counter-offer responses for retention. TRIGGERS - Use when user needs help with counter-offer-response related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fcounter-offer-response\u002FSKILL.md",[],[],{"skillNo":97,"name":98,"nameZh":6,"description":99,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":100,"categories":101,"keywords":102,"publishedAt":67},"01M7FFFDGEHV8Y7J9N40HZXW52","continuous-feedback-system","Design continuous feedback systems. TRIGGERS - Use when user needs help with continuous-feedback-system related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fcontinuous-feedback-system\u002FSKILL.md",[],[]]