[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M71AX3ZJ10D1BN7JACEASMAA":3,"skill-same:01M71AX3ZJ10D1BN7JACEASMAA":84},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":43,"linkOnly":44,"markdownContent":45,"markdownBody":46,"frontMatter":47,"refName":54,"commitSha":55,"skillDirectory":56,"forksCount":57,"repositoryHtmlUrl":58,"repositoryHomepageUrl":59,"ownerHtmlUrl":60,"ownerBlogUrl":61,"ownerDescription":62,"updatedAt":63,"install":64,"resources":77},"01M71AX3ZJ10D1BN7JACEASMAA","eval",null,"Measure whether a change to AI behavior actually improved things, the way a\ncode change is measured: binary quality dimensions, a grid-sampled synthetic\neval set with negative controls and a generated response side, a hand-labeled\nanswer key, a judge written like code, then judge calibration with\ntoolkit\u002Fcalibrate.py (TP\u002FTN\u002FFP\u002FFN counts, TPR, TNR, accuracy, precision, and\na harsh\u002Flenient bias reading) before gating on thresholds set before the run.\nUse when the user changed AI behavior (prompt wording, a skill description or\nbody, an agent loop, a RAG pipeline) and needs to know whether it helped, or\nasks to \"build an eval set\", \"calibrate a judge\", or \"evaluate the prompt\".\nNot the harness command `claude plugin eval`, and not a substitute for\nordinary code tests: those check deterministic code, this measures judgment\nbehavior.","ariadoss\u002Fsuperskills","ariadoss","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F414341?v=4","01M71AC7383PT6JCRZSQJ4AVTM","Danilo Stern-Sapad","01M71AC73Y7NTBTJ9KQYNENRRE",12,"MIT","https:\u002F\u002Fgithub.com\u002Fariadoss\u002Fsuperskills\u002Fblob\u002Fmain\u002Fskills\u002Feval\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[5,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42],"hand","write","read","rag","run","command","change","plugin eval","dimensions","synthetic","code","Bash","when","gating","loop","generated","rag pipeline","pipeline","SET","2026-11-10 10:12:30",false,"---\nname: eval\ndescription: |\n  Measure whether a change to AI behavior actually improved things, the way a\n  code change is measured: binary quality dimensions, a grid-sampled synthetic\n  eval set with negative controls and a generated response side, a hand-labeled\n  answer key, a judge written like code, then judge calibration with\n  toolkit\u002Fcalibrate.py (TP\u002FTN\u002FFP\u002FFN counts, TPR, TNR, accuracy, precision, and\n  a harsh\u002Flenient bias reading) before gating on thresholds set before the run.\n  Use when the user changed AI behavior (prompt wording, a skill description or\n  body, an agent loop, a RAG pipeline) and needs to know whether it helped, or\n  asks to \"build an eval set\", \"calibrate a judge\", or \"evaluate the prompt\".\n  Not the harness command `claude plugin eval`, and not a substitute for\n  ordinary code tests: those check deterministic code, this measures judgment\n  behavior.\nallowed-tools:\n  - Bash\n  - Read\n  - Write\n  - Edit\n  - Grep\n---\n\n# Eval: measure AI behavior changes like code changes\n\nA change to AI behavior (prompt wording, a skill's description or body, an\nagent loop, a RAG pipeline) is a code change with no compiler: without a\nmeasurement you cannot tell help from harm, and neither can a reviewer. This\nskill runs the loop **Generate, Run, Evaluate, Analyze, Improve**, repeated:\nevaluation is a cycle, not an event, and the goal is steady visible movement,\nnot a perfect score in one pass. Measure **before** changing anything, or you\ncannot confirm the change helped.\n\nThe toolkit does the arithmetic. **You, the agent, make the judgment calls**:\nwhich dimensions matter, which grid cells to sample, what each label means,\nwhere the rubric wobbles, and whether a failure is a wording bug or a\nstructure bug. Those calls are the skill; the script only keeps the numbers\nhonest.\n\n`TOOLKIT` below means the `toolkit\u002F` directory inside this skill's own\ndirectory. The host prints that directory when the skill loads (Claude Code:\n\"Base directory for this skill: …\"); use it. Do **not** derive it from `$0`:\nin your shell that names the shell, not this file. Confirm the path before\nrunning anything, so a wrong one fails loudly instead of phase 5 running\nnothing:\n\n```bash\nTOOLKIT=\"\u003Cbase directory for this skill>\u002Ftoolkit\"\n# Fallback, only if the host printed no base directory. Never the shell's cwd:\n# that would run whatever toolkit\u002F happens to sit there.\nfor d in \"$TOOLKIT\" \"$HOME\u002F.claude\u002Fskills\u002Feval\u002Ftoolkit\"; do\n  [ -f \"$d\u002Fcalibrate.py\" ] && { TOOLKIT=\"$d\"; break; }\ndone\n[ -f \"$TOOLKIT\u002Fcalibrate.py\" ] || { echo \"eval toolkit not found\" >&2; exit 1; }\n```\n\n## Prerequisites\n\nPython 3, standard library only. Nothing to install. The toolkit never calls\na model: judging is the judge prompt's job, calibration is pure arithmetic.\n\n## Phases\n\n**1. Define \"good\" as binary dimensions.**\n\nPick the dimensions the change is supposed to move (selection, safety,\ncorrectness, outcome) or the project's own, and define each one as a binary\nlabel: pass\u002Ffail, resolved\u002Fnot-resolved. Graded scales multiply the\ndisagreement surface before you can even measure it, so stay binary until the\ntaxonomy is stable. Decide, in writing, which label is the **positive**\nclass for reporting; FAIL is the usual choice when judging quality, because a\nfalse positive then means \"the judge failed a run the human passed\", which is\nthe harsh direction. State the positive class next to every number you\nreport: swapping it swaps TPR and TNR and changes what precision means.\n\n**2. Build the eval set on a declared grid, not at random.**\n\nAsking a model for \"100 random questions\" produces repetitive, obvious cases\nclustered around the easy middle. Sample on a deliberate grid instead:\n\n- **intent × persona × complexity**: every cell that makes sense gets cases;\n- **negative controls are part of the plan**, not an afterthought: requests\n  that should *not* trigger the behavior, including near-miss neighbors;\n- **edge cases on purpose**: ambiguity, misspellings, multi-turn context,\n  adversarial inputs. Testing only typical inputs means production failures\n  are discovered by users first;\n- **mix synthetic and real traffic** where you have real traffic; synthetic\n  data speeds iteration but must reflect the real distribution.\n\nGenerate the **response side too**: for each case, produce responses of each\nkind (complete, partial, workaround, escalation, vague\u002Fincorrect) so the\njudge is tested on every grade, not just the easy passes. Decide up front how\nworkarounds, escalations and partial answers map onto the binary labels; a\nbinary judge does not need a label per response type.\n\nTwo hygiene rules. Keep eval data unseen: never reuse evaluation cases as\nfew-shot examples in the thing being graded, that is data leakage. And one\nobserved failure is a class: when a case fails, generate more cases like it\nand fix the class, never a hard-coded fix for the one case you saw. Check the\ngenerated set before trusting it (quality, diversity, realism, coverage\ngaps); a judge calibrated on a narrow set is only calibrated for that narrow\nset.\n\n**3. Hand-label a representative subset: the answer key.**\n\nLabel a representative subset of runs by hand. This subset is the ground\ntruth every judge metric is computed against, so it must span the grid,\nincluding the awkward cells. Keep the labels binary per phase 1. After the\njudge runs, the calibration file is one pair per line, human first:\n\n```json\n{\"human\": \"FAIL\", \"judge\": \"PASS\"}\n```\n\n**4. Write the judge like code.**\n\nThe judge is a prompt with a job description, and it is writable, testable\nand improvable exactly like code:\n\n1. **Exact label definitions.** What counts as pass? Does pass require a\n   complete fix, a workaround, or a correct escalation?\n2. **Boundary cases decided in advance**: the accurate-but-jargon-heavy\n   answer, the correct redirect of an out-of-scope request, the partial\n   resolution that needs one more user reply. Write the ruling into the\n   rubric before running, not after seeing the judge wobble.\n3. **Few-shot examples with reasoning**: a positive, a negative, and *why*\n   each is what it is. The reasoning is the part that transfers.\n4. **Low temperature (≤ 0.2)** so judgments repeat.\n5. **Structured output**: JSON with a label and a one-line reason, so grading\n   is a parser, not a string hunt.\n\n```text\nYou grade whether the response resolved the user's issue.\nLabel definitions: PASS means ...; FAIL means ...\nBoundary cases: a correct workaround is PASS; an accurate but jargon-heavy\nanswer is PASS; an out-of-scope request correctly redirected is PASS; ...\n\nExample 1 (PASS, with reasoning): ...\nExample 2 (FAIL, with reasoning): ...\n\nUser issue: {case}\nResponse: {response}\nAnswer with JSON only: {\"label\": \"PASS\"|\"FAIL\", \"reason\": \"...\"}\n```\n\n**5. Calibrate the judge with the toolkit.**\n\nBefore trusting the judge at scale, measure the judge itself against the\nanswer key. Run the judge over the labeled subset, write the human\u002Fjudge\npairs to a file (a two-column CSV, human first, also reads), then:\n\n```bash\npython3 \"$TOOLKIT\u002Fcalibrate.py\" labels.jsonl                  # FAIL positive, the usual\npython3 \"$TOOLKIT\u002Fcalibrate.py\" labels.jsonl --positive PASS  # the swapped convention\n```\n\nYou get the confusion counts (TP\u002FTN\u002FFP\u002FFN), TPR (recall), TNR\n(specificity), accuracy, precision, all at 4 decimals, and a one-line bias\nreading: harsh when the judge fails good work more than it waves bad work\nthrough, lenient when the reverse, computed from the two error rates for the\npositive class you declared.\n\nSanity anchor, the method's worked example. 80 human-labeled runs, 50 FAIL\nand 30 PASS; the judge fails 44 of the 50 and passes 21 of the 30. Then\nTP=44 TN=21 FP=9 FN=6, TPR 0.8800, TNR 0.7000, accuracy 0.8125, precision\n0.8302, and the reading is harsh: wrongly failed 9\u002F30 (0.3000) versus waved\nthrough 6\u002F50 (0.1200). Run the same input through the tool once when you set\nup; if your numbers disagree with this anchor, the input or the tool is\nwrong, not the method.\n\nHow to read the output:\n\n- **Report TPR, TNR, accuracy and precision together, never accuracy\n  alone.** Accuracy is blind to error direction: 85% correct could be a\n  judge that fails good work or one that waves bad work through.\n- **A wide TPR-TNR gap is directional bias.** Name the direction from the\n  error rates, not a hunch: the wrongly-failed share versus the waved-through\n  share. The bias line does this for the declared positive class; declaring\n  the other class swaps TPR and TNR but must not flip the verdict.\n- **At small n the counts are the signal.** TPR 0.88 over 50 labeled runs is\n  a measurement; over 6 it is noise dressed up as a percentage.\n- **Every disagreement is a rubric bug before it is a model bug.** Sharpen\n  the label definitions, add a few-shot aimed at the direction it errs in,\n  re-judge, re-measure. Do not ship a judge whose TPR and TNR disagree\n  widely.\n\n**6. Gate and iterate, thresholds first.**\n\nSet pass thresholds **before** looking at results (choosing them after is how\nA\u002FB tests lie), keep a rollback path, and feed observed failures back as new\neval cases. The suite grows toward the stopping rule: add cases until new\ntraces stop producing new failure modes.\n\nKeep two dashboards separate. **Judge metrics** (phase 5) measure the judge\nagainst humans; a judge can agree perfectly with humans about a system that\nis failing. **System metrics** measure the thing you shipped: pass rate per\ndimension, failure rate per category, percent resolved per issue type, plus\nbusiness outcomes (execution rate, escalation rate, latency). Compute system\nmetrics per category while iterating, not only globally; an aggregate can\nhold still while a category burns. For agents, judge outcome plus process\nsanity, never step sequences: different valid paths reach the same answer, so\ngrade decision quality and final-output correctness, with budgets (max steps,\ncost, timeout) and tool-error counts in the report. For RAG, retrieval\nmetrics are leading indicators (Recall@k, Precision@k, MRR) before touching\ngeneration; fix retrieval first, a faithful generator over a broken retriever\nstill fails.\n\nWhen a gate fails, fix **wording before structure**: sharper label\ndefinitions, a better few-shot, a clearer boundary ruling. Change structure\nonly if the measured category rate does not move after the wording fix.\n\n## Report\n\nEnd with evidence, not a verdict word: the eval-set shape (grid cells,\nnegative controls, counts), the answer-key size, the judge's four metrics\nwith the positive class named, the bias reading with counts, the thresholds\nthat were set beforehand, and which gates passed or failed. Name the top\nfailure classes observed and what was fed back as new cases.\n\n## Never\n\n- Never report accuracy alone.\n- Never set or adjust thresholds after seeing the results.\n- Never reuse eval cases as few-shot examples in the thing being graded.\n- Never hard-code a fix for the one case you saw; one observed failure is a\n  class.\n- Never let the judge see the human label, and never let the judge's own past\n  output stand in for the answer key.\n\n## Further reading\n\n`evals\u002FMETHODOLOGY.md` in the superskills repo is the long-form method this\nskill operationalizes; that path does not exist inside a standalone export of\nthis skill, which is why this file is self-contained.\n","# Eval: measure AI behavior changes like code changes\n\nA change to AI behavior (prompt wording, a skill's description or body, an\nagent loop, a RAG pipeline) is a code change with no compiler: without a\nmeasurement you cannot tell help from harm, and neither can a reviewer. This\nskill runs the loop **Generate, Run, Evaluate, Analyze, Improve**, repeated:\nevaluation is a cycle, not an event, and the goal is steady visible movement,\nnot a perfect score in one pass. Measure **before** changing anything, or you\ncannot confirm the change helped.\n\nThe toolkit does the arithmetic. **You, the agent, make the judgment calls**:\nwhich dimensions matter, which grid cells to sample, what each label means,\nwhere the rubric wobbles, and whether a failure is a wording bug or a\nstructure bug. Those calls are the skill; the script only keeps the numbers\nhonest.\n\n`TOOLKIT` below means the `toolkit\u002F` directory inside this skill's own\ndirectory. The host prints that directory when the skill loads (Claude Code:\n\"Base directory for this skill: …\"); use it. Do **not** derive it from `$0`:\nin your shell that names the shell, not this file. Confirm the path before\nrunning anything, so a wrong one fails loudly instead of phase 5 running\nnothing:\n\n```bash\nTOOLKIT=\"\u003Cbase directory for this skill>\u002Ftoolkit\"\n# Fallback, only if the host printed no base directory. Never the shell's cwd:\n# that would run whatever toolkit\u002F happens to sit there.\nfor d in \"$TOOLKIT\" \"$HOME\u002F.claude\u002Fskills\u002Feval\u002Ftoolkit\"; do\n  [ -f \"$d\u002Fcalibrate.py\" ] && { TOOLKIT=\"$d\"; break; }\ndone\n[ -f \"$TOOLKIT\u002Fcalibrate.py\" ] || { echo \"eval toolkit not found\" >&2; exit 1; }\n```\n\n## Prerequisites\n\nPython 3, standard library only. Nothing to install. The toolkit never calls\na model: judging is the judge prompt's job, calibration is pure arithmetic.\n\n## Phases\n\n**1. Define \"good\" as binary dimensions.**\n\nPick the dimensions the change is supposed to move (selection, safety,\ncorrectness, outcome) or the project's own, and define each one as a binary\nlabel: pass\u002Ffail, resolved\u002Fnot-resolved. Graded scales multiply the\ndisagreement surface before you can even measure it, so stay binary until the\ntaxonomy is stable. Decide, in writing, which label is the **positive**\nclass for reporting; FAIL is the usual choice when judging quality, because a\nfalse positive then means \"the judge failed a run the human passed\", which is\nthe harsh direction. State the positive class next to every number you\nreport: swapping it swaps TPR and TNR and changes what precision means.\n\n**2. Build the eval set on a declared grid, not at random.**\n\nAsking a model for \"100 random questions\" produces repetitive, obvious cases\nclustered around the easy middle. Sample on a deliberate grid instead:\n\n- **intent × persona × complexity**: every cell that makes sense gets cases;\n- **negative controls are part of the plan**, not an afterthought: requests\n  that should *not* trigger the behavior, including near-miss neighbors;\n- **edge cases on purpose**: ambiguity, misspellings, multi-turn context,\n  adversarial inputs. Testing only typical inputs means production failures\n  are discovered by users first;\n- **mix synthetic and real traffic** where you have real traffic; synthetic\n  data speeds iteration but must reflect the real distribution.\n\nGenerate the **response side too**: for each case, produce responses of each\nkind (complete, partial, workaround, escalation, vague\u002Fincorrect) so the\njudge is tested on every grade, not just the easy passes. Decide up front how\nworkarounds, escalations and partial answers map onto the binary labels; a\nbinary judge does not need a label per response type.\n\nTwo hygiene rules. Keep eval data unseen: never reuse evaluation cases as\nfew-shot examples in the thing being graded, that is data leakage. And one\nobserved failure is a class: when a case fails, generate more cases like it\nand fix the class, never a hard-coded fix for the one case you saw. Check the\ngenerated set before trusting it (quality, diversity, realism, coverage\ngaps); a judge calibrated on a narrow set is only calibrated for that narrow\nset.\n\n**3. Hand-label a representative subset: the answer key.**\n\nLabel a representative subset of runs by hand. This subset is the ground\ntruth every judge metric is computed against, so it must span the grid,\nincluding the awkward cells. Keep the labels binary per phase 1. After the\njudge runs, the calibration file is one pair per line, human first:\n\n```json\n{\"human\": \"FAIL\", \"judge\": \"PASS\"}\n```\n\n**4. Write the judge like code.**\n\nThe judge is a prompt with a job description, and it is writable, testable\nand improvable exactly like code:\n\n1. **Exact label definitions.** What counts as pass? Does pass require a\n   complete fix, a workaround, or a correct escalation?\n2. **Boundary cases decided in advance**: the accurate-but-jargon-heavy\n   answer, the correct redirect of an out-of-scope request, the partial\n   resolution that needs one more user reply. Write the ruling into the\n   rubric before running, not after seeing the judge wobble.\n3. **Few-shot examples with reasoning**: a positive, a negative, and *why*\n   each is what it is. The reasoning is the part that transfers.\n4. **Low temperature (≤ 0.2)** so judgments repeat.\n5. **Structured output**: JSON with a label and a one-line reason, so grading\n   is a parser, not a string hunt.\n\n```text\nYou grade whether the response resolved the user's issue.\nLabel definitions: PASS means ...; FAIL means ...\nBoundary cases: a correct workaround is PASS; an accurate but jargon-heavy\nanswer is PASS; an out-of-scope request correctly redirected is PASS; ...\n\nExample 1 (PASS, with reasoning): ...\nExample 2 (FAIL, with reasoning): ...\n\nUser issue: {case}\nResponse: {response}\nAnswer with JSON only: {\"label\": \"PASS\"|\"FAIL\", \"reason\": \"...\"}\n```\n\n**5. Calibrate the judge with the toolkit.**\n\nBefore trusting the judge at scale, measure the judge itself against the\nanswer key. Run the judge over the labeled subset, write the human\u002Fjudge\npairs to a file (a two-column CSV, human first, also reads), then:\n\n```bash\npython3 \"$TOOLKIT\u002Fcalibrate.py\" labels.jsonl                  # FAIL positive, the usual\npython3 \"$TOOLKIT\u002Fcalibrate.py\" labels.jsonl --positive PASS  # the swapped convention\n```\n\nYou get the confusion counts (TP\u002FTN\u002FFP\u002FFN), TPR (recall), TNR\n(specificity), accuracy, precision, all at 4 decimals, and a one-line bias\nreading: harsh when the judge fails good work more than it waves bad work\nthrough, lenient when the reverse, computed from the two error rates for the\npositive class you declared.\n\nSanity anchor, the method's worked example. 80 human-labeled runs, 50 FAIL\nand 30 PASS; the judge fails 44 of the 50 and passes 21 of the 30. Then\nTP=44 TN=21 FP=9 FN=6, TPR 0.8800, TNR 0.7000, accuracy 0.8125, precision\n0.8302, and the reading is harsh: wrongly failed 9\u002F30 (0.3000) versus waved\nthrough 6\u002F50 (0.1200). Run the same input through the tool once when you set\nup; if your numbers disagree with this anchor, the input or the tool is\nwrong, not the method.\n\nHow to read the output:\n\n- **Report TPR, TNR, accuracy and precision together, never accuracy\n  alone.** Accuracy is blind to error direction: 85% correct could be a\n  judge that fails good work or one that waves bad work through.\n- **A wide TPR-TNR gap is directional bias.** Name the direction from the\n  error rates, not a hunch: the wrongly-failed share versus the waved-through\n  share. The bias line does this for the declared positive class; declaring\n  the other class swaps TPR and TNR but must not flip the verdict.\n- **At small n the counts are the signal.** TPR 0.88 over 50 labeled runs is\n  a measurement; over 6 it is noise dressed up as a percentage.\n- **Every disagreement is a rubric bug before it is a model bug.** Sharpen\n  the label definitions, add a few-shot aimed at the direction it errs in,\n  re-judge, re-measure. Do not ship a judge whose TPR and TNR disagree\n  widely.\n\n**6. Gate and iterate, thresholds first.**\n\nSet pass thresholds **before** looking at results (choosing them after is how\nA\u002FB tests lie), keep a rollback path, and feed observed failures back as new\neval cases. The suite grows toward the stopping rule: add cases until new\ntraces stop producing new failure modes.\n\nKeep two dashboards separate. **Judge metrics** (phase 5) measure the judge\nagainst humans; a judge can agree perfectly with humans about a system that\nis failing. **System metrics** measure the thing you shipped: pass rate per\ndimension, failure rate per category, percent resolved per issue type, plus\nbusiness outcomes (execution rate, escalation rate, latency). Compute system\nmetrics per category while iterating, not only globally; an aggregate can\nhold still while a category burns. For agents, judge outcome plus process\nsanity, never step sequences: different valid paths reach the same answer, so\ngrade decision quality and final-output correctness, with budgets (max steps,\ncost, timeout) and tool-error counts in the report. For RAG, retrieval\nmetrics are leading indicators (Recall@k, Precision@k, MRR) before touching\ngeneration; fix retrieval first, a faithful generator over a broken retriever\nstill fails.\n\nWhen a gate fails, fix **wording before structure**: sharper label\ndefinitions, a better few-shot, a clearer boundary ruling. Change structure\nonly if the measured category rate does not move after the wording fix.\n\n## Report\n\nEnd with evidence, not a verdict word: the eval-set shape (grid cells,\nnegative controls, counts), the answer-key size, the judge's four metrics\nwith the positive class named, the bias reading with counts, the thresholds\nthat were set beforehand, and which gates passed or failed. Name the top\nfailure classes observed and what was fed back as new cases.\n\n## Never\n\n- Never report accuracy alone.\n- Never set or adjust thresholds after seeing the results.\n- Never reuse eval cases as few-shot examples in the thing being graded.\n- Never hard-code a fix for the one case you saw; one observed failure is a\n  class.\n- Never let the judge see the human label, and never let the judge's own past\n  output stand in for the answer key.\n\n## Further reading\n\n`evals\u002FMETHODOLOGY.md` in the superskills repo is the long-form method this\nskill operationalizes; that path does not exist inside a standalone export of\nthis skill, which is why this file is self-contained.\n",{"name":5,"description":48,"allowed-tools":49},"Measure whether a change to AI behavior actually improved things, the way a\ncode change is measured: binary quality dimensions, a grid-sampled synthetic\neval set with negative controls and a generated response side, a hand-labeled\nanswer key, a judge written like code, then judge calibration with\ntoolkit\u002Fcalibrate.py (TP\u002FTN\u002FFP\u002FFN counts, TPR, TNR, accuracy, precision, and\na harsh\u002Flenient bias reading) before gating on thresholds set before the run.\nUse when the user changed AI behavior (prompt wording, a skill description or\nbody, an agent loop, a RAG pipeline) and needs to know whether it helped, or\nasks to \"build an eval set\", \"calibrate a judge\", or \"evaluate the prompt\".\nNot the harness command `claude plugin eval`, and not a substitute for\nordinary code tests: those check deterministic code, this measures judgment\nbehavior.\n",[35,50,51,52,53],"Read","Write","Edit","Grep","main","c7ec8d78e5dc37484c2cfe90ab04de163d97dd41","skills\u002Feval",0,"https:\u002F\u002Fgithub.com\u002Fariadoss\u002Fsuperskills","https:\u002F\u002Fhyperion360.com\u002Fblog\u002Fparallel-ai-agents-engineering-workflow\u002F","https:\u002F\u002Fgithub.com\u002Fariadoss","https:\u002F\u002Fdanilosapad.com\u002F","CEO & Founder of @Hyperion360","2026-11-01 12:36:50",{"source":8,"skillName":5,"commands":65,"prompt":75,"zipAvailable":76},[66,69,72],{"tool":67,"command":68},"npx","npx skills add ariadoss\u002Fsuperskills --skill eval",{"tool":70,"command":71},"bunx","bunx skills add ariadoss\u002Fsuperskills --skill eval",{"tool":73,"command":74},"pnpm","pnpm dlx skills add ariadoss\u002Fsuperskills --skill eval","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fariadoss\u002Fsuperskills\u002Fblob\u002Fmain\u002Fskills\u002Feval\u002FSKILL.md",true,[78],{"path":79,"resourceType":80,"resourceTypeDesc":81,"sizeBytes":82,"sourceHtmlUrl":83},"toolkit\u002Fcalibrate.py","SCRIPT","脚本",9652,"https:\u002F\u002Fgithub.com\u002Fariadoss\u002Fsuperskills\u002Fblob\u002Fmain\u002Fskills\u002Feval\u002Ftoolkit\u002Fcalibrate.py",[85,108,132,151,178,194],{"skillNo":86,"name":87,"nameZh":6,"description":88,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":89,"categories":90,"keywords":94,"publishedAt":43},"01M71AZE3MPF891RM354EMRYA1","youtube","Download YouTube video transcripts when user provides a YouTube URL or asks to download\u002Fget\u002Ffetch a transcript from YouTube. Also use when user wants to transcribe or get captions\u002Fsubtitles from a YouTube video.","https:\u002F\u002Fgithub.com\u002Fariadoss\u002Fsuperskills\u002Fblob\u002Fmain\u002Fskills\u002Fyoutube\u002FSKILL.md",[91],{"categoryNo":92,"categoryName":93,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000016","图像与多媒体",[87,95,96,97,25,98,99,35,100,36,101,102,103,104,26,105,106,107],"user","download","transcribe","transcript","video transcripts","URL","Captions","get","fetch","Video","use","subtitles","transcripts",{"skillNo":109,"name":110,"nameZh":6,"description":111,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":112,"categories":113,"keywords":114,"publishedAt":43},"01M71AZAYR85PW6BJGSQXS4BN2","write-plan","Use when you have a spec or requirements for a multi-step task, before touching code. Produces an engineering plan complete with TDD tasks, DRY\u002FSOLID\u002FYAGNI principles, a required Test Plan & Verification section, and a coverage target — then auto-runs \u002Fplan-eng-review as an independent double-check before execution.","https:\u002F\u002Fgithub.com\u002Fariadoss\u002Fsuperskills\u002Fblob\u002Fmain\u002Fskills\u002Fwrite-plan\u002FSKILL.md",[],[115,116,25,117,118,119,120,34,121,36,122,123,124,125,126,127,128,129,130,131],"write plan","plan","dry","verification","review","tasks","you","auto","eng","coverage","complete","section","for","test plan","target","as","yagni",{"skillNo":133,"name":134,"nameZh":6,"description":135,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":136,"categories":137,"keywords":138,"publishedAt":43},"01M71AZ8NDDP5ZMGEE6FSC4FKR","worktrees","Use when starting feature work that needs isolation from current workspace or before executing implementation plans - creates isolated git worktrees with smart directory selection and safety verification","https:\u002F\u002Fgithub.com\u002Fariadoss\u002Fsuperskills\u002Fblob\u002Fmain\u002Fskills\u002Fworktrees\u002FSKILL.md",[],[134,139,140,141,105,142,143,144,145,146,147,118,148,36,149,150],"implementation","workspace","with","plans","feature","isolation","git","directory","work","git worktrees","selection","Safety",{"skillNo":152,"name":153,"nameZh":6,"description":154,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":155,"categories":156,"keywords":158,"publishedAt":43},"01M71AZ4ZQHTNWFRZVYHKQFP08","web-perf","Analyzes web performance using Chrome DevTools MCP. Measures Core Web Vitals (LCP, INP, CLS) and supplementary metrics (FCP, TBT, Speed Index), identifies render-blocking resources, network dependency chains, layout shifts, caching issues, and accessibility gaps. Use when asked to audit, profile, debug, or optimize page load performance, Lighthouse scores, or site speed. Biases towards retrieval from current documentation over pre-trained knowledge.","https:\u002F\u002Fgithub.com\u002Fariadoss\u002Fsuperskills\u002Fblob\u002Fmain\u002Fskills\u002Fweb-perf\u002FSKILL.md",[157],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[159,160,161,162,163,164,165,166,167,36,168,169,170,171,172,173,174,175,176,177],"web","perf","web perf","resources","MCP","layout","optimize","documentation","caching","qa","knowledge","devtools","issues","render","re","performance","chrome devtools","vitals","never",{"skillNo":179,"name":180,"nameZh":6,"description":181,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":182,"categories":183,"keywords":184,"publishedAt":43},"01M71AZ2KZNWYEGKKRMJH8PYG2","verify","Use when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always","https:\u002F\u002Fgithub.com\u002Fariadoss\u002Fsuperskills\u002Fblob\u002Fmain\u002Fskills\u002Fverify\u002FSKILL.md",[],[180,125,185,186,147,118,187,188,36,189,105,190,191,192,193],"commands","about","claims","output","assertions","claim","prs","evidence","running",{"skillNo":195,"name":196,"nameZh":6,"description":197,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":198,"categories":199,"keywords":200,"publishedAt":43},"01M71AZ0ACCK7N1QMWQK1FV2HJ","validate-idea","Validate a business idea using the minimalist entrepreneur framework. Use when someone has a business idea and wants to test if it's worth pursuing before building anything.","https:\u002F\u002Fgithub.com\u002Fariadoss\u002Fsuperskills\u002Fblob\u002Fmain\u002Fskills\u002Fvalidate-idea\u002FSKILL.md",[],[201,202,203,204,205,206,207,36,208,105,209],"validate","idea","validate idea","framework","it","minimalist","test","business","before building"]