[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7DD63XGPTQAZZ3FHTK32HM3":3,"skill-same:01M7DD63XGPTQAZZ3FHTK32HM3":52},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":24,"commitSha":25,"skillDirectory":26,"forksCount":27,"repositoryHtmlUrl":28,"repositoryHomepageUrl":29,"ownerHtmlUrl":30,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":31,"install":32,"resources":45},"01M7DD63XGPTQAZZ3FHTK32HM3","setup",null,"Read this first, before the reeval skill, whenever you're asked to set up, install, add, try or get started with reeval (the eval runner for coding agents), in a project or an empty directory, including \"set up reeval here\" and \"add evals\". It detects what's set up, asks what to evaluate, installs reeval, writes one to three small evals that show its value (a skill with and without, when the project has skills), runs `reeval doctor` for keys and Docker, and explains the results.","reeval-ai\u002Freeval","reeval-ai","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F354599760?v=4","01M7DCZWK0K84BEW1ECSTKP6MK","reeval.ai","01M7DCZWM22FTSWRP1FNVPA7NK",13,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Freeval-ai\u002Freeval\u002Fblob\u002Fmain\u002Fskills\u002Fsetup\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: setup\ndescription: >\n  Read this first, before the reeval skill, whenever you're asked to set up, install, add,\n  try or get started with reeval (the eval runner for coding agents), in a project or an empty\n  directory, including \"set up reeval here\" and \"add evals\". It detects what's set up, asks what\n  to evaluate, installs reeval, writes one to three small evals that show its value (a skill with\n  and without, when the project has skills), runs `reeval doctor` for keys and Docker, and\n  explains the results.\n---\n\n# Set up reeval\n\nYour job is the user's first good run: a few small evals that pass `--dry-run`, the keys and\nDocker sorted, and a short explanation of what the evals show. Small and readable beats complete.\nFor the authoring API in depth, the reeval skill (`skills\u002Freeval\u002FSKILL.md` in\ngithub.com\u002Freeval-ai\u002Freeval) has it; this file has everything a first setup needs.\n\nRules that hold throughout:\n\n- **Never overwrite or re-initialize.** Don't run `reeval init` where a `reeval.config.*` or\n  `reeval.yaml` exists, never pass `--force`, never edit or delete an existing eval, and add new\n  files beside the user's.\n- **Never write, print or ask for a key's value.** Name the variable and the file it goes in.\n- **Ask before you build,** then stop and wait (step 2): always, even when reeval is half set\n  up or the right eval looks obvious. Only a message that already says what to evaluate\n  (\"add an eval for the postgres skill\") skips the question.\n\n## 1. Look before you touch anything\n\n```bash\nls -a; ls reeval.config.* reeval.yaml evals 2>\u002Fdev\u002Fnull\ngrep -n '\"reeval\"' package.json 2>\u002Fdev\u002Fnull\nls pnpm-lock.yaml yarn.lock bun.lock package-lock.json 2>\u002Fdev\u002Fnull   # the package manager\nfind . -name SKILL.md -not -path '*\u002Fnode_modules\u002F*' | head -20       # the project's skills\nls AGENTS.md CLAUDE.md GEMINI.md .cursor\u002Frules .github\u002Fcopilot-instructions.md 2>\u002Fdev\u002Fnull  # rules\nls CONTRIBUTING.md .mcp.json .cursor\u002Fmcp.json docs 2>\u002Fdev\u002Fnull\ngrep -o '^[A-Z_]*=' .env.local .env 2>\u002Fdev\u002Fnull                      # key names only\n```\n\n| You find | State | Then |\n|---|---|---|\n| A `reeval.config.*` or `reeval.yaml` | Set up | No init. Read the config and the evals and run `npx reeval run --dry-run`; after step 2, add evals beside them |\n| `reeval` in `package.json`, no config | Installed | `npx reeval init --bare --harness \u003Ch>` |\n| Neither | New | Install, then `init --bare` (step 3) |\n\nThe directory is a **project** if it has code, docs or skills (`package.json`, `src\u002F`,\n`README.md`, `skills\u002F`, a `.git` with commits); it's **empty** if there's nothing but dotfiles.\n\n## 2. Agree what to evaluate\n\n\"Set up reeval\" says to set reeval up, not what to evaluate, so it doesn't answer this step,\nhowever clear the choice looks from the repo. Your first reply is the proposal and the\nquestion, with nothing installed or written yet: end your turn there and wait for the answer.\n\n**In a project,** say in two or three sentences what you found and what you propose, then ask\nand stop. For example: \"This repo has two skills. I'd set up reeval with one eval that runs Cursor\non a billing task with and without `acme-billing`, to show what the skill changes. Shall I set\nup evals for this project?\" With no skills, propose the rules file or a task in the code\ninstead.\n\n**In an empty directory, or when the user says no,** ask what they want to evaluate (an SDK, an\nAPI, a library, docs or a skill they're writing, a kind of task) and stop. When they answer,\nresearch it before you write a line: read its docs, changelog and README (search the web, fetch\nthe pages), note the current version, and find a few things a coding agent gets wrong without\nhelp: a renamed API, a new default, a required option, a convention. Those are your evals.\n\n## 3. Install and initialize\n\nUse the project's package manager (its lockfile), and pick the harness the user has a key for:\n\n| Harness | `--harness` | Key |\n|---|---|---|\n| Cursor (default) | `cursor` | `CURSOR_API_KEY` |\n| Claude Code | `claude-code` | `ANTHROPIC_API_KEY` |\n| Codex | `codex` | `OPENAI_API_KEY` |\n| Antigravity | `antigravity` | `GEMINI_API_KEY` |\n| OpenCode, Goose, pi | `opencode`, `goose`, `pi` | The model's provider's key, e.g. `GOOGLE_GENERATIVE_AI_API_KEY` for `google\u002F…` |\n\n```bash\nnpm init -y                                  # only if there's no package.json\nnpm i -D reeval                              # pnpm add -D reeval · yarn add -D reeval\nnpx reeval init --bare --harness cursor      # pnpm reeval init … · yarn reeval init …\n```\n\n`--bare` writes only `reeval.config.ts`, the `.gitignore` entries (`.reeval\u002F`, `.env*`) and the\n`eval` script; it merges into existing files and skips the rest. It exits 2 if reeval is set up\nalready: that's your cue to stop initializing, not to `--force`. The config sets\n`analysis.model` for `reeval analyze` (needs `GOOGLE_GENERATIVE_AI_API_KEY`) and `pr` and `nightly`\nprofiles. Leave them.\n\n## 4. Write one to three evals\n\nPick the low-hanging fruit, in this order:\n\n1. **The project has skills:** a compare of one skill, with and without. The control is an empty\n   workspace, the variant the same plus the skill; the task is one the skill's own rules decide.\n   This is the eval that shows lift. Template A in\n   [references\u002Ftemplates.md](references\u002Ftemplates.md).\n2. **No skills, but agent rules** (`AGENTS.md`, `CLAUDE.md`, `GEMINI.md`, `.cursor\u002Frules`): a\n   compare of the rules file, with and without, on a small task in the real repo. Template D.\n3. **Any other codebase (an app, a CLI, a library):** a small task in the real repo, checked by\n   its typecheck, its tests or a hidden test. Template B.\n4. **An empty directory:** the researched subject, a compare of a short skill or doc you write\n   from the research (`skills\u002F\u003Csubject>\u002FSKILL.md`) against none, or a single eval. Template C.\n\nWork in the user's code, not a copy you make up: seed the repo with `directory('..')` rather\nthan stub files, which hand the agent the convention you meant to test. Don't write a skill or\nrules from what you already know to compare against: your checks would restate what you wrote,\nso the compare measures nothing. Write one only from research (step 2, Template C).\n\nEach eval:\n\n- **Check rules only the skill teaches.** Read the skill (or rules file) and pick two or three\n  of its most specific rules that one task touches: a house convention, a helper or file it\n  names, a version-specific API, a non-obvious default. Ask of each: would a strong agent do\n  this unprompted? If yes (index a foreign key, deny public access, validate input), both arms\n  pass it; pick another. In a real repo, the agent copies what the code around the change\n  already shows (the error class, the logger), so prefer rules the code can't show: a changelog\n  entry, where tests go, a file to register in. Grep the repo for each rule: one that\n  `CONTRIBUTING.md` or the docs repeat reaches the control too, so leave those files out of\n  the control as well, or pick another rule. Check that the task triggers each rule: a rule for\n  user-facing changes needs a user-facing task. One labeled check per rule, so the check\n  table shows which ones the skill changed.\n- **One task, one prompt, that doesn't name the answer.** Ask for the outcome in the user's\n  words (\"an invoices table with its total\"), never the rule the check looks for (\"store the\n  total in cents\"), and not in the rule's terms either (\"a user-visible fix\" hints at the\n  changelog). Reread the prompt: if it restates the rule, the control passes too. Name the\n  file the task is about (`src\u002FprojectId.ts`) so a check can read it, but not the files a rule\n  decides (where the test goes): check those with a glob or the one path the rule allows.\n- **Deterministic checks first:** a file's content, a command's exit code, a script that imports\n  the agent's code. Label each requirement with `expect(x, 'label')`; a failed one is recorded and the eval carries on. Add a judge\n  (`c.judge`) only for what no check can see, with the inputs its question reads\n  (`{ diff: ['src\u002F**'] }`, `{ output: true }`), and then add `judge:` to the config.\n- **A name that states the question:** `stores invoice totals in cents with the billing skill`,\n  not `billing test`. The results and `reeval analyze` read in its terms.\n- **Small:** leave `runs` unset: a compare runs 4 a cell (8 attempts), the fewest for a lift\n  verdict, and anything else once. Seed only the files the task\n  needs. If it needs the repo, seed it with `directory('..', { glob: ['!.env*', '!.reeval\u002F',\n  '!evals\u002F', '!reeval.config.*'] })`: `directory()` doesn't skip `.env*`, keys must not reach\n  the sandbox, and the agent must not read your checks or the oracle's answer in the config.\n- **Eval files go in `evals\u002F*.eval.ts`,** environments in `evals\u002Fenvironments.ts`. Paths in\n  `skill.local()`, `file()` and `directory()` are relative to the file that calls them. If\n  `evals\u002Fenvironments.ts` exists, append your environments below what's there and leave its\n  lines, comments included, as they are.\n\nWhen a check is a regex or a script you wrote, add two fake-agent profiles (Template A shows\nboth): a **canary** that does the task wrong must fail every check, so the checks check; an\n**oracle** that does it right, written another way than you'd expect (another helper name,\nanother order, schema-qualified `public.notes`), must pass, so the checks don't demand one\nspelling: a regex that wants `on notes (` fails the agent that writes `on public.notes (`. A\ncheck that runs a build or the project's tests needs no oracle.\n\n## 5. Check it plans, then the machine\n\n```bash\nnpx reeval run --dry-run        # exit 0; the evals, cells and attempts you meant\nnpx reeval doctor               # Docker, image, keys; exit 1 names each fix\n```\n\n- Rerun `--dry-run` after every edit. Exit 2 is a config or eval mistake with a precise\n  message; fix it, don't work around it. Check the counts: a compare is 2 cells.\n- `doctor` lists each missing key as `fix: add NAME=... to .env.local`. Tell the user exactly\n  those lines, and that `.env.local` goes next to `reeval.config.ts` (it's gitignored). The\n  analysis key is a warning only: runs pass or fail the same without it.\n- No Docker, or `doctor` says the image isn't built: the user installs or starts Docker, then\n  runs `npx reeval image build` once (minutes, cached).\n- With Docker: `npx reeval run --profile canary \u003Ceval-name>` must fail every\n  check you wrote, and `--profile oracle` must pass them. Name the eval: a fake agent writes the\n  same files for every eval, so it answers only the one it was written for. Needs no keys,\n  seconds each. A canary that passes means a check isn't checking; an oracle that fails means a\n  check is too narrow. Fix the check.\n\nOnly start a real `npx reeval run` if the user asked for it or agrees: it runs a real agent on\ntheir key. One compare (8 attempts) on Cursor is several minutes.\n\n## 6. Hand over\n\nEnd with a short message, not a report:\n\n1. What you added, one line per file, and what each eval asks.\n2. Each key the config needs by name (`CURSOR_API_KEY`, the analysis key) and whether `doctor`\n   found it; then what's missing, as commands: the `.env.local` lines, `npx reeval image\n   build`, Docker.\n3. The command to run: `npx reeval run` (or `npx reeval run \u003Ceval-name>` for one).\n4. How to read the output, in a few lines. `npx reeval run` prints a line per eval and a sentence\n   for each way one failed; `npx reeval show` prints its arms and checks. A real Cursor run of\n   Template A, from `npx reeval show`:\n\n```text\npass             stores invoice totals in cents with the billing skill\n  arm                     pass\n  plain                   0 of 2\n  with-billing-skill      2 of 2  helped\n  check                   plain   with-billing-skill\n  total in integer cents  0 of 2  2 of 2\n  a currency column       0 of 2  2 of 2\n  +1 more passed\n\n1 pass\n```\n\n- Each arm's `N of M` is passing attempts. The variant's word is its effect against the control:\n  `helped`, `made it worse`, or `same`; on a small sample (below 4 runs a side, or a `same` at\n  2 of 4 against 2 of 4) a `too-few-tries` remark says it doesn't hold yet and names the `runs`. When\n  both arms pass every run, or both fail every run, at 4 runs a side or more, a `cant-tell-apart`\n  remark says more runs won't help: make the task harder, or check the eval. When both pass but\n  another measure differs, the line under the variant names it (`Same pass rate · 5× faster.`),\n  and a measure that only leans gets a `too-few-tries` remark with the runs that would confirm it.\n  The check table shows which check told the arms apart.\n- `Worth knowing:` under an eval in `reeval run`'s list names what about it needs work, with the line to fix. Remarks never\n  change a verdict.\n- Both arms passing is a result, not a broken eval: the agent already does this without the\n  skill. Say so, and suggest a harder task or a more specific rule from the skill next.\n- A check that fails every attempt on both arms is more often a broken check than a hard task.\n  Open one attempt's `diff.patch` before you say the agent can't do it.\n- Exit 0 means every gated cell passed; only the variant is gated in a compare.\n- Details: `.reeval\u002Fruns\u002F\u003Crun>\u002Fsummary.md`, each attempt's `transcript.jsonl` and `diff.patch`;\n  `npx reeval studio` browses them; `npx reeval query effort` compares tools, time and tokens.\n","# Set up reeval\n\nYour job is the user's first good run: a few small evals that pass `--dry-run`, the keys and\nDocker sorted, and a short explanation of what the evals show. Small and readable beats complete.\nFor the authoring API in depth, the reeval skill (`skills\u002Freeval\u002FSKILL.md` in\ngithub.com\u002Freeval-ai\u002Freeval) has it; this file has everything a first setup needs.\n\nRules that hold throughout:\n\n- **Never overwrite or re-initialize.** Don't run `reeval init` where a `reeval.config.*` or\n  `reeval.yaml` exists, never pass `--force`, never edit or delete an existing eval, and add new\n  files beside the user's.\n- **Never write, print or ask for a key's value.** Name the variable and the file it goes in.\n- **Ask before you build,** then stop and wait (step 2): always, even when reeval is half set\n  up or the right eval looks obvious. Only a message that already says what to evaluate\n  (\"add an eval for the postgres skill\") skips the question.\n\n## 1. Look before you touch anything\n\n```bash\nls -a; ls reeval.config.* reeval.yaml evals 2>\u002Fdev\u002Fnull\ngrep -n '\"reeval\"' package.json 2>\u002Fdev\u002Fnull\nls pnpm-lock.yaml yarn.lock bun.lock package-lock.json 2>\u002Fdev\u002Fnull   # the package manager\nfind . -name SKILL.md -not -path '*\u002Fnode_modules\u002F*' | head -20       # the project's skills\nls AGENTS.md CLAUDE.md GEMINI.md .cursor\u002Frules .github\u002Fcopilot-instructions.md 2>\u002Fdev\u002Fnull  # rules\nls CONTRIBUTING.md .mcp.json .cursor\u002Fmcp.json docs 2>\u002Fdev\u002Fnull\ngrep -o '^[A-Z_]*=' .env.local .env 2>\u002Fdev\u002Fnull                      # key names only\n```\n\n| You find | State | Then |\n|---|---|---|\n| A `reeval.config.*` or `reeval.yaml` | Set up | No init. Read the config and the evals and run `npx reeval run --dry-run`; after step 2, add evals beside them |\n| `reeval` in `package.json`, no config | Installed | `npx reeval init --bare --harness \u003Ch>` |\n| Neither | New | Install, then `init --bare` (step 3) |\n\nThe directory is a **project** if it has code, docs or skills (`package.json`, `src\u002F`,\n`README.md`, `skills\u002F`, a `.git` with commits); it's **empty** if there's nothing but dotfiles.\n\n## 2. Agree what to evaluate\n\n\"Set up reeval\" says to set reeval up, not what to evaluate, so it doesn't answer this step,\nhowever clear the choice looks from the repo. Your first reply is the proposal and the\nquestion, with nothing installed or written yet: end your turn there and wait for the answer.\n\n**In a project,** say in two or three sentences what you found and what you propose, then ask\nand stop. For example: \"This repo has two skills. I'd set up reeval with one eval that runs Cursor\non a billing task with and without `acme-billing`, to show what the skill changes. Shall I set\nup evals for this project?\" With no skills, propose the rules file or a task in the code\ninstead.\n\n**In an empty directory, or when the user says no,** ask what they want to evaluate (an SDK, an\nAPI, a library, docs or a skill they're writing, a kind of task) and stop. When they answer,\nresearch it before you write a line: read its docs, changelog and README (search the web, fetch\nthe pages), note the current version, and find a few things a coding agent gets wrong without\nhelp: a renamed API, a new default, a required option, a convention. Those are your evals.\n\n## 3. Install and initialize\n\nUse the project's package manager (its lockfile), and pick the harness the user has a key for:\n\n| Harness | `--harness` | Key |\n|---|---|---|\n| Cursor (default) | `cursor` | `CURSOR_API_KEY` |\n| Claude Code | `claude-code` | `ANTHROPIC_API_KEY` |\n| Codex | `codex` | `OPENAI_API_KEY` |\n| Antigravity | `antigravity` | `GEMINI_API_KEY` |\n| OpenCode, Goose, pi | `opencode`, `goose`, `pi` | The model's provider's key, e.g. `GOOGLE_GENERATIVE_AI_API_KEY` for `google\u002F…` |\n\n```bash\nnpm init -y                                  # only if there's no package.json\nnpm i -D reeval                              # pnpm add -D reeval · yarn add -D reeval\nnpx reeval init --bare --harness cursor      # pnpm reeval init … · yarn reeval init …\n```\n\n`--bare` writes only `reeval.config.ts`, the `.gitignore` entries (`.reeval\u002F`, `.env*`) and the\n`eval` script; it merges into existing files and skips the rest. It exits 2 if reeval is set up\nalready: that's your cue to stop initializing, not to `--force`. The config sets\n`analysis.model` for `reeval analyze` (needs `GOOGLE_GENERATIVE_AI_API_KEY`) and `pr` and `nightly`\nprofiles. Leave them.\n\n## 4. Write one to three evals\n\nPick the low-hanging fruit, in this order:\n\n1. **The project has skills:** a compare of one skill, with and without. The control is an empty\n   workspace, the variant the same plus the skill; the task is one the skill's own rules decide.\n   This is the eval that shows lift. Template A in\n   [references\u002Ftemplates.md](references\u002Ftemplates.md).\n2. **No skills, but agent rules** (`AGENTS.md`, `CLAUDE.md`, `GEMINI.md`, `.cursor\u002Frules`): a\n   compare of the rules file, with and without, on a small task in the real repo. Template D.\n3. **Any other codebase (an app, a CLI, a library):** a small task in the real repo, checked by\n   its typecheck, its tests or a hidden test. Template B.\n4. **An empty directory:** the researched subject, a compare of a short skill or doc you write\n   from the research (`skills\u002F\u003Csubject>\u002FSKILL.md`) against none, or a single eval. Template C.\n\nWork in the user's code, not a copy you make up: seed the repo with `directory('..')` rather\nthan stub files, which hand the agent the convention you meant to test. Don't write a skill or\nrules from what you already know to compare against: your checks would restate what you wrote,\nso the compare measures nothing. Write one only from research (step 2, Template C).\n\nEach eval:\n\n- **Check rules only the skill teaches.** Read the skill (or rules file) and pick two or three\n  of its most specific rules that one task touches: a house convention, a helper or file it\n  names, a version-specific API, a non-obvious default. Ask of each: would a strong agent do\n  this unprompted? If yes (index a foreign key, deny public access, validate input), both arms\n  pass it; pick another. In a real repo, the agent copies what the code around the change\n  already shows (the error class, the logger), so prefer rules the code can't show: a changelog\n  entry, where tests go, a file to register in. Grep the repo for each rule: one that\n  `CONTRIBUTING.md` or the docs repeat reaches the control too, so leave those files out of\n  the control as well, or pick another rule. Check that the task triggers each rule: a rule for\n  user-facing changes needs a user-facing task. One labeled check per rule, so the check\n  table shows which ones the skill changed.\n- **One task, one prompt, that doesn't name the answer.** Ask for the outcome in the user's\n  words (\"an invoices table with its total\"), never the rule the check looks for (\"store the\n  total in cents\"), and not in the rule's terms either (\"a user-visible fix\" hints at the\n  changelog). Reread the prompt: if it restates the rule, the control passes too. Name the\n  file the task is about (`src\u002FprojectId.ts`) so a check can read it, but not the files a rule\n  decides (where the test goes): check those with a glob or the one path the rule allows.\n- **Deterministic checks first:** a file's content, a command's exit code, a script that imports\n  the agent's code. Label each requirement with `expect(x, 'label')`; a failed one is recorded and the eval carries on. Add a judge\n  (`c.judge`) only for what no check can see, with the inputs its question reads\n  (`{ diff: ['src\u002F**'] }`, `{ output: true }`), and then add `judge:` to the config.\n- **A name that states the question:** `stores invoice totals in cents with the billing skill`,\n  not `billing test`. The results and `reeval analyze` read in its terms.\n- **Small:** leave `runs` unset: a compare runs 4 a cell (8 attempts), the fewest for a lift\n  verdict, and anything else once. Seed only the files the task\n  needs. If it needs the repo, seed it with `directory('..', { glob: ['!.env*', '!.reeval\u002F',\n  '!evals\u002F', '!reeval.config.*'] })`: `directory()` doesn't skip `.env*`, keys must not reach\n  the sandbox, and the agent must not read your checks or the oracle's answer in the config.\n- **Eval files go in `evals\u002F*.eval.ts`,** environments in `evals\u002Fenvironments.ts`. Paths in\n  `skill.local()`, `file()` and `directory()` are relative to the file that calls them. If\n  `evals\u002Fenvironments.ts` exists, append your environments below what's there and leave its\n  lines, comments included, as they are.\n\nWhen a check is a regex or a script you wrote, add two fake-agent profiles (Template A shows\nboth): a **canary** that does the task wrong must fail every check, so the checks check; an\n**oracle** that does it right, written another way than you'd expect (another helper name,\nanother order, schema-qualified `public.notes`), must pass, so the checks don't demand one\nspelling: a regex that wants `on notes (` fails the agent that writes `on public.notes (`. A\ncheck that runs a build or the project's tests needs no oracle.\n\n## 5. Check it plans, then the machine\n\n```bash\nnpx reeval run --dry-run        # exit 0; the evals, cells and attempts you meant\nnpx reeval doctor               # Docker, image, keys; exit 1 names each fix\n```\n\n- Rerun `--dry-run` after every edit. Exit 2 is a config or eval mistake with a precise\n  message; fix it, don't work around it. Check the counts: a compare is 2 cells.\n- `doctor` lists each missing key as `fix: add NAME=... to .env.local`. Tell the user exactly\n  those lines, and that `.env.local` goes next to `reeval.config.ts` (it's gitignored). The\n  analysis key is a warning only: runs pass or fail the same without it.\n- No Docker, or `doctor` says the image isn't built: the user installs or starts Docker, then\n  runs `npx reeval image build` once (minutes, cached).\n- With Docker: `npx reeval run --profile canary \u003Ceval-name>` must fail every\n  check you wrote, and `--profile oracle` must pass them. Name the eval: a fake agent writes the\n  same files for every eval, so it answers only the one it was written for. Needs no keys,\n  seconds each. A canary that passes means a check isn't checking; an oracle that fails means a\n  check is too narrow. Fix the check.\n\nOnly start a real `npx reeval run` if the user asked for it or agrees: it runs a real agent on\ntheir key. One compare (8 attempts) on Cursor is several minutes.\n\n## 6. Hand over\n\nEnd with a short message, not a report:\n\n1. What you added, one line per file, and what each eval asks.\n2. Each key the config needs by name (`CURSOR_API_KEY`, the analysis key) and whether `doctor`\n   found it; then what's missing, as commands: the `.env.local` lines, `npx reeval image\n   build`, Docker.\n3. The command to run: `npx reeval run` (or `npx reeval run \u003Ceval-name>` for one).\n4. How to read the output, in a few lines. `npx reeval run` prints a line per eval and a sentence\n   for each way one failed; `npx reeval show` prints its arms and checks. A real Cursor run of\n   Template A, from `npx reeval show`:\n\n```text\npass             stores invoice totals in cents with the billing skill\n  arm                     pass\n  plain                   0 of 2\n  with-billing-skill      2 of 2  helped\n  check                   plain   with-billing-skill\n  total in integer cents  0 of 2  2 of 2\n  a currency column       0 of 2  2 of 2\n  +1 more passed\n\n1 pass\n```\n\n- Each arm's `N of M` is passing attempts. The variant's word is its effect against the control:\n  `helped`, `made it worse`, or `same`; on a small sample (below 4 runs a side, or a `same` at\n  2 of 4 against 2 of 4) a `too-few-tries` remark says it doesn't hold yet and names the `runs`. When\n  both arms pass every run, or both fail every run, at 4 runs a side or more, a `cant-tell-apart`\n  remark says more runs won't help: make the task harder, or check the eval. When both pass but\n  another measure differs, the line under the variant names it (`Same pass rate · 5× faster.`),\n  and a measure that only leans gets a `too-few-tries` remark with the runs that would confirm it.\n  The check table shows which check told the arms apart.\n- `Worth knowing:` under an eval in `reeval run`'s list names what about it needs work, with the line to fix. Remarks never\n  change a verdict.\n- Both arms passing is a result, not a broken eval: the agent already does this without the\n  skill. Say so, and suggest a harder task or a more specific rule from the skill next.\n- A check that fails every attempt on both arms is more often a broken check than a hard task.\n  Open one attempt's `diff.patch` before you say the agent can't do it.\n- Exit 0 means every gated cell passed; only the variant is gated in a compare.\n- Details: `.reeval\u002Fruns\u002F\u003Crun>\u002Fsummary.md`, each attempt's `transcript.jsonl` and `diff.patch`;\n  `npx reeval studio` browses them; `npx reeval query effort` compares tools, time and tokens.\n",{"name":5,"description":7},"main","6fb66482952dcf7c440a6c82595cf359dbb7f6ed","skills\u002Fsetup",0,"https:\u002F\u002Fgithub.com\u002Freeval-ai\u002Freeval","https:\u002F\u002Fdocs.reeval.ai","https:\u002F\u002Fgithub.com\u002Freeval-ai","2026-11-15 06:04:49",{"source":8,"skillName":5,"commands":33,"prompt":43,"zipAvailable":44},[34,37,40],{"tool":35,"command":36},"npx","npx skills add reeval-ai\u002Freeval --skill setup",{"tool":38,"command":39},"bunx","bunx skills add reeval-ai\u002Freeval --skill setup",{"tool":41,"command":42},"pnpm","pnpm dlx skills add reeval-ai\u002Freeval --skill setup","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Freeval-ai\u002Freeval\u002Fblob\u002Fmain\u002Fskills\u002Fsetup\u002FSKILL.md",true,[46],{"path":47,"resourceType":48,"resourceTypeDesc":49,"sizeBytes":50,"sourceHtmlUrl":51},"references\u002Ftemplates.md","REFERENCE","参考文档",10420,"https:\u002F\u002Fgithub.com\u002Freeval-ai\u002Freeval\u002Fblob\u002Fmain\u002Fskills\u002Fsetup\u002Freferences\u002Ftemplates.md",[53,60,67,74,81,88],{"skillNo":54,"name":55,"nameZh":6,"description":56,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":57,"categories":58,"keywords":59,"publishedAt":19},"01M7DD5X2NMVWR1TTKSYM24FBZ","reeval","How to write, run and debug evals with reeval, the eval runner for coding agents (npm package reeval; CLI `reeval`, alias `re`). Use whenever a task mentions reeval, reeval.config.ts, *.eval.ts files, `reeval run`, evals for a coding agent, comparing skills, MCP servers or docs, or lift. Not for changing reeval's own source: contributors follow the repository's AGENTS.md.","https:\u002F\u002Fgithub.com\u002Freeval-ai\u002Freeval\u002Fblob\u002Fmain\u002Fskills\u002Freeval\u002FSKILL.md",[],[],{"skillNo":61,"name":62,"nameZh":6,"description":63,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":64,"categories":65,"keywords":66,"publishedAt":19},"01M7DD5SATZ1PJ799YFKHW92GF","eval-rubric","The rubric reeval holds evals to, one section per remark kind. Load it when writing or reviewing a reeval eval's name, prompts, judge questions or compare, or when a reeval run prints a Remarks block.","https:\u002F\u002Fgithub.com\u002Freeval-ai\u002Freeval\u002Fblob\u002Fmain\u002Fskills\u002Feval-rubric\u002FSKILL.md",[],[],{"skillNo":68,"name":69,"nameZh":6,"description":70,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":71,"categories":72,"keywords":73,"publishedAt":19},"01M7DD5NQ68B0V41YP6KVM0JBB","house-greeting","The house style for greetings in this organization. Use when writing any greeting function, message or text.","https:\u002F\u002Fgithub.com\u002Freeval-ai\u002Freeval\u002Fblob\u002Fmain\u002Fpackages\u002Freeval\u002Ftest\u002Freal-agents\u002Fskills\u002Fhouse-greeting\u002FSKILL.md",[],[],{"skillNo":75,"name":76,"nameZh":6,"description":77,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":78,"categories":79,"keywords":80,"publishedAt":19},"01M7DD5MR0VM7CJ1D79XA4VKJS","greet","Greets people.","https:\u002F\u002Fgithub.com\u002Freeval-ai\u002Freeval\u002Fblob\u002Fmain\u002Fpackages\u002Freeval\u002Ftest\u002Fplan\u002Fyaml-paths\u002Fproject\u002Fskills\u002Fgreet\u002FSKILL.md",[],[],{"skillNo":82,"name":83,"nameZh":6,"description":84,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":85,"categories":86,"keywords":87,"publishedAt":19},"01M7DD5KGCB8MA5ATWSJDDM0SV","lint","The project's own lint skill.","https:\u002F\u002Fgithub.com\u002Freeval-ai\u002Freeval\u002Fblob\u002Fmain\u002Fpackages\u002Freeval\u002Ftest\u002Fplan\u002Fplugins\u002Fskills\u002Flint\u002FSKILL.md",[],[],{"skillNo":89,"name":90,"nameZh":6,"description":91,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":92,"categories":93,"keywords":94,"publishedAt":19},"01M7DD5G8NVWN927KN0WNKA8MQ","review-lint","Lints a change the house way.","https:\u002F\u002Fgithub.com\u002Freeval-ai\u002Freeval\u002Fblob\u002Fmain\u002Fpackages\u002Freeval\u002Ftest\u002Fplan\u002Fplugins\u002Fplugins\u002Freview\u002Fskills\u002Flint\u002FSKILL.md",[],[]]