[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7AADS2NSGSNTBA01N2F41WX":3,"skill-same:01M7AADS2NSGSNTBA01N2F41WX":47},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":24,"commitSha":25,"skillDirectory":26,"forksCount":27,"repositoryHtmlUrl":28,"repositoryHomepageUrl":6,"ownerHtmlUrl":29,"ownerBlogUrl":30,"ownerDescription":31,"updatedAt":32,"install":33,"resources":46},"01M7AADS2NSGSNTBA01N2F41WX","create-cases",null,"Author a shakedown cases file (`shakedowns\u002F\u003Cslug>.cases.toml`) for an existing agent skill: read the skill, turn what it promises into cases, supply fixtures for the commands it must not really run, and check the result with `shakedown case validate`. Use this whenever someone wants to measure, smoke-test, or add conformance cases for a skill, asks \"how do I test this skill\", points at a SKILL.md and wants cases written, or is editing a `.cases.toml`. Use it even when they only name the skill (\"write cases for connect-agent-to-hub\") without mentioning shakedown or TOML.","JasperHG90\u002Fshakedown","JasperHG90","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F8079803?v=4","01M7AAAKN62DYZ9XRCXAN0HGEF","Jasper Ginn","01M7AAAKP8D78B8ANVKEV7MRK8",1,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FJasperHG90\u002Fshakedown\u002Fblob\u002Fmain\u002Fplugins\u002Fgemini\u002Fskills\u002Fcreate-cases\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: create-cases\ndescription: >-\n  Author a shakedown cases file (`shakedowns\u002F\u003Cslug>.cases.toml`) for an\n  existing agent skill: read the skill, turn what it promises into cases,\n  supply fixtures for the commands it must not really run, and check the\n  result with `shakedown case validate`. Use this whenever someone wants to\n  measure, smoke-test, or add conformance cases for a skill, asks \"how do I\n  test this skill\", points at a SKILL.md and wants cases written, or is\n  editing a `.cases.toml`. Use it even when they only name the skill\n  (\"write cases for connect-agent-to-hub\") without mentioning shakedown or\n  TOML.\n---\n\n# Writing cases for a skill\n\nA cases file says what a skill promises and how to tell whether it kept the\npromise. shakedown then runs the skill on every harness and model you ship\nto and reports which promises survive.\n\nEverything here is one file plus, sometimes, a directory of stand-ins. You\nare writing both, then proving they load.\n\n## Step 1: Find the skill\n\nThe user may have named it already (\"write cases for\n`skills\u002Fconnect-agent-to-hub`\"). If not, ask for the path, and offer what\nyou can see: `ls skills\u002F` or a `**\u002FSKILL.md` glob usually finds the\ncandidates, and a short list is easier to answer than an open question.\n\nRead its `SKILL.md` completely before writing anything. You are looking for\nfour things, and the rest of this skill is what to do with them:\n\n- **What it produces.** A file, usually. That is `artifact_created`.\n- **What it goes through.** A CLI the skill tells the agent to run rather\n  than doing the work itself. That is `tool_used`.\n- **What it needs from the user.** Anything the skill says to ask for\n  rather than guess. That is `inputs_resolved`.\n- **What it calls that reaches the world.** `gh`, `kubectl`, `aws`, `curl`.\n  Those decide whether you need fixtures.\n\n## Step 2: Map the promises onto the three checks\n\nshakedown measures at most three things per case, plus one precondition it\nalways checks: did the skill activate at all. A case declares only the\nchecks that apply to it, and one it does not declare reports `unsupported`\nrather than failing. Do not invent a check to fill a column.\n\n| The skill promises | The case declares |\n|---|---|\n| writes `PLAN.md` | `[[case.artifacts]]` with `path`, and `contains` for the values that must land in it |\n| goes through `planctl` | `tool = \"planctl\"` |\n| asks before guessing the owner | `[[case.answers]]` with a `match` pattern and the `reply` |\n\nAn answer needs somewhere to land. `inputs_resolved` is proved by the\nreply appearing in an artifact, so a case with `answers` and no\n`[[case.artifacts]]` measures nothing at all — `case validate` will say so.\n\n`tool` is a substring, matched against a call's name **and its arguments**.\nA tool whose name turns up in what the skill writes is satisfied by the\nwriting: `tool = \"shakedown\"` passes on a `Write` to `shakedown.toml`, with\nthe CLI never run. Name enough of the command to rule that out —\n`tool = \"shakedown doctor\"` — and check it against a plausible transcript\nbefore believing it.\n\n**Double every backslash in `match`.** It is a TOML basic string, where\n`\\b` is a backspace character rather than a word boundary, so\n`match = \"(?i)\\bowner\\b\"` compiles to a pattern that can never fire. The\ncase then passes validation, costs a real run, and fails with \"no `match`\nfired\", quoting back a question the agent asked plainly. The fault is in\nthis file.\n\n```toml\n[[case]]\nname   = \"withholds-the-owner\"\nprompt = \"Write a project plan titled Billing migration.\"\ntool   = \"planctl\"\n\n  [[case.artifacts]]\n  path     = \"PLAN.md\"\n  contains = [\"platform-team\"]\n\n  [[case.answers]]\n  match = \"(?i)\\\\bowner\\\\b|who owns\"\n  reply = \"platform-team\"\n```\n\n`match` has to be loose enough to catch the question however the agent\nphrases it, and `reply` has to be a string that could appear no other way.\n`platform-team` reaching `PLAN.md` proves the harness asked; a reply of\n`yes` proves nothing.\n\nTwo or three cases is usually right, and they should differ in kind rather\nthan in wording:\n\n1. **Fully specified.** Everything the skill needs is in the prompt. This\n   measures whether it does the job at all.\n2. **Something withheld.** Leave out one fact the skill says it must ask\n   for, and supply the reply through `answers`. The proof is that the reply\n   reaches the artifact: it is only ever given in answer to a question, so\n   it cannot appear unless the harness asked. This is the case that catches\n   a model inventing a plausible value, which is the most expensive failure\n   a skill has, because the output looks right.\n3. **A judgment call.** Where the skill maps the user's words onto its own\n   categories — an environment onto a tier, a description onto a type.\n\n### Make `contains` fail when the answer is wrong\n\nThis is where most cases are too weak. A `contains` that names a value\nalone passes whichever slot it landed in, so a skill that put production\ninto the test tier still scores 100%. Assert the pairing, across the\nnewline if the format has one:\n\n```toml\ncontains = [\"  test:\\n    backend: https:\u002F\u002F…\u002Fcheckout-test-4417\u002F…\"]\n```\n\nBefore you believe a case, ask what a plausible wrong answer looks like and\ncheck that the case rejects it. If you cannot think of one, the case is\nprobably measuring the format rather than the judgment.\n\n## Step 3: Decide what must not really run\n\nA skill that clones a repository, opens a pull request, or calls a cloud\nAPI cannot be measured by letting it do those things: it needs credentials\nthe sandbox has not got, and a run that did authenticate would leave real\nside effects behind.\n\nSearch the skill and its scripts for what it shells out to, and show the\nuser the list rather than deciding for them:\n\n```bash\ngrep -rnoE '\\b(gh|git|kubectl|aws|gcloud|terraform|curl|docker)\\b' \u003Cskill-dir> | sort -u\n```\n\nThen ask, as a decision rather than a prose question, which of these should\nbe replaced. Some should not: `git` operating on a local clone is real work\nworth measuring, and faking it would measure the fake.\n\n`fixtures` names a directory of stand-in executables, seeded into the\nsandbox's `bin\u002F` ahead of everything else on PATH. Several directories are\nallowed and later wins, so a double shared between skills combines with the\nones only this skill needs:\n\n```toml\nfixtures = [\"fixtures\u002Fcommon\", \"fixtures\u002Fregister-service\"]\n```\n\nFixtures live beside the cases, never inside the skill — a fake `gh`\nshipped in `\u003Cskill>\u002Fbin\u002F` installs onto the machine of everyone who uses\nthe skill, and shakedown refuses a `fixtures` path that points inside.\n\n### Writing a double that does not lie\n\nA double is only useful if a broken skill still fails. Four rules earn\nthat, and each one comes from a way a double has quietly passed a bad run:\n\n- **Fake the boundary, not the work.** Replace only what leaves the\n  machine. If the skill clones and edits a repository, serve the clone from\n  a local repository the double builds on first call, and let real `git` do\n  the branch, the diff and the commit.\n- **Die on anything unrecognized.** A double that exits 0 for a verb it\n  does not answer hands the skill an empty success, and the run gets scored\n  on it.\n- **Leave the evidence where checks can read it.** A skill that works in a\n  temp directory it deletes leaves nothing to assert on. Have the double\n  copy the result into the workspace — `pr\u002F`, say — and point the artifact\n  check there.\n- **Record the calls.** Appending each invocation to a log file in the\n  workspace turns \"did it try\" into something a `contains` can check, and\n  tells you what happened when a run fails.\n\nThe double locates the workspace itself, so the same one works for any\nskill:\n\n```bash\nWORK=\"$(cd \"$(dirname \"$0\")\u002F..\" && pwd)\"\n```\n\nRecording is then one line at the top:\n\n```bash\necho \"$(basename \"$0\") $*\" >> \"$WORK\u002F$(basename \"$0\")-calls.log\"\n```\n\nWrite each double at `shakedowns\u002Ffixtures\u002F\u003Cslug>\u002F\u003Ccommand>` — the directory\nthe `fixtures` key names, resolved relative to the cases file — and make it\nexecutable. Nothing checks the bit for you, and a double at mode 644 does\nnot fall through to the real command: the shell refuses it outright, and\nthe run fails with an error naming neither the fixture nor the permission.\n\n```bash\nchmod +x shakedowns\u002Ffixtures\u002F\u003Cslug>\u002F*\n```\n\n### Check the call log like any other file\n\nThe log is a file in the workspace, so an artifact check reads it:\n\n```toml\n  [[case.artifacts]]\n  path = \"gh-calls.log\"\n  contains = [\"pr create --base main --head register-checkout\"]\n```\n\nCheck the log for what the result cannot show. `register-checkout` is a\nbranch name only `registerctl` builds, so an agent that ignored the CLI and\nran `git` and `gh pr create` by hand fails here while `pr\u002Fservices\u002Fcheckout.yaml`\nstill matches. That is stricter than `tool`, which passes on\n`registerctl --help`. Where the artifact already carries the evidence,\nchecking the log repeats it and breaks whenever the CLI changes its flags.\n\nTwo things the log cannot do:\n\n- **It is not the command line.** `echo \"$*\"` joins the arguments with\n  spaces and throws the quoting away, so `--title \"Register checkout\"` is\n  logged unquoted and `--body \"\"` disappears. Match unquoted text, and never\n  let a substring straddle an argument that might contain a space.\n- **It must not go in a case that declares `answers`.** `inputs_resolved`\n  looks for the reply across every artifact the case declares. A log holding\n  the command line lets a reply prove itself by appearing in an argument the\n  skill *passed*, when the check exists to prove it reached what the skill\n  *produced*.\n\nLeave temp paths and generated titles out of the substring — they change\nbetween runs.\n\n## Step 4: Ask what you missed\n\nYou have read one file and grepped for command names. The user knows things\nyou cannot see: a service that must be reachable, a config the skill reads,\nan environment variable, a second CLI invoked only on a branch you did not\nnotice. Ask plainly what else the skill depends on before you call the file\ndone — this catches more than another pass over the source would.\n\n## Step 5: Write it, then prove it loads\n\nWrite to `shakedowns\u002F\u003Cslug>.cases.toml`, where `\u003Cslug>` is the skill\ndirectory's name. `skill` and `fixtures` are file-level keys and belong\nabove the first `[[case]]`: TOML gives a bare key written after a table to\nthat table, so one placed below it silently becomes part of that case.\n\n```toml\nversion = 1\nskill   = \"..\u002Fskills\u002Fmy-skill\"\n\n[[case]]\nname   = \"fully-specified\"\nprompt = \"…\"\ntool   = \"myctl\"\n\n  [[case.artifacts]]\n  path     = \"OUT.md\"\n  contains = [\"…\"]\n```\n\nThen check it, which costs nothing:\n\n```bash\nshakedown case validate shakedowns\u002F\u003Cslug>.cases.toml\n```\n\nIt prints each case and the checks it measures. This is free and\ndeterministic, so run it after every edit rather than once at the end, and\ndo not offer the paid run until it is clean.\n\nRead all of what it says, not just the exit code:\n\n- **It exits 2.** The file does not load. The message names the key and,\n  where there is one, the remedy — a `fixtures` path that is not a\n  directory, a `skill` that resolves to something without a `SKILL.md`, a\n  key placed under a `[[case]]` that belongs above the first one. Fix and\n  re-run; do not work around it by deleting the key.\n- **A case measures \"nothing but skill_fired\".** It will pass no matter\n  what the skill does. Give it something to prove, or delete it.\n- **\"answers with no artifact\".** The reply has nowhere to land, so asking\n  is not measured. Add the artifact the answer should reach.\n- **A name it warns about.** Whitespace or a duplicate means `--case`\n  cannot select that case later.\n\n## Step 6: Offer to run it\n\nEverything so far worked on files alone, which is why no harness was needed\nto get here — cases are often written on a machine that never runs them.\nRunning is different, so check the ground before offering it:\n\n```bash\ncommand -v \u003Cthe harness CLI named in shakedown.toml>\n```\n\nA missing binary makes the offer a waste of the operator's time, and the\nfailure it produces reads like a broken skill rather than an absent CLI.\nIf none is installed, say which one the config names and stop there; the\ncases file is still finished and still worth committing. Same if there is\nno `shakedown.toml` to be found: the run needs one, and `init` writes it.\n\nThen offer rather than assume, and say roughly what it will cost: one model\nround trip per case per target, plus a second turn for any case that\nwithholds something.\n\n```bash\nshakedown case run shakedowns\u002F\u003Cslug>.cases.toml --harness \u003Cone> --keep\n```\n\nNarrow to one harness first. `--keep` leaves the workspaces, which is what\nmakes a failure diagnosable.\n\nRead the outcome before changing anything, because the three failures want\ndifferent fixes:\n\n- **`not_triggered` (\"never activated\").** Nothing was measured, so treat\n  this as a harness question before a skill question. On Claude Code the\n  run's `.shakedown-turn0.jsonl` opens with an init event listing the\n  skills the model was offered: if yours is missing from it, the sandbox\n  or the harness config is at fault, not your description. Other harnesses\n  do not emit that list, so there read the tool calls instead and see\n  whether the activation tool was called at all. A skill that was offered\n  and activated never reports `not_triggered`, so if you are staring at\n  this status with the name in the init list, suspect a mismatch between\n  the directory name and the `name` in the front matter.\n- **`tool_used` failed.** Either the model did the work itself, which is a\n  real finding about the skill, or the CLI was not on PATH, which is yours\n  to fix.\n- **`artifact_created` failed.** Read the file the run actually produced\n  before assuming the skill is wrong. Half the time the case asserted a\n  format detail the skill never promised.\n\nIterate until it passes for the right reason. A green run whose cases could\nnot have failed is worse than a red one, because it gets believed.\n","# Writing cases for a skill\n\nA cases file says what a skill promises and how to tell whether it kept the\npromise. shakedown then runs the skill on every harness and model you ship\nto and reports which promises survive.\n\nEverything here is one file plus, sometimes, a directory of stand-ins. You\nare writing both, then proving they load.\n\n## Step 1: Find the skill\n\nThe user may have named it already (\"write cases for\n`skills\u002Fconnect-agent-to-hub`\"). If not, ask for the path, and offer what\nyou can see: `ls skills\u002F` or a `**\u002FSKILL.md` glob usually finds the\ncandidates, and a short list is easier to answer than an open question.\n\nRead its `SKILL.md` completely before writing anything. You are looking for\nfour things, and the rest of this skill is what to do with them:\n\n- **What it produces.** A file, usually. That is `artifact_created`.\n- **What it goes through.** A CLI the skill tells the agent to run rather\n  than doing the work itself. That is `tool_used`.\n- **What it needs from the user.** Anything the skill says to ask for\n  rather than guess. That is `inputs_resolved`.\n- **What it calls that reaches the world.** `gh`, `kubectl`, `aws`, `curl`.\n  Those decide whether you need fixtures.\n\n## Step 2: Map the promises onto the three checks\n\nshakedown measures at most three things per case, plus one precondition it\nalways checks: did the skill activate at all. A case declares only the\nchecks that apply to it, and one it does not declare reports `unsupported`\nrather than failing. Do not invent a check to fill a column.\n\n| The skill promises | The case declares |\n|---|---|\n| writes `PLAN.md` | `[[case.artifacts]]` with `path`, and `contains` for the values that must land in it |\n| goes through `planctl` | `tool = \"planctl\"` |\n| asks before guessing the owner | `[[case.answers]]` with a `match` pattern and the `reply` |\n\nAn answer needs somewhere to land. `inputs_resolved` is proved by the\nreply appearing in an artifact, so a case with `answers` and no\n`[[case.artifacts]]` measures nothing at all — `case validate` will say so.\n\n`tool` is a substring, matched against a call's name **and its arguments**.\nA tool whose name turns up in what the skill writes is satisfied by the\nwriting: `tool = \"shakedown\"` passes on a `Write` to `shakedown.toml`, with\nthe CLI never run. Name enough of the command to rule that out —\n`tool = \"shakedown doctor\"` — and check it against a plausible transcript\nbefore believing it.\n\n**Double every backslash in `match`.** It is a TOML basic string, where\n`\\b` is a backspace character rather than a word boundary, so\n`match = \"(?i)\\bowner\\b\"` compiles to a pattern that can never fire. The\ncase then passes validation, costs a real run, and fails with \"no `match`\nfired\", quoting back a question the agent asked plainly. The fault is in\nthis file.\n\n```toml\n[[case]]\nname   = \"withholds-the-owner\"\nprompt = \"Write a project plan titled Billing migration.\"\ntool   = \"planctl\"\n\n  [[case.artifacts]]\n  path     = \"PLAN.md\"\n  contains = [\"platform-team\"]\n\n  [[case.answers]]\n  match = \"(?i)\\\\bowner\\\\b|who owns\"\n  reply = \"platform-team\"\n```\n\n`match` has to be loose enough to catch the question however the agent\nphrases it, and `reply` has to be a string that could appear no other way.\n`platform-team` reaching `PLAN.md` proves the harness asked; a reply of\n`yes` proves nothing.\n\nTwo or three cases is usually right, and they should differ in kind rather\nthan in wording:\n\n1. **Fully specified.** Everything the skill needs is in the prompt. This\n   measures whether it does the job at all.\n2. **Something withheld.** Leave out one fact the skill says it must ask\n   for, and supply the reply through `answers`. The proof is that the reply\n   reaches the artifact: it is only ever given in answer to a question, so\n   it cannot appear unless the harness asked. This is the case that catches\n   a model inventing a plausible value, which is the most expensive failure\n   a skill has, because the output looks right.\n3. **A judgment call.** Where the skill maps the user's words onto its own\n   categories — an environment onto a tier, a description onto a type.\n\n### Make `contains` fail when the answer is wrong\n\nThis is where most cases are too weak. A `contains` that names a value\nalone passes whichever slot it landed in, so a skill that put production\ninto the test tier still scores 100%. Assert the pairing, across the\nnewline if the format has one:\n\n```toml\ncontains = [\"  test:\\n    backend: https:\u002F\u002F…\u002Fcheckout-test-4417\u002F…\"]\n```\n\nBefore you believe a case, ask what a plausible wrong answer looks like and\ncheck that the case rejects it. If you cannot think of one, the case is\nprobably measuring the format rather than the judgment.\n\n## Step 3: Decide what must not really run\n\nA skill that clones a repository, opens a pull request, or calls a cloud\nAPI cannot be measured by letting it do those things: it needs credentials\nthe sandbox has not got, and a run that did authenticate would leave real\nside effects behind.\n\nSearch the skill and its scripts for what it shells out to, and show the\nuser the list rather than deciding for them:\n\n```bash\ngrep -rnoE '\\b(gh|git|kubectl|aws|gcloud|terraform|curl|docker)\\b' \u003Cskill-dir> | sort -u\n```\n\nThen ask, as a decision rather than a prose question, which of these should\nbe replaced. Some should not: `git` operating on a local clone is real work\nworth measuring, and faking it would measure the fake.\n\n`fixtures` names a directory of stand-in executables, seeded into the\nsandbox's `bin\u002F` ahead of everything else on PATH. Several directories are\nallowed and later wins, so a double shared between skills combines with the\nones only this skill needs:\n\n```toml\nfixtures = [\"fixtures\u002Fcommon\", \"fixtures\u002Fregister-service\"]\n```\n\nFixtures live beside the cases, never inside the skill — a fake `gh`\nshipped in `\u003Cskill>\u002Fbin\u002F` installs onto the machine of everyone who uses\nthe skill, and shakedown refuses a `fixtures` path that points inside.\n\n### Writing a double that does not lie\n\nA double is only useful if a broken skill still fails. Four rules earn\nthat, and each one comes from a way a double has quietly passed a bad run:\n\n- **Fake the boundary, not the work.** Replace only what leaves the\n  machine. If the skill clones and edits a repository, serve the clone from\n  a local repository the double builds on first call, and let real `git` do\n  the branch, the diff and the commit.\n- **Die on anything unrecognized.** A double that exits 0 for a verb it\n  does not answer hands the skill an empty success, and the run gets scored\n  on it.\n- **Leave the evidence where checks can read it.** A skill that works in a\n  temp directory it deletes leaves nothing to assert on. Have the double\n  copy the result into the workspace — `pr\u002F`, say — and point the artifact\n  check there.\n- **Record the calls.** Appending each invocation to a log file in the\n  workspace turns \"did it try\" into something a `contains` can check, and\n  tells you what happened when a run fails.\n\nThe double locates the workspace itself, so the same one works for any\nskill:\n\n```bash\nWORK=\"$(cd \"$(dirname \"$0\")\u002F..\" && pwd)\"\n```\n\nRecording is then one line at the top:\n\n```bash\necho \"$(basename \"$0\") $*\" >> \"$WORK\u002F$(basename \"$0\")-calls.log\"\n```\n\nWrite each double at `shakedowns\u002Ffixtures\u002F\u003Cslug>\u002F\u003Ccommand>` — the directory\nthe `fixtures` key names, resolved relative to the cases file — and make it\nexecutable. Nothing checks the bit for you, and a double at mode 644 does\nnot fall through to the real command: the shell refuses it outright, and\nthe run fails with an error naming neither the fixture nor the permission.\n\n```bash\nchmod +x shakedowns\u002Ffixtures\u002F\u003Cslug>\u002F*\n```\n\n### Check the call log like any other file\n\nThe log is a file in the workspace, so an artifact check reads it:\n\n```toml\n  [[case.artifacts]]\n  path = \"gh-calls.log\"\n  contains = [\"pr create --base main --head register-checkout\"]\n```\n\nCheck the log for what the result cannot show. `register-checkout` is a\nbranch name only `registerctl` builds, so an agent that ignored the CLI and\nran `git` and `gh pr create` by hand fails here while `pr\u002Fservices\u002Fcheckout.yaml`\nstill matches. That is stricter than `tool`, which passes on\n`registerctl --help`. Where the artifact already carries the evidence,\nchecking the log repeats it and breaks whenever the CLI changes its flags.\n\nTwo things the log cannot do:\n\n- **It is not the command line.** `echo \"$*\"` joins the arguments with\n  spaces and throws the quoting away, so `--title \"Register checkout\"` is\n  logged unquoted and `--body \"\"` disappears. Match unquoted text, and never\n  let a substring straddle an argument that might contain a space.\n- **It must not go in a case that declares `answers`.** `inputs_resolved`\n  looks for the reply across every artifact the case declares. A log holding\n  the command line lets a reply prove itself by appearing in an argument the\n  skill *passed*, when the check exists to prove it reached what the skill\n  *produced*.\n\nLeave temp paths and generated titles out of the substring — they change\nbetween runs.\n\n## Step 4: Ask what you missed\n\nYou have read one file and grepped for command names. The user knows things\nyou cannot see: a service that must be reachable, a config the skill reads,\nan environment variable, a second CLI invoked only on a branch you did not\nnotice. Ask plainly what else the skill depends on before you call the file\ndone — this catches more than another pass over the source would.\n\n## Step 5: Write it, then prove it loads\n\nWrite to `shakedowns\u002F\u003Cslug>.cases.toml`, where `\u003Cslug>` is the skill\ndirectory's name. `skill` and `fixtures` are file-level keys and belong\nabove the first `[[case]]`: TOML gives a bare key written after a table to\nthat table, so one placed below it silently becomes part of that case.\n\n```toml\nversion = 1\nskill   = \"..\u002Fskills\u002Fmy-skill\"\n\n[[case]]\nname   = \"fully-specified\"\nprompt = \"…\"\ntool   = \"myctl\"\n\n  [[case.artifacts]]\n  path     = \"OUT.md\"\n  contains = [\"…\"]\n```\n\nThen check it, which costs nothing:\n\n```bash\nshakedown case validate shakedowns\u002F\u003Cslug>.cases.toml\n```\n\nIt prints each case and the checks it measures. This is free and\ndeterministic, so run it after every edit rather than once at the end, and\ndo not offer the paid run until it is clean.\n\nRead all of what it says, not just the exit code:\n\n- **It exits 2.** The file does not load. The message names the key and,\n  where there is one, the remedy — a `fixtures` path that is not a\n  directory, a `skill` that resolves to something without a `SKILL.md`, a\n  key placed under a `[[case]]` that belongs above the first one. Fix and\n  re-run; do not work around it by deleting the key.\n- **A case measures \"nothing but skill_fired\".** It will pass no matter\n  what the skill does. Give it something to prove, or delete it.\n- **\"answers with no artifact\".** The reply has nowhere to land, so asking\n  is not measured. Add the artifact the answer should reach.\n- **A name it warns about.** Whitespace or a duplicate means `--case`\n  cannot select that case later.\n\n## Step 6: Offer to run it\n\nEverything so far worked on files alone, which is why no harness was needed\nto get here — cases are often written on a machine that never runs them.\nRunning is different, so check the ground before offering it:\n\n```bash\ncommand -v \u003Cthe harness CLI named in shakedown.toml>\n```\n\nA missing binary makes the offer a waste of the operator's time, and the\nfailure it produces reads like a broken skill rather than an absent CLI.\nIf none is installed, say which one the config names and stop there; the\ncases file is still finished and still worth committing. Same if there is\nno `shakedown.toml` to be found: the run needs one, and `init` writes it.\n\nThen offer rather than assume, and say roughly what it will cost: one model\nround trip per case per target, plus a second turn for any case that\nwithholds something.\n\n```bash\nshakedown case run shakedowns\u002F\u003Cslug>.cases.toml --harness \u003Cone> --keep\n```\n\nNarrow to one harness first. `--keep` leaves the workspaces, which is what\nmakes a failure diagnosable.\n\nRead the outcome before changing anything, because the three failures want\ndifferent fixes:\n\n- **`not_triggered` (\"never activated\").** Nothing was measured, so treat\n  this as a harness question before a skill question. On Claude Code the\n  run's `.shakedown-turn0.jsonl` opens with an init event listing the\n  skills the model was offered: if yours is missing from it, the sandbox\n  or the harness config is at fault, not your description. Other harnesses\n  do not emit that list, so there read the tool calls instead and see\n  whether the activation tool was called at all. A skill that was offered\n  and activated never reports `not_triggered`, so if you are staring at\n  this status with the name in the init list, suspect a mismatch between\n  the directory name and the `name` in the front matter.\n- **`tool_used` failed.** Either the model did the work itself, which is a\n  real finding about the skill, or the CLI was not on PATH, which is yours\n  to fix.\n- **`artifact_created` failed.** Read the file the run actually produced\n  before assuming the skill is wrong. Half the time the case asserted a\n  format detail the skill never promised.\n\nIterate until it passes for the right reason. A green run whose cases could\nnot have failed is worse than a red one, because it gets believed.\n",{"name":5,"description":7},"main","3ac632ac167bd480d8b07868b7001fc52e2a402c","plugins\u002Fgemini\u002Fskills\u002Fcreate-cases",0,"https:\u002F\u002Fgithub.com\u002FJasperHG90\u002Fshakedown","https:\u002F\u002Fgithub.com\u002FJasperHG90","https:\u002F\u002Fjasperhg90.github.io\u002F","Data Engineer @ Xebia Data. I am interested in data analysis, history, politics, education, policy making and writing. ","2026-09-21 04:18:57",{"source":8,"skillName":5,"commands":34,"prompt":44,"zipAvailable":45},[35,38,41],{"tool":36,"command":37},"npx","npx skills add JasperHG90\u002Fshakedown --skill create-cases",{"tool":39,"command":40},"bunx","bunx skills add JasperHG90\u002Fshakedown --skill create-cases",{"tool":42,"command":43},"pnpm","pnpm dlx skills add JasperHG90\u002Fshakedown --skill create-cases","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FJasperHG90\u002Fshakedown\u002Fblob\u002Fmain\u002Fplugins\u002Fgemini\u002Fskills\u002Fcreate-cases\u002FSKILL.md",true,[],[48,55,62,67,74,81],{"skillNo":49,"name":50,"nameZh":6,"description":51,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":52,"categories":53,"keywords":54,"publishedAt":19},"01M7AAFD8Q4CRZ5V34GB5655FX","fake-skill","A skill used only by shakedown's own offline tests.","https:\u002F\u002Fgithub.com\u002FJasperHG90\u002Fshakedown\u002Fblob\u002Fmain\u002Ftests\u002Ffake\u002Fskill\u002FSKILL.md",[],[],{"skillNo":56,"name":57,"nameZh":6,"description":58,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":59,"categories":60,"keywords":61,"publishedAt":19},"01M7AAFB03PDQMX0DY841ADDR9","shakedown-canary","Run the shakedown canary check. Use whenever someone asks to run the canary check, verify the harness, or confirm shakedown is wired up correctly.","https:\u002F\u002Fgithub.com\u002FJasperHG90\u002Fshakedown\u002Fblob\u002Fmain\u002Fsrc\u002Fshakedown\u002Fcanary\u002FSKILL.md",[],[],{"skillNo":63,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":64,"categories":65,"keywords":66,"publishedAt":19},"01M7AAF55DP6C2XSF7N23STFP3","https:\u002F\u002Fgithub.com\u002FJasperHG90\u002Fshakedown\u002Fblob\u002Fmain\u002Fskills\u002Fcreate-cases\u002FSKILL.md",[],[],{"skillNo":68,"name":69,"nameZh":6,"description":70,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":71,"categories":72,"keywords":73,"publishedAt":19},"01M7AAEXCDG258QRVSSFQV0MV9","analyze-results","Read a shakedown report (`shakedown-report.json`) and say which file to open: the measured skill's own `SKILL.md`, the `shakedowns\u002F\u003Cslug>.cases.toml` holding its cases, the harness block in `shakedown.toml`, or a flag on the run itself. Use this whenever someone has run `shakedown case run` and wants to know what a number means or what to do about it, pastes a report or the PR comment rendered from one, asks why `skill_fired`, `tool_used`, `artifact_created` or `inputs_resolved` came out low, asks whether a rate is worth acting on, or says a conformance run failed and they cannot tell whose fault it is. Use it even when they only quote the percentage (\"my skill scores 60% on inputs_resolved, now what\") without naming shakedown or the report file.","https:\u002F\u002Fgithub.com\u002FJasperHG90\u002Fshakedown\u002Fblob\u002Fmain\u002Fskills\u002Fanalyze-results\u002FSKILL.md",[],[],{"skillNo":75,"name":76,"nameZh":6,"description":77,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":78,"categories":79,"keywords":80,"publishedAt":19},"01M7AAEMM68JP8A128WHMTJJTX","add-harness","Add a new agent harness (opencode, Gemini CLI, Codex, Amp, a local Ollama-backed runner, or any CLI that loads skills) to a shakedown config so its skill conformance can be measured, and diagnose a harness that `shakedown doctor` reports as not qualifying. Use this whenever someone wants shakedown to run against a harness it does not yet support, asks why a harness fails doctor, asks how to point an existing harness at a different model or provider, or is editing the `[harness.*]` block of a shakedown.toml. Use it even when they only name the tool (\"can this test opencode?\") without mentioning config or TOML.","https:\u002F\u002Fgithub.com\u002FJasperHG90\u002Fshakedown\u002Fblob\u002Fmain\u002Fskills\u002Fadd-harness\u002FSKILL.md",[],[],{"skillNo":82,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":83,"categories":84,"keywords":85,"publishedAt":19},"01M7AAEEKJAS0KAJRM74K7FEFJ","https:\u002F\u002Fgithub.com\u002FJasperHG90\u002Fshakedown\u002Fblob\u002Fmain\u002Fplugins\u002Fopencode\u002Fskills\u002Fcreate-cases\u002FSKILL.md",[],[]]