create-project-verification-skill
Use for /create-project-verification-skill, /create-project-verification-skill quick, or when asked to set up verification for a repo.
- License 许可
- MIT
- 最近更新
- 2026-11-15
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- create-project-verification-skill
- metadata
- { "harness": "claude-code, codex, grok, cursor" }
- description
- Use for /create-project-verification-skill, /create-project-verification-skill quick, or when asked to set up verification for a repo.
- disable-model-invocation
- true
02 资源文件 16
- 参考文档agents/openai.yaml43 B
- 参考文档references/agents-verification.md1.3 KB
- 参考文档references/areas8.9 KB
- 参考文档references/claude-stub.md286 B
- 参考文档references/discovery.md3.8 KB
- 参考文档references/full.md4.9 KB
- 参考文档references/harvest2.9 KB
- 脚本references/lib.sh2.2 KB
- 参考文档references/lifecycle.md7.4 KB
- 参考文档references/pw-close1.2 KB
- 脚本references/pw.sh4.1 KB
- 参考文档references/reach20.9 KB
- 参考文档references/recorders.md5.7 KB
- 参考文档references/upgrade8.0 KB
- 参考文档references/verify45.0 KB
- 参考文档references/verify-skill.md28.5 KB
03 SKILL.md 正文
Create a project verification skill
This skill gives a project a verify-<app> skill. That skill has an isolated environment,
one bash recipe per feature, and the shipped runner bin/verify, which drives the recipes
in parallel without an agent. Agents then drive the features a change touches before they
call work done. You write for the next agent, which reads the result in the middle of
another task, with no other context. Recorders are subagents. Each one writes the recipes
of one area of the app.
Modes
/create-project-verification-skill(default): record every feature./create-project-verification-skill quick: write the environment and one recipe, with no recorders. Do steps 1 to 3, steps 4.1 and 4.3, the lifecycle checks of step 5, and step 6. The one recipe proves that the environment, sign-in, the browser helper and cleanup work, and agents copy it. The other recipes grow with the code:verify affectedfails while a change adds a screen, route or action no recipe covers,verify uncoveredlists the rest, and/verify-<app> fullrecords them when the user asks.
A request with the same meaning picks the same mode. Do not ask which mode to use. Keep your own context small: write the environment and the first recipe yourself, and leave the other recipes to recorders.
When you need me to make a choice before you can continue, use your question tool if you have one, or ask in your reply if you don't. If the question gets no answer, ask again in your reply and wait. Do not answer it yourself.
If .agents/skills/verify-* already exists, stop before writing anything. Ask whether to
replace it (delete and generate again) or upgrade it with
/maintain-project-verification-skill.
<generator> below is this skill's folder.
What you write
.agents/skills/verify-<app>/
SKILL.md <generator>/references/verify-skill.md, with <app> filled in
FULL.md <generator>/references/full.md, with <app> filled in
RECORDERS.md copied from <generator>/references/recorders.md, unchanged
PROJECT.md this project's facts (step 3)
bin/verify copied from <generator>/references/verify, unchanged
bin/lib.sh copied from <generator>/references/lib.sh, unchanged
bin/harvest copied from <generator>/references/harvest, unchanged
bin/areas copied from <generator>/references/areas, unchanged (needs python3)
bin/reach copied from <generator>/references/reach, unchanged (needs python3)
bin/pw.sh bin/pw-close copied from <generator>/references, unchanged, for a web UI; pw-version beside SKILL.md
bin/launch.sh bin/cleanup.sh bin/doctor.sh bin/observe.sh bin/server-errors.sh
bin/env.sh bin/after-feature.sh optional
bin/entrypoints.sh bin/actions.sh
bin/<driver helpers>
features/<feature>.sh
.gitignore holds .cache/
apps companion repo only
.claude/skills/verify-<app>/SKILL.md stub from <generator>/references/claude-stub.md
.claude/settings.json "Bash(sudo:*)" merged into permissions.deny
<app> is a short lowercase name. Codex, Cursor, Grok and OpenCode read .agents/skills/.
Claude Code reads only .claude/skills/, so the stub exists for it. Never use symlinks.
SKILL.md, FULL.md and the files marked unchanged are the shipped files. An upgrade replaces
them, so never edit them for the project. Everything project-specific goes into
PROJECT.md, the other bin/ helpers and features/.
Companion repo
Some projects must not show that agents work on them. Their skill lives in a companion repo: a separate git repo next to the app checkouts that holds only agent files.
Run one git ls-files here. When it lists no app source, this is a companion:
- Ask for the path of each app checkout. Write
.agents/skills/verify-<app>/appswith one line<name> <path>per app, the path relative to this repo's root. bin/verifyexports each path asAPP_<NAME>(upper case,-as_). A value the caller set wins.- Recipes,
# touches:lines and helpers name app files as<name>/<path>and reach them through$APP_<NAME>.
Nothing goes into an app checkout: no file, no dependency, no build output. Build from a
copy under .cache/, or send build output outside the checkout. Run git against a checkout
with GIT_OPTIONAL_LOCKS=0. Report a checkout that does not build; do not fix it.
Playwright, when needed, goes into the companion's own package.json.
1. Read the repo
Answer from the codebase. Ask only what you cannot observe.
- Surfaces: web UI, API, CLI or TUI, desktop, mobile, firmware, jobs.
- Run: how the app starts locally, its dependencies, ports, seed data and sign-in.
- Harness: existing end-to-end specs, sign-in or seed helpers, curl-able endpoints.
- Entry points: the files that define routes, screens, commands or endpoints, found
with one grep. That grep becomes
bin/entrypoints.sh. - Checks: the focused test, lint and type-check commands.
- Sibling repos: another repo the app needs at run time, such as a backend. Use the path
its docs name, else
../<name>, and write it intoPROJECT.md.
If the app does not build or start as it is, fix that or report it before going on.
2. Isolation
Choose the isolation level once, following
references/lifecycle.md: full (default), shared or
checks-only. That file also says which ports a run may use and how to use
local-stack/stack. Ask the user only when full is not possible. Recipes read every URL
and port from env.sh, never localhost:<default>.
3. Write the environment and PROJECT.md
At level checks-only the app cannot run, so there is no environment to write. Do only
this, then go to step 6: write PROJECT.md with the focused checks, copy the shipped files
as the tree above shows, and write the Claude stub and .gitignore. Write no environment
scripts and no recipes, and launch nothing.
At the other levels, write the scripts below into bin/ and make each one executable.
Environment scripts
launch.shcreates and starts the run's environment. It registers each resource as it creates it, and waits until the environment is ready, with a deadline.$VERIFY_RUN_IDnames the run and stays the same untilbin/verify down.- With
VERIFY_APP=devit runs the app with the developers' own dev command, such asnext devor./mvnw spring-boot:run, as a process the run records. A code change then needs no image build. - With
VERIFY_APP=buildit runs the production build. - A run uses
devunless its firstbin/verify upsetsVERIFY_APP=build.verify alland lanes always usebuild. - It is idempotent. When the run's environment already exists, it restarts the app, so a
second
bin/verify uptakes seconds.
cleanup.shremoves only what the run created, after success, failure and interruption, and checks that none of it remains. It never checks resources that existed before. With no run recorded, it exits 0.env.sh(optional) exports the run's values that recipes and helpers need, such as base URLs, ports and keys. It reads them from the run's record. The runner sources it before each recipe.after-feature.sh(optional) runs after every recipe'scleanup, for example to close the feature's browser session. A finished feature then holds nothing untilbin/verify down.doctor.shis one fast, read-only check that the environment belongs to this run and is healthy: processes up, right checkout, ports owned, sign-in works. It never treats a process outside the run's record as the run's.observe.sh <path>saves the app's current state for a failed step: the snapshot or screenshot of the feature's browser session ($VERIFY_SESSION), the last HTTP response, or the last command output.server-errors.sh <epoch seconds>, for an app with a server, prints the errors the server logged since that time, each with its stack trace. It prints nothing when there are none.- It reads the errors where this stack writes them, such as
docker logs --sincefor a container, the app server's log file, or the output of the dev process the run records. It reads the copy that$VERIFY_RUN_IDnames. - The summary line shows only the first line, so start each error with the request and the exception.
- When a step fails,
lib.shcalls it with the time the step began and saves its output as evidence. A UI timeout caused by a server error then explains itself. - A failed
bin/verify upcalls it too, with the time the launch began. So it must also work on a run whose app never became ready, such as one that stopped at a migration. - Check it once by causing a server error, such as a request the app answers with HTTP 500.
- It reads the errors where this stack writes them, such as
Scripts that list what to cover
entrypoints.shprints every entry point, one per line, as<entry point><TAB><file>: a route with its method (POST /api/notes), a screen's path, a command (todo add), a job or message handler.- It searches the source, such as route annotations or a router file, so it finds new entry points by itself.
- Some frameworks build entry points at run time, from configuration or the database,
where no source search finds them. When the project's framework is in
references/discovery.md, list them the way it says. - List only what a user or client calls: not update or deploy hooks, migrations, compose services or schemas.
bin/verify uncoveredcompares its output with the recipes'# covers:lines.
actions.shprints every user action, one per line, as<name><TAB><file>: each function a screen runs when the user acts (event handlers such ashandleDelete, form submits). For a project without a UI, it prints each handler method behind an entry point. It searches the source, likeentrypoints.sh.- Name an inline handler (
onClick={() => remove(id)}) after its element's label or test id, such asDelete button. Never use a line number: it changes whenever the file shifts. - A screen can get a handler from a hook or a parent
(
const { start } = useDeviceControl()). List it under the screen's file too. A search forconst handle...alone misses it. bin/verify uncoveredlists each action that no recipe's# expects:line cites by its file and name.bin/verify affectedlists those in changed files.
- Name an inline handler (
Rules for every helper
- A helper also works when an agent calls it by hand, outside
bin/verify. It sets: "${VERIFY_SKILL:=$(cd "$(dirname "$0")/.." && pwd)}"and gives the otherVERIFY_*variables it reads a default. - Its waits and error output follow the "Waits and errors" rules in the shipped
SKILL.md. A helper that inserts rows directly follows the fixture rule in that file's Data section.verify lintchecks only recipes, so check helpers yourself. - A helper that runs a command inside a container passes environment overrides through
(
bin/cli -e KEY=VALUE <command>). A recipe can then point one command at its own schema, prefix or folder.
Driver helpers
A driver helper that keeps state keeps it per $VERIFY_SESSION.
For a web UI without its own harness:
- Copy
pw.shandpw-closetobin/unchanged. The header ofpw.shsays how to call it. - Write into the skill's
pw-versionfile a Playwright version whose browser build is installed (ls ~/.cache/ms-playwright).pw.shinstalls that version under.cache/on first use. after-feature.shrunsbin/pw-close feature, andcleanup.shrunsbin/pw-close run. No recorder callspw-close, because other drives share the run's browser.
Then write:
- A sign-in helper. It saves each account's storage state once per run and loads it in one call. It signs each account into a browser session of its own: a second account in the same session is signed in as the first, through the sign-in provider's cookie.
- Account helpers. Creating or deleting an account also removes that account's cached state and token.
- Every cached file (token, storage state) through a temporary file of its own (
mktempin the same folder) andmv. Features sign in at the same time, and a shared temporary name makes one feature'smvfail.
For an API, an api helper wraps curl with the base URL, auth and --max-time, and
saves the last response for observe.sh.
PROJECT.md
Write PROJECT.md, about 10 KB, with these sections:
- Drivers: the tools that drive the app, such as
Playwright CLI and curl, and why. - Isolation:
Isolation level: <level>, one line on why, what the run owns and its markers, and the success paths the isolation cannot reach (such as a real payment that a stub replaces). - Environment: what
launch.sh,cleanup.shanddoctor.shdo, and how long a first and a secondbin/verify uptake. - Sign-in and data: the test accounts and baseline data a recipe can count on, each checked with one command against the running environment.
- Stubs: for each stub or fake service in the run, what it returns and stores and what it cannot do (such as an ML stub that stores nothing, or a collector that returns no data), each checked with one call. Recorders read this instead of finding it out in failed runs.
- Helpers: one line per helper with one real call as a recipe writes it, what it checks or returns, and its deadline when it waits.
- Focused checks: the test, lint and type-check commands.
- Gotchas: traps that hold for several features, one line each with the fix. A trap of
one feature goes in that recipe's header instead. Start with the driver's traps. For the
Playwright CLI:
- After a click that navigates, wait for the new URL or state before the next check.
- After
gotoorreload, wait for an element only the client renders before the first click. A click before the page is interactive does nothing. - A
confirm()dialog blocks every command until it is handled. - A role name matches as a substring unless
exact: true. - The
run-codefunction runs in a bare sandbox. Node and browser globals such asURL,requireandconsoleare not there, so parse URLs and log insidepage.evaluate, or return the value.
Bring it up
- Copy the shipped files as the tree above shows, and write the Claude stub and
.gitignore. - Run
VERIFY_APP=build bin/verify upin the background. While it builds, do the parts of step 4 that come before the first recipe. - Run
bin/verify upagain, which keeps the production build. Time both calls.
Generation uses the production build. A dev server holds far more memory, and with many
features at once it loses clicks that land before a page is ready. The first up also
measures how much memory and time one copy of the environment takes. The runner needs both
to decide on lanes (extra copies of the environment for # alone features).
For a CLI or a library there is no process to restart. launch.sh prepares the run's data
folder and builds once, and doctor.sh checks that the folder is the run's and the build is
current.
Build the environment for speed
Every recipe waits on the environment in every run.
- Derive every per-run file name and port from
$VERIFY_RUN_ID, never a fixed path such as.cache/run/, so lanes can run side by side. - Shorten every timer the features wait on, in the run's own configuration: job intervals,
retry delays, grace periods, expiries, polling.
- When the project's stack fixes the environment and offers no way to override it, add one to the stack as part of the repo, such as a compose override file it reads. Or note the gap and start the job once per recipe.
- Write each override and the recipes that rely on it into
PROJECT.md. - Before shortening a job for all features, check which recipes set up the state that job consumes.
- Keep each copy small so lanes fit in memory. Cap worker counts and heap sizes: a JVM
without
-Xmxtakes a quarter of the machine. Start the project's own JVM apps with-XX:TieredStopAtLevel=1 -XX:+UseSerialGC, which shortens startup. - Give every service of the run a memory limit (
mem_limitfor a container), third-party ones included. Step 4.3 sizes it, once the example recipe passes.- A runtime that sizes its heap from the container, such as a JVM, then stays inside it.
- Change only the limit of a third-party service, never its startup flags. Keycloak's
start script picks a garbage collector that clashes with
UseSerialGC, and then Keycloak does not start. - When the services come from the project's local stack, the limit goes into that stack
(
create-project-local-stack), not into the run.
- A stub or helper that starts a process first stops any leftover of the same feature, so an interrupted run never blocks the next one.
- Lanes share one build. Build images or artifacts once, under a lock, and let every lane
start from them. Only when the build bakes ports into an image (such as
NEXT_PUBLIC_*URLs) does each lane build its own. launch.shand every helper that restarts the app, such as for a clock shift, wait until a real page answers, signed in where the app has sign-in, as "Waits and errors" says. They poll every half second and never run the whole doctor.cleanup.shstops throwaway containers without a grace period (docker compose down -t 1).doctor.shretries each probe a few times with a 30 s limit, so a loaded machine never stops a drive with a false failure.
4. Record every feature
-
Run
bin/areas > .cache/areas.txt. It groups the entry points into areas of about 1,000 lines of source, one line per area:<area>: <n> lines; entry points: <...>; files: <...>. The files are the entry points' folders and the project files they import. One recorder records one area. Review the file:- Leave a line marked
split:as it is. It is one file over the size, and its recorder takes longer. - Rename an area whose name does not say what the user calls the screen, such as one
named
area2-<name>. - Move a file the lines miss, such as a component two screens share, to the area that uses it most.
- Make sure no area name equals another or starts with another followed by
-. Features are named<area>-<goal>, andbin/verify uncovered <area>finds an area's features by that prefix.
Do not name features. Each recorder lists every behavior of its area from the source and splits that list into features itself.
- Leave a line marked
-
Write each area's expectation list, as the Expectation lists section of the copied
FULL.mdsays. -
Write the first recipe yourself, for the simplest feature. Follow the Recipes section of the copied
SKILL.md, and name the recipe<area>-<goal>after its area. That area's recorder keeps it. Runbin/verify <feature>until it passes. Every recorder copies this recipe as its example, so make it clean. Then:- Drive the main create path by hand once: create the same record twice, and read it as
a second tenant or user. Write what that shows about ids, uniqueness and shared data
into
PROJECT.md's Gotchas before any recorder starts, so no recorder hits the same trap. - Write the setup helpers several features need, such as signing in as each role, creating an account with a role, provisioning a subscription or workspace, adding a member, or reading the run's mail.
- Write the fixture helper for each row several features insert directly, such as a user (the Data section's rule for such rows), before any recorder writes that insert into a recipe.
- Call each helper once the way a recipe calls it, after deleting its cached state (such as saved sign-ins), so its slow path runs too. A cache can hide a wrong argument until the first fresh drive.
- Size each service's memory limit: about one and a half times what the service uses
after a busy run. A busy run is the example recipe's flow plus a few hundred sign-ins,
30 at a time. Read the use with
docker stats. Never size the limit from an idle stack.
- Drive the main create path by hand once: create the same record twice, and read it as
a second tenant or user. Write what that shows about ids, uniqueness and shared data
into
-
Run
bin/verify up --lanes 1. A feature marked# alonewaits until its copy is free of other features, so on a single copy every other drive stops meanwhile. With the extra copy, every drive runs its# alonefeatures there instead. Then start recorders as the Recorders section of the copiedFULL.mdsays: one per area, every area at once, heaviest first. -
Read only their result files. Add their traps to
PROJECT.md's Gotchas.
5. Prove it
Run bin/verify down. Then do steps 2 to 5 of Full in the copied FULL.md. Its first
verify all starts a fresh environment on the production build, so every recipe runs once
more with every other feature beside it. Send a failing feature, an uncovered line or a
not-driven reason that does not hold back to the recorder that recorded its area, instead
of starting a new one. In the final verify all, every feature runs on an environment
where it already ran, which also proves each recipe repeats.
The runner starts up to as many features at once as the machine has cores, each only when
the machine-wide queue has room. Write a jobs file only when the project needs a lower
limit, such as a service with few connections. When # alone features remain, write into
the skill's lanes file the most extra copies a drive may start: about their total time
divided by the longest parallel feature.
Then check the lifecycle once:
- Run
bin/verify lint. Fix every# expects:line it lists as citing code the checkouts do not have:bin/verify affectedcan never select a feature through such a line. - Commit, change one line in a function that an
# expects:line cites, and check thatbin/verify affected --dry --base HEADlists that feature under "Check the edited code". Without--base HEADon this branch, every new recipe counts as changed. Undo the change. In a companion, change a copy of the app checkout instead and pointAPP_<NAME>at it. - Check that
bin/verify downleaves nothing of the run outside.cache/: no process, port, container, browser session or file.
6. Instructions and hand-over
Write one ## Verification section in the project's AGENTS.md from
references/agents-verification.md, with <app> and
the focused check commands filled in. In a companion repo it goes in the companion's
AGENTS.md. Create the file when there is none.
- Integrate an existing verification section instead of adding a second one. Rewrite a section that points at a verify skill this run replaces.
- If there is no
CLAUDE.md, create one holding only@AGENTS.md. If one exists without that import, put the same section there too. - Replace agent instructions for launching the app by hand with a pointer to the skill. Leave the README's instructions for people.
- Report an older verification system instead of deleting it.
Merge Bash(sudo:*) into permissions.deny of .claude/settings.json, keeping the rest.
Commit following the repo's conventions. Report: the isolation level, how long recording and
bin/verify all took, one line per feature (pass, fail with its kind, or blocked with what
was tried), and product defects found.
04 同仓库更多 Skills
with-verification
Use when the user adds "with verify" or "with verification" to a task as an instruction to verify it with the project's verify skill. Not when verification is only the topic.
triple-review
Use when the user asks for a triple review, or for three independent reviewers of a change.
tourik
Use when the user asks for "tourik mode".
takeover
Use when the user asks to take over someone else's PR and get it ready for the user to merge.
t3-fan-out
Use when the user wants several T3 threads or delegates started and coordinated, one per task.
squash-merge-pr
Use only when the user explicitly invokes squash-merge-pr.