qa-verify-before-verdict
Use before any review_criterion approval or column move - the evidence each verdict needs, how to treat skipped, blocked and intermittent cases, and which exit a blocker takes
- License 许可
- Apache-2.0
- 最近更新
- 2026-11-15
- Stars
- ★ 122
- Forks
- 18
01 前言 Front Matter
- name
- qa-verify-before-verdict
- source
- obra/superpowers (MIT), adapted
- category
- qa
- description
- Use before any review_criterion approval or column move - the evidence each verdict needs, how to treat skipped, blocked and intermittent cases, and which exit a blocker takes
03 SKILL.md 正文
Verify Before Verdict
No verdict without fresh executed evidence in this run.
- Every case on the task gets a result you recorded yourself:
passedorfailedwith the exact command and its observed output (actual), UI cases additionally with a one-line evidence string (width/device, URL, viewport report, what it showed — never a screenshot path, there isn't one; frontend-manual-testing / mobile-manual-testing). The result belongs on the case (set_test_case_result), not in a comment — that is where the next reader looks, and it survives the run. - A case you could not execute is
skippedwith what blocked it (no device, no stage deploy, missing credential), neverpassed. A case stillplannedwhen you try to hand the task on refuses the move. - A pass verdict rests on ONE thing this iteration: every case executed manually by you, with its evidence. Writing automated tests is out of scope for now — do not add suites and do not hold a verdict waiting for one. The repository's existing pipeline is still read (
get_pipeline_status): red is a finding, green is not a substitute for your own run. - A criterion is approved (
approved=true) only when you executed and observed it pass this run AND at least onepassedtest case carries itscriterion_id— an approval with no linked passed case is a claim, not proof. A criterion you could not execute is never approved: say which one, what blocked it and what would unblock it. - Both verdicts are moves out of
in_qa— the column you took the task into before testing. All cases green → move it to human_uat with the evidence on the cases and the criteria approved, same "no comment" rule — backend-only, API-only and infra changes included. Any case fails → move it to need_revision with, per failure, the acceptance criterion, exact reproduction command, EXPECTED vs ACTUAL (and, for a visual defect, the width/URL and what the screen showed). A task left sitting inin_qaat the end of a run is an unfinished verdict, not a result. - What actually fails a task vs. what is a note on a passing criterion: see bug-report-writing's rubric. Taste and polish beyond the team's UI floor are notes, never failures.
Blocker triage
Not every "I could not verify this" exits the same way:
| Observation | Owner | Exit |
|---|---|---|
| Behaviour contradicts a criterion, human comment or the UI floor | dev | reject criterion, need_revision |
| Boot fails with the repo's own documented command (missing migration, crash on start, undocumented required env var) | dev | need_revision, quote command + log |
| Your environment: port busy, stale server, Docker absent, npm cache | you | fix it and re-run, at most 2 attempts; never a finding. Still broken after that → the next row |
| Your environment, and you cannot fix it from inside the run (headless browser or Chrome will not launch, a missing system library, a tool you cannot install) | human | stop after 2 failed attempts at fixing it. Never need_revision (the developer cannot fix your machine) and never hand the task back and forth between stages. One ask_user (choice mode) that parks the task as blocked with the exact failing command and its error, options such as "I fixed the environment, retry" and "skip browser checks and verify by other means" |
| No device / no credential / no stage / an integration the criterion depends on is not configured in this install (e.g. no hosted-preview provider, no payment sandbox) / ambiguous criterion | human | ask_user (after one get_repo_tree — it is refused until the run has read the repo), 1 call, choice mode with options such as "attach the device and resume", "provide a sandbox key", "accept reading A", "move to need_revision" |
| Device busy | system | stop; the task resumes by itself |
A criterion that depends on something missing from the install is not the developer's defect even when you reject it with not verified: never pair that rejection with need_revision. The developer has nothing to change, and the system will not re-run QA on a diff it already rejected — the task parks for a person anyway, one round later.
Count attempts per problem, not per run: the same environment failure showing up again on a later round, or after the task came back to you from another stage, is the same problem and keeps the count. A human's comment that they fixed it resets it.
Intermittent failures
A failure you saw once is not automatically a defect, and it is not automatically your environment either:
- Re-run it from a clean state (fresh page load / fresh request) up to 3 times.
- Fails ≥1 of 3 with the product's own behaviour → a real defect: report it "intermittent, 1/3", with the condition that seems to trigger it. Races are bugs, not noise.
- Fails only through your harness (your
sleep, a selector that matched before render, a port collision) → fix the harness and re-run; never a finding. - Wait for the condition, not a duration:
browser_wait_forthe post-action element, and for async poll with a timeout (for i in $(seq 30); do <check> && break; sleep 1; done). Nevernetworkidle— Playwright's own docs mark it discouraged for assertions.
Bug-type tasks
For task_type=bug, the reporter's own repro steps are case #1 and must pass on the branch — that's the red-green check that proves the fix actually touches the reported symptom, not a nearby path that happens to look related. When the pass looks ambiguous (intermittent, environment-dependent), confirm the same steps still reproduce the bug on the default branch or stage before trusting that the fix explains the difference.
04 同仓库更多 Skills
visual-craft
暂无简介
ux-copy
Use when you write any words a user reads in a mockup or a hand-off - content language and voice, buttons, labels, errors, empty states, confirmations, numbers and dates, and the longest-string check
screen-mockup-html
Use when you draw screens as the design task's HTML mockup - ONE canvas document holding every screen and every variant side by side, the sanitizer-safe recipe, phone and wide frames, the states grid, static control states, and the loopback self-check with screenshots at 375, 768 and 1440 attached to the task
performance-awareness
How the performance score moves for your role. Use when the score context shows a declining trend, or before handing off work you have not self-checked against every criterion.
design-variants
Use when a screen design needs variants - how many, variants as side-by-side rows on the one canvas, what makes two variants genuinely different, recommending one, how the human's choice is read, and narrowing to the chosen one in revision
design-system-authoring
Use when you derive, change or extend a design system - where tokens live per stack, the DTCG token tree, role-based names for both themes, base vs layer vs drift, the DESIGN.md section order, the inventory format, the contrast check and the lint every proposal returns