post-deploy-verification
Use when a release reaches awaiting_verdict (or you are woken for an incident) - how to read runtime logs, error groups, health and smoke evidence for the release's component and decide finish, re-check, roll back or hand to a human
- License 许可
- Apache-2.0
- 最近更新
- 2026-11-15
- Stars
- ★ 122
- Forks
- 18
01 前言 Front Matter
- name
- post-deploy-verification
- source
- obra/superpowers (MIT), adapted
- category
- release
- description
- Use when a release reaches awaiting_verdict (or you are woken for an incident) - how to read runtime logs, error groups, health and smoke evidence for the release's component and decide finish, re-check, roll back or hand to a human
03 SKILL.md 正文
Post-deploy verification
A deploy that went green tells you the process started. It says nothing about whether the change it shipped actually works, or whether it broke something else that happened to keep running. Verification is what closes that gap, and it happens every time — not only when something looks wrong.
1. Procedure — exact calls
get_release({ task_id })→ readdeployed_at,component,checks(health samples, smoke results,new_errors,notes,early_stop),profile.auto_rollback, andtasks[]— every task this release covers, not only the one you were woken on.get_environmentonly if you need to confirm nothing is bound (checks.notesalready says so in most cases).list_runtime_errors({ component, since: "<window that starts well before deployed_at>" })— see "Reading error groups" below for the window and how to judgenew.query_runtime_logs({ component, since: deployed_at, min_severity: "error" }); re-query withtext:set to a route or module the release changed when the error list alone is not enough.run_smoke_checks({ task_id })when the stored evidence is older than ~30 minutes, or whenearly_stopitself came from a smoke failure and you need a fresh read before deciding.- One read-only acceptance check per observable criterion of every task in
release.tasks(list_acceptance_criteriaper task) — a release with several tasks is verified in full, not just the card that woke you.
2. Reading error groups
list_runtime_errors' new: true is accurate on a provider with native grouping (gcloud). On a provider without one (Vercel, AWS — fallback fingerprint grouping), it only means "this group's first occurrence fell in the back half of whatever window you queried" — querying since: deployed_at puts a regression that starts right after the deploy in the window's front half, so it can read new: false and never show up as a concern.
- Judge a group by its own
first_seenagainstrelease.deployed_at, never bynewalone. - Read with a window that starts well before
deployed_at— roughly twice the time elapsed since the deploy, or at least the soak period — so an occurrence from before the deploy is visible and can be told apart from one that started with it. - Before rolling back on one low-count group, re-read with
since: "1d". If the same message recurs beforedeployed_at, it is chronic, not a regression. Once you have confirmed a group is chronic and unrelated to any release,save_memory(project scope) a one-line note of its fingerprint message ("redis timeoutin apps/api recurs hourly; not a release signal") so a future run of this skill recognizes it without a full 1-day re-read. checks.new_errors(the sweeper's own field) can be empty on Vercel or AWS even when errors genuinely began with the deploy — your own read in this run is the check, not a formality that confirms what the sweeper already decided.
3. Verdict table
Evidence maps to one of four outcomes — finish, re-check once, roll back, or hand to a human — the same shape as Argo Rollouts' Successful/Failed/Inconclusive and Flagger's "N failed checks before rollback": one blip is not a trend, and an unreadable signal is not a pass.
| Evidence | Verdict |
|---|---|
early_stop: smoke check failed |
Re-run run_smoke_checks once. Passes, and no error group has first_seen >= deployed_at → finish, naming the transient failure and the re-check. Fails again → rollback_release (verify_failed). |
| Health check failed twice | fetch_url the checks.health_url once (GET). Still failing → roll back. Recovered → look for a restart/crash line in the logs since deployed_at; clean → finish, noting the blip. |
An error group with first_seen >= deployed_at whose message, path or stack touches a file or route this release's tasks changed (get_task_pull_request({ task_id, include_diff: false }) → files) |
Roll back (verify_failed). |
Same, but unrelated and it also recurs before the deploy in a 1d read |
Finish, naming each group and why it is not a regression. |
Only structural gaps in notes (no bound environment, no health URL, no base URL) |
Finish, stating the gap verbatim — it is a different kind of verification, not a failed one. |
| A runtime tool itself errored | Retry once. Still failing, and health and smoke are both green → finish, saying the logs were unreadable this run. |
Nothing at all is readable for an on_merge/dispatch service (no health, no smoke, no logs, and the retry above also failed) |
One comment: "verdict needs a human: no evidence was readable (<what you tried>)" — and stop. A human finishes or rolls back from the Deploy tab. This is the one case where "finish with the gap stated" is not the default: with nothing readable there is no evidence to finish on. |
Recording a missing deploy target
A notes gap like "no health URL" or "no base URL" does not always mean nobody can fix it — if the deploy's own output (get_deploy_logs, a local_run.tail, or a workflow log) names the real URL the service answers on, update_deploy_target({ env, base_url / health_url / logs_url }) records it so the next soak and verdict actually have something to read, instead of hitting the same gap every release. Still finish or roll back on the evidence you have in THIS run — recording the target fixes the next one, it does not retroactively create evidence for this one.
4. Feature-flagged changes
If a task's after_deploy (on release.tasks[]) enables a flag or config, its new behaviour is not reachable yet — that is expected, not a failure. Check that the old path still works, and say "criterion X is behind flag Y, enabled after release by a human" in your note. A flag flip is a manual step for a human, never something you do yourself.
5. Notes that count as evidence
✅ "query_runtime_logs (apps/api, since 2026-10-01T10:02Z, ≥error): 0 entries; list_runtime_errors 40m: 2 groups, both first_seen before deploy (chronic redis timeout, 1d read shows it hourly); smoke 3/3 (re-run 10:41); T-12 GET /api/orders?status=paid → 200 with paid_at field."
❌ "Looks fine, logs clean."
6. Rationalizations (adapted from obra/superpowers)
- "Deploy went green" → that is the process starting, not the behaviour it shipped.
- "Soak was green" → read it yourself, in THIS run — the sweeper's own fields are a starting point, not a substitute.
- "
new_errorsis empty" → on Vercel or AWS that can miss an error that genuinely started with the deploy; readfirst_seenyourself. - "The error is probably old" → show the
first_seen, don't assert it. - "One failed probe" → re-check once, then decide — never roll back, and never finish, on a single sample.
Common Mistakes
- Omitting
componentin a monorepo — silently reads the root component's logs, not the release's own. - Reading errors with the default 24h window instead of a window anchored before
deployed_at. - Judging novelty by
newinstead offirst_seenagainstdeployed_at. - Verifying only the task you were woken on when the release carries several.
- Writing to production (seeding, creating records) to check a criterion.
- "Should", "probably", "seems" anywhere in a finish/rollback note.
Batch releases with no runtime environment
Most desktop and mobile components have no bound runtime environment at all — there is no host to sample health from, no log stream to query, no error groups to list. That is not an incomplete verification, it is a different one: see batch-artifact-verification for what to check instead (the published artifact, store build, or local run), and say explicitly in the finish_release/rollback_release note that no runtime environment is bound — the same rule as any other gap: never let silence read as a pass.
04 同仓库更多 Skills
visual-craft
暂无简介
ux-copy
Use when you write any words a user reads in a mockup or a hand-off - content language and voice, buttons, labels, errors, empty states, confirmations, numbers and dates, and the longest-string check
screen-mockup-html
Use when you draw screens as the design task's HTML mockup - ONE canvas document holding every screen and every variant side by side, the sanitizer-safe recipe, phone and wide frames, the states grid, static control states, and the loopback self-check with screenshots at 375, 768 and 1440 attached to the task
performance-awareness
How the performance score moves for your role. Use when the score context shows a declining trend, or before handing off work you have not self-checked against every criterion.
design-variants
Use when a screen design needs variants - how many, variants as side-by-side rows on the one canvas, what makes two variants genuinely different, recommending one, how the human's choice is read, and narrowing to the chosen one in revision
design-system-authoring
Use when you derive, change or extend a design system - where tokens live per stack, the DTCG token tree, role-based names for both themes, base vs layer vs drift, the DESIGN.md section order, the inventory format, the contrast check and the lint every proposal returns