evidence
Capture truthful before/after evidence for one mapped scenario - real screenshots, short screencasts, CLI transcripts, or HTTP responses from the base and the candidate builds - and compare them into one manifest. Works in any clone without sum or Herdr.
- License 许可
- MIT
- 最近更新
- 2026-11-13
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- evidence
- description
- Capture truthful before/after evidence for one mapped scenario - real screenshots, short screencasts, CLI transcripts, or HTTP responses from the base and the candidate builds - and compare them into one manifest. Works in any clone without sum or Herdr.
02 资源文件 3
- 参考文档references/ATTRIBUTION.md2.1 KB
- 许可证references/LICENSE-before-and-after.txt1.2 KB
- 脚本scripts/evidence_browser.mjs12.3 KB
03 SKILL.md 正文
Evidence
This skill is portable: it needs the sum-verify tool that .agents/skills/verify/scripts/sum-verify runs, and for browser captures Node 22+ and a Chromium-family browser already on the machine.
It does not require sum, Herdr, a .sum directory, or any absolute path outside this checkout.
It never starts, stops, or resets an application and never drives a person's browser profile; you run the base and the candidate yourself, in their own checkouts, and point each capture at the one you drove.
Evidence complements .agents/skills/verify/: a green mise run verify says the automated checks passed, this skill shows a person the user path failing before and passing after.
Images alone prove nothing; every capture also records what the page, command, or response said, and the comparison refuses to call two captures a red/green pair unless the assertions say so.
Procedure
-
Run
.agents/skills/verify/scripts/sum-verify evidence capabilitiesfirst.browser: supportedneeds Node 22+ and a browser binary (EVIDENCE_BROWSER=/pathoverrides discovery). A recipe this machine cannot drive is recordedblocked, never simulated; report such scenarios with the runner's--scenarioinstead. -
Pick the scenario id from the feature maps (
VERIFY.md->feature_maps) and its kind:bugfix(a defect the user could see),feature(nothing to fail before),visual(an intended change of appearance), ornonvisual(CLI/API/library; visual proof is not applicable and no browser screenshot is manufactured). -
Identify the two revisions: the base (merge base of the branch) and the final candidate SHA. Check out each into its own directory (
git worktree add ../base <sha>), build and start each with the repository's own commands on distinct ports. Never reuse a checkout someone else is editing. -
Capture the before state from the base checkout, then the after state from the candidate, with one shared run id and identical settings (viewport, theme, locale, timezone, scenario data, flags):
RUN=$(date -u +%Y%m%dT%H%M%SZ)-$RANDOM .agents/skills/verify/scripts/sum-verify evidence capture --scenario <id> --role before --kind bugfix --run $RUN --checkout ../base --expect-sha <base-sha> \ browser --url http://127.0.0.1:<base-port>/ --step click='#add' --step wait=500 --observe '#count' --expect-selector '#count=Count: 1' .agents/skills/verify/scripts/sum-verify evidence capture --scenario <id> --role after --kind bugfix --run $RUN --checkout ../candidate --expect-sha <candidate-sha> \ browser --url http://127.0.0.1:<candidate-port>/ --step click='#add' --step wait-text='Count: 1' --observe '#count' --expect-selector '#count=Count: 1' .agents/skills/verify/scripts/sum-verify evidence compare --scenario <id> --run $RUN --base <base-sha> --candidate <candidate-sha>The browser recipe dispatches real mouse and keyboard events (
click=,type=SEL=TEXT,press=Enter), waits on visible text (wait-text=), records a screenshot and a screencast of the whole path, and reads the visible text back as the observable end state. Add--side-effect GET <url> --side-effect-text '...'when the journey must also change server state. Add--convertfor an mp4 beside the AVI original when ffmpeg is present. For a CLI or API change usecli -- <command>(transcript with exit code) orhttp METHOD URL(status and body) with--kind nonvisual; the manifest saysvisual_proof: not-applicable. When the base cannot be run (no access, no build), record it honestly instead of inventing a red screenshot:.agents/skills/verify/scripts/sum-verify evidence unavailable --scenario <id> --role before --reason '...'; the comparison is thenafter-onlywith the labelbefore-unavailable. Do not break code on purpose to produce red. A bug fix that already passes at the base is reportedbefore-also-passes. -
Read the comparison.
red-greenmeans the base failed the assertions and the candidate passed them with matching environments and builds.mismatchnames a viewport, theme, locale, recipe, or kind difference you did not declare with--intentional key=value, a capture taken from the wrong SHA (stale build), or an original whose hash changed.capture-failedkeeps the diagnostics (browser-diagnostics.log, driver exit, console) and is never a pass. -
Report the paths:
<evidence>/<run>/<scenario>/comparison.json, thebefore-<sha>/andafter-<sha>/directories, andcomparison.html(a derived two-up view that references the originals and only scales them with CSS). When the checkout is disposable, run.agents/skills/verify/scripts/sum-verify evidence promote --run $RUN --to <durable-dir>before cleanup; it copies the run and re-verifies every recorded content hash, or setVERIFY_EVIDENCE_ROOTto a durable directory before capturing.
Publish into the pull request
.agents/skills/verify/scripts/sum-verify publish places captured screenshots, screencasts, CLI transcripts, and HTTP responses into one pull request's description as a single marked block (<!-- before-and-after:start --> ... <!-- before-and-after:end -->).
Capture, publication, and the block format stay separate: this command never drives a browser or converts video, and the only upload path is the GitHub CLI's own gh pr edit --attach (GitHub CLI 2.99+), running as whoever invokes it with whatever authorization they already hold.
Publication is a deliberate, authorized operation against your own repository; capture never needs it, and a worker or verification script is never handed write credentials to perform it.
.agents/skills/verify/scripts/sum-verify publish capabilitiesreads the installedgh: its version, whetherpr edit --attachexists, and whether it is authenticated. Without--attach, text-only publication still works; a plan containing images or videos is explicitly deferred. Nothing is downloaded or installed per publication..agents/skills/verify/scripts/sum-verify publish plan --run $RUN --repo owner/name --pr N --candidate <sha> [--base <sha>] [--scenario <id>]...validates every comparison in the run and stages approved publish copies under<run>/publish/media/(or--publish-dir), uploading nothing. A scenario is planned only when its comparison names the same candidate, its verdict shows the candidate passing (red-green,before-after,after-only,before-also-passes; nevermismatch,capture-failed,after-fails), every media file resolves inside the run directory without symlinks or parent components, its bytes match the comparison and capture hashes and re-read dimensions, it is within GitHub's limits (10 MiB images, 100 MiB videos), and no secret-shaped text (the capture redaction patterns plus--redact) appears in anything that would reach the PR. A capture whose text was redacted is refused with its screenshot, because a screenshot cannot be redacted. Text/plain transcript and response files are hash-checked against both manifests, checked for secret-shaped content, and staged as immutable publish copies. Text is shown in labeled before/after code blocks with its capture hash; redacted transcripts are permitted, while raw secrets are refused. Code fences accommodate captured backticks, and HTML comment openers are visibly escaped so captured output cannot impersonate publication markers. The full PR body must fit the 65,536-character limit; oversized publication is refused before writing and evidence is never silently truncated. Capture-time clipping remains labeled in the transcript. Only PNG/JPEG/GIF/WebP and MP4/MOV/WebM are attachable; an AVI-only screencast is named as unsupported and the screenshot pair is published without it (capture with--convertto get an mp4)..agents/skills/verify/scripts/sum-verify publish publish --plan <plan.json> --receipts <path> --visibility public|private|internalreads the destination (gh repo view: exact name, visibility, write permission) and the latest PR body, refuses a PR that is not open or whose head is not the candidate (--allow-head-mismatchpublishes it labelled), then edits only the marked block: append when none exists, replace in place when the last block came from these receipts, refuse when the block was written or edited by someone else (--replace-foreign-blocktakes it over after you inspected it). Prose outside the block is copied byte for byte from the body read immediately before the edit; the body is read again before writing and the edit is recomputed when it moved. Text publish copies are hash-checked again before rendering and text-only plans need no uploads. Receipts persist one entry per attachment content hash and destination: a second publication reuses the attachment URLs, uploads nothing, and replaces the same block. Videos go up on their own lines (GitHub renders a player) and become a side-by-side HTML<video>table in a second edit once every URL is known. After every edit the body is read back and checked: the expected evidence content, no unresolved media references, a URL for each attached hash, and unchanged prose; incomplete attachment rewriting may restore the previous body and reportfailed, with uploads retained in receipts. If the observed content or surrounding prose differs from what was sent, preserve that body asbody-observed.mdand returnuncertainwithout overwriting a possible concurrent edit. Missing GitHub CLI or an incomplete capability probe leaves a durabledeferredresult. A timeout after a possible write isuncertainuntil the PR is read back: attachments found there are recorded; an unchanged body means nothing was applied. Exactly-once attachment creation is not promised where the API cannot establish it; receipts andresults/*.jsonunder the publish directory record what happened.- Keep the receipts and publish directory where they outlive a disposable checkout (
--receipts,--publish-dir), as sum does under the task record; the originals under the evidence root are never modified or removed by publication.
The block carries the scenario and feature id, verdict, candidate and base SHAs, the evidence run, cited verification run ids, and who captured it, and says in words that worker evidence is a claim about the candidate build and not independent verification.
Block markup follows vercel-labs/before-and-after (see references/ATTRIBUTION.md and references/LICENSE-before-and-after.txt); its markers are the same, so a block written by that skill is recognised as foreign and left alone unless you take it over explicitly.
What a capture records
capture.json beside the media: scenario and feature id, role and kind, the recipe and its exact steps, the checkout path, HEAD SHA, dirty state and branch, the expected SHA and whether it matched, an optional build id, start/end times, environment (viewport, theme, locale, timezone, browser version, platform, Node version; environment variable names for CLI, never values), every media file with type, bytes, sha256, and real dimensions, frame count and duration read from the bytes, the assertions with met/unmet, observations (redacted), console messages, limitations, intentional differences, and the redaction count.
Screencasts are the original JPEG frames under frames/ plus a Motion-JPEG AVI assembled from them at a fixed frame rate; held repeats are timing only and are counted in the manifest. A converted mp4 is marked derived with the tool and command.
Video is bounded: at most 60 seconds, 1920x1080, 25 MB; the manifest says when a bound stopped the recording.
Recorded text is redacted for bearer tokens, key=/token=/password= shapes, emails, and common API key prefixes (--redact REGEX adds patterns); a redacted transcript is named *.redacted.txt. Screenshots cannot be redacted, so use synthetic data. The browser always runs with a fresh temporary profile, so no cookies, sessions, or extensions of the person leak in. inspect PATH validates any media file by reading it.
Rules
- A capture is never overwritten, edited, or regenerated to look better; a second attempt is a new run id with the earlier one kept.
- Observations are read-only (visible text, title, URL, one selector's text); never manipulate the DOM or internal state to reach the end state.
- The evidence root comes from
evidenceinVERIFY.md(default.artifacts/evidence, Git-ignored) orVERIFY_EVIDENCE_ROOT; run ids keep concurrent captures apart, and each role directory carries the SHA it was taken from. - A comparison is evidence for a person or a root session to read; it does not certify a candidate, replace the verify runner, or bypass the independent review.
- Publication changes nothing about that: a block in a PR is a copy of the worker's claim for reviewers to see, never verification, approval, or a merge.
04 同仓库更多 Skills
verify
Verify a repository through its project-root VERIFY.md contract - run the canonical `mise run verify`, record candidate-bound evidence, and report manual scenarios truthfully. Works in any clone without sum or Herdr.
maintain-verification
Keep a repository's VERIFY.md and feature maps honest when behavior changes - audit stale tasks, paths, links, and coverage claims, find the features a change touches, record a no-map-change rationale for internal changes, and keep authored map edits separate from run results. Works in any clone without sum or Herdr.
create-verification
Bootstrap or extend a repository's project-local verification - the VERIFY.md contract, a `verify` mise task built from checks the project already declares, a feature-map index with seed feature files, and the vendored verify/maintain skills - then prove the generated instructions by running them once. Works in any clone without sum or Herdr.
Sum Verify
Run a distinct coordinator verification of a worker result. Use when a worker reports a candidate or findings, before treating that report as done.
Sum Sweep
Put standing sweeps on a dedicated Sweep worker plus routine. Use for inbox digests and similar repeating work, not for the weekday Inbox status.
Sum Status
Reconcile /workspace/sum/inbox.md with live worker Bots. Use at the start of a turn when work is active, and when the user asks what is outstanding.