AGENT SKILL

canary-lab-run

Use when running or healing an EXISTING Canary Lab feature locally — "run <feature>", "fix the failing run", "rerun 7cvh", "drive the heal loop" — through the repair MCP tools (start_run, wait_for_heal_task, signal_run, get_heal_context, pause_run, cancel_heal, abort_run). Fix failing runs by editing app/service code, not tests. For end-to-end onboarding of a new repo use canary-lab (flight); for deployed-environment checks use canary-lab-verify.

License 许可
MIT
最近更新
2026-11-05
Stars
★ 4
Forks
0

01 前言 Front Matter

name
canary-lab-run
type
skill
description
Use when running or healing an EXISTING Canary Lab feature locally — "run <feature>", "fix the failing run", "rerun 7cvh", "drive the heal loop" — through the repair MCP tools (start_run, wait_for_heal_task, signal_run, get_heal_context, pause_run, cancel_heal, abort_run). Fix failing runs by editing app/service code, not tests. For end-to-end onboarding of a new repo use canary-lab (flight); for deployed-environment checks use canary-lab-verify.

03 SKILL.md 正文

Canary Lab — Run + Heal Loop

User input through MCP 2.0

Use the native Canary connector in the requesting chat so SDK 2.0 elicitation (input_required) reaches the human there. Do not substitute a shell MCP client, answer a form yourself, or recreate the question in chat. Existing explicit instructions and autopilot choices still apply without another ask.

Form decisions have one shared record. The native form links to the same approval in Canary Notifications, also surfaced above the Flight page. On needs-input with approvalId and reviewUrl, show that link and call wait_for_approval with the ID; repeat on still_waiting. The human answers once in either surface. A browser answer resolves the server decision; some clients keep their native form visible until the human dismisses it. Dismissal then returns the stored result. Client decline/cancel alone does not represent a human decision.

Without an approvalId, leave needs-input pending after decline/cancel, stale input, or an unfinished UI action; do not automatically retry or repeat the question. Setup/reconnection can use chat while MCP is unavailable. Never collect passwords, API keys, or access tokens in chat or form elicitation; use the returned Canary UI URL for secret entry.

MCP Invocation

Setup and the plugin expose one public Canary Lab MCP tool: exec (usually rendered as mcp__Canary_Lab__exec). Every Canary Lab tool name below is the exact command value, not a separate public tool. For a feature-scoped command, replace both placeholders in this shape:

{"command":"<exact_tool_name>","arguments":{"feature":"<feature_name>"}}

This is the envelope shape, not every command's complete schema. Add the fields that command declares inside arguments; call describe_tool when a field is uncertain.

Never invent a wrapper verb such as learn or call, embed JSON in a command string, or turn arguments into flags. Keep fields such as confirm: true inside arguments. Use list_tools, search_tools, or describe_tool as the command when discovery is needed. A deliberately selected focused or full profile still exposes atomic tools for debugging; the setup-installed path is compact + exec.

Canary Lab owns the run verdicts and artifacts; this client applies the fixes. These tools arrive via the Canary Lab MCP server. If this client is already connected (the plugin connects with compact), skip this step. To configure the same connection manually: npx canary-lab mcp --profile compact.

Arguments

An invocation argument (/canary-lab-run <suite> — the Getting Started guide's "Repair a Broken Suite" card emits exactly this shape) is a suite (feature) name in the connected workspace: pass it to start_run directly.

Workspace Bootstrap

  1. Find the LIVE server first: read ~/.canary-lab/active-servers.json, which records projectRoot, port and pid for every UI that registered. A stopped server's entry LINGERS — the file is only rewritten when the next server registers — so an entry is a candidate, not proof: the health check below is what confirms it. One entry → that is your server and its port. Several → take the one whose projectRoot is the workspace the user means. None → fall back to ~/.canary-lab/workspaces.json (Windows: %USERPROFILE%\.canary-lab\workspaces.json): one workspace → use it, several → ask which, none → ask the user to run npx canary-lab setup. Do NOT start from a guessed port.
  2. Then CONFIRM it is the right server: curl -s http://127.0.0.1:<port>/mcp/health and check that projectRoot is the workspace you intended. A healthy response does not settle the question on its own — a stale UI left behind by a demo or a tarball smoke test answers a port just as convincingly as the right one, and that is how a flight ends up running in someone's throwaway workspace. projectRoot matches what you intended → continue and tell the user which workspace. It names a DIFFERENT workspace → this is the wrong server; go back to step 1 rather than adopting it. It is under a temp directory (/tmp, /private/var/folders, %TEMP%) → never auto-select it; those are throwaway demo workspaces, so use one only when the user names it explicitly. Only when no live server serves the workspace you want does one need starting.
  3. If the health check fails, start npx canary-lab ui from the workspace in a visible long-running terminal; if this client cannot run long-lived commands, ask the user to run npx canary-lab ui from the workspace and confirm when it's up.
  4. A healthy /mcp/health means the server is live. On the setup-installed compact profile, atomic names such as get_feature_coverage are deliberately absent from tools/list; only exec is public. Call exec with {"command":"list_tools","arguments":{}} before concluding the connection is missing. Only an unknown-tool error for exec means this session is not connected — ask the user to run npx canary-lab setup --force and reconnect/restart the client, then retry. Never drive /mcp with a hand-written HTTP/JSON-RPC client (curl included; the health check above is the only direct HTTP use): a custom client bypasses client detection and reconnect handling.

Keep ownership until an explicit outcome

A successful heal claim makes this run your unfinished task. Claiming does not start a background repair agent.

  • After start_run returns an ordinary run with your heal claim, or claim_heal succeeds, your next run action is wait_for_heal_task with that runId and the same session_id. If you needed to load this skill or get_workflow_guide(workflow: "repair"), make that call immediately after reading it. A promise to wait is not the call.
  • On still_waiting, call the bounded wait again. On needs_heal, work on this run's repair, then signal_run once and wait again. Give progress updates between calls; do not end the turn with “done while waiting” or switch to suite authoring, coverage mapping, or another run while this duty is unresolved.
  • The wait sends heartbeats only while its call is open. During a long repair, use the run-specific heartbeat with status: "healing" at work milestones before the claim goes stale. Chat, unrelated tool calls, and get_run_actions do not supervise this run. Do not send heartbeats merely to keep an abandoned task appearing active.
  • If a tool fails, distinguish a retryable call error from a permission denial or missing connection. Correct an invalid argument and retry the same run; never bypass a denied action through shell commands or a different API. If you cannot continue, tell the user the run ID, last confirmed state, exact blocker, and the action needed to resume or stop. Do not claim it is paused, cancelled, or supervised unless a tool result confirms that outcome.
  • Before yielding to unrelated work or ending with an active run, resolve who will drive it or follow the user's stop instruction. To stop an in-flight heal when the user authorizes stopping repair, call cancel_heal and check its result; it ends the run as failed. Do not substitute abort_run for this heal action or claim there is no direct stop tool without checking cancel_heal. An abort_run denial does not prove cancel_heal is unavailable, but a denied stop action is not permission to try alternate tools for the same effect. abort_run is for a user-requested full abort and requires the human stop form. pause_run enters heal mode, and release_heal only releases ownership: neither parks nor cancels the run. If stopping is blocked or needs user input, report that the run is still active and needs attention.

External Run Loop

If start_run returns type: "getting_started_busy", a Getting Started demo already owns the workspace. Follow the returned active target (a run, Flight, coverage job, draft, portify, or export) in its current owner; do not start another workflow.

  1. Call list_features and choose the requested feature. Then pick the mode from the target: a run boots the suite's services from its repos and repairs that code. When the target is a deployed, staging or live host the suite does not boot ("the staging run", "check prod"), use canary-lab-verify instead — a repair cycle cannot change a remote deployment, so the only "fix" left would be editing the test.

  2. Call start_run with claim_heal: true, a stable session_id, and a useful conversation_name. Do not pass client_kind — the MCP bridge auto-detects it from the connection; guessing it yourself can mis-set it and suppress heal claim. Heal claiming is open to interactive Claude/Codex clients (Desktop or CLI alike) — only runner-spawned PTY agents are blocked — so an ordinary CLI session like this one can own the heal loop. For requests like "rerun 7cvh", pass run_ref: "7cvh". Before a fresh run, stale test-to-requirement coverage opens a human elicitation: update coverage first, or run now with stale coverage. Never answer it yourself. Update-first returns type: "coverage_update_required" and starts nothing: respect activeJobId / flightId ownership, complete the returned coverage next steps, confirm freshness, then retry start_run. Run-now is diagnostic; coverageStale: true means the historical percentage is still not current. Decline/cancel starts nothing. Reusing an active run or restarting a failed/aborted run with run_ref continues its recorded suite without this gate. Before a fresh run, a suite that boots no services and whose envset names a deployed host opens a human elicitation: verify the deployed target, or run with repair anyway. It is asked before coverage. Never answer it yourself. Verify returns type: "verify_instead" with the deployed targetOrigins and starts nothing — continue with canary-lab-verify (list_verification_configs → execute_verification). Only if the fallback type: "remote_target_requires_choice" reaches you, ask the user; only after they choose run-anyway re-call start_run with remote_target: "run". Never pass it on your own.

  3. A run boots the commit the feature's repo checkout is sitting on — not the branch's latest. If the user wants the current upstream tip ("run against latest main", "pull first"), pass update_repos: true: before the worktree is cut, Canary Lab does git fetch + git merge --ff-only on each pinned repo. A repo declared with track: 'upstream' in feature.config.cjs does this on every run without the flag; update_repos: false boots the checkout as-is. If start_run returns type: "repo_update_refused", nothing started: repos[] says per repo why (dirty, diverged, detached, wrong-branch, fetch-failed, in-use). Tell the user which repo and why and let them reconcile — never stash, reset, or discard their work to make the update go through — then re-call start_run (or re-call with update_repos: false to boot the checked-out commit anyway). get_feature_repo_status reports behindUpstream / aheadUpstream so you can say how stale a checkout is before starting. If start_run returns type: "repo_collision_requires_choice", another run is using the same app/repo. The tool requests this choice through elicitation when supported. Only if this fallback result reaches you, ask the user whether to run this one isolated in a per-run git worktree (runs now, concurrently) or queue it until the other run finishes, then re-call start_run with isolation: "worktree" or isolation: "queue". Do not guess. If start_run returns queued: true, tell the user the run is parked (queueReason) and will start automatically when capacity frees; wait_for_heal_task still blocks until it starts and needs fixes.

  4. If start_run returns an active run, continue that run. But if it returns type: "boot_session" (or executionType: "boot"), the run is a held boot-only session — services are up, no tests run, and there is no heal task. Do not claim heal or call wait_for_heal_task; report that services are ready and that the user can stop them through the abort_run human confirmation form when done. A service that fails its readiness probe is marked failed (its status shows timeout) but the session stays held — boot never self-aborts on a health-check failure, so report which services came up and which failed; only abort_run tears it down.

  5. If start_run reports already-claimed, stop and tell the user which session owns the run. If it returns claimSuppressed: true, this session is a runner-spawned PTY agent (the benchmark/portify sessions Canary Lab launches itself) and cannot own the heal loop — interactive clients are not suppressed, so you normally won't hit this. The run still runs in External-client heal mode (it does not fall back to the project's configured heal agent — it waits for a drive); do not call wait_for_heal_task. Report the run id and tell the user to drive heal from an interactive Claude/Codex client or the web UI.

  6. Handle user interrupts explicitly: "pause", "intercept", or "pause and heal" means call pause_run; "stop heal" or "cancel repair" means call cancel_heal; "abort", "kill the run", or "stop everything" means call abort_run only with the required confirmation.

  7. Call wait_for_heal_task with the runId from start_run and the same session_id. It blocks for a short bounded window and heartbeats for you. If it returns type: "still_waiting", the run is still active and the window just elapsed — this is not terminal: immediately call wait_for_heal_task again with the same runId + session_id. Loop on still_waiting until you get needs_heal / passed / failed. (If it ever returns type: "boot_session", the run is a held boot-only session — report services are up and stop here; do not wait again.)

  8. If it returns passed, summarize using result.counts.statusLine and stop.

  9. If it returns failed, report the terminal status using result.counts.statusLine and relevant failure summary.

    • If result.newRunRequired is present, the suite's one external-effect attempt was already claimed. The app repair is captured but unverified. Review/promote its Changes patch, obtain the suite's required approval, and start a fresh run without run_ref. Do not restart or hand off the spent run.
    • If result.serverExited is present, the Canary Lab server driving the run stopped mid-run, so its services and heal loop stopped too and the run is settled aborted — no verdict, nothing left to signal. signal_run, pause_run and cancel_heal on such a run answer server-exited the same way. Relay serverExited.message, then restart it with start_run and run_ref set to its run id; its recorded suite and journal carry over.
    • If the result carries dirtyTests (a test file changed since the last green run — can appear on passed, failed, or needs_heal), relay result.dirtyTests.message to the user verbatim (e.g. "⚠️ Tests have been modified, please review.") alongside the outcome. Do not block, gate, re-run, revert, or edit the test files to "fix" it — it's an awareness signal so the user can review or commit the change, not an error to act on.
    • If the result carries specEdits (test files changed after this run started — can appear on passed, failed, or needs_heal), the run used the tests recorded at run start, so the changed files were not tested and the result says nothing about them. Relay result.specEdits.message and follow result.specEdits.nextSteps: call get_test_review(runId), show the exact patch (read patchPath if needed), then call review_test_changes(runId, review_revision) WITHOUT wait_for_decision. The human chooses Accept & commit or Restore recorded files; cancel/decline leaves pending. Acceptance binds approval to the displayed revision, commits every reviewed file including helpers/fixtures/config, and returns a durable receipt with Git and execution outcomes. For an active run continue with wait_for_heal_task and do not signal twice; for an ended passed/failed/aborted run call start_run without run_ref when the receipt says new-run-required. Acceptance preserves the old result and is never a pass. reviewUrl is optional side-by-side inspection, never a required operational step. If elicitation is unavailable or the client declines without a recorded decision, say no human decision was recorded; with a blocked request_id use the browser watcher rule below. For other reviews, use a capable client or the human-chosen browser fallback. A Git commit, clean tree, restart, or chat statement is not a persisted review receipt; only the explicit decision creates one. Changed revisions require a fresh review. No MCP tool can self-approve a test-file change, and a passed run with pending specEdits is a pass of the recorded tests — never report the changed tests as passed. result.specEdits.hints is the advisory check: a kind: "weaker" hint means an assertion was removed or loosened relative to the test that ran — restore it, a weaker assertion is never a repair; kind: "cannot-classify" means review by hand. Quote specEdits.disclosure with any hint you quote.
  10. If it returns needs_heal, treat the returned heal context as the compact first-stop packet — which part of it matters depends on the situation:

    • context.singleAttempt.claimed true: the suite's receipt says this run used its one external-effect attempt. Fix app/service code, then signal_run once with the journal fields. Canary finalizes this run failed/unverified and captures its patch; it does not restart services or rerun Playwright. Call wait_for_heal_task for the terminal result. A fresh approved run with a new ID is required to verify the promoted fix.

    • context.failedTests non-empty, first cycle (context.healPrompt present): inspect context.healPrompt.startHere first, then use context.healPrompt.resources, current failures, and the checked-out source code. The packet is slim — context.healIndex and context.journal are paths (Read them when needed), and each context.failedTests[] entry carries a failureId plus pointer dirs (errorPath, traceDir, playwrightMcpDir).

    • cycle ≥ 2 (context.guidance present, no context.healPrompt): context.healPrompt and context.nextSteps are sent on the first needs_heal only — on repeat cycles the context carries just the changed failure packet plus a context.guidance breadcrumb. Reuse the cycle-1 map, or call get_heal_context to re-fetch it.

    • context.escalation present: the same tests failed 3+ cycles running — you're stuck. Read context.escalation.readFirst and follow context.escalation.tactics (change tactic — revert or build on the prior diff, don't fire a fresh unrelated hypothesis) instead of repeating the last fix.

    • context.failedTests empty AND context.bootFailure set: a service failed to boot — no tests ran. Inspect its reason/classification, command/cwd, exit/signal, and bounded redacted excerpt before Reading the full logPath. classification: "underlying-cause-not-preserved" means an outer wrapper did not preserve the underlying rejection or child-process failure: do not guess a root cause; fix that product wrapper to log and rethrow. Then fix the service/app code and signal_run kind: "restart" (context.nextSteps already reflects this).

    • context.serviceFailure set: a required service failed after readiness. Playwright may have partial results; do not infer failures or passes for tests that did not run. Inspect the recorded kind, excerpt, and service log, fix the service/app code, then signal_run kind: "restart". This failure enters healing automatically; signal_run means the fix is ready for runner verification.

    • context.dependencyBlockers present: startup is blocked by incompatible dependencies. Each record names the repository, all affected services, a stable cause, requiredAction, worktreePath, and logPath when available. Repair deterministic in-scope problems without elicitation. Ask only for a genuine user choice or missing authority, such as changing another shared checkout. Never offer continue-anyway, mark-compatible, or a revision change as a bypass; different generator inputs do not prove that generated output differs. Fix app/service dependencies, not tests or the validator. Then signal_run kind: "restart" on the same run. Both restart and rerun perform a fresh dependency preflight and persist its evidence before any service spawn or test verification. These blockers remain in repeat waits; get_heal_context recovers them after reconnect.

    Call get_run_snapshot only when you need the verbose raw summary, full counts, or deeper debugging fields.

  11. Follow the run diagnosis policy in context.nextSteps or its heal prompt. The default is parent-only: read the failure index yourself, diagnose every failure in the parent, and do not spawn diagnosis sub-agents. Keep a failure-ID ledger — each failure with its evidence, candidate cause, and addressed or explicitly unresolved status; similar symptoms or a shared service log alone do not prove a common cause. If contradictory evidence or a failed verification cycle needs delegation, record the escalation (failure IDs + reason) in your hypothesis first; escalated children are read-only and return evidence plus a proposed patch. An explicit per-failure or adaptive policy overrides this default for that run. Under per-failure, when several tests fail, dispatch one read-only sub-agent per failure in a single parallel round (up to 5 at once), hand each the failureId, and have it call get_failure_detail(runId, failureId) to investigate just that slice and report back a hypothesis plus a concrete proposed patch (the exact file edits / unified diff). Under every policy, sub-agents are read-only — they must not touch the working tree or call signal_run — and you apply all fixes yourself, serially, reconciling by hand if two touch the same file, then signal_run once. Applying and signalling stay single-threaded. A sub-agent that comes back empty, or a failure you could not explain, has not been cleared — say so in the hypothesis rather than signalling as though its test were addressed.

  12. Fix app/service code, not tests, unless the test is provably wrong.

    • Edit where the run actually booted. When the heal context carries context.worktrees (repo name → path), this run is isolated and the services under test were booted from those per-run git worktrees — every portified feature runs this way, and so does any run the user isolated after a collision. Apply your fix under the context.worktrees path. context.repoBranches[].path is the product repo, which an isolated run never reads (repoBranches[].sha is the commit the run booted, and updatedFromUpstream says when the start pulled to get there): patch only that and the rerun comes back byte-identical, which reads as "my fix did not work" when the fix was right and simply never loaded. The worktree edits are captured at teardown, so nothing is lost. When context.worktrees is absent, the run boots the repos in place and repoBranches[].path is the edit target.
  13. Call signal_run once per cycle with kind: "rerun" for test-only/app-code fixes that do not need service restart, or kind: "restart" when services or env need restarting. Include hypothesis and fixDescription; Canary Lab writes the journal from that signal and its observed git diff. One accountable signal per cycle, even when you fixed several failures.

  14. Do not call a separate journal-writing tool; the runner records failing tests, changed files, signal, outcome, and diff.

  15. Repeat from wait_for_heal_task (looping on still_waiting) until the run passes or reaches terminal failure.

wait_for_heal_task result types

Result Action
still_waiting Not terminal — re-call with the same runId + session_id.
needs_heal Fix (see step 10), signal_run once, then wait again.
passed Report result.counts.statusLine, plus what happened to your fix (see below), then stop.
failed Report result.counts.statusLine + failure summary, stop.
boot_session Report which services came up, stop — never wait again.

What happens to your fix when the run passes

Your edits are never left only in your head or only on disk. At teardown Canary Lab diffs the run's working copy, saves the result as a patch per repo, and — unless the workspace turned it off — opens a draft pull request from it. So on passed:

  • Do not open a pull request yourself, and do not push a branch. The run does it, on a branch named per feature + repo, so healing the same feature again updates that same pull request instead of opening a second one.
  • Report what the run reports: the pull request URL when there is one, or the per-repo reason there isn't (gh not signed in, no push rights, the patch no longer applies). Both live on the run's prAttempt; the diff itself is on the run's Changes tab in the Canary Lab UI.
  • A run that ends red, or that gives up after its cycle cap, opens nothing — a fix that didn't make the tests pass is not a fix to propose.

Guardrails

  • Envset values belong in features/<feature>/envsets/<env>/<slot> in the selected workspace. A suite .env is a materialized consumer target, not a second source. Never create pointer-only envsets or target .runtime/envsets, an envset source, or a personal source checkout merely because an old variable points there. If the user authorizes an envset repair, read get_workflow_guide(workflow:"author") and trace source → target → consumer before updating the values and consumer together. Otherwise report the mapping problem; repair permission for app code does not authorize a suite migration.
  • session_id recipe: generate one id at the start of the conversation (any unique string) and pass that identical value in every tool call for the rest of the conversation.
  • When a blocked start returns request_id, carry it into get_test_review and review_test_changes. On an accepted receipt, resume with start_run(feature, request_id, session_id) in the original external client; Restore recorded files cancels that external request, so stop without starting. Canary resumes internal requests itself. Never create a replacement request or switch owners. If the form is unavailable or the client declines without a recorded human decision, show reviewUrl and keep waiting: spawn exactly one background agent for this request if supported, otherwise wait in this turn. The watcher calls get_test_review in its own MCP session for a fresh read-only browser_wait_token, then repeats review_test_changes with wait_for_decision:true. It stops on restored/cancelled or changed revision. After reconnect, get a fresh token and read the same durable request. A wait, transport notification, Git commit, or agent-written argument never approves changes.
  • A FLIGHT can hand you a run via an external-work checkpoint carrying a runId: the run was started in external-heal mode UNCLAIMED for you — claim_heal it with your own session_id and drive this exact loop, then re-call get_flight and release with respond_flight_checkpoint(flightId, choice: "submit") once the run is terminal (a failed run is a valid terminal answer; Canary reads the verdict from the run record). If that re-check carries checkpoint.data.takeoverRequestedAt, stop your repair work and release with choice: "run-internally" instead — do not submit. Never abort_run a flight-owned run to escape it, and never start another run for that feature while the flight owns it. The flight will wait on you indefinitely — a parked hand-off has no deadline — so do not end your turn with it open; after 45 minutes with no get_flight contact the read reports handOffIdle on it.
  • heartbeat is a low-level liveness refresh for long local repair stretches. wait_for_heal_task heartbeats while waiting, and signal_run and get_heal_context refresh liveness, so call explicit heartbeat only before or after a long stretch of local Read / Edit / Write / Bash work.
  • start_run is the single entrypoint for start/resume/restart intent. A fresh run pauses for a human choice when coverage mapping is stale; no agent flag bypasses that gate. A matching active run is continued even with force_new:true; intentional concurrent runs of the same feature start from the Run panel. Across different features, a same-app collision returns repo_collision_requires_choice (resolve with isolation: "worktree" or "queue"); over the resource budget, the run is queued and starts automatically.
  • For requests like "rerun 7cvh", start_run resolves the run suffix and restarts that same failed/aborted run in remaining-test mode. Canary Lab reruns failed tests first, then skipped tests, then pending/not-run tests; do not tell the user no test filter exists.
  • To re-execute a run, reuse it: prefer this rerun (start_run with run_ref) over abort_run + a fresh start — rerun re-executes only the failed/skipped/pending tests and avoids the abort + repo-collision dance. The rerun already re-runs skipped and pending tests, so it is complete — do not force_new just to avoid "skipped" tests. On a portified feature, force_new spins a brand-new per-run worktree and resets the heal journal to Iteration 1, discarding the prior cycles. Start a fresh full run only when prior passes are genuinely invalidated (e.g. a global data/state change), and you rarely need to abort_run first to do so.
  • After changing code or tests, never call start_run to verify. The signal requests runner verification; it is not a claim that the fix already passes. Do not start services or run Playwright, smoke, end-to-end, or other runtime checks yourself. Canary Lab owns affected-service restart, health checks, and targeted Playwright verification after the signal. If an edit command failed or syntax is uncertain, run at most one fast non-network static check before signalling. Verification means signal_run with hypothesis and fixDescription, then wait_for_heal_task on the same runId.
  • wait_for_heal_task waits while a submitted signal is pending and while a live runner transitions from a test failure into healing. Only a finalized run returns a terminal result; keep waiting on this run.
  • During normal healing, omit run_ref/force_new so start_run continues the active run. force_new cannot replace an active run. Pending suite changes block fresh starts until get_test_review then review_test_changes records the human's Accept & commit or Restore recorded files decision for the exact revision. The acceptance receipt says whether the active run reran or a new run is required; start the latter without run_ref. Restarting preserves recorded tests and review history but cannot turn acceptance into tested evidence.
  • Environment exclusions must be declared in the recorded suite before execution with the canary:environments annotation. counts.notApplicable stays part of counts.skipped, never counts.passed; other skips and not-run tests still block completion. Never add an exclusion to evade a failure. Changing applicability during a repair requires the same human test review as any other test edit.
  • Never compute passed count as summary.total - summary.failed.length. Use result.counts.statusLine, result.counts.passed, or summary.passed for pass counts. Treat tests absent from passedNames, failed, and skippedNames as not run, not passed.
  • Do not call abort_run unless the user asks. Its human form must be accepted; agent-written confirm:true is not authorization. If forms are unavailable, the human uses Stop in the Run panel. Never answer that form yourself.
  • Prefer compact get_heal_context when you need to refresh failure artifacts outside the wait loop. Use get_run_snapshot only for verbose fallback/debugging context.
  • Read responses are slim by default to protect your context, but the full data is always one step away — pull it whenever a failure needs deeper detail (just never in a wait loop): get_run omits the raw lifecycleEvents/playwrightArtifacts/playbackEvents (call it again with includeRaw:true to inline them); get_run_snapshot returns the heal-index and journal as paths to Read; list_runs returns the newest 20 (raise limit).
  • On any terminal run, recommend the exact next skill invocation /canary-lab-export <runId>, replacing the placeholder with the terminal run's returned ID. Do not substitute npx canary-lab export; the export skill uses the connected MCP server and preserves the exact run selection.
  • Record concise, factual journal notes. Do not paste raw transcripts.
  • When the run is waiting for external heal, Canary Lab is the source of truth for status, artifacts, and rerun/restart signals.

04 同仓库更多 Skills

ferterahadi
ferterahadi/canary-lab

off-by-one

暂无简介

测试与质量Skill与智能体开发
★ 4
ferterahadi
ferterahadi/canary-lab

multi-failure-cascade

暂无简介

测试与质量Skill与智能体开发
★ 4
ferterahadi
ferterahadi/canary-lab

broken-delete-contract

暂无简介

律师与法务测试与质量Skill与智能体开发
★ 4
ferterahadi
ferterahadi/canary-lab

canary-lab

Use when the user wants Canary Lab to take a product repo end to end — "test this app", "onboard this repo", "run a flight", "evaluate what I just built" — through the flight pipeline (start_flight / get_flight / respond_flight_checkpoint over MCP). One conducted pipeline continues a configured suite or takes bare repo(s) to a green, covered, healed run and publishes a downloadable Report; Parallel setup continues as Canary-owned background work. Use a focused skill only when the user asks for that single capability without asking for a Flight or /canary-lab.

测试与质量Skill与智能体开发
★ 4
ferterahadi
ferterahadi/canary-lab

canary-lab-verify

Use when verifying a Canary Lab feature against a RUNNING target — a deployed environment ("verify checkout on staging", "run the saved verify config", "check the deployed app") or a locally booted app ("verify a running app") — through the verify MCP tools (list/get/create/update_verification_config, boot_services, execute_verification, get_verification_result). No heal loop. For run + heal use canary-lab-run; for end-to-end onboarding use canary-lab (flight).

测试与质量Skill与智能体开发
★ 4
ferterahadi
ferterahadi/canary-lab

canary-lab-repair-discovery

Repair Playwright test discovery for an existing Canary Lab suite, reporting live progress to the Tests column through MCP. Use when the test list cannot load or Canary offers an In your agent discovery-repair command. Does not execute tests or repair a failing test run.

测试与质量Skill与智能体开发
★ 4