verification-quality

verification-quality — Evidence Before Claims

Overview

Claims without evidence are noise. "It works", "tests pass", "I fixed it" — none of these mean anything until verified against the codebase, the test suite, and the build.

This skill wires four concepts to monomind's real command surface: truth scoring (a 0.0–1.0 confidence score from citable evidence), an evidence-before-claims protocol (require file:line, test names, or build output before marking done), auto-rollback (regression detection via analyze diff, revert via git), and a multi-angle quality workflow (correctness, tests, security, performance, docs).

Core principle: NO CLAIM IS TRUE UNTIL VERIFIED AGAINST EVIDENCE.

Iron Law:

NO "DONE" WITHOUT A TRUTH SCORE ≥ 0.95 BACKED BY CITABLE EVIDENCE

If you cannot point to a file:line, a passing test name, or a green build, you have not finished. You have started.

When to Use

Use for ANY task that ends in a claim of completion: "I implemented X", "the bug is fixed", "tests pass", "ready to ship", or any agent-returned work.

Use this ESPECIALLY when: the agent (or you) is in a hurry — that's when false claims slip in; the change touches security, money, auth, or data integrity; a previous attempt already failed; you're about to commit, push, open a PR, or merge.

Don't skip when: "it's a tiny change" (tiny changes break tests too) or "I'm sure" (confidence without evidence is the failure mode this skill prevents).

The Real Command Surface

These are the ONLY commands this skill wires to. Anything else is invented.

Command	What it does	Phase
monomind analyze diff	Git diff risk + change classification	Evidence, regression
monomind analyze code	Static code analysis	Verification
monomind analyze deps --security	Dependency CVEs	Verification
monomind analyze complexity	Cyclomatic complexity	Verification
monomind analyze symbols	Extract functions/classes/types	Evidence
monomind analyze imports	Import graph	Verification
monomind security scan	Vulnerability + secret scan	Verification
monomind security secrets	Dedicated secret detection	Verification
monomind security audit	Security audit log	Verification
monomind performance benchmark	Run benchmarks (wasm/memory/search)	Evidence
monomind performance metrics	View/export metrics	Monitoring
monomind performance bottleneck	Identify bottlenecks	Verification
monomind doctor / doctor --fix	28 health-check categories	Baseline, monitoring
monomind hooks metrics	Learning-hook metrics	Monitoring
monomind hooks intelligence	Neural/MoE/HNSW status	Monitoring
monomind monograph search	Knowledge graph search (BM25/semantic/hybrid)	Evidence
monomind monograph build	Build/rebuild the knowledge graph	Baseline
monomind tokens dashboard	Token spend	Monitoring

Use npx monomind@latest ... from outside the repo; inside the repo node packages/@monomind/cli/bin/cli.js ... works too. Never use monomind@alpha — it does not exist.

MCP tools (called by Claude Code, not the CLI)

Tool	Use
mcp__monomind__hooks_pre-task	Capture task intent + acceptance criteria before work
mcp__monomind__hooks_post-task	Record outcome + evidence after work
mcp__monomind__monograph_query	Find file:line for a symbol before citing it
mcp__monomind__monograph_impact	Blast radius before risky edits
mcp__monomind__monograph_context	360° callers/callees for the change site
mcp__monomind__system_health	Snapshot system health before declaring done
mcp__monomind__system_metrics	Objective metrics for the verification record

---

Core Concept: The Truth Score

A truth score is a 0.0–1.0 confidence value derived from evidence you can cite, not a feeling. The default ship threshold is 0.95. The score is computed in Phase 3 from real command output; the action mapping lives in Phase 4. You do not invent it.

---

The Four Phases

Complete each phase before moving on. Skipping a phase produces unverified claims.

Phase 1: Evidence Collection

BEFORE claiming work is done, gather evidence it actually works.

1a. State the claim precisely:

"I claim X is done. Acceptance criteria: [list]. Evidence required: [list]."

1b. Capture task intent (MCP) and cite the change site via the knowledge graph:

mcp__monomind__hooks_pre-task({ task: "...", acceptance: ["...", "..."] })
mcp__monomind__monograph_query(symbol: "refreshToken")     # cite before editing
mcp__monomind__monograph_impact(file: "src/auth/refresh.ts")

Every cited location must be file:line. "Somewhere in auth" is not a citation.

1c. Capture the diff as evidence:

npx monomind@latest analyze diff --risk --classify -v > evidence-diff.json
npx monomind@latest analyze imports src/auth --external    # import-graph blast radius

Success criteria: acceptance criteria written, diff classified on disk, every changed symbol cited as file:line.

---

Phase 2: Multi-Angle Verification

Run each angle that applies. All applicable angles must pass or the truth score drops. Skip an angle only when it genuinely does not apply (and say so).

Angle 1 — Correctness (always applies):

npx monomind@latest analyze code src/auth/    # static analysis on touched paths
npm run build && npm run typecheck            # use the project's real commands

Angle 2 — Tests (always applies when tests exist):

npm test -- --reporter=spec

The evidence is test names + counts, not "tests passed".

Angle 3 — Security (applies to auth, crypto, input boundaries, deps):

npx monomind@latest security scan
npx monomind@latest security secrets
npx monomind@latest analyze deps --security

Angle 4 — Performance (applies to hot paths, queries, bundles):

npx monomind@latest performance benchmark -s all -i 100 -o json > bench.json
npx monomind@latest performance bottleneck
npx monomind@latest analyze complexity src/auth/ -t 10

Angle 5 — Documentation (applies to public APIs, behavior changes):

npx monomind@latest analyze symbols src/auth/refresh.ts   # did docs track changes?

If exported symbols changed and docs didn't, this angle fails.

Angle 6 — System health:

npx monomind@latest doctor

A red doctor category blocks the claim, even if the code looks fine.

---

Phase 3: Truth Score Computation

Score each applicable angle — judgment against a checklist, not a vibe:

Angle	1.0 (full)	0.5 (partial)	0.0 (fail/no evidence)
Correctness	Build + typecheck + analyze code clean	Typecheck clean, build warnings	Build or typecheck fails
Tests	All relevant tests pass, names captured	New tests pass, one pre-existing flake	Any relevant test fails
Security	security scan, secrets, deps --security clean	One informational finding, no exploit path	Any HIGH/CRITICAL or leaked secret
Performance	Benchmark within baseline, no new hotspot	Within ±5% of baseline	Regression vs. baseline
Documentation	All changed symbols documented	Minor export undocumented	Public API change with no doc update
System health	doctor green	Yellows acknowledged	Any red category

Composite score formula:

truth_score = (sum of angle scores) / (number of applicable angles)

A failing angle zeroes its row. Any 0.0 angle caps the composite at 0.85 — critical findings always block shipping regardless of the average.

Write the score and per-angle evidence to the task record:

mcp__monomind__hooks_post-task({
  task: "Fix auth refresh race", outcome: "complete", truth_score: 0.96,
  evidence: {
    diff: "evidence-diff.json",
    tests: "auth.test.ts: 42 passed, 0 failed",
    security: "scan clean; deps --security 0 HIGH",
    performance: "benchmark within 1.2% of baseline",
    health: "doctor green",
    citations: ["src/auth/refresh.ts:87", "src/auth/refresh.ts:134"]
  }
})

---

Phase 4: Decision — Ship, Fix, or Rollback

Use the composite score from Phase 3:

Score	Decision	Required action
≥ 0.95	Ship	Record evidence via hooks_post-task; proceed to commit/PR
0.85–0.94	Ship with caveats	Record the gaps explicitly in the PR description
0.75–0.84	Fix	Return to Phase 1 with the failing angle as the new task
< 0.75	Rollback	See Auto-Rollback below; do not leave broken code on the branch

3 or more fix loops without reaching 0.95 → architectural problem. Stop, discuss with the user. Do not attempt a 4th loop. (Same rule as mastermind-debug Phase 4.5.)

---

Methodology: Auto-Rollback

When a change scores below 0.75, or a regression is detected after merge, revert. monomind does not have a magic verify rollback subcommand — rollback is git + evidence from analyze diff.

1. Confirm the regression is real:

npx monomind@latest analyze diff --risk -v              # was it high-risk at review?
npx monomind@latest analyze complexity src/ -t 15 -f json
npx monomind@latest performance benchmark -s all -i 100 -o json > now.json
# diff now.json against the saved baseline

2. Roll back to the last known-good state:

git log --oneline -10
git revert <bad-commit> --no-edit          # preserves the diagnosis in history
# or, if nothing downstream depends on it:
git reset --hard <last-good-commit>

3. Prevent recurrence:

# Add a regression test for the failure mode BEFORE re-attempting (see mastermind-debug Phase 4)
npx monomind@latest doctor                  # re-verify the rolled-back state
npx monomind@latest security scan

Rules:

• Never rollback silently — record what failed and why in the postmortem.
• Selective rollback (revert one file, keep another) is fine if analyze diff shows the changes are independent. Otherwise revert as a unit.
• Always re-verify the rolled-back state passes the failing angle.

---

Methodology: CI/CD Integration

Wire the same four phases into CI so unverified work cannot merge.

GitHub Action — quality gate on PRs:

name: Quality Verification
on: [pull_request]
jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }   # analyze diff needs history
      - run: npm ci
      - name: Build + typecheck + tests   # Angle 1 + 2
        run: |
          npm run build
          npm run typecheck
          npm test
      - name: Diff risk + security        # Angle 3
        run: |
          npx monomind@latest analyze diff main..HEAD --risk --classify --format json > diff-risk.json
          npx monomind@latest security scan
          npx monomind@latest analyze deps --security
      - name: Health + benchmark          # Angle 4 + 6
        run: |
          npx monomind@latest doctor
          npx monomind@latest performance benchmark -s all -i 50 -o json > bench.json
      - uses: actions/upload-artifact@v4
        with: { name: verification-evidence, path: "diff-risk.json\nbench.json" }

Prefer risk classification (qualitative, stable) over hard latency thresholds (fragile, flaky) for the gate. Use metrics for trend analysis offline.

---

Methodology: Continuous Monitoring

Verification is not just a PR gate. Keep watching after merge:

npx monomind@latest doctor                                          # daily health
npx monomind@latest performance metrics -t 7d -f json > "metrics-$(date +%Y%m%d).json"
npx monomind@latest hooks metrics                                   # what hooks learned
npx monomind@latest hooks intelligence                              # neural/HNSW status (usually not-loaded)
npx monomind@latest tokens dashboard -p week --no-interactive       # spend surprises → quality problems

For long-term storage, pipe performance metrics -f prometheus into Prometheus and alert on trend, not on single values.

---

Red Flags — STOP and Return to Phase 1

Thought / Action	What it means
"It works" with no test names or file:line	No evidence. Phase 1.
"Tests pass" with no output captured	Untested claim. Re-run and capture.
"It's a tiny change, skip verification"	Tiny changes break tests too. Phase 2.
"Security probably isn't affected"	Probably ≠ verified. If auth/crypto/input touched, run security scan.
"Performance feels fine"	Feeling is not measurement. Run performance benchmark.
Skipping an angle without saying why	Silent skips are how bugs ship. State "N/A because…".
"Doctor has a red but it's unrelated"	Verify the unrelated-ness, don't assume.
3+ fix loops, still < 0.95	Architectural problem. Stop, discuss design.
Merging with score 0.85 "to unblock"	Below threshold is below threshold. Fix the gap.
"The agent said it's done" / "it compiled"	Agent claims and clean compiles are inputs to verify, not conclusions.

---

Related Skills

• mastermind-debug — root-cause methodology when verification finds a failure; Phase 4 covers failing-test-first evidence collection
• performance-analysis — Phase 2 Angle 4 deep-dive (pack extras — monomind packs add extras)
• mastermind-receive-review — same rigor applied to incoming review feedback
• swarm-orchestration — every agent output runs through Phase 1–4 before merge

Quick Reference

Phase	Key commands	Success criteria
1. Evidence	analyze diff --risk, monograph query/impact, hooks_pre-task	Acceptance criteria + cited file:line on disk
2. Verification	analyze code, security scan, performance benchmark, analyze complexity, doctor	Every applicable angle scored
3. Truth score	Composite formula above	Numeric score + per-angle evidence recorded via hooks_post-task
4. Decision	git (rollback when < 0.75)	Ship ≥ 0.95, fix 0.75–0.94, rollback < 0.75

---

Version: 2.0.0 · Last Updated: 2026-08-12