noyalib-coverage-campaign
Raise noyalib line, branch, and function coverage toward the 98% target. Use for 'raise coverage', llvm-cov gaps, coverage-gap triage, ratcheting fail-under thresholds, or picking the next coverage target. Fences off the known wrong turns (chasing 100%, weakening thresholds).
- License 许可
- MIT
- 最近更新
- 2026-11-09
- Stars
- ★ 1
- Forks
- 0
01 前言 Front Matter
- name
- noyalib-coverage-campaign
- license
- MIT
- metadata
- { "agtmls-owner": "Sebastien Rousseau", "agtmls-bundle": "noyalib", "agtmls-maturity": "project", "agtmls-risk-level": "medium", "agtmls-writes-files": "true", "agtmls-network-access": "optional", "agtmls-handles-secrets": "false", "agtmls-executes-commands": "true", "agtmls-requires-human-review": "true" }
- description
- Raise noyalib line, branch, and function coverage toward the 98% target. Use for 'raise coverage', llvm-cov gaps, coverage-gap triage, ratcheting fail-under thresholds, or picking the next coverage target. Fences off the known wrong turns (chasing 100%, weakening thresholds).
- allowed-tools
- Read Glob Grep Write Edit Bash WebFetch WebSearch
- compatibility
- Requires cargo, git. Tested with Claude Code, Codex, and Aider skill layouts
02 资源文件 2
- 参考文档metadata.json411 B
- 参考文档reference.md17.4 KB
03 SKILL.md 正文
noyalib-coverage-campaign
Purpose. Run the coverage-hardening campaign to specification, unsupervised, on a Sonnet-class session. This skill is the decision procedure, not a tutorial: at each phase there is a command, an expected observation, and a gate that says whether to proceed, retry, or escalate.
Scope guard. This skill covers only the coverage campaign. For the wider "how do we test noyalib at all" question (test layers, proptest, fuzz, Miri, spec suite) use noyalib-validation-and-qa. For interpreting a specific tool's output (llvm-cov flags, cargo-vet, deny) use noyalib-diagnostics-and-tooling. Do not duplicate those skills here.
Progressive disclosure. This SKILL.md is the campaign spine. Extended
material — bucket-classification walkthroughs, verbatim house-pattern
snippets, "why not" rationale for each fenced wrong-path, expanded
solution-menu prefer-when notes, and the invariant_violated
file:line inventory — is in
reference.md. Load reference.md when the compact
guidance below is insufficient; each section flags its
cross-reference.
0. Baseline numbers (VOLATILE — re-verify at campaign start)
Captured 2026-07-08 from .github/workflows/ci.yml:160-162,
.github/workflows/shared-coverage.yml reusable defaults,
doc/TESTING.md §"Coverage gate", and PLAN.md §1 "Coverage baseline":
| axis | live workspace gate (ci.yml) |
shared-coverage.yml default | doc/TESTING.md (stale) |
target |
|---|---|---|---|---|
| functions | 96 % | 96 % | 95 % | 98 % |
| lines | 94 % | 94 % | 93 % | 98 % |
| regions | 93 % | 93 % | 92 % | 98 % |
Two threshold layers exist and both must be understood:
- Live workspace gate (
.github/workflows/ci.yml:160-162, 96/94/93) — the values a PR is actually measured against today. This matches theshared-coverage.ymlreusable-workflow defaults; the workspace caller does not override them. doc/TESTING.mddocumented axis (95/93/92) — documentation drift. The doc lags the workflow files by one ratchet step. Trust the workflows; fix the doc in the same PR that next moves the gate.
These numbers drift. Before touching anything, run Phase 0 and treat the axis printed there as ground truth. If Phase 0 disagrees with this table by more than 1 pp on any axis, update this section in the same PR that closes the gap that caused the drift.
Phase 0 — Baseline: what do we actually cover today?
Command (canonical):
make coverage-gap
# equivalent to:
# ./scripts/coverage-gap-report.sh # 98 % threshold (default)
# ./scripts/coverage-gap-report.sh 95 # ratchet to a custom floor
Under the hood this runs (verbatim from scripts/coverage-gap-report.sh):
NOYALIB_COVERAGE=1 cargo +nightly llvm-cov \
--workspace --all-features --no-fail-fast \
--ignore-filename-regex '' \
--summary-only
Expected observation. A TSV block, header file regions lines functions,
one row per file below the threshold, and on stderr a one-liner
<N> files below <T> %. The full raw summary is teed to
/tmp/noyalib-coverage.log.
Gate.
- Zero files below threshold → campaign done for this ratchet step; advance to Phase 5 to ratchet the gate numbers.
- N > 0 → continue to Phase 1 with the punchlist.
If the tool errors: toolchain mismatch → cargo +nightly;
missing nightly → rustup toolchain install nightly --component llvm-tools-preview. Numbers wildly off → confirm --all-features.
CI cache poisoning symptom (cargo check under 2 s + drifting
numbers) → coverage jobs use CARGO_TARGET_DIR=target-coverage
(memory feedback_ci_cache_poisoning); locally cargo clean and
rerun. Extended error triage in
reference.md §6.
Phase 1 — Target selection: pick the ≤ 3 files that matter most
Rule of thumb. Rank Phase 0 rows by
(uncovered_lines × load_bearingness) where load-bearingness is:
| tier | modules | weight |
|---|---|---|
| S | parser/, scanner/, loader.rs, de/, ser/, error.rs |
3 |
| A | value.rs, borrowed.rs, cst/, document.rs |
2 |
| B | fmt.rs, path.rs, public re-exports |
1 |
| C | convenience wrappers, Display shims, example-only helpers |
0.5 |
Parser / loader / scanner / error paths outrank convenience code, always.
A file at 90 % in parser/events.rs beats a file at 85 % in a Display
shim (memory feedback_security_first_posture: correctness on
adversarial input IS a security control).
PR discipline gate. Pick ≤ 3 files per PR. One logical change
per PR (memories project_main_branch_protection +
feedback_no_premature_phasing). Bundling five files into one "coverage
sweep" makes the ratchet PR hard to revert per-file if one addition
regresses.
Entangled files — e.g. parser/events.rs gaps only close when
scanner.rs gaps close — may live in the same PR as a single logical
change. Say so in the description.
Phase 2 — Gap analysis: what kind of gap is each uncovered region?
Command. For a specific file:
NOYALIB_COVERAGE=1 cargo +nightly llvm-cov show \
--workspace --all-features \
--show-line-counts-or-regions \
--ignore-filename-regex '' \
-- crates/noyalib/src/parser/events.rs # example
Or the whole workspace with per-file annotated HTML:
NOYALIB_COVERAGE=1 cargo +nightly llvm-cov --html --workspace --all-features
open target/llvm-cov/html/index.html
Classification. Every uncovered region falls into one of four buckets. This decision drives the rest of the campaign:
| bucket | signal | action |
|---|---|---|
| (a) | Reachable error path (an Err(Error::…) return, a bail on bad input) |
Write a test that triggers exactly that input; assert the ErrorKind variant. |
| (b) | Feature-gated code (#[cfg(feature = "…")], #[cfg(target_arch = …)]) |
Feature-matrix test leg — the workspace's --all-features still misses cfg-arch. |
| (c) | Defensive-unreachable — a _ => arm that calls invariant_violated |
NOT a test target. Confirm the annotation, document exclusion in PR. |
| (d) | Dead code — genuinely unreachable, not defensive | Do not test. Open a change-control PR to remove it. Coverage rises by subtraction. |
How to spot (c) vs (d). (c) is guarded by
#[cfg_attr(noyalib_coverage, coverage(off))] and calls
crate::error::invariant_violated(...). If an arm is NOT annotated,
that's the bug — annotate it. (d) is code you can prove no input can
reach; if unsure, write the test first — failure to trigger is
empirical evidence for (d).
Extended walkthroughs (signal / action / common trap per bucket) and
the current invariant_violated file:line inventory are in
reference.md §1 and §3.
Phase 3 — Test authoring: the house pattern
Location. crates/noyalib/tests/coverage_<slug>.rs. The existing
family is ~20 files including coverage_scanner.rs,
coverage_error.rs, coverage_loader{,_full}.rs,
coverage_borrowed.rs, coverage_ser.rs, coverage_de.rs,
coverage_fmt.rs, coverage_value.rs,
coverage_spanned_anchors.rs, plus historical sweep files
(coverage_final*.rs, coverage_100.rs, coverage_remaining.rs).
Naming. Prefer topic-scoped names (coverage_scanner) over sweep
names (coverage_final_sweep4). Sweep names are historical; don't
add more.
Every file must:
- Start with the SPDX header and a module doc naming the source file whose gaps it closes.
- Assert semantics — every test asserts a specific behaviour, not "some output happened". Wrong-but-green is the declared enemy.
- On error-path tests,
matches!on theErrorKindvariant — noterr.to_string().
Canonical style references (mirror these three verbatim):
coverage_scanner.rs, coverage_error.rs, coverage_loader_full.rs.
Verbatim snippets (SPDX header, error-path shape, anti-pattern,
"any outcome is fine" exception) are in
reference.md §2.
Doctest option. If the file being covered has a public API without
a # Examples section, a doctest closes the coverage gap AND satisfies
the rustdoc-coverage goal in PLAN.md §1 (missing_docs = warn → deny,
≥ 98 % items). Double win; prefer this on the public surface.
Phase 4 — Verification gate
Command. Re-run Phase 0:
make coverage-gap
Expected. Targeted files no longer appear in the punchlist, or appear with all three axes ≥ 98 %. Workspace totals rise by the uncovered-region contribution of the files closed.
Gate.
- Targeted files ≥ 98 on all three axes → proceed to Phase 5.
- Improved but still below 98 → back to Phase 2 for the residual regions. Common cause: an error path with three variants, test hit two.
- Unchanged → the test isn't actually executing the intended path.
dbg!on entry, runcargo test --test coverage_<slug> -- <name> --nocaptureunderNOYALIB_COVERAGE=1, confirm the region enters.
If a file refuses to close the last percent: check for
#[cfg(target_arch = …)] (cross-target is out of scope per PLAN.md
§7.8), macro-expanded lines (llvm-cov show quirk on macro_rules!;
document as at-ceiling), or #[cfg]-conditional match arms needing
a different feature subset (--no-default-features --features alloc
for the no_std leg — only worth it for tier-S). Extended notes in
reference.md §6.
Principled exclusion. If a region is genuinely (c) or (d) but
cannot be excluded via #[cfg_attr(noyalib_coverage, coverage(off))],
document location + reason in the PR — change-control question, see
noyalib-change-control.
Phase 5 — Promotion: normal change control
Standard flow, per memory project_main_branch_protection:
- Branch from
main, feature branch. - Local run of the full pre-push gate (see noyalib-build-and-env or noyalib-ci-and-release for the exact command list).
- PR into
main—mainrejects direct pushes. - Wait for CI green on all shared workflows including the coverage
gate (memory
feedback_ci_must_be_green). gh pr merge --squash --adminonce green.- If the campaign has fully closed a threshold step (e.g. all files
≥ 98 % lines), a separate PR ratchets the numbers in both:
doc/TESTING.md§"Coverage gate"- the workspace's caller of
shared-coverage.yml(itsfail-under-*inputs) - and, if this is the reference for satellites, the defaults in
.github/workflows/shared-coverage.ymlNever bundle a threshold ratchet with a test-authoring PR — two separate logical changes, different risk profiles.
Success is measured, not eyeballed. The number of record is the
coverage-gate job (currently 96 fn / 94 lines / 93 regions per
ci.yml:160-162 + shared-coverage.yml defaults), not a local
terminal readout. If CI says "failed under 94 lines" and local says
"94.02", CI wins — check CARGO_TARGET_DIR isolation and re-run.
Known wrong paths (fenced — terse list)
Per-item "why not" rationale is in reference.md §4.
- Chasing 100 %. Unreachable by construction; ceiling is ≥ 98 %.
- Weakening thresholds to pass. Never lower a
fail-under-*inshared-coverage.ymlordoc/TESTING.mdto green a red PR. - Assert-free coverage tests. Wrong-but-green is the declared enemy.
- Duplicating existing coverage under a new name. Grep the
coverage_*.rsfamily first; extend if it exists. - Testing
invariant_violatedarms directly. Annotate them; do not reach them viaunsafeor hand-forged inputs. - Touching
${1:-98}inscripts/coverage-gap-report.shsilently. It's contract with the maintainer. - Bundling with unrelated refactors. Two logical changes = two PRs. Coverage PRs must be trivially revertible per-file.
The solution menu (ranked — brief)
Given a specific gap, prefer higher-ranked techniques first. Extended
when-to-prefer rationale in reference.md §5.
- Targeted unit / integration test — default. Use for (a) reachable-error-path gaps and single-function invariants.
- Proptest generator — for combinatorial / shape-family gaps (scalar shapes × chomping × indent; escape sequences; type tags).
- Doctest on the public API — closes coverage AND moves rustdoc coverage toward PLAN.md §1 in one commit.
- Fixture-driven spec test — YAML file under
tests/fixtures/plus expected event stream / value. For YAML-semantic gaps (anchor/alias corners, merge-key ordering, spec-clause edges).
None replaces the others — a well-covered file uses all four.
Provenance
scripts/coverage-gap-report.sh— exactcargo +nightly llvm-covinvocation, TSV output, default threshold 98 %..github/workflows/ci.yml:160-162— livecoverage-gateinputs: 96 fn / 94 lines / 93 regions..github/workflows/shared-coverage.yml— reusable workflow. Defaults 96/94/93 (caller doesn't override).CARGO_TARGET_DIR: target-coverageisolation is load-bearing (memoryfeedback_ci_cache_poisoning).doc/TESTING.md§"Coverage gate" — stale (95/93/92); trust the workflows. Also lists legitimate exclusions (noyalib-wasm/src/lib.rs, MCP/LSPprotocol.rs).PLAN.md§1 + §7.8 — 98 % target, ratchet plan, cross-target matrix as future work.crates/noyalib/tests/coverage_*.rs— ~20-file house pattern. Style refs:coverage_scanner.rs,coverage_error.rs,coverage_loader_full.rs.crates/noyalib/src/error.rs::invariant_violated(line 1697) — the#[cfg_attr(noyalib_coverage, coverage(off))]mechanism. Full call-site inventory inreference.md§3.- Makefile
coverage-gaptarget — canonical entry point.
Sibling skills (out of scope; delegate):
- noyalib-validation-and-qa — proptest / fuzz / Miri / spec-suite.
- noyalib-diagnostics-and-tooling — llvm-cov flags, cargo-vet / deny.
- noyalib-change-control — ratchet-PR mechanics.
- noyalib-ci-and-release — pre-push gate; shared-workflow
caller-permission unioning (memory
feedback_shared_workflow_caller_perms).
04 同仓库更多 Skills
example-skill
Use when the user needs this reusable workflow. Replace this with trigger-rich wording that says when to load the skill, what task it handles, and when not to use it.
writing-plans
Turn a vague or multi-step request into an ordered list of small steps, each with its own done-condition and verification. Load before starting work that touches several files, has unclear scope, or spans more than one session, and when a request needs decomposing before any code is written. Produces a plan a reviewer can check against, not a to-do list.
web-research-and-source-triage
Use when gathering current web evidence, comparing sources, triaging credibility, reading GitHub repositories, checking docs, or deciding whether a claim needs live verification.
vetting-a-skill-before-install
Use before installing, importing or enabling a third-party skill, plugin, marketplace or skills repository in Claude Code, Codex, Antigravity, Cursor or any agent. Pins the source to a commit, audits every skill and the agent configuration it ships, reads what the audit could not cover, and ends in install, mirror or refuse. Not for auditing this registry's own skills.
verification-before-completion
The gate between doing work and claiming it is done. Load before saying finished, fixed, working, or ready, before marking a task complete, and before reporting an outcome to a human. Converts each claim into the observation that would justify it, and names the rationalizations that let unverified work ship. Not the project's own evidence bar or test taxonomy.
using-agtmls
Meta-router for the AgtMLS skill catalog. Read first when unsure which AgtMLS skill applies, when several look relevant, when asking 'is there a skill for porting or noyalib work', or when onboarding to how the hub's skills and shared references fit together.