AGENT SKILL

deniz-dotnet-general.test-gap-analysis

Performs report-only pseudo-mutation analysis of existing tests: find caller-visible production-code changes that current assertions would not catch. Use when asked whether a bug or mutation could survive, where tests are behaviorally blind, or which production outcomes lack protection. Polyglot. DO NOT USE FOR: writing tests, proof that tests catch an identified fix or feature (test-value-evidence), suite organization or taxonomy, coverage/CRAP interpretation, assertion/smell audits, or running a mutation-testing tool.

License 许可
MIT
最近更新
2026-11-13
Stars
★ 1
Forks
0

01 前言 Front Matter

name
deniz-dotnet-general.test-gap-analysis
description
Performs report-only pseudo-mutation analysis of existing tests: find caller-visible production-code changes that current assertions would not catch. Use when asked whether a bug or mutation could survive, where tests are behaviorally blind, or which production outcomes lack protection. Polyglot. DO NOT USE FOR: writing tests, proof that tests catch an identified fix or feature (test-value-evidence), suite organization or taxonomy, coverage/CRAP interpretation, assertion/smell audits, or running a mutation-testing tool.

02 资源文件 1

03 SKILL.md 正文

Test Gap Analysis

Answer one question: which caller-visible production behaviors could change without an existing test failing? Mutation reasoning is a probe, not the goal. Inventory public outcomes first, then verify only credible gaps.

Decision flow

1. Set scope

Discover production and test files from manifests and file types. After a narrow search misses, inspect the current directory broadly before asking for paths.

Request Action
One component or named risk Inventory every high-risk public outcome in scope; do not edit production code unless verification was requested
General small-component review Inventory distinct outcomes and report caller-visible gaps from source/assertion mapping
Explicit survivor verification Inventory all requested outcomes; execute one representative observable candidate for each distinct high-risk outcome under verification, then classify it as Survived or Killed
Explicit exhaustive audit Read references/mutation-catalog.md and classify all meaningful candidates
Add or create tests Report the verified gaps; test writing is outside this skill

When the request names a risk, turn it into a one-line public-outcome allowlist before reading code. An outcome is not in scope merely because the same method writes it. For money math, allow computed or returned amounts, rates, tier/boundary choice, percentage base/order, floors/caps, and rounding; exclude non-monetary state predicates (including derived booleans), identity, and formatting. Private code is in scope only to trace an allowed outcome.

Do not expand a focused request into a repository audit, plan artifact, or dashboard. Use source and tests directly for familiar frameworks. When discovery or assertion semantics are unclear, load the deniz-dotnet-general.test-analysis-extensions skill and read its matching language file; do not search installation directories. If it is absent, state the limitation and use the pinned APIs and rules here.

2. Establish one baseline

For a source-only advisory review, skip execution and label empirical survival unverified. A baseline is required when the user requests execution or survivor verification. Run the narrowest existing test command once. Choose it from the project manifest; Microsoft.Testing.Platform executables may require dotnet run. Confirm tests executed: exit 0 with build-only output is not green. If that one attempt cannot run the suite, do not troubleshoot the runner or try alternate commands for an advisory review; continue statically and label all candidates unverified. Do not infer a project-configuration cause from missing output; name a cause only when the command reports it.

Missing runner output limits only claims of empirical mutation survival. It does not make source-proven facts tentative: a public outcome with no reaching test is still No coverage, and an exact expected value derived from the unmodified implementation is still actionable. State the baseline limitation once, then give the static source/assertion conclusion directly instead of hedging every row.

For an advisory review such as "would tests catch this?", stop execution after that baseline. Source-to-assertion mapping is sufficient evidence for No coverage and Candidate survivor (unverified). Trace or run the unmodified code once only when an original value is unclear. Apply mutations only for explicit verification or an exhaustive audit.

Any focused mutation budget limits execution, not discovery. Keep every distinct unasserted public outcome in the inventory.

3. Inventory public outcomes

For each public entry point, map:

  • input partitions: classifier arms, compound conditions, invalid and nearest-valid guard boundaries, and default cases;
  • each independent observation: returned field/variant, exception type, invalid-input acceptance, public state transition, or external side effect;
  • private-helper composition, constants/rates, rounding, retries, cancellation, and error propagation as observed through the public caller.

Use public input/sequence -> expected outcome -> existing assertion -> gap. One asserted return field does not cover another. One allowed result does not cover its denial.

Money math: inventory the no-op path, every rate/tier and exact boundary, operation order, percentage base or composition, floor/cap, and rounding. Trace private helpers through the public result. A test asserting only a broad range does not pin any exact amount. For each actionable money row, derive one witness input and its exact original result through the complete call chain; do not recommend a generic "assert the exact amount" without supplying that amount.

Ordered guards and retries: inventory invalid below minimum | first valid | last allowed or retryable | first blocked | later blocked. For an upper guard such as value >= limit, use limit - 1, limit, and limit + 1; the last witness exposes narrowing to value == limit. Inventory every accepted and rejected error class. When type matching is polymorphic, include a representative derived accepted type that would expose exact-runtime-type narrowing. A test at the first blocked value does not protect the last allowed or later blocked value.

Authorization: enumerate each relevant identity/role, resource class, and action from the caller's view. Untested false, forbidden, and unchanged-role outcomes are first-class security gaps. Do not analyze variants of an allowed path while a denial outcome remains uninventoried. Check each public surface:

  • permission-returning APIs: every distinct role/resource class and every returned capability independently;
  • action-dispatch APIs: each read/write/delete-style action branch, especially paths that must return denial;
  • role/state transitions: accepted, rejected, invalid, null, and empty inputs, including outcomes that must leave state unchanged.

Reserve execution for wholly untested public branches before another variant of a partially covered helper. If more than five high-risk behaviors are unasserted, execute the top 3-5 and keep the rest visible as No coverage or Candidate survivor (unverified).

Execution never replaces the ledger. Before mutating or answering, classify every required outcome, including each invalid input, guard boundary, classifier arm, action, and denial.

Completeness checkpoint: before selecting findings, explicitly account for every independent mode/flag, both zero and negative for a <= 0 guard, every accepted exception class, and a representative derived accepted exception when matching is polymorphic. For a removed guard, trace the fallthrough: if it still produces the same public exception type, it is equivalent unless finer exception metadata is an established contract.

4. Admit only observable candidates

First replay each exact mutation against every existing asserted input or sequence with all arguments fixed. Any changed return, exception, state, or side effect is Likely killed; a dedicated single-purpose test is unnecessary. Never compare the mutant on one input with the original on another.

For survivors, choose a witness before execution or reporting and state witness -> original observation -> mutant observation. Reuse it in the smallest test. Admit it only when the last two differ publicly after tracing the full call chain; otherwise choose a distinguishing witness or drop it.

Exclude:

  • edits that require inserting or reordering statements rather than changing or removing an existing expression, condition, constant, return, or side effect;
  • edits that do not compile, including removal of a declaration whose value is still referenced;
  • overflow behavior, exception message/ParamName metadata, or other semantics not established by the current contract, source intent, or tests;
  • a removed guard or short-circuit that falls through to the same result, exception, state, and side effects;
  • private representation changes that every public input sequence observes identically, even if the suite stays green;
  • a mutation whose proposed test passes against both original and mutant;
  • boundary edits that return the same value on the distinguishing input; for example, changing result < floor ? floor : result to <= is equivalent at equality because both branches return floor;
  • a standalone auto-property or trivial one-line wrapper/predicate with no meaningful branch, calculation, or side effect, unless the user names it;
  • hypothetical future impact, generated code, logging/formatting-only changes, impossible values, and duplicate syntax variants.

Missing direct assertions do not prove No coverage: first trace existing assertions through public callers and shared branches. Missing assertions make an observable candidate a survivor; they do not make an inert mutation meaningful.

5. Rank and classify

Rank: (1) security denials, financial outcomes, errors, and state changes; (2) wholly unasserted public outcomes; (3) boundaries or exact values reached by weak assertions; (4) alternate variants of already-asserted behavior.

Finish the inventory before selecting mutations or a verdict. One killed attempt, exception type, or switch arm does not clear its siblings.

Choose the verdict from the completed inventory:

  • Strong when core branches and primary boundaries are protected and only a few validation or default-case variants remain;
  • Mixed when meaningful coverage exists but at least one important outcome partition is unprotected;
  • Weak when important outcomes are broadly unprotected.

A handful of validation gaps does not make an otherwise broad suite Mixed unless validation is the named risk or the gaps threaten security, data, or other contract-critical behavior.

When the inventory meets the Strong criteria above, lead with Strong and name the protected boundaries and dual assertions before listing minor gaps. Do not open with Mixed, "only core paths", or a risk-heavy dashboard.

Stop when existing assertions kill the remaining candidates or no credible public survivor remains. Do not mutate every operator merely to fill a report or calculate a score.

Result Meaning
Likely killed An existing assertion observes the changed outcome
Candidate survivor (unverified) Observable change appears unasserted; not executed
Killed Exact observable mutation executed and a relevant assertion failed
Survived Exact observable mutation executed and tests stayed green
No coverage No test reaches the public outcome; report the missing branch without inventing a survivor
Equivalent No public observation changes; omit from findings

Outside explicit verification or an exhaustive audit, execute no mutations. Do not mutate to confirm obvious no coverage. For explicit verification, execute one representative candidate per distinct high-risk outcome in scope; do not stop after the first one or two while another guard, action branch, error class, or denial remains unclassified. Omit equivalent syntax variants.

6. Verify without creating false positives

Enter this phase only for explicit verification or an exhaustive audit.

  1. Apply one candidate and confirm the diff changes exactly one intended expression.
  2. Run the narrowest covering test and confirm tests executed: green means Survived; a failure at a relevant assertion means Killed, for that edit only. Build, discovery, infrastructure, or unrelated failures leave the candidate unverified, not Killed.
  3. Revert immediately and confirm the clean source/test baseline.
  4. After a green run, re-check the public counterfactual. Execution proves the suite missed the edit, not that the edit changes behavior; drop inert or unobservable mutants.

Never leave mutations in the workspace. Before reporting, reconcile every unasserted high-risk outcome as Survived, Candidate survivor (unverified), No coverage, or omitted Equivalent. Stop when no credible public gap remains; do not fill a report with internal details or calculate a score unless the user requested an exhaustive audit.

Output contract

Scale the response to the request.

For focused or small analysis, return:

  1. A one-line verdict: Strong, Mixed, or Weak, with the reason.

  2. For a Strong suite, one short strengths sentence naming the concrete protected boundaries, guards, or paired observations that justify the verdict.

  3. One compact row per actionable Survived, Candidate survivor (unverified), or No coverage outcome. Before adding a row, apply the outcome allowlist when the request names a risk, then apply the observable-candidate rules; omit any candidate that fails either filter. Include every high-risk outcome, use one row per distinct public outcome, and consolidate only related low-risk variants:

    Risk Public outcome Change Result/evidence Smallest test

    Every gap needs a distinguishing witness and a concrete smallest test. An error-path gap must name an invalid input and the expected error/result.

  4. For a Mixed or Weak suite, one short strengths sentence naming important killed behavior.

  5. When the request names exclusions, one short scope sentence naming the generated, trivial, or unrelated code intentionally skipped.

Do not repeat the table in prose or report discarded mutants, tool chronology, or in-flight reasoning.

For an exhaustive audit, separate executed Killed / Survived counts from static Likely killed / Candidate survivor (unverified) classifications and No coverage / Equivalent inventory counts. Never include static reasoning in an empirical killed/total ratio. Group findings by risk.

Reliability rules

  • A passing test that does not assert the changed outcome does not kill a mutation.
  • Coverage is per behavior partition. One switch/ternary arm or compound input does not prove siblings: allow does not prove deny; read does not prove write; null does not prove empty or whitespace when those inputs have different caller-visible outcomes. A kill clears only the edit and path that ran.
  • Private helpers reached through a public method remain in scope.
  • Error semantics are language-specific: in Rust, ? propagation versus panic is observable behavior; in C#, exception type and whether an input guard accepts or rejects a value are observable behavior.
  • Cross-check every exact amount or boundary result against the unmodified implementation or an existing exact assertion. If it cannot be checked, state the behavioral relation without inventing a number.
  • Do not label a finding high-risk merely because a mutation survived.
  • Never recommend a redundant test for behavior the existing suite already protects.

Validation

  • [ ] Scope stayed proportional to the request
  • [ ] The original suite passed, or static-only limits are explicit
  • [ ] Every high-risk public outcome in scope was inventoried
  • [ ] Original and mutant have different caller-visible observations
  • [ ] Every outcome labeled Survived was executed; unexecuted candidates use Candidate survivor (unverified)
  • [ ] Every temporary mutation was reverted
  • [ ] Findings exclude trivial, generated, and equivalent changes
  • [ ] Recommendations target only demonstrated gaps
  • [ ] Every public entry-point branch and each accepted exception type in scope is explicitly accounted for

04 同仓库更多 Skills

Blind-Striker
Blind-Striker/agent-skills-and-plugins

writing-tunit-tests

Write tests in TUnit — the attributes, the async assertion model, data-driven cases, lifecycle hooks, and the parallelism rules. USE FOR writing or fixing a test in a TUnit project, translating a test idea into TUnit syntax, or reviewing TUnit test code for API mistakes. DO NOT USE FOR deciding what to test, judging whether a test is any good, or running tests.

★ 1
Blind-Striker
Blind-Striker/agent-skills-and-plugins

dotnet-walls

Walls for a .NET change in a repository without its own quality gates — pinned, ephemeral analyzer, format, clone and scanner commands for the verify step, their identity ratchet, and the build traps. Use during a walls preflight, or when a worker's verify step needs .NET quality gates.

★ 1
Blind-Striker
Blind-Striker/agent-skills-and-plugins

csharp-15

Write, review and modernize C# 15 (.NET 11) code correctly. Covers every C# 15 feature - collection expression arguments (`[with(...), ...]`), union types (`union`, `[Union]`, `IUnion`, custom unions), closed class hierarchies (`closed`), extension indexers, labeled `break`/`continue`, non-virtual static interface members on runtimes without default interface methods, and the preview memory-safety model (`unsafe(expr)`, `safe`, requires-unsafe members). Use when code targets net11.0 or LangVersion 15/preview, when the user mentions any of these features, when replacing marker interfaces, OneOf-style results, `goto`/flag loop exits or capacity/comparer boilerplate, or when the build reports CS8121, CS8509, CS8655, CS8510, CS0121/CS0411 on a `with(...)` element, CS9370, CS9380 or CS9363.

★ 1
Blind-Striker
Blind-Striker/agent-skills-and-plugins

verified-waves

Run one task as verified multi-agent waves. Plans the fan-out, sends findings to fresh skeptics, keeps a ledger, and reports N of M done; small tasks stay solo.

★ 1
Blind-Striker
Blind-Striker/agent-skills-and-plugins

harness

The bundled dispatch harness, which runs a task file on worker models through OpenCode or Codex, fences their writes, escalates their asks to the operator, settles each result from git and its verify commands, and records every step in a ledger. Its commands, task file, modes and write fence, configuration, runs and their recovery, ledger and exit codes.

★ 1
Blind-Striker
Blind-Striker/agent-skills-and-plugins

delegating-to-worker-models

Use when you hand off implementation, a plan, research a decision will rely on, or a review, and you brief the worker and check what comes back. A subagent that searches, reads in parallel, or keeps your context small is ordinary tool use. Covers what stays with you, the mechanism, assigning work by strength, the walls preflight, prompts without contradictions, the worker contract and model profiles, evidence from your own run, review, and the fix-up loop.

★ 1