ln-41-test-strategy-planner

Test Strategy Planner

Goal: Design a read-only test strategy that maximizes confidence in important local behavior. Select test levels by the boundary that must be crossed and the defect that must be detected, not by a fixed pyramid, coverage target, or scenario count.

Execution contract: Treat the ordered checkbox workflow below as this skill's Definition of Done. Work through every item in order, and mark it complete only when its action and required evidence are complete. N/A, skipped, unavailable, or delegated items remain incomplete. Before returning, apply this skill's verdict, decision, and approval rules to every incomplete item and prepend Checklist: X/Y complete<br>Incomplete: None | section/item — reason; outcome impact; exact next action; list every incomplete item.

Tool Routing

Need	Preferred tool	Use it when	Fallback
Requirements and repository rules	Native file reads plus Git	Establishing scope, current work, acceptance criteria, and supported commands	User-provided requirements with explicit limitations
Existing test surface	File listing, search, manifests, runner configuration, and CI	Mapping test levels, fixtures, environments, and conventions	Repository tree and known test entrypoints
Behavior and boundaries	Language server or host-native code intelligence	Tracing entrypoints, consumers, trust boundaries, persistence, queues, and external contracts	Narrow search followed by direct inspection
Existing evidence	Safe repository-defined test and coverage commands	Determining what behavior is already proved and where confidence is weak	Inspect tests and CI; mark execution unavailable
Current external failure modes	Official documentation, specifications, advisories, and primary field evidence	An external contract or real user failure can change scenarios or priority	Mark the claim UNVERIFIED; do not invent risk

Keep the run read-only. Do not create tests, fixtures, snapshots, tasks, or documentation, and do not update the reviewed implementation.

Evidence Rules

• Test level follows the observable boundary: use the lowest level that can expose the intended defect without bypassing the behavior under test.
• Coverage is discovery evidence, not proof. Require an oracle that would fail for the named defect.
• Prioritize by impact, plausible failure, uniqueness, detectability, and recovery cost; do not convert those judgments into universal numeric thresholds.
• Existing tests reduce a gap only when their setup and assertions prove the same behavior and failure mode.
• Framework, language, ORM, serializer, or library behavior is not a product test unless local configuration or integration changes its contract.
• External research is actionable only when it adds a concrete failure mode, boundary, or oracle to this plan.

Checklist

1. Establish Scope and Evidence

• [ ] Resolve the feature, requirements, acceptance criteria, actors, explicit non-goals, and protected human or system outcomes; separate the requested mechanism from the result it must enable and return BLOCKED if there is no concrete behavior to plan for.
• [ ] Read applicable repository instructions and inspect Git state so current work and unrelated changes are not mistaken for established behavior.
• [ ] Detect languages, frameworks, runners, test directories, fixtures, factories, environments, CI gates, coverage, contract tests, and manual test surfaces.
• [ ] Map existing evidence to each requirement and mark it PROVED, PARTIAL, MISSING, or UNAVAILABLE based on the actual oracle.
• [ ] Inspect manual, exploratory, incident, and production evidence when it reveals behavior that automated suites do not cover.
• [ ] Identify environment, data, credentials, services, devices, browsers, and destructive-state constraints before proposing scenarios.
• [ ] Record assumptions and unknowns that can change test level, priority, or feasibility, and ask one concise question only when different interpretations materially change the strategy.

2. Build the Risk Map

• [ ] Trace critical flows from actor trigger through entrypoint, runtime wiring, state change, and durable or user-visible outcome.
• [ ] Identify uniquely important local behavior involving money, authentication, authorization, ownership, data integrity, destructive actions, migrations, public contracts, or irreversible workflows.
• [ ] Enumerate plausible defect classes: incorrect success, rejected valid input, accepted invalid input, boundary error, partial failure, duplicate delivery, ordering, timeout, retry, cancellation, race, rollback, recovery, and compatibility drift; state what protected outcome is lost or what concrete harm follows.
• [ ] Separate product risks from implementation details and behavior already guaranteed by a dependency; exclude technically representable states that protect no unique local outcome or decision.
• [ ] Identify privacy-sensitive or regulated test data and require synthetic, minimized, or explicitly approved fixtures.
• [ ] Use current external evidence only when version-sensitive contracts, recurring user failures, abuse patterns, or interoperability risks can change the map.
• [ ] Rank risks qualitatively and explain ties or uncertainty; do not manufacture precision from missing frequency or impact data.

3. Select Test Levels and Oracles

• [ ] Choose unit tests for isolated local rules, contract tests for producer-consumer agreement, integration tests for owned boundaries, and end-to-end tests for production-shaped journeys whose terminal outcome cannot be proved lower.
• [ ] Avoid duplicating the same behavior at every level unless each level detects a distinct failure class.
• [ ] Define one independent oracle per scenario: returned contract, durable state, emitted event, rendered behavior, external effect, invariant, or deterministic artifact.
• [ ] Check that mocks and fakes do not bypass the boundary or failure semantics the scenario claims to prove.
• [ ] Include positive, invalid, boundary, authorization, error, recovery, concurrency, and compatibility cases only where the risk map makes them material.
• [ ] Specify non-default configuration, time, locale, randomness, ordering, or data scale when defaults could conceal hard-coded behavior.
• [ ] Add browser, device, operating-system, runtime, or version cells only when the supported contract or a known risk makes them decision-relevant.
• [ ] Prefer deterministic setup and bounded data; identify where real dependencies, emulators, disposable environments, or production-like topology are necessary.

4. Produce a Prioritized Test Matrix

• [ ] For every scenario, name the protected outcome, requirement, risk, defect class, test level, setup, action, oracle, expected evidence, and required environment.
• [ ] Mark existing scenarios that should be retained or strengthened and new scenarios that should be added; do not redesign the entire suite without a demonstrated need.
• [ ] Order scenarios so safety-critical and high-information checks run before expensive breadth, while preserving prerequisite and state dependencies.
• [ ] Identify which scenarios can run in parallel and which share mutable state, rate limits, accounts, devices, or environment setup.
• [ ] Separate release-gating scenarios from slower diagnostic or exploratory coverage so the plan does not make routine delivery impractical.
• [ ] State exclusions explicitly, including scenarios with no unique protected outcome or defect signal, low-value duplication, framework behavior, infeasible environments, and accepted residual risks.
• [ ] Use READY when the strategy is executable and decision-complete, INCONCLUSIVE when useful partial planning is possible but material evidence is missing, and BLOCKED when requirements or a safety-critical boundary cannot be established.
• [ ] Return the verdict, risk map, existing evidence, prioritized matrix, environment needs, exclusions, limitations, and residual risks.
• [ ] State the smallest next evidence-gathering action for every INCONCLUSIVE or BLOCKED area.

Output Contract

# Test Strategy

**Verdict:** READY | INCONCLUSIVE | BLOCKED

## Scope and existing evidence
- Requirements, actors, and outcomes
- Existing suites, commands, and environments
- Assumptions and unavailable evidence

## Risk map
| Protected outcome | Behavior | Failure or defect class | Impact | Existing proof | Priority rationale |
|---|---|---|---|---|---|
| ... | ... | ... | ... | PROVED / PARTIAL / MISSING / UNAVAILABLE | ... |

## Prioritized scenarios
| Priority | Protected outcome | Requirement | Level | Scenario | Oracle and expected evidence | Environment |
|---:|---|---|---|---|---|---|
| ... | ... | ... | unit / contract / integration / E2E | ... | ... | ... |

## Next evidence-gathering actions
Exact repository, environment, contract, or user decision needed for each `INCONCLUSIVE` or `BLOCKED` area.

## Exclusions and residual risks
Low-value duplication, unavailable environments, accepted gaps, and evidence still required.