chaos-study-scope

chaos-study-scope — decide what the study can test

Chaos studies often go wrong here, not during execution. They run an action nothing was measuring, over a scope that could not receive it, to answer a question that could not have come back false.

This skill's job is to catch that before any resource is touched.

Principles

The workspace is the lifecycle root. Scopes, discovered resources, scenarios, configurations and runs all hang off one workspace. Scoping resolves it first, and everything the plan records is anchored to it.

Ask the platform; never assume. The list of available actions is fetched from Microsoft.Chaos/locations/{region}/actions for the workspace's region, on every scope. That response is authoritative: its canonicalId, actionType, appliesTo and parametersSchema are what the plan records. Scenarios come from the workspace's own recommendation list.

No bundled catalogue, no fallback. This skill ships no action definitions. If the actions endpoint cannot be reached — not logged in, no permission, region unresolved — scoping stops with exit 16 and says so. It does not degrade to a stale local list, because a stale list is how you plan a study against an action that does not exist.

What is in scope is read, not asserted. The resources a run will touch come from the workspace's discovered resources, not from the operator's mental model. The two diverge constantly — a scope grows a resource nobody remembered, or a resource the plan names was never onboarded and would have been silently skipped.

A steady state must be falsifiable. successRate >= 99.5 can be violated. "the service is healthy" cannot. Scoping rejects the second kind.

Check the fit before promising the study. An empty scope, or an action that applies to no resource type present in the scope, means the study cannot prove anything. Scoping says so up front (exit 10 / 14) instead of failing halfway through a run.

Parameters are validated against the service's schema. Not against a copy of it. Required properties, enums and types come from the live parametersSchema, so validation cannot drift from the platform. Scenario knobs and action knobs are different documents: -Parameters is checked against the scenario's schema and is what the configuration carries, -ActionParameters against the action's. Neither is ever silently checked against the other.

Freeze what was approved. The plan is written once and hashed. The run skill re-computes that hash; if it changed, the run refuses (exit 12). What you reviewed is what executes.

Discovery is read-only. Scoping issues only reads, unless you explicitly ask for -CreateWorkspace. Nothing else here mutates anything.

Usage

List what the platform offers — start here; the action constrains the question, and only the service knows what exists in that region today:

./scripts/Invoke-ChaosStudyScope.ps1 -ListActions `
    -SubscriptionId $sub -ResourceGroup rg-prod -WorkspaceName ws-payments

./scripts/Invoke-ChaosStudyScope.ps1 -ListScenarios `
    -SubscriptionId $sub -ResourceGroup rg-prod -WorkspaceName ws-payments

Scope a study:

./scripts/Invoke-ChaosStudyScope.ps1 `
    -SubscriptionId $sub -ResourceGroup rg-prod -WorkspaceName ws-payments `
    -Scenario '<scenario from -ListScenarios>' `
    -Action '<action from -ListActions>' `
    -SteadyState 'successRate >= 99.5' `
    -SignalSource 'metrics:Availability' `
    -DurationMinutes 10 -BaselineMinutes 5 -RecoveryMinutes 10

-Action accepts the canonical URN, the action name or the display name; a partial match is accepted only when it is unambiguous.

Useful switches:

• -CreateWorkspace -Scope <armId...> — create the workspace over those scopes when it does not exist. Without it, a missing workspace is an error rather than an implicit provisioning action.
• -Location — region for a workspace being created
• -Parameters @{ ... } — scenario knobs, validated against the live scenario schema; these are what the configuration carries
• -ActionParameters @{ ... } — action knobs, validated against the action's own schema and recorded on the plan. The V2 configuration body takes scenario parameters only, so these are declared and frozen, never transmitted.
• -FilterLocation / -FilterZone / -ExcludeResource / -ExcludeType / -ExcludeTag — the blast radius, frozen onto the plan as the configuration's filters and exclusions. A declared filter is not verified discovery metadata. Discovery does not report a location or zone, so resources missing those attributes are retained as candidates with the attribute unverified, the uncertainty is printed, and the declared filter is still transmitted for the service to enforce. The preview cannot narrow what it cannot see, and it must not widen what you asked for either.
• -SignalSource — the evidence to collect (metrics: / logs:); no defaults are invented for you
• -Hypothesis — what you expect to happen, recorded for honesty at report time
• -SkipDiscovery — plan without contacting Azure. The plan then carries limitation L10 and records the scenario and action as unverified-offline; no metadata is fabricated, and the run still validates before executing.

Output is a frozen study-plan.v3.json plus a study id. Nothing is executed.

What the plan records

The plan is the contract the rest of the suite reads:

Section	Why it matters
workspace	the lifecycle root — subscription, group, name, id, region, scopes
scope	what is actually in scope: discovered resources, types, blast radius, and how many survive it
action	live action metadata: canonical id, type, what it applies to, schema, roles
scenario	the scenario that will execute, its version and parameters
discovery	which endpoint answered, when, and how much it returned
question	the steady-state predicate, parsed and normalised
windows	baseline / inject / recovery minutes
signals	what evidence must be collected for the verdict to mean anything
readiness	gate results and any limitation codes they imply
planHash	integrity seal checked before execution

Choosing windows

Baseline exists to establish what "normal" looked like today, not last week. Recovery exists to distinguish a transient dip from real damage — without it, a system that never recovers looks identical to one that recovers instantly.

Injection windows of three minutes or less are recorded as limitation L7: too short to distinguish resilience from luck.

Progressive discovery

Action identifiers, parameter shapes and required permissions are not documented here. They are the service's to state, and restating them locally is exactly the drift this design removes. Use -ListActions and -ListScenarios.

What lives locally is the method:

• ../chaos-study/references/study-method.md — the method behind the phases
• ../chaos-study/references/report-contract.md — what a report may claim

Exit codes

Code	Meaning
0	Plan frozen
1	Error
10	Readiness gates failed
14	Scope unverified — nothing in scope, or the action does not fit it
16	Live action discovery unavailable — there is no fallback
18	Paused: an Azure operation must be executed by the host, then resumed
19	Paused: role assignments need their own, separate approval
20	Scenario would run fewer legs than it declares, and that was not accepted
21	The run would not exercise the failure often enough to mean anything
22	No adapter can reach Azure — pass -Adapter explicitly
23	Study was written by an older contract version

The pauses

18, 19, 20 and 21 are stops with a specific missing input, not errors. Supply it and re-run the same command; scoping resumes rather than restarting.

• 18 — under -Adapter external this phase cannot call Azure itself. It wrote the exact request to the study's operations/ directory. Execute it with your own authenticated tooling, write the response back as the matching result file, then re-run. Never answer the request from memory or inference: the result is hashed and bound to the request, and invented evidence cannot seal.
• 19 — preflight validation found the workspace identity is missing role assignments. Granting them is a separate decision from approving the fault, so it needs its own approval phrase. Approving the injection never implies it.
• 20 — the service's execution plan skips legs the scenario name implies. Fix the scope, or accept the reduced scenario explicitly.
• 21 — the exposure arithmetic says the fault would rarely meet the code path. Strengthen the exercise, or accept a weak one explicitly and let the study report "Not exercised" rather than false resilience.

Next

Review the plan, then run it with explicit consent:

chaos-study-run -StudyId <studyId> -DryRun:$false -Consent '<phrase from the plan>'