chaos-study

chaos-study — guided Chaos studies, end to end

Chaos engineering works best as an experiment. This skill helps structure one: it starts from a question the run could answer no to, runs one action against your resources only with your consent, and reports the signals it collected for you to review.

Does <steady state> hold when I execute <action> across <workspace scope>?

You do not need that sentence to start — the front door builds it with you. "Test resilience" is not a study: nothing could falsify it.

Principles

A study is a falsifiable claim. Every study needs a steady-state predicate measurable beforehand that could plausibly break during the run.

What to test is decided before it is scoped, and never by this skill alone. Picking an action because it is available is how studies end up proving nothing. The front door therefore always runs design first: it reads the system, then stops and asks you, one question at a time, until the purpose and the risk you will accept are stated in your own words and confirmed. There is no flag that skips the interview.

The platform decides what is possible, not this plugin. The available actions are read live from Chaos Studio for the workspace's region, every time. This suite ships no action catalogue, and there is no offline fallback: if the service cannot be asked, scoping stops with exit 16 rather than guessing. Likewise the resources a run will touch come from the workspace's discovered resources, not from an assertion in the plan.

Evidence beats vibes. A verdict is only as good as the signals behind it. If a signal was not collected, the report says not measured — never 0, never a plausible-looking guess.

Nothing runs without consent. -DryRun defaults to $true. Arming a study takes both -DryRun:$false and typing back the exact phrase the plan prints. There is no flag that skips this.

Blast radius is bounded before it is armed. Workspace, scope, filters, exclusions, action, parameters and duration are frozen into the plan and hashed. If the plan changed after you approved it, the run refuses.

The six skills

This is a suite, not a monolith. Use the front door — it is the only path that enforces the design gate. Calling a phase skill directly is the expert escape hatch and skips that enforcement deliberately.

Skill	Owns
chaos-study	chains the phases below; the default entry point
chaos-study-design	reading the system, interviewing you, recommending what to test
chaos-study-scope	workspace, scope, live action discovery, freezing the plan
chaos-study-run	configuration, validation, consent, execution, evidence
chaos-study-report	interpretation, findings, the HTML report
chaos-study-history	list, compare, and re-run past studies

Each phase persists its own output, so a chain that fails partway resumes there.

Usage

1. Name the system. That is all you start with. The front door always opens the design phase first: it reads the system, ranks candidate hypotheses against real dependency edges, then holds the study at exactly one question:

./scripts/Invoke-ChaosStudy.ps1 -System orders-api `
    -SubscriptionId $sub -ResourceGroup rg-prod -WorkspaceName ws-payments

Exit 26 means the study is open and waiting on a person — not that it failed. One question is printed, and repeated in a CHAOS-QUESTION JSON block. Put that question to the customer with ask_user (or your host's equivalent interactive mechanism), wait for the reply, then resume the same brief:

./scripts/Invoke-ChaosStudy.ps1 -System orders-api -BriefId <id> `
    -QuestionId <id from the block> -Answer '<what they actually said>'

One question per turn. Do not batch them, do not print a questionnaire, do not answer on the customer's behalf, and do not fall through to scoping. Exit 24 means the answer was too vague to test and the same question is still open. Passing -Scenario/-Action up front does not bypass any of this: nothing is scoped until the brief is CONFIRMED by the customer.

2. Ask the platform what it can do here. The answer depends on the region and what is in scope, and changes as Chaos Studio ships new actions:

./scripts/Invoke-ChaosStudy.ps1 -ListActions `
    -SubscriptionId $sub -ResourceGroup rg-prod -WorkspaceName ws-payments
# ...and -ListScenarios for the scenarios the workspace recommends

Each action is printed with its canonical id, type, what it applies to and its parameter schema — all as the service reported them.

3. Plan and preview. Once the brief is confirmed the same command carries on into scoping and stops there, executing nothing. Read the preview: what is in scope after filters and exclusions, the windows, and the configuration that would be created. Overrides such as -SteadyState apply on top of the brief. 4. Arm it. Only after the preview matches your intent:

./scripts/Invoke-ChaosStudy.ps1 ... -DryRun:$false -Consent '<phrase from the preview>'

Reading the verdict

Verdict	Means
Steady state held	The action landed and the objective survived it
Degraded but recovered	Breached during the run, recovered afterwards
Steady state breached	Breached and still breached after recovery
Not exercised	The action never applied enough work to test anything
Inconclusive	The action could not be proven to have landed

Severity follows recovery, not drama: a breach that never recovers is critical; the same breach that self-heals is medium.

Choosing signals

The suite collects only what you name, because what counts as evidence depends on the system under study. Two forms:

• metrics:<name> — an Azure Monitor metric on the single scoped resource, or @<resourceId> to pin one, |<aggregation> to override the aggregation. A scope holding several resources has no implied metric target, so pin it.
• logs:<workspaceId>#<kql> — a Log Analytics query

Name at least one signal expected to move under the action. That is what separates "resilient" from "the action never landed". Scoping checks readiness first: a study whose objective cannot be measured, or whose parameters do not satisfy the live schema, teaches nothing, so exit 10 is a real answer.

Progressive discovery

Action identifiers, parameters and required permissions are not documented here, because they are the service's to state and this plugin's to relay. Get them from -ListActions — Kubernetes/AKS guidance is deliberately absent until those actions appear there. What lives here is the method:

• references/design-method.md — how a study is chosen before it is scoped
• references/study-method.md — the six-phase method and why it is split
• references/report-contract.md — what the report may and may not claim

Exit codes

Code	Meaning
0	Success
1	Error
4	Reserved by the shipped chaos skills (broad fix unconsented) — this suite never emits it
10	Readiness gates failed
11	Consent declined or phrase mismatch
12	Plan changed after it was frozen
13	Study or brief already sealed
14	Scope unverified — nothing in scope, or the action does not fit it
15	Studies not comparable
16	Live action discovery unavailable — there is no fallback
17	Scenario configuration failed validation
18	Paused: an Azure operation must be executed by the host, then resumed
19	Paused: role assignments need their own, separate approval
20	Scenario would run fewer legs than it declares, and that was not accepted
21	The run would not exercise the failure often enough to mean anything
22	No adapter can reach Azure — say which one to use
23	Study was written by an older contract version
24	Design incomplete — an unread area, a vague answer, or a missing field
25	A past study cannot be re-run exactly, so no command was printed
26	Paused: waiting on the customer. One question is open — ask it

18, 19 and 21 are pauses, not failures. Each names the one thing it needs; supply it and re-run the same command. Do not fill in the missing part yourself — a study that cannot show where its evidence came from cannot be sealed.

Running where az is not usable

Every Azure call goes through one adapter seam, selected with -Adapter: local-az runs az in-process; external cannot, so it writes each operation into the study's operations/ directory and exits 18. Under external you are the executor: read the pending request, run exactly that call with your own authenticated tooling, write the response back as the matching result file, and re-run the identical command. Results are bound to the request hash and folded into the study's provenance; resuming is idempotent.

Notes

• Chaos Studio V2 only: workspaces are the lifecycle root and resource selection flows through workspace scopes. There is no V1 path or fallback.
• Core is az / Azure Resource Manager: no SRE Agent or MCP dependency.
• Sealed studies are immutable — re-testing scopes a new study.
• -Parameters (scenario), -ActionParameters (action), -AcknowledgeFaultDurationSeconds and -AbortCriteria (the customer's stop rule: @{ statement; source = 'customer'; signal; condition; resourceCorrelation }) all cross the front door.