[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7AC85QF7YKFQWQHYYR7N8YY":3,"skill-same:01M7AC85QF7YKFQWQHYYR7N8YY":151},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":44,"commitSha":45,"skillDirectory":46,"forksCount":47,"repositoryHtmlUrl":48,"repositoryHomepageUrl":6,"ownerHtmlUrl":49,"ownerBlogUrl":6,"ownerDescription":50,"updatedAt":51,"install":52,"resources":65},"01M7AC85QF7YKFQWQHYYR7N8YY","secrets-manager-rotation-troubleshooter",null,"Diagnoses AWS Secrets Manager rotation failures through a ten-category diagnostic tree: rotation Lambda errors at the database (wrong host, port, database name, credential creation), rotation schedule not triggering (EventBridge rule deleted, disabled, or wrong schedule expression), cross-account secret access denied (rotation role lacks kms:Decrypt or secretsmanager:GetSecretValue on the secret), Master Secret ARN misconfigured in the rotation template, rotation Lambda timeout (default 3s too low, or 15s insufficient for slow DB), VPC connectivity (rotation Lambda not attached to DB subnets, missing NAT\u002Fendpoint for Secrets Manager API), rotation strategy conflict (Alternating Users on a engine without CREATE USER, Single User on a twin-secret setup), twin secrets not synced across regions\u002Faccounts, rotation token missing (a caller invoked the Lambda directly without a ClientRequestToken), and superuser permissions insufficient for the rotation Lambda's DB user. Walks symptoms to a verified root cause wi...","jhkchan\u002Faws-agent-skills","jhkchan","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F1083902?v=4","01M7A2SVQYP9GQTPTDXP5JKYQ4","Jacky Chan","01M7A2SVS117YWCTZABB75HZR1",4,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: secrets-manager-rotation-troubleshooter\ndescription: 'Diagnoses AWS Secrets Manager rotation failures through a ten-category diagnostic tree: rotation Lambda errors at the database (wrong host, port, database name, credential creation), rotation schedule not triggering (EventBridge rule deleted, disabled, or wrong schedule expression), cross-account secret access denied (rotation role lacks kms:Decrypt or secretsmanager:GetSecretValue on the secret), Master Secret ARN misconfigured in the rotation template, rotation Lambda timeout (default 3s too low, or 15s insufficient for slow DB), VPC connectivity (rotation Lambda not attached to DB subnets, missing NAT\u002Fendpoint for Secrets Manager API), rotation strategy conflict (Alternating Users on a engine without CREATE USER, Single User on a twin-secret setup), twin secrets not synced across regions\u002Faccounts, rotation token missing (a caller invoked the Lambda directly without a ClientRequestToken), and superuser permissions insufficient for the rotation Lambda''s DB user. Walks symptoms to a verified root cause wi...'\nlicense: Apache-2.0\ncompatibility: Agent runtime that reads SKILL.md (Claude Code, Cursor, Windsurf, Codex, Gemini). Offline symptom classification works from pasted error messages and rotation configuration. Live-account diagnosis uses aws secretsmanager describe-secret, aws secretsmanager\nmetadata:\n  domain: aws-cloudops\n  complexity: high\n  requires_llm: 'true'\n  phase: '2'\n  supports_pipeline: 'true'\n  entry_point: 'false'\n  family: Security\n  task_type: troubleshoot\n  skill_class: capability\n  lifecycle_status: active\n  verdict_shape: ROOT_CAUSE_IDENTIFIED | INSUFFICIENT_DATA | ESCALATE\n  when_to_use: Diagnosing an AWS Secrets Manager secret rotation failure (rotation Lambda error, schedule not triggering, cross-account access denied, Master Secret misconfiguration, rotation Lambda timeout, VPC connectivity to the database, Alternating vs Single User strategy conflict, twin secrets not synced, rotation token missing, superuser permissions insufficient), walking a symptom to the failed layer with verify and fix commands, validating why a secret's LastRotatedDate is stale, or triaging a \"secret rotation is broken\" page where the root cause may be rotation Lambda, EventBridge schedule, IAM, network, KMS, or database permissions — not necessarily the rotation template itself.\n  when_not_to_use: Initial rotation configuration (use the Secrets Manager console or CloudFormation rotation template setup), secret value retrieval at application runtime (use the application's SDK path), audit of all secrets lacking rotation (use secrets-manager-rotation-coverage-auditor), encryption-key rotation for the underlying CMK (use kms-key-rotation-operator), or password-policy enforcement (use iam-password-policy-auditor). This skill diagnoses rotation-time failures; it does not design rotation schedules or audit steady-state rotation posture.\n  activation_triggers: ''\n  invocation_schema: '''Input: either (a) a symptom description (error message from the rotation Lambda CloudWatch logs, observed behaviour such as \"LastRotatedDate is 30 days ago\", \"rotation succeeds but applications cannot connect\"), optionally paired with the secret''s describe-secret output and recent rotation Lambda logs, OR (b) a SecretId plus caller context for live-account diagnosis. Output: a deterministic TARGET\u002FVERDICT\u002FREASON\u002FLAYER\u002FEVIDENCE\u002FREMEDIATION block where VERDICT ∈ {ROOT_CAUSE_IDENTIFIED, INSUFFICIENT_DATA, ESCALATE} and LAYER ∈ {ROTATION_LAMBDA_TIMEOUT, ROTATION_LAMBDA_VPC, ROTATION_LAMBDA_DB_CREDENTIAL, ROTATION_LAMBDA_DB_ENDPOINT, SCHEDULE_MISSING, SCHEDULE_DISABLED, MASTER_SECRET_MISCONFIGURED, PERMISSION_ROTATION_ROLE, PERMISSION_CROSS_ACCOUNT, KMS_DECRYPT_ROLE, ROTATION_TOKEN_MISSING, SUPERUSER_INSUFFICIENT, STRATEGY_CONFLICT, TWIN_SECRETS_NOT_SYNCED, PREVIOUS_CREDENTIAL_NOT_STORED, REDSHIFT_ROTATION_FUNCTION, UNKNOWN}.'''\n  invocation_example: '\"# Minimal valid input (offline symptom classification):\\nSymptom: \\\"Secret prod\u002Fdb\u002Fpayments-primary has not rotated in 7 days;\\nLastRotatedDate is 2026-07-30. The rotation Lambda''s last execution\\nlogged ''Task timed out after 3.00 seconds''.\\\"\\nSecretId: prod\u002Fdb\u002Fpayments-primary\\nRotationEnabled: true\\nRotationLambdaARN: arn:aws:lambda:us-east-1:111111111111:function:SecretsManagerRotation-prod-db-payments\\nRotationRules: {ScheduleExpression: ''rate(1d)''}\\nLastRotatedDate: 2026-07-30T03:17:22Z\\nOwningService: (none — customer-managed)\\nKmsKeyId: alias\u002Faws\u002Fsecretsmanager\\nRotationLambda LastLog: ''Task timed out after 3.00 seconds''\"'\n  version: 0.1.0\n  author: Jacky Chan — AWS Community Builder\n  keywords: Secrets Manager, rotation, rotation Lambda, Master Secret, Alternating Users, Single User, rotation strategy, EventBridge schedule, rotation token, ClientRequestToken, cross-account secret, twin secrets, RotateSecret, rotation template, Superuser, database credential, rotation disabled, rate(1d), VPC connectivity, KMS decrypt\n---\n\n# Secrets Manager Rotation Troubleshooter\n\n## Quick start\n\n- **Symptom → layer map (first plausible match drives the first probe):**\n  rotation Lambda `Task timed out` and the timeout is 3s (default)\n  → `ROTATION_LAMBDA_TIMEOUT`; rotation Lambda `Task timed out` at 15s\n  with a DB in another subnet → `ROTATION_LAMBDA_VPC`; rotation Lambda\n  logged `AccessDenied` calling `secretsmanager:GetSecretValue` on the\n  Master Secret → `PERMISSION_ROTATION_ROLE`; `LastRotatedDate` is days\n  stale and the EventBridge rule is missing → `SCHEDULE_MISSING`;\n  rotation Lambda logged `Could not connect to database at host ...` or\n  `wrong port` → `ROTATION_LAMBDA_DB_ENDPOINT`; rotation Lambda logged\n  `Master Secret ARN ... does not exist` → `MASTER_SECRET_MISCONFIGURED`;\n  rotation Lambda logged `Rotating back to previous credential` on every\n  attempt → `PREVIOUS_CREDENTIAL_NOT_STORED`; the Lambda was invoked\n  manually without a `ClientRequestToken` → `ROTATION_TOKEN_MISSING`.\n- **Always verify with a probe, never guess.** Each layer has a single\n  command that proves or disproves it. A `ROOT_CAUSE_IDENTIFIED` verdict\n  requires positive evidence — a failing probe that matches the symptom\n  — not a process of elimination.\n- **Rotation is a four-step protocol: `createSecret`, `setSecret`,\n  `testSecret`, `finishSecret`.** Most rotation failures surface in\n  `setSecret` (the step that issues the SQL `ALTER USER` \u002F `CREATE USER`\n  against the target database). Read the Lambda logs to identify the\n  step that failed before guessing the layer.\n- **Rotation strategy is set at template-deploy time, not at the\n  secret.** `Alternating Users` creates a clone of the current user and\n  rotates the clone's password (safer; requires `CREATE USER` +\n  `GRANT`); `Single User` updates the existing user's password in place\n  (atomic; requires no extra privileges but risks brief connection\n  failures during the rotation step). A rotation Lambda written for one\n  strategy cannot rotate a secret configured for the other without a\n  code change.\n- **ESCALATE for AWS-side incidents.** A regional Secrets Manager or\n  Lambda outage, or an AWS Health event affecting the rotation service,\n  is not customer-fixable — escalate to AWS Support and surface the\n  event ARN.\n\n## Mindset\nFull diagnostic mindset (rotation incidents are Lambda config, schedule, or DB-permission incidents, not template bugs): [references\u002Fadvanced-patterns.md](references\u002Fadvanced-patterns.md).\nLoad on demand for the senior-engineer framing.\n\n## Philosophy\nThe four senior-engineer behaviours (rotation step name drives diagnostic order, LastRotatedDate is the canonical health signal, execution role vs DB credential, rate(1d) is a schedule not a guarantee): [references\u002Fadvanced-patterns.md](references\u002Fadvanced-patterns.md).\nLoad on demand when routing by rotation step name.\n\n## Quick reference — symptom triage table\n\n| Symptom phrase \u002F log line | Most likely layer | First probe |\n|---|---|---|\n| `Task timed out after 3.00 seconds` (rotation Lambda default) | `ROTATION_LAMBDA_TIMEOUT` | `lambda get-function-configuration` (Timeout), CloudWatch Duration vs Timeout |\n| `Task timed out after 15.00 seconds` + DB in another subnet | `ROTATION_LAMBDA_VPC` | `lambda get-function-configuration` (VpcConfig), `ec2 describe-route-tables` for the subnet |\n| `Could not connect to database at host ... port ...` | `ROTATION_LAMBDA_DB_ENDPOINT` | Rotation Lambda env vars \u002F Master Secret value (`host`, `port`, `dbname`), `rds describe-db-instances` |\n| `AccessDenied` calling `secretsmanager:GetSecretValue` | `PERMISSION_ROTATION_ROLE` | `iam simulate-principal-policy` on the rotation role for `secretsmanager:GetSecretValue` on the secret ARN |\n| `AccessDenied` calling `kms:Decrypt` | `KMS_DECRYPT_ROLE` | `kms describe-key`, simulate `kms:Decrypt` on the rotation role |\n| `Master Secret ARN ... does not exist` or `ResourceNotFoundException` | `MASTER_SECRET_MISCONFIGURED` | Rotation Lambda env var `SECRETS_MANAGER_MASTER_ID` vs actual Master Secret ARN |\n| `LastRotatedDate` is days\u002Fweeks stale; EventBridge rule missing | `SCHEDULE_MISSING` | `events describe-rule` for the rotation rule; `events list-targets-by-rule` |\n| `LastRotatedDate` is stale; EventBridge rule `State: DISABLED` | `SCHEDULE_DISABLED` | `events describe-rule --name \u003Crule> --query State` |\n| `Rotating back to previous credential` on every attempt | `PREVIOUS_CREDENTIAL_NOT_STORED` | Lambda logs in the `setSecret` step; verify `AWSPREVIOUS` staging label exists |\n| `Rotation request missing ClientRequestToken` | `ROTATION_TOKEN_MISSING` | CloudTrail `RotateSecret` event; check for direct Lambda invocation |\n| `permission denied for table mysql.user` (or pg_roles) | `SUPERUSER_INSUFFICIENT` | Master Secret DB user's `SUPERUSER` \u002F `rds_superuser` attribute |\n| `CREATE USER failed ... already exists` (Alternating) | `STRATEGY_CONFLICT` | Rotation Lambda code: `ROTATION_STRATEGY` env var or template family |\n| Redshift secret rotated but `staging_*` user creation failed | `REDSHIFT_ROTATION_FUNCTION` | Rotation Lambda uses the Redshift-specific template, not the generic MySQL one |\n| Cross-account: rotation Lambda in account A cannot read secret in account B | `PERMISSION_CROSS_ACCOUNT` | Secret resource policy grants `secretsmanager:GetSecretValue` to the rotation role ARN in account A |\n| Region-paired twin: `prod\u002Fdb\u002Fpayments` rotated but `eu-west-1\u002Fdb\u002Fpayments` did not | `TWIN_SECRETS_NOT_SYNCED` | Compare `LastRotatedDate` across regions; verify the second region's rotation config |\n| None of the above, region-wide Secrets Manager outage | `ESCALATE` | `aws health describe-events` for `AWS_SECRETS_MANAGER` |\n\n## Pre-flight: secret state and gather-info gate\n\nBefore running symptom-specific probes, gather the canonical secret\nconfiguration and short-circuit on secret states that mimic rotation\nfailures.\n\n### Account-wide pre-flight commands\nAccount-wide pre-flight commands 1-6 (describe-secret, rotation Lambda log filter, get-resource-policy, get-function-configuration, EventBridge rules, AWS Health): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nRun these before any symptom-specific probe.\n\n### Secret-state short-circuit\nThe secret-state short-circuit table (healthy vs stale vs RotationEnabled:false vs DeletedDate vs missing AWSCURRENT vs AWSPENDING stuck vs OwningService): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nConsult it right after describe-secret to short-circuit mimics.\n\nIf the input is malformed (missing SecretId, absent symptom\ndescription, no caller context for live diagnosis), emit:\n\nThe INSUFFICIENT_DATA re-prompt template for malformed\u002Fmissing input: [references\u002Fworked-examples.md](references\u002Fworked-examples.md).\nEmit it whenever required context is missing.\n\n## Process — Diagnostic decision tree (apply in symptom order)\n\nThe diagnostic tree is symptom-driven. Pick the entry point based on the\nobserved symptom, then walk the layer-specific probes in order. **Never\nemit `ROOT_CAUSE_IDENTIFIED` without a failing probe that matches the\nsymptom.**\n\n### Step 0: Rotation protocol and non-obvious behaviours\nThe ten non-obvious behaviours (four-step protocol, 3s default timeout, schedule jitter, Master Secret vs rotating secret, cross-account dual policy, CMK decrypt, Lambda-VPC attachment, AWSPENDING recovery, Alternating engine support, AWSPREVIOUS on first rotation): [references\u002Fadvanced-patterns.md](references\u002Fadvanced-patterns.md).\nLoad on demand when the failing step is unclear.\n\n### Step 1: Symptom entry — pick the diagnostic branch\n\n| Symptom | Branch |\n|---|---|\n| Rotation Lambda `Task timed out after 3.00 seconds` | Step 2 — Lambda timeout |\n| Rotation Lambda `Task timed out` at > 3s; DB in another subnet\u002Fregion | Step 3 — VPC connectivity |\n| Rotation Lambda `Could not connect to database at host ...` \u002F `wrong port` \u002F `unknown database` | Step 4 — DB endpoint |\n| Rotation Lambda `AccessDenied` calling `secretsmanager:GetSecretValue` \u002F `kms:Decrypt` | Step 5 — IAM permissions |\n| Rotation Lambda `Master Secret ARN ... does not exist` | Step 6 — Master Secret |\n| `LastRotatedDate` stale; rotation never invoked | Step 7 — EventBridge schedule |\n| Rotation Lambda logged `permission denied for table mysql.user` \u002F `must be superuser` | Step 8 — DB superuser |\n| Rotation Lambda logged `CREATE USER failed ... already exists` \u002F `Rotating back` | Step 9 — Strategy conflict |\n| Cross-account rotation Lambda cannot read secret in another account | Step 10 — Cross-account |\n| Rotation invoked but `LastRotatedDate` did not advance; `AWSPENDING` stuck | Step 11 — Recovery failure |\n| Region-paired twin did not rotate when primary did | Step 12 — Twin secrets |\n| None of the above | Step 13 — Escalate \u002F INSUFFICIENT_DATA |\n\n### Step 2: Rotation Lambda timeout (default 3s)\n\nSymptom: rotation Lambda CloudWatch logs show `Task timed out after\n3.00 seconds`. `LastRotatedDate` is stale. The default Lambda timeout\nis 3s; the AWS-managed rotation templates override this to 30s at\ndeploy time, but a manually created rotation Lambda may inherit the\ndefault.\n\nProbes (lambda get-function-configuration Timeout\u002FMemory\u002FRuntime, CloudWatch Duration statistics vs configured Timeout): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nLoad on demand to run the timeout probes.\n\nIf `Maximum` Duration is at or just above the configured `Timeout`, the\nfunction is being killed before completing the four-step protocol. If\nthe configured Timeout is 3, **ROOT_CAUSE_IDENTIFIED** with\n`LAYER: ROTATION_LAMBDA_TIMEOUT`.\n\nFix command (lambda update-function-configuration --timeout 30) plus the do-not-just-raise-the-timeout note: [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nLoad on demand when applying the ROTATION_LAMBDA_TIMEOUT fix.\n\n### Step 3: VPC connectivity — rotation Lambda cannot reach database\n\nSymptom: rotation Lambda logs `Could not connect to database at host\n\u003Cendpoint>` or `Connection timed out` against the RDS\u002FAurora\u002FRedshift\nendpoint. The Lambda's Timeout is ≥ 15s but the connection never\nestablishes.\n\nProbes (lambda get-function-configuration VpcConfig, subnet route tables, Lambda\u002FDB security groups): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nLoad on demand to run the VPC probes.\n\n`VpcConfig` empty or null → the Lambda is NOT in a VPC and cannot\nreach a private database. **ROOT_CAUSE_IDENTIFIED** with\n`LAYER: ROTATION_LAMBDA_VPC`. Fix: attach the Lambda to the database's\nsubnets (`update-function-configuration --vpc-config ...`).\n\nIf `VpcConfig` is populated, verify the subnet route table:\n\nRoute-table reading guide (local route, peered VPC, PrivateLink ENI) and security-group checks per DB port: [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nLoad on demand when VpcConfig is populated but the DB is unreachable.\n\n**Verdicts:**\n- Lambda not VPC-attached, DB is private: ROOT_CAUSE_IDENTIFIED,\n  `LAYER: ROTATION_LAMBDA_VPC`.\n- Lambda in wrong subnet, no route to DB CIDR: ROOT_CAUSE_IDENTIFIED,\n  `LAYER: ROTATION_LAMBDA_VPC`.\n- SG egress or ingress missing the DB port: ROOT_CAUSE_IDENTIFIED,\n  `LAYER: ROTATION_LAMBDA_VPC`.\n\n### Step 4: Database endpoint — wrong host, port, or database name\n\nSymptom: rotation Lambda logs `Could not connect to database at host\n\u003Chost> port \u003Cport>` or `unknown database \"\u003Cdbname>\"`. The connection\nestablishes (no VPC issue) but authentication or routing fails.\n\nConnection-string probes (Lambda env vars, secret value host\u002Fport\u002Fdbname, rds describe-db-instances, redshift describe-clusters): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nLoad on demand to cross-reference the database endpoint.\n\nCommon mismatches:\n\n| Symptom | Cause |\n|---|---|\n| Secret's `host` is the writer endpoint but the rotation tries to connect to a reader | Aurora reader endpoint resolves to a read-only instance; `ALTER USER` fails on read-only. Use the cluster writer endpoint. |\n| Secret's `port` is 3306 but the engine is PostgreSQL (5432) | Cross-engine template confusion; the secret was created for one engine but the rotation Lambda uses another engine's template. |\n| Secret's `dbname` does not exist on the DB instance | Database was renamed or never created; the connection succeeds but `USE \u003Cdbname>` fails. |\n| Secret's `host` is an IP that no longer resolves | RDS failover moved the endpoint; the secret's value was never updated. Re-rotate or fix the secret value. |\n\n**Verdict:** ROOT_CAUSE_IDENTIFIED, `LAYER: ROTATION_LAMBDA_DB_ENDPOINT`.\nFix: update the secret value (or Lambda env var) to the correct host,\nport, or dbname, then trigger a manual rotation.\n\n### Step 5: IAM — rotation role lacks permissions\n\nSymptom: rotation Lambda logs `AccessDenied` calling\n`secretsmanager:GetSecretValue`, `secretsmanager:PutSecretValue`, or\n`kms:Decrypt`. The Lambda runs but fails at the first Secrets Manager\nAPI call.\n\nIAM probes (rotation role ARN, simulate-principal-policy for GetSecretValue\u002FPutSecretValue\u002FDescribeSecret and kms:Decrypt, CloudTrail lookup for the denied API) plus implicitDeny vs explicitDeny reading: [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nLoad on demand on any AccessDenied symptom.\n\n**Verdicts:**\n- Rotation role lacks `secretsmanager:GetSecretValue` on the secret or\n  Master Secret: ROOT_CAUSE_IDENTIFIED,\n  `LAYER: PERMISSION_ROTATION_ROLE`.\n- Rotation role lacks `kms:Decrypt` on the customer-managed CMK:\n  ROOT_CAUSE_IDENTIFIED, `LAYER: KMS_DECRYPT_ROLE`.\n- Same-account resource policy does not matter (identity policy\n  suffices); cross-account requires both — see Step 10.\n\n### Step 6: Master Secret ARN misconfiguration\n\nSymptom: rotation Lambda logs `ResourceNotFoundException: Master\nSecret ARN ... does not exist` or `AccessDenied` reading a Master\nSecret that the secret actually points to.\n\nMaster Secret probes (Lambda env vars SECRETS_MANAGER_MASTER_ID \u002F MASTER_ARN, describe-secret on the Master ARN): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nLoad on demand when the Master Secret is suspect.\n\nCommon patterns:\n\n| Symptom | Cause |\n|---|---|\n| Env var value is a secret NAME, not an ARN; cross-account rotation fails | Same-account tolerates the name; cross-account requires the full ARN with the account ID. Use the ARN. |\n| Env var points to a Master Secret in a different region | Rotation Lambda cannot read cross-region without a replication setup; the Master Secret must be in the same region. |\n| Env var was updated but the Lambda was not published as a new version | The Lambda alias still points to the old version with the old ARN. Publish and update the alias. |\n| Master Secret was deleted but the rotation Lambda still references it | Restore the Master Secret or reconfigure rotation to use the rotating secret itself as Master (single-user strategy). |\n\n**Verdict:** ROOT_CAUSE_IDENTIFIED,\n`LAYER: MASTER_SECRET_MISCONFIGURED`. Fix: update the Lambda env var\nto the correct Master Secret ARN, publish a new version, and shift the\nalias.\n\n### Step 7: EventBridge schedule — rotation never triggers\n\nSymptom: `LastRotatedDate` is days or weeks stale. The rotation Lambda\nhas no recent CloudWatch log streams. The secret's\n`RotationRules.ScheduleExpression` is correct (e.g., `rate(1d)`), but\nnothing fires.\n\nSchedule probes (list-targets-by-rule, list-rules filtered for the rotation Lambda, per-rule target verification): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nLoad on demand when LastRotatedDate is stale.\n\nFor each candidate rule:\n\n| Field | Effect |\n|---|---|\n| Rule not found | The rule was deleted. **ROOT_CAUSE_IDENTIFIED**, `LAYER: SCHEDULE_MISSING`. Recreate via `events put-rule` + `put-targets`. |\n| `State: DISABLED` | The rule exists but is disabled. **ROOT_CAUSE_IDENTIFIED**, `LAYER: SCHEDULE_DISABLED`. Re-enable via `events enable-rule`. |\n| `ScheduleExpression` does not match `RotationRules.ScheduleExpression` | Schedule drift; the rule fires at the wrong cadence. Update via `events put-rule`. |\n| Targets empty or target ARN ≠ rotation Lambda ARN | The rule fires but invokes the wrong Lambda (or no Lambda). Update via `events put-targets`. |\n| Target Lambda permission missing (`lambda:InvokeFunction` for the EventBridge principal) | The rule fires but Lambda rejects the invocation. Verify via `lambda get-policy` for the resource-based statement allowing `events.amazonaws.com`. |\n\n**Verdicts:**\n- Rule deleted: ROOT_CAUSE_IDENTIFIED, `LAYER: SCHEDULE_MISSING`.\n- Rule disabled: ROOT_CAUSE_IDENTIFIED, `LAYER: SCHEDULE_DISABLED`.\n- Rule target wrong \u002F missing: ROOT_CAUSE_IDENTIFIED,\n  `LAYER: SCHEDULE_MISSING`.\n\n### Step 8: Database superuser privileges insufficient\nStep 8 deep dive (engine-by-engine privilege requirements, Master Secret username read, SUPERUSER_INSUFFICIENT verdict and fix): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md) and [references\u002Ferror-handling.md](references\u002Ferror-handling.md).\nBranch when the Lambda logs `permission denied for table mysql.user` or `must be superuser`.\n\n### Step 9: Rotation strategy conflict (Alternating vs Single User)\nStep 9 deep dive (Single vs Alternating template families, ROTATION_STRATEGY env probes, STRATEGY_CONFLICT patterns and verdict): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nBranch when the Lambda logs `CREATE USER failed ... already exists` or `Rotating back`.\n\n### Step 10: Cross-account secret access denied\nStep 10 deep dive (cross-account simulate + get-resource-policy probes, required resource-policy statement, KMS key policy, PERMISSION_CROSS_ACCOUNT verdict): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nBranch when a rotation Lambda in account A cannot read a secret in account B.\n\n### Step 11: Rotation token missing or recovery failure\nStep 11 deep dive (CloudTrail RotateSecret lookup, AWSPENDING staging-label check, recovery rotation, ROTATION_TOKEN_MISSING verdict): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nBranch on manual invocation without a ClientRequestToken or a stuck AWSPENDING.\n\n### Step 12: Twin secrets not synced across regions\nStep 12 deep dive (primary\u002Fsecondary describe-secret comparison, ReplicationStatus, TWIN_SECRETS_NOT_SYNCED verdict and fix): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nBranch when a region-paired twin did not rotate with the primary.\n\n### Step 12b: Redshift rotation function (engine-specific template)\nStep 12b deep dive (Redshift-specific handler and engine probes, REDSHIFT_ROTATION_FUNCTION verdict): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nBranch when a Redshift secret fails in setSecret with a SQL syntax error.\n\n### Step 13: Escalate or INSUFFICIENT_DATA\n\nIf none of the above produced a positive root-cause match, OR the\nsymptom clearly indicates an AWS-side incident (region event, Secrets\nManager outage, Lambda outage), emit one of:\n\n- **ESCALATE** — AWS-side incident. Surface the AWS Health event ARN,\n  the secret's `describe-secret` output, and the relevant CloudTrail\n  error. Recommend opening a Support case. Do NOT continue diagnosing;\n  the cause is outside the customer's control.\n- **INSUFFICIENT_DATA** — A specific probe requires operator input.\n  List the missing pieces (rotation Lambda name, Master Secret ARN,\n  recent CloudWatch log stream, database endpoint) and the next probe\n  to run once the info is available.\n\n## Output format\n\n```text\nTARGET: \u003Csecret-id or ARN>\nVERDICT: ROOT_CAUSE_IDENTIFIED | INSUFFICIENT_DATA | ESCALATE\nREASON: \u003C1-2 sentences naming the failed layer and the failing probe>\nLAYER: \u003CROTATION_LAMBDA_TIMEOUT | ROTATION_LAMBDA_VPC |\n        ROTATION_LAMBDA_DB_CREDENTIAL | ROTATION_LAMBDA_DB_ENDPOINT |\n        SCHEDULE_MISSING | SCHEDULE_DISABLED |\n        MASTER_SECRET_MISCONFIGURED | PERMISSION_ROTATION_ROLE |\n        PERMISSION_CROSS_ACCOUNT | KMS_DECRYPT_ROLE |\n        ROTATION_TOKEN_MISSING | SUPERUSER_INSUFFICIENT |\n        STRATEGY_CONFLICT | TWIN_SECRETS_NOT_SYNCED |\n        PREVIOUS_CREDENTIAL_NOT_STORED | REDSHIFT_ROTATION_FUNCTION |\n        UNKNOWN>\nEVIDENCE:\n  - \u003Cobserved symptom — error string or LastRotatedDate delta>\n  - \u003Cfailing probe — command and its output that confirms the cause>\n  - \u003Cpassing probes — layers ruled out>\nREMEDIATION:\n  1. \u003Cspecific action with CLI command>\n  2. \u003Cverification command after the fix>\nCONFIRM: Before executing any state-changing CLI, emit and await\n  operator approval: \"CONFIRM: About to \u003Caction> on \u003Csecret-id> in\n  \u003Cregion>. Proceed? (yes\u002Fno)\"\n```\n\n### Worked example — ROTATION_LAMBDA_TIMEOUT (default 3s)\n\n```text\nTARGET: prod\u002Fdb\u002Fpayments-primary\nVERDICT: ROOT_CAUSE_IDENTIFIED\nREASON: Rotation Lambda SecretRotation-prod-db-payments has Timeout=3\n  (the Lambda default); CloudWatch Duration Maximum is 3.00s on every\n  rotation attempt. The Lambda is killed before the four-step protocol\n  can complete the setSecret step. The LastRotatedDate is 2026-07-30;\n  today is 2026-08-05 — six schedule intervals have been missed.\nLAYER: ROTATION_LAMBDA_TIMEOUT\nEVIDENCE:\n  - Symptom: LastRotatedDate is 2026-07-30T03:17:22Z; RotationRules\n    is rate(1d); RotationEnabled is true.\n  - Probe: aws lambda get-function-configuration returns Timeout=3,\n    Runtime=python3.12, LastModified=2026-07-25 (creation; never\n    reconfigured).\n  - Probe: aws logs filter-log-events returns \"Task timed out after\n    3.00 seconds\" in 6 of 6 rotation attempts in the last 24 hours;\n    the preceding log line is \"createSecret: staging new password\".\n  - Passing: VpcConfig is correctly attached to the DB subnets;\n    rotation role has secretsmanager:GetSecretValue and kms:Decrypt\n    on the CMK; EventBridge rule is ENABLED with the correct target.\nREMEDIATION:\n  1. Raise the rotation Lambda timeout to 30 seconds:\n     aws lambda update-function-configuration \\\n       --function-name SecretRotation-prod-db-payments --timeout 30 \\\n       --profile \u003Cp>\n  2. Trigger an immediate rotation to verify:\n     aws secretsmanager rotate-secret \\\n       --secret-id prod\u002Fdb\u002Fpayments-primary \\\n       --rotation-rule AutomaticallyAfterDays=1 --profile \u003Cp>\n  3. Verify LastRotatedDate advances within 5 minutes:\n     aws secretsmanager describe-secret \\\n       --secret-id prod\u002Fdb\u002Fpayments-primary \\\n       --query LastRotatedDate --output text\nCONFIRM: Before updating the Lambda timeout, emit and await:\n  \"CONFIRM: About to raise SecretRotation-prod-db-payments timeout\n   to 30s. Proceed? (yes\u002Fno)\"\n```\n\n### Worked example — SCHEDULE_MISSING (EventBridge rule deleted)\nFull worked example (recreate deleted EventBridge rule + add-permission + verify rotation): [references\u002Fworked-examples.md](references\u002Fworked-examples.md).\nThe ROTATION_LAMBDA_TIMEOUT example above is the primary worked example.\n\n### Worked example — INSUFFICIENT_DATA\nFull INSUFFICIENT_DATA worked example (missing Lambda name, connection keys, RDS identifier, re-prompt): [references\u002Fworked-examples.md](references\u002Fworked-examples.md).\nLoad on demand when required context is missing.\n\n## Anti-Patterns — NEVER\n\n- NEVER declare `ROOT_CAUSE_IDENTIFIED` without a failing probe that\n  matches the symptom. A \"process of elimination\" diagnosis erodes\n  operator trust when the real cause is elsewhere.\n\n- NEVER raise the rotation Lambda timeout above 30s without first\n  ruling out VPC connectivity and DB endpoint issues. A rotation that\n  \"needs\" 900s is almost always blocking on an unreachable database;\n  raising the timeout just delays the failure and consumes Lambda\n  compute budget.\n\n- NEVER assume the rotation Lambda is in the same VPC as the database.\n  Rotation is \"managed\" by Secrets Manager, but the Lambda runs in\n  YOUR account with YOUR VPC config. Always verify `VpcConfig`.\n\n- NEVER confuse the rotation role's IAM permissions with the Master\n  Secret's database privileges. IAM permissions let the Lambda read\n  the secret; database privileges let the Lambda execute `ALTER USER`.\n  A \"permission denied\" in the Lambda logs must be routed by where the\n  error originated: a CloudTrail `AccessDenied` event is IAM; a SQL\n  `permission denied` is database.\n\n- NEVER re-deploy the rotation template as the first remediation step.\n  The template is rarely the cause; the rotation Lambda's config\n  (timeout, VPC, role, env vars), the EventBridge schedule, and the\n  database privileges are the common culprits. Re-deploying the\n  template without identifying the layer burns time and may reset\n  working configuration.\n\n- NEVER trigger a manual rotation via `lambda invoke`. Direct Lambda\n  invocation does not pass a `ClientRequestToken` and the Lambda will\n  fail at the first step. Always use `secretsmanager rotate-secret` to\n  trigger rotation; it injects the token correctly.\n\n- NEVER assume `LastRotatedDate` advances on a successful Lambda\n  invocation. The date advances only when the Lambda completes\n  `finishSecret`. A Lambda that succeeds at `createSecret`,\n  `setSecret`, and `testSecret` but throws in `finishSecret` will log\n  success but leave `LastRotatedDate` stale and a version stuck in\n  `AWSPENDING`.\n\n- NEVER conclude a rotation is healthy just because EventBridge is\n  firing. A schedule that fires but invokes a Lambda that immediately\n  errors produces a healthy invocation count and a stale\n  `LastRotatedDate`. Always cross-reference `LastRotatedDate` against\n  `RotationRules.ScheduleExpression`.\n\n- NEVER mix rotation template families across engines. The MySQL\n  template does not work on PostgreSQL; the PostgreSQL template does\n  not work on Redshift; the SQL Server template has separate Single\n  User and Multi User variants. Always match the template to the\n  engine AND the strategy.\n\n- NEVER assume the Master Secret is in the same account. Cross-account\n  rotation is common; the Master Secret's ARN must include the\n  account ID, the resource-based policy must list the rotation role,\n  and the KMS key policy must grant the rotation role if the Master is\n  encrypted with a customer-managed CMK.\n\n- NEVER assume the `AWSPREVIOUS` staging label exists on a brand-new\n  secret. The first successful rotation populates `AWSPREVIOUS`; a\n  rollback attempted before the first success fails with \"no previous\n  version available\".\n\n- NEVER rotate a secret without confirming the application will pick\n  up the new value. Applications that cache the secret value at\n  startup (and never call `GetSecretValue` again) will continue using\n  the old credential after rotation. The rotation succeeds; the\n  application breaks. Pair rotation with an application-side refresh\n  pattern (polling, refresh trigger, or `SecretRotation` EventBridge\n  event subscription).\n\n## Pre-flight safety checks (run before any state-changing CLI)\nFull safety guidance (confirm gate, read-only-first, rotate-secret non-blocking, vpc-config ENI recreation, batch limit of 5): [references\u002Ferror-handling.md](references\u002Ferror-handling.md).\nLoad on demand before any state-changing CLI.\n\n## Remediation guidance\nPer-layer fix and verify commands (ROTATION_LAMBDA_TIMEOUT, VPC, DB_ENDPOINT, SCHEDULE_MISSING\u002FDISABLED, MASTER_SECRET, PERMISSION_ROTATION_ROLE, KMS_DECRYPT_ROLE, CROSS_ACCOUNT, TOKEN, SUPERUSER, STRATEGY_CONFLICT, TWIN_SECRETS): [references\u002Ferror-handling.md](references\u002Ferror-handling.md).\nLoad on demand after a ROOT_CAUSE_IDENTIFIED verdict.\n\n## Deep reference: Secrets Manager rotation layer model\nThe full layer model (symptom-to-layer decision matrix, rotation-step-to-layer routing, execution-role minimum policy, EventBridge rule template, rotation strategy matrix): [references\u002Fadvanced-patterns.md](references\u002Fadvanced-patterns.md).\nLoad on demand for offline classification or template details.\n\n## Recent AWS features (2024-2026)\nRecent AWS features 2024-2026 (cross-account rotation, 30s default on managed templates, Redshift multi-user GA, EventBridge Scheduler, RotationFailed events): [references\u002Fadvanced-patterns.md](references\u002Fadvanced-patterns.md).\nLoad on demand when a trigger mechanism looks new.\n\n## References (load on demand)\n\n- [Diagnostic commands](references\u002Fdiagnostic-commands.md) — account-wide pre-flight gather-info commands, the secret-state short-circuit table, and every step's probe commands (Steps 2-12b)\n- [Worked examples](references\u002Fworked-examples.md) — SCHEDULE_MISSING and INSUFFICIENT_DATA worked examples plus the malformed-input re-prompt template\n- [Error handling](references\u002Ferror-handling.md) — pre-flight safety checks before state-changing CLIs and per-layer remediation guidance with fix and verify commands\n- [Advanced patterns](references\u002Fadvanced-patterns.md) — mindset, philosophy, Step 0 non-obvious behaviours, the rotation layer model, recent AWS features (2024-2026)\n- [Rotation Lambda reference](references\u002Frotation-lambda-reference.md) — rotation Lambda internals, four-step protocol contract, configuration defaults\n- [Rotation strategy reference](references\u002Frotation-strategy-reference.md) — Alternating vs Single User strategy and rotation template family detail\n\n## Domain\n\nAWS CloudOps \u002F Secrets Manager, Rotation Lambda, Database Credential\nLifecycle, EventBridge Scheduling, Cross-Account IAM, and Customer\nManaged Key encryption.\n\n## AWS documentation\n\n- **AWS Secrets Manager User Guide — Rotating secrets** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fsecretsmanager\u002Flatest\u002Fuserguide\u002Frotating-secrets.html\n- **Rotation Lambda templates** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fsecretsmanager\u002Flatest\u002Fuserguide\u002Freference_available-rotation-templates.html\n- **Rotation function internals (four-step protocol)** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fsecretsmanager\u002Flatest\u002Fuserguide\u002Frotating-secrets_lambda-app-set-up.html\n- **Cross-account rotation** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fsecretsmanager\u002Flatest\u002Fuserguide\u002Frotating-secrets_create-generic-template.html\n- **Twin secrets and replication** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fsecretsmanager\u002Flatest\u002Fuserguide\u002Fcreate-manage-multi-region-secrets.html\n- **EventBridge rules for rotation** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fsecretsmanager\u002Flatest\u002Fuserguide\u002Frotate-secrets_schedule.html\n- **KMS key policy for Secrets Manager** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fsecretsmanager\u002Flatest\u002Fuserguide\u002Fsecurity-encryption.html\n- **AWS Lambda execution role for rotation** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fsecretsmanager\u002Flatest\u002Fuserguide\u002Frotating-secrets-required-permissions.html\n- **AWS Health** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fhealth\u002Flatest\u002Fug\u002F\n","# Secrets Manager Rotation Troubleshooter\n\n## Quick start\n\n- **Symptom → layer map (first plausible match drives the first probe):**\n  rotation Lambda `Task timed out` and the timeout is 3s (default)\n  → `ROTATION_LAMBDA_TIMEOUT`; rotation Lambda `Task timed out` at 15s\n  with a DB in another subnet → `ROTATION_LAMBDA_VPC`; rotation Lambda\n  logged `AccessDenied` calling `secretsmanager:GetSecretValue` on the\n  Master Secret → `PERMISSION_ROTATION_ROLE`; `LastRotatedDate` is days\n  stale and the EventBridge rule is missing → `SCHEDULE_MISSING`;\n  rotation Lambda logged `Could not connect to database at host ...` or\n  `wrong port` → `ROTATION_LAMBDA_DB_ENDPOINT`; rotation Lambda logged\n  `Master Secret ARN ... does not exist` → `MASTER_SECRET_MISCONFIGURED`;\n  rotation Lambda logged `Rotating back to previous credential` on every\n  attempt → `PREVIOUS_CREDENTIAL_NOT_STORED`; the Lambda was invoked\n  manually without a `ClientRequestToken` → `ROTATION_TOKEN_MISSING`.\n- **Always verify with a probe, never guess.** Each layer has a single\n  command that proves or disproves it. A `ROOT_CAUSE_IDENTIFIED` verdict\n  requires positive evidence — a failing probe that matches the symptom\n  — not a process of elimination.\n- **Rotation is a four-step protocol: `createSecret`, `setSecret`,\n  `testSecret`, `finishSecret`.** Most rotation failures surface in\n  `setSecret` (the step that issues the SQL `ALTER USER` \u002F `CREATE USER`\n  against the target database). Read the Lambda logs to identify the\n  step that failed before guessing the layer.\n- **Rotation strategy is set at template-deploy time, not at the\n  secret.** `Alternating Users` creates a clone of the current user and\n  rotates the clone's password (safer; requires `CREATE USER` +\n  `GRANT`); `Single User` updates the existing user's password in place\n  (atomic; requires no extra privileges but risks brief connection\n  failures during the rotation step). A rotation Lambda written for one\n  strategy cannot rotate a secret configured for the other without a\n  code change.\n- **ESCALATE for AWS-side incidents.** A regional Secrets Manager or\n  Lambda outage, or an AWS Health event affecting the rotation service,\n  is not customer-fixable — escalate to AWS Support and surface the\n  event ARN.\n\n## Mindset\nFull diagnostic mindset (rotation incidents are Lambda config, schedule, or DB-permission incidents, not template bugs): [references\u002Fadvanced-patterns.md](references\u002Fadvanced-patterns.md).\nLoad on demand for the senior-engineer framing.\n\n## Philosophy\nThe four senior-engineer behaviours (rotation step name drives diagnostic order, LastRotatedDate is the canonical health signal, execution role vs DB credential, rate(1d) is a schedule not a guarantee): [references\u002Fadvanced-patterns.md](references\u002Fadvanced-patterns.md).\nLoad on demand when routing by rotation step name.\n\n## Quick reference — symptom triage table\n\n| Symptom phrase \u002F log line | Most likely layer | First probe |\n|---|---|---|\n| `Task timed out after 3.00 seconds` (rotation Lambda default) | `ROTATION_LAMBDA_TIMEOUT` | `lambda get-function-configuration` (Timeout), CloudWatch Duration vs Timeout |\n| `Task timed out after 15.00 seconds` + DB in another subnet | `ROTATION_LAMBDA_VPC` | `lambda get-function-configuration` (VpcConfig), `ec2 describe-route-tables` for the subnet |\n| `Could not connect to database at host ... port ...` | `ROTATION_LAMBDA_DB_ENDPOINT` | Rotation Lambda env vars \u002F Master Secret value (`host`, `port`, `dbname`), `rds describe-db-instances` |\n| `AccessDenied` calling `secretsmanager:GetSecretValue` | `PERMISSION_ROTATION_ROLE` | `iam simulate-principal-policy` on the rotation role for `secretsmanager:GetSecretValue` on the secret ARN |\n| `AccessDenied` calling `kms:Decrypt` | `KMS_DECRYPT_ROLE` | `kms describe-key`, simulate `kms:Decrypt` on the rotation role |\n| `Master Secret ARN ... does not exist` or `ResourceNotFoundException` | `MASTER_SECRET_MISCONFIGURED` | Rotation Lambda env var `SECRETS_MANAGER_MASTER_ID` vs actual Master Secret ARN |\n| `LastRotatedDate` is days\u002Fweeks stale; EventBridge rule missing | `SCHEDULE_MISSING` | `events describe-rule` for the rotation rule; `events list-targets-by-rule` |\n| `LastRotatedDate` is stale; EventBridge rule `State: DISABLED` | `SCHEDULE_DISABLED` | `events describe-rule --name \u003Crule> --query State` |\n| `Rotating back to previous credential` on every attempt | `PREVIOUS_CREDENTIAL_NOT_STORED` | Lambda logs in the `setSecret` step; verify `AWSPREVIOUS` staging label exists |\n| `Rotation request missing ClientRequestToken` | `ROTATION_TOKEN_MISSING` | CloudTrail `RotateSecret` event; check for direct Lambda invocation |\n| `permission denied for table mysql.user` (or pg_roles) | `SUPERUSER_INSUFFICIENT` | Master Secret DB user's `SUPERUSER` \u002F `rds_superuser` attribute |\n| `CREATE USER failed ... already exists` (Alternating) | `STRATEGY_CONFLICT` | Rotation Lambda code: `ROTATION_STRATEGY` env var or template family |\n| Redshift secret rotated but `staging_*` user creation failed | `REDSHIFT_ROTATION_FUNCTION` | Rotation Lambda uses the Redshift-specific template, not the generic MySQL one |\n| Cross-account: rotation Lambda in account A cannot read secret in account B | `PERMISSION_CROSS_ACCOUNT` | Secret resource policy grants `secretsmanager:GetSecretValue` to the rotation role ARN in account A |\n| Region-paired twin: `prod\u002Fdb\u002Fpayments` rotated but `eu-west-1\u002Fdb\u002Fpayments` did not | `TWIN_SECRETS_NOT_SYNCED` | Compare `LastRotatedDate` across regions; verify the second region's rotation config |\n| None of the above, region-wide Secrets Manager outage | `ESCALATE` | `aws health describe-events` for `AWS_SECRETS_MANAGER` |\n\n## Pre-flight: secret state and gather-info gate\n\nBefore running symptom-specific probes, gather the canonical secret\nconfiguration and short-circuit on secret states that mimic rotation\nfailures.\n\n### Account-wide pre-flight commands\nAccount-wide pre-flight commands 1-6 (describe-secret, rotation Lambda log filter, get-resource-policy, get-function-configuration, EventBridge rules, AWS Health): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nRun these before any symptom-specific probe.\n\n### Secret-state short-circuit\nThe secret-state short-circuit table (healthy vs stale vs RotationEnabled:false vs DeletedDate vs missing AWSCURRENT vs AWSPENDING stuck vs OwningService): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nConsult it right after describe-secret to short-circuit mimics.\n\nIf the input is malformed (missing SecretId, absent symptom\ndescription, no caller context for live diagnosis), emit:\n\nThe INSUFFICIENT_DATA re-prompt template for malformed\u002Fmissing input: [references\u002Fworked-examples.md](references\u002Fworked-examples.md).\nEmit it whenever required context is missing.\n\n## Process — Diagnostic decision tree (apply in symptom order)\n\nThe diagnostic tree is symptom-driven. Pick the entry point based on the\nobserved symptom, then walk the layer-specific probes in order. **Never\nemit `ROOT_CAUSE_IDENTIFIED` without a failing probe that matches the\nsymptom.**\n\n### Step 0: Rotation protocol and non-obvious behaviours\nThe ten non-obvious behaviours (four-step protocol, 3s default timeout, schedule jitter, Master Secret vs rotating secret, cross-account dual policy, CMK decrypt, Lambda-VPC attachment, AWSPENDING recovery, Alternating engine support, AWSPREVIOUS on first rotation): [references\u002Fadvanced-patterns.md](references\u002Fadvanced-patterns.md).\nLoad on demand when the failing step is unclear.\n\n### Step 1: Symptom entry — pick the diagnostic branch\n\n| Symptom | Branch |\n|---|---|\n| Rotation Lambda `Task timed out after 3.00 seconds` | Step 2 — Lambda timeout |\n| Rotation Lambda `Task timed out` at > 3s; DB in another subnet\u002Fregion | Step 3 — VPC connectivity |\n| Rotation Lambda `Could not connect to database at host ...` \u002F `wrong port` \u002F `unknown database` | Step 4 — DB endpoint |\n| Rotation Lambda `AccessDenied` calling `secretsmanager:GetSecretValue` \u002F `kms:Decrypt` | Step 5 — IAM permissions |\n| Rotation Lambda `Master Secret ARN ... does not exist` | Step 6 — Master Secret |\n| `LastRotatedDate` stale; rotation never invoked | Step 7 — EventBridge schedule |\n| Rotation Lambda logged `permission denied for table mysql.user` \u002F `must be superuser` | Step 8 — DB superuser |\n| Rotation Lambda logged `CREATE USER failed ... already exists` \u002F `Rotating back` | Step 9 — Strategy conflict |\n| Cross-account rotation Lambda cannot read secret in another account | Step 10 — Cross-account |\n| Rotation invoked but `LastRotatedDate` did not advance; `AWSPENDING` stuck | Step 11 — Recovery failure |\n| Region-paired twin did not rotate when primary did | Step 12 — Twin secrets |\n| None of the above | Step 13 — Escalate \u002F INSUFFICIENT_DATA |\n\n### Step 2: Rotation Lambda timeout (default 3s)\n\nSymptom: rotation Lambda CloudWatch logs show `Task timed out after\n3.00 seconds`. `LastRotatedDate` is stale. The default Lambda timeout\nis 3s; the AWS-managed rotation templates override this to 30s at\ndeploy time, but a manually created rotation Lambda may inherit the\ndefault.\n\nProbes (lambda get-function-configuration Timeout\u002FMemory\u002FRuntime, CloudWatch Duration statistics vs configured Timeout): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nLoad on demand to run the timeout probes.\n\nIf `Maximum` Duration is at or just above the configured `Timeout`, the\nfunction is being killed before completing the four-step protocol. If\nthe configured Timeout is 3, **ROOT_CAUSE_IDENTIFIED** with\n`LAYER: ROTATION_LAMBDA_TIMEOUT`.\n\nFix command (lambda update-function-configuration --timeout 30) plus the do-not-just-raise-the-timeout note: [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nLoad on demand when applying the ROTATION_LAMBDA_TIMEOUT fix.\n\n### Step 3: VPC connectivity — rotation Lambda cannot reach database\n\nSymptom: rotation Lambda logs `Could not connect to database at host\n\u003Cendpoint>` or `Connection timed out` against the RDS\u002FAurora\u002FRedshift\nendpoint. The Lambda's Timeout is ≥ 15s but the connection never\nestablishes.\n\nProbes (lambda get-function-configuration VpcConfig, subnet route tables, Lambda\u002FDB security groups): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nLoad on demand to run the VPC probes.\n\n`VpcConfig` empty or null → the Lambda is NOT in a VPC and cannot\nreach a private database. **ROOT_CAUSE_IDENTIFIED** with\n`LAYER: ROTATION_LAMBDA_VPC`. Fix: attach the Lambda to the database's\nsubnets (`update-function-configuration --vpc-config ...`).\n\nIf `VpcConfig` is populated, verify the subnet route table:\n\nRoute-table reading guide (local route, peered VPC, PrivateLink ENI) and security-group checks per DB port: [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nLoad on demand when VpcConfig is populated but the DB is unreachable.\n\n**Verdicts:**\n- Lambda not VPC-attached, DB is private: ROOT_CAUSE_IDENTIFIED,\n  `LAYER: ROTATION_LAMBDA_VPC`.\n- Lambda in wrong subnet, no route to DB CIDR: ROOT_CAUSE_IDENTIFIED,\n  `LAYER: ROTATION_LAMBDA_VPC`.\n- SG egress or ingress missing the DB port: ROOT_CAUSE_IDENTIFIED,\n  `LAYER: ROTATION_LAMBDA_VPC`.\n\n### Step 4: Database endpoint — wrong host, port, or database name\n\nSymptom: rotation Lambda logs `Could not connect to database at host\n\u003Chost> port \u003Cport>` or `unknown database \"\u003Cdbname>\"`. The connection\nestablishes (no VPC issue) but authentication or routing fails.\n\nConnection-string probes (Lambda env vars, secret value host\u002Fport\u002Fdbname, rds describe-db-instances, redshift describe-clusters): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nLoad on demand to cross-reference the database endpoint.\n\nCommon mismatches:\n\n| Symptom | Cause |\n|---|---|\n| Secret's `host` is the writer endpoint but the rotation tries to connect to a reader | Aurora reader endpoint resolves to a read-only instance; `ALTER USER` fails on read-only. Use the cluster writer endpoint. |\n| Secret's `port` is 3306 but the engine is PostgreSQL (5432) | Cross-engine template confusion; the secret was created for one engine but the rotation Lambda uses another engine's template. |\n| Secret's `dbname` does not exist on the DB instance | Database was renamed or never created; the connection succeeds but `USE \u003Cdbname>` fails. |\n| Secret's `host` is an IP that no longer resolves | RDS failover moved the endpoint; the secret's value was never updated. Re-rotate or fix the secret value. |\n\n**Verdict:** ROOT_CAUSE_IDENTIFIED, `LAYER: ROTATION_LAMBDA_DB_ENDPOINT`.\nFix: update the secret value (or Lambda env var) to the correct host,\nport, or dbname, then trigger a manual rotation.\n\n### Step 5: IAM — rotation role lacks permissions\n\nSymptom: rotation Lambda logs `AccessDenied` calling\n`secretsmanager:GetSecretValue`, `secretsmanager:PutSecretValue`, or\n`kms:Decrypt`. The Lambda runs but fails at the first Secrets Manager\nAPI call.\n\nIAM probes (rotation role ARN, simulate-principal-policy for GetSecretValue\u002FPutSecretValue\u002FDescribeSecret and kms:Decrypt, CloudTrail lookup for the denied API) plus implicitDeny vs explicitDeny reading: [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nLoad on demand on any AccessDenied symptom.\n\n**Verdicts:**\n- Rotation role lacks `secretsmanager:GetSecretValue` on the secret or\n  Master Secret: ROOT_CAUSE_IDENTIFIED,\n  `LAYER: PERMISSION_ROTATION_ROLE`.\n- Rotation role lacks `kms:Decrypt` on the customer-managed CMK:\n  ROOT_CAUSE_IDENTIFIED, `LAYER: KMS_DECRYPT_ROLE`.\n- Same-account resource policy does not matter (identity policy\n  suffices); cross-account requires both — see Step 10.\n\n### Step 6: Master Secret ARN misconfiguration\n\nSymptom: rotation Lambda logs `ResourceNotFoundException: Master\nSecret ARN ... does not exist` or `AccessDenied` reading a Master\nSecret that the secret actually points to.\n\nMaster Secret probes (Lambda env vars SECRETS_MANAGER_MASTER_ID \u002F MASTER_ARN, describe-secret on the Master ARN): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nLoad on demand when the Master Secret is suspect.\n\nCommon patterns:\n\n| Symptom | Cause |\n|---|---|\n| Env var value is a secret NAME, not an ARN; cross-account rotation fails | Same-account tolerates the name; cross-account requires the full ARN with the account ID. Use the ARN. |\n| Env var points to a Master Secret in a different region | Rotation Lambda cannot read cross-region without a replication setup; the Master Secret must be in the same region. |\n| Env var was updated but the Lambda was not published as a new version | The Lambda alias still points to the old version with the old ARN. Publish and update the alias. |\n| Master Secret was deleted but the rotation Lambda still references it | Restore the Master Secret or reconfigure rotation to use the rotating secret itself as Master (single-user strategy). |\n\n**Verdict:** ROOT_CAUSE_IDENTIFIED,\n`LAYER: MASTER_SECRET_MISCONFIGURED`. Fix: update the Lambda env var\nto the correct Master Secret ARN, publish a new version, and shift the\nalias.\n\n### Step 7: EventBridge schedule — rotation never triggers\n\nSymptom: `LastRotatedDate` is days or weeks stale. The rotation Lambda\nhas no recent CloudWatch log streams. The secret's\n`RotationRules.ScheduleExpression` is correct (e.g., `rate(1d)`), but\nnothing fires.\n\nSchedule probes (list-targets-by-rule, list-rules filtered for the rotation Lambda, per-rule target verification): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nLoad on demand when LastRotatedDate is stale.\n\nFor each candidate rule:\n\n| Field | Effect |\n|---|---|\n| Rule not found | The rule was deleted. **ROOT_CAUSE_IDENTIFIED**, `LAYER: SCHEDULE_MISSING`. Recreate via `events put-rule` + `put-targets`. |\n| `State: DISABLED` | The rule exists but is disabled. **ROOT_CAUSE_IDENTIFIED**, `LAYER: SCHEDULE_DISABLED`. Re-enable via `events enable-rule`. |\n| `ScheduleExpression` does not match `RotationRules.ScheduleExpression` | Schedule drift; the rule fires at the wrong cadence. Update via `events put-rule`. |\n| Targets empty or target ARN ≠ rotation Lambda ARN | The rule fires but invokes the wrong Lambda (or no Lambda). Update via `events put-targets`. |\n| Target Lambda permission missing (`lambda:InvokeFunction` for the EventBridge principal) | The rule fires but Lambda rejects the invocation. Verify via `lambda get-policy` for the resource-based statement allowing `events.amazonaws.com`. |\n\n**Verdicts:**\n- Rule deleted: ROOT_CAUSE_IDENTIFIED, `LAYER: SCHEDULE_MISSING`.\n- Rule disabled: ROOT_CAUSE_IDENTIFIED, `LAYER: SCHEDULE_DISABLED`.\n- Rule target wrong \u002F missing: ROOT_CAUSE_IDENTIFIED,\n  `LAYER: SCHEDULE_MISSING`.\n\n### Step 8: Database superuser privileges insufficient\nStep 8 deep dive (engine-by-engine privilege requirements, Master Secret username read, SUPERUSER_INSUFFICIENT verdict and fix): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md) and [references\u002Ferror-handling.md](references\u002Ferror-handling.md).\nBranch when the Lambda logs `permission denied for table mysql.user` or `must be superuser`.\n\n### Step 9: Rotation strategy conflict (Alternating vs Single User)\nStep 9 deep dive (Single vs Alternating template families, ROTATION_STRATEGY env probes, STRATEGY_CONFLICT patterns and verdict): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nBranch when the Lambda logs `CREATE USER failed ... already exists` or `Rotating back`.\n\n### Step 10: Cross-account secret access denied\nStep 10 deep dive (cross-account simulate + get-resource-policy probes, required resource-policy statement, KMS key policy, PERMISSION_CROSS_ACCOUNT verdict): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nBranch when a rotation Lambda in account A cannot read a secret in account B.\n\n### Step 11: Rotation token missing or recovery failure\nStep 11 deep dive (CloudTrail RotateSecret lookup, AWSPENDING staging-label check, recovery rotation, ROTATION_TOKEN_MISSING verdict): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nBranch on manual invocation without a ClientRequestToken or a stuck AWSPENDING.\n\n### Step 12: Twin secrets not synced across regions\nStep 12 deep dive (primary\u002Fsecondary describe-secret comparison, ReplicationStatus, TWIN_SECRETS_NOT_SYNCED verdict and fix): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nBranch when a region-paired twin did not rotate with the primary.\n\n### Step 12b: Redshift rotation function (engine-specific template)\nStep 12b deep dive (Redshift-specific handler and engine probes, REDSHIFT_ROTATION_FUNCTION verdict): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\nBranch when a Redshift secret fails in setSecret with a SQL syntax error.\n\n### Step 13: Escalate or INSUFFICIENT_DATA\n\nIf none of the above produced a positive root-cause match, OR the\nsymptom clearly indicates an AWS-side incident (region event, Secrets\nManager outage, Lambda outage), emit one of:\n\n- **ESCALATE** — AWS-side incident. Surface the AWS Health event ARN,\n  the secret's `describe-secret` output, and the relevant CloudTrail\n  error. Recommend opening a Support case. Do NOT continue diagnosing;\n  the cause is outside the customer's control.\n- **INSUFFICIENT_DATA** — A specific probe requires operator input.\n  List the missing pieces (rotation Lambda name, Master Secret ARN,\n  recent CloudWatch log stream, database endpoint) and the next probe\n  to run once the info is available.\n\n## Output format\n\n```text\nTARGET: \u003Csecret-id or ARN>\nVERDICT: ROOT_CAUSE_IDENTIFIED | INSUFFICIENT_DATA | ESCALATE\nREASON: \u003C1-2 sentences naming the failed layer and the failing probe>\nLAYER: \u003CROTATION_LAMBDA_TIMEOUT | ROTATION_LAMBDA_VPC |\n        ROTATION_LAMBDA_DB_CREDENTIAL | ROTATION_LAMBDA_DB_ENDPOINT |\n        SCHEDULE_MISSING | SCHEDULE_DISABLED |\n        MASTER_SECRET_MISCONFIGURED | PERMISSION_ROTATION_ROLE |\n        PERMISSION_CROSS_ACCOUNT | KMS_DECRYPT_ROLE |\n        ROTATION_TOKEN_MISSING | SUPERUSER_INSUFFICIENT |\n        STRATEGY_CONFLICT | TWIN_SECRETS_NOT_SYNCED |\n        PREVIOUS_CREDENTIAL_NOT_STORED | REDSHIFT_ROTATION_FUNCTION |\n        UNKNOWN>\nEVIDENCE:\n  - \u003Cobserved symptom — error string or LastRotatedDate delta>\n  - \u003Cfailing probe — command and its output that confirms the cause>\n  - \u003Cpassing probes — layers ruled out>\nREMEDIATION:\n  1. \u003Cspecific action with CLI command>\n  2. \u003Cverification command after the fix>\nCONFIRM: Before executing any state-changing CLI, emit and await\n  operator approval: \"CONFIRM: About to \u003Caction> on \u003Csecret-id> in\n  \u003Cregion>. Proceed? (yes\u002Fno)\"\n```\n\n### Worked example — ROTATION_LAMBDA_TIMEOUT (default 3s)\n\n```text\nTARGET: prod\u002Fdb\u002Fpayments-primary\nVERDICT: ROOT_CAUSE_IDENTIFIED\nREASON: Rotation Lambda SecretRotation-prod-db-payments has Timeout=3\n  (the Lambda default); CloudWatch Duration Maximum is 3.00s on every\n  rotation attempt. The Lambda is killed before the four-step protocol\n  can complete the setSecret step. The LastRotatedDate is 2026-07-30;\n  today is 2026-08-05 — six schedule intervals have been missed.\nLAYER: ROTATION_LAMBDA_TIMEOUT\nEVIDENCE:\n  - Symptom: LastRotatedDate is 2026-07-30T03:17:22Z; RotationRules\n    is rate(1d); RotationEnabled is true.\n  - Probe: aws lambda get-function-configuration returns Timeout=3,\n    Runtime=python3.12, LastModified=2026-07-25 (creation; never\n    reconfigured).\n  - Probe: aws logs filter-log-events returns \"Task timed out after\n    3.00 seconds\" in 6 of 6 rotation attempts in the last 24 hours;\n    the preceding log line is \"createSecret: staging new password\".\n  - Passing: VpcConfig is correctly attached to the DB subnets;\n    rotation role has secretsmanager:GetSecretValue and kms:Decrypt\n    on the CMK; EventBridge rule is ENABLED with the correct target.\nREMEDIATION:\n  1. Raise the rotation Lambda timeout to 30 seconds:\n     aws lambda update-function-configuration \\\n       --function-name SecretRotation-prod-db-payments --timeout 30 \\\n       --profile \u003Cp>\n  2. Trigger an immediate rotation to verify:\n     aws secretsmanager rotate-secret \\\n       --secret-id prod\u002Fdb\u002Fpayments-primary \\\n       --rotation-rule AutomaticallyAfterDays=1 --profile \u003Cp>\n  3. Verify LastRotatedDate advances within 5 minutes:\n     aws secretsmanager describe-secret \\\n       --secret-id prod\u002Fdb\u002Fpayments-primary \\\n       --query LastRotatedDate --output text\nCONFIRM: Before updating the Lambda timeout, emit and await:\n  \"CONFIRM: About to raise SecretRotation-prod-db-payments timeout\n   to 30s. Proceed? (yes\u002Fno)\"\n```\n\n### Worked example — SCHEDULE_MISSING (EventBridge rule deleted)\nFull worked example (recreate deleted EventBridge rule + add-permission + verify rotation): [references\u002Fworked-examples.md](references\u002Fworked-examples.md).\nThe ROTATION_LAMBDA_TIMEOUT example above is the primary worked example.\n\n### Worked example — INSUFFICIENT_DATA\nFull INSUFFICIENT_DATA worked example (missing Lambda name, connection keys, RDS identifier, re-prompt): [references\u002Fworked-examples.md](references\u002Fworked-examples.md).\nLoad on demand when required context is missing.\n\n## Anti-Patterns — NEVER\n\n- NEVER declare `ROOT_CAUSE_IDENTIFIED` without a failing probe that\n  matches the symptom. A \"process of elimination\" diagnosis erodes\n  operator trust when the real cause is elsewhere.\n\n- NEVER raise the rotation Lambda timeout above 30s without first\n  ruling out VPC connectivity and DB endpoint issues. A rotation that\n  \"needs\" 900s is almost always blocking on an unreachable database;\n  raising the timeout just delays the failure and consumes Lambda\n  compute budget.\n\n- NEVER assume the rotation Lambda is in the same VPC as the database.\n  Rotation is \"managed\" by Secrets Manager, but the Lambda runs in\n  YOUR account with YOUR VPC config. Always verify `VpcConfig`.\n\n- NEVER confuse the rotation role's IAM permissions with the Master\n  Secret's database privileges. IAM permissions let the Lambda read\n  the secret; database privileges let the Lambda execute `ALTER USER`.\n  A \"permission denied\" in the Lambda logs must be routed by where the\n  error originated: a CloudTrail `AccessDenied` event is IAM; a SQL\n  `permission denied` is database.\n\n- NEVER re-deploy the rotation template as the first remediation step.\n  The template is rarely the cause; the rotation Lambda's config\n  (timeout, VPC, role, env vars), the EventBridge schedule, and the\n  database privileges are the common culprits. Re-deploying the\n  template without identifying the layer burns time and may reset\n  working configuration.\n\n- NEVER trigger a manual rotation via `lambda invoke`. Direct Lambda\n  invocation does not pass a `ClientRequestToken` and the Lambda will\n  fail at the first step. Always use `secretsmanager rotate-secret` to\n  trigger rotation; it injects the token correctly.\n\n- NEVER assume `LastRotatedDate` advances on a successful Lambda\n  invocation. The date advances only when the Lambda completes\n  `finishSecret`. A Lambda that succeeds at `createSecret`,\n  `setSecret`, and `testSecret` but throws in `finishSecret` will log\n  success but leave `LastRotatedDate` stale and a version stuck in\n  `AWSPENDING`.\n\n- NEVER conclude a rotation is healthy just because EventBridge is\n  firing. A schedule that fires but invokes a Lambda that immediately\n  errors produces a healthy invocation count and a stale\n  `LastRotatedDate`. Always cross-reference `LastRotatedDate` against\n  `RotationRules.ScheduleExpression`.\n\n- NEVER mix rotation template families across engines. The MySQL\n  template does not work on PostgreSQL; the PostgreSQL template does\n  not work on Redshift; the SQL Server template has separate Single\n  User and Multi User variants. Always match the template to the\n  engine AND the strategy.\n\n- NEVER assume the Master Secret is in the same account. Cross-account\n  rotation is common; the Master Secret's ARN must include the\n  account ID, the resource-based policy must list the rotation role,\n  and the KMS key policy must grant the rotation role if the Master is\n  encrypted with a customer-managed CMK.\n\n- NEVER assume the `AWSPREVIOUS` staging label exists on a brand-new\n  secret. The first successful rotation populates `AWSPREVIOUS`; a\n  rollback attempted before the first success fails with \"no previous\n  version available\".\n\n- NEVER rotate a secret without confirming the application will pick\n  up the new value. Applications that cache the secret value at\n  startup (and never call `GetSecretValue` again) will continue using\n  the old credential after rotation. The rotation succeeds; the\n  application breaks. Pair rotation with an application-side refresh\n  pattern (polling, refresh trigger, or `SecretRotation` EventBridge\n  event subscription).\n\n## Pre-flight safety checks (run before any state-changing CLI)\nFull safety guidance (confirm gate, read-only-first, rotate-secret non-blocking, vpc-config ENI recreation, batch limit of 5): [references\u002Ferror-handling.md](references\u002Ferror-handling.md).\nLoad on demand before any state-changing CLI.\n\n## Remediation guidance\nPer-layer fix and verify commands (ROTATION_LAMBDA_TIMEOUT, VPC, DB_ENDPOINT, SCHEDULE_MISSING\u002FDISABLED, MASTER_SECRET, PERMISSION_ROTATION_ROLE, KMS_DECRYPT_ROLE, CROSS_ACCOUNT, TOKEN, SUPERUSER, STRATEGY_CONFLICT, TWIN_SECRETS): [references\u002Ferror-handling.md](references\u002Ferror-handling.md).\nLoad on demand after a ROOT_CAUSE_IDENTIFIED verdict.\n\n## Deep reference: Secrets Manager rotation layer model\nThe full layer model (symptom-to-layer decision matrix, rotation-step-to-layer routing, execution-role minimum policy, EventBridge rule template, rotation strategy matrix): [references\u002Fadvanced-patterns.md](references\u002Fadvanced-patterns.md).\nLoad on demand for offline classification or template details.\n\n## Recent AWS features (2024-2026)\nRecent AWS features 2024-2026 (cross-account rotation, 30s default on managed templates, Redshift multi-user GA, EventBridge Scheduler, RotationFailed events): [references\u002Fadvanced-patterns.md](references\u002Fadvanced-patterns.md).\nLoad on demand when a trigger mechanism looks new.\n\n## References (load on demand)\n\n- [Diagnostic commands](references\u002Fdiagnostic-commands.md) — account-wide pre-flight gather-info commands, the secret-state short-circuit table, and every step's probe commands (Steps 2-12b)\n- [Worked examples](references\u002Fworked-examples.md) — SCHEDULE_MISSING and INSUFFICIENT_DATA worked examples plus the malformed-input re-prompt template\n- [Error handling](references\u002Ferror-handling.md) — pre-flight safety checks before state-changing CLIs and per-layer remediation guidance with fix and verify commands\n- [Advanced patterns](references\u002Fadvanced-patterns.md) — mindset, philosophy, Step 0 non-obvious behaviours, the rotation layer model, recent AWS features (2024-2026)\n- [Rotation Lambda reference](references\u002Frotation-lambda-reference.md) — rotation Lambda internals, four-step protocol contract, configuration defaults\n- [Rotation strategy reference](references\u002Frotation-strategy-reference.md) — Alternating vs Single User strategy and rotation template family detail\n\n## Domain\n\nAWS CloudOps \u002F Secrets Manager, Rotation Lambda, Database Credential\nLifecycle, EventBridge Scheduling, Cross-Account IAM, and Customer\nManaged Key encryption.\n\n## AWS documentation\n\n- **AWS Secrets Manager User Guide — Rotating secrets** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fsecretsmanager\u002Flatest\u002Fuserguide\u002Frotating-secrets.html\n- **Rotation Lambda templates** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fsecretsmanager\u002Flatest\u002Fuserguide\u002Freference_available-rotation-templates.html\n- **Rotation function internals (four-step protocol)** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fsecretsmanager\u002Flatest\u002Fuserguide\u002Frotating-secrets_lambda-app-set-up.html\n- **Cross-account rotation** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fsecretsmanager\u002Flatest\u002Fuserguide\u002Frotating-secrets_create-generic-template.html\n- **Twin secrets and replication** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fsecretsmanager\u002Flatest\u002Fuserguide\u002Fcreate-manage-multi-region-secrets.html\n- **EventBridge rules for rotation** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fsecretsmanager\u002Flatest\u002Fuserguide\u002Frotate-secrets_schedule.html\n- **KMS key policy for Secrets Manager** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fsecretsmanager\u002Flatest\u002Fuserguide\u002Fsecurity-encryption.html\n- **AWS Lambda execution role for rotation** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fsecretsmanager\u002Flatest\u002Fuserguide\u002Frotating-secrets-required-permissions.html\n- **AWS Health** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fhealth\u002Flatest\u002Fug\u002F\n",{"name":5,"license":15,"metadata":24,"description":7,"compatibility":43},{"phase":25,"author":26,"domain":27,"family":28,"version":29,"keywords":30,"task_type":31,"complexity":32,"entry_point":33,"skill_class":34,"when_to_use":35,"requires_llm":36,"verdict_shape":37,"when_not_to_use":38,"lifecycle_status":39,"invocation_schema":40,"supports_pipeline":36,"invocation_example":41,"activation_triggers":42},"2","Jacky Chan — AWS Community Builder","aws-cloudops","Security","0.1.0","Secrets Manager, rotation, rotation Lambda, Master Secret, Alternating Users, Single User, rotation strategy, EventBridge schedule, rotation token, ClientRequestToken, cross-account secret, twin secrets, RotateSecret, rotation template, Superuser, database credential, rotation disabled, rate(1d), VPC connectivity, KMS decrypt","troubleshoot","high","false","capability","Diagnosing an AWS Secrets Manager secret rotation failure (rotation Lambda error, schedule not triggering, cross-account access denied, Master Secret misconfiguration, rotation Lambda timeout, VPC connectivity to the database, Alternating vs Single User strategy conflict, twin secrets not synced, rotation token missing, superuser permissions insufficient), walking a symptom to the failed layer with verify and fix commands, validating why a secret's LastRotatedDate is stale, or triaging a \"secret rotation is broken\" page where the root cause may be rotation Lambda, EventBridge schedule, IAM, network, KMS, or database permissions — not necessarily the rotation template itself.","true","ROOT_CAUSE_IDENTIFIED | INSUFFICIENT_DATA | ESCALATE","Initial rotation configuration (use the Secrets Manager console or CloudFormation rotation template setup), secret value retrieval at application runtime (use the application's SDK path), audit of all secrets lacking rotation (use secrets-manager-rotation-coverage-auditor), encryption-key rotation for the underlying CMK (use kms-key-rotation-operator), or password-policy enforcement (use iam-password-policy-auditor). This skill diagnoses rotation-time failures; it does not design rotation schedules or audit steady-state rotation posture.","active","'Input: either (a) a symptom description (error message from the rotation Lambda CloudWatch logs, observed behaviour such as \"LastRotatedDate is 30 days ago\", \"rotation succeeds but applications cannot connect\"), optionally paired with the secret's describe-secret output and recent rotation Lambda logs, OR (b) a SecretId plus caller context for live-account diagnosis. Output: a deterministic TARGET\u002FVERDICT\u002FREASON\u002FLAYER\u002FEVIDENCE\u002FREMEDIATION block where VERDICT ∈ {ROOT_CAUSE_IDENTIFIED, INSUFFICIENT_DATA, ESCALATE} and LAYER ∈ {ROTATION_LAMBDA_TIMEOUT, ROTATION_LAMBDA_VPC, ROTATION_LAMBDA_DB_CREDENTIAL, ROTATION_LAMBDA_DB_ENDPOINT, SCHEDULE_MISSING, SCHEDULE_DISABLED, MASTER_SECRET_MISCONFIGURED, PERMISSION_ROTATION_ROLE, PERMISSION_CROSS_ACCOUNT, KMS_DECRYPT_ROLE, ROTATION_TOKEN_MISSING, SUPERUSER_INSUFFICIENT, STRATEGY_CONFLICT, TWIN_SECRETS_NOT_SYNCED, PREVIOUS_CREDENTIAL_NOT_STORED, REDSHIFT_ROTATION_FUNCTION, UNKNOWN}.'","\"# Minimal valid input (offline symptom classification):\\nSymptom: \\\"Secret prod\u002Fdb\u002Fpayments-primary has not rotated in 7 days;\\nLastRotatedDate is 2026-07-30. The rotation Lambda's last execution\\nlogged 'Task timed out after 3.00 seconds'.\\\"\\nSecretId: prod\u002Fdb\u002Fpayments-primary\\nRotationEnabled: true\\nRotationLambdaARN: arn:aws:lambda:us-east-1:111111111111:function:SecretsManagerRotation-prod-db-payments\\nRotationRules: {ScheduleExpression: 'rate(1d)'}\\nLastRotatedDate: 2026-07-30T03:17:22Z\\nOwningService: (none — customer-managed)\\nKmsKeyId: alias\u002Faws\u002Fsecretsmanager\\nRotationLambda LastLog: 'Task timed out after 3.00 seconds'\"","","Agent runtime that reads SKILL.md (Claude Code, Cursor, Windsurf, Codex, Gemini). Offline symptom classification works from pasted error messages and rotation configuration. Live-account diagnosis uses aws secretsmanager describe-secret, aws secretsmanager","main","eeace51bd5dd1db2030729d83eeab988861553aa","skills\u002Fsecrets-manager-rotation-troubleshooter",0,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills","https:\u002F\u002Fgithub.com\u002Fjhkchan","CTO & Co-Founder @Beever-AI  & @votee. CAISP. AWS Community Builder. Organizer@GDGCloudHK. MongoDB Champion. Technopreneur + Infornographer.","2026-09-24 08:54:10",{"source":8,"skillName":5,"commands":53,"prompt":63,"zipAvailable":64},[54,57,60],{"tool":55,"command":56},"npx","npx skills add jhkchan\u002Faws-agent-skills --skill secrets-manager-rotation-troubleshooter",{"tool":58,"command":59},"bunx","bunx skills add jhkchan\u002Faws-agent-skills --skill secrets-manager-rotation-troubleshooter",{"tool":61,"command":62},"pnpm","pnpm dlx skills add jhkchan\u002Faws-agent-skills --skill secrets-manager-rotation-troubleshooter","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002FSKILL.md",true,[66,72,76,80,84,88,91,95,99,103,107,111,115,119,123,127,131,135,139,143,147],{"path":67,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":70,"sourceHtmlUrl":71},"eval\u002Ftest-cases.yaml","REFERENCE","参考文档",15842,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Feval\u002Ftest-cases.yaml",{"path":73,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":74,"sourceHtmlUrl":75},"evals\u002Fbaselines\u002Falternating-users-strategy-conflict.md",834,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Fevals\u002Fbaselines\u002Falternating-users-strategy-conflict.md",{"path":77,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":78,"sourceHtmlUrl":79},"evals\u002Fbaselines\u002Fcross-account-secret-access-denied.md",889,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Fevals\u002Fbaselines\u002Fcross-account-secret-access-denied.md",{"path":81,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":82,"sourceHtmlUrl":83},"evals\u002Fbaselines\u002Feventbridge-schedule-deleted.md",876,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Fevals\u002Fbaselines\u002Feventbridge-schedule-deleted.md",{"path":85,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":86,"sourceHtmlUrl":87},"evals\u002Fbaselines\u002Fmaster-secret-arn-misconfigured.md",909,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Fevals\u002Fbaselines\u002Fmaster-secret-arn-misconfigured.md",{"path":89,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":78,"sourceHtmlUrl":90},"evals\u002Fbaselines\u002Frotation-lambda-vpc-timeout.md","https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Fevals\u002Fbaselines\u002Frotation-lambda-vpc-timeout.md",{"path":92,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":93,"sourceHtmlUrl":94},"evals\u002Feval_config.yaml",2102,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Fevals\u002Feval_config.yaml",{"path":96,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":97,"sourceHtmlUrl":98},"evals\u002Fevals.json",5238,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Fevals\u002Fevals.json",{"path":100,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":101,"sourceHtmlUrl":102},"evals\u002Fprompts\u002Falternating-users-strategy-conflict.md",3316,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Fevals\u002Fprompts\u002Falternating-users-strategy-conflict.md",{"path":104,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":105,"sourceHtmlUrl":106},"evals\u002Fprompts\u002Fcross-account-secret-access-denied.md",3139,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Fevals\u002Fprompts\u002Fcross-account-secret-access-denied.md",{"path":108,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":109,"sourceHtmlUrl":110},"evals\u002Fprompts\u002Feventbridge-schedule-deleted.md",1977,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Fevals\u002Fprompts\u002Feventbridge-schedule-deleted.md",{"path":112,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":113,"sourceHtmlUrl":114},"evals\u002Fprompts\u002Fmaster-secret-arn-misconfigured.md",2279,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Fevals\u002Fprompts\u002Fmaster-secret-arn-misconfigured.md",{"path":116,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":117,"sourceHtmlUrl":118},"evals\u002Fprompts\u002Frotation-lambda-vpc-timeout.md",2618,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Fevals\u002Fprompts\u002Frotation-lambda-vpc-timeout.md",{"path":120,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":121,"sourceHtmlUrl":122},"examples\u002FREADME.md",9188,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Fexamples\u002FREADME.md",{"path":124,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":125,"sourceHtmlUrl":126},"references\u002Fadvanced-patterns.md",12663,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Freferences\u002Fadvanced-patterns.md",{"path":128,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":129,"sourceHtmlUrl":130},"references\u002Fdiagnostic-commands.md",19313,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Freferences\u002Fdiagnostic-commands.md",{"path":132,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":133,"sourceHtmlUrl":134},"references\u002Ferror-handling.md",8394,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Freferences\u002Ferror-handling.md",{"path":136,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":137,"sourceHtmlUrl":138},"references\u002Frotation-lambda-reference.md",10433,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Freferences\u002Frotation-lambda-reference.md",{"path":140,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":141,"sourceHtmlUrl":142},"references\u002Frotation-strategy-reference.md",15094,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Freferences\u002Frotation-strategy-reference.md",{"path":144,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":145,"sourceHtmlUrl":146},"references\u002Fworked-examples.md",3801,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Freferences\u002Fworked-examples.md",{"path":148,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":149,"sourceHtmlUrl":150},"skill.usf.yaml",4691,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fsecrets-manager-rotation-troubleshooter\u002Fskill.usf.yaml",[152,159,166,173,180,187],{"skillNo":153,"name":154,"nameZh":6,"description":155,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":156,"categories":157,"keywords":158,"publishedAt":19},"01M7ADVD60X4PCAPKSE58P9NDC","xray-tracing-deployer","Deploys AWS X-Ray distributed tracing with production-grade configuration: X-Ray daemon deployment (EC2 systemd, ECS sidecar, EKS DaemonSet, Lambda built-in), SDK instrumentation (Java, Python, Node.js, Go, .NET) with annotations and metadata, sampling rules (default reservoir + rate vs custom rules with URL \u002F method \u002F service-name predicates), service map generation, trace query and retrieval, groups and insights, CloudWatch ServiceLens integration, IAM permissions (PutTraceSegments, GetSamplingRules), and latest features (Lambda Powertools tracing, OpenTelemetry \u002F ADOT Collector support, W3C trace context). Emits a READY_TO_DEPLOY checklist with every configuration item verified. Use when enabling X-Ray on a new application, adding tracing to ECS \u002F EKS \u002F Lambda, validating a sampling configuration, or generating daemon deployment CLI and IaC templates. Triggers: X-Ray, distributed tracing, X-Ray daemon, sampling rules, service map, X-Ray SDK, OpenTelemetry, ADOT, Lambda Powertools, CloudWatch ServiceLens.","https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fxray-tracing-deployer\u002FSKILL.md",[],[],{"skillNo":160,"name":161,"nameZh":6,"description":162,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":163,"categories":164,"keywords":165,"publishedAt":19},"01M7ADTVJ7DCBM9CB1FY51GH7J","wellarchitected-workload-auditor","Audits AWS Well-Architected Tool workloads for review staleness, per-pillar high-risk issue counts, milestone tracking gaps, and remediation plan completeness. Emits a deterministic verdict (STALE_REVIEW | HIGH_RISK | CONFIG_GAP | OK) per workload with per-pillar risk breakdown, milestone trend analysis, and specific remediation. Use when reviewing Well-Architected workloads, checking WA review freshness, auditing high-risk issues per pillar, validating milestone coverage, or tracking remediation plans.","https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fwellarchitected-workload-auditor\u002FSKILL.md",[],[],{"skillNo":167,"name":168,"nameZh":6,"description":169,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":170,"categories":171,"keywords":172,"publishedAt":19},"01M7ADT59G6FTTNZYHT2QRPNNP","wellarchitected-review-operator","Operates AWS Well-Architected Tool reviews end-to-end — creates workloads, runs pillar reviews (operational excellence, security, reliability, performance, cost optimization, sustainability, Prosperity), answers questions with risk-tier rationale, generates reports and improvement plans, creates milestones, and integrates with Trusted Advisor and Well-Architected Labs. Runs deterministic pre-checks (workload existence, lens availability, pillar coverage), emits the exact update-answer \u002F get-consolidated-report CLI behind a CONFIRM gate, and verifies risk-tier transitions. Emits READY | BLOCKED | COMPLETED. Use when creating a workload, answering pillar questions, generating an improvement plan, creating a milestone, or integrating with Trusted Advisor. Triggers: well-architected tool, well architected review, workload review, pillar, operational excellence, security, reliability, performance, cost optimization, sustainability, prosperity, improvement plan, milestone, trusted advisor.","https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fwellarchitected-review-operator\u002FSKILL.md",[],[],{"skillNo":174,"name":175,"nameZh":6,"description":176,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":177,"categories":178,"keywords":179,"publishedAt":19},"01M7ADSH5JA9QHNPPHQCD7EVA7","wafv2-web-acl-deployer","Provisions AWS WAFv2 Web ACLs with correct production defaults: scope selection (CLOUDFRONT vs REGIONAL), managed rule groups (AWSManagedRulesCommonRuleSet, SQLiRuleSet, LinuxRuleSet, WindowsRuleSet, IPReputationList, BotControlRuleSet, AmazonIpReputationList, ATP), custom rules (byte match, IP set, regex pattern, geo match, size constraint, rate-based), rule priority ordering, logging destinations (Kinesis Firehose, CloudWatch Logs, S3), resource association (ALB, API Gateway, CloudFront), rate-based rules with aggregate keys (IP, forwarded IP), Challenge and CAPTCHA actions, and account takeover prevention. Emits a READY_TO_DEPLOY checklist. Use when creating a Web ACL, attaching managed rules, configuring rate limiting, enabling WAF logging, associating a Web ACL with an ALB\u002FAPI Gateway\u002FCloudFront, or configuring CAPTCHA\u002FChallenge actions. Triggers: create WAF Web ACL, WAFv2, managed rule groups, WAF rate limiting, WAF logging, WAF CAPTCHA, WAF Challenge action, bot control, ATP managed rules.","https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fwafv2-web-acl-deployer\u002FSKILL.md",[],[],{"skillNo":181,"name":182,"nameZh":6,"description":183,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":184,"categories":185,"keywords":186,"publishedAt":19},"01M7ADRYFAT8R73FBNCHRC573X","wafv2-web-acl-auditor","Audits AWS WAFv2 Web ACL configurations to determine whether the ACL provides effective protection: default-action posture (Allow vs Block), managed-rule-group coverage gaps, rule effectiveness (BLOCK vs COUNT, shadow\u002Fbypass rules, stale exclusions), rate-based rule correctness, logging and visibility configuration, and text-transformation bypass vectors. Emits a deterministic verdict per Web ACL. Use when reviewing a WAFv2 Web ACL for security posture, checking managed-rule coverage, validating rate-limit configuration, auditing WAF logging, or hardening a WAF before production deployment. Triggers: WAF, WAFv2, Web ACL, managed rule group, AWSManagedRules, rate-based rule, CAPTCHA, Challenge, OverrideAction, Count mode, forwarded IP, Bot Control, ATP, text transformation, WAF logging, WAF visibility, OWASP, SQLi, XSS, RFI.","https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fwafv2-web-acl-auditor\u002FSKILL.md",[],[],{"skillNo":188,"name":189,"nameZh":6,"description":190,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":191,"categories":192,"keywords":193,"publishedAt":19},"01M7ADR8W7G72JX4KFGN03QK8Q","waf-rule-deployer","Provisions AWS WAFv2 rule sets with production defaults: Web ACL creation (CloudFront scope in us-east-1 vs Regional scope), rule groups (managed vs custom) with capacity planning in WCU (Web ACL Capacity Units, 1500 default per ACL), managed rule groups (AWSManagedRulesCommonRuleSet, AWSManagedRulesSQLiRuleSet, AWSManagedRulesAmazonIpReputationList, AWSManagedRulesLinuxRuleSet, AWSManagedRulesWindowsRuleSet, AWSManagedRulesUnixRuleSet), custom rules (byte-match, regex pattern set, geo-match, rate-based), IP set and regex pattern set resources, rule priority ordering (lowest number evaluated first), action types (allow, block, count, CAPTCHA, Challenge), bot control managed rule group, account takeover. Triggers: create WAF rule, WAF managed rule group, WAF custom rule, WAF byte match, WAF geo match, WAF rate based rule, WAF IP set, WAF regex pattern set, WAF WCU budget, WAF CloudFront ACL, WAF logging Kinesis Firehose, WAF bot control, WAF ATP, WAF label matching, WAF Challenge action, WAF CAPTCHA.","https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fwaf-rule-deployer\u002FSKILL.md",[],[]]