[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7A2YHMSS0BCT8S5W7D28S87":3,"skill-same:01M7A2YHMSS0BCT8S5W7D28S87":148},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":44,"commitSha":45,"skillDirectory":46,"forksCount":47,"repositoryHtmlUrl":48,"repositoryHomepageUrl":6,"ownerHtmlUrl":49,"ownerBlogUrl":6,"ownerDescription":50,"updatedAt":51,"install":52,"resources":65},"01M7A2YHMSS0BCT8S5W7D28S87","alb-5xx-troubleshooter",null,"Diagnoses ALB and NLB 5xx errors (500 InternalServerError, 502 BadGateway, 503 ServiceUnavailable, 504 GatewayTimeout, 561 Unauthorized) through a systematic diagnostic tree covering target health check failures (wrong path, port, matcher, timeout), target security group misconfiguration (ALB SG cannot reach target SG on target port), idle timeout exceeded (60s default), deregistration draining stuck, listener rule misconfiguration (wrong target group, wrong priority), WAF blocks, and invalid backend responses. Walks symptoms to root cause with describe-target-health, ALB access logs in S3 (target_processing_time, target_status_code, error_reason), security group analysis, and listener rule inspection. Emits ROOT_CAUSE_FOUND with the specific failure layer or ESCALATE. Use when ALB returns 5xx errors, targets unhealthy, no healthy targets, or intermittent target failures.","jhkchan\u002Faws-agent-skills","jhkchan","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F1083902?v=4","01M7A2SVQYP9GQTPTDXP5JKYQ4","Jacky Chan","01M7A2SVS117YWCTZABB75HZR1",4,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: alb-5xx-troubleshooter\ndescription: Diagnoses ALB and NLB 5xx errors (500 InternalServerError, 502 BadGateway, 503 ServiceUnavailable, 504 GatewayTimeout, 561 Unauthorized) through a systematic diagnostic tree covering target health check failures (wrong path, port, matcher, timeout), target security group misconfiguration (ALB SG cannot reach target SG on target port), idle timeout exceeded (60s default), deregistration draining stuck, listener rule misconfiguration (wrong target group, wrong priority), WAF blocks, and invalid backend responses. Walks symptoms to root cause with describe-target-health, ALB access logs in S3 (target_processing_time, target_status_code, error_reason), security group analysis, and listener rule inspection. Emits ROOT_CAUSE_FOUND with the specific failure layer or ESCALATE. Use when ALB returns 5xx errors, targets unhealthy, no healthy targets, or intermittent target failures.\nlicense: Apache-2.0\ncompatibility: Agent runtime that reads SKILL.md (Claude Code, Cursor, Windsurf, Codex, Gemini). Offline symptom classification works from pasted error codes and target group metadata. Live-account diagnosis uses aws elbv2 describe-target-health, describe-target-groups, describe-listeners, describe-rules, describe-load-balancers, describe-load-balancer-attributes, aws ec2 describe-security-groups, aws s3 cp (for ALB access logs), and aws cloudwatch get-metric-statistics (AWS CLI v2, SSO or key-based...\nmetadata:\n  domain: aws-cloudops\n  complexity: high\n  requires_llm: 'true'\n  phase: '2'\n  supports_pipeline: 'true'\n  entry_point: 'false'\n  family: Networking\n  task_type: troubleshoot\n  skill_class: capability\n  lifecycle_status: active\n  verdict_shape: ROOT_CAUSE_FOUND | NEED_MORE_INFO | ESCALATE\n  when_to_use: Diagnosing an ALB or NLB 5xx error (500, 502, 503, 504, 561), walking a symptom to the failed target, security group, health check, or listener layer with verify and fix commands, validating target health failures, identifying unhealthy target groups, diagnosing idle timeout exceeded, or triaging a \"the load balancer returns 5xx\" page where the root cause may be target health, SG rules, health check configuration, deregistration, WAF, or listener rules — not necessarily the load balancer itself.\n  when_not_to_use: Configuration posture audits (use elbv2-load-balancer-auditor for TLS policy, access-log enablement, deletion protection, SG exposure), TLS\u002Fcipher negotiation issues (use elbv2-load-balancer-auditor INSECURE_LISTENER), or capacity\u002Fthroughput sizing (use the optimize task type). This skill diagnoses runtime 5xx failures and target health, not config posture.\n  activation_triggers: ALB 5xx error, ALB 502 BadGateway, ALB 503 ServiceUnavailable, ALB 504 GatewayTimeout, ALB 561 Unauthorized, NLB 5xx, target unhealthy, no healthy targets, target group health check failed, ALB access logs 5xx, target security group blocked, deregistration draining stuck, WAF blocked request ALB, troubleshoot ALB, troubleshoot load balancer 5xx\n  invocation_schema: 'Input: either (a) a symptom description (5xx error code, observed pattern, failing target group), optionally paired with the load balancer and target group metadata (describe-load-balancers, describe-target-groups, describe-target-health output), OR (b) a load balancer ARN or target group ARN for live-account diagnosis. Output: a deterministic TARGET\u002FVERDICT\u002FREASON\u002FLAYER\u002FEVIDENCE\u002F REMEDIATION block where VERDICT ∈ {ROOT_CAUSE_FOUND, NEED_MORE_INFO, ESCALATE} and LAYER ∈ {TARGET_HEALTH_CHECK, TARGET_SG_BLOCKED, TARGET_NONE_HEALTHY, TARGET_TIMEOUT, TARGET_INVALID_RESPONSE, LISTENER_MISCONFIGURED, WAF_BLOCKED, DEREGISTRATION_STUCK, ALB_INTERNAL, UNKNOWN}.'\n  version: 0.1.0\n  author: Jacky Chan — AWS Community Builder\n  keywords: ALB, NLB, ELBv2, load balancer, 5xx, 500, 502, 503, 504, 561, BadGateway, ServiceUnavailable, GatewayTimeout, target health, target group, health check, security group, idle timeout, deregistration, draining, WAF, listener rule, access logs, troubleshooting\n  tags: elbv2, alb, nlb, load-balancer, networking, troubleshooting, 5xx, target-health\n---\n\n# ALB 5xx Troubleshooter\n\n## Quick start\n\n- **Error code → layer map (first plausible match drives the first probe):**\n  502 → TARGET_INVALID_RESPONSE \u002F TARGET_SG_BLOCKED; 503 →\n  TARGET_NONE_HEALTHY \u002F TARGET_HEALTH_CHECK \u002F DEREGISTRATION_STUCK \u002F\n  TARGET_SG_BLOCKED; 504 → TARGET_TIMEOUT; 561 → WAF_BLOCKED;\n  500 → ALB_INTERNAL (rare — check AWS Health Dashboard).\n- **Always verify with a probe, never guess.** Each layer has a single\n  command (or access-log field) that proves or disproves it. A\n  ROOT_CAUSE_FOUND verdict requires positive evidence — a failing probe\n  that matches the symptom — not \"must be the targets.\"\n- **Target health is the root of most 5xx.** A 503 with no healthy\n  targets, a 502 from an unreachable target, and a 504 from a slow\n  target all originate in the target group. Probe `describe-target-health`\n  first — it is the single highest-signal command.\n- **ESCALATE for AWS-side incidents.** A sustained 500 with healthy\n  targets, no recent change, and clean configs is an AWS-side event —\n  surface the Health Dashboard link and open a Support case.\n\n## Mindset\n\nFull diagnostic mindset (the ALB is the messenger, not the cause): [references\u002Fadvanced-patterns.md](references\u002Fadvanced-patterns.md).\n\n## Philosophy\n\nThe four senior-engineer behaviours (5xx code tells WHERE, health check config is the top root cause, SG rules are bidirectional, access logs are the forensic trail): [references\u002Fadvanced-patterns.md](references\u002Fadvanced-patterns.md).\n\n## Quick navigation\n\n| If the symptom is... | Go to | First probe |\n|---|---|---|\n| 502 with `Target.InvalidResponse` or `Target.ConnectionFailed` | Step 2a | `describe-target-health` + direct curl on target |\n| 502 with target SG not allowing ALB SG | Step 2b | `describe-security-groups` on target SG |\n| 503 with all targets `unhealthy` | Step 3a | `describe-target-health` + health check config |\n| 503 with all targets `draining` | Step 3c | `deregistration_delay.timeout_seconds` |\n| 503 with zero registered targets | Step 3b | `describe-target-groups` TargetType |\n| 504 with `Target.Timeout` in access logs | Step 4 | ALB idle timeout + target response time |\n| 561 from WAF | Step 5 | WAF Web ACL logs |\n| 502\u002F503 from wrong target group on listener | Step 6 | `describe-rules` for listener priorities |\n| 500 sustained, healthy targets, no change | Step 7 | AWS Health Dashboard — ESCALATE |\n| Need the access log format and fields | Reference | `references\u002Ftarget-health-reference.md` |\n| Need full probes for any Step | Reference | `references\u002Fdiagnostic-commands.md` |\n| Need more worked examples | Reference | `references\u002Fworked-examples.md` |\n| Need per-layer fix commands | Reference | `references\u002Ferror-handling.md` |\n| Need Step 0 gotchas in full | Reference | `references\u002Fadvanced-patterns.md` |\n\n## Pre-flight: load balancer type and gather-info gate\n\nBefore running code-specific probes, gather the canonical load balancer,\nlistener, target group, and target health metadata. Misidentifying the\nload balancer type (ALB vs NLB) or the target type (instance vs ip vs\nlambda) produces false root causes.\n\n### Account-wide pre-flight commands\n\nAccount-wide pre-flight commands (describe-load-balancers, -listeners, -rules, -target-groups, -target-health, attributes, SGs, CloudWatch 5xx metrics, AWS Health): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n### Load-balancer-type short-circuit\n\n| Attribute | Value | Effect on diagnosis |\n|---|---|---|\n| `Type` | `application` | **ALB.** L7. Full 5xx diagnostic applies. Access logs include `target_status_code`, `target_processing_time`, `error_reason`. WAF can be associated (561 errors). Health checks are HTTP\u002FHTTPS. |\n| `Type` | `network` | **NLB.** L4. Access logs do NOT include `error_reason` or `target_status_code` (connection-level only). No WAF (no 561). Health checks can be TCP (misleading — a TCP handshake succeeds even when the app is broken). Cross-zone off by default. |\n| `Scheme` | `internet-facing` | ALB has public IPs. SG allowing 0.0.0.0\u002F0 on ports 80\u002F443 is expected. |\n| `Scheme` | `internal` | ALB has private IPs. SG allowing 0.0.0.0\u002F0 on any port is suspicious. |\n| `State` | `failed` | ALB is in a failed provisioning state. Output ERROR — do not classify. |\n\n### Target-type short-circuit\n\n| `TargetType` | Effect on diagnosis |\n|---|---|\n| `instance` | Target registered by EC2 instance ID. Uses the instance's primary private IP. Target SG is the instance's SG. |\n| `ip` | Target registered by IP address. Can be outside the VPC (peered VPC, on-prem via Direct Connect). SG rules must use CIDR (not referenced SG) for non-VPC IPs. |\n| `alb` | Target is another ALB (nested). Health check is on the parent ALB. Rare — usually a misconfiguration. |\n| `lambda` | Target is a Lambda function (ALB-only). No health check — Lambda invocations either succeed or fail. 502 from a Lambda target is a function error (similar to API Gateway Lambda proxy). |\n\nIf the input is malformed (missing load balancer ARN, missing target\ngroup ARN, ambiguous error code), emit:\n\nNEED_MORE_INFO re-prompt template for malformed\u002Fmissing input: [references\u002Fworked-examples.md](references\u002Fworked-examples.md).\n\n## Process — Diagnostic decision tree (apply in error-code order)\n\nThe diagnostic tree is error-code-driven. Pick the entry point based on\nthe observed 5xx code, then walk the layer-specific probes in order. Each\nlayer ends with either a positive root-cause confirmation (failing probe\nthat matches the symptom) or a pass that moves to the next layer. **Never\nemit ROOT_CAUSE_FOUND without a failing probe that matches the symptom.**\n\n### Step 0: Non-obvious behaviours that change diagnosis\n\nThe 13 non-obvious behaviours (shallow \u002Fhealth checks, NLB TCP false-healthy, ALB-SG-vs-IP rules, draining window, S3 log delivery, error_reason semantics, idle-timeout scope, NLB cross-zone default, rule priority order, instance-ID registration, WAF 561, Lambda 502, grace period, AZ reachability): [references\u002Fadvanced-patterns.md](references\u002Fadvanced-patterns.md).\n\n### Step 1: Symptom entry — pick the diagnostic branch\n\nMap the 5xx error code to a branch and jump to that branch's section.\n\n| Error code | Meaning | Branch |\n|---|---|---|\n| **502 BadGateway** | Target returned invalid response or connection failed | Step 2 |\n| **503 ServiceUnavailable** | No healthy targets available | Step 3 |\n| **504 GatewayTimeout** | Target did not respond within idle timeout | Step 4 |\n| **561 Unauthorized** | WAF blocked the request | Step 5 |\n| **500 InternalServerError** | ALB internal failure (rare) | Step 7 |\n| Ambiguous \u002F intermittent \u002F mixed codes | Gather access logs first | Step 1b |\n\n### Step 1b: Gather access logs (when the code is ambiguous)\n\nAccess-log location (access_logs.s3.bucket), fetch\u002Fparse commands, and the space-delimited field format (target_processing_time, target_status_code, error_reason): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n### Step 2: 502 BadGateway — target returned invalid response\n\nSymptom: client receives `502 BadGateway`. The ALB reached the target\n(or tried to) but received no valid HTTP response.\n\n#### 2a: Target returned invalid HTTP or connection failed\n\nProbes (describe-target-health, direct curl on target, access-log error_reason): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n**Verdict signals:**\n- Access log `error_reason: Target.InvalidResponse` → target returned\n  malformed HTTP (non-HTTP bytes, truncated response). **ROOT_CAUSE_FOUND**,\n  `LAYER: TARGET_INVALID_RESPONSE`. Fix: the target application.\n- Access log `error_reason: Target.ConnectionFailed` → connection to\n  target refused or reset. If targets are `healthy`, the target port\n  closed between health check and request. **ROOT_CAUSE_FOUND**,\n  `LAYER: TARGET_INVALID_RESPONSE`.\n- Target is `unhealthy` with `Target.FailedHealthChecks` → the target\n  is failing health checks; the ALB has no healthy target for this\n  request. Move to Step 3a (503 diagnosis — same root cause, different\n  manifestation).\n\n#### 2b: Target security group does not allow ALB SG\n\nProbes (target SG inbound rules vs ALB SG on target port): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n**Verdict signals:**\n- Target SG has NO inbound rule allowing the ALB SG on the target port\n  → **ROOT_CAUSE_FOUND**, `LAYER: TARGET_SG_BLOCKED`. The ALB cannot\n  reach the target. Fix: add an inbound rule to the target SG allowing\n  the ALB SG on the target port.\n- Target SG allows a specific IP\u002FCIDR but NOT the ALB SG → if the\n  allowed IP is a stale ALB IP, the rule is dead. **ROOT_CAUSE_FOUND**,\n  `LAYER: TARGET_SG_BLOCKED`. Fix: replace with an ALB-SG-referenced rule.\n- Target SG allows the ALB SG on the correct port → SG is not the\n  cause. Move to Step 2a (target application issue).\n\n### Step 3: 503 ServiceUnavailable — no healthy targets\n\nSymptom: client receives `503 ServiceUnavailable`. The ALB has no\nhealthy target in the target group to receive the request.\n\n#### 3a: All targets unhealthy (health check failing)\n\nProbes (target health reason, HealthCheckConfig fields, direct curl of the health path): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n**Verdict signals:**\n- Target health check path returns non-200 (e.g., 404, 500) → the\n  health check endpoint itself is broken. **ROOT_CAUSE_FOUND**,\n  `LAYER: TARGET_HEALTH_CHECK`. Fix: update the health check path to an\n  endpoint that returns 200 when healthy, or fix the endpoint.\n- `Matcher.HttpCode` does not include the status code the target returns\n  (e.g., matcher is `200` but target returns `204` on the health path)\n  → **ROOT_CAUSE_FOUND**, `LAYER: TARGET_HEALTH_CHECK`. Fix: update the\n  matcher to include the expected code(s) (e.g., `200,204`).\n- Health check timeout too short (`HealthCheckTimeoutSeconds` \u003C target\n  response time) → health check fails before the target responds.\n  **ROOT_CAUSE_FOUND**, `LAYER: TARGET_HEALTH_CHECK`. Fix: increase the\n  timeout.\n- Health check on wrong port (`HealthCheckPort` != application port)\n  → health check hits a port with no listener. **ROOT_CAUSE_FOUND**,\n  `LAYER: TARGET_HEALTH_CHECK`.\n\n#### 3b: Zero registered targets\n\nProbes (TargetType + registered target list): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n**Verdict signals:**\n- `TargetHealthDescriptions` is empty → no targets registered.\n  **ROOT_CAUSE_FOUND**, `LAYER: TARGET_NONE_HEALTHY`. Fix: register\n  targets (`register-targets`). This is common for newly created target\n  groups or after an Auto Scaling group scaled to zero.\n- All targets in `unused` state → targets are registered but the target\n  group is not attached to any listener rule. **ROOT_CAUSE_FOUND**,\n  `LAYER: LISTENER_MISCONFIGURED`. Fix: attach the target group to a\n  listener rule.\n\n#### 3c: All targets in draining state\n\nProbes (draining state + deregistration_delay.timeout_seconds): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n**Verdict signals:**\n- All targets `State: draining` with a long deregistration delay →\n  targets were deregistered but are still in the draining window.\n  **ROOT_CAUSE_FOUND**, `LAYER: DEREGISTRATION_STUCK`. Fix: register\n  new targets, or reduce the deregistration delay if the draining\n  window is too long for the deployment pattern.\n\n### Step 4: 504 GatewayTimeout — target did not respond in time\n\nSymptom: client receives `504 GatewayTimeout`. The target did not\nrespond within the ALB idle timeout.\n\nProbes (idle_timeout.timeout_seconds, access-log target_processing_time, TargetResponseTime metric, direct timed curl): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n**Verdict signals:**\n- `idle_timeout.timeout_seconds` is 60 (default), target response time\n  > 60s → **ROOT_CAUSE_FOUND**, `LAYER: TARGET_TIMEOUT`. Fix: raise the\n  idle timeout (if the workload legitimately needs it) or optimize the\n  target application.\n- Access log `error_reason: Target.Timeout` → confirms the target did\n  not respond within the idle timeout. **ROOT_CAUSE_FOUND**,\n  `LAYER: TARGET_TIMEOUT`.\n- `TargetResponseTime` metric Maximum approaches the idle timeout → the\n  target is slow. **ROOT_CAUSE_FOUND**, `LAYER: TARGET_TIMEOUT`.\n\n### Step 5: 561 Unauthorized — WAF blocked the request\n\nSymptom: client receives `561 Unauthorized`. This is a custom error from\nWAF blocking the request.\n\nProbes (get-web-acl-for-resource, list-web-acls, WAF BLOCK log filter): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n**Verdict signals:**\n- WAF Web ACL is associated with the ALB, and WAF logs show BLOCK actions\n  for the failing requests → **ROOT_CAUSE_FOUND**, `LAYER: WAF_BLOCKED`.\n  Fix: review the WAF rules — a false positive blocking legitimate\n  traffic needs the rule adjusted (add an IP exemption, tune the rate\n  limit, update the regex pattern).\n\n### Step 6: Listener rule misconfiguration\n\nSymptom: the \"wrong\" targets are receiving traffic, or a specific path\nreturns 503 while the root path works. This is a routing issue, not a\ntarget health issue.\n\nProbes (describe-rules priority order, listener DefaultActions): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n**Verdict signals:**\n- A high-priority rule (low number) matches all paths (`path-pattern: \u002F*`)\n  and forwards to the wrong target group → **ROOT_CAUSE_FOUND**,\n  `LAYER: LISTENER_MISCONFIGURED`. Fix: adjust rule priorities and\n  conditions.\n- Default action points to a deleted or empty target group →\n  **ROOT_CAUSE_FOUND**, `LAYER: LISTENER_MISCONFIGURED`. Fix: update the\n  default action.\n- A rule condition mismatches the incoming request (e.g., host-header\n  condition does not match) → requests fall through to the default action.\n  **ROOT_CAUSE_FOUND**, `LAYER: LISTENER_MISCONFIGURED`.\n\n### Step 7: 500 InternalServerError — rare ALB internal failure\n\nSymptom: client receives `500 InternalServerError`. This is rare and\nusually indicates an AWS-side issue.\n\nProbe (aws health describe-events for ELB): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n**Verdict signals:**\n- 500 sustained, all targets healthy, no recent change → **ESCALATE**,\n  `LAYER: ALB_INTERNAL`. Surface the AWS Health event ARN and open a\n  Support case.\n\n### Step 8: Escalate or NEED_MORE_INFO\n\nIf none of the above produced a positive root-cause match, OR the\nsymptom clearly indicates an AWS-side incident (region event, sustained\n500), emit one of:\n\n- **ESCALATE** — AWS-side incident. Surface the AWS Health event ARN,\n  the load balancer ARN, and the relevant access-log evidence. Recommend\n  opening a Support case.\n- **NEED_MORE_INFO** — A specific probe requires operator input. List\n  the missing pieces (access logs not enabled, target group ARN\n  unknown, health check endpoint unclear) and the next probe to run\n  once the info is available.\n\n## Output format\n\n```text\nTARGET: \u003Clb-arn>\u002F\u003Ctarget-group-arn>\nVERDICT: ROOT_CAUSE_FOUND | NEED_MORE_INFO | ESCALATE\nREASON: \u003C1-2 sentences naming the failed layer and the failing probe>\nLAYER: \u003CTARGET_HEALTH_CHECK | TARGET_SG_BLOCKED | TARGET_NONE_HEALTHY |\n        TARGET_TIMEOUT | TARGET_INVALID_RESPONSE | LISTENER_MISCONFIGURED |\n        WAF_BLOCKED | DEREGISTRATION_STUCK | ALB_INTERNAL | UNKNOWN>\nEVIDENCE:\n  - \u003Cobserved symptom — error code, error_reason, target state>\n  - \u003Cfailing probe — command and its output that confirms the cause>\n  - \u003Cpassing probes — layers ruled out>\nREMEDIATION:\n  1. \u003Cspecific action with CLI command>\n  2. \u003Cverification command after the fix>\nCONFIRM: Before executing any state-changing CLI, emit and await operator\n  approval: \"CONFIRM: About to \u003Caction> on \u003Clb-arn>\u002F\u003Ctg-arn>. Proceed?\n  (yes\u002Fno)\"\n```\n\n### Worked example — 503 from wrong health check path\n\n```text\nTARGET: arn:aws:elasticloadbalancing:us-east-1:111:load-balancer\u002Fapp\u002Fprod-web\u002Fabc \u002F target-group\u002Ftg-web\u002Fdef\nVERDICT: ROOT_CAUSE_FOUND\nREASON: The target group health check path \u002Fhealth returns 404 on all\n  targets — the application serves health on \u002Fhealthz. All targets are\n  marked unhealthy, and the ALB has no healthy target for incoming\n  requests (Step 3a).\nLAYER: TARGET_HEALTH_CHECK\nEVIDENCE:\n  - Symptom: clients receive 503 ServiceUnavailable on all paths. ALB\n    access logs show target_group_arn for tg-web but no\n    target_status_code (no target was selected).\n  - Probe: aws elbv2 describe-target-health --target-group-arn \u003Ctg-web>\n    returns all targets State: unhealthy, Reason:\n    Target.FailedHealthChecks.\n  - Probe: aws elbv2 describe-target-groups --target-group-arns \u003Ctg-web>\n    returns HealthCheckPath: \u002Fhealth, Matcher.HttpCode: 200.\n  - Probe: ssh \u003Cbastion> \"curl -v http:\u002F\u002F\u003Ctarget-ip>:8080\u002Fhealth\"\n    returns HTTP\u002F1.1 404 Not Found. \"curl http:\u002F\u002F\u003Ctarget-ip>:8080\u002Fhealthz\"\n    returns HTTP\u002F1.1 200 OK.\n  - Passing: target SG allows ALB SG on port 8080; ALB idle timeout is\n    60s; listener default action points to tg-web; no WAF associated.\nREMEDIATION:\n  1. Update the health check path to \u002Fhealthz:\n     aws elbv2 modify-target-group --target-group-arn \u003Ctg-web>\n       --health-check-path \u002Fhealthz --profile \u003Cp>\n  2. Wait for healthy_threshold_count consecutive successful checks\n     (default: 3-5 checks at HealthCheckIntervalSeconds).\n  3. Verify: describe-target-health shows State: healthy for all\n     targets; clients no longer see 503.\nCONFIRM: Before modifying the target group, emit and await:\n  \"CONFIRM: About to modify-target-group on tg-web (health-check-path\n   \u002Fhealth → \u002Fhealthz). Proceed? (yes\u002Fno)\"\n```\n\n### Worked example — 502 from target SG not allowing ALB SG\n\nFull worked example (502, healthy targets, SG missing tcp\u002F443 from ALB SG; LAYER: TARGET_SG_BLOCKED): [references\u002Fworked-examples.md](references\u002Fworked-examples.md).\n\n### Worked example — 504 from target exceeding idle timeout\n\nFull worked example (504 at exactly 60s, Target.Timeout; LAYER: TARGET_TIMEOUT): [references\u002Fworked-examples.md](references\u002Fworked-examples.md).\n\n## Anti-Patterns — NEVER\n\n- NEVER declare ROOT_CAUSE_FOUND without a failing probe that matches the\n  symptom. \"Must be the targets\" without running `describe-target-health`\n  or checking access logs erodes operator trust.\n\n- NEVER confuse 502 and 503. A 502 means the ALB reached the target but\n  got an invalid response (or connection failed). A 503 means no healthy\n  target was available (the ALB never sent the request to a target).\n  Probing target health for a 502 when all targets are healthy wastes\n  the incident window — the target application is the cause, not target\n  selection.\n\n- NEVER assume a `healthy` target is actually serving the application\n  correctly. A health check on `\u002Fhealth` returning 200 does NOT mean\n  the application works on its real endpoints. A target can be `healthy`\n  while returning 500 to clients — the ALB routes to it, and clients\n  see the 500. Always test the actual endpoint, not just the health\n  check path.\n\n- NEVER trust an NLB `healthy` status without verifying\n  `health_check.type`. The default TCP health check succeeds on a bare\n  handshake — an app returning 500s is marked `healthy`. For HTTP\u002FHTTPS\n  targets behind an NLB, require `health_check.type = HTTP\u002FHTTPS` with\n  a real `path`. A TCP check on an HTTP target is the most common cause\n  of \"healthy but broken\" NLB incidents.\n\n- NEVER configure a target SG rule with the ALB's IP address. ALB IPs\n  change when the ALB scales. Use the ALB security group ID as the\n  source — this is stable across scaling events.\n\n- NEVER assume access logs are delivered just because\n  `access_logs.s3.enabled` is `true`. The S3 bucket policy MUST grant\n  `elasticloadbalancing.amazonaws.com` write access with\n  `aws:SourceAccount` condition. A misconfigured bucket policy causes\n  logs to silently fail. Verify with `aws s3 ls` on the prefix after\n  5-10 minutes.\n\n- NEVER recommend raising the ALB idle timeout without verifying the\n  target can actually finish within the new timeout. A target that takes\n  300s will still 504 with a 120s timeout. For long-running requests,\n  migrate to an async pattern (202 + polling).\n\n- NEVER forget that ALB cross-zone load balancing is ALWAYS ON. It cannot\n  be disabled. Do not flag an ALB for cross-zone issues. Only NLB has\n  the toggle (off by default).\n\n- NEVER treat 561 as a backend error. 561 is a WAF block — the request\n  never reached the target. Check the WAF Web ACL, not the target health.\n\n- NEVER conflate deregistration delay with unhealthy targets. Targets in\n  `draining` state are NOT unhealthy — they were deliberately deregistered\n  and are serving in-flight requests. The fix is to register new targets,\n  not to \"fix\" the draining targets.\n\n- NEVER assume the listener default action points to the right target\n  group. A misconfigured default action (pointing to an empty or deleted\n  target group) produces 503 for any request that does not match a\n  specific rule. Always verify `DefaultActions` in `describe-listeners`.\n\n- NEVER evaluate listener rules out of priority order. Rules are\n  evaluated lowest-number-first. A low-priority rule with a broad\n  condition (`path-pattern: \u002F*`) shadows higher-priority rules. Always\n  list rules sorted by priority when diagnosing routing issues.\n\n- NEVER recommend deleting a target group or load balancer as\n  remediation. These resources have listener rules, Auto Scaling groups,\n  and DNS records that depend on them. Remediation is always to fix the\n  targets, health check, or security group — not to remove the LB.\n\n- NEVER ignore the health check grace period for new deployments. New\n  targets start in `initial` state and must pass\n  `healthy_threshold_count` checks before becoming `healthy`. If all\n  old targets were deregistered simultaneously, the ALB has no healthy\n  targets during this window. Use a blue-green or rolling deployment.\n\n- NEVER assume an IP-type target in a peered VPC uses a referenced SG.\n  IP-type targets outside the VPC (peered VPC, on-prem) cannot use\n  referenced SG rules — they require CIDR-based rules. A target SG rule\n  referencing the ALB SG fails for a cross-VPC IP target.\n\n## Pre-flight safety checks (run before any state-changing CLI)\n\n- **MANDATORY CONFIRMATION GATE.** Before any state-changing operation\n  (`modify-target-group`, `modify-load-balancer-attributes`,\n  `authorize-security-group-ingress`, `modify-listener`,\n  `modify-rule`), emit and await operator approval. Do NOT execute the\n  CLI until the operator confirms.\n\n- **Read-only first.** Every probe in the diagnostic tree is read-only\n  (`describe-*`, `aws s3 ls\u002Fcp`, `get-metric-statistics`, `curl`). Do\n  not perform state-changing operations as diagnostic probes.\n\n- **Health check path changes affect ALL targets.** Modifying\n  `HealthCheckPath` causes all targets to be re-evaluated. If the new\n  path is wrong, ALL targets go `unhealthy` simultaneously, producing a\n  full outage. Test the new path on one target first.\n\n- **Idle timeout changes apply immediately** and affect all connections.\n  Raising the timeout holds connections longer (more memory on targets);\n  lowering it may cut off slow legitimate requests.\n\n- **Security group changes can affect multiple targets.** A target SG\n  rule change applies to all instances using that SG. Verify the scope\n  before modifying.\n\n- **Listener rule priority changes are live immediately.** Reordering\n  rules affects traffic routing instantly. Test in a staging environment\n  first.\n\n- **Access log enablement is non-disruptive** but requires a valid S3\n  bucket with the correct policy. Verify the bucket exists and has the\n  ELB write policy before enabling.\n\n## Remediation guidance\n\nPer-LAYER fix playbooks (TARGET_HEALTH_CHECK, TARGET_SG_BLOCKED, TARGET_NONE_HEALTHY, TARGET_TIMEOUT, TARGET_INVALID_RESPONSE, DEREGISTRATION_STUCK, WAF_BLOCKED, LISTENER_MISCONFIGURED, ESCALATE): [references\u002Ferror-handling.md](references\u002Ferror-handling.md).\n\n## Domain\n\nAWS CloudOps \u002F ELBv2 Load Balancer Diagnostics, Target Health Analysis,\nSecurity Group Verification, and Incident Diagnosis.\n\n## Recent AWS features (2024-2026)\n\nFull feature list with dates: [references\u002Fadvanced-patterns.md](references\u002Fadvanced-patterns.md).\n\n## References (load on demand)\n\n- [Diagnostic commands](references\u002Fdiagnostic-commands.md) — account-wide pre-flight gather-info gate, Step 1b access-log fetch, and every step's probe commands (2a-7)\n- [Worked examples](references\u002Fworked-examples.md) — 502 from target SG not allowing ALB SG, 504 from target exceeding idle timeout, NEED_MORE_INFO re-prompt\n- [Error handling](references\u002Ferror-handling.md) — per-LAYER remediation guidance with fix and verify commands\n- [Advanced patterns](references\u002Fadvanced-patterns.md) — diagnostic mindset, philosophy, Step 0 non-obvious behaviours, recent AWS features\n- [Target health reference](references\u002Ftarget-health-reference.md) — error-code catalog, reason codes, access-log fields, SG evaluation, metrics\n\n## AWS documentation\n\n- **Elastic Load Balancing User Guide** — https:\u002F\u002Fdocs.aws.amazon.com\u002Felasticloadbalancing\u002Flatest\u002Fapplication\u002Fintroduction.html\n- **Target groups** — https:\u002F\u002Fdocs.aws.amazon.com\u002Felasticloadbalancing\u002Flatest\u002Fapplication\u002Fload-balancer-target-groups.html\n- **Target health for Application Load Balancers** — https:\u002F\u002Fdocs.aws.amazon.com\u002Felasticloadbalancing\u002Flatest\u002Fapplication\u002Ftarget-group-health-checks.html\n- **Access logs for Application Load Balancers** — https:\u002F\u002Fdocs.aws.amazon.com\u002Felasticloadbalancing\u002Flatest\u002Fapplication\u002Fload-balancer-access-logs.html\n- **ALB troubleshooting** — https:\u002F\u002Fdocs.aws.amazon.com\u002Felasticloadbalancing\u002Flatest\u002Fapplication\u002Fts-elb-error-codes.html\n- **ELB Security** — https:\u002F\u002Fdocs.aws.amazon.com\u002Felasticloadbalancing\u002Flatest\u002Fapplication\u002Fsecurity.html\n- **AWS CLI ELBv2 reference** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fcli\u002Flatest\u002Freference\u002Felbv2\u002F\n- **AWS Health** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fhealth\u002Flatest\u002Fug\u002F\n","# ALB 5xx Troubleshooter\n\n## Quick start\n\n- **Error code → layer map (first plausible match drives the first probe):**\n  502 → TARGET_INVALID_RESPONSE \u002F TARGET_SG_BLOCKED; 503 →\n  TARGET_NONE_HEALTHY \u002F TARGET_HEALTH_CHECK \u002F DEREGISTRATION_STUCK \u002F\n  TARGET_SG_BLOCKED; 504 → TARGET_TIMEOUT; 561 → WAF_BLOCKED;\n  500 → ALB_INTERNAL (rare — check AWS Health Dashboard).\n- **Always verify with a probe, never guess.** Each layer has a single\n  command (or access-log field) that proves or disproves it. A\n  ROOT_CAUSE_FOUND verdict requires positive evidence — a failing probe\n  that matches the symptom — not \"must be the targets.\"\n- **Target health is the root of most 5xx.** A 503 with no healthy\n  targets, a 502 from an unreachable target, and a 504 from a slow\n  target all originate in the target group. Probe `describe-target-health`\n  first — it is the single highest-signal command.\n- **ESCALATE for AWS-side incidents.** A sustained 500 with healthy\n  targets, no recent change, and clean configs is an AWS-side event —\n  surface the Health Dashboard link and open a Support case.\n\n## Mindset\n\nFull diagnostic mindset (the ALB is the messenger, not the cause): [references\u002Fadvanced-patterns.md](references\u002Fadvanced-patterns.md).\n\n## Philosophy\n\nThe four senior-engineer behaviours (5xx code tells WHERE, health check config is the top root cause, SG rules are bidirectional, access logs are the forensic trail): [references\u002Fadvanced-patterns.md](references\u002Fadvanced-patterns.md).\n\n## Quick navigation\n\n| If the symptom is... | Go to | First probe |\n|---|---|---|\n| 502 with `Target.InvalidResponse` or `Target.ConnectionFailed` | Step 2a | `describe-target-health` + direct curl on target |\n| 502 with target SG not allowing ALB SG | Step 2b | `describe-security-groups` on target SG |\n| 503 with all targets `unhealthy` | Step 3a | `describe-target-health` + health check config |\n| 503 with all targets `draining` | Step 3c | `deregistration_delay.timeout_seconds` |\n| 503 with zero registered targets | Step 3b | `describe-target-groups` TargetType |\n| 504 with `Target.Timeout` in access logs | Step 4 | ALB idle timeout + target response time |\n| 561 from WAF | Step 5 | WAF Web ACL logs |\n| 502\u002F503 from wrong target group on listener | Step 6 | `describe-rules` for listener priorities |\n| 500 sustained, healthy targets, no change | Step 7 | AWS Health Dashboard — ESCALATE |\n| Need the access log format and fields | Reference | `references\u002Ftarget-health-reference.md` |\n| Need full probes for any Step | Reference | `references\u002Fdiagnostic-commands.md` |\n| Need more worked examples | Reference | `references\u002Fworked-examples.md` |\n| Need per-layer fix commands | Reference | `references\u002Ferror-handling.md` |\n| Need Step 0 gotchas in full | Reference | `references\u002Fadvanced-patterns.md` |\n\n## Pre-flight: load balancer type and gather-info gate\n\nBefore running code-specific probes, gather the canonical load balancer,\nlistener, target group, and target health metadata. Misidentifying the\nload balancer type (ALB vs NLB) or the target type (instance vs ip vs\nlambda) produces false root causes.\n\n### Account-wide pre-flight commands\n\nAccount-wide pre-flight commands (describe-load-balancers, -listeners, -rules, -target-groups, -target-health, attributes, SGs, CloudWatch 5xx metrics, AWS Health): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n### Load-balancer-type short-circuit\n\n| Attribute | Value | Effect on diagnosis |\n|---|---|---|\n| `Type` | `application` | **ALB.** L7. Full 5xx diagnostic applies. Access logs include `target_status_code`, `target_processing_time`, `error_reason`. WAF can be associated (561 errors). Health checks are HTTP\u002FHTTPS. |\n| `Type` | `network` | **NLB.** L4. Access logs do NOT include `error_reason` or `target_status_code` (connection-level only). No WAF (no 561). Health checks can be TCP (misleading — a TCP handshake succeeds even when the app is broken). Cross-zone off by default. |\n| `Scheme` | `internet-facing` | ALB has public IPs. SG allowing 0.0.0.0\u002F0 on ports 80\u002F443 is expected. |\n| `Scheme` | `internal` | ALB has private IPs. SG allowing 0.0.0.0\u002F0 on any port is suspicious. |\n| `State` | `failed` | ALB is in a failed provisioning state. Output ERROR — do not classify. |\n\n### Target-type short-circuit\n\n| `TargetType` | Effect on diagnosis |\n|---|---|\n| `instance` | Target registered by EC2 instance ID. Uses the instance's primary private IP. Target SG is the instance's SG. |\n| `ip` | Target registered by IP address. Can be outside the VPC (peered VPC, on-prem via Direct Connect). SG rules must use CIDR (not referenced SG) for non-VPC IPs. |\n| `alb` | Target is another ALB (nested). Health check is on the parent ALB. Rare — usually a misconfiguration. |\n| `lambda` | Target is a Lambda function (ALB-only). No health check — Lambda invocations either succeed or fail. 502 from a Lambda target is a function error (similar to API Gateway Lambda proxy). |\n\nIf the input is malformed (missing load balancer ARN, missing target\ngroup ARN, ambiguous error code), emit:\n\nNEED_MORE_INFO re-prompt template for malformed\u002Fmissing input: [references\u002Fworked-examples.md](references\u002Fworked-examples.md).\n\n## Process — Diagnostic decision tree (apply in error-code order)\n\nThe diagnostic tree is error-code-driven. Pick the entry point based on\nthe observed 5xx code, then walk the layer-specific probes in order. Each\nlayer ends with either a positive root-cause confirmation (failing probe\nthat matches the symptom) or a pass that moves to the next layer. **Never\nemit ROOT_CAUSE_FOUND without a failing probe that matches the symptom.**\n\n### Step 0: Non-obvious behaviours that change diagnosis\n\nThe 13 non-obvious behaviours (shallow \u002Fhealth checks, NLB TCP false-healthy, ALB-SG-vs-IP rules, draining window, S3 log delivery, error_reason semantics, idle-timeout scope, NLB cross-zone default, rule priority order, instance-ID registration, WAF 561, Lambda 502, grace period, AZ reachability): [references\u002Fadvanced-patterns.md](references\u002Fadvanced-patterns.md).\n\n### Step 1: Symptom entry — pick the diagnostic branch\n\nMap the 5xx error code to a branch and jump to that branch's section.\n\n| Error code | Meaning | Branch |\n|---|---|---|\n| **502 BadGateway** | Target returned invalid response or connection failed | Step 2 |\n| **503 ServiceUnavailable** | No healthy targets available | Step 3 |\n| **504 GatewayTimeout** | Target did not respond within idle timeout | Step 4 |\n| **561 Unauthorized** | WAF blocked the request | Step 5 |\n| **500 InternalServerError** | ALB internal failure (rare) | Step 7 |\n| Ambiguous \u002F intermittent \u002F mixed codes | Gather access logs first | Step 1b |\n\n### Step 1b: Gather access logs (when the code is ambiguous)\n\nAccess-log location (access_logs.s3.bucket), fetch\u002Fparse commands, and the space-delimited field format (target_processing_time, target_status_code, error_reason): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n### Step 2: 502 BadGateway — target returned invalid response\n\nSymptom: client receives `502 BadGateway`. The ALB reached the target\n(or tried to) but received no valid HTTP response.\n\n#### 2a: Target returned invalid HTTP or connection failed\n\nProbes (describe-target-health, direct curl on target, access-log error_reason): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n**Verdict signals:**\n- Access log `error_reason: Target.InvalidResponse` → target returned\n  malformed HTTP (non-HTTP bytes, truncated response). **ROOT_CAUSE_FOUND**,\n  `LAYER: TARGET_INVALID_RESPONSE`. Fix: the target application.\n- Access log `error_reason: Target.ConnectionFailed` → connection to\n  target refused or reset. If targets are `healthy`, the target port\n  closed between health check and request. **ROOT_CAUSE_FOUND**,\n  `LAYER: TARGET_INVALID_RESPONSE`.\n- Target is `unhealthy` with `Target.FailedHealthChecks` → the target\n  is failing health checks; the ALB has no healthy target for this\n  request. Move to Step 3a (503 diagnosis — same root cause, different\n  manifestation).\n\n#### 2b: Target security group does not allow ALB SG\n\nProbes (target SG inbound rules vs ALB SG on target port): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n**Verdict signals:**\n- Target SG has NO inbound rule allowing the ALB SG on the target port\n  → **ROOT_CAUSE_FOUND**, `LAYER: TARGET_SG_BLOCKED`. The ALB cannot\n  reach the target. Fix: add an inbound rule to the target SG allowing\n  the ALB SG on the target port.\n- Target SG allows a specific IP\u002FCIDR but NOT the ALB SG → if the\n  allowed IP is a stale ALB IP, the rule is dead. **ROOT_CAUSE_FOUND**,\n  `LAYER: TARGET_SG_BLOCKED`. Fix: replace with an ALB-SG-referenced rule.\n- Target SG allows the ALB SG on the correct port → SG is not the\n  cause. Move to Step 2a (target application issue).\n\n### Step 3: 503 ServiceUnavailable — no healthy targets\n\nSymptom: client receives `503 ServiceUnavailable`. The ALB has no\nhealthy target in the target group to receive the request.\n\n#### 3a: All targets unhealthy (health check failing)\n\nProbes (target health reason, HealthCheckConfig fields, direct curl of the health path): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n**Verdict signals:**\n- Target health check path returns non-200 (e.g., 404, 500) → the\n  health check endpoint itself is broken. **ROOT_CAUSE_FOUND**,\n  `LAYER: TARGET_HEALTH_CHECK`. Fix: update the health check path to an\n  endpoint that returns 200 when healthy, or fix the endpoint.\n- `Matcher.HttpCode` does not include the status code the target returns\n  (e.g., matcher is `200` but target returns `204` on the health path)\n  → **ROOT_CAUSE_FOUND**, `LAYER: TARGET_HEALTH_CHECK`. Fix: update the\n  matcher to include the expected code(s) (e.g., `200,204`).\n- Health check timeout too short (`HealthCheckTimeoutSeconds` \u003C target\n  response time) → health check fails before the target responds.\n  **ROOT_CAUSE_FOUND**, `LAYER: TARGET_HEALTH_CHECK`. Fix: increase the\n  timeout.\n- Health check on wrong port (`HealthCheckPort` != application port)\n  → health check hits a port with no listener. **ROOT_CAUSE_FOUND**,\n  `LAYER: TARGET_HEALTH_CHECK`.\n\n#### 3b: Zero registered targets\n\nProbes (TargetType + registered target list): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n**Verdict signals:**\n- `TargetHealthDescriptions` is empty → no targets registered.\n  **ROOT_CAUSE_FOUND**, `LAYER: TARGET_NONE_HEALTHY`. Fix: register\n  targets (`register-targets`). This is common for newly created target\n  groups or after an Auto Scaling group scaled to zero.\n- All targets in `unused` state → targets are registered but the target\n  group is not attached to any listener rule. **ROOT_CAUSE_FOUND**,\n  `LAYER: LISTENER_MISCONFIGURED`. Fix: attach the target group to a\n  listener rule.\n\n#### 3c: All targets in draining state\n\nProbes (draining state + deregistration_delay.timeout_seconds): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n**Verdict signals:**\n- All targets `State: draining` with a long deregistration delay →\n  targets were deregistered but are still in the draining window.\n  **ROOT_CAUSE_FOUND**, `LAYER: DEREGISTRATION_STUCK`. Fix: register\n  new targets, or reduce the deregistration delay if the draining\n  window is too long for the deployment pattern.\n\n### Step 4: 504 GatewayTimeout — target did not respond in time\n\nSymptom: client receives `504 GatewayTimeout`. The target did not\nrespond within the ALB idle timeout.\n\nProbes (idle_timeout.timeout_seconds, access-log target_processing_time, TargetResponseTime metric, direct timed curl): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n**Verdict signals:**\n- `idle_timeout.timeout_seconds` is 60 (default), target response time\n  > 60s → **ROOT_CAUSE_FOUND**, `LAYER: TARGET_TIMEOUT`. Fix: raise the\n  idle timeout (if the workload legitimately needs it) or optimize the\n  target application.\n- Access log `error_reason: Target.Timeout` → confirms the target did\n  not respond within the idle timeout. **ROOT_CAUSE_FOUND**,\n  `LAYER: TARGET_TIMEOUT`.\n- `TargetResponseTime` metric Maximum approaches the idle timeout → the\n  target is slow. **ROOT_CAUSE_FOUND**, `LAYER: TARGET_TIMEOUT`.\n\n### Step 5: 561 Unauthorized — WAF blocked the request\n\nSymptom: client receives `561 Unauthorized`. This is a custom error from\nWAF blocking the request.\n\nProbes (get-web-acl-for-resource, list-web-acls, WAF BLOCK log filter): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n**Verdict signals:**\n- WAF Web ACL is associated with the ALB, and WAF logs show BLOCK actions\n  for the failing requests → **ROOT_CAUSE_FOUND**, `LAYER: WAF_BLOCKED`.\n  Fix: review the WAF rules — a false positive blocking legitimate\n  traffic needs the rule adjusted (add an IP exemption, tune the rate\n  limit, update the regex pattern).\n\n### Step 6: Listener rule misconfiguration\n\nSymptom: the \"wrong\" targets are receiving traffic, or a specific path\nreturns 503 while the root path works. This is a routing issue, not a\ntarget health issue.\n\nProbes (describe-rules priority order, listener DefaultActions): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n**Verdict signals:**\n- A high-priority rule (low number) matches all paths (`path-pattern: \u002F*`)\n  and forwards to the wrong target group → **ROOT_CAUSE_FOUND**,\n  `LAYER: LISTENER_MISCONFIGURED`. Fix: adjust rule priorities and\n  conditions.\n- Default action points to a deleted or empty target group →\n  **ROOT_CAUSE_FOUND**, `LAYER: LISTENER_MISCONFIGURED`. Fix: update the\n  default action.\n- A rule condition mismatches the incoming request (e.g., host-header\n  condition does not match) → requests fall through to the default action.\n  **ROOT_CAUSE_FOUND**, `LAYER: LISTENER_MISCONFIGURED`.\n\n### Step 7: 500 InternalServerError — rare ALB internal failure\n\nSymptom: client receives `500 InternalServerError`. This is rare and\nusually indicates an AWS-side issue.\n\nProbe (aws health describe-events for ELB): [references\u002Fdiagnostic-commands.md](references\u002Fdiagnostic-commands.md).\n\n**Verdict signals:**\n- 500 sustained, all targets healthy, no recent change → **ESCALATE**,\n  `LAYER: ALB_INTERNAL`. Surface the AWS Health event ARN and open a\n  Support case.\n\n### Step 8: Escalate or NEED_MORE_INFO\n\nIf none of the above produced a positive root-cause match, OR the\nsymptom clearly indicates an AWS-side incident (region event, sustained\n500), emit one of:\n\n- **ESCALATE** — AWS-side incident. Surface the AWS Health event ARN,\n  the load balancer ARN, and the relevant access-log evidence. Recommend\n  opening a Support case.\n- **NEED_MORE_INFO** — A specific probe requires operator input. List\n  the missing pieces (access logs not enabled, target group ARN\n  unknown, health check endpoint unclear) and the next probe to run\n  once the info is available.\n\n## Output format\n\n```text\nTARGET: \u003Clb-arn>\u002F\u003Ctarget-group-arn>\nVERDICT: ROOT_CAUSE_FOUND | NEED_MORE_INFO | ESCALATE\nREASON: \u003C1-2 sentences naming the failed layer and the failing probe>\nLAYER: \u003CTARGET_HEALTH_CHECK | TARGET_SG_BLOCKED | TARGET_NONE_HEALTHY |\n        TARGET_TIMEOUT | TARGET_INVALID_RESPONSE | LISTENER_MISCONFIGURED |\n        WAF_BLOCKED | DEREGISTRATION_STUCK | ALB_INTERNAL | UNKNOWN>\nEVIDENCE:\n  - \u003Cobserved symptom — error code, error_reason, target state>\n  - \u003Cfailing probe — command and its output that confirms the cause>\n  - \u003Cpassing probes — layers ruled out>\nREMEDIATION:\n  1. \u003Cspecific action with CLI command>\n  2. \u003Cverification command after the fix>\nCONFIRM: Before executing any state-changing CLI, emit and await operator\n  approval: \"CONFIRM: About to \u003Caction> on \u003Clb-arn>\u002F\u003Ctg-arn>. Proceed?\n  (yes\u002Fno)\"\n```\n\n### Worked example — 503 from wrong health check path\n\n```text\nTARGET: arn:aws:elasticloadbalancing:us-east-1:111:load-balancer\u002Fapp\u002Fprod-web\u002Fabc \u002F target-group\u002Ftg-web\u002Fdef\nVERDICT: ROOT_CAUSE_FOUND\nREASON: The target group health check path \u002Fhealth returns 404 on all\n  targets — the application serves health on \u002Fhealthz. All targets are\n  marked unhealthy, and the ALB has no healthy target for incoming\n  requests (Step 3a).\nLAYER: TARGET_HEALTH_CHECK\nEVIDENCE:\n  - Symptom: clients receive 503 ServiceUnavailable on all paths. ALB\n    access logs show target_group_arn for tg-web but no\n    target_status_code (no target was selected).\n  - Probe: aws elbv2 describe-target-health --target-group-arn \u003Ctg-web>\n    returns all targets State: unhealthy, Reason:\n    Target.FailedHealthChecks.\n  - Probe: aws elbv2 describe-target-groups --target-group-arns \u003Ctg-web>\n    returns HealthCheckPath: \u002Fhealth, Matcher.HttpCode: 200.\n  - Probe: ssh \u003Cbastion> \"curl -v http:\u002F\u002F\u003Ctarget-ip>:8080\u002Fhealth\"\n    returns HTTP\u002F1.1 404 Not Found. \"curl http:\u002F\u002F\u003Ctarget-ip>:8080\u002Fhealthz\"\n    returns HTTP\u002F1.1 200 OK.\n  - Passing: target SG allows ALB SG on port 8080; ALB idle timeout is\n    60s; listener default action points to tg-web; no WAF associated.\nREMEDIATION:\n  1. Update the health check path to \u002Fhealthz:\n     aws elbv2 modify-target-group --target-group-arn \u003Ctg-web>\n       --health-check-path \u002Fhealthz --profile \u003Cp>\n  2. Wait for healthy_threshold_count consecutive successful checks\n     (default: 3-5 checks at HealthCheckIntervalSeconds).\n  3. Verify: describe-target-health shows State: healthy for all\n     targets; clients no longer see 503.\nCONFIRM: Before modifying the target group, emit and await:\n  \"CONFIRM: About to modify-target-group on tg-web (health-check-path\n   \u002Fhealth → \u002Fhealthz). Proceed? (yes\u002Fno)\"\n```\n\n### Worked example — 502 from target SG not allowing ALB SG\n\nFull worked example (502, healthy targets, SG missing tcp\u002F443 from ALB SG; LAYER: TARGET_SG_BLOCKED): [references\u002Fworked-examples.md](references\u002Fworked-examples.md).\n\n### Worked example — 504 from target exceeding idle timeout\n\nFull worked example (504 at exactly 60s, Target.Timeout; LAYER: TARGET_TIMEOUT): [references\u002Fworked-examples.md](references\u002Fworked-examples.md).\n\n## Anti-Patterns — NEVER\n\n- NEVER declare ROOT_CAUSE_FOUND without a failing probe that matches the\n  symptom. \"Must be the targets\" without running `describe-target-health`\n  or checking access logs erodes operator trust.\n\n- NEVER confuse 502 and 503. A 502 means the ALB reached the target but\n  got an invalid response (or connection failed). A 503 means no healthy\n  target was available (the ALB never sent the request to a target).\n  Probing target health for a 502 when all targets are healthy wastes\n  the incident window — the target application is the cause, not target\n  selection.\n\n- NEVER assume a `healthy` target is actually serving the application\n  correctly. A health check on `\u002Fhealth` returning 200 does NOT mean\n  the application works on its real endpoints. A target can be `healthy`\n  while returning 500 to clients — the ALB routes to it, and clients\n  see the 500. Always test the actual endpoint, not just the health\n  check path.\n\n- NEVER trust an NLB `healthy` status without verifying\n  `health_check.type`. The default TCP health check succeeds on a bare\n  handshake — an app returning 500s is marked `healthy`. For HTTP\u002FHTTPS\n  targets behind an NLB, require `health_check.type = HTTP\u002FHTTPS` with\n  a real `path`. A TCP check on an HTTP target is the most common cause\n  of \"healthy but broken\" NLB incidents.\n\n- NEVER configure a target SG rule with the ALB's IP address. ALB IPs\n  change when the ALB scales. Use the ALB security group ID as the\n  source — this is stable across scaling events.\n\n- NEVER assume access logs are delivered just because\n  `access_logs.s3.enabled` is `true`. The S3 bucket policy MUST grant\n  `elasticloadbalancing.amazonaws.com` write access with\n  `aws:SourceAccount` condition. A misconfigured bucket policy causes\n  logs to silently fail. Verify with `aws s3 ls` on the prefix after\n  5-10 minutes.\n\n- NEVER recommend raising the ALB idle timeout without verifying the\n  target can actually finish within the new timeout. A target that takes\n  300s will still 504 with a 120s timeout. For long-running requests,\n  migrate to an async pattern (202 + polling).\n\n- NEVER forget that ALB cross-zone load balancing is ALWAYS ON. It cannot\n  be disabled. Do not flag an ALB for cross-zone issues. Only NLB has\n  the toggle (off by default).\n\n- NEVER treat 561 as a backend error. 561 is a WAF block — the request\n  never reached the target. Check the WAF Web ACL, not the target health.\n\n- NEVER conflate deregistration delay with unhealthy targets. Targets in\n  `draining` state are NOT unhealthy — they were deliberately deregistered\n  and are serving in-flight requests. The fix is to register new targets,\n  not to \"fix\" the draining targets.\n\n- NEVER assume the listener default action points to the right target\n  group. A misconfigured default action (pointing to an empty or deleted\n  target group) produces 503 for any request that does not match a\n  specific rule. Always verify `DefaultActions` in `describe-listeners`.\n\n- NEVER evaluate listener rules out of priority order. Rules are\n  evaluated lowest-number-first. A low-priority rule with a broad\n  condition (`path-pattern: \u002F*`) shadows higher-priority rules. Always\n  list rules sorted by priority when diagnosing routing issues.\n\n- NEVER recommend deleting a target group or load balancer as\n  remediation. These resources have listener rules, Auto Scaling groups,\n  and DNS records that depend on them. Remediation is always to fix the\n  targets, health check, or security group — not to remove the LB.\n\n- NEVER ignore the health check grace period for new deployments. New\n  targets start in `initial` state and must pass\n  `healthy_threshold_count` checks before becoming `healthy`. If all\n  old targets were deregistered simultaneously, the ALB has no healthy\n  targets during this window. Use a blue-green or rolling deployment.\n\n- NEVER assume an IP-type target in a peered VPC uses a referenced SG.\n  IP-type targets outside the VPC (peered VPC, on-prem) cannot use\n  referenced SG rules — they require CIDR-based rules. A target SG rule\n  referencing the ALB SG fails for a cross-VPC IP target.\n\n## Pre-flight safety checks (run before any state-changing CLI)\n\n- **MANDATORY CONFIRMATION GATE.** Before any state-changing operation\n  (`modify-target-group`, `modify-load-balancer-attributes`,\n  `authorize-security-group-ingress`, `modify-listener`,\n  `modify-rule`), emit and await operator approval. Do NOT execute the\n  CLI until the operator confirms.\n\n- **Read-only first.** Every probe in the diagnostic tree is read-only\n  (`describe-*`, `aws s3 ls\u002Fcp`, `get-metric-statistics`, `curl`). Do\n  not perform state-changing operations as diagnostic probes.\n\n- **Health check path changes affect ALL targets.** Modifying\n  `HealthCheckPath` causes all targets to be re-evaluated. If the new\n  path is wrong, ALL targets go `unhealthy` simultaneously, producing a\n  full outage. Test the new path on one target first.\n\n- **Idle timeout changes apply immediately** and affect all connections.\n  Raising the timeout holds connections longer (more memory on targets);\n  lowering it may cut off slow legitimate requests.\n\n- **Security group changes can affect multiple targets.** A target SG\n  rule change applies to all instances using that SG. Verify the scope\n  before modifying.\n\n- **Listener rule priority changes are live immediately.** Reordering\n  rules affects traffic routing instantly. Test in a staging environment\n  first.\n\n- **Access log enablement is non-disruptive** but requires a valid S3\n  bucket with the correct policy. Verify the bucket exists and has the\n  ELB write policy before enabling.\n\n## Remediation guidance\n\nPer-LAYER fix playbooks (TARGET_HEALTH_CHECK, TARGET_SG_BLOCKED, TARGET_NONE_HEALTHY, TARGET_TIMEOUT, TARGET_INVALID_RESPONSE, DEREGISTRATION_STUCK, WAF_BLOCKED, LISTENER_MISCONFIGURED, ESCALATE): [references\u002Ferror-handling.md](references\u002Ferror-handling.md).\n\n## Domain\n\nAWS CloudOps \u002F ELBv2 Load Balancer Diagnostics, Target Health Analysis,\nSecurity Group Verification, and Incident Diagnosis.\n\n## Recent AWS features (2024-2026)\n\nFull feature list with dates: [references\u002Fadvanced-patterns.md](references\u002Fadvanced-patterns.md).\n\n## References (load on demand)\n\n- [Diagnostic commands](references\u002Fdiagnostic-commands.md) — account-wide pre-flight gather-info gate, Step 1b access-log fetch, and every step's probe commands (2a-7)\n- [Worked examples](references\u002Fworked-examples.md) — 502 from target SG not allowing ALB SG, 504 from target exceeding idle timeout, NEED_MORE_INFO re-prompt\n- [Error handling](references\u002Ferror-handling.md) — per-LAYER remediation guidance with fix and verify commands\n- [Advanced patterns](references\u002Fadvanced-patterns.md) — diagnostic mindset, philosophy, Step 0 non-obvious behaviours, recent AWS features\n- [Target health reference](references\u002Ftarget-health-reference.md) — error-code catalog, reason codes, access-log fields, SG evaluation, metrics\n\n## AWS documentation\n\n- **Elastic Load Balancing User Guide** — https:\u002F\u002Fdocs.aws.amazon.com\u002Felasticloadbalancing\u002Flatest\u002Fapplication\u002Fintroduction.html\n- **Target groups** — https:\u002F\u002Fdocs.aws.amazon.com\u002Felasticloadbalancing\u002Flatest\u002Fapplication\u002Fload-balancer-target-groups.html\n- **Target health for Application Load Balancers** — https:\u002F\u002Fdocs.aws.amazon.com\u002Felasticloadbalancing\u002Flatest\u002Fapplication\u002Ftarget-group-health-checks.html\n- **Access logs for Application Load Balancers** — https:\u002F\u002Fdocs.aws.amazon.com\u002Felasticloadbalancing\u002Flatest\u002Fapplication\u002Fload-balancer-access-logs.html\n- **ALB troubleshooting** — https:\u002F\u002Fdocs.aws.amazon.com\u002Felasticloadbalancing\u002Flatest\u002Fapplication\u002Fts-elb-error-codes.html\n- **ELB Security** — https:\u002F\u002Fdocs.aws.amazon.com\u002Felasticloadbalancing\u002Flatest\u002Fapplication\u002Fsecurity.html\n- **AWS CLI ELBv2 reference** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fcli\u002Flatest\u002Freference\u002Felbv2\u002F\n- **AWS Health** — https:\u002F\u002Fdocs.aws.amazon.com\u002Fhealth\u002Flatest\u002Fug\u002F\n",{"name":5,"license":15,"metadata":24,"description":7,"compatibility":43},{"tags":25,"phase":26,"author":27,"domain":28,"family":29,"version":30,"keywords":31,"task_type":32,"complexity":33,"entry_point":34,"skill_class":35,"when_to_use":36,"requires_llm":37,"verdict_shape":38,"when_not_to_use":39,"lifecycle_status":40,"invocation_schema":41,"supports_pipeline":37,"activation_triggers":42},"elbv2, alb, nlb, load-balancer, networking, troubleshooting, 5xx, target-health","2","Jacky Chan — AWS Community Builder","aws-cloudops","Networking","0.1.0","ALB, NLB, ELBv2, load balancer, 5xx, 500, 502, 503, 504, 561, BadGateway, ServiceUnavailable, GatewayTimeout, target health, target group, health check, security group, idle timeout, deregistration, draining, WAF, listener rule, access logs, troubleshooting","troubleshoot","high","false","capability","Diagnosing an ALB or NLB 5xx error (500, 502, 503, 504, 561), walking a symptom to the failed target, security group, health check, or listener layer with verify and fix commands, validating target health failures, identifying unhealthy target groups, diagnosing idle timeout exceeded, or triaging a \"the load balancer returns 5xx\" page where the root cause may be target health, SG rules, health check configuration, deregistration, WAF, or listener rules — not necessarily the load balancer itself.","true","ROOT_CAUSE_FOUND | NEED_MORE_INFO | ESCALATE","Configuration posture audits (use elbv2-load-balancer-auditor for TLS policy, access-log enablement, deletion protection, SG exposure), TLS\u002Fcipher negotiation issues (use elbv2-load-balancer-auditor INSECURE_LISTENER), or capacity\u002Fthroughput sizing (use the optimize task type). This skill diagnoses runtime 5xx failures and target health, not config posture.","active","Input: either (a) a symptom description (5xx error code, observed pattern, failing target group), optionally paired with the load balancer and target group metadata (describe-load-balancers, describe-target-groups, describe-target-health output), OR (b) a load balancer ARN or target group ARN for live-account diagnosis. Output: a deterministic TARGET\u002FVERDICT\u002FREASON\u002FLAYER\u002FEVIDENCE\u002F REMEDIATION block where VERDICT ∈ {ROOT_CAUSE_FOUND, NEED_MORE_INFO, ESCALATE} and LAYER ∈ {TARGET_HEALTH_CHECK, TARGET_SG_BLOCKED, TARGET_NONE_HEALTHY, TARGET_TIMEOUT, TARGET_INVALID_RESPONSE, LISTENER_MISCONFIGURED, WAF_BLOCKED, DEREGISTRATION_STUCK, ALB_INTERNAL, UNKNOWN}.","ALB 5xx error, ALB 502 BadGateway, ALB 503 ServiceUnavailable, ALB 504 GatewayTimeout, ALB 561 Unauthorized, NLB 5xx, target unhealthy, no healthy targets, target group health check failed, ALB access logs 5xx, target security group blocked, deregistration draining stuck, WAF blocked request ALB, troubleshoot ALB, troubleshoot load balancer 5xx","Agent runtime that reads SKILL.md (Claude Code, Cursor, Windsurf, Codex, Gemini). Offline symptom classification works from pasted error codes and target group metadata. Live-account diagnosis uses aws elbv2 describe-target-health, describe-target-groups, describe-listeners, describe-rules, describe-load-balancers, describe-load-balancer-attributes, aws ec2 describe-security-groups, aws s3 cp (for ALB access logs), and aws cloudwatch get-metric-statistics (AWS CLI v2, SSO or key-based...","main","eeace51bd5dd1db2030729d83eeab988861553aa","skills\u002Falb-5xx-troubleshooter",0,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills","https:\u002F\u002Fgithub.com\u002Fjhkchan","CTO & Co-Founder @Beever-AI  & @votee. CAISP. AWS Community Builder. Organizer@GDGCloudHK. MongoDB Champion. Technopreneur + Infornographer.","2026-09-24 08:54:10",{"source":8,"skillName":5,"commands":53,"prompt":63,"zipAvailable":64},[54,57,60],{"tool":55,"command":56},"npx","npx skills add jhkchan\u002Faws-agent-skills --skill alb-5xx-troubleshooter",{"tool":58,"command":59},"bunx","bunx skills add jhkchan\u002Faws-agent-skills --skill alb-5xx-troubleshooter",{"tool":61,"command":62},"pnpm","pnpm dlx skills add jhkchan\u002Faws-agent-skills --skill alb-5xx-troubleshooter","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002FSKILL.md",true,[66,72,76,80,84,88,92,96,100,104,108,112,116,120,124,128,132,136,140,144],{"path":67,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":70,"sourceHtmlUrl":71},"eval\u002Ftest-cases.yaml","REFERENCE","参考文档",11228,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Feval\u002Ftest-cases.yaml",{"path":73,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":74,"sourceHtmlUrl":75},"evals\u002Fbaselines\u002F502-target-invalid-response.md",834,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Fevals\u002Fbaselines\u002F502-target-invalid-response.md",{"path":77,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":78,"sourceHtmlUrl":79},"evals\u002Fbaselines\u002F502-target-sg-blocks-alb.md",778,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Fevals\u002Fbaselines\u002F502-target-sg-blocks-alb.md",{"path":81,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":82,"sourceHtmlUrl":83},"evals\u002Fbaselines\u002F503-targets-draining-stuck.md",760,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Fevals\u002Fbaselines\u002F503-targets-draining-stuck.md",{"path":85,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":86,"sourceHtmlUrl":87},"evals\u002Fbaselines\u002F503-wrong-health-check-path.md",836,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Fevals\u002Fbaselines\u002F503-wrong-health-check-path.md",{"path":89,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":90,"sourceHtmlUrl":91},"evals\u002Fbaselines\u002F504-idle-timeout-exceeded.md",785,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Fevals\u002Fbaselines\u002F504-idle-timeout-exceeded.md",{"path":93,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":94,"sourceHtmlUrl":95},"evals\u002Feval_config.yaml",1961,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Fevals\u002Feval_config.yaml",{"path":97,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":98,"sourceHtmlUrl":99},"evals\u002Fevals.json",4872,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Fevals\u002Fevals.json",{"path":101,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":102,"sourceHtmlUrl":103},"evals\u002Fprompts\u002F502-target-invalid-response.md",1638,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Fevals\u002Fprompts\u002F502-target-invalid-response.md",{"path":105,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":106,"sourceHtmlUrl":107},"evals\u002Fprompts\u002F502-target-sg-blocks-alb.md",1398,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Fevals\u002Fprompts\u002F502-target-sg-blocks-alb.md",{"path":109,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":110,"sourceHtmlUrl":111},"evals\u002Fprompts\u002F503-targets-draining-stuck.md",1562,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Fevals\u002Fprompts\u002F503-targets-draining-stuck.md",{"path":113,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":114,"sourceHtmlUrl":115},"evals\u002Fprompts\u002F503-wrong-health-check-path.md",1378,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Fevals\u002Fprompts\u002F503-wrong-health-check-path.md",{"path":117,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":118,"sourceHtmlUrl":119},"evals\u002Fprompts\u002F504-idle-timeout-exceeded.md",1452,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Fevals\u002Fprompts\u002F504-idle-timeout-exceeded.md",{"path":121,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":122,"sourceHtmlUrl":123},"examples\u002FREADME.md",5910,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Fexamples\u002FREADME.md",{"path":125,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":126,"sourceHtmlUrl":127},"references\u002Fadvanced-patterns.md",10236,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Freferences\u002Fadvanced-patterns.md",{"path":129,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":130,"sourceHtmlUrl":131},"references\u002Fdiagnostic-commands.md",7835,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Freferences\u002Fdiagnostic-commands.md",{"path":133,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":134,"sourceHtmlUrl":135},"references\u002Ferror-handling.md",3175,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Freferences\u002Ferror-handling.md",{"path":137,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":138,"sourceHtmlUrl":139},"references\u002Ftarget-health-reference.md",12990,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Freferences\u002Ftarget-health-reference.md",{"path":141,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":142,"sourceHtmlUrl":143},"references\u002Fworked-examples.md",4460,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Freferences\u002Fworked-examples.md",{"path":145,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":146,"sourceHtmlUrl":147},"skill.usf.yaml",4581,"https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Falb-5xx-troubleshooter\u002Fskill.usf.yaml",[149,156,163,170,177,184],{"skillNo":150,"name":151,"nameZh":6,"description":152,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":153,"categories":154,"keywords":155,"publishedAt":19},"01M7ADVD60X4PCAPKSE58P9NDC","xray-tracing-deployer","Deploys AWS X-Ray distributed tracing with production-grade configuration: X-Ray daemon deployment (EC2 systemd, ECS sidecar, EKS DaemonSet, Lambda built-in), SDK instrumentation (Java, Python, Node.js, Go, .NET) with annotations and metadata, sampling rules (default reservoir + rate vs custom rules with URL \u002F method \u002F service-name predicates), service map generation, trace query and retrieval, groups and insights, CloudWatch ServiceLens integration, IAM permissions (PutTraceSegments, GetSamplingRules), and latest features (Lambda Powertools tracing, OpenTelemetry \u002F ADOT Collector support, W3C trace context). Emits a READY_TO_DEPLOY checklist with every configuration item verified. Use when enabling X-Ray on a new application, adding tracing to ECS \u002F EKS \u002F Lambda, validating a sampling configuration, or generating daemon deployment CLI and IaC templates. Triggers: X-Ray, distributed tracing, X-Ray daemon, sampling rules, service map, X-Ray SDK, OpenTelemetry, ADOT, Lambda Powertools, CloudWatch ServiceLens.","https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fxray-tracing-deployer\u002FSKILL.md",[],[],{"skillNo":157,"name":158,"nameZh":6,"description":159,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":160,"categories":161,"keywords":162,"publishedAt":19},"01M7ADTVJ7DCBM9CB1FY51GH7J","wellarchitected-workload-auditor","Audits AWS Well-Architected Tool workloads for review staleness, per-pillar high-risk issue counts, milestone tracking gaps, and remediation plan completeness. Emits a deterministic verdict (STALE_REVIEW | HIGH_RISK | CONFIG_GAP | OK) per workload with per-pillar risk breakdown, milestone trend analysis, and specific remediation. Use when reviewing Well-Architected workloads, checking WA review freshness, auditing high-risk issues per pillar, validating milestone coverage, or tracking remediation plans.","https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fwellarchitected-workload-auditor\u002FSKILL.md",[],[],{"skillNo":164,"name":165,"nameZh":6,"description":166,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":167,"categories":168,"keywords":169,"publishedAt":19},"01M7ADT59G6FTTNZYHT2QRPNNP","wellarchitected-review-operator","Operates AWS Well-Architected Tool reviews end-to-end — creates workloads, runs pillar reviews (operational excellence, security, reliability, performance, cost optimization, sustainability, Prosperity), answers questions with risk-tier rationale, generates reports and improvement plans, creates milestones, and integrates with Trusted Advisor and Well-Architected Labs. Runs deterministic pre-checks (workload existence, lens availability, pillar coverage), emits the exact update-answer \u002F get-consolidated-report CLI behind a CONFIRM gate, and verifies risk-tier transitions. Emits READY | BLOCKED | COMPLETED. Use when creating a workload, answering pillar questions, generating an improvement plan, creating a milestone, or integrating with Trusted Advisor. Triggers: well-architected tool, well architected review, workload review, pillar, operational excellence, security, reliability, performance, cost optimization, sustainability, prosperity, improvement plan, milestone, trusted advisor.","https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fwellarchitected-review-operator\u002FSKILL.md",[],[],{"skillNo":171,"name":172,"nameZh":6,"description":173,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":174,"categories":175,"keywords":176,"publishedAt":19},"01M7ADSH5JA9QHNPPHQCD7EVA7","wafv2-web-acl-deployer","Provisions AWS WAFv2 Web ACLs with correct production defaults: scope selection (CLOUDFRONT vs REGIONAL), managed rule groups (AWSManagedRulesCommonRuleSet, SQLiRuleSet, LinuxRuleSet, WindowsRuleSet, IPReputationList, BotControlRuleSet, AmazonIpReputationList, ATP), custom rules (byte match, IP set, regex pattern, geo match, size constraint, rate-based), rule priority ordering, logging destinations (Kinesis Firehose, CloudWatch Logs, S3), resource association (ALB, API Gateway, CloudFront), rate-based rules with aggregate keys (IP, forwarded IP), Challenge and CAPTCHA actions, and account takeover prevention. Emits a READY_TO_DEPLOY checklist. Use when creating a Web ACL, attaching managed rules, configuring rate limiting, enabling WAF logging, associating a Web ACL with an ALB\u002FAPI Gateway\u002FCloudFront, or configuring CAPTCHA\u002FChallenge actions. Triggers: create WAF Web ACL, WAFv2, managed rule groups, WAF rate limiting, WAF logging, WAF CAPTCHA, WAF Challenge action, bot control, ATP managed rules.","https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fwafv2-web-acl-deployer\u002FSKILL.md",[],[],{"skillNo":178,"name":179,"nameZh":6,"description":180,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":181,"categories":182,"keywords":183,"publishedAt":19},"01M7ADRYFAT8R73FBNCHRC573X","wafv2-web-acl-auditor","Audits AWS WAFv2 Web ACL configurations to determine whether the ACL provides effective protection: default-action posture (Allow vs Block), managed-rule-group coverage gaps, rule effectiveness (BLOCK vs COUNT, shadow\u002Fbypass rules, stale exclusions), rate-based rule correctness, logging and visibility configuration, and text-transformation bypass vectors. Emits a deterministic verdict per Web ACL. Use when reviewing a WAFv2 Web ACL for security posture, checking managed-rule coverage, validating rate-limit configuration, auditing WAF logging, or hardening a WAF before production deployment. Triggers: WAF, WAFv2, Web ACL, managed rule group, AWSManagedRules, rate-based rule, CAPTCHA, Challenge, OverrideAction, Count mode, forwarded IP, Bot Control, ATP, text transformation, WAF logging, WAF visibility, OWASP, SQLi, XSS, RFI.","https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fwafv2-web-acl-auditor\u002FSKILL.md",[],[],{"skillNo":185,"name":186,"nameZh":6,"description":187,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":188,"categories":189,"keywords":190,"publishedAt":19},"01M7ADR8W7G72JX4KFGN03QK8Q","waf-rule-deployer","Provisions AWS WAFv2 rule sets with production defaults: Web ACL creation (CloudFront scope in us-east-1 vs Regional scope), rule groups (managed vs custom) with capacity planning in WCU (Web ACL Capacity Units, 1500 default per ACL), managed rule groups (AWSManagedRulesCommonRuleSet, AWSManagedRulesSQLiRuleSet, AWSManagedRulesAmazonIpReputationList, AWSManagedRulesLinuxRuleSet, AWSManagedRulesWindowsRuleSet, AWSManagedRulesUnixRuleSet), custom rules (byte-match, regex pattern set, geo-match, rate-based), IP set and regex pattern set resources, rule priority ordering (lowest number evaluated first), action types (allow, block, count, CAPTCHA, Challenge), bot control managed rule group, account takeover. Triggers: create WAF rule, WAF managed rule group, WAF custom rule, WAF byte match, WAF geo match, WAF rate based rule, WAF IP set, WAF regex pattern set, WAF WCU budget, WAF CloudFront ACL, WAF logging Kinesis Firehose, WAF bot control, WAF ATP, WAF label matching, WAF Challenge action, WAF CAPTCHA.","https:\u002F\u002Fgithub.com\u002Fjhkchan\u002Faws-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fwaf-rule-deployer\u002FSKILL.md",[],[]]