[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M77PAQDSKZXMP0PSC3ZGA682":3,"skill-same:01M77PAQDSKZXMP0PSC3ZGA682":47},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":25,"commitSha":26,"skillDirectory":27,"forksCount":28,"repositoryHtmlUrl":29,"repositoryHomepageUrl":6,"ownerHtmlUrl":30,"ownerBlogUrl":31,"ownerDescription":6,"updatedAt":32,"install":33,"resources":46},"01M77PAQDSKZXMP0PSC3ZGA682","aws-cloudwatch-investigation",null,"Reusable investigation patterns for AWS CloudWatch: Logs Insights query templates, alarm-to-deployment correlation, blast-radius narrowing decision tree, and PromQL-style metric query patterns for structured incident triage.","Reihantt6\u002Fsns-myagent","Reihantt6","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F219822095?v=4","01M77KK94PV8B7C2D9XWXNS8DJ","Mohamad Reihan tegar prasetya","01M77KK958QFMTHYXGXFV7KZ79",2,"MIT","https:\u002F\u002Fgithub.com\u002FReihantt6\u002Fsns-myagent\u002Fblob\u002Fmain\u002Fagent\u002Fskills\u002Faws-cloudwatch-investigation\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\ndescription: \"Reusable investigation patterns for AWS CloudWatch: Logs Insights query templates, alarm-to-deployment correlation, blast-radius narrowing decision tree, and PromQL-style metric query patterns for structured incident triage.\\n\"\n---\n# AWS CloudWatch Investigation Skill\n\nReusable patterns for investigating production incidents using CloudWatch Logs, Metrics, and Alarms. These patterns are designed to be composed together during incident triage.\n\n---\n\n## Pattern 1: Logs Insights Query Templates\n\n### Error Spike Detection\n\nFind the top errors in a time window, grouped by error type:\n\n```\nfields @timestamp, @message, @logStream\n| filter @message like \u002F(?i)(error|exception|fatal|critical)\u002F\n| stats count(*) as errorCount by bin(5m), @logStream\n| sort errorCount desc\n| limit 20\n```\n\n### P99 Latency Breakdown by Operation\n\nIdentify which operations are driving latency spikes:\n\n```\nfields @timestamp, @duration, operation\n| filter ispresent(@duration)\n| stats avg(@duration) as avgMs,\n        pct(@duration, 50) as p50Ms,\n        pct(@duration, 95) as p95Ms,\n        pct(@duration, 99) as p99Ms,\n        count(*) as invocations\n  by operation\n| sort p99Ms desc\n| limit 15\n```\n\n### Lambda Cold Start Detection\n\nQuantify cold start impact during an incident:\n\n```\nfields @timestamp, @duration, @initDuration, @memorySize, @maxMemoryUsed\n| filter ispresent(@initDuration)\n| stats count(*) as coldStarts,\n        avg(@initDuration) as avgInitMs,\n        max(@initDuration) as maxInitMs,\n        avg(@duration) as avgDurationMs\n  by bin(5m)\n| sort @timestamp desc\n```\n\n### Out-of-Memory (OOM) Detection\n\nFind Lambda functions or containers killed by memory pressure:\n\n```\nfields @timestamp, @message, @logStream, @memorySize, @maxMemoryUsed\n| filter @message like \u002FRuntime exited|out of memory|OOMKilled|Cannot allocate memory|MemoryError\u002F\n| stats count(*) as oomEvents by @logStream, bin(10m)\n| sort oomEvents desc\n| limit 10\n```\n\nFor memory utilization trending before OOM:\n\n```\nfields @timestamp, @maxMemoryUsed, @memorySize\n| filter ispresent(@maxMemoryUsed)\n| stats max(@maxMemoryUsed \u002F @memorySize * 100) as peakMemPct,\n        avg(@maxMemoryUsed \u002F @memorySize * 100) as avgMemPct\n  by bin(5m)\n| sort @timestamp desc\n```\n\n### Timeout Detection\n\nFind invocations that hit the configured timeout:\n\n```\nfields @timestamp, @duration, @logStream, @requestId\n| filter @message like \u002FTask timed out\u002F or @duration > 28000\n| stats count(*) as timeouts by @logStream, bin(5m)\n| sort timeouts desc\n```\n\n---\n\n## Pattern 2: Alarm History to Deploy-Event Correlation\n\n### Process\n\n1. **Get alarm transition time** — note the exact timestamp when the alarm entered ALARM state.\n2. **Query CloudTrail** for deployment-related events in a window of [alarm_time - 30min, alarm_time]:\n\n```\n# CloudTrail Lake query for deployment events\nSELECT eventTime, eventName, userIdentity.arn, requestParameters\nFROM \u003Cevent-data-store-id>\nWHERE eventTime > '\u003Calarm_time_minus_30m>'\n  AND eventTime \u003C '\u003Calarm_time>'\n  AND eventName IN (\n    'UpdateFunctionCode', 'UpdateFunctionConfiguration',\n    'UpdateService', 'CreateDeployment', 'RegisterTaskDefinition',\n    'CreateChangeSet', 'ExecuteChangeSet',\n    'StartPipelineExecution', 'PutImage'\n  )\nORDER BY eventTime DESC\n```\n\n3. **Correlation criteria** — a deploy is \"correlated\" if:\n   - It targets the same service\u002Fresource as the alarm\n   - It completed within 15 minutes before the alarm transition\n   - The deployer identity matches a CI\u002FCD role (not a human applying a hotfix)\n\n4. **Strengthening the correlation:**\n   - Check if the same alarm was healthy in the previous deployment cycle\n   - Verify no other environmental changes (scaling events, config changes) in the same window\n   - Look for canary\u002Fsynthetic monitor failures that started at the same time\n\n### Output Format\n\n```\nDeploy Correlation:\n  Event: UpdateFunctionCode\n  Time: 2024-03-15T14:23:07Z (12 min before alarm)\n  Actor: arn:aws:sts::123456789012:assumed-role\u002Fgithub-actions-deploy\u002Fsession\n  Resource: arn:aws:lambda:us-east-1:123456789012:function:payment-processor\n  Correlation: STRONG — same resource, CI\u002FCD actor, alarm was OK prior cycle\n```\n\n---\n\n## Pattern 3: Narrow the Blast Radius Decision Tree\n\nUse this tree to systematically scope an incident from broadest to most specific:\n\n```\nSTART\n  |\n  v\n[1] ACCOUNT — Which account(s) show the alarm?\n  |  - Check: Are alarms firing in multiple accounts?\n  |  - If yes → suspect shared service (SSO, networking, shared deployment pipeline)\n  |  - If no → proceed to Region\n  v\n[2] REGION — Which region(s) are affected?\n  |  - Check: Same alarm in other regions?\n  |  - If multi-region → suspect global service (IAM, Route53, S3 global)\n  |  - If single-region → proceed to Service\n  v\n[3] SERVICE — Which service namespace shows degradation?\n  |  - Check CloudWatch namespace: AWS\u002FLambda, AWS\u002FECS, AWS\u002FApiGateway, etc.\n  |  - If multiple services → suspect shared dependency (VPC, NAT, DNS, IAM)\n  |  - If single service → proceed to Operation\n  v\n[4] OPERATION — Which API action or function is failing?\n  |  - For Lambda: which function name?\n  |  - For ECS: which service\u002Ftask definition?\n  |  - For API GW: which stage\u002Fresource\u002Fmethod?\n  |  - If all operations → suspect service-level issue (throttling, quota)\n  |  - If specific operation → proceed to Resource\n  v\n[5] RESOURCE — Which specific resource instance?\n     - Function ARN, Task ID, DB instance identifier\n     - This is your investigation target\n     - Proceed to log and trace analysis scoped to this resource\n```\n\n### Shared Dependency Investigation\n\nWhen blast radius spans multiple services, investigate in this order:\n\n1. **VPC\u002FNetworking** — NAT Gateway ErrorPortAllocation, packet drops, DNS resolution failures\n2. **IAM\u002FSTS** — ThrottlingException on AssumeRole, token vending latency\n3. **Downstream dependency** — shared database, cache, or external API\n4. **Deployment pipeline** — simultaneous deploys across services from same pipeline run\n5. **AWS service event** — check AWS Health Dashboard and Service Health for the region\n\n---\n\n## Pattern 4: PromQL-Style Metric Query Patterns\n\nThese patterns use CloudWatch metric math and GetMetricData to build composite signals. Express them as metric queries for dashboards or programmatic retrieval.\n\n### Error Rate as Percentage\n\n```\nMetricDataQueries:\n  - Id: errors\n    MetricStat:\n      Metric:\n        Namespace: AWS\u002FLambda\n        MetricName: Errors\n        Dimensions: [{Name: FunctionName, Value: TARGET}]\n      Period: 60\n      Stat: Sum\n  - Id: invocations\n    MetricStat:\n      Metric:\n        Namespace: AWS\u002FLambda\n        MetricName: Invocations\n        Dimensions: [{Name: FunctionName, Value: TARGET}]\n      Period: 60\n      Stat: Sum\n  - Id: error_rate\n    Expression: \"errors \u002F invocations * 100\"\n    Label: \"Error Rate %\"\n```\n\n### Latency Anomaly Detection (Compare to Baseline)\n\n```\nMetricDataQueries:\n  - Id: current_p99\n    MetricStat:\n      Metric:\n        Namespace: AWS\u002FLambda\n        MetricName: Duration\n        Dimensions: [{Name: FunctionName, Value: TARGET}]\n      Period: 300\n      Stat: p99\n  - Id: baseline_p99\n    MetricStat:\n      Metric:\n        Namespace: AWS\u002FLambda\n        MetricName: Duration\n        Dimensions: [{Name: FunctionName, Value: TARGET}]\n      Period: 300\n      Stat: p99\n    # Use StartTime\u002FEndTime set to same window last week\n  - Id: anomaly_ratio\n    Expression: \"current_p99 \u002F baseline_p99\"\n    Label: \"Latency vs Baseline (ratio > 2 = anomaly)\"\n```\n\n### Throttling Pressure Score\n\nCombine multiple throttling signals into a single pressure metric:\n\n```\nMetricDataQueries:\n  - Id: lambda_throttles\n    MetricStat:\n      Metric: {Namespace: AWS\u002FLambda, MetricName: Throttles}\n      Period: 60\n      Stat: Sum\n  - Id: api_gw_429s\n    MetricStat:\n      Metric: {Namespace: AWS\u002FApiGateway, MetricName: 4XXError, Dimensions: [{Name: ApiName, Value: TARGET}]}\n      Period: 60\n      Stat: Sum\n  - Id: dynamo_throttles\n    MetricStat:\n      Metric: {Namespace: AWS\u002FDynamoDB, MetricName: ThrottledRequests, Dimensions: [{Name: TableName, Value: TARGET}]}\n      Period: 60\n      Stat: Sum\n  - Id: throttle_pressure\n    Expression: \"lambda_throttles + api_gw_429s + dynamo_throttles\"\n    Label: \"Combined Throttle Pressure\"\n```\n\n### Concurrent Execution Headroom\n\n```\nMetricDataQueries:\n  - Id: concurrent\n    MetricStat:\n      Metric: {Namespace: AWS\u002FLambda, MetricName: ConcurrentExecutions}\n      Period: 60\n      Stat: Maximum\n  - Id: headroom\n    Expression: \"1000 - concurrent\"\n    Label: \"Remaining Concurrency (account limit 1000)\"\n```\n\n---\n\n## Pattern 5: Incident Timeline Reconstruction\n\n### Process\n\nReconstruct a precise timeline by merging data from multiple sources:\n\n1. **Collect timestamps:**\n\n| Source | Query | Yields |\n|--------|-------|--------|\n| CloudWatch Alarms | Alarm history API | State transition times |\n| CloudWatch Metrics | GetMetricData with 1-min period | First anomaly point |\n| CloudWatch Logs | Logs Insights with `earliest(@timestamp)` | First error occurrence |\n| CloudTrail | LookupEvents filtered by time | Deployment\u002Fchange events |\n| AWS Health | DescribeEvents | AWS-side incidents |\n\n2. **Build the timeline:**\n\n```\nfields @timestamp, @message\n| filter @message like \u002FERROR|WARN|timeout|refused|denied\u002F\n| stats earliest(@timestamp) as firstSeen, latest(@timestamp) as lastSeen, count(*) as occurrences\n  by @message\n| sort firstSeen asc\n| limit 20\n```\n\n3. **Identify the sequence:**\n\n```\nTimeline:\n  T-15m: CloudTrail — UpdateFunctionCode by CI\u002FCD role\n  T-12m: Logs — first error \"Connection refused to payments-api.internal\"\n  T-10m: Metrics — Error count crosses 5\u002Fmin threshold\n  T-8m:  Alarm — PaymentProcessorErrors enters ALARM\n  T-5m:  Metrics — p99 latency spikes to 28s (timeout)\n  T-0:   Current — error rate at 45%, alarm still firing\n```\n\n4. **Determine root event** — the earliest change that preceded all symptoms. Walk backward from the first symptom to the most recent mutation (deploy, config change, scaling event, or external dependency shift).\n\n### Gotchas\n\n- CloudWatch metric timestamps are end-of-period. A 1-minute datapoint at 14:05 covers 14:04-14:05.\n- CloudTrail events can have up to 15-minute delivery delay. Use `eventTime`, not ingestion time.\n- Log group timestamps depend on the agent\u002FSDK flush interval. Allow for 30-60s of clock skew.\n- Alarm state changes have a built-in evaluation delay (periods x evaluation periods). The actual anomaly started earlier.\n","# AWS CloudWatch Investigation Skill\n\nReusable patterns for investigating production incidents using CloudWatch Logs, Metrics, and Alarms. These patterns are designed to be composed together during incident triage.\n\n---\n\n## Pattern 1: Logs Insights Query Templates\n\n### Error Spike Detection\n\nFind the top errors in a time window, grouped by error type:\n\n```\nfields @timestamp, @message, @logStream\n| filter @message like \u002F(?i)(error|exception|fatal|critical)\u002F\n| stats count(*) as errorCount by bin(5m), @logStream\n| sort errorCount desc\n| limit 20\n```\n\n### P99 Latency Breakdown by Operation\n\nIdentify which operations are driving latency spikes:\n\n```\nfields @timestamp, @duration, operation\n| filter ispresent(@duration)\n| stats avg(@duration) as avgMs,\n        pct(@duration, 50) as p50Ms,\n        pct(@duration, 95) as p95Ms,\n        pct(@duration, 99) as p99Ms,\n        count(*) as invocations\n  by operation\n| sort p99Ms desc\n| limit 15\n```\n\n### Lambda Cold Start Detection\n\nQuantify cold start impact during an incident:\n\n```\nfields @timestamp, @duration, @initDuration, @memorySize, @maxMemoryUsed\n| filter ispresent(@initDuration)\n| stats count(*) as coldStarts,\n        avg(@initDuration) as avgInitMs,\n        max(@initDuration) as maxInitMs,\n        avg(@duration) as avgDurationMs\n  by bin(5m)\n| sort @timestamp desc\n```\n\n### Out-of-Memory (OOM) Detection\n\nFind Lambda functions or containers killed by memory pressure:\n\n```\nfields @timestamp, @message, @logStream, @memorySize, @maxMemoryUsed\n| filter @message like \u002FRuntime exited|out of memory|OOMKilled|Cannot allocate memory|MemoryError\u002F\n| stats count(*) as oomEvents by @logStream, bin(10m)\n| sort oomEvents desc\n| limit 10\n```\n\nFor memory utilization trending before OOM:\n\n```\nfields @timestamp, @maxMemoryUsed, @memorySize\n| filter ispresent(@maxMemoryUsed)\n| stats max(@maxMemoryUsed \u002F @memorySize * 100) as peakMemPct,\n        avg(@maxMemoryUsed \u002F @memorySize * 100) as avgMemPct\n  by bin(5m)\n| sort @timestamp desc\n```\n\n### Timeout Detection\n\nFind invocations that hit the configured timeout:\n\n```\nfields @timestamp, @duration, @logStream, @requestId\n| filter @message like \u002FTask timed out\u002F or @duration > 28000\n| stats count(*) as timeouts by @logStream, bin(5m)\n| sort timeouts desc\n```\n\n---\n\n## Pattern 2: Alarm History to Deploy-Event Correlation\n\n### Process\n\n1. **Get alarm transition time** — note the exact timestamp when the alarm entered ALARM state.\n2. **Query CloudTrail** for deployment-related events in a window of [alarm_time - 30min, alarm_time]:\n\n```\n# CloudTrail Lake query for deployment events\nSELECT eventTime, eventName, userIdentity.arn, requestParameters\nFROM \u003Cevent-data-store-id>\nWHERE eventTime > '\u003Calarm_time_minus_30m>'\n  AND eventTime \u003C '\u003Calarm_time>'\n  AND eventName IN (\n    'UpdateFunctionCode', 'UpdateFunctionConfiguration',\n    'UpdateService', 'CreateDeployment', 'RegisterTaskDefinition',\n    'CreateChangeSet', 'ExecuteChangeSet',\n    'StartPipelineExecution', 'PutImage'\n  )\nORDER BY eventTime DESC\n```\n\n3. **Correlation criteria** — a deploy is \"correlated\" if:\n   - It targets the same service\u002Fresource as the alarm\n   - It completed within 15 minutes before the alarm transition\n   - The deployer identity matches a CI\u002FCD role (not a human applying a hotfix)\n\n4. **Strengthening the correlation:**\n   - Check if the same alarm was healthy in the previous deployment cycle\n   - Verify no other environmental changes (scaling events, config changes) in the same window\n   - Look for canary\u002Fsynthetic monitor failures that started at the same time\n\n### Output Format\n\n```\nDeploy Correlation:\n  Event: UpdateFunctionCode\n  Time: 2024-03-15T14:23:07Z (12 min before alarm)\n  Actor: arn:aws:sts::123456789012:assumed-role\u002Fgithub-actions-deploy\u002Fsession\n  Resource: arn:aws:lambda:us-east-1:123456789012:function:payment-processor\n  Correlation: STRONG — same resource, CI\u002FCD actor, alarm was OK prior cycle\n```\n\n---\n\n## Pattern 3: Narrow the Blast Radius Decision Tree\n\nUse this tree to systematically scope an incident from broadest to most specific:\n\n```\nSTART\n  |\n  v\n[1] ACCOUNT — Which account(s) show the alarm?\n  |  - Check: Are alarms firing in multiple accounts?\n  |  - If yes → suspect shared service (SSO, networking, shared deployment pipeline)\n  |  - If no → proceed to Region\n  v\n[2] REGION — Which region(s) are affected?\n  |  - Check: Same alarm in other regions?\n  |  - If multi-region → suspect global service (IAM, Route53, S3 global)\n  |  - If single-region → proceed to Service\n  v\n[3] SERVICE — Which service namespace shows degradation?\n  |  - Check CloudWatch namespace: AWS\u002FLambda, AWS\u002FECS, AWS\u002FApiGateway, etc.\n  |  - If multiple services → suspect shared dependency (VPC, NAT, DNS, IAM)\n  |  - If single service → proceed to Operation\n  v\n[4] OPERATION — Which API action or function is failing?\n  |  - For Lambda: which function name?\n  |  - For ECS: which service\u002Ftask definition?\n  |  - For API GW: which stage\u002Fresource\u002Fmethod?\n  |  - If all operations → suspect service-level issue (throttling, quota)\n  |  - If specific operation → proceed to Resource\n  v\n[5] RESOURCE — Which specific resource instance?\n     - Function ARN, Task ID, DB instance identifier\n     - This is your investigation target\n     - Proceed to log and trace analysis scoped to this resource\n```\n\n### Shared Dependency Investigation\n\nWhen blast radius spans multiple services, investigate in this order:\n\n1. **VPC\u002FNetworking** — NAT Gateway ErrorPortAllocation, packet drops, DNS resolution failures\n2. **IAM\u002FSTS** — ThrottlingException on AssumeRole, token vending latency\n3. **Downstream dependency** — shared database, cache, or external API\n4. **Deployment pipeline** — simultaneous deploys across services from same pipeline run\n5. **AWS service event** — check AWS Health Dashboard and Service Health for the region\n\n---\n\n## Pattern 4: PromQL-Style Metric Query Patterns\n\nThese patterns use CloudWatch metric math and GetMetricData to build composite signals. Express them as metric queries for dashboards or programmatic retrieval.\n\n### Error Rate as Percentage\n\n```\nMetricDataQueries:\n  - Id: errors\n    MetricStat:\n      Metric:\n        Namespace: AWS\u002FLambda\n        MetricName: Errors\n        Dimensions: [{Name: FunctionName, Value: TARGET}]\n      Period: 60\n      Stat: Sum\n  - Id: invocations\n    MetricStat:\n      Metric:\n        Namespace: AWS\u002FLambda\n        MetricName: Invocations\n        Dimensions: [{Name: FunctionName, Value: TARGET}]\n      Period: 60\n      Stat: Sum\n  - Id: error_rate\n    Expression: \"errors \u002F invocations * 100\"\n    Label: \"Error Rate %\"\n```\n\n### Latency Anomaly Detection (Compare to Baseline)\n\n```\nMetricDataQueries:\n  - Id: current_p99\n    MetricStat:\n      Metric:\n        Namespace: AWS\u002FLambda\n        MetricName: Duration\n        Dimensions: [{Name: FunctionName, Value: TARGET}]\n      Period: 300\n      Stat: p99\n  - Id: baseline_p99\n    MetricStat:\n      Metric:\n        Namespace: AWS\u002FLambda\n        MetricName: Duration\n        Dimensions: [{Name: FunctionName, Value: TARGET}]\n      Period: 300\n      Stat: p99\n    # Use StartTime\u002FEndTime set to same window last week\n  - Id: anomaly_ratio\n    Expression: \"current_p99 \u002F baseline_p99\"\n    Label: \"Latency vs Baseline (ratio > 2 = anomaly)\"\n```\n\n### Throttling Pressure Score\n\nCombine multiple throttling signals into a single pressure metric:\n\n```\nMetricDataQueries:\n  - Id: lambda_throttles\n    MetricStat:\n      Metric: {Namespace: AWS\u002FLambda, MetricName: Throttles}\n      Period: 60\n      Stat: Sum\n  - Id: api_gw_429s\n    MetricStat:\n      Metric: {Namespace: AWS\u002FApiGateway, MetricName: 4XXError, Dimensions: [{Name: ApiName, Value: TARGET}]}\n      Period: 60\n      Stat: Sum\n  - Id: dynamo_throttles\n    MetricStat:\n      Metric: {Namespace: AWS\u002FDynamoDB, MetricName: ThrottledRequests, Dimensions: [{Name: TableName, Value: TARGET}]}\n      Period: 60\n      Stat: Sum\n  - Id: throttle_pressure\n    Expression: \"lambda_throttles + api_gw_429s + dynamo_throttles\"\n    Label: \"Combined Throttle Pressure\"\n```\n\n### Concurrent Execution Headroom\n\n```\nMetricDataQueries:\n  - Id: concurrent\n    MetricStat:\n      Metric: {Namespace: AWS\u002FLambda, MetricName: ConcurrentExecutions}\n      Period: 60\n      Stat: Maximum\n  - Id: headroom\n    Expression: \"1000 - concurrent\"\n    Label: \"Remaining Concurrency (account limit 1000)\"\n```\n\n---\n\n## Pattern 5: Incident Timeline Reconstruction\n\n### Process\n\nReconstruct a precise timeline by merging data from multiple sources:\n\n1. **Collect timestamps:**\n\n| Source | Query | Yields |\n|--------|-------|--------|\n| CloudWatch Alarms | Alarm history API | State transition times |\n| CloudWatch Metrics | GetMetricData with 1-min period | First anomaly point |\n| CloudWatch Logs | Logs Insights with `earliest(@timestamp)` | First error occurrence |\n| CloudTrail | LookupEvents filtered by time | Deployment\u002Fchange events |\n| AWS Health | DescribeEvents | AWS-side incidents |\n\n2. **Build the timeline:**\n\n```\nfields @timestamp, @message\n| filter @message like \u002FERROR|WARN|timeout|refused|denied\u002F\n| stats earliest(@timestamp) as firstSeen, latest(@timestamp) as lastSeen, count(*) as occurrences\n  by @message\n| sort firstSeen asc\n| limit 20\n```\n\n3. **Identify the sequence:**\n\n```\nTimeline:\n  T-15m: CloudTrail — UpdateFunctionCode by CI\u002FCD role\n  T-12m: Logs — first error \"Connection refused to payments-api.internal\"\n  T-10m: Metrics — Error count crosses 5\u002Fmin threshold\n  T-8m:  Alarm — PaymentProcessorErrors enters ALARM\n  T-5m:  Metrics — p99 latency spikes to 28s (timeout)\n  T-0:   Current — error rate at 45%, alarm still firing\n```\n\n4. **Determine root event** — the earliest change that preceded all symptoms. Walk backward from the first symptom to the most recent mutation (deploy, config change, scaling event, or external dependency shift).\n\n### Gotchas\n\n- CloudWatch metric timestamps are end-of-period. A 1-minute datapoint at 14:05 covers 14:04-14:05.\n- CloudTrail events can have up to 15-minute delivery delay. Use `eventTime`, not ingestion time.\n- Log group timestamps depend on the agent\u002FSDK flush interval. Allow for 30-60s of clock skew.\n- Alarm state changes have a built-in evaluation delay (periods x evaluation periods). The actual anomaly started earlier.\n",{"description":24},"Reusable investigation patterns for AWS CloudWatch: Logs Insights query templates, alarm-to-deployment correlation, blast-radius narrowing decision tree, and PromQL-style metric query patterns for structured incident triage.\n","main","58843b159d7a4050d6b785a98b908cce505c0b92","agent\u002Fskills\u002Faws-cloudwatch-investigation",0,"https:\u002F\u002Fgithub.com\u002FReihantt6\u002Fsns-myagent","https:\u002F\u002Fgithub.com\u002FReihantt6","reihsns.web.id","2026-08-19 20:08:18",{"source":8,"skillName":5,"commands":34,"prompt":44,"zipAvailable":45},[35,38,41],{"tool":36,"command":37},"npx","npx skills add Reihantt6\u002Fsns-myagent --skill aws-cloudwatch-investigation",{"tool":39,"command":40},"bunx","bunx skills add Reihantt6\u002Fsns-myagent --skill aws-cloudwatch-investigation",{"tool":42,"command":43},"pnpm","pnpm dlx skills add Reihantt6\u002Fsns-myagent --skill aws-cloudwatch-investigation","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FReihantt6\u002Fsns-myagent\u002Fblob\u002Fmain\u002Fagent\u002Fskills\u002Faws-cloudwatch-investigation\u002FSKILL.md",true,[],[48,55,62,69,76,83],{"skillNo":49,"name":50,"nameZh":6,"description":51,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":52,"categories":53,"keywords":54,"publishedAt":19},"01M77XDCYZ407XYMF2T4P6F7VT","youtube-transcript","Extract transcripts from YouTube videos. Use when the user asks for a transcript, subtitles, or captions of a YouTube video and provides a YouTube URL (youtube.com\u002Fwatch?v=, youtu.be\u002F, or similar). Supports output with or without timestamps.","https:\u002F\u002Fgithub.com\u002FReihantt6\u002Fsns-myagent\u002Fblob\u002Fmain\u002Fagent\u002Fskills\u002Fyoutube-transcript\u002FSKILL.md",[],[],{"skillNo":56,"name":57,"nameZh":6,"description":58,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":59,"categories":60,"keywords":61,"publishedAt":19},"01M77XD7PBC98J6MXGJJ2TJ8DX","youtube-thumbnail-design","YouTube thumbnail design with specific dimensions, contrast rules, and mobile preview optimization. Covers safe zones, text placement, face expression psychology, and A\u002FB testing. Use for: YouTube thumbnails, video cover images, click-through optimization. Triggers: youtube thumbnail, thumbnail design, video thumbnail, click through rate, ctr optimization, youtube cover, video cover image, thumbnail maker, thumbnail tips, youtube design, video preview image","https:\u002F\u002Fgithub.com\u002FReihantt6\u002Fsns-myagent\u002Fblob\u002Fmain\u002Fagent\u002Fskills\u002Fyoutube-thumbnail-design\u002FSKILL.md",[],[],{"skillNo":63,"name":64,"nameZh":6,"description":65,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":66,"categories":67,"keywords":68,"publishedAt":19},"01M77XD2541C8HXJ6RHYZP4NGJ","youtube-short-maker","Make YouTube Shorts that hold attention with Pexo — vertical 9:16, fast-paced, hook in the first 3 seconds. Describe your topic and Pexo writes the script, generates the shots, picks the models, and assembles a finished Short with music and captions. Use for YouTube Shorts: \"youtube short\", \"make a youtube short\", \"shorts video\", \"short-form video\". NOT for long-form or landscape video.","https:\u002F\u002Fgithub.com\u002FReihantt6\u002Fsns-myagent\u002Fblob\u002Fmain\u002Fagent\u002Fskills\u002Fyoutube-short-maker\u002FSKILL.md",[],[],{"skillNo":70,"name":71,"nameZh":6,"description":72,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":73,"categories":74,"keywords":75,"publishedAt":19},"01M77XCXKVSYK7KTZ3PQBVVGFD","xcode-project-setup","Safely modifies Xcode projects (.pbxproj) to add Swift Packages and link files. Use this skill whenever an iOS project needs dependencies installed (e.g. Firebase, Alamofire).","https:\u002F\u002Fgithub.com\u002FReihantt6\u002Fsns-myagent\u002Fblob\u002Fmain\u002Fagent\u002Fskills\u002Fxcode-project-setup\u002FSKILL.md",[],[],{"skillNo":77,"name":78,"nameZh":6,"description":79,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":80,"categories":81,"keywords":82,"publishedAt":19},"01M77XCR1DV8ZQK2NHWKVY4EWX","x402","Set up Browser Use Cloud payments with x402 — pay per request from a crypto wallet (USDC on Base mainnet), no signup or API key. Two setups it works out up front — \"just use it\" (set up a wallet so you or Claude Code can run cloud browser tasks paid from the wallet — Claude writes and runs throwaway scripts, nothing touches your codebase) or \"build it in\" (install the SDK and write the key + code into your project). Walks through wallet setup, funding, .env, and a ~$1 test run. Use when the user asks about x402, pay-per-use, USDC payments, or wants Browser Use Cloud without an API key. For the free-tier signup (reverse-CAPTCHA → API key), use `browser-use cloud signup` or the `cloud` skill instead.","https:\u002F\u002Fgithub.com\u002FReihantt6\u002Fsns-myagent\u002Fblob\u002Fmain\u002Fagent\u002Fskills\u002Fx402\u002FSKILL.md",[],[],{"skillNo":84,"name":85,"nameZh":6,"description":86,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":87,"categories":88,"keywords":89,"publishedAt":19},"01M77XCMD3R3ZBZQQSW6E0793B","x-twitter-scraper","Build GitHub Copilot workflows with Xquik X API SDKs, REST endpoints, hosted Apify Actor runs, MCP tools, TweetClaw OpenClaw plugin installs, signed webhooks, tweet search, user lookup, follower exports, media actions, and agent automation.","https:\u002F\u002Fgithub.com\u002FReihantt6\u002Fsns-myagent\u002Fblob\u002Fmain\u002Fagent\u002Fskills\u002Fx-twitter-scraper\u002FSKILL.md",[],[]]