[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6F72KV2HTNAPDKR9E2M2M8X":3,"skill-same:01M6F72KV2HTNAPDKR9E2M2M8X":81},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":22,"publishedAt":44,"linkOnly":45,"markdownContent":46,"markdownBody":47,"frontMatter":48,"refName":60,"commitSha":61,"skillDirectory":62,"forksCount":63,"repositoryHtmlUrl":64,"repositoryHomepageUrl":6,"ownerHtmlUrl":65,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":66,"install":67,"resources":80},"01M6F72KV2HTNAPDKR9E2M2M8X","graceful-shutdown",null,"Implement graceful shutdown for servers and workers: drain connections, finish in-flight work, release resources, and exit cleanly on SIGTERM\u002FSIGINT.","boisenoise\u002Fskills-collections","boisenoise","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F193180?v=4","01M6CDBDQK6BDN55ZMA6B6PKBM","01M6CDBDVSJ3JPTBYN26FZ7TZK",12,"MIT","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-graceful-shutdown\u002FSKILL.md",[17],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000011","运维与部署","SCENARIO","使用场景",[23,24,25,26,27,5,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43],"graceful shutdown","shutdown","2026","safe","for","resources","signals","self","health-check","connections","exit","workers","finish","work","release","development","resilience","implement","kubernetes","docker","gemini","2026-11-10 10:12:30",false,"---\nname: graceful-shutdown\ndescription: \"Implement graceful shutdown for servers and workers: drain connections, finish in-flight work, release resources, and exit cleanly on SIGTERM\u002FSIGINT.\"\ncategory: development\nrisk: safe\nsource: self\nsource_type: self\ndate_added: \"2026-08-27\"\nauthor: Prajeeth-12\ntags: [graceful-shutdown, signals, SIGTERM, SIGINT, drain, health-check, kubernetes, docker, production, resilience]\ntools: [claude, cursor, codex, gemini]\nlicense: \"MIT\"\n---\n\n# Graceful Shutdown\n\n## Overview\n\nA skill for implementing graceful shutdown in servers, workers, and long-running processes. Ensures in-flight requests complete, background jobs finish or checkpoint, database connections close cleanly, and the process exits with a proper status code. Essential for zero-downtime deployments in container orchestrators (Kubernetes, ECS, Docker Compose) and bare-metal process managers (systemd, PM2).\n\n## When to Use This Skill\n\n- Use when building an HTTP server that must not drop active connections during deploys\n- Use when writing a background worker that processes jobs from a queue\n- Use when deploying to Kubernetes, Docker, or any environment that sends SIGTERM before killing\n- Use when the user says \"graceful shutdown\", \"drain connections\", \"handle SIGTERM\", \"zero downtime\", or \"don't kill active requests\"\n- Use when implementing health check endpoints (`\u002Fhealthz`, `\u002Freadyz`) for orchestrators\n\n## How It Works\n\n### Step 1: Register signal handlers early\n\nTrap `SIGTERM` (orchestrator shutdown) and `SIGINT` (Ctrl+C) at process startup. Set a flag so the application knows it is shutting down.\n\n```typescript\nlet isShuttingDown = false;\n\nfunction onShutdownSignal(signal: string): void {\n  if (isShuttingDown) return; \u002F\u002F prevent double-shutdown\n  isShuttingDown = true;\n  console.log(`Received ${signal}, starting graceful shutdown...`);\n  void shutdown().catch(() => { process.exitCode = 1; });\n}\n\nprocess.on(\"SIGTERM\", () => onShutdownSignal(\"SIGTERM\"));\nprocess.on(\"SIGINT\", () => onShutdownSignal(\"SIGINT\"));\n```\n\n### Step 2: Stop accepting new work\n\nImmediately stop the server from accepting new connections. For HTTP servers, call `server.close()`. For queue workers, stop polling for new jobs.\n\n```typescript\nasync function shutdown(): Promise\u003Cvoid> {\n  \u002F\u002F 1. Stop accepting new connections\n  server.close(() => {\n    console.log(\"Server closed — no new connections accepted\");\n  });\n\n  \u002F\u002F 2. Mark health check as not-ready so load balancers stop routing\n  \u002F\u002F    (readiness probe returns 503 from this point)\n}\n```\n\n### Step 3: Drain in-flight work with a deadline\n\nWait for active requests and background tasks to finish, but enforce a hard deadline so the process never hangs indefinitely.\n\n```typescript\nconst DRAIN_TIMEOUT_MS = 25_000; \u002F\u002F must be less than orchestrator's terminationGracePeriodSeconds\n\nasync function drainAndExit(): Promise\u003Cvoid> {\n  const deadline = setTimeout(() => {\n    console.error(\"Drain timeout reached — forcing exit\");\n    process.exit(1);\n  }, DRAIN_TIMEOUT_MS);\n  deadline.unref(); \u002F\u002F don't keep the event loop alive just for the timer\n\n  try {\n    \u002F\u002F Wait for active connections to finish\n    await waitForActiveConnections();\n\n    \u002F\u002F Flush buffered data (logs, metrics, queues)\n    await flushBuffers();\n\n    \u002F\u002F Close external resource handles\n    await closeResources();\n\n    console.log(\"Graceful shutdown complete\");\n    process.exit(0);\n  } catch (err) {\n    console.error(\"Error during shutdown:\", err);\n    process.exit(1);\n  }\n}\n```\n\n### Step 4: Implement readiness and liveness probes\n\nOrchestrators use these to decide whether to route traffic and whether to restart the container. Liveness proves the process is alive; readiness controls whether traffic is routed. While the listener is still available during a drain, keep liveness healthy and return 503 only from readiness. After the listener closes, new probes cannot connect, so do not promise that HTTP liveness remains reachable for the entire termination window.\n\n```typescript\nimport { createServer, IncomingMessage, ServerResponse } from \"node:http\";\n\nfunction handleHealthCheck(req: IncomingMessage, res: ServerResponse): void {\n  if (req.url === \"\u002Fhealthz\") {\n    \u002F\u002F Keep liveness distinct from readiness while the listener is available.\n    \u002F\u002F Drain-rejection middleware must not turn this endpoint into a 503.\n    res.writeHead(200).end(\"ok\");\n    return;\n  }\n\n  if (req.url === \"\u002Freadyz\") {\n    \u002F\u002F Readiness: 503 during shutdown so the load balancer stops routing.\n    if (isShuttingDown) {\n      res.writeHead(503).end(\"shutting down\");\n    } else {\n      res.writeHead(200).end(\"ready\");\n    }\n    return;\n  }\n}\n```\n\n### Step 5: Track active connections\n\nMaintain a count of in-flight requests so you know when draining is complete. Use a once guard covering both `finish` and `close` events so that client disconnects (aborted requests) correctly decrement the counter.\n\n```typescript\nlet activeConnections = 0;\nlet drainResolve: (() => void) | null = null;\n\nfunction trackRequest(res: ServerResponse): void {\n  activeConnections++;\n  let counted = true;\n  function release(): void {\n    if (!counted) return;\n    counted = false;\n    activeConnections--;\n    if (isShuttingDown && activeConnections === 0 && drainResolve) {\n      drainResolve();\n    }\n  }\n  res.on(\"finish\", release);\n  res.on(\"close\", release);\n}\n\nfunction waitForActiveConnections(): Promise\u003Cvoid> {\n  if (activeConnections === 0) return Promise.resolve();\n  return new Promise((resolve) => {\n    drainResolve = resolve;\n  });\n}\n```\n\n## Examples\n\n### Example 1: Express.js server with graceful shutdown\n\n```typescript\nimport express from \"express\";\nimport { createServer } from \"node:http\";\n\nconst app = express();\nconst server = createServer(app);\nlet isShuttingDown = false;\nlet activeRequests = 0;\nlet drainResolve: (() => void) | null = null;\n\n\u002F\u002F Health endpoints — registered BEFORE the drain-rejection middleware so it\n\u002F\u002F cannot turn liveness into a 503 while the listener is still available.\napp.get(\"\u002Fhealthz\", (_, res) => res.send(\"ok\"));\napp.get(\"\u002Freadyz\", (_, res) => {\n  res.status(isShuttingDown ? 503 : 200).send(isShuttingDown ? \"draining\" : \"ready\");\n});\n\n\u002F\u002F Track in-flight requests and reject new application work during drain.\napp.use((req, res, next) => {\n  if (isShuttingDown) {\n    res.setHeader(\"Connection\", \"close\");\n    res.status(503).json({ error: \"Server is shutting down\" });\n    return;\n  }\n\n  activeRequests++;\n  let counted = true;\n  function release(): void {\n    if (!counted) return;\n    counted = false;\n    activeRequests--;\n    if (isShuttingDown && activeRequests === 0 && drainResolve) {\n      drainResolve();\n    }\n  }\n  \u002F\u002F Listen for both finish (normal) and close (client abort) so the\n  \u002F\u002F counter always decrements. The once guard prevents double-decrement\n  \u002F\u002F when both events fire.\n  res.on(\"finish\", release);\n  res.on(\"close\", release);\n  next();\n});\n\n\u002F\u002F Application routes\napp.get(\"\u002Fapi\u002Fdata\", async (req, res) => {\n  const data = await fetchData();\n  res.json(data);\n});\n\n\u002F\u002F Graceful shutdown\nfunction shutdown(signal: string): void {\n  if (isShuttingDown) return;\n  isShuttingDown = true;\n  console.log(`${signal} received — draining ${activeRequests} active requests`);\n\n  server.close();\n\n  const forceExit = setTimeout(() => {\n    console.error(\"Forced exit — drain timeout exceeded\");\n    process.exit(1);\n  }, 25_000);\n  forceExit.unref();\n\n  if (activeRequests === 0) {\n    console.log(\"No active requests — exiting cleanly\");\n    process.exit(0);\n  }\n\n  drainResolve = () => {\n    console.log(\"All requests drained — exiting cleanly\");\n    process.exit(0);\n  };\n}\n\nprocess.on(\"SIGTERM\", () => shutdown(\"SIGTERM\"));\nprocess.on(\"SIGINT\", () => shutdown(\"SIGINT\"));\n\nserver.listen(3000, () => console.log(\"Server ready on :3000\"));\n```\n\n### Example 2: Python FastAPI under Uvicorn\n\nUvicorn owns SIGTERM handling and request draining. It stops accepting new connections, asks existing connections to shut down, waits for connections and tasks up to `--timeout-graceful-shutdown`, and only then sends the ASGI lifespan shutdown event. Do not replace its signal handler or wait for requests again inside `lifespan`; use that hook to release application resources after Uvicorn's drain.\n\n```python\nfrom contextlib import asynccontextmanager\nfrom fastapi import FastAPI\n\n\n@asynccontextmanager\nasync def lifespan(app: FastAPI):\n    app.state.db_pool = await open_database_pool()\n    try:\n        yield\n    finally:\n        # Uvicorn has already completed or timed out its request drain.\n        await app.state.db_pool.close()\n\n\napp = FastAPI(lifespan=lifespan)\n\n\n@app.get(\"\u002Fhealthz\")\nasync def healthz():\n    return {\"status\": \"ok\"}\n\n\n@app.get(\"\u002Freadyz\")\nasync def readyz():\n    return {\"status\": \"ready\"}\n```\n\nRun Uvicorn with a deadline shorter than the orchestrator's kill timeout:\n\n```bash\nuvicorn app:app --timeout-graceful-shutdown 25\n```\n\nIf readiness must turn 503 before SIGTERM, coordinate a separately secured and tested pre-stop drain signal plus a propagation delay. FastAPI's lifespan shutdown hook is too late for that transition because Uvicorn invokes it after request draining.\n\n### Example 3: Process-level worker with a bounded drain\n\n`JobQueue` is an application-owned adapter. Its poll must honor the timeout, and each job must be idempotent because a crash after work but before ack causes redelivery. This is a process, not a Node worker thread (process signals are not delivered to worker threads).\n\n```typescript\nlet stopping = false;\nfunction stop(): void { stopping = true; }\nprocess.once(\"SIGTERM\", stop);\nprocess.once(\"SIGINT\", stop);\n\nasync function processJobs(queue: JobQueue): Promise\u003Cvoid> {\n  let deadline: ReturnType\u003Ctypeof setTimeout> | undefined;\n  const startDeadline = () => {\n    if (deadline) return;\n    stopping = true;\n    deadline = setTimeout(() => process.exit(1), 25_000);\n  };\n  process.once(\"SIGTERM\", startDeadline);\n  process.once(\"SIGINT\", startDeadline);\n  try {\n    while (!stopping) {\n      const job = await queue.poll({ timeout: 5000 });\n      if (!job) continue;\n      if (stopping) break; \u002F\u002F do not start newly polled work; leave it unacked\n      await job.execute();\n      await queue.ack(job.id); \u002F\u002F only after successful work\n    }\n    await queue.close(); \u002F\u002F application adapter stops polling and closes handles\n    process.exitCode = 0;\n  } finally {\n    if (deadline) clearTimeout(deadline);\n    process.off(\"SIGTERM\", startDeadline);\n    process.off(\"SIGINT\", startDeadline);\n  }\n}\n\u002F\u002F Caller must await\u002Fcatch processJobs; on failure close resources and set exitCode=1.\n```\n\nFor checkpointing, design cooperative cancellation inside job.execute and await a confirmed checkpoint before releasing the job. Do not concurrently checkpoint mutable job state from a signal handler while execute\u002Fack still run.\n\n## Best Practices\n\n- Always set a drain timeout shorter than the orchestrator's kill timeout (`terminationGracePeriodSeconds` in Kubernetes defaults to 30s — use 25s for your drain)\n- Return `Connection: close` header on responses sent during draining so HTTP\u002F1.1 clients don't reuse the connection\n- Reject new requests with 503 during shutdown so load balancers learn faster\n- Unref your force-exit timer so it doesn't keep the event loop alive after all work is done\n- Flush async buffers (log transports, metric aggregators, write-ahead logs) before exiting\n- Use `process.exit(0)` for clean shutdown and `process.exit(1)` for timeout\u002Ferror so orchestrators can distinguish the two\n- Test shutdown behavior explicitly — simulate SIGTERM in integration tests and verify no requests are dropped\n\n## Common Pitfalls\n\n- **Problem:** Drain-rejection middleware turns both readiness and liveness into 503 while the HTTP listener is still available.\n  **Solution:** Register health routes before that middleware and flip only readiness. Once the listener closes, new probes may no longer connect; the shutdown deadline, not a promise of HTTP liveness, bounds termination.\n\n- **Problem:** Drain hangs until the force-exit timeout even though all clients have disconnected.\n  **Solution:** Track request completion with both `finish` and `close` events (Node.js) or equivalent. If a client aborts the connection, `finish` may never fire — `close` will. Use a once guard to prevent double-decrementing the counter.\n\n- **Problem:** A FastAPI application replaces Uvicorn's SIGTERM handler or waits for in-flight requests inside lifespan shutdown.\n  **Solution:** Let Uvicorn own signal handling, connection\u002Ftask draining, and `--timeout-graceful-shutdown`. Use lifespan shutdown for resource cleanup; it runs after Uvicorn's request-drain phase.\n\n- **Problem:** Kubernetes kills the pod before connections drain because `terminationGracePeriodSeconds` is too short.\n  **Solution:** Set it to at least drain timeout + 5s buffer. If your longest request takes 60s, use `terminationGracePeriodSeconds: 70` and drain timeout of 65s.\n\n- **Problem:** Load balancer keeps sending traffic after SIGTERM because readiness probe still returns 200.\n  **Solution:** Flip the readiness probe to 503 immediately on signal receipt — before starting to drain.\n\n- **Problem:** `server.close()` resolves instantly but connections remain open (keep-alive).\n  **Solution:** Track connections manually and destroy idle keep-alive sockets on shutdown. Active sockets with in-flight requests should drain normally.\n\n- **Problem:** Double shutdown from both SIGTERM and SIGINT (e.g., Docker sends SIGTERM then user hits Ctrl+C).\n  **Solution:** Guard with a `isShuttingDown` flag — ignore the second signal.\n\n- **Problem:** Deadlocked process never exits because drain waits forever.\n  **Solution:** Always have a hard force-exit timeout as the final backstop.\n\n## Kubernetes Configuration\n\n```yaml\napiVersion: apps\u002Fv1\nkind: Deployment\nspec:\n  template:\n    spec:\n      terminationGracePeriodSeconds: 30\n      containers:\n        - name: app\n          livenessProbe:\n            httpGet:\n              path: \u002Fhealthz\n              port: 3000\n            initialDelaySeconds: 5\n            periodSeconds: 10\n          readinessProbe:\n            httpGet:\n              path: \u002Freadyz\n              port: 3000\n            initialDelaySeconds: 2\n            periodSeconds: 5\n```\n\n## Limitations\n\n- This skill does not replace environment-specific validation, testing, or expert review.\n- WebSocket and SSE connections require application-level close frames before severing — `server.close()` alone won't gracefully end them.\n- In clustered\u002Fmulti-process setups, coordinate master and child shutdown explicitly; do not assume signals are forwarded. Worker threads need an explicit message\u002Fabort protocol.\n- Some cloud platforms (Heroku, Railway) send SIGTERM with very short grace periods (10-30s) — adjust drain timeouts accordingly.\n\n## Related Skills\n\n- `@api-rate-limit-handler` — Resilient retry and backoff for outbound requests\n- `@circuit-breaker` — When to stop retrying entirely and fail fast\n- `@error-handling` — Structured error handling patterns\n","# Graceful Shutdown\n\n## Overview\n\nA skill for implementing graceful shutdown in servers, workers, and long-running processes. Ensures in-flight requests complete, background jobs finish or checkpoint, database connections close cleanly, and the process exits with a proper status code. Essential for zero-downtime deployments in container orchestrators (Kubernetes, ECS, Docker Compose) and bare-metal process managers (systemd, PM2).\n\n## When to Use This Skill\n\n- Use when building an HTTP server that must not drop active connections during deploys\n- Use when writing a background worker that processes jobs from a queue\n- Use when deploying to Kubernetes, Docker, or any environment that sends SIGTERM before killing\n- Use when the user says \"graceful shutdown\", \"drain connections\", \"handle SIGTERM\", \"zero downtime\", or \"don't kill active requests\"\n- Use when implementing health check endpoints (`\u002Fhealthz`, `\u002Freadyz`) for orchestrators\n\n## How It Works\n\n### Step 1: Register signal handlers early\n\nTrap `SIGTERM` (orchestrator shutdown) and `SIGINT` (Ctrl+C) at process startup. Set a flag so the application knows it is shutting down.\n\n```typescript\nlet isShuttingDown = false;\n\nfunction onShutdownSignal(signal: string): void {\n  if (isShuttingDown) return; \u002F\u002F prevent double-shutdown\n  isShuttingDown = true;\n  console.log(`Received ${signal}, starting graceful shutdown...`);\n  void shutdown().catch(() => { process.exitCode = 1; });\n}\n\nprocess.on(\"SIGTERM\", () => onShutdownSignal(\"SIGTERM\"));\nprocess.on(\"SIGINT\", () => onShutdownSignal(\"SIGINT\"));\n```\n\n### Step 2: Stop accepting new work\n\nImmediately stop the server from accepting new connections. For HTTP servers, call `server.close()`. For queue workers, stop polling for new jobs.\n\n```typescript\nasync function shutdown(): Promise\u003Cvoid> {\n  \u002F\u002F 1. Stop accepting new connections\n  server.close(() => {\n    console.log(\"Server closed — no new connections accepted\");\n  });\n\n  \u002F\u002F 2. Mark health check as not-ready so load balancers stop routing\n  \u002F\u002F    (readiness probe returns 503 from this point)\n}\n```\n\n### Step 3: Drain in-flight work with a deadline\n\nWait for active requests and background tasks to finish, but enforce a hard deadline so the process never hangs indefinitely.\n\n```typescript\nconst DRAIN_TIMEOUT_MS = 25_000; \u002F\u002F must be less than orchestrator's terminationGracePeriodSeconds\n\nasync function drainAndExit(): Promise\u003Cvoid> {\n  const deadline = setTimeout(() => {\n    console.error(\"Drain timeout reached — forcing exit\");\n    process.exit(1);\n  }, DRAIN_TIMEOUT_MS);\n  deadline.unref(); \u002F\u002F don't keep the event loop alive just for the timer\n\n  try {\n    \u002F\u002F Wait for active connections to finish\n    await waitForActiveConnections();\n\n    \u002F\u002F Flush buffered data (logs, metrics, queues)\n    await flushBuffers();\n\n    \u002F\u002F Close external resource handles\n    await closeResources();\n\n    console.log(\"Graceful shutdown complete\");\n    process.exit(0);\n  } catch (err) {\n    console.error(\"Error during shutdown:\", err);\n    process.exit(1);\n  }\n}\n```\n\n### Step 4: Implement readiness and liveness probes\n\nOrchestrators use these to decide whether to route traffic and whether to restart the container. Liveness proves the process is alive; readiness controls whether traffic is routed. While the listener is still available during a drain, keep liveness healthy and return 503 only from readiness. After the listener closes, new probes cannot connect, so do not promise that HTTP liveness remains reachable for the entire termination window.\n\n```typescript\nimport { createServer, IncomingMessage, ServerResponse } from \"node:http\";\n\nfunction handleHealthCheck(req: IncomingMessage, res: ServerResponse): void {\n  if (req.url === \"\u002Fhealthz\") {\n    \u002F\u002F Keep liveness distinct from readiness while the listener is available.\n    \u002F\u002F Drain-rejection middleware must not turn this endpoint into a 503.\n    res.writeHead(200).end(\"ok\");\n    return;\n  }\n\n  if (req.url === \"\u002Freadyz\") {\n    \u002F\u002F Readiness: 503 during shutdown so the load balancer stops routing.\n    if (isShuttingDown) {\n      res.writeHead(503).end(\"shutting down\");\n    } else {\n      res.writeHead(200).end(\"ready\");\n    }\n    return;\n  }\n}\n```\n\n### Step 5: Track active connections\n\nMaintain a count of in-flight requests so you know when draining is complete. Use a once guard covering both `finish` and `close` events so that client disconnects (aborted requests) correctly decrement the counter.\n\n```typescript\nlet activeConnections = 0;\nlet drainResolve: (() => void) | null = null;\n\nfunction trackRequest(res: ServerResponse): void {\n  activeConnections++;\n  let counted = true;\n  function release(): void {\n    if (!counted) return;\n    counted = false;\n    activeConnections--;\n    if (isShuttingDown && activeConnections === 0 && drainResolve) {\n      drainResolve();\n    }\n  }\n  res.on(\"finish\", release);\n  res.on(\"close\", release);\n}\n\nfunction waitForActiveConnections(): Promise\u003Cvoid> {\n  if (activeConnections === 0) return Promise.resolve();\n  return new Promise((resolve) => {\n    drainResolve = resolve;\n  });\n}\n```\n\n## Examples\n\n### Example 1: Express.js server with graceful shutdown\n\n```typescript\nimport express from \"express\";\nimport { createServer } from \"node:http\";\n\nconst app = express();\nconst server = createServer(app);\nlet isShuttingDown = false;\nlet activeRequests = 0;\nlet drainResolve: (() => void) | null = null;\n\n\u002F\u002F Health endpoints — registered BEFORE the drain-rejection middleware so it\n\u002F\u002F cannot turn liveness into a 503 while the listener is still available.\napp.get(\"\u002Fhealthz\", (_, res) => res.send(\"ok\"));\napp.get(\"\u002Freadyz\", (_, res) => {\n  res.status(isShuttingDown ? 503 : 200).send(isShuttingDown ? \"draining\" : \"ready\");\n});\n\n\u002F\u002F Track in-flight requests and reject new application work during drain.\napp.use((req, res, next) => {\n  if (isShuttingDown) {\n    res.setHeader(\"Connection\", \"close\");\n    res.status(503).json({ error: \"Server is shutting down\" });\n    return;\n  }\n\n  activeRequests++;\n  let counted = true;\n  function release(): void {\n    if (!counted) return;\n    counted = false;\n    activeRequests--;\n    if (isShuttingDown && activeRequests === 0 && drainResolve) {\n      drainResolve();\n    }\n  }\n  \u002F\u002F Listen for both finish (normal) and close (client abort) so the\n  \u002F\u002F counter always decrements. The once guard prevents double-decrement\n  \u002F\u002F when both events fire.\n  res.on(\"finish\", release);\n  res.on(\"close\", release);\n  next();\n});\n\n\u002F\u002F Application routes\napp.get(\"\u002Fapi\u002Fdata\", async (req, res) => {\n  const data = await fetchData();\n  res.json(data);\n});\n\n\u002F\u002F Graceful shutdown\nfunction shutdown(signal: string): void {\n  if (isShuttingDown) return;\n  isShuttingDown = true;\n  console.log(`${signal} received — draining ${activeRequests} active requests`);\n\n  server.close();\n\n  const forceExit = setTimeout(() => {\n    console.error(\"Forced exit — drain timeout exceeded\");\n    process.exit(1);\n  }, 25_000);\n  forceExit.unref();\n\n  if (activeRequests === 0) {\n    console.log(\"No active requests — exiting cleanly\");\n    process.exit(0);\n  }\n\n  drainResolve = () => {\n    console.log(\"All requests drained — exiting cleanly\");\n    process.exit(0);\n  };\n}\n\nprocess.on(\"SIGTERM\", () => shutdown(\"SIGTERM\"));\nprocess.on(\"SIGINT\", () => shutdown(\"SIGINT\"));\n\nserver.listen(3000, () => console.log(\"Server ready on :3000\"));\n```\n\n### Example 2: Python FastAPI under Uvicorn\n\nUvicorn owns SIGTERM handling and request draining. It stops accepting new connections, asks existing connections to shut down, waits for connections and tasks up to `--timeout-graceful-shutdown`, and only then sends the ASGI lifespan shutdown event. Do not replace its signal handler or wait for requests again inside `lifespan`; use that hook to release application resources after Uvicorn's drain.\n\n```python\nfrom contextlib import asynccontextmanager\nfrom fastapi import FastAPI\n\n\n@asynccontextmanager\nasync def lifespan(app: FastAPI):\n    app.state.db_pool = await open_database_pool()\n    try:\n        yield\n    finally:\n        # Uvicorn has already completed or timed out its request drain.\n        await app.state.db_pool.close()\n\n\napp = FastAPI(lifespan=lifespan)\n\n\n@app.get(\"\u002Fhealthz\")\nasync def healthz():\n    return {\"status\": \"ok\"}\n\n\n@app.get(\"\u002Freadyz\")\nasync def readyz():\n    return {\"status\": \"ready\"}\n```\n\nRun Uvicorn with a deadline shorter than the orchestrator's kill timeout:\n\n```bash\nuvicorn app:app --timeout-graceful-shutdown 25\n```\n\nIf readiness must turn 503 before SIGTERM, coordinate a separately secured and tested pre-stop drain signal plus a propagation delay. FastAPI's lifespan shutdown hook is too late for that transition because Uvicorn invokes it after request draining.\n\n### Example 3: Process-level worker with a bounded drain\n\n`JobQueue` is an application-owned adapter. Its poll must honor the timeout, and each job must be idempotent because a crash after work but before ack causes redelivery. This is a process, not a Node worker thread (process signals are not delivered to worker threads).\n\n```typescript\nlet stopping = false;\nfunction stop(): void { stopping = true; }\nprocess.once(\"SIGTERM\", stop);\nprocess.once(\"SIGINT\", stop);\n\nasync function processJobs(queue: JobQueue): Promise\u003Cvoid> {\n  let deadline: ReturnType\u003Ctypeof setTimeout> | undefined;\n  const startDeadline = () => {\n    if (deadline) return;\n    stopping = true;\n    deadline = setTimeout(() => process.exit(1), 25_000);\n  };\n  process.once(\"SIGTERM\", startDeadline);\n  process.once(\"SIGINT\", startDeadline);\n  try {\n    while (!stopping) {\n      const job = await queue.poll({ timeout: 5000 });\n      if (!job) continue;\n      if (stopping) break; \u002F\u002F do not start newly polled work; leave it unacked\n      await job.execute();\n      await queue.ack(job.id); \u002F\u002F only after successful work\n    }\n    await queue.close(); \u002F\u002F application adapter stops polling and closes handles\n    process.exitCode = 0;\n  } finally {\n    if (deadline) clearTimeout(deadline);\n    process.off(\"SIGTERM\", startDeadline);\n    process.off(\"SIGINT\", startDeadline);\n  }\n}\n\u002F\u002F Caller must await\u002Fcatch processJobs; on failure close resources and set exitCode=1.\n```\n\nFor checkpointing, design cooperative cancellation inside job.execute and await a confirmed checkpoint before releasing the job. Do not concurrently checkpoint mutable job state from a signal handler while execute\u002Fack still run.\n\n## Best Practices\n\n- Always set a drain timeout shorter than the orchestrator's kill timeout (`terminationGracePeriodSeconds` in Kubernetes defaults to 30s — use 25s for your drain)\n- Return `Connection: close` header on responses sent during draining so HTTP\u002F1.1 clients don't reuse the connection\n- Reject new requests with 503 during shutdown so load balancers learn faster\n- Unref your force-exit timer so it doesn't keep the event loop alive after all work is done\n- Flush async buffers (log transports, metric aggregators, write-ahead logs) before exiting\n- Use `process.exit(0)` for clean shutdown and `process.exit(1)` for timeout\u002Ferror so orchestrators can distinguish the two\n- Test shutdown behavior explicitly — simulate SIGTERM in integration tests and verify no requests are dropped\n\n## Common Pitfalls\n\n- **Problem:** Drain-rejection middleware turns both readiness and liveness into 503 while the HTTP listener is still available.\n  **Solution:** Register health routes before that middleware and flip only readiness. Once the listener closes, new probes may no longer connect; the shutdown deadline, not a promise of HTTP liveness, bounds termination.\n\n- **Problem:** Drain hangs until the force-exit timeout even though all clients have disconnected.\n  **Solution:** Track request completion with both `finish` and `close` events (Node.js) or equivalent. If a client aborts the connection, `finish` may never fire — `close` will. Use a once guard to prevent double-decrementing the counter.\n\n- **Problem:** A FastAPI application replaces Uvicorn's SIGTERM handler or waits for in-flight requests inside lifespan shutdown.\n  **Solution:** Let Uvicorn own signal handling, connection\u002Ftask draining, and `--timeout-graceful-shutdown`. Use lifespan shutdown for resource cleanup; it runs after Uvicorn's request-drain phase.\n\n- **Problem:** Kubernetes kills the pod before connections drain because `terminationGracePeriodSeconds` is too short.\n  **Solution:** Set it to at least drain timeout + 5s buffer. If your longest request takes 60s, use `terminationGracePeriodSeconds: 70` and drain timeout of 65s.\n\n- **Problem:** Load balancer keeps sending traffic after SIGTERM because readiness probe still returns 200.\n  **Solution:** Flip the readiness probe to 503 immediately on signal receipt — before starting to drain.\n\n- **Problem:** `server.close()` resolves instantly but connections remain open (keep-alive).\n  **Solution:** Track connections manually and destroy idle keep-alive sockets on shutdown. Active sockets with in-flight requests should drain normally.\n\n- **Problem:** Double shutdown from both SIGTERM and SIGINT (e.g., Docker sends SIGTERM then user hits Ctrl+C).\n  **Solution:** Guard with a `isShuttingDown` flag — ignore the second signal.\n\n- **Problem:** Deadlocked process never exits because drain waits forever.\n  **Solution:** Always have a hard force-exit timeout as the final backstop.\n\n## Kubernetes Configuration\n\n```yaml\napiVersion: apps\u002Fv1\nkind: Deployment\nspec:\n  template:\n    spec:\n      terminationGracePeriodSeconds: 30\n      containers:\n        - name: app\n          livenessProbe:\n            httpGet:\n              path: \u002Fhealthz\n              port: 3000\n            initialDelaySeconds: 5\n            periodSeconds: 10\n          readinessProbe:\n            httpGet:\n              path: \u002Freadyz\n              port: 3000\n            initialDelaySeconds: 2\n            periodSeconds: 5\n```\n\n## Limitations\n\n- This skill does not replace environment-specific validation, testing, or expert review.\n- WebSocket and SSE connections require application-level close frames before severing — `server.close()` alone won't gracefully end them.\n- In clustered\u002Fmulti-process setups, coordinate master and child shutdown explicitly; do not assume signals are forwarded. Worker threads need an explicit message\u002Fabort protocol.\n- Some cloud platforms (Heroku, Railway) send SIGTERM with very short grace periods (10-30s) — adjust drain timeouts accordingly.\n\n## Related Skills\n\n- `@api-rate-limit-handler` — Resilient retry and backoff for outbound requests\n- `@circuit-breaker` — When to stop retrying entirely and fail fast\n- `@error-handling` — Structured error handling patterns\n",{"name":5,"risk":26,"tags":49,"tools":54,"author":58,"source":30,"license":14,"category":38,"date_added":59,"description":7,"source_type":30},[5,29,50,51,52,31,41,42,53,39],"SIGTERM","SIGINT","drain","production",[55,56,57,43],"claude","cursor","codex","Prajeeth-12","2026-08-27","main","7bf2279c8a65758319526a721d5c4b75683aec9c","skills\u002Fantigravity-graceful-shutdown",3,"https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections","https:\u002F\u002Fgithub.com\u002Fboisenoise","2026-11-12 14:21:24",{"source":8,"skillName":5,"commands":68,"prompt":78,"zipAvailable":79},[69,72,75],{"tool":70,"command":71},"npx","npx skills add boisenoise\u002Fskills-collections --skill graceful-shutdown",{"tool":73,"command":74},"bunx","bunx skills add boisenoise\u002Fskills-collections --skill graceful-shutdown",{"tool":76,"command":77},"pnpm","pnpm dlx skills add boisenoise\u002Fskills-collections --skill graceful-shutdown","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-graceful-shutdown\u002FSKILL.md",true,[],[82,110,136,161,180,203],{"skillNo":83,"name":84,"nameZh":6,"description":85,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":86,"licenseFileUrl":87,"sourceHtmlUrl":88,"categories":89,"keywords":90,"publishedAt":44},"01M6WNH92BHVFXHGPDZCPJ0AG0","web-access","Use Treg for public web or platform data only when existing search, browser and connected tools cannot cover the task, with explicit cost and data boundaries.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-web-access\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-web-access\u002FSKILL.md",[],[91,92,93,94,38,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,25],"web","access","web access","official","browser","search","data","Tools","read","fetch","only","youtube","free tools","platform","critical","free","cover","normal","when",{"skillNo":111,"name":112,"nameZh":6,"description":113,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":86,"licenseFileUrl":114,"sourceHtmlUrl":115,"categories":116,"keywords":120,"publishedAt":44},"01M6WNF9XAYAB2CZDDJBAKNM93","treg","Use the Treg CLI or MCP catalog to discover, price and call authorized external tools with team-scoped credentials and a user-approved budget.","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-treg\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-treg\u002FSKILL.md",[117],{"categoryNo":118,"categoryName":119,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000023","Skill与智能体开发",[112,121,122,123,94,38,124,98,125,126,127,128,129,130,131,105,132,133,134,135,25],"MCP","cli","github","license","Budget","credentials","team","catalog","use","call","discover","external tools","with","scoped","blob",{"skillNo":137,"name":138,"nameZh":6,"description":139,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":86,"licenseFileUrl":140,"sourceHtmlUrl":141,"categories":142,"keywords":148,"publishedAt":44},"01M6WN93MEAZ4E41DYAHJRPHQT","recruit","Turn a job description into an evidence-linked candidate shortlist through Treg, with job-related ranking and optional authorized business contact enrichment.","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-recruit\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-recruit\u002FSKILL.md",[143],{"categoryNo":144,"categoryName":145,"taxonomyType":146,"taxonomyTypeDesc":147},"01JBSEEDCAT000000000000038","人力资源","PROFESSION","适用职业",[138,123,94,124,149,150,151,152,153,154,105,155,156,157,133,27,158,159,160,135,112,25],"https","email","enrichment","business","evidence","phone","ranking","post","job description","person","candidate","reason",{"skillNo":162,"name":163,"nameZh":6,"description":164,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":86,"licenseFileUrl":165,"sourceHtmlUrl":166,"categories":167,"keywords":168,"publishedAt":44},"01M6WN8R5RXSMHM9B4D9FAFX42","rea-growth","Analyze a company’s public growth and content through Treg, distinguish observations from hypotheses, and produce an evidence-linked playbook within an approved budget.","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-rea-growth\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-rea-growth\u002FSKILL.md",[],[169,170,171,123,94,124,172,173,149,174,125,175,176,177,153,105,178,179,135,112,25],"growth","rea","rea growth","produce","content","playbook","marketing","analyze","public","observations","company",{"skillNo":181,"name":182,"nameZh":6,"description":183,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":86,"licenseFileUrl":184,"sourceHtmlUrl":185,"categories":186,"keywords":187,"publishedAt":44},"01M6WN2EVE3RQRGWAKBT4S7YT2","make-ugc","Create original AI UGC-style product videos through Treg: research hooks, approve a licensed presenter and voice, generate clips, add captions and report costs.","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-make-ugc\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-make-ugc\u002FSKILL.md",[],[188,189,123,190,94,191,124,192,193,194,195,196,105,197,198,199,200,201,202,135],"make","make ugc","research","create","Video","Captions","ads","short","hooks","head","style","niche","trending","talking-head","approve",{"skillNo":204,"name":205,"nameZh":6,"description":206,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":86,"licenseFileUrl":207,"sourceHtmlUrl":208,"categories":209,"keywords":210,"publishedAt":44},"01M6WN10ARAA63Y8K1YNGXJ2RM","lead-signals","Find and qualify public buying signals through Treg, rank fitting business accounts and optionally enrich authorized business contact information.","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-lead-signals\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-lead-signals\u002FSKILL.md",[],[29,211,212,123,94,124,213,214,152,215,216,105,217,218,219,220,221,222,223,135],"lead","lead signals","how","find","ready","as","how to","qualify","people","information","enrich","changes","funding"]