[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M73B3DNKYHB06Y25DREECPBA":3,"skill-same:01M73B3DNKYHB06Y25DREECPBA":47},{"skillNo":4,"name":5,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":18,"linkOnly":19,"markdownContent":20,"markdownBody":21,"frontMatter":22,"refName":23,"commitSha":24,"skillDirectory":25,"forksCount":26,"repositoryHtmlUrl":27,"repositoryHomepageUrl":28,"ownerHtmlUrl":29,"ownerBlogUrl":30,"ownerDescription":31,"updatedAt":32,"install":33,"resources":46},"01M73B3DNKYHB06Y25DREECPBA","disaggregated-prefill-decode",null,"Horonomos-laps\u002FYana-AI","Horonomos-laps","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F355299122?v=4","01M76B3QPXE88ET2SQPDPVATZT"," Horonomos-laps","01M72ZQRJPCYPSSWT4D00VJR2A",13,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fdisaggregated-prefill-decode\u002FSKILL.md",[],[],"2026-11-11 15:11:14",false,"---\nname: disaggregated-prefill-decode\ndescription: Disaggregated prefill\u002Fdecode — separate GPU pools for compute-bound prefill and memory-bound decode. KV cache transfer via NIXL (RDMA\u002FTCP). NVIDIA Dynamo (stack-above) vs llm-d (K8s-native). 6x DeepSeek-R1 on GB200+Dynamo. 30–40% cost savings on $2M+ inference budgets. Sources: rohitg00\u002Fai-engineering-from-scratch (Apache-2.0).\norigin: yana-ai — synthesized from rohitg00\u002Fai-engineering-from-scratch (Apache-2.0)\nlicense: Apache-2.0\nversion: 1.0.0\ncompatibility: yana-ai >= 1.3.54\n---\n\n# \u002Fdisaggregated-prefill-decode\n\n## When to Use\n\n- Inference spend >$500K\u002Fyear on colocated serving — disaggregation can save 30–40%\n- Mixed workloads with both long-prompt (prefill-heavy) and long-output (decode-heavy) requests\n- MoE models (DeepSeek-R1, Mixtral) where colocation wastes H100 compute on memory-bound decode\n- Already on Kubernetes and want CNCF-native orchestration (llm-d)\n\n## Do NOT use for\n\n- Short prompts + short outputs (\u003C512 tokens each) — NIXL transfer tax exceeds savings\n- Small-scale serving (\u003C10 GPUs) — orchestration overhead not justified\n- Development \u002F testing environments — complexity far outweighs benefit\n\n---\n\n## Why the bottlenecks differ\n\n```\nPrefill  — full forward pass over input prompt → matrix multiplications dominate\n           → COMPUTE-BOUND — H100 FP8 ~2,000 TFLOPS\n           → batch many tokens in one forward → high GPU utilization\n\nDecode   — one token per step, reading full weights each time\n           → MEMORY-BANDWIDTH-BOUND — HBM3 ~3 TB\u002Fs\n           → only amortizes at high concurrency\n\nColocation problem:\n  You buy H100 compute for decode (memory bound → compute wasted)\n  You buy H100 HBM for prefill (compute bound → memory wasted)\n  20–40% GPU time wasted on the wrong resource at typical workloads.\n```\n\n---\n\n## Architecture diagram\n\n```\n            ┌──────────────┐\n  Request → │    Router    │ ─────────────────────────┐\n            └──────┬───────┘                          │\n                   │ (prompt only)                    │\n                   ▼                                  │\n            ┌──────────────┐   KV cache via NIXL   ┌──▼───────────┐\n            │ Prefill pool │ ──────────────────────► │ Decode pool  │\n            │ (compute)    │                         │ (memory)     │\n            └──────────────┘                         └──────┬───────┘\n                                                            │ tokens\n                                                            ▼\n                                                         Client\n\nNIXL: NVIDIA inter-node transport\n  - Uses RDMA\u002FInfiniBand when available\n  - TCP fallback otherwise\n  - Typical KV transfer latency: 20–80 ms for 4K-token prompt on 70B FP8\n  → Short prompts (\u003C512 tokens): transfer tax > compute savings → don't disaggregate\n```\n\n---\n\n## NVIDIA Dynamo (GTC 2025 GA)\n\n```bash\n# Dynamo sits ABOVE vLLM \u002F SGLang \u002F TRT-LLM as an orchestrator\n# Rust core, Python extensibility\n\n# Install\npip install nvidia-dynamo\n\n# Launch with disaggregated P\u002FD\ndynamo serve \\\n  --model deepseek-ai\u002FDeepSeek-R1 \\\n  --prefill-replicas 2 \\\n  --decode-replicas 4 \\\n  --kv-transfer nixl \\\n  --sla-planner auto\n\n# Dynamo Planner Profiler auto-measures workload and sets P:D ratio\n# Published results (developer.nvidia.com, 2025-06):\n#   DeepSeek-R1 MoE on GB200 NVL72 + Dynamo: ~6x throughput in medium-latency regime\n#   GB300 NVL72 + Dynamo: up to 50x MoE throughput vs Hopper (product page, undated)\n```\n\n---\n\n## llm-d (Red Hat + AWS, Kubernetes-native)\n\n```yaml\n# llm-d separates prefill \u002F decode \u002F router as independent K8s Services\n# Per-role HPA based on correct signals for each role\n\n# Prefill Deployment — scale on queue depth\napiVersion: apps\u002Fv1\nkind: Deployment\nmetadata:\n  name: prefill-server\nspec:\n  replicas: 2\n  template:\n    spec:\n      containers:\n        - name: prefill\n          image: llmd\u002Fvllm-prefill:latest\n          env:\n            - name: ROLE\n              value: prefill\n            - name: NIXL_ENDPOINT\n              value: decode-service:8001\n\n---\n# Decode Deployment — scale on KV utilization\napiVersion: apps\u002Fv1\nkind: Deployment\nmetadata:\n  name: decode-server\nspec:\n  replicas: 4\n  template:\n    spec:\n      containers:\n        - name: decode\n          image: llmd\u002Fvllm-decode:latest\n          env:\n            - name: ROLE\n              value: decode\n\n---\n# HPA for prefill (queue depth signal)\napiVersion: autoscaling\u002Fv2\nkind: HorizontalPodAutoscaler\nmetadata:\n  name: prefill-hpa\nspec:\n  scaleTargetRef:\n    apiVersion: apps\u002Fv1\n    kind: Deployment\n    name: prefill-server\n  minReplicas: 1\n  maxReplicas: 8\n  metrics:\n    - type: External\n      external:\n        metric:\n          name: llmd_prefill_queue_depth\n        target:\n          type: AverageValue\n          averageValue: \"10\"\n```\n\n---\n\n## When NOT to disaggregate\n\n```python\ndef should_disaggregate(\n    avg_prompt_tokens: int,\n    avg_output_tokens: int,\n    monthly_gpu_spend: float,\n) -> tuple[bool, str]:\n    if avg_prompt_tokens \u003C 512 and avg_output_tokens \u003C 512:\n        return False, \"Short prompts + short outputs: NIXL transfer tax exceeds savings\"\n    if monthly_gpu_spend \u003C 40_000:\n        return False, \"Orchestration complexity not justified under ~$40K\u002Fmonth\"\n    return True, f\"Expected savings: 30–40% on ${monthly_gpu_spend:,.0f}\u002Fmonth\"\n```\n\n---\n\n## Dynamo vs llm-d: choose based on context\n\n```\nDynamo  → already using NVIDIA stack (TRT-LLM, vLLM)\n           want managed P:D ratio tuning (SLA Planner auto-configures)\n           comfortable with NVIDIA licensing + support\n\nllm-d   → committed to Kubernetes \u002F CNCF ecosystem\n           need GitOps-friendly declarative config (YAML deployments)\n           want scale-to-zero + LoRA routing + hierarchical KV offloading (v0.5)\n           Red Hat\u002FAWS support preferred\n```\n\n---\n\n## Anti-Fake-Pass Checklist\n\n```\n❌ Short prompts (\u003C512 tokens): NIXL transfer latency dominates → no throughput gain\n❌ Prefill:decode ratio not tuned to workload → wrong pool sizes → one is idle, one is bottlenecked\n❌ RDMA not available → TCP fallback → higher latency → reduces the savings window\n❌ Same GPU type for both pools → defeats the purpose; prefill wants compute, decode wants memory bandwidth\n❌ No KV cache compression before NIXL transfer → 70B 4K-prompt KV cache can be 4–8 GB per request\n❌ Citing \"30x\" as a guaranteed number — it's a community aggregate on specific Blackwell+Dynamo+R1 stacks\n```\n","# \u002Fdisaggregated-prefill-decode\n\n## When to Use\n\n- Inference spend >$500K\u002Fyear on colocated serving — disaggregation can save 30–40%\n- Mixed workloads with both long-prompt (prefill-heavy) and long-output (decode-heavy) requests\n- MoE models (DeepSeek-R1, Mixtral) where colocation wastes H100 compute on memory-bound decode\n- Already on Kubernetes and want CNCF-native orchestration (llm-d)\n\n## Do NOT use for\n\n- Short prompts + short outputs (\u003C512 tokens each) — NIXL transfer tax exceeds savings\n- Small-scale serving (\u003C10 GPUs) — orchestration overhead not justified\n- Development \u002F testing environments — complexity far outweighs benefit\n\n---\n\n## Why the bottlenecks differ\n\n```\nPrefill  — full forward pass over input prompt → matrix multiplications dominate\n           → COMPUTE-BOUND — H100 FP8 ~2,000 TFLOPS\n           → batch many tokens in one forward → high GPU utilization\n\nDecode   — one token per step, reading full weights each time\n           → MEMORY-BANDWIDTH-BOUND — HBM3 ~3 TB\u002Fs\n           → only amortizes at high concurrency\n\nColocation problem:\n  You buy H100 compute for decode (memory bound → compute wasted)\n  You buy H100 HBM for prefill (compute bound → memory wasted)\n  20–40% GPU time wasted on the wrong resource at typical workloads.\n```\n\n---\n\n## Architecture diagram\n\n```\n            ┌──────────────┐\n  Request → │    Router    │ ─────────────────────────┐\n            └──────┬───────┘                          │\n                   │ (prompt only)                    │\n                   ▼                                  │\n            ┌──────────────┐   KV cache via NIXL   ┌──▼───────────┐\n            │ Prefill pool │ ──────────────────────► │ Decode pool  │\n            │ (compute)    │                         │ (memory)     │\n            └──────────────┘                         └──────┬───────┘\n                                                            │ tokens\n                                                            ▼\n                                                         Client\n\nNIXL: NVIDIA inter-node transport\n  - Uses RDMA\u002FInfiniBand when available\n  - TCP fallback otherwise\n  - Typical KV transfer latency: 20–80 ms for 4K-token prompt on 70B FP8\n  → Short prompts (\u003C512 tokens): transfer tax > compute savings → don't disaggregate\n```\n\n---\n\n## NVIDIA Dynamo (GTC 2025 GA)\n\n```bash\n# Dynamo sits ABOVE vLLM \u002F SGLang \u002F TRT-LLM as an orchestrator\n# Rust core, Python extensibility\n\n# Install\npip install nvidia-dynamo\n\n# Launch with disaggregated P\u002FD\ndynamo serve \\\n  --model deepseek-ai\u002FDeepSeek-R1 \\\n  --prefill-replicas 2 \\\n  --decode-replicas 4 \\\n  --kv-transfer nixl \\\n  --sla-planner auto\n\n# Dynamo Planner Profiler auto-measures workload and sets P:D ratio\n# Published results (developer.nvidia.com, 2025-06):\n#   DeepSeek-R1 MoE on GB200 NVL72 + Dynamo: ~6x throughput in medium-latency regime\n#   GB300 NVL72 + Dynamo: up to 50x MoE throughput vs Hopper (product page, undated)\n```\n\n---\n\n## llm-d (Red Hat + AWS, Kubernetes-native)\n\n```yaml\n# llm-d separates prefill \u002F decode \u002F router as independent K8s Services\n# Per-role HPA based on correct signals for each role\n\n# Prefill Deployment — scale on queue depth\napiVersion: apps\u002Fv1\nkind: Deployment\nmetadata:\n  name: prefill-server\nspec:\n  replicas: 2\n  template:\n    spec:\n      containers:\n        - name: prefill\n          image: llmd\u002Fvllm-prefill:latest\n          env:\n            - name: ROLE\n              value: prefill\n            - name: NIXL_ENDPOINT\n              value: decode-service:8001\n\n---\n# Decode Deployment — scale on KV utilization\napiVersion: apps\u002Fv1\nkind: Deployment\nmetadata:\n  name: decode-server\nspec:\n  replicas: 4\n  template:\n    spec:\n      containers:\n        - name: decode\n          image: llmd\u002Fvllm-decode:latest\n          env:\n            - name: ROLE\n              value: decode\n\n---\n# HPA for prefill (queue depth signal)\napiVersion: autoscaling\u002Fv2\nkind: HorizontalPodAutoscaler\nmetadata:\n  name: prefill-hpa\nspec:\n  scaleTargetRef:\n    apiVersion: apps\u002Fv1\n    kind: Deployment\n    name: prefill-server\n  minReplicas: 1\n  maxReplicas: 8\n  metrics:\n    - type: External\n      external:\n        metric:\n          name: llmd_prefill_queue_depth\n        target:\n          type: AverageValue\n          averageValue: \"10\"\n```\n\n---\n\n## When NOT to disaggregate\n\n```python\ndef should_disaggregate(\n    avg_prompt_tokens: int,\n    avg_output_tokens: int,\n    monthly_gpu_spend: float,\n) -> tuple[bool, str]:\n    if avg_prompt_tokens \u003C 512 and avg_output_tokens \u003C 512:\n        return False, \"Short prompts + short outputs: NIXL transfer tax exceeds savings\"\n    if monthly_gpu_spend \u003C 40_000:\n        return False, \"Orchestration complexity not justified under ~$40K\u002Fmonth\"\n    return True, f\"Expected savings: 30–40% on ${monthly_gpu_spend:,.0f}\u002Fmonth\"\n```\n\n---\n\n## Dynamo vs llm-d: choose based on context\n\n```\nDynamo  → already using NVIDIA stack (TRT-LLM, vLLM)\n           want managed P:D ratio tuning (SLA Planner auto-configures)\n           comfortable with NVIDIA licensing + support\n\nllm-d   → committed to Kubernetes \u002F CNCF ecosystem\n           need GitOps-friendly declarative config (YAML deployments)\n           want scale-to-zero + LoRA routing + hierarchical KV offloading (v0.5)\n           Red Hat\u002FAWS support preferred\n```\n\n---\n\n## Anti-Fake-Pass Checklist\n\n```\n❌ Short prompts (\u003C512 tokens): NIXL transfer latency dominates → no throughput gain\n❌ Prefill:decode ratio not tuned to workload → wrong pool sizes → one is idle, one is bottlenecked\n❌ RDMA not available → TCP fallback → higher latency → reduces the savings window\n❌ Same GPU type for both pools → defeats the purpose; prefill wants compute, decode wants memory bandwidth\n❌ No KV cache compression before NIXL transfer → 70B 4K-prompt KV cache can be 4–8 GB per request\n❌ Citing \"30x\" as a guaranteed number — it's a community aggregate on specific Blackwell+Dynamo+R1 stacks\n```\n",{},"main","481dcd7dd2e288b9d627a2bd017db70e64e6c6fa","core\u002Fskills\u002Fdisaggregated-prefill-decode",2,"https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI","https:\u002F\u002Fyana.vutam.link\u002F","https:\u002F\u002Fgithub.com\u002FHoronomos-laps","https:\u002F\u002Fyana.vutam.link","An open-source governed runtime & execution environment for multi-agent AI systems.","2026-11-14 13:24:09",{"source":7,"skillName":5,"commands":34,"prompt":44,"zipAvailable":45},[35,38,41],{"tool":36,"command":37},"npx","npx skills add Horonomos-laps\u002FYana-AI --skill disaggregated-prefill-decode",{"tool":39,"command":40},"bunx","bunx skills add Horonomos-laps\u002FYana-AI --skill disaggregated-prefill-decode",{"tool":42,"command":43},"pnpm","pnpm dlx skills add Horonomos-laps\u002FYana-AI --skill disaggregated-prefill-decode","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fdisaggregated-prefill-decode\u002FSKILL.md",true,[],[48,56,63,71,79,87],{"skillNo":49,"name":50,"nameZh":6,"description":51,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":52,"licenseFileUrl":6,"sourceHtmlUrl":53,"categories":54,"keywords":55,"publishedAt":18},"01M735GKET5YVYKAY62W2P5YXB","openai--sentry--sentry","Use when the user asks to inspect Sentry issues or events, summarize recent production errors, or pull basic Sentry health data via the Sentry API; perform read-only queries with the bundled script and require `SENTRY_AUTH_TOKEN`.","MIT","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fopenai--sentry--sentry\u002FSKILL.md",[],[],{"skillNo":57,"name":58,"nameZh":6,"description":59,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":60,"categories":61,"keywords":62,"publishedAt":18},"01M73DCAP83T17HY7SMGK7MGFE","ito-basket-compare","Compare Itô prediction-market baskets against a user's knowledge base, portfolio notes, financial context, watchlist, or research thesis. Use for read-only basket comparison and gap analysis without investment advice or live trading.","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fito-basket-compare\u002FSKILL.md",[],[],{"skillNo":64,"name":65,"nameZh":6,"description":66,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":67,"sourceHtmlUrl":68,"categories":69,"keywords":70,"publishedAt":18},"01M734CYEMZFDR3KH6SW0JP8TD","implementing-network-traffic-baselining","Build network traffic baselines from NetFlow\u002FIPFIX data using Python pandas for statistical analysis, z-score anomaly detection, and hourly\u002Fdaily traffic pattern profiling","https:\u002F\u002Fgithub.com\u002Fyanacuti1121\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-network-traffic-baselining\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-network-traffic-baselining\u002FSKILL.md",[],[],{"skillNo":72,"name":73,"nameZh":6,"description":74,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":75,"sourceHtmlUrl":76,"categories":77,"keywords":78,"publishedAt":18},"01M73ASZ6NJRJM44Z2P84NGQ3Q","detecting-evasion-techniques-in-endpoint-logs","Detects defense evasion techniques used by adversaries in endpoint logs including log tampering, timestomping, process injection, and security tool disabling. Use when investigating suspicious endpoint behavior, building detection rules for evasion tactics, or conducting threat hunting for stealthy adversary activity. Activates for requests involving evasion detection, defense evasion analysis, log tampering detection, or MITRE ATT&CK TA0005.","https:\u002F\u002Fgithub.com\u002Fyanacuti1121\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fdetecting-evasion-techniques-in-endpoint-logs\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fdetecting-evasion-techniques-in-endpoint-logs\u002FSKILL.md",[],[],{"skillNo":80,"name":81,"nameZh":6,"description":82,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":83,"sourceHtmlUrl":84,"categories":85,"keywords":86,"publishedAt":18},"01M734AFNXD06Z78QRK0VKGT7Q","implementing-microsegmentation-with-guardicore","Implementing microsegmentation using Akamai Guardicore Segmentation to map application dependencies, create granular network policies, visualize east-west traffic flows, and enforce least-privilege communication between workloads across data centers and cloud.","https:\u002F\u002Fgithub.com\u002Fyanacuti1121\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-microsegmentation-with-guardicore\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-microsegmentation-with-guardicore\u002FSKILL.md",[],[],{"skillNo":88,"name":89,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":90,"categories":91,"keywords":92,"publishedAt":18},"01M736TK27QMCXK0DVCH25SJN5","ramdisk-tmpfs-patterns","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Framdisk-tmpfs-patterns\u002FSKILL.md",[],[]]