[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M73FSJ7NPFKD5DMYYK7X73JP":3,"skill-same:01M73FSJ7NPFKD5DMYYK7X73JP":47},{"skillNo":4,"name":5,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":18,"linkOnly":19,"markdownContent":20,"markdownBody":21,"frontMatter":22,"refName":23,"commitSha":24,"skillDirectory":25,"forksCount":26,"repositoryHtmlUrl":27,"repositoryHomepageUrl":28,"ownerHtmlUrl":29,"ownerBlogUrl":30,"ownerDescription":31,"updatedAt":32,"install":33,"resources":46},"01M73FSJ7NPFKD5DMYYK7X73JP","sglang-radix-attention",null,"Horonomos-laps\u002FYana-AI","Horonomos-laps","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F355299122?v=4","01M76B3QPXE88ET2SQPDPVATZT"," Horonomos-laps","01M72ZQRJPCYPSSWT4D00VJR2A",13,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fsglang-radix-attention\u002FSKILL.md",[],[],"2026-11-11 15:11:14",false,"---\nname: sglang-radix-attention\ndescription: sGLang RadixAttention — KV cache stored in a radix tree, reused across requests sharing common prefixes. Cache-aware scheduling (depth-first, LRU at branch level). 29% throughput edge vs vLLM on ShareGPT, 6.4x on RAG workloads, 86% hit rate on voice. Deployed 400k+ GPUs 2026. Sources: rohitg00\u002Fai-engineering-from-scratch (Apache-2.0).\norigin: yana-ai — synthesized from rohitg00\u002Fai-engineering-from-scratch (Apache-2.0)\nlicense: Apache-2.0\nversion: 1.0.0\ncompatibility: yana-ai >= 1.3.54\n---\n\n# \u002Fsglang-radix-attention\n\n## When to Use\n\n- RAG workloads: same system prompt + same retrieval preamble + varying question\n- Agentic workloads: same tool schemas + system prompt repeated across thousands of requests\n- Chat with long, stable system prompts (>1K tokens)\n- Voice\u002Fvision with repeated preambles (audio feature header, image encoder prefix)\n\n## Do NOT use for\n\n- Open-ended chat with unique prompts per request (no shared prefix = no cache benefit)\n- Single-shot code completion where every prompt is distinct\n- Short prompts (\u003C200 tokens) where cache lookup overhead exceeds savings\n\n---\n\n## How RadixAttention works\n\n```\nThe radix tree stores token sequences as paths from root to leaf.\nEach node owns KV blocks for the token range on its path.\nA new request WALKS the tree: matching nodes → KV blocks reused → no prefill.\n\nExample tree:\n  root\n   └─ \"You are a helpful assistant...\" (2,000 tokens = 124 KV blocks)\n       ├─ \"Context: \u003Cdoc A>...\" (500 tokens, 31 blocks)\n       │   ├─ \"Question: Alice\"  (80 tokens, 5 blocks — NEW: only 5 blocks prefilled)\n       │   └─ \"Question: Bob\"    (95 tokens, 6 blocks)\n       └─ \"Context: \u003Cdoc B>...\" (520 tokens, 33 blocks)\n\nWithout tree: request for doc-A + Alice prefills 2,625 tokens (160 blocks).\nWith tree:    prefills only 80 tokens (5 blocks). ~32x prefill savings.\n```\n\n---\n\n## The ordering invariant (engineer's key lever)\n\n```\nThe 6.4x improvement REQUIRES consistent prompt template ordering.\nA shared prefix to a human ≠ shared prefix to the radix tree.\n\n[system, tools, context, history, question]   ← correct (immutable first)\n[system, context, tools, history, question]   ← different order → tree miss\n\nRule: put everything immutable (system, tools, schemas) FIRST.\n      put retrieval context next.\n      put user question LAST.\n      NEVER interleave dynamic content into the stable prefix.\n\nReal case: one deployment went from 7% → 74% cache hit rate\n           by moving dynamic content out of the cacheable prefix.\n```\n\n---\n\n## Cache-aware scheduling\n\n```\nProblem with FCFS (first-come, first-served):\n  Request A shares 2,000-token prefix.\n  Request B arrives first, shares only 50 tokens.\n  FCFS serves B → 2,000-token branch evicted before A is served.\n\nCache-aware scheduling policies:\n  1. Depth-first dispatch    — prefer requests rooted at the same branch as running set\n  2. LRU at branch level     — evict whole branches (not individual blocks), shortest-used first\n```\n\n---\n\n## Benchmark numbers to remember\n\n```\nBenchmark                                       SGLang vs vLLM\n──────────────────────────────────────────────  ──────────────────\nLlama 3.1 8B, H100, ShareGPT 1K prompts         ~16,200 vs ~12,500 tok\u002Fs  (+29%)\nPrefix-heavy RAG (same doc, varying question)    up to 6.4x\nVoice cloning workloads                          86.4% prefix cache hit rate\nProduction hit rates across customers            50–99% (depends on prompt discipline)\nDeployment scale (2026)                          400,000+ GPUs\n\nUse sGLang over vLLM when: prefix-heavy workload + prompt ordering is controlled.\nUse vLLM when: unique-prompt workloads or you need PagedAttention's continuous batching.\n```\n\n---\n\n## sGLang deployment (Docker)\n\n```bash\n# Start sGLang server (OpenAI-compatible API)\ndocker run --gpus all \\\n  -p 30000:30000 \\\n  lmsysorg\u002Fsglang:latest \\\n  python -m sglang.launch_server \\\n    --model-path meta-llama\u002FMeta-Llama-3.1-8B-Instruct \\\n    --port 30000 \\\n    --enable-prefix-caching \\\n    --cache-radix-tree \\\n    --mem-fraction-static 0.88\n\n# OpenAI-compatible: point any client at http:\u002F\u002Flocalhost:30000\n```\n\n---\n\n## Prompt template discipline (Python)\n\n```python\nfrom dataclasses import dataclass\n\n@dataclass\nclass PromptBuilder:\n    \"\"\"Enforces immutable-first ordering for maximum prefix cache hits.\"\"\"\n    system_prompt: str          # IMMUTABLE — goes first\n    tool_schemas:  list[str]    # IMMUTABLE — goes second\n    few_shot_examples: list[str] = ()  # IMMUTABLE if fixed\n\n    def build(self, context: str, history: list[str], question: str) -> str:\n        parts = [\n            f\"[SYSTEM]\\n{self.system_prompt}\",\n            \"[TOOLS]\\n\" + \"\\n\".join(self.tool_schemas),\n        ]\n        if self.few_shot_examples:\n            parts.append(\"[EXAMPLES]\\n\" + \"\\n\".join(self.few_shot_examples))\n        # Dynamic parts LAST — do not reorder\n        if context:\n            parts.append(f\"[CONTEXT]\\n{context}\")\n        if history:\n            parts.append(\"[HISTORY]\\n\" + \"\\n\".join(history))\n        parts.append(f\"[QUESTION]\\n{question}\")\n        return \"\\n\\n\".join(parts)\n\n    def validate_ordering(self) -> None:\n        \"\"\"Fail fast if dynamic content would corrupt the immutable prefix.\"\"\"\n        if \"{\" in self.system_prompt or \"{\" in \" \".join(self.tool_schemas):\n            raise ValueError(\"System prompt and tool schemas must not contain dynamic f-string placeholders\")\n```\n\n---\n\n## Anti-Fake-Pass Checklist\n\n```\n❌ Dynamic content interleaved into system prompt → radix tree cannot find shared prefix → 6.4x → 1.0x\n❌ Prompt template varies across requests → cache miss rate > 90% → sGLang offers no advantage over vLLM\n❌ FCFS scheduling not replaced → hot branch evicted before high-prefix-share requests → cache churns\n❌ All-in-one vLLM used for RAG workload → missing 29–6.4x throughput; switch to sGLang\n❌ Short prompts (\u003C200 tokens) → lookup overhead > savings; RadixAttention not worth it\n❌ Assuming \"cache hit\" without measuring → instrument with SGLang's cache_hit_rate metric\n```\n","# \u002Fsglang-radix-attention\n\n## When to Use\n\n- RAG workloads: same system prompt + same retrieval preamble + varying question\n- Agentic workloads: same tool schemas + system prompt repeated across thousands of requests\n- Chat with long, stable system prompts (>1K tokens)\n- Voice\u002Fvision with repeated preambles (audio feature header, image encoder prefix)\n\n## Do NOT use for\n\n- Open-ended chat with unique prompts per request (no shared prefix = no cache benefit)\n- Single-shot code completion where every prompt is distinct\n- Short prompts (\u003C200 tokens) where cache lookup overhead exceeds savings\n\n---\n\n## How RadixAttention works\n\n```\nThe radix tree stores token sequences as paths from root to leaf.\nEach node owns KV blocks for the token range on its path.\nA new request WALKS the tree: matching nodes → KV blocks reused → no prefill.\n\nExample tree:\n  root\n   └─ \"You are a helpful assistant...\" (2,000 tokens = 124 KV blocks)\n       ├─ \"Context: \u003Cdoc A>...\" (500 tokens, 31 blocks)\n       │   ├─ \"Question: Alice\"  (80 tokens, 5 blocks — NEW: only 5 blocks prefilled)\n       │   └─ \"Question: Bob\"    (95 tokens, 6 blocks)\n       └─ \"Context: \u003Cdoc B>...\" (520 tokens, 33 blocks)\n\nWithout tree: request for doc-A + Alice prefills 2,625 tokens (160 blocks).\nWith tree:    prefills only 80 tokens (5 blocks). ~32x prefill savings.\n```\n\n---\n\n## The ordering invariant (engineer's key lever)\n\n```\nThe 6.4x improvement REQUIRES consistent prompt template ordering.\nA shared prefix to a human ≠ shared prefix to the radix tree.\n\n[system, tools, context, history, question]   ← correct (immutable first)\n[system, context, tools, history, question]   ← different order → tree miss\n\nRule: put everything immutable (system, tools, schemas) FIRST.\n      put retrieval context next.\n      put user question LAST.\n      NEVER interleave dynamic content into the stable prefix.\n\nReal case: one deployment went from 7% → 74% cache hit rate\n           by moving dynamic content out of the cacheable prefix.\n```\n\n---\n\n## Cache-aware scheduling\n\n```\nProblem with FCFS (first-come, first-served):\n  Request A shares 2,000-token prefix.\n  Request B arrives first, shares only 50 tokens.\n  FCFS serves B → 2,000-token branch evicted before A is served.\n\nCache-aware scheduling policies:\n  1. Depth-first dispatch    — prefer requests rooted at the same branch as running set\n  2. LRU at branch level     — evict whole branches (not individual blocks), shortest-used first\n```\n\n---\n\n## Benchmark numbers to remember\n\n```\nBenchmark                                       SGLang vs vLLM\n──────────────────────────────────────────────  ──────────────────\nLlama 3.1 8B, H100, ShareGPT 1K prompts         ~16,200 vs ~12,500 tok\u002Fs  (+29%)\nPrefix-heavy RAG (same doc, varying question)    up to 6.4x\nVoice cloning workloads                          86.4% prefix cache hit rate\nProduction hit rates across customers            50–99% (depends on prompt discipline)\nDeployment scale (2026)                          400,000+ GPUs\n\nUse sGLang over vLLM when: prefix-heavy workload + prompt ordering is controlled.\nUse vLLM when: unique-prompt workloads or you need PagedAttention's continuous batching.\n```\n\n---\n\n## sGLang deployment (Docker)\n\n```bash\n# Start sGLang server (OpenAI-compatible API)\ndocker run --gpus all \\\n  -p 30000:30000 \\\n  lmsysorg\u002Fsglang:latest \\\n  python -m sglang.launch_server \\\n    --model-path meta-llama\u002FMeta-Llama-3.1-8B-Instruct \\\n    --port 30000 \\\n    --enable-prefix-caching \\\n    --cache-radix-tree \\\n    --mem-fraction-static 0.88\n\n# OpenAI-compatible: point any client at http:\u002F\u002Flocalhost:30000\n```\n\n---\n\n## Prompt template discipline (Python)\n\n```python\nfrom dataclasses import dataclass\n\n@dataclass\nclass PromptBuilder:\n    \"\"\"Enforces immutable-first ordering for maximum prefix cache hits.\"\"\"\n    system_prompt: str          # IMMUTABLE — goes first\n    tool_schemas:  list[str]    # IMMUTABLE — goes second\n    few_shot_examples: list[str] = ()  # IMMUTABLE if fixed\n\n    def build(self, context: str, history: list[str], question: str) -> str:\n        parts = [\n            f\"[SYSTEM]\\n{self.system_prompt}\",\n            \"[TOOLS]\\n\" + \"\\n\".join(self.tool_schemas),\n        ]\n        if self.few_shot_examples:\n            parts.append(\"[EXAMPLES]\\n\" + \"\\n\".join(self.few_shot_examples))\n        # Dynamic parts LAST — do not reorder\n        if context:\n            parts.append(f\"[CONTEXT]\\n{context}\")\n        if history:\n            parts.append(\"[HISTORY]\\n\" + \"\\n\".join(history))\n        parts.append(f\"[QUESTION]\\n{question}\")\n        return \"\\n\\n\".join(parts)\n\n    def validate_ordering(self) -> None:\n        \"\"\"Fail fast if dynamic content would corrupt the immutable prefix.\"\"\"\n        if \"{\" in self.system_prompt or \"{\" in \" \".join(self.tool_schemas):\n            raise ValueError(\"System prompt and tool schemas must not contain dynamic f-string placeholders\")\n```\n\n---\n\n## Anti-Fake-Pass Checklist\n\n```\n❌ Dynamic content interleaved into system prompt → radix tree cannot find shared prefix → 6.4x → 1.0x\n❌ Prompt template varies across requests → cache miss rate > 90% → sGLang offers no advantage over vLLM\n❌ FCFS scheduling not replaced → hot branch evicted before high-prefix-share requests → cache churns\n❌ All-in-one vLLM used for RAG workload → missing 29–6.4x throughput; switch to sGLang\n❌ Short prompts (\u003C200 tokens) → lookup overhead > savings; RadixAttention not worth it\n❌ Assuming \"cache hit\" without measuring → instrument with SGLang's cache_hit_rate metric\n```\n",{},"main","481dcd7dd2e288b9d627a2bd017db70e64e6c6fa","core\u002Fskills\u002Fsglang-radix-attention",1,"https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI","https:\u002F\u002Fyana.vutam.link\u002F","https:\u002F\u002Fgithub.com\u002FHoronomos-laps","https:\u002F\u002Fyana.vutam.link","An open-source governed runtime & execution environment for multi-agent AI systems.","2026-11-11 23:56:26",{"source":7,"skillName":5,"commands":34,"prompt":44,"zipAvailable":45},[35,38,41],{"tool":36,"command":37},"npx","npx skills add Horonomos-laps\u002FYana-AI --skill sglang-radix-attention",{"tool":39,"command":40},"bunx","bunx skills add Horonomos-laps\u002FYana-AI --skill sglang-radix-attention",{"tool":42,"command":43},"pnpm","pnpm dlx skills add Horonomos-laps\u002FYana-AI --skill sglang-radix-attention","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fsglang-radix-attention\u002FSKILL.md",true,[],[48,56,63,71,79,87],{"skillNo":49,"name":50,"nameZh":6,"description":51,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":52,"licenseFileUrl":6,"sourceHtmlUrl":53,"categories":54,"keywords":55,"publishedAt":18},"01M735GKET5YVYKAY62W2P5YXB","openai--sentry--sentry","Use when the user asks to inspect Sentry issues or events, summarize recent production errors, or pull basic Sentry health data via the Sentry API; perform read-only queries with the bundled script and require `SENTRY_AUTH_TOKEN`.","MIT","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fopenai--sentry--sentry\u002FSKILL.md",[],[],{"skillNo":57,"name":58,"nameZh":6,"description":59,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":60,"categories":61,"keywords":62,"publishedAt":18},"01M73DCAP83T17HY7SMGK7MGFE","ito-basket-compare","Compare Itô prediction-market baskets against a user's knowledge base, portfolio notes, financial context, watchlist, or research thesis. Use for read-only basket comparison and gap analysis without investment advice or live trading.","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fito-basket-compare\u002FSKILL.md",[],[],{"skillNo":64,"name":65,"nameZh":6,"description":66,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":67,"sourceHtmlUrl":68,"categories":69,"keywords":70,"publishedAt":18},"01M734CYEMZFDR3KH6SW0JP8TD","implementing-network-traffic-baselining","Build network traffic baselines from NetFlow\u002FIPFIX data using Python pandas for statistical analysis, z-score anomaly detection, and hourly\u002Fdaily traffic pattern profiling","https:\u002F\u002Fgithub.com\u002Fyanacuti1121\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-network-traffic-baselining\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-network-traffic-baselining\u002FSKILL.md",[],[],{"skillNo":72,"name":73,"nameZh":6,"description":74,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":75,"sourceHtmlUrl":76,"categories":77,"keywords":78,"publishedAt":18},"01M73ASZ6NJRJM44Z2P84NGQ3Q","detecting-evasion-techniques-in-endpoint-logs","Detects defense evasion techniques used by adversaries in endpoint logs including log tampering, timestomping, process injection, and security tool disabling. Use when investigating suspicious endpoint behavior, building detection rules for evasion tactics, or conducting threat hunting for stealthy adversary activity. Activates for requests involving evasion detection, defense evasion analysis, log tampering detection, or MITRE ATT&CK TA0005.","https:\u002F\u002Fgithub.com\u002Fyanacuti1121\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fdetecting-evasion-techniques-in-endpoint-logs\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fdetecting-evasion-techniques-in-endpoint-logs\u002FSKILL.md",[],[],{"skillNo":80,"name":81,"nameZh":6,"description":82,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":83,"sourceHtmlUrl":84,"categories":85,"keywords":86,"publishedAt":18},"01M734AFNXD06Z78QRK0VKGT7Q","implementing-microsegmentation-with-guardicore","Implementing microsegmentation using Akamai Guardicore Segmentation to map application dependencies, create granular network policies, visualize east-west traffic flows, and enforce least-privilege communication between workloads across data centers and cloud.","https:\u002F\u002Fgithub.com\u002Fyanacuti1121\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-microsegmentation-with-guardicore\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-microsegmentation-with-guardicore\u002FSKILL.md",[],[],{"skillNo":88,"name":89,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":90,"categories":91,"keywords":92,"publishedAt":18},"01M736TK27QMCXK0DVCH25SJN5","ramdisk-tmpfs-patterns","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Framdisk-tmpfs-patterns\u002FSKILL.md",[],[]]