[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M737M38TQ14X3DFRQ0ATZP9D":3,"skill-same:01M737M38TQ14X3DFRQ0ATZP9D":47},{"skillNo":4,"name":5,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":18,"linkOnly":19,"markdownContent":20,"markdownBody":21,"frontMatter":22,"refName":23,"commitSha":24,"skillDirectory":25,"forksCount":26,"repositoryHtmlUrl":27,"repositoryHomepageUrl":28,"ownerHtmlUrl":29,"ownerBlogUrl":30,"ownerDescription":31,"updatedAt":32,"install":33,"resources":46},"01M737M38TQ14X3DFRQ0ATZP9D","streaming-llm-infinite-context",null,"Horonomos-laps\u002FYana-AI","Horonomos-laps","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F355299122?v=4","01M76B3QPXE88ET2SQPDPVATZT"," Horonomos-laps","01M72ZQRJPCYPSSWT4D00VJR2A",13,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fstreaming-llm-infinite-context\u002FSKILL.md",[],[],"2026-11-11 15:11:14",false,"---\nname: streaming-llm-infinite-context\ndescription: StreamingLLM attention sink patterns for infinite-context LLM inference. Attention sink tokens, rolling KV cache window, no recomputation on eviction, and enabling agents to process arbitrarily long sessions. Sources: mit-han-lab\u002Fstreaming-llm (MIT).\norigin: yana-ai — synthesized from mit-han-lab\u002Fstreaming-llm (MIT)\nlicense: Apache-2.0\nversion: 1.0.0\ncompatibility: yana-ai >= 1.3.51\n---\n\n# \u002Fstreaming-llm-infinite-context\n\n## When to Use\n\n- Agent sessions that run indefinitely without context window reset\n- Long documents that exceed model context: process chunk-by-chunk without restart\n- Streaming decode without out-of-memory as sequence length grows\n- Avoid \"attention sink\" collapse that causes models to fail past their trained context\n\n## Do NOT use for\n\n- Tasks requiring recall from early in the session (evicted tokens are GONE)\n- Small contexts \u003C 4k tokens (standard KV cache is fine)\n\n---\n\n## Attention sink discovery\n\n```\nProblem:\n  Standard KV cache grows linearly → OOM at context limit\n  Dense attention past training length → incoherent output\n\nObservation (Xiao et al. 2023):\n  Attention scores massively concentrate on the first 4 tokens\n  regardless of their semantic content. These are \"attention sinks.\"\n\nStreamingLLM insight:\n  Keep: [attention sink tokens (first 4)] + [recent W tokens]\n  Evict: everything older than W except the sinks\n  Result: stable perplexity even at 4M tokens (tested)\n\nMemory: constant = (4 + W) × KV_size_per_token\n```\n\n---\n\n## Streaming generation with attention sinks\n\n```python\nfrom streaming_llm.enable_streaming_llm import enable_streaming_llm\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\nmodel     = AutoModelForCausalLM.from_pretrained('meta-llama\u002FLlama-3-8B', torch_dtype=torch.float16)\ntokenizer = AutoTokenizer.from_pretrained('meta-llama\u002FLlama-3-8B')\nmodel     = model.cuda()\n\n# Enable StreamingLLM: patch KV cache to use rolling window\nkv_cache = enable_streaming_llm(\n  model,\n  start_size   = 4,     # number of sink tokens to keep\n  recent_size  = 2000,  # sliding window of recent tokens\n)\n\n# Generate tokens indefinitely — memory stays flat\ndef stream_generate(prompt: str, max_tokens: int = 10_000):\n  inputs  = tokenizer(prompt, return_tensors='pt').to('cuda')\n  past_kv = None\n\n  for _ in range(max_tokens):\n    with torch.no_grad():\n      outputs = model(**inputs, past_key_values=past_kv, use_cache=True)\n\n    logits  = outputs.logits[:, -1, :]\n    next_id = logits.argmax(dim=-1, keepdim=True)\n\n    # Roll the KV cache: evict middle, keep sinks + recent\n    past_kv = kv_cache(outputs.past_key_values)\n\n    yield tokenizer.decode(next_id[0])\n    inputs = {'input_ids': next_id, 'attention_mask': torch.ones_like(next_id)}\n```\n\n---\n\n## When to reset vs stream\n\n```\nUSE streaming (no reset):\n  ✅ Continuous agent session (log reader, monitoring agent)\n  ✅ Long document processing (PDF → summary in chunks)\n  ✅ When most recent context is sufficient (chat, Q&A)\n\nUSE fresh context (reset):\n  ✅ When early conversation facts must be recalled precisely\n  ✅ Code generation that references declarations from session start\n  ✅ When accuracy > throughput (pay the recomputation cost)\n\nHybrid: summarize evicted window → inject summary as new context\n```\n\n---\n\n## Perplexity vs window size tradeoff\n\n```\nWindow (W)   Memory(8B)   Perplexity degradation\n────────────────────────────────────────────────\n  512          0.5 GB      +15% vs oracle\n 1000          1.0 GB      +5%\n 2000          2.0 GB      +2%    ← sweet spot\n 4000          4.0 GB      \u003C1%\n```\n\n---\n\n## Anti-Fake-Pass Checklist\n\n```\n❌ start_size: 0 → no attention sinks → model collapses on long contexts (validated by paper)\n❌ recent_size too small (\u003C 256) → model loses critical recent context; output degrades\n❌ Using streaming-llm without the monkey-patch → standard KV cache grows unbounded anyway\n❌ Expecting recall of evicted tokens → they are gone; no retrieval possible\n❌ Applying to models not supported (non-Llama architectures) → attention pattern may differ\n❌ Mixing streaming and standard cache → shape mismatch error in attention layers\n```\n","# \u002Fstreaming-llm-infinite-context\n\n## When to Use\n\n- Agent sessions that run indefinitely without context window reset\n- Long documents that exceed model context: process chunk-by-chunk without restart\n- Streaming decode without out-of-memory as sequence length grows\n- Avoid \"attention sink\" collapse that causes models to fail past their trained context\n\n## Do NOT use for\n\n- Tasks requiring recall from early in the session (evicted tokens are GONE)\n- Small contexts \u003C 4k tokens (standard KV cache is fine)\n\n---\n\n## Attention sink discovery\n\n```\nProblem:\n  Standard KV cache grows linearly → OOM at context limit\n  Dense attention past training length → incoherent output\n\nObservation (Xiao et al. 2023):\n  Attention scores massively concentrate on the first 4 tokens\n  regardless of their semantic content. These are \"attention sinks.\"\n\nStreamingLLM insight:\n  Keep: [attention sink tokens (first 4)] + [recent W tokens]\n  Evict: everything older than W except the sinks\n  Result: stable perplexity even at 4M tokens (tested)\n\nMemory: constant = (4 + W) × KV_size_per_token\n```\n\n---\n\n## Streaming generation with attention sinks\n\n```python\nfrom streaming_llm.enable_streaming_llm import enable_streaming_llm\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\nmodel     = AutoModelForCausalLM.from_pretrained('meta-llama\u002FLlama-3-8B', torch_dtype=torch.float16)\ntokenizer = AutoTokenizer.from_pretrained('meta-llama\u002FLlama-3-8B')\nmodel     = model.cuda()\n\n# Enable StreamingLLM: patch KV cache to use rolling window\nkv_cache = enable_streaming_llm(\n  model,\n  start_size   = 4,     # number of sink tokens to keep\n  recent_size  = 2000,  # sliding window of recent tokens\n)\n\n# Generate tokens indefinitely — memory stays flat\ndef stream_generate(prompt: str, max_tokens: int = 10_000):\n  inputs  = tokenizer(prompt, return_tensors='pt').to('cuda')\n  past_kv = None\n\n  for _ in range(max_tokens):\n    with torch.no_grad():\n      outputs = model(**inputs, past_key_values=past_kv, use_cache=True)\n\n    logits  = outputs.logits[:, -1, :]\n    next_id = logits.argmax(dim=-1, keepdim=True)\n\n    # Roll the KV cache: evict middle, keep sinks + recent\n    past_kv = kv_cache(outputs.past_key_values)\n\n    yield tokenizer.decode(next_id[0])\n    inputs = {'input_ids': next_id, 'attention_mask': torch.ones_like(next_id)}\n```\n\n---\n\n## When to reset vs stream\n\n```\nUSE streaming (no reset):\n  ✅ Continuous agent session (log reader, monitoring agent)\n  ✅ Long document processing (PDF → summary in chunks)\n  ✅ When most recent context is sufficient (chat, Q&A)\n\nUSE fresh context (reset):\n  ✅ When early conversation facts must be recalled precisely\n  ✅ Code generation that references declarations from session start\n  ✅ When accuracy > throughput (pay the recomputation cost)\n\nHybrid: summarize evicted window → inject summary as new context\n```\n\n---\n\n## Perplexity vs window size tradeoff\n\n```\nWindow (W)   Memory(8B)   Perplexity degradation\n────────────────────────────────────────────────\n  512          0.5 GB      +15% vs oracle\n 1000          1.0 GB      +5%\n 2000          2.0 GB      +2%    ← sweet spot\n 4000          4.0 GB      \u003C1%\n```\n\n---\n\n## Anti-Fake-Pass Checklist\n\n```\n❌ start_size: 0 → no attention sinks → model collapses on long contexts (validated by paper)\n❌ recent_size too small (\u003C 256) → model loses critical recent context; output degrades\n❌ Using streaming-llm without the monkey-patch → standard KV cache grows unbounded anyway\n❌ Expecting recall of evicted tokens → they are gone; no retrieval possible\n❌ Applying to models not supported (non-Llama architectures) → attention pattern may differ\n❌ Mixing streaming and standard cache → shape mismatch error in attention layers\n```\n",{},"main","481dcd7dd2e288b9d627a2bd017db70e64e6c6fa",".claude\u002Fskills\u002Fstreaming-llm-infinite-context",2,"https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI","https:\u002F\u002Fyana.vutam.link\u002F","https:\u002F\u002Fgithub.com\u002FHoronomos-laps","https:\u002F\u002Fyana.vutam.link","An open-source governed runtime & execution environment for multi-agent AI systems.","2026-11-14 13:24:09",{"source":7,"skillName":5,"commands":34,"prompt":44,"zipAvailable":45},[35,38,41],{"tool":36,"command":37},"npx","npx skills add Horonomos-laps\u002FYana-AI --skill streaming-llm-infinite-context",{"tool":39,"command":40},"bunx","bunx skills add Horonomos-laps\u002FYana-AI --skill streaming-llm-infinite-context",{"tool":42,"command":43},"pnpm","pnpm dlx skills add Horonomos-laps\u002FYana-AI --skill streaming-llm-infinite-context","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fstreaming-llm-infinite-context\u002FSKILL.md",true,[],[48,56,63,71,79,87],{"skillNo":49,"name":50,"nameZh":6,"description":51,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":52,"licenseFileUrl":6,"sourceHtmlUrl":53,"categories":54,"keywords":55,"publishedAt":18},"01M735GKET5YVYKAY62W2P5YXB","openai--sentry--sentry","Use when the user asks to inspect Sentry issues or events, summarize recent production errors, or pull basic Sentry health data via the Sentry API; perform read-only queries with the bundled script and require `SENTRY_AUTH_TOKEN`.","MIT","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fopenai--sentry--sentry\u002FSKILL.md",[],[],{"skillNo":57,"name":58,"nameZh":6,"description":59,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":60,"categories":61,"keywords":62,"publishedAt":18},"01M73DCAP83T17HY7SMGK7MGFE","ito-basket-compare","Compare Itô prediction-market baskets against a user's knowledge base, portfolio notes, financial context, watchlist, or research thesis. Use for read-only basket comparison and gap analysis without investment advice or live trading.","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fito-basket-compare\u002FSKILL.md",[],[],{"skillNo":64,"name":65,"nameZh":6,"description":66,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":67,"sourceHtmlUrl":68,"categories":69,"keywords":70,"publishedAt":18},"01M734CYEMZFDR3KH6SW0JP8TD","implementing-network-traffic-baselining","Build network traffic baselines from NetFlow\u002FIPFIX data using Python pandas for statistical analysis, z-score anomaly detection, and hourly\u002Fdaily traffic pattern profiling","https:\u002F\u002Fgithub.com\u002Fyanacuti1121\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-network-traffic-baselining\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-network-traffic-baselining\u002FSKILL.md",[],[],{"skillNo":72,"name":73,"nameZh":6,"description":74,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":75,"sourceHtmlUrl":76,"categories":77,"keywords":78,"publishedAt":18},"01M73ASZ6NJRJM44Z2P84NGQ3Q","detecting-evasion-techniques-in-endpoint-logs","Detects defense evasion techniques used by adversaries in endpoint logs including log tampering, timestomping, process injection, and security tool disabling. Use when investigating suspicious endpoint behavior, building detection rules for evasion tactics, or conducting threat hunting for stealthy adversary activity. Activates for requests involving evasion detection, defense evasion analysis, log tampering detection, or MITRE ATT&CK TA0005.","https:\u002F\u002Fgithub.com\u002Fyanacuti1121\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fdetecting-evasion-techniques-in-endpoint-logs\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fdetecting-evasion-techniques-in-endpoint-logs\u002FSKILL.md",[],[],{"skillNo":80,"name":81,"nameZh":6,"description":82,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":83,"sourceHtmlUrl":84,"categories":85,"keywords":86,"publishedAt":18},"01M734AFNXD06Z78QRK0VKGT7Q","implementing-microsegmentation-with-guardicore","Implementing microsegmentation using Akamai Guardicore Segmentation to map application dependencies, create granular network policies, visualize east-west traffic flows, and enforce least-privilege communication between workloads across data centers and cloud.","https:\u002F\u002Fgithub.com\u002Fyanacuti1121\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-microsegmentation-with-guardicore\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-microsegmentation-with-guardicore\u002FSKILL.md",[],[],{"skillNo":88,"name":89,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":90,"categories":91,"keywords":92,"publishedAt":18},"01M736TK27QMCXK0DVCH25SJN5","ramdisk-tmpfs-patterns","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Framdisk-tmpfs-patterns\u002FSKILL.md",[],[]]