[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M74G5B1CDM2FNHCJ6EGCH0GC":3,"skill-same:01M74G5B1CDM2FNHCJ6EGCH0GC":50},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":24,"commitSha":25,"skillDirectory":26,"forksCount":14,"repositoryHtmlUrl":27,"repositoryHomepageUrl":6,"ownerHtmlUrl":28,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":29,"install":30,"resources":43},"01M74G5B1CDM2FNHCJ6EGCH0GC","llm-cost-latency",null,"Cut LLM cost and latency — model routing\u002Fcascade, prompt caching, batching, streaming, context trimming, output-token control, and a per-request cost\u002Flatency budget.","MohammadBafkar\u002Fpolymath","MohammadBafkar","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F34940462?v=4","01M74G540TQFZWK6KVAPKFVGNR","Mohammad Bafkar","01M74G541HG19QP9V5DQCJVKYE",0,"MIT","https:\u002F\u002Fgithub.com\u002FMohammadBafkar\u002Fpolymath\u002Fblob\u002Fmain\u002Fplugins\u002Fpolymath-ai\u002Fskills\u002Fllm-cost-latency\u002FSKILL.md",[],[],"2026-11-11 15:11:14",false,"---\nname: llm-cost-latency\ndescription: Cut LLM cost and latency — model routing\u002Fcascade, prompt caching, batching, streaming, context trimming, output-token control, and a per-request cost\u002Flatency budget.\n---\n\n# llm-cost-latency\n\n> LLM spend is dominated by tokens × model tier × call count. Most savings come from sending fewer tokens to a smaller model less often — measure the request first, then cut the biggest line.\n\n**Mode:** analytical — judgment inside a fixed frame. Keep the output shape; every verdict cites its evidence; unknowns are marked `[TBD]` or `[VERIFY]`, never filled in.\n\n## When to use\n\n- An LLM feature is too slow or too expensive, or you are sizing one before launch.\n- You need a per-request cost\u002Flatency budget and a path to hit it.\n- A workflow invokes `polymath-ai:llm-cost-latency`.\n\nNot this when: this is a non-LLM service's latency\u002Fcost — that is `polymath-performance:backend-tail-latency` \u002F `perf-budget`. For prompt wording quality, `polymath-ai:prompt-engineer`.\n\n## Inputs\n\n- The current model, prompt sizes (system + context + output), call pattern, and measured cost\u002Flatency.\n- The quality bar the cheaper path must still clear.\n\n## Procedure\n\n1. **Account the request**: break cost into input tokens, output tokens, and model tier; break latency into time-to-first-token, generation time, and tool\u002Fround-trips. Optimize the largest contributor, not the easiest.\n2. **Right-size the model**: route easy cases to a smaller\u002Fcheaper model and escalate only hard ones (a cascade). Reserve the top tier for tasks that measurably need it — verify with `polymath-ai:eval-plan` that the cheaper model still passes.\n3. **Cut input tokens**: trim\u002Fcompress context, retrieve less but better (tune `rag-design`), drop stale transcript, and reuse a stable system prompt to exploit **prompt caching** (order prefix-stable content first; check the provider's minimum cacheable length, cache TTL, and the write premium vs read discount, then measure the cache-hit rate). In agent loops, input cost grows with every turn because the history is re-sent — caching and summarization matter most there.\n4. **Cut output tokens**: ask for the minimal format (structured\u002Fshort), cap max output, and stop sequences — output tokens are priced several times higher than input and dominate generation latency.\n5. **Cut call count**: batch independent requests (asynchronous batch APIs trade latency for a large discount on non-interactive work), collapse multi-call chains where one call suffices, and cache deterministic results.\n6. **Hide latency**: stream tokens to first paint; parallelize independent tool calls; precompute where the input is known ahead of time.\n7. **Set the budget + tripwire**: a per-request cost and p95 latency target, with an alert when exceeded — the same discipline as a perf budget.\n\n## Output\n\n```text\nLLM cost\u002Flatency plan: support-reply drafting\n\nPer request (measured, 1,000-request sample):\n  input  6,200 tok (system 1,800 · retrieved 3,900 · history 500) × $3.00\u002FM = $0.0186\n  output   450 tok × $15.00\u002FM                                              = $0.0068\n  total  $0.0254\u002Frequest · p95 latency 7.8 s (TTFT 1.1 s, generation 6.7 s)\n  largest line: retrieved context (61% of input cost)\n\nCuts (expected)\n  1. retrieve top-4 not top-10 chunks (eval: answer quality 91% → 90%, within noise)   −$0.0070\n  2. prompt caching on the 1,800-token system prefix (hit rate 85% measured in staging)  −$0.0041\n  3. cap output at 300 tokens + \"3 short paragraphs\" format                             −$0.0023, −2.2 s p95\n  4. route \"FAQ-match\" requests (38%) to the small tier after eval-plan passes on that slice\nStreaming: on — first paint at TTFT\nBudget + tripwire: ≤ $0.015\u002Frequest and p95 ≤ 5 s; alert at 120% of either for 1 h\n```\n\nPrices are the provider's current list prices, cited with the date checked; an unknown price is `[TBD: price]`, never remembered.\n\n## Quality bar\n\n- The optimization targets the measured largest cost\u002Flatency contributor, not a guess.\n- Any model downgrade is gated by an eval that shows quality holds.\n- Prompt caching is exploited via prefix-stable prompt ordering where the provider supports it.\n- A per-request budget + tripwire is named.\n","# llm-cost-latency\n\n> LLM spend is dominated by tokens × model tier × call count. Most savings come from sending fewer tokens to a smaller model less often — measure the request first, then cut the biggest line.\n\n**Mode:** analytical — judgment inside a fixed frame. Keep the output shape; every verdict cites its evidence; unknowns are marked `[TBD]` or `[VERIFY]`, never filled in.\n\n## When to use\n\n- An LLM feature is too slow or too expensive, or you are sizing one before launch.\n- You need a per-request cost\u002Flatency budget and a path to hit it.\n- A workflow invokes `polymath-ai:llm-cost-latency`.\n\nNot this when: this is a non-LLM service's latency\u002Fcost — that is `polymath-performance:backend-tail-latency` \u002F `perf-budget`. For prompt wording quality, `polymath-ai:prompt-engineer`.\n\n## Inputs\n\n- The current model, prompt sizes (system + context + output), call pattern, and measured cost\u002Flatency.\n- The quality bar the cheaper path must still clear.\n\n## Procedure\n\n1. **Account the request**: break cost into input tokens, output tokens, and model tier; break latency into time-to-first-token, generation time, and tool\u002Fround-trips. Optimize the largest contributor, not the easiest.\n2. **Right-size the model**: route easy cases to a smaller\u002Fcheaper model and escalate only hard ones (a cascade). Reserve the top tier for tasks that measurably need it — verify with `polymath-ai:eval-plan` that the cheaper model still passes.\n3. **Cut input tokens**: trim\u002Fcompress context, retrieve less but better (tune `rag-design`), drop stale transcript, and reuse a stable system prompt to exploit **prompt caching** (order prefix-stable content first; check the provider's minimum cacheable length, cache TTL, and the write premium vs read discount, then measure the cache-hit rate). In agent loops, input cost grows with every turn because the history is re-sent — caching and summarization matter most there.\n4. **Cut output tokens**: ask for the minimal format (structured\u002Fshort), cap max output, and stop sequences — output tokens are priced several times higher than input and dominate generation latency.\n5. **Cut call count**: batch independent requests (asynchronous batch APIs trade latency for a large discount on non-interactive work), collapse multi-call chains where one call suffices, and cache deterministic results.\n6. **Hide latency**: stream tokens to first paint; parallelize independent tool calls; precompute where the input is known ahead of time.\n7. **Set the budget + tripwire**: a per-request cost and p95 latency target, with an alert when exceeded — the same discipline as a perf budget.\n\n## Output\n\n```text\nLLM cost\u002Flatency plan: support-reply drafting\n\nPer request (measured, 1,000-request sample):\n  input  6,200 tok (system 1,800 · retrieved 3,900 · history 500) × $3.00\u002FM = $0.0186\n  output   450 tok × $15.00\u002FM                                              = $0.0068\n  total  $0.0254\u002Frequest · p95 latency 7.8 s (TTFT 1.1 s, generation 6.7 s)\n  largest line: retrieved context (61% of input cost)\n\nCuts (expected)\n  1. retrieve top-4 not top-10 chunks (eval: answer quality 91% → 90%, within noise)   −$0.0070\n  2. prompt caching on the 1,800-token system prefix (hit rate 85% measured in staging)  −$0.0041\n  3. cap output at 300 tokens + \"3 short paragraphs\" format                             −$0.0023, −2.2 s p95\n  4. route \"FAQ-match\" requests (38%) to the small tier after eval-plan passes on that slice\nStreaming: on — first paint at TTFT\nBudget + tripwire: ≤ $0.015\u002Frequest and p95 ≤ 5 s; alert at 120% of either for 1 h\n```\n\nPrices are the provider's current list prices, cited with the date checked; an unknown price is `[TBD: price]`, never remembered.\n\n## Quality bar\n\n- The optimization targets the measured largest cost\u002Flatency contributor, not a guess.\n- Any model downgrade is gated by an eval that shows quality holds.\n- Prompt caching is exploited via prefix-stable prompt ordering where the provider supports it.\n- A per-request budget + tripwire is named.\n",{"name":5,"description":7},"main","23f6e21841a39580c9da2b5ec941f39dda17cd86","plugins\u002Fpolymath-ai\u002Fskills\u002Fllm-cost-latency","https:\u002F\u002Fgithub.com\u002FMohammadBafkar\u002Fpolymath","https:\u002F\u002Fgithub.com\u002FMohammadBafkar","2026-10-29 18:34:59",{"source":8,"skillName":5,"commands":31,"prompt":41,"zipAvailable":42},[32,35,38],{"tool":33,"command":34},"npx","npx skills add MohammadBafkar\u002Fpolymath --skill llm-cost-latency",{"tool":36,"command":37},"bunx","bunx skills add MohammadBafkar\u002Fpolymath --skill llm-cost-latency",{"tool":39,"command":40},"pnpm","pnpm dlx skills add MohammadBafkar\u002Fpolymath --skill llm-cost-latency","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FMohammadBafkar\u002Fpolymath\u002Fblob\u002Fmain\u002Fplugins\u002Fpolymath-ai\u002Fskills\u002Fllm-cost-latency\u002FSKILL.md",true,[44],{"path":45,"resourceType":46,"resourceTypeDesc":47,"sizeBytes":48,"sourceHtmlUrl":49},"routing.yaml","REFERENCE","参考文档",391,"https:\u002F\u002Fgithub.com\u002FMohammadBafkar\u002Fpolymath\u002Fblob\u002Fmain\u002Fplugins\u002Fpolymath-ai\u002Fskills\u002Fllm-cost-latency\u002Frouting.yaml",[51,58,65,72,79,86],{"skillNo":52,"name":53,"nameZh":6,"description":54,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":55,"categories":56,"keywords":57,"publishedAt":19},"01M74GN87EBRHM9N87M7AP1HDD","runbook","Write or refresh an operational runbook for an alert\u002Fsymptom; structured as TL;DR + Pre-checks + Steps + Verification + Rollback + Escalation.","https:\u002F\u002Fgithub.com\u002FMohammadBafkar\u002Fpolymath\u002Fblob\u002Fmain\u002Fplugins\u002Fpolymath-writing\u002Fskills\u002Frunbook\u002FSKILL.md",[],[],{"skillNo":59,"name":60,"nameZh":6,"description":61,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":62,"categories":63,"keywords":64,"publishedAt":19},"01M74GN5GPKEC1JEKP2DFQNEQJ","rfc","Draft a lightweight design RFC to docs\u002Frfcs\u002F\u003Cslug>.md; Summary, Motivation, Detailed design, Drawbacks, Alternatives, Unresolved questions. A proposal to debate, not a recorded decision (adr).","https:\u002F\u002Fgithub.com\u002FMohammadBafkar\u002Fpolymath\u002Fblob\u002Fmain\u002Fplugins\u002Fpolymath-writing\u002Fskills\u002Frfc\u002FSKILL.md",[],[],{"skillNo":66,"name":67,"nameZh":6,"description":68,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":69,"categories":70,"keywords":71,"publishedAt":19},"01M74GN2F4CWPMV4X09TG4E304","editorial-pass","Review and tighten prose in a Markdown artifact for a target tone (terse, formal, friendly) without dropping information; preserve quotations and load-bearing constraints.","https:\u002F\u002Fgithub.com\u002FMohammadBafkar\u002Fpolymath\u002Fblob\u002Fmain\u002Fplugins\u002Fpolymath-writing\u002Fskills\u002Feditorial-pass\u002FSKILL.md",[],[],{"skillNo":73,"name":74,"nameZh":6,"description":75,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":76,"categories":77,"keywords":78,"publishedAt":19},"01M74GMZWGYMZ37C4DYGT6P23G","architecture-review-facilitation","Run an architecture review as a process — entry criteria, reviewer checklist, decision log, and follow-up actions — pairing the design artifact with a red-team pass.","https:\u002F\u002Fgithub.com\u002FMohammadBafkar\u002Fpolymath\u002Fblob\u002Fmain\u002Fplugins\u002Fpolymath-writing\u002Fskills\u002Farchitecture-review-facilitation\u002FSKILL.md",[],[],{"skillNo":80,"name":81,"nameZh":6,"description":82,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":83,"categories":84,"keywords":85,"publishedAt":19},"01M74GMXVX0YX3ETDXS527XDGM","architecture-doc","Write a C4-style architecture document; System Context + Containers (+ Components when interesting) with mermaid diagrams. Describes structure, not a decision (adr) or proposal (rfc).","https:\u002F\u002Fgithub.com\u002FMohammadBafkar\u002Fpolymath\u002Fblob\u002Fmain\u002Fplugins\u002Fpolymath-writing\u002Fskills\u002Farchitecture-doc\u002FSKILL.md",[],[],{"skillNo":87,"name":88,"nameZh":6,"description":89,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":90,"categories":91,"keywords":92,"publishedAt":19},"01M74GMVG1WSMCK066S3KE6V5K","adr","Write an Architecture Decision Record (Michael Nygard format) to docs\u002Fadrs\u002FNNNN-\u003Cslug>.md; frontmatter validated by the ADR artifact schema. Records a made decision, not a proposal to debate (rfc).","https:\u002F\u002Fgithub.com\u002FMohammadBafkar\u002Fpolymath\u002Fblob\u002Fmain\u002Fplugins\u002Fpolymath-writing\u002Fskills\u002Fadr\u002FSKILL.md",[],[]]