[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7AVFGA1GRGS2EGSWRBM7GZC":3,"skill-same:01M7AVFGA1GRGS2EGSWRBM7GZC":44},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":24,"commitSha":25,"skillDirectory":26,"forksCount":14,"repositoryHtmlUrl":27,"repositoryHomepageUrl":6,"ownerHtmlUrl":28,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":29,"install":30,"resources":43},"01M7AVFGA1GRGS2EGSWRBM7GZC","inference-service",null,"Use when designing, reviewing, or debugging TypeScript\u002FJavaScript inference backends involving request admission, queues, engine batching, SSE streaming, cancellation, caching, fallbacks, circuit breakers, or GPU capacity.","MinhDuyDEV\u002Fmd-pikit","MinhDuyDEV","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F94704296?v=4","01M7AVDRJB5JM5VG5B6K49X0EZ","Nguyễn Viết Minh Duy","01M7AVDRKDB3JPGHFE4HGK1Z5M",0,"MIT","https:\u002F\u002Fgithub.com\u002FMinhDuyDEV\u002Fmd-pikit\u002Fblob\u002Fmain\u002F.pi\u002Fskills\u002Finference-service\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: inference-service\ndescription: Use when designing, reviewing, or debugging TypeScript\u002FJavaScript inference backends involving request admission, queues, engine batching, SSE streaming, cancellation, caching, fallbacks, circuit breakers, or GPU capacity.\n---\n\n# Inference Service\n\n\u003CHARD-GATE>\nThere is no universal inference topology. Before choosing components, define the latency SLO, quality contract, delivery semantics, tenant isolation, privacy\u002Fretention rules, cancellation behavior, and measured load. Never trade correctness or isolation for an assumed availability improvement.\n\u003C\u002FHARD-GATE>\n\n## Contract First\n\nRecord:\n\n- synchronous response versus accepted asynchronous job;\n- maximum prompt\u002Foutput size, concurrency, and rate limits per tenant;\n- model\u002Fversion\u002Fpolicy pinning and whether degradation is allowed;\n- at-most-once, at-least-once, or best-effort processing;\n- retry\u002Fidempotency contract and retention\u002Fdeletion requirements;\n- latency to first token, total latency, throughput, and cost budgets.\n\nDo not promise zero loss. Durable brokers still require replicated storage, producer acknowledgement, consumer acknowledgement, idempotency, replay policy, dead-letter handling, and tested disaster recovery.\n\n## Select Architecture from Evidence\n\n- A direct service-to-engine call can be correct at low measured load when the request remains synchronous and bounded.\n- A queue is optional: add one for durable accepted jobs, admission smoothing, or explicit backpressure—not by habit. Specify when a request becomes accepted and what cancellation means after enqueue.\n- Keep GPU\u002Fmodel runtimes isolated from the Node.js control plane when their failure, memory, or scaling characteristics differ.\n- Prefer engine batching when supported, but tune from throughput\u002Flatency measurements. Client batching is justified only by a documented engine or transport limitation.\n\n## Admission and Streaming\n\nAuthenticate and authorize before enqueue; validate model, tenant, token budget, and request schema. Apply per-tenant quotas and idempotency keys.\n\nFor SSE, propagate an `AbortSignal`, bound buffers and output tokens, implement backpressure, heartbeat only when infrastructure requires it, and handle disconnect cleanup idempotently. Do not acknowledge durable work merely because the client disconnected; follow the declared delivery contract.\n\n## Cache, Breaker, and Fallback Safety\n\nA semantic cache is opt-in, not a default. Disable it for sensitive data, rapidly changing policy, or outputs where semantic similarity cannot preserve correctness. If permitted, partition and key by tenant, authorization scope, model, version, policy, parameters, and safety configuration; define TTL, invalidation, encryption, and auditability.\n\nUse a circuit breaker around a named failing dependency with measured thresholds, minimum sample size, cooldown, and half-open probes. Preserve accepted-work semantics when it opens.\n\nUse a fallback only when the quality contract explicitly permits the alternate model, stale\u002Fcache age, or asynchronous response. Label degraded output and meter it. Never silently substitute another tenant's cache entry, model\u002Fversion, or lower-quality answer.\n\n## Observability and Verification\n\nMeasure admission rejects, queue depth\u002Fage, time to first token, total latency, tokens\u002Fsecond, cancellation latency, retry\u002Freplay counts, cache hits by tenant\u002Fmodel\u002Fpolicy, breaker state, fallback rate, GPU utilization\u002Fmemory, OOMs, and cost.\n\nTest overload, duplicate delivery, process death before\u002Fafter acknowledgement, broker outage, partial SSE disconnect, tenant isolation, cache invalidation, model rollout, breaker recovery, fallback prohibition, and graceful shutdown. Use fault injection before claiming durability or degradation behavior.\n","# Inference Service\n\n\u003CHARD-GATE>\nThere is no universal inference topology. Before choosing components, define the latency SLO, quality contract, delivery semantics, tenant isolation, privacy\u002Fretention rules, cancellation behavior, and measured load. Never trade correctness or isolation for an assumed availability improvement.\n\u003C\u002FHARD-GATE>\n\n## Contract First\n\nRecord:\n\n- synchronous response versus accepted asynchronous job;\n- maximum prompt\u002Foutput size, concurrency, and rate limits per tenant;\n- model\u002Fversion\u002Fpolicy pinning and whether degradation is allowed;\n- at-most-once, at-least-once, or best-effort processing;\n- retry\u002Fidempotency contract and retention\u002Fdeletion requirements;\n- latency to first token, total latency, throughput, and cost budgets.\n\nDo not promise zero loss. Durable brokers still require replicated storage, producer acknowledgement, consumer acknowledgement, idempotency, replay policy, dead-letter handling, and tested disaster recovery.\n\n## Select Architecture from Evidence\n\n- A direct service-to-engine call can be correct at low measured load when the request remains synchronous and bounded.\n- A queue is optional: add one for durable accepted jobs, admission smoothing, or explicit backpressure—not by habit. Specify when a request becomes accepted and what cancellation means after enqueue.\n- Keep GPU\u002Fmodel runtimes isolated from the Node.js control plane when their failure, memory, or scaling characteristics differ.\n- Prefer engine batching when supported, but tune from throughput\u002Flatency measurements. Client batching is justified only by a documented engine or transport limitation.\n\n## Admission and Streaming\n\nAuthenticate and authorize before enqueue; validate model, tenant, token budget, and request schema. Apply per-tenant quotas and idempotency keys.\n\nFor SSE, propagate an `AbortSignal`, bound buffers and output tokens, implement backpressure, heartbeat only when infrastructure requires it, and handle disconnect cleanup idempotently. Do not acknowledge durable work merely because the client disconnected; follow the declared delivery contract.\n\n## Cache, Breaker, and Fallback Safety\n\nA semantic cache is opt-in, not a default. Disable it for sensitive data, rapidly changing policy, or outputs where semantic similarity cannot preserve correctness. If permitted, partition and key by tenant, authorization scope, model, version, policy, parameters, and safety configuration; define TTL, invalidation, encryption, and auditability.\n\nUse a circuit breaker around a named failing dependency with measured thresholds, minimum sample size, cooldown, and half-open probes. Preserve accepted-work semantics when it opens.\n\nUse a fallback only when the quality contract explicitly permits the alternate model, stale\u002Fcache age, or asynchronous response. Label degraded output and meter it. Never silently substitute another tenant's cache entry, model\u002Fversion, or lower-quality answer.\n\n## Observability and Verification\n\nMeasure admission rejects, queue depth\u002Fage, time to first token, total latency, tokens\u002Fsecond, cancellation latency, retry\u002Freplay counts, cache hits by tenant\u002Fmodel\u002Fpolicy, breaker state, fallback rate, GPU utilization\u002Fmemory, OOMs, and cost.\n\nTest overload, duplicate delivery, process death before\u002Fafter acknowledgement, broker outage, partial SSE disconnect, tenant isolation, cache invalidation, model rollout, breaker recovery, fallback prohibition, and graceful shutdown. Use fault injection before claiming durability or degradation behavior.\n",{"name":5,"description":7},"main","01207a96514af9f9c4d7d6cb594b936c6a9ddb10",".pi\u002Fskills\u002Finference-service","https:\u002F\u002Fgithub.com\u002FMinhDuyDEV\u002Fmd-pikit","https:\u002F\u002Fgithub.com\u002FMinhDuyDEV","2026-08-20 02:00:33",{"source":8,"skillName":5,"commands":31,"prompt":41,"zipAvailable":42},[32,35,38],{"tool":33,"command":34},"npx","npx skills add MinhDuyDEV\u002Fmd-pikit --skill inference-service",{"tool":36,"command":37},"bunx","bunx skills add MinhDuyDEV\u002Fmd-pikit --skill inference-service",{"tool":39,"command":40},"pnpm","pnpm dlx skills add MinhDuyDEV\u002Fmd-pikit --skill inference-service","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FMinhDuyDEV\u002Fmd-pikit\u002Fblob\u002Fmain\u002F.pi\u002Fskills\u002Finference-service\u002FSKILL.md",true,[],[45,52,59,66,73,80],{"skillNo":46,"name":47,"nameZh":6,"description":48,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":49,"categories":50,"keywords":51,"publishedAt":19},"01M7AVGJR1RSRVA3VJT8ZW4Y27","writing-skills","Use when creating new skills, editing existing skills, or verifying skills work before deployment - applies TDD to process documentation by testing with subagents before writing, iterating until bulletproof against rationalization. Includes complete pressure testing methodology.","https:\u002F\u002Fgithub.com\u002FMinhDuyDEV\u002Fmd-pikit\u002Fblob\u002Fmain\u002F.pi\u002Fskills\u002Fwriting-skills\u002FSKILL.md",[],[],{"skillNo":53,"name":54,"nameZh":6,"description":55,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":56,"categories":57,"keywords":58,"publishedAt":19},"01M7AVGE076BNB3J7JDBGAKD2T","verification-before-completion","Use when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always","https:\u002F\u002Fgithub.com\u002FMinhDuyDEV\u002Fmd-pikit\u002Fblob\u002Fmain\u002F.pi\u002Fskills\u002Fverification-before-completion\u002FSKILL.md",[],[],{"skillNo":60,"name":61,"nameZh":6,"description":62,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":63,"categories":64,"keywords":65,"publishedAt":19},"01M7AVGAYJ6APXRB9QWBG7482S","typescript-coding-standards","Use when writing, refactoring, or reviewing TypeScript code that needs strong domain modeling, typed errors, schema parsing, safe adapters, test seams, or maintainable module boundaries.","https:\u002F\u002Fgithub.com\u002FMinhDuyDEV\u002Fmd-pikit\u002Fblob\u002Fmain\u002F.pi\u002Fskills\u002Ftypescript-coding-standards\u002FSKILL.md",[],[],{"skillNo":67,"name":68,"nameZh":6,"description":69,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":70,"categories":71,"keywords":72,"publishedAt":19},"01M7AVG8CE5BTR2CRFPHHVJCJZ","test-driven-development","Use when implementing a feature, bugfix, or behavior-changing refactor — require a failing behavior test before production code, then make the smallest change that passes.","https:\u002F\u002Fgithub.com\u002FMinhDuyDEV\u002Fmd-pikit\u002Fblob\u002Fmain\u002F.pi\u002Fskills\u002Ftest-driven-development\u002FSKILL.md",[],[],{"skillNo":74,"name":75,"nameZh":6,"description":76,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":77,"categories":78,"keywords":79,"publishedAt":19},"01M7AVG5FJH3WTG9XR2G0608ND","source-driven-development","Use when unfamiliar libraries, dependency internals, external APIs, framework behavior, migrations, or current ecosystem guidance require official documentation, version-matched source, read the tests, and cited evidence.","https:\u002F\u002Fgithub.com\u002FMinhDuyDEV\u002Fmd-pikit\u002Fblob\u002Fmain\u002F.pi\u002Fskills\u002Fsource-driven-development\u002FSKILL.md",[],[],{"skillNo":81,"name":82,"nameZh":6,"description":83,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":84,"categories":85,"keywords":86,"publishedAt":19},"01M7AVG37PZ77CF35KAR6E93ZJ","security-and-hardening","Use when auditing security, implementing authentication or authorization, handling secrets, or validating data across external trust boundaries.","https:\u002F\u002Fgithub.com\u002FMinhDuyDEV\u002Fmd-pikit\u002Fblob\u002Fmain\u002F.pi\u002Fskills\u002Fsecurity-and-hardening\u002FSKILL.md",[],[]]