[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7EN2NT3AMVKGD3Z2AMTJ1X5":3,"skill-same:01M7EN2NT3AMVKGD3Z2AMTJ1X5":47},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":25,"commitSha":26,"skillDirectory":27,"forksCount":28,"repositoryHtmlUrl":29,"repositoryHomepageUrl":6,"ownerHtmlUrl":30,"ownerBlogUrl":31,"ownerDescription":6,"updatedAt":32,"install":33,"resources":46},"01M7EN2NT3AMVKGD3Z2AMTJ1X5","friendli-ai",null,"High-throughput inference on FriendliAI — optimized serving engine for low-latency, cost-efficient LLMs.","TanNhatCMS\u002Fjarvis","TanNhatCMS","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F113796420?v=4","01M753MV17KG5CD3152G3WQEQY","KietNT","01M7EKEDYVYQ6PWVJJMZ911VMX",0,"MIT","https:\u002F\u002Fgithub.com\u002FTanNhatCMS\u002Fjarvis\u002Fblob\u002Fmain\u002Fskills\u002Fams-friendli-ai\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: friendli-ai\ndescription: High-throughput inference on FriendliAI — optimized serving engine for low-latency, cost-efficient LLMs.\ncategory: ai-research\n---\n\n## Overview\n\nFriendliAI builds inference infrastructure with a focus on serving efficiency:\ntheir serving engine (with techniques like iteration-level scheduling and\noptimized kernels) targets high throughput and low latency per dollar, offered\nboth as a managed inference service and as software for self-hosted deployment.\nThe positioning is infrastructure-grade efficiency — for teams serving LLMs at\nscale where serving economics dominate the bill.\n\nThe distinctive angle is the engine itself: FriendliAI's research in efficient\nserving (published work on scheduling and optimization) informs both their\ncloud service and their deployable software. For organizations serving open\nmodels at volume — whether on FriendliAI's cloud or their own GPUs with\nFriendliAI's stack — the value is measured in tokens per dollar at target\nlatency.\n\nEvaluate FriendliAI where serving efficiency is the decision criterion: high\nvolume, latency SLOs, and cost-per-token pressure.\n\n## When to use\n\n- High-volume LLM serving where cost-per-token at scale decides.\n- Latency-sensitive serving needing efficient scheduling (low TTFT + high\n  throughput).\n- Self-hosted inference wanting a high-efficiency serving engine.\n- Multi-tenant serving with mixed workload shapes.\n- Throughput-heavy batch + online mixed workloads.\n- Teams optimizing serving economics as a core competency.\n\n## Core concepts\n\n- **Serving engine efficiency**: iteration-level scheduling, optimized\n  attention kernels, and quantization support — the technical sources of\n  throughput-per-dollar. Understand what's applied in your deployment.\n- **Managed service vs. software**: use their cloud, or deploy their engine\n  on your infrastructure. The choice hinges on your ops capacity and data\n  control needs.\n- **Throughput per dollar**: the headline metric — tokens\u002Fsec per unit cost\n  at your latency SLO. Benchmark this, not raw tokens\u002Fsec alone.\n- **Latency SLO adherence**: efficiency that violates p99 latency isn't\n  useful. Measure the latency distribution, not just the mean throughput.\n- **Quantization integration**: efficient serving often involves quantization\n  — validate output quality on your task for the quantized configuration.\n- **Mixed workload handling**: batch and interactive traffic on shared\n  infrastructure — scheduling that protects interactive latency while\n  maintaining batch throughput.\n- **Dedicated capacity**: reserved deployments for predictable performance.\n  Match to your SLO requirements.\n- **Observability**: throughput, latency percentiles, GPU utilization, queue\n  depths — the metrics that reveal serving health.\n\n## Practical workflow\n\n1. **Define the efficiency target.** Target cost per million tokens at your\n   required p99 latency. This is the benchmark everything else serves.\n2. **Benchmark throughput per dollar.** Your models, your prompt shapes, your\n   concurrency: measure tokens\u002Fsec per dollar at acceptable latency. Compare\n   against your current serving.\n3. **Validate quantized quality.** If the efficient configuration uses\n   quantization: run your eval set. Efficiency that degrades quality below\n   your bar is disqualified.\n4. **Test mixed workloads.** If you serve batch + interactive: verify\n   interactive latency holds under batch load. Scheduling quality shows here.\n5. **Decide managed vs. self-hosted.** Ops capacity, data control, and\n   economics — model both. Self-hosting the engine trades ops burden for\n   control.\n6. **Load-test at scale.** Sustained peak concurrency; watch latency\n   percentiles, queue behavior, and failure modes under overload.\n7. **Monitor serving economics continuously.** Track cost per token, latency\n   SLOs, and utilization. Serving efficiency drifts with traffic shape —\n   review regularly.\n\nChecklist for FriendliAI in production:\n- Cost per token at required p99 latency benchmarked.\n- Quantized configurations quality-validated.\n- Mixed-workload latency verified (if applicable).\n- Managed vs. self-hosted decision modeled.\n- Serving metrics monitored (throughput, latency, utilization, cost).\n\n## Common pitfalls\n\n- **Throughput without latency SLOs.** Optimizing tokens\u002Fsec while p99 latency\n  violates UX requirements. Efficiency is throughput per dollar at acceptable\n  latency.\n- **Unvalidated quantization.** Assuming quantized serving preserves quality.\n  Validate on your task — always.\n- **Single-workload benchmarks.** Testing only uniform prompts when production\n  mixes short\u002Flong, batch\u002Finteractive. Test the mix.\n- **Ignoring queue behavior.** Throughput looks fine until queues build under\n  burst. Test bursts; monitor queue depths.\n- **Managed-vs-self-hosted on vibes.** Choosing without modeling ops cost,\n  data control needs, and economics. Decide on numbers.\n- **Utilization blindness.** Low GPU utilization means wasted money; saturation\n  means latency risk. Right-size from metrics.\n- **No overload plan.** What happens past capacity — graceful degradation or\n  collapse? Define and test the overload behavior.\n- **Static efficiency assumptions.** Traffic shape changes alter serving\n  efficiency. Re-benchmark when workloads shift.\n","## Overview\n\nFriendliAI builds inference infrastructure with a focus on serving efficiency:\ntheir serving engine (with techniques like iteration-level scheduling and\noptimized kernels) targets high throughput and low latency per dollar, offered\nboth as a managed inference service and as software for self-hosted deployment.\nThe positioning is infrastructure-grade efficiency — for teams serving LLMs at\nscale where serving economics dominate the bill.\n\nThe distinctive angle is the engine itself: FriendliAI's research in efficient\nserving (published work on scheduling and optimization) informs both their\ncloud service and their deployable software. For organizations serving open\nmodels at volume — whether on FriendliAI's cloud or their own GPUs with\nFriendliAI's stack — the value is measured in tokens per dollar at target\nlatency.\n\nEvaluate FriendliAI where serving efficiency is the decision criterion: high\nvolume, latency SLOs, and cost-per-token pressure.\n\n## When to use\n\n- High-volume LLM serving where cost-per-token at scale decides.\n- Latency-sensitive serving needing efficient scheduling (low TTFT + high\n  throughput).\n- Self-hosted inference wanting a high-efficiency serving engine.\n- Multi-tenant serving with mixed workload shapes.\n- Throughput-heavy batch + online mixed workloads.\n- Teams optimizing serving economics as a core competency.\n\n## Core concepts\n\n- **Serving engine efficiency**: iteration-level scheduling, optimized\n  attention kernels, and quantization support — the technical sources of\n  throughput-per-dollar. Understand what's applied in your deployment.\n- **Managed service vs. software**: use their cloud, or deploy their engine\n  on your infrastructure. The choice hinges on your ops capacity and data\n  control needs.\n- **Throughput per dollar**: the headline metric — tokens\u002Fsec per unit cost\n  at your latency SLO. Benchmark this, not raw tokens\u002Fsec alone.\n- **Latency SLO adherence**: efficiency that violates p99 latency isn't\n  useful. Measure the latency distribution, not just the mean throughput.\n- **Quantization integration**: efficient serving often involves quantization\n  — validate output quality on your task for the quantized configuration.\n- **Mixed workload handling**: batch and interactive traffic on shared\n  infrastructure — scheduling that protects interactive latency while\n  maintaining batch throughput.\n- **Dedicated capacity**: reserved deployments for predictable performance.\n  Match to your SLO requirements.\n- **Observability**: throughput, latency percentiles, GPU utilization, queue\n  depths — the metrics that reveal serving health.\n\n## Practical workflow\n\n1. **Define the efficiency target.** Target cost per million tokens at your\n   required p99 latency. This is the benchmark everything else serves.\n2. **Benchmark throughput per dollar.** Your models, your prompt shapes, your\n   concurrency: measure tokens\u002Fsec per dollar at acceptable latency. Compare\n   against your current serving.\n3. **Validate quantized quality.** If the efficient configuration uses\n   quantization: run your eval set. Efficiency that degrades quality below\n   your bar is disqualified.\n4. **Test mixed workloads.** If you serve batch + interactive: verify\n   interactive latency holds under batch load. Scheduling quality shows here.\n5. **Decide managed vs. self-hosted.** Ops capacity, data control, and\n   economics — model both. Self-hosting the engine trades ops burden for\n   control.\n6. **Load-test at scale.** Sustained peak concurrency; watch latency\n   percentiles, queue behavior, and failure modes under overload.\n7. **Monitor serving economics continuously.** Track cost per token, latency\n   SLOs, and utilization. Serving efficiency drifts with traffic shape —\n   review regularly.\n\nChecklist for FriendliAI in production:\n- Cost per token at required p99 latency benchmarked.\n- Quantized configurations quality-validated.\n- Mixed-workload latency verified (if applicable).\n- Managed vs. self-hosted decision modeled.\n- Serving metrics monitored (throughput, latency, utilization, cost).\n\n## Common pitfalls\n\n- **Throughput without latency SLOs.** Optimizing tokens\u002Fsec while p99 latency\n  violates UX requirements. Efficiency is throughput per dollar at acceptable\n  latency.\n- **Unvalidated quantization.** Assuming quantized serving preserves quality.\n  Validate on your task — always.\n- **Single-workload benchmarks.** Testing only uniform prompts when production\n  mixes short\u002Flong, batch\u002Finteractive. Test the mix.\n- **Ignoring queue behavior.** Throughput looks fine until queues build under\n  burst. Test bursts; monitor queue depths.\n- **Managed-vs-self-hosted on vibes.** Choosing without modeling ops cost,\n  data control needs, and economics. Decide on numbers.\n- **Utilization blindness.** Low GPU utilization means wasted money; saturation\n  means latency risk. Right-size from metrics.\n- **No overload plan.** What happens past capacity — graceful degradation or\n  collapse? Define and test the overload behavior.\n- **Static efficiency assumptions.** Traffic shape changes alter serving\n  efficiency. Re-benchmark when workloads shift.\n",{"name":5,"category":24,"description":7},"ai-research","main","af388a77c2fe5e0b4fd879cca4300a0ffa9a6579","skills\u002Fams-friendli-ai",1,"https:\u002F\u002Fgithub.com\u002FTanNhatCMS\u002Fjarvis","https:\u002F\u002Fgithub.com\u002FTanNhatCMS","https:\u002F\u002Fmrkiet.name.vn","2026-11-07 14:23:08",{"source":8,"skillName":5,"commands":34,"prompt":44,"zipAvailable":45},[35,38,41],{"tool":36,"command":37},"npx","npx skills add TanNhatCMS\u002Fjarvis --skill friendli-ai",{"tool":39,"command":40},"bunx","bunx skills add TanNhatCMS\u002Fjarvis --skill friendli-ai",{"tool":42,"command":43},"pnpm","pnpm dlx skills add TanNhatCMS\u002Fjarvis --skill friendli-ai","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FTanNhatCMS\u002Fjarvis\u002Fblob\u002Fmain\u002Fskills\u002Fams-friendli-ai\u002FSKILL.md",true,[],[48,55,63,70,77,84],{"skillNo":49,"name":50,"nameZh":6,"description":51,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":52,"categories":53,"keywords":54,"publishedAt":19},"01M7EWYJR0J8DARY7E59Z6Y9MV","web-scrape","Turn web pages into LLM-ready markdown via the Firecrawl API (renders JS, anti-bot handling). Use when browser_open\u002Ffetch fails on JS-heavy pages, or when you need clean markdown\u002Fstructured extraction from a site. Requires FIRECRAWL_API_KEY.","https:\u002F\u002Fgithub.com\u002FTanNhatCMS\u002Fjarvis\u002Fblob\u002Fmain\u002Fskills\u002Fweb-scrape\u002FSKILL.md",[],[],{"skillNo":56,"name":57,"nameZh":6,"description":58,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":59,"sourceHtmlUrl":60,"categories":61,"keywords":62,"publishedAt":19},"01M7EWYBN5P6F2AJG3FR24T9BP","design-taste-frontend","tasteskill v2, the anti-slop frontend skill. Art direction for websites, distilled from a study of 247 curated gallery sites (One Page Love, Awwwards Sites of the Day). Deals researched directions (idea, composition, type, palette, photographs, section blueprints) from a library, enforces a strict no-slop house style with a real-browser inspector, and runs a look-and-fix loop on screenshots. Use this whenever you build, design, redesign, restyle, polish or review a landing page, marketing site, portfolio, product page, one-pager, hero or any public-facing web page, in any stack (HTML\u002FCSS, React, Next.js, Vue, Svelte, Astro, Tailwind) — also when the user says a page looks generic, templated, \"AI-made\", sloppy, or asks to make it look designed, premium, award-level or less like every other site. Not for dashboards or app screens.","https:\u002F\u002Fgithub.com\u002FTanNhatCMS\u002Fjarvis\u002Fblob\u002Fmain\u002Fskills\u002Ftaste-design\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FTanNhatCMS\u002Fjarvis\u002Fblob\u002Fmain\u002Fskills\u002Ftaste-design\u002FSKILL.md",[],[],{"skillNo":64,"name":65,"nameZh":6,"description":66,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":67,"categories":68,"keywords":69,"publishedAt":19},"01M7EWY8FQFNQ2GB7NZZWGSSDX","test-driven-development","Use when implementing any feature or bugfix, before writing implementation code","https:\u002F\u002Fgithub.com\u002FTanNhatCMS\u002Fjarvis\u002Fblob\u002Fmain\u002Fskills\u002Fsuperpowers-tdd\u002FSKILL.md",[],[],{"skillNo":71,"name":72,"nameZh":6,"description":73,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":74,"categories":75,"keywords":76,"publishedAt":19},"01M7EWY5SHBFRH1HDHVWM9K7TV","systematic-debugging","Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes","https:\u002F\u002Fgithub.com\u002FTanNhatCMS\u002Fjarvis\u002Fblob\u002Fmain\u002Fskills\u002Fsuperpowers-debugging\u002FSKILL.md",[],[],{"skillNo":78,"name":79,"nameZh":6,"description":80,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":81,"categories":82,"keywords":83,"publishedAt":19},"01M7EWY3NM0Z0JAKS7HDZ24K66","requesting-code-review","Use when completing tasks, implementing major features, or before merging to verify work meets requirements","https:\u002F\u002Fgithub.com\u002FTanNhatCMS\u002Fjarvis\u002Fblob\u002Fmain\u002Fskills\u002Fsuperpowers-code-review\u002FSKILL.md",[],[],{"skillNo":85,"name":86,"nameZh":6,"description":87,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":88,"categories":89,"keywords":90,"publishedAt":19},"01M7EWY0WRMWX4M9ZFY0WKC593","brainstorming","You MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation).","https:\u002F\u002Fgithub.com\u002FTanNhatCMS\u002Fjarvis\u002Fblob\u002Fmain\u002Fskills\u002Fsuperpowers-brainstorming\u002FSKILL.md",[],[]]