dynamo-router-starter
Start or patch Dynamo router modes and run router endpoint smoke checks. Use for round-robin, KV-aware, least-loaded, or device-aware routing setup; use recipe-runner for recipe deployment and troubleshoot for failure diagnosis.
- License 许可
- Apache-2.0
- 最近更新
- 2026-07-19
- Stars
- ★ 1
- Forks
- 1
01 前言 Front Matter
- name
- dynamo-router-starter
- license
- Apache-2.0
- metadata
- { "tags": [ "dynamo", "router", "smoke-test", "bring-up" ], "author": "Dan Gil <dagil@nvidia.com>" }
- description
- Start or patch Dynamo router modes and run router endpoint smoke checks. Use for round-robin, KV-aware, least-loaded, or device-aware routing setup; use recipe-runner for recipe deployment and troubleshoot for failure diagnosis.
02 资源文件 7
- 参考文档agents/openai.yaml303 B
- 参考文档BENCHMARK.md2.6 KB
- 参考文档evals/evals.json3.0 KB
- 参考文档references/router-modes.md2.1 KB
- 脚本scripts/check_router_health.py4.7 KB
- 参考文档skill-card.md2.4 KB
- 其他skill.oms.sig5.0 KB
03 SKILL.md 正文
Dynamo Router Starter
<!-- SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. SPDX-License-Identifier: CC-BY-4.0 -->
Purpose
Make Dynamo routing feel easy by getting a baseline router mode running, enabling KV-aware routing when appropriate, and proving the endpoint works. Keep the user focused on exact commands and success signals, not router internals.
Prerequisites
- Python 3.10+ with the
dynamopackage importable (python3 -m dynamo.frontend --helpworks). - For Kubernetes runs:
kubectlconfigured with access to the target namespace and a deployed Dynamo recipe. - Network reachability to the frontend service (port-forward or direct).
- A model already loaded into at least one worker (
/v1/modelsreturns at least one entry).
Required Inputs
Collect or infer:
- local Python/CLI or Kubernetes recipe path
- desired mode:
round-robin,kv,least-loaded,device-aware-weighted,direct, orrandom - frontend port or Kubernetes frontend service
- whether workers publish KV events; if not, use approximate KV mode
- model name for smoke requests, if
/v1/modelscannot discover it
Instructions
1. Establish A Baseline
For local bring-up with already registered workers:
python3 -m dynamo.frontend --router-mode round-robin --http-port 8000
For Kubernetes, inspect the selected recipe deploy.yaml and locate the
frontend service. If the recipe is not already deployed, use
dynamo-recipe-runner first.
2. Enable KV Routing
For local frontend:
python3 -m dynamo.frontend --router-mode kv --http-port 8000
For Kubernetes, patch only the frontend service env:
envs:
- name: DYN_ROUTER_MODE
value: kv
If backend workers are not publishing KV cache events, set approximate mode instead of leaving the router waiting for events:
envs:
- name: DYN_ROUTER_USE_KV_EVENTS
value: "false"
3. Smoke Test
After port-forwarding the frontend service or starting local frontend, run:
python3 scripts/check_router_health.py \
--base-url http://127.0.0.1:8000
This must verify /v1/models and, when a model is discoverable, one
/v1/chat/completions request.
4. Compare Modes Carefully
When comparing round-robin vs KV routing:
- use the same model, workers, prompt set, concurrency, and sampling settings
- send repeated-prefix prompts if demonstrating KV reuse
- label the result as a smoke comparison unless enough benchmark samples were collected
- do not claim throughput improvement from a single chat request
If the endpoint is unhealthy or workers are missing, switch to
dynamo-troubleshoot.
Available Scripts
| Script | Purpose | Arguments |
|---|---|---|
scripts/check_router_health.py |
Smoke-test /v1/models and one chat completion against a Dynamo frontend |
--base-url, --retries, --timeout |
Invoke via the agentskills.io run_script() protocol:
run_script("scripts/check_router_health.py", args=["--base-url", "http://127.0.0.1:8000"])
Examples
Local KV-routed frontend on port 8000, then smoke-test it:
python3 -m dynamo.frontend --router-mode kv --http-port 8000 &
python3 scripts/check_router_health.py --base-url http://127.0.0.1:8000
Kubernetes-deployed frontend reachable via port-forward:
kubectl port-forward svc/qwen-vllm-disagg-frontend 8000:8000 -n dynamo-demo &
python3 scripts/check_router_health.py --base-url http://127.0.0.1:8000 --retries 3
Equivalent through the agent protocol:
run_script("scripts/check_router_health.py", args=["--base-url", "http://127.0.0.1:8000", "--retries", "3"])
Output Contract
Return:
- mode selected and why
- local command or Kubernetes env patch
- frontend service or URL
- smoke-test result
- any limitation, such as approximate KV mode or missing worker KV events
- next command to run for a fuller comparison
Limitations
- Smoke test is one chat completion; it is not a benchmark. Use
dynamo-benchmarkfor throughput/latency numbers. - KV-aware mode without worker KV-event publication degrades to approximate mode; this skill flags but does not fix the underlying worker config.
- Mode comparisons require matched workloads; cross-mode latency claims need separate benchmark runs.
Troubleshooting
| Symptom | Likely cause | Next step |
|---|---|---|
/v1/models returns empty list |
No worker registered with the frontend | Verify worker pods are Ready; confirm they connect to the same etcd/NATS |
| Smoke chat request times out | Frontend up, workers not serving | Switch to dynamo-troubleshoot; inspect worker logs |
| KV mode hangs | Workers do not publish KV cache events | Set DYN_ROUTER_USE_KV_EVENTS=false (approximate mode) |
| Connection refused on port-forward | Port-forward dropped or wrong service name | Re-run port-forward; verify the frontend service name matches the recipe |
Benchmark
See BENCHMARK.md for the NVCARPS-EVAL performance report (auto-generated by the NVSkills CI pipeline). To refresh, re-run /nvskills-ci on an upstream PR touching this skill.
References
- Read
references/router-modes.mdfor the compact mode/env map. - Use
scripts/check_router_health.pyfor endpoint smoke tests.
04 同仓库更多 Skills
Zotero
Use Zotero Desktop from Codex to enable/probe the local API, search a local Zotero library, list items/collections/tags, export BibTeX, insert citation keys into LaTeX or Markdown drafts, read indexed full text when requested, and import BibTeX/RIS records into Zotero through the connector server. Use when the user mentions Zotero, citations, references.bib, BibTeX export, local Zotero API, localhost:23119, or adding citations from a Zotero library.
zoom-apps-sdk
Use when using Apps SDK.
youtube
Obtain accurate, timestamped YouTube video transcripts (preferring manual captions over auto-generated) as the primary deliverable, then derive summaries, key claims, or notes from that transcript. Use when the user provides a YouTube URL (or video ID) and wants the transcript, a faithful summary, key claims with attribution, timestamped notes, or "what does this video actually say". The skill is transcript-first: never substitute metadata, title, description, or search snippets for the spoken content. Triggers: "youtube transcript", "get the transcript of this video", "summarize this youtube video", "what did they say in https://youtube.com/...", "key claims from this talk".
yeet
Publish local changes to GitHub by confirming scope, committing intentionally, pushing the branch, and opening a draft PR through the GitHub app from this plugin, with `gh` used only as a fallback where connector coverage is insufficient.
x-post-creator
Create X-ready post threads and co-located story packages from the active conversation or referenced project material. Use when the user says turn this into an X-post, /x-post, make this into an X thread, or wants plain-language social posts with supporting images saved to docs/x-posts/[story-name]/.
x-api
Search, read, and write X posts through the local X API MCP server (the same backend used by the Codex x-api skill). Use for searching X/x.com, reading threads and replies, creating lists, staging/publishing posts for @alltheputs, finding high-visibility reply targets for research outreach, ingesting public context, or any request that mentions /x-api or "search X".