AGENT SKILL

dynamo-router-starter

Start or patch Dynamo router modes and run router endpoint smoke checks. Use for round-robin, KV-aware, least-loaded, or device-aware routing setup; use recipe-runner for recipe deployment and troubleshoot for failure diagnosis.

License 许可
Apache-2.0
最近更新
2026-11-14
Stars
★ 12
Forks
5

01 前言 Front Matter

name
dynamo-router-starter
license
Apache-2.0
metadata
{ "tags": [ "dynamo", "router", "smoke-test", "bring-up" ], "author": "Dan Gil <dagil@nvidia.com>" }
description
Start or patch Dynamo router modes and run router endpoint smoke checks. Use for round-robin, KV-aware, least-loaded, or device-aware routing setup; use recipe-runner for recipe deployment and troubleshoot for failure diagnosis.

02 资源文件 7

03 SKILL.md 正文

Dynamo Router Starter

<!-- SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. SPDX-License-Identifier: CC-BY-4.0 -->

Purpose

Make Dynamo routing feel easy by getting a baseline router mode running, enabling KV-aware routing when appropriate, and proving the endpoint works. Keep the user focused on exact commands and success signals, not router internals.

Prerequisites

  • Python 3.10+ with the dynamo package importable (python3 -m dynamo.frontend --help works).
  • For Kubernetes runs: kubectl configured with access to the target namespace and a deployed Dynamo recipe.
  • Network reachability to the frontend service (port-forward or direct).
  • A model already loaded into at least one worker (/v1/models returns at least one entry).

Required Inputs

Collect or infer:

  • local Python/CLI or Kubernetes recipe path
  • desired mode: round-robin, kv, least-loaded, device-aware-weighted, direct, or random
  • frontend port or Kubernetes frontend service
  • whether workers publish KV events; if not, use approximate KV mode
  • model name for smoke requests, if /v1/models cannot discover it

Instructions

1. Establish A Baseline

For local bring-up with already registered workers:

python3 -m dynamo.frontend --router-mode round-robin --http-port 8000

For Kubernetes, inspect the selected recipe deploy.yaml and locate the frontend service. If the recipe is not already deployed, use dynamo-recipe-runner first.

2. Enable KV Routing

For local frontend:

python3 -m dynamo.frontend --router-mode kv --http-port 8000

For Kubernetes, patch only the frontend service env:

envs:
  - name: DYN_ROUTER_MODE
    value: kv

If backend workers are not publishing KV cache events, set approximate mode instead of leaving the router waiting for events:

envs:
  - name: DYN_ROUTER_USE_KV_EVENTS
    value: "false"

3. Smoke Test

After port-forwarding the frontend service or starting local frontend, run:

python3 scripts/check_router_health.py \
  --base-url http://127.0.0.1:8000

This must verify /v1/models and, when a model is discoverable, one /v1/chat/completions request.

4. Compare Modes Carefully

When comparing round-robin vs KV routing:

  • use the same model, workers, prompt set, concurrency, and sampling settings
  • send repeated-prefix prompts if demonstrating KV reuse
  • label the result as a smoke comparison unless enough benchmark samples were collected
  • do not claim throughput improvement from a single chat request

If the endpoint is unhealthy or workers are missing, switch to dynamo-troubleshoot.

Available Scripts

Script Purpose Arguments
scripts/check_router_health.py Smoke-test /v1/models and one chat completion against a Dynamo frontend --base-url, --retries, --timeout

Invoke via the agentskills.io run_script() protocol:

run_script("scripts/check_router_health.py", args=["--base-url", "http://127.0.0.1:8000"])

Examples

Local KV-routed frontend on port 8000, then smoke-test it:

python3 -m dynamo.frontend --router-mode kv --http-port 8000 &
python3 scripts/check_router_health.py --base-url http://127.0.0.1:8000

Kubernetes-deployed frontend reachable via port-forward:

kubectl port-forward svc/qwen-vllm-disagg-frontend 8000:8000 -n dynamo-demo &
python3 scripts/check_router_health.py --base-url http://127.0.0.1:8000 --retries 3

Equivalent through the agent protocol:

run_script("scripts/check_router_health.py", args=["--base-url", "http://127.0.0.1:8000", "--retries", "3"])

Output Contract

Return:

  • mode selected and why
  • local command or Kubernetes env patch
  • frontend service or URL
  • smoke-test result
  • any limitation, such as approximate KV mode or missing worker KV events
  • next command to run for a fuller comparison

Limitations

  • Smoke test is one chat completion; it is not a benchmark. Use dynamo-benchmark for throughput/latency numbers.
  • KV-aware mode without worker KV-event publication degrades to approximate mode; this skill flags but does not fix the underlying worker config.
  • Mode comparisons require matched workloads; cross-mode latency claims need separate benchmark runs.

Troubleshooting

Symptom Likely cause Next step
/v1/models returns empty list No worker registered with the frontend Verify worker pods are Ready; confirm they connect to the same etcd/NATS
Smoke chat request times out Frontend up, workers not serving Switch to dynamo-troubleshoot; inspect worker logs
KV mode hangs Workers do not publish KV cache events Set DYN_ROUTER_USE_KV_EVENTS=false (approximate mode)
Connection refused on port-forward Port-forward dropped or wrong service name Re-run port-forward; verify the frontend service name matches the recipe

Benchmark

See BENCHMARK.md for the NVCARPS-EVAL performance report (auto-generated by the NVSkills CI pipeline). To refresh, re-run /nvskills-ci on an upstream PR touching this skill.

References

  • Read references/router-modes.md for the compact mode/env map.
  • Use scripts/check_router_health.py for endpoint smoke tests.

04 同仓库更多 Skills

stanfish06
stanfish06/skillquarium

zarr-python

Stores and queries chunked N-D scientific arrays with Zarr-Python 3, including codecs, sharding, S3/GCS storage, and NumPy/Dask/Xarray integration. Use for array layout, bounded I/O, format migration, or scientific metadata preservation.

★ 12
stanfish06
stanfish06/skillquarium

yeet

Publish local changes to GitHub by confirming scope, committing intentionally, pushing the branch, and opening a draft PR through the GitHub app from this plugin, with `gh` used only as a fallback where connector coverage is insufficient.

★ 12
stanfish06
stanfish06/skillquarium

xarray

N-dimensional labeled arrays and datasets in Python. Built on top of NumPy and Dask. It introduces labels in the form of dimensions, coordinates, and attributes on top of raw NumPy-like arrays, making data analysis in physical sciences more intuitive and less error-prone. Use for working with multi-dimensional scientific data, NetCDF/GRIB/Zarr files, climate/weather/oceanographic datasets, remote sensing, geospatial imaging, large out-of-memory datasets with Dask, and labeled array operations.

★ 12
stanfish06
stanfish06/skillquarium

writing-mstest-tests

Use when asked to fix, rewrite, update, improve, modernize, show corrected code for, or explain existing MSTest tests or MSTest-specific configuration. Use for "review" when corrected code or edits are wanted, even for one pasted assertion or passing tests with bad failure output. Covers expected/actual labels; generic Boolean, collection, string, numeric, null, identity, exception, hard-cast, and object[] checks; TestContext/lifecycle; timeout/cancellation; OS/CI conditions, retry, cleanup, parallelization, MSTest.Sdk project setup, and MSTESTxxxx. Honor the installed MSTest version. DO NOT USE to design new test cases (code-testing), perform report-only audits, create project files rather than explain MSTest setup, run tests, migrate frameworks, or handle non-MSTest/non-.NET code.

★ 12
stanfish06
stanfish06/skillquarium

wgs-prs

End-to-end WGS to polygenic risk score pipeline. Takes paired-end FASTQ files (or a pre-existing VCF) through nf-core/sarek for variant calling, applies VCF QC (normalisation, hard filtering, Ti/Tv and Het/Hom checks), then computes polygenic risk scores via the PGS Catalog. Fills the FASTQ to VCF gap upstream of the gwas-prs skill.

★ 12
stanfish06
stanfish06/skillquarium

webapp-testing

Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.

★ 12