[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M78HW5W4YBZAGYJMPPZRHCF6":3,"skill-same:01M78HW5W4YBZAGYJMPPZRHCF6":81},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":18,"linkOnly":19,"markdownContent":20,"markdownBody":21,"frontMatter":22,"refName":30,"commitSha":31,"skillDirectory":32,"forksCount":13,"repositoryHtmlUrl":33,"repositoryHomepageUrl":34,"ownerHtmlUrl":35,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":36,"install":37,"resources":50},"01M78HW5W4YBZAGYJMPPZRHCF6","dynamo-troubleshoot",null,"Diagnose failed or unhealthy Dynamo deployments. Use when pods, model-cache jobs, PVCs, workers, frontend\u002Frouter health, endpoints, or benchmark jobs fail; use recipe-runner\u002Frouter-starter before this for normal bring-up.","matheussluzz\u002FZeuZ-Agent","matheussluzz","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F209255758?v=4","01M78HKHH1D1ASC9KC9BG583DH","01M78HKHHP6W7XRXQM2SBCAM3R",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Fdynamo-troubleshoot\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: dynamo-troubleshoot\ndescription: Diagnose failed or unhealthy Dynamo deployments. Use when pods, model-cache jobs, PVCs, workers, frontend\u002Frouter health, endpoints, or benchmark jobs fail; use recipe-runner\u002Frouter-starter before this for normal bring-up.\nlicense: Apache-2.0\nmetadata:\n  author: Dan Gil \u003Cdagil@nvidia.com>\n  tags:\n    - dynamo\n    - kubernetes\n    - troubleshooting\n    - day-2\n---\n\n# Dynamo Troubleshoot\n\n\u003C!--\nSPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.\nSPDX-License-Identifier: CC-BY-4.0\n-->\n\n## Purpose\n\nTurn a Dynamo failure into a clear problem class, strongest signal, and next\naction. Start with read-only evidence, avoid secrets, and fix one layer at a\ntime.\n\n## Prerequisites\n\n- Python 3.10+ on the operator machine.\n- `kubectl` configured with read access to the target namespace.\n- Permission to read pods, events, jobs, PVCs, and `DynamoGraphDeployment` resources (NOT secrets).\n- Network reachability to the cluster API server.\n\n## Instructions\n\n### 1. Collect A Read-Only Bundle\n\nRun:\n\n```bash\npython3 scripts\u002Fcollect_dynamo_debug_bundle.py \\\n  --namespace \"${NAMESPACE}\"\n```\n\nIf the user names a deployment, include it:\n\n```bash\npython3 scripts\u002Fcollect_dynamo_debug_bundle.py \\\n  --namespace \"${NAMESPACE}\" \\\n  --deployment-name \u003Cdeployment-name>\n```\n\nDo not collect Kubernetes secrets. Do not print Hugging Face tokens.\n\n### 2. Classify The Failure\n\nUse `references\u002Ffailure-decision-tree.md` and classify into one primary bucket:\n\n- cluster\u002Fplatform\n- namespace\u002Fsecret\n- model cache\u002FPVC\u002Fdownload\n- image pull\u002Fruntime image\n- GPU scheduling\u002Fresources\n- operator\u002FDynamoGraphDeployment reconciliation\n- frontend\u002Frouter\n- worker\u002Fbackend\n- endpoint\u002FAPI\n- benchmark\u002Fperf job\n\n### 3. Debug Top Down\n\nCheck in this order:\n\n1. namespace, storage class, GPU nodes, and HF secret existence\n2. PVC and model-download job\n3. `DynamoGraphDeployment` status and events\n4. pod status, `describe pod`, and container logs\n5. frontend service and port-forward\n6. `\u002Fv1\u002Fmodels`\n7. `\u002Fv1\u002Fchat\u002Fcompletions`\n8. benchmark job only after endpoint smoke test passes\n\n### 4. Fix One Layer At A Time\n\nPrefer the smallest reversible change:\n\n- create missing namespace or HF secret\n- patch `storageClassName`\n- patch image tag or image pull secret\n- reduce GPU request only if the recipe can still be valid\n- switch KV router to approximate mode only if workers do not publish events\n- restart failed jobs after fixing the underlying config\n\nAfter each fix, rerun the relevant readiness check before moving deeper.\n\n## Available Scripts\n\n| Script | Purpose | Arguments |\n|---|---|---|\n| `scripts\u002Fcollect_dynamo_debug_bundle.py` | Collect a read-only debug bundle (pods, events, jobs, PVCs, CR status) | `--namespace`, `--deployment-name`, `--output-dir` |\n\nInvoke via the agentskills.io `run_script()` protocol:\n\n```python\nrun_script(\"scripts\u002Fcollect_dynamo_debug_bundle.py\", args=[\"--namespace\", \"dynamo-demo\"])\n```\n\n## Examples\n\nCollect everything in a namespace for triage:\n\n```bash\npython3 scripts\u002Fcollect_dynamo_debug_bundle.py --namespace dynamo-demo\n```\n\nScope to a single failing deployment:\n\n```bash\npython3 scripts\u002Fcollect_dynamo_debug_bundle.py \\\n  --namespace dynamo-demo \\\n  --deployment-name qwen-vllm-disagg\n```\n\nEquivalent through the agent protocol:\n\n```python\nrun_script(\"scripts\u002Fcollect_dynamo_debug_bundle.py\", args=[\"--namespace\", \"dynamo-demo\", \"--deployment-name\", \"qwen-vllm-disagg\"])\n```\n\n## Output Contract\n\nReturn:\n\n- problem class\n- evidence checked\n- strongest signal\n- likely cause\n- exact next command or patch\n- what was ruled out\n- whether it is safe to continue deployment or benchmarking\n\n## Limitations\n\n- Read-only. Never mutates the cluster; remediation commands are returned, not executed.\n- Will not collect secrets or print Hugging Face tokens; some failure modes (auth) may need user-side inspection.\n- Bundle size grows with deployment size; on very large namespaces, scope with `--deployment-name`.\n- Does not validate disagg transport — use `dynamo-interconnect-check` for that.\n\n## Troubleshooting\n\n| Symptom | Likely cause | Next step |\n|---|---|---|\n| `kubectl` returns Forbidden on events\u002Fpods | Service account lacks read RBAC | Ask operator for read-only role binding on the namespace |\n| Bundle missing `DynamoGraphDeployment` status | Operator not installed or different namespace | Verify `dynamo-platform` operator is installed and watching the namespace |\n| Model-download job in `Pending` | PVC unbound or HF secret missing | Fix PVC binding or create the named HF secret, then rerun the job |\n| Worker pods `CrashLoopBackOff` | Image\u002Fruntime mismatch or GPU not available | Inspect container logs; check `nvidia.com\u002Fgpu` allocatable on nodes |\n\n## Benchmark\n\nSee `BENCHMARK.md` for the NVCARPS-EVAL performance report (auto-generated by the NVSkills CI pipeline). To refresh, re-run `\u002Fnvskills-ci` on an upstream PR touching this skill.\n\n## References\n\n- Read `references\u002Ffailure-decision-tree.md` for bucket-specific checks.\n- Use `scripts\u002Fcollect_dynamo_debug_bundle.py` for read-only bundle collection.\n","# Dynamo Troubleshoot\n\n\u003C!--\nSPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.\nSPDX-License-Identifier: CC-BY-4.0\n-->\n\n## Purpose\n\nTurn a Dynamo failure into a clear problem class, strongest signal, and next\naction. Start with read-only evidence, avoid secrets, and fix one layer at a\ntime.\n\n## Prerequisites\n\n- Python 3.10+ on the operator machine.\n- `kubectl` configured with read access to the target namespace.\n- Permission to read pods, events, jobs, PVCs, and `DynamoGraphDeployment` resources (NOT secrets).\n- Network reachability to the cluster API server.\n\n## Instructions\n\n### 1. Collect A Read-Only Bundle\n\nRun:\n\n```bash\npython3 scripts\u002Fcollect_dynamo_debug_bundle.py \\\n  --namespace \"${NAMESPACE}\"\n```\n\nIf the user names a deployment, include it:\n\n```bash\npython3 scripts\u002Fcollect_dynamo_debug_bundle.py \\\n  --namespace \"${NAMESPACE}\" \\\n  --deployment-name \u003Cdeployment-name>\n```\n\nDo not collect Kubernetes secrets. Do not print Hugging Face tokens.\n\n### 2. Classify The Failure\n\nUse `references\u002Ffailure-decision-tree.md` and classify into one primary bucket:\n\n- cluster\u002Fplatform\n- namespace\u002Fsecret\n- model cache\u002FPVC\u002Fdownload\n- image pull\u002Fruntime image\n- GPU scheduling\u002Fresources\n- operator\u002FDynamoGraphDeployment reconciliation\n- frontend\u002Frouter\n- worker\u002Fbackend\n- endpoint\u002FAPI\n- benchmark\u002Fperf job\n\n### 3. Debug Top Down\n\nCheck in this order:\n\n1. namespace, storage class, GPU nodes, and HF secret existence\n2. PVC and model-download job\n3. `DynamoGraphDeployment` status and events\n4. pod status, `describe pod`, and container logs\n5. frontend service and port-forward\n6. `\u002Fv1\u002Fmodels`\n7. `\u002Fv1\u002Fchat\u002Fcompletions`\n8. benchmark job only after endpoint smoke test passes\n\n### 4. Fix One Layer At A Time\n\nPrefer the smallest reversible change:\n\n- create missing namespace or HF secret\n- patch `storageClassName`\n- patch image tag or image pull secret\n- reduce GPU request only if the recipe can still be valid\n- switch KV router to approximate mode only if workers do not publish events\n- restart failed jobs after fixing the underlying config\n\nAfter each fix, rerun the relevant readiness check before moving deeper.\n\n## Available Scripts\n\n| Script | Purpose | Arguments |\n|---|---|---|\n| `scripts\u002Fcollect_dynamo_debug_bundle.py` | Collect a read-only debug bundle (pods, events, jobs, PVCs, CR status) | `--namespace`, `--deployment-name`, `--output-dir` |\n\nInvoke via the agentskills.io `run_script()` protocol:\n\n```python\nrun_script(\"scripts\u002Fcollect_dynamo_debug_bundle.py\", args=[\"--namespace\", \"dynamo-demo\"])\n```\n\n## Examples\n\nCollect everything in a namespace for triage:\n\n```bash\npython3 scripts\u002Fcollect_dynamo_debug_bundle.py --namespace dynamo-demo\n```\n\nScope to a single failing deployment:\n\n```bash\npython3 scripts\u002Fcollect_dynamo_debug_bundle.py \\\n  --namespace dynamo-demo \\\n  --deployment-name qwen-vllm-disagg\n```\n\nEquivalent through the agent protocol:\n\n```python\nrun_script(\"scripts\u002Fcollect_dynamo_debug_bundle.py\", args=[\"--namespace\", \"dynamo-demo\", \"--deployment-name\", \"qwen-vllm-disagg\"])\n```\n\n## Output Contract\n\nReturn:\n\n- problem class\n- evidence checked\n- strongest signal\n- likely cause\n- exact next command or patch\n- what was ruled out\n- whether it is safe to continue deployment or benchmarking\n\n## Limitations\n\n- Read-only. Never mutates the cluster; remediation commands are returned, not executed.\n- Will not collect secrets or print Hugging Face tokens; some failure modes (auth) may need user-side inspection.\n- Bundle size grows with deployment size; on very large namespaces, scope with `--deployment-name`.\n- Does not validate disagg transport — use `dynamo-interconnect-check` for that.\n\n## Troubleshooting\n\n| Symptom | Likely cause | Next step |\n|---|---|---|\n| `kubectl` returns Forbidden on events\u002Fpods | Service account lacks read RBAC | Ask operator for read-only role binding on the namespace |\n| Bundle missing `DynamoGraphDeployment` status | Operator not installed or different namespace | Verify `dynamo-platform` operator is installed and watching the namespace |\n| Model-download job in `Pending` | PVC unbound or HF secret missing | Fix PVC binding or create the named HF secret, then rerun the job |\n| Worker pods `CrashLoopBackOff` | Image\u002Fruntime mismatch or GPU not available | Inspect container logs; check `nvidia.com\u002Fgpu` allocatable on nodes |\n\n## Benchmark\n\nSee `BENCHMARK.md` for the NVCARPS-EVAL performance report (auto-generated by the NVSkills CI pipeline). To refresh, re-run `\u002Fnvskills-ci` on an upstream PR touching this skill.\n\n## References\n\n- Read `references\u002Ffailure-decision-tree.md` for bucket-specific checks.\n- Use `scripts\u002Fcollect_dynamo_debug_bundle.py` for read-only bundle collection.\n",{"name":5,"license":14,"metadata":23,"description":7},{"tags":24,"author":29},[25,26,27,28],"dynamo","kubernetes","troubleshooting","day-2","Dan Gil \u003Cdagil@nvidia.com>","main","fc4f2c7759ae400c66cd38656db70ead882a0d08","catalog\u002Fbundles\u002Fnvidia\u002Fdynamo-troubleshoot","https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent","https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent#readme","https:\u002F\u002Fgithub.com\u002Fmatheussluzz","2026-08-27 05:21:51",{"source":8,"skillName":5,"commands":38,"prompt":48,"zipAvailable":49},[39,42,45],{"tool":40,"command":41},"npx","npx skills add matheussluzz\u002FZeuZ-Agent --skill dynamo-troubleshoot",{"tool":43,"command":44},"bunx","bunx skills add matheussluzz\u002FZeuZ-Agent --skill dynamo-troubleshoot",{"tool":46,"command":47},"pnpm","pnpm dlx skills add matheussluzz\u002FZeuZ-Agent --skill dynamo-troubleshoot","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Fdynamo-troubleshoot\u002FSKILL.md",true,[51,57,61,67,71,77],{"path":52,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":55,"sourceHtmlUrl":56},"BENCHMARK.md","REFERENCE","参考文档",3022,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Fdynamo-troubleshoot\u002FBENCHMARK.md",{"path":58,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":59,"sourceHtmlUrl":60},"references\u002Ffailure-decision-tree.md",3834,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Fdynamo-troubleshoot\u002Freferences\u002Ffailure-decision-tree.md",{"path":62,"resourceType":63,"resourceTypeDesc":64,"sizeBytes":65,"sourceHtmlUrl":66},"scripts\u002Fcollect_dynamo_debug_bundle.py","SCRIPT","脚本",8539,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Fdynamo-troubleshoot\u002Fscripts\u002Fcollect_dynamo_debug_bundle.py",{"path":68,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":69,"sourceHtmlUrl":70},"skill-card.md",2844,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Fdynamo-troubleshoot\u002Fskill-card.md",{"path":72,"resourceType":73,"resourceTypeDesc":74,"sizeBytes":75,"sourceHtmlUrl":76},"skill.oms.sig","OTHER","其他",5097,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Fdynamo-troubleshoot\u002Fskill.oms.sig",{"path":78,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":79,"sourceHtmlUrl":80},"zeuz.manifest.yaml",172,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Fdynamo-troubleshoot\u002Fzeuz.manifest.yaml",[82,90,97,104,111,118],{"skillNo":83,"name":84,"nameZh":6,"description":85,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":86,"licenseFileUrl":6,"sourceHtmlUrl":87,"categories":88,"keywords":89,"publishedAt":18},"01M78JQ2R0QR8H2Z6662RFX804","gamma","Fixture skill gamma for discovery benchmarks.","MIT","https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Ftest\u002Ffixtures\u002Fskill-registry-mini\u002Fskills\u002Fgamma\u002FSKILL.md",[],[],{"skillNo":91,"name":92,"nameZh":6,"description":93,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":86,"licenseFileUrl":6,"sourceHtmlUrl":94,"categories":95,"keywords":96,"publishedAt":18},"01M78JQ1KCVZ15APZG3FFF29K0","beta","Fixture skill beta for discovery benchmarks.","https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Ftest\u002Ffixtures\u002Fskill-registry-mini\u002Fskills\u002Fbeta\u002FSKILL.md",[],[],{"skillNo":98,"name":99,"nameZh":6,"description":100,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":86,"licenseFileUrl":6,"sourceHtmlUrl":101,"categories":102,"keywords":103,"publishedAt":18},"01M78JQ0FJTK4Q9JKZMX9SVSEN","alpha","Fixture skill alpha for discovery benchmarks.","https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Ftest\u002Ffixtures\u002Fskill-registry-mini\u002Fskills\u002Falpha\u002FSKILL.md",[],[],{"skillNo":105,"name":106,"nameZh":6,"description":107,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":86,"licenseFileUrl":6,"sourceHtmlUrl":108,"categories":109,"keywords":110,"publishedAt":18},"01M78JPYVYPY7R8QS10KADQPWE","writing-for-agents","Writing documents for agents. Use when creating or editing skills, or modifying AGENTS.md or CLAUDE.md.","https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fskills\u002Fwriting-for-agents\u002FSKILL.md",[],[],{"skillNo":112,"name":113,"nameZh":6,"description":114,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":86,"licenseFileUrl":6,"sourceHtmlUrl":115,"categories":116,"keywords":117,"publishedAt":18},"01M78JPVYHB88RF5ATZVGX84RR","wizard","Generate an interactive bash wizard that walks a human through steps only they can perform. Use when provisioning infrastructure, setting up credentials or CI secrets, walking an unfamiliar third-party dashboard, or running a one-off migration or cutover. Don't invoke this for steps the agent can perform itself.","https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fskills\u002Fwizard\u002FSKILL.md",[],[],{"skillNo":119,"name":120,"nameZh":6,"description":121,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":86,"licenseFileUrl":6,"sourceHtmlUrl":122,"categories":123,"keywords":124,"publishedAt":18},"01M78JPSSWNTZW89DWGJTX7716","wayfinder","Plan a huge chunk of work (more than one agent session can hold) as a shared map of decision tickets on your issue tracker, and resolve them one at a time until the way to the destination is clear.","https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fskills\u002Fwayfinder\u002FSKILL.md",[],[]]