[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M753JNEEGWDMXT7GDAWSC3YN":3,"skill-same:01M753JNEEGWDMXT7GDAWSC3YN":71},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":18,"linkOnly":19,"markdownContent":20,"markdownBody":21,"frontMatter":22,"refName":27,"commitSha":28,"skillDirectory":29,"forksCount":30,"repositoryHtmlUrl":31,"repositoryHomepageUrl":6,"ownerHtmlUrl":32,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":33,"install":34,"resources":46},"01M753JNEEGWDMXT7GDAWSC3YN","distributed-training-backends",null,"Select, configure, and diagnose Accelerate distributed training backends including DeepSpeed, FSDP\u002FFSDP2, Megatron-LM, torch native parallelism, TPU\u002FXLA, FP8, quantization, compilation, Local SGD, and DDP communication hooks.","VectorSpaceLab\u002FAREX-Skill","VectorSpaceLab","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F180637102?v=4","01M7538C5WPSJGWF7KHR4EFZ9F","01M7538C6R2GFP53NPXTCZNBRT",351,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Frepositories\u002Frepo-skills\u002Faccelerate\u002Fsub-skills\u002Fdistributed-training-backends\u002FSKILL.md",[],[],"2026-11-12 13:55:43",false,"---\nname: distributed-training-backends\ndescription: \"Select, configure, and diagnose Accelerate distributed training\n  backends including DeepSpeed, FSDP\u002FFSDP2, Megatron-LM, torch native\n  parallelism, TPU\u002FXLA, FP8, quantization, compilation, Local SGD, and DDP\n  communication hooks.\"\ndisable-model-invocation: true\nmetadata:\n  disco-role: operating\nlicense: Apache 2.0\n---\n\n# Distributed Training Backends\n\nUse this sub-skill when an agent must choose or configure an Accelerate backend beyond a plain single-process training loop: DeepSpeed ZeRO, FSDP\u002FFSDP2, Megatron-LM, torch native tensor\u002Fcontext\u002Fsequence parallelism, TPU\u002FXLA, Gaudi\u002FHPU, FP8 or low precision, bitsandbytes\u002Ftorchao quantization, `torch.compile`, Local SGD, or DDP communication hooks.\n\n## Routing\n\n- For `accelerate config`, `accelerate launch`, config-file locations, or generic CLI flag syntax, use `..\u002Fconfiguration-and-cli\u002F` first, then return here for backend-specific keys and compatibility.\n- For where to instantiate `Accelerator`, how to call `prepare`, `backward`, `accumulate`, or unwrap models in a training loop, use `..\u002Ftraining-loop-integration\u002F`.\n- For `save_state`, `load_state`, FSDP weight merging, DeepSpeed checkpoint folders, or tracker artifacts, use `..\u002Fcheckpointing-and-tracking\u002F`.\n- For backend choice, plugin objects, optional dependencies, hardware constraints, or config-vs-plugin precedence, stay in this sub-skill.\n\n## Fast Backend Triage\n\n- Choose **DeepSpeed** for ZeRO optimizer\u002Fparameter partitioning, CPU\u002FNVMe offload, DeepSpeed-specific optimizers\u002Fschedulers, or existing DeepSpeed JSON; see `references\u002Fdeepspeed-fsdp.md`.\n- Choose **FSDP\u002FFSDP2** for PyTorch-native sharding, `transformer_based_wrap`, state-dict control, CPU-RAM-efficient Transformers loading, and FSDP2 composition with native parallelism; see `references\u002Fdeepspeed-fsdp.md`.\n- Choose **torch native parallelism** when using `parallelism_config` for data replication\u002Fsharding, tensor parallelism, context parallelism, or DeepSpeed sequence parallelism; see `references\u002Fparallelism-and-precision.md`.\n- Choose **Megatron-LM** only when the training stack is Megatron-aware and needs tensor\u002Fpipeline\u002Fsequence parallelism plus Megatron dummy optimizer\u002Fscheduler integration.\n- Choose **TPU\u002FXLA** only when `torch_xla` and TPU runtime are available; CPU-only inspection can validate config shape, not TPU execution.\n- Choose **FP8\u002Fquantization\u002Fcompile\u002Fcommunication options** as add-ons after the distributed backend is selected; verify optional dependency and hardware support before promising speedups.\n\n## Required References\n\n- `references\u002Fdeepspeed-fsdp.md` — DeepSpeed, FSDP\u002FFSDP2 selection, launch\u002Fconfig implications, and plugin usage.\n- `references\u002Fparallelism-and-precision.md` — torch native parallelism, Megatron-LM, TPU\u002FXLA, Gaudi\u002FHPU, FP8, low precision, quantization, compilation, Local SGD, and DDP hooks.\n- `references\u002Fapi-reference.md` — primary Accelerate plugin\u002Fconfig classes, common constructor fields, environment\u002Fconfig key mapping, and minimal snippets.\n- `references\u002Ftroubleshooting.md` — dependency, hardware, ZeRO\u002Foffload, FSDP wrapping, FP8 mismatch, multi-node, and precedence diagnostics.\n\n## Bundled Helper\n\nUse `scripts\u002Fvalidate_backend_config.py` to statically validate JSON or YAML snippets for backend key shape and common contradictions:\n\n```bash\npython sub-skills\u002Fdistributed-training-backends\u002Fscripts\u002Fvalidate_backend_config.py path\u002Fto\u002Fconfig.yaml\npython sub-skills\u002Fdistributed-training-backends\u002Fscripts\u002Fvalidate_backend_config.py --format deepspeed deepspeed.json\npython sub-skills\u002Fdistributed-training-backends\u002Fscripts\u002Fvalidate_backend_config.py --check-imports config.yaml\n```\n\nThe helper does not launch distributed jobs, import Accelerate, initialize process groups, or require GPUs\u002FTPUs. It reports missing optional packages only as hints when `--check-imports` is passed.\n\n## Safety Notes\n\n- Label DeepSpeed, FSDP2, Megatron-LM, TPU\u002FXLA, FP8, bitsandbytes, torchao, transformer-engine, ms-amp, and Gaudi as optional dependency or hardware paths.\n- Do not promise that config parsing proves runtime correctness; distributed execution still depends on process count, network\u002Frendezvous setup, device availability, backend package versions, and model compatibility.\n- Prefer explicit plugin objects in Python when code must be reproducible, and prefer config files when users rely on `accelerate config`\u002F`launch` workflows. If both are present, check precedence and mismatch rules in `references\u002Ftroubleshooting.md`.\n","# Distributed Training Backends\n\nUse this sub-skill when an agent must choose or configure an Accelerate backend beyond a plain single-process training loop: DeepSpeed ZeRO, FSDP\u002FFSDP2, Megatron-LM, torch native tensor\u002Fcontext\u002Fsequence parallelism, TPU\u002FXLA, Gaudi\u002FHPU, FP8 or low precision, bitsandbytes\u002Ftorchao quantization, `torch.compile`, Local SGD, or DDP communication hooks.\n\n## Routing\n\n- For `accelerate config`, `accelerate launch`, config-file locations, or generic CLI flag syntax, use `..\u002Fconfiguration-and-cli\u002F` first, then return here for backend-specific keys and compatibility.\n- For where to instantiate `Accelerator`, how to call `prepare`, `backward`, `accumulate`, or unwrap models in a training loop, use `..\u002Ftraining-loop-integration\u002F`.\n- For `save_state`, `load_state`, FSDP weight merging, DeepSpeed checkpoint folders, or tracker artifacts, use `..\u002Fcheckpointing-and-tracking\u002F`.\n- For backend choice, plugin objects, optional dependencies, hardware constraints, or config-vs-plugin precedence, stay in this sub-skill.\n\n## Fast Backend Triage\n\n- Choose **DeepSpeed** for ZeRO optimizer\u002Fparameter partitioning, CPU\u002FNVMe offload, DeepSpeed-specific optimizers\u002Fschedulers, or existing DeepSpeed JSON; see `references\u002Fdeepspeed-fsdp.md`.\n- Choose **FSDP\u002FFSDP2** for PyTorch-native sharding, `transformer_based_wrap`, state-dict control, CPU-RAM-efficient Transformers loading, and FSDP2 composition with native parallelism; see `references\u002Fdeepspeed-fsdp.md`.\n- Choose **torch native parallelism** when using `parallelism_config` for data replication\u002Fsharding, tensor parallelism, context parallelism, or DeepSpeed sequence parallelism; see `references\u002Fparallelism-and-precision.md`.\n- Choose **Megatron-LM** only when the training stack is Megatron-aware and needs tensor\u002Fpipeline\u002Fsequence parallelism plus Megatron dummy optimizer\u002Fscheduler integration.\n- Choose **TPU\u002FXLA** only when `torch_xla` and TPU runtime are available; CPU-only inspection can validate config shape, not TPU execution.\n- Choose **FP8\u002Fquantization\u002Fcompile\u002Fcommunication options** as add-ons after the distributed backend is selected; verify optional dependency and hardware support before promising speedups.\n\n## Required References\n\n- `references\u002Fdeepspeed-fsdp.md` — DeepSpeed, FSDP\u002FFSDP2 selection, launch\u002Fconfig implications, and plugin usage.\n- `references\u002Fparallelism-and-precision.md` — torch native parallelism, Megatron-LM, TPU\u002FXLA, Gaudi\u002FHPU, FP8, low precision, quantization, compilation, Local SGD, and DDP hooks.\n- `references\u002Fapi-reference.md` — primary Accelerate plugin\u002Fconfig classes, common constructor fields, environment\u002Fconfig key mapping, and minimal snippets.\n- `references\u002Ftroubleshooting.md` — dependency, hardware, ZeRO\u002Foffload, FSDP wrapping, FP8 mismatch, multi-node, and precedence diagnostics.\n\n## Bundled Helper\n\nUse `scripts\u002Fvalidate_backend_config.py` to statically validate JSON or YAML snippets for backend key shape and common contradictions:\n\n```bash\npython sub-skills\u002Fdistributed-training-backends\u002Fscripts\u002Fvalidate_backend_config.py path\u002Fto\u002Fconfig.yaml\npython sub-skills\u002Fdistributed-training-backends\u002Fscripts\u002Fvalidate_backend_config.py --format deepspeed deepspeed.json\npython sub-skills\u002Fdistributed-training-backends\u002Fscripts\u002Fvalidate_backend_config.py --check-imports config.yaml\n```\n\nThe helper does not launch distributed jobs, import Accelerate, initialize process groups, or require GPUs\u002FTPUs. It reports missing optional packages only as hints when `--check-imports` is passed.\n\n## Safety Notes\n\n- Label DeepSpeed, FSDP2, Megatron-LM, TPU\u002FXLA, FP8, bitsandbytes, torchao, transformer-engine, ms-amp, and Gaudi as optional dependency or hardware paths.\n- Do not promise that config parsing proves runtime correctness; distributed execution still depends on process count, network\u002Frendezvous setup, device availability, backend package versions, and model compatibility.\n- Prefer explicit plugin objects in Python when code must be reproducible, and prefer config files when users rely on `accelerate config`\u002F`launch` workflows. If both are present, check precedence and mismatch rules in `references\u002Ftroubleshooting.md`.\n",{"name":5,"license":23,"metadata":24,"description":7,"disable-model-invocation":26},"Apache 2.0",{"disco-role":25},"operating",true,"main","ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6","skills\u002Frepositories\u002Frepo-skills\u002Faccelerate\u002Fsub-skills\u002Fdistributed-training-backends",27,"https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab","2026-09-03 10:25:21",{"source":8,"skillName":5,"commands":35,"prompt":45,"zipAvailable":26},[36,39,42],{"tool":37,"command":38},"npx","npx skills add VectorSpaceLab\u002FAREX-Skill --skill distributed-training-backends",{"tool":40,"command":41},"bunx","bunx skills add VectorSpaceLab\u002FAREX-Skill --skill distributed-training-backends",{"tool":43,"command":44},"pnpm","pnpm dlx skills add VectorSpaceLab\u002FAREX-Skill --skill distributed-training-backends","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Frepositories\u002Frepo-skills\u002Faccelerate\u002Fsub-skills\u002Fdistributed-training-backends\u002FSKILL.md",[47,53,57,61,65],{"path":48,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":51,"sourceHtmlUrl":52},"references\u002Fapi-reference.md","REFERENCE","参考文档",6799,"https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Frepositories\u002Frepo-skills\u002Faccelerate\u002Fsub-skills\u002Fdistributed-training-backends\u002Freferences\u002Fapi-reference.md",{"path":54,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":55,"sourceHtmlUrl":56},"references\u002Fdeepspeed-fsdp.md",8626,"https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Frepositories\u002Frepo-skills\u002Faccelerate\u002Fsub-skills\u002Fdistributed-training-backends\u002Freferences\u002Fdeepspeed-fsdp.md",{"path":58,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":59,"sourceHtmlUrl":60},"references\u002Fparallelism-and-precision.md",9315,"https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Frepositories\u002Frepo-skills\u002Faccelerate\u002Fsub-skills\u002Fdistributed-training-backends\u002Freferences\u002Fparallelism-and-precision.md",{"path":62,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":63,"sourceHtmlUrl":64},"references\u002Ftroubleshooting.md",6734,"https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Frepositories\u002Frepo-skills\u002Faccelerate\u002Fsub-skills\u002Fdistributed-training-backends\u002Freferences\u002Ftroubleshooting.md",{"path":66,"resourceType":67,"resourceTypeDesc":68,"sizeBytes":69,"sourceHtmlUrl":70},"scripts\u002Fvalidate_backend_config.py","SCRIPT","脚本",21704,"https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Frepositories\u002Frepo-skills\u002Faccelerate\u002Fsub-skills\u002Fdistributed-training-backends\u002Fscripts\u002Fvalidate_backend_config.py",[72,80,87,94,101,108],{"skillNo":73,"name":74,"nameZh":6,"description":75,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":76,"categories":77,"keywords":78,"publishedAt":79},"01M799NYSH1R3RVAHSM2D88Y84","passnet-triton-opt","Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block\u002Fgrid\u002Fwarp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Ftask-oriented\u002FPassNet\u002Fskills\u002Fpassnet-triton-opt\u002FSKILL.md",[],[],"2026-11-15 14:19:17",{"skillNo":81,"name":82,"nameZh":6,"description":83,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":84,"categories":85,"keywords":86,"publishedAt":79},"01M799NS5APEC0N1FZ7XZVW6BN","passnet-solve","END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Ftask-oriented\u002FPassNet\u002Fskills\u002Fpassnet-solve\u002FSKILL.md",[],[],{"skillNo":88,"name":89,"nameZh":6,"description":90,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":91,"categories":92,"keywords":93,"publishedAt":79},"01M799NNY9BKSM21V5R7QVTBJX","passnet-skill","PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under .\u002Fpass_dir\u002F, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Ftask-oriented\u002FPassNet\u002Fskills\u002Fpassnet-skill\u002FSKILL.md",[],[],{"skillNo":95,"name":96,"nameZh":6,"description":97,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":98,"categories":99,"keywords":100,"publishedAt":79},"01M799NHH6NP7Z1RK77KGXAN6S","passnet-pattern-fusion","Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()\u002Freplacement_args()\u002Freplacement_func() files, when a pass \"failed to match\", or when deciding how to fuse multiple kernels\u002Fops.","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Ftask-oriented\u002FPassNet\u002Fskills\u002Fpassnet-pattern-fusion\u002FSKILL.md",[],[],{"skillNo":102,"name":103,"nameZh":6,"description":104,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":105,"categories":106,"keywords":107,"publishedAt":79},"01M799ND2J4WNC2PGAZ0A06NDF","passnet-orchestrate","Round\u002Fbatch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep\u002Frevert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Ftask-oriented\u002FPassNet\u002Fskills\u002Fpassnet-orchestrate\u002FSKILL.md",[],[],{"skillNo":109,"name":110,"nameZh":6,"description":111,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":112,"categories":113,"keywords":114,"publishedAt":79},"01M799N8SBTMY02ZDG2AZF5ZWE","passnet-feedback","Fast, deterministic feedback for PassNet work: pre-flight pattern\u002Fmatch verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Ftask-oriented\u002FPassNet\u002Fskills\u002Fpassnet-feedback\u002FSKILL.md",[],[]]