[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M792J2PF0CFQAD7G5P9B2C26":3,"skill-same:01M792J2PF0CFQAD7G5P9B2C26":67},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":18,"linkOnly":19,"markdownContent":20,"markdownBody":21,"frontMatter":22,"refName":27,"commitSha":28,"skillDirectory":29,"forksCount":30,"repositoryHtmlUrl":31,"repositoryHomepageUrl":6,"ownerHtmlUrl":32,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":33,"install":34,"resources":46},"01M792J2PF0CFQAD7G5P9B2C26","training-and-integrations",null,"Use PEFT with Transformers Trainer, Accelerate, FSDP, DeepSpeed, TRL SFT, Diffusers training examples, memory-efficient training, torch.compile, and backend-specific training caveats.","VectorSpaceLab\u002FAREX-Skill","VectorSpaceLab","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F180637102?v=4","01M7538C5WPSJGWF7KHR4EFZ9F","01M7538C6R2GFP53NPXTCZNBRT",351,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Frepositories\u002Frepo-skills\u002Fpeft\u002Fsub-skills\u002Ftraining-and-integrations\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: training-and-integrations\ndescription: \"Use PEFT with Transformers Trainer, Accelerate, FSDP, DeepSpeed,\n  TRL SFT, Diffusers training examples, memory-efficient training,\n  torch.compile, and backend-specific training caveats.\"\ndisable-model-invocation: true\nmetadata:\n  disco-role: operating\nlicense: Apache 2.0\n---\n\n# Training and Integrations\n\nUse this sub-skill when the task is about fitting PEFT into a training workflow rather than choosing adapter method parameters. It covers `Trainer`, `Seq2SeqTrainer`, `SFTTrainer`, raw `Accelerate`, FSDP, DeepSpeed, Diffusers DreamBooth\u002FControlNet-style scripts, memory-reduction choices, quantized training backends, and adapting PEFT example workflows safely.\n\n## Route Here For\n\n- Converting a single-GPU PEFT training script to `accelerate launch`, FSDP, DeepSpeed, or multi-GPU SFT.\n- Deciding where PEFT wrapping belongs in `Trainer`, `SFTTrainer`, custom training loops, or Diffusers pipelines.\n- Troubleshooting distributed save behavior, mixed precision gradient errors, gradient checkpointing\u002Foffload interactions, optional backend dependencies, or example scripts that drift with upstream APIs.\n- Building a training readiness checklist without downloading data\u002Fmodels or running training.\n- Deciding whether `torch.compile`, quantized loading, CPU\u002FGPU offload, or gradient checkpointing is safe for a PEFT workflow.\n\n## Route Elsewhere\n\n- Core `LoraConfig`, `PeftConfig`, `get_peft_model`, adapter lifecycle, target module discovery, and `modules_to_save`: use `..\u002Fadapter-core\u002FSKILL.md`.\n- LoRA variants, QLoRA configuration, LoftQ, QALoRA, DoRA, quantization parameter choices, and trainable tokens: use `..\u002Flora-and-quantization\u002FSKILL.md`.\n- Prompt tuning, prefix tuning, p-tuning, and soft prompt method behavior: use `..\u002Fprompt-and-soft-methods\u002FSKILL.md`.\n- Non-LoRA tuner families such as IA3, BOFT\u002FOFT, LoHa\u002FLoKr, VeRA, HRA, RandLoRA, and similar method-specific choices: use `..\u002Fspecialized-tuners\u002FSKILL.md`.\n- Saving, loading, merging, conversion, deployment packaging, or checkpoint format questions after training: use `..\u002Fsave-load-merge\u002FSKILL.md`.\n\n## Start Fast\n\n1. Run the bundled no-training checklist builder from this sub-skill directory:\n\n   ```bash\n   python scripts\u002Fbuild_training_checklist.py --workflow trainer --backend single-gpu\n   python scripts\u002Fbuild_training_checklist.py --workflow trainer --backend fsdp --quantized\n   python scripts\u002Fbuild_training_checklist.py --workflow diffusers --backend deepspeed\n   ```\n\n2. Read `references\u002Ftraining-recipes.md` for PEFT placement in `Trainer`, `SFTTrainer`, `Accelerate`, and Diffusers-style scripts.\n3. Read `references\u002Fdistributed-and-backends.md` before changing FSDP, DeepSpeed, quantization, offload, or `torch.compile` settings.\n4. Read `references\u002Ftroubleshooting.md` when failures mention FP16 gradient unscale, gradient checkpointing, optional packages, FSDP\u002FDeepSpeed save behavior, stale examples, or backend-specific dtype limitations.\n\n## Core Workflow Pattern\n\n- Prepare the base model\u002Ftokenizer\u002Fdataset exactly as the training framework expects, then apply PEFT before the trainer or accelerator prepares the model.\n- For Transformers `Trainer`, pass a `PeftModel` as `model`, or use a trainer integration such as TRL `SFTTrainer(peft_config=...)` when that trainer is responsible for wrapping.\n- For raw `Accelerate`, build the PEFT model before `accelerator.prepare(...)`; after preparation, save adapters with the unwrapped model on the main process.\n- For FSDP, set PEFT-aware wrapping through `peft.utils.other.fsdp_auto_wrap_policy`, keep `fsdp_use_orig_params: false` when memory savings matter, and switch to a full state dict before final adapter save when using trainer-managed FSDP.\n- For DeepSpeed ZeRO-3, align `accelerate` config and training arguments, keep gradient accumulation consistent across both, and use trainer\u002Faccelerator save helpers rather than rank-local saves.\n- For quantized PEFT training, route method\u002Fbackend parameters to `lora-and-quantization`, but keep integration checks here: installed optional backend, `prepare_model_for_kbit_training`, compatible gradient checkpointing mode, and compatible distributed backend.\n\n## Safety Rules For Adapting Examples\n\n- Treat examples as patterns, not stable APIs: re-check current `transformers`, `accelerate`, `trl`, `diffusers`, `bitsandbytes`, and `torch` versions when adapting.\n- Do not run example training as a smoke test unless the user explicitly wants downloads, GPU time, and dataset access; use parser\u002Fhelp checks or dry configuration review first.\n- Replace hard-coded model IDs, dataset names, hub push flags, precision flags, and output directories with user-controlled arguments.\n- Preserve the final adapter save path and distributed save semantics when porting examples to FSDP or DeepSpeed.\n- Keep PEFT adapter configuration decisions separate from launcher\u002Fbackend decisions so changes are reviewable.\n\n## Bundled Materials\n\n- `references\u002Ftraining-recipes.md`: framework-specific recipes for `Trainer`, `Accelerate`, TRL SFT, Diffusers training, memory-efficient training, and `torch.compile` placement.\n- `references\u002Fdistributed-and-backends.md`: FSDP, DeepSpeed, quantization, offload, precision, and backend compatibility guidance.\n- `references\u002Ftroubleshooting.md`: diagnosis paths for stale examples, dtype unscale errors, checkpointing\u002Foffload issues, optional packages, distributed saves, and compile limitations.\n- `scripts\u002Fbuild_training_checklist.py`: parser-only checklist generator for `trainer`, `accelerate`, and `diffusers` workflows across `single-gpu`, `fsdp`, `deepspeed`, and `cpu` backends.\n","# Training and Integrations\n\nUse this sub-skill when the task is about fitting PEFT into a training workflow rather than choosing adapter method parameters. It covers `Trainer`, `Seq2SeqTrainer`, `SFTTrainer`, raw `Accelerate`, FSDP, DeepSpeed, Diffusers DreamBooth\u002FControlNet-style scripts, memory-reduction choices, quantized training backends, and adapting PEFT example workflows safely.\n\n## Route Here For\n\n- Converting a single-GPU PEFT training script to `accelerate launch`, FSDP, DeepSpeed, or multi-GPU SFT.\n- Deciding where PEFT wrapping belongs in `Trainer`, `SFTTrainer`, custom training loops, or Diffusers pipelines.\n- Troubleshooting distributed save behavior, mixed precision gradient errors, gradient checkpointing\u002Foffload interactions, optional backend dependencies, or example scripts that drift with upstream APIs.\n- Building a training readiness checklist without downloading data\u002Fmodels or running training.\n- Deciding whether `torch.compile`, quantized loading, CPU\u002FGPU offload, or gradient checkpointing is safe for a PEFT workflow.\n\n## Route Elsewhere\n\n- Core `LoraConfig`, `PeftConfig`, `get_peft_model`, adapter lifecycle, target module discovery, and `modules_to_save`: use `..\u002Fadapter-core\u002FSKILL.md`.\n- LoRA variants, QLoRA configuration, LoftQ, QALoRA, DoRA, quantization parameter choices, and trainable tokens: use `..\u002Flora-and-quantization\u002FSKILL.md`.\n- Prompt tuning, prefix tuning, p-tuning, and soft prompt method behavior: use `..\u002Fprompt-and-soft-methods\u002FSKILL.md`.\n- Non-LoRA tuner families such as IA3, BOFT\u002FOFT, LoHa\u002FLoKr, VeRA, HRA, RandLoRA, and similar method-specific choices: use `..\u002Fspecialized-tuners\u002FSKILL.md`.\n- Saving, loading, merging, conversion, deployment packaging, or checkpoint format questions after training: use `..\u002Fsave-load-merge\u002FSKILL.md`.\n\n## Start Fast\n\n1. Run the bundled no-training checklist builder from this sub-skill directory:\n\n   ```bash\n   python scripts\u002Fbuild_training_checklist.py --workflow trainer --backend single-gpu\n   python scripts\u002Fbuild_training_checklist.py --workflow trainer --backend fsdp --quantized\n   python scripts\u002Fbuild_training_checklist.py --workflow diffusers --backend deepspeed\n   ```\n\n2. Read `references\u002Ftraining-recipes.md` for PEFT placement in `Trainer`, `SFTTrainer`, `Accelerate`, and Diffusers-style scripts.\n3. Read `references\u002Fdistributed-and-backends.md` before changing FSDP, DeepSpeed, quantization, offload, or `torch.compile` settings.\n4. Read `references\u002Ftroubleshooting.md` when failures mention FP16 gradient unscale, gradient checkpointing, optional packages, FSDP\u002FDeepSpeed save behavior, stale examples, or backend-specific dtype limitations.\n\n## Core Workflow Pattern\n\n- Prepare the base model\u002Ftokenizer\u002Fdataset exactly as the training framework expects, then apply PEFT before the trainer or accelerator prepares the model.\n- For Transformers `Trainer`, pass a `PeftModel` as `model`, or use a trainer integration such as TRL `SFTTrainer(peft_config=...)` when that trainer is responsible for wrapping.\n- For raw `Accelerate`, build the PEFT model before `accelerator.prepare(...)`; after preparation, save adapters with the unwrapped model on the main process.\n- For FSDP, set PEFT-aware wrapping through `peft.utils.other.fsdp_auto_wrap_policy`, keep `fsdp_use_orig_params: false` when memory savings matter, and switch to a full state dict before final adapter save when using trainer-managed FSDP.\n- For DeepSpeed ZeRO-3, align `accelerate` config and training arguments, keep gradient accumulation consistent across both, and use trainer\u002Faccelerator save helpers rather than rank-local saves.\n- For quantized PEFT training, route method\u002Fbackend parameters to `lora-and-quantization`, but keep integration checks here: installed optional backend, `prepare_model_for_kbit_training`, compatible gradient checkpointing mode, and compatible distributed backend.\n\n## Safety Rules For Adapting Examples\n\n- Treat examples as patterns, not stable APIs: re-check current `transformers`, `accelerate`, `trl`, `diffusers`, `bitsandbytes`, and `torch` versions when adapting.\n- Do not run example training as a smoke test unless the user explicitly wants downloads, GPU time, and dataset access; use parser\u002Fhelp checks or dry configuration review first.\n- Replace hard-coded model IDs, dataset names, hub push flags, precision flags, and output directories with user-controlled arguments.\n- Preserve the final adapter save path and distributed save semantics when porting examples to FSDP or DeepSpeed.\n- Keep PEFT adapter configuration decisions separate from launcher\u002Fbackend decisions so changes are reviewable.\n\n## Bundled Materials\n\n- `references\u002Ftraining-recipes.md`: framework-specific recipes for `Trainer`, `Accelerate`, TRL SFT, Diffusers training, memory-efficient training, and `torch.compile` placement.\n- `references\u002Fdistributed-and-backends.md`: FSDP, DeepSpeed, quantization, offload, precision, and backend compatibility guidance.\n- `references\u002Ftroubleshooting.md`: diagnosis paths for stale examples, dtype unscale errors, checkpointing\u002Foffload issues, optional packages, distributed saves, and compile limitations.\n- `scripts\u002Fbuild_training_checklist.py`: parser-only checklist generator for `trainer`, `accelerate`, and `diffusers` workflows across `single-gpu`, `fsdp`, `deepspeed`, and `cpu` backends.\n",{"name":5,"license":23,"metadata":24,"description":7,"disable-model-invocation":26},"Apache 2.0",{"disco-role":25},"operating",true,"main","ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6","skills\u002Frepositories\u002Frepo-skills\u002Fpeft\u002Fsub-skills\u002Ftraining-and-integrations",27,"https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab","2026-09-03 10:25:21",{"source":8,"skillName":5,"commands":35,"prompt":45,"zipAvailable":26},[36,39,42],{"tool":37,"command":38},"npx","npx skills add VectorSpaceLab\u002FAREX-Skill --skill training-and-integrations",{"tool":40,"command":41},"bunx","bunx skills add VectorSpaceLab\u002FAREX-Skill --skill training-and-integrations",{"tool":43,"command":44},"pnpm","pnpm dlx skills add VectorSpaceLab\u002FAREX-Skill --skill training-and-integrations","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Frepositories\u002Frepo-skills\u002Fpeft\u002Fsub-skills\u002Ftraining-and-integrations\u002FSKILL.md",[47,53,57,61],{"path":48,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":51,"sourceHtmlUrl":52},"references\u002Fdistributed-and-backends.md","REFERENCE","参考文档",6672,"https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Frepositories\u002Frepo-skills\u002Fpeft\u002Fsub-skills\u002Ftraining-and-integrations\u002Freferences\u002Fdistributed-and-backends.md",{"path":54,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":55,"sourceHtmlUrl":56},"references\u002Ftraining-recipes.md",7177,"https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Frepositories\u002Frepo-skills\u002Fpeft\u002Fsub-skills\u002Ftraining-and-integrations\u002Freferences\u002Ftraining-recipes.md",{"path":58,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":59,"sourceHtmlUrl":60},"references\u002Ftroubleshooting.md",6173,"https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Frepositories\u002Frepo-skills\u002Fpeft\u002Fsub-skills\u002Ftraining-and-integrations\u002Freferences\u002Ftroubleshooting.md",{"path":62,"resourceType":63,"resourceTypeDesc":64,"sizeBytes":65,"sourceHtmlUrl":66},"scripts\u002Fbuild_training_checklist.py","SCRIPT","脚本",5499,"https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Frepositories\u002Frepo-skills\u002Fpeft\u002Fsub-skills\u002Ftraining-and-integrations\u002Fscripts\u002Fbuild_training_checklist.py",[68,75,82,89,96,103],{"skillNo":69,"name":70,"nameZh":6,"description":71,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":72,"categories":73,"keywords":74,"publishedAt":18},"01M799NYSH1R3RVAHSM2D88Y84","passnet-triton-opt","Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block\u002Fgrid\u002Fwarp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Ftask-oriented\u002FPassNet\u002Fskills\u002Fpassnet-triton-opt\u002FSKILL.md",[],[],{"skillNo":76,"name":77,"nameZh":6,"description":78,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":79,"categories":80,"keywords":81,"publishedAt":18},"01M799NS5APEC0N1FZ7XZVW6BN","passnet-solve","END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Ftask-oriented\u002FPassNet\u002Fskills\u002Fpassnet-solve\u002FSKILL.md",[],[],{"skillNo":83,"name":84,"nameZh":6,"description":85,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":86,"categories":87,"keywords":88,"publishedAt":18},"01M799NNY9BKSM21V5R7QVTBJX","passnet-skill","PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under .\u002Fpass_dir\u002F, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Ftask-oriented\u002FPassNet\u002Fskills\u002Fpassnet-skill\u002FSKILL.md",[],[],{"skillNo":90,"name":91,"nameZh":6,"description":92,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":93,"categories":94,"keywords":95,"publishedAt":18},"01M799NHH6NP7Z1RK77KGXAN6S","passnet-pattern-fusion","Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()\u002Freplacement_args()\u002Freplacement_func() files, when a pass \"failed to match\", or when deciding how to fuse multiple kernels\u002Fops.","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Ftask-oriented\u002FPassNet\u002Fskills\u002Fpassnet-pattern-fusion\u002FSKILL.md",[],[],{"skillNo":97,"name":98,"nameZh":6,"description":99,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":100,"categories":101,"keywords":102,"publishedAt":18},"01M799ND2J4WNC2PGAZ0A06NDF","passnet-orchestrate","Round\u002Fbatch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep\u002Frevert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Ftask-oriented\u002FPassNet\u002Fskills\u002Fpassnet-orchestrate\u002FSKILL.md",[],[],{"skillNo":104,"name":105,"nameZh":6,"description":106,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":107,"categories":108,"keywords":109,"publishedAt":18},"01M799N8SBTMY02ZDG2AZF5ZWE","passnet-feedback","Fast, deterministic feedback for PassNet work: pre-flight pattern\u002Fmatch verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Ftask-oriented\u002FPassNet\u002Fskills\u002Fpassnet-feedback\u002FSKILL.md",[],[]]