[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6QEVSHERWV14YQWSG5JS0M8":3,"skill-same:01M6QEVSHERWV14YQWSG5JS0M8":87},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":39,"linkOnly":40,"markdownContent":41,"markdownBody":42,"frontMatter":43,"refName":50,"commitSha":51,"skillDirectory":52,"forksCount":14,"repositoryHtmlUrl":53,"repositoryHomepageUrl":54,"ownerHtmlUrl":55,"ownerBlogUrl":56,"ownerDescription":57,"updatedAt":58,"install":59,"resources":72},"01M6QEVSHERWV14YQWSG5JS0M8","huggingface-accelerate",null,"Wraps existing PyTorch training scripts with HuggingFace Accelerate (Accelerator class, accelerate config, accelerate launch) so the same code runs on CPU, single GPU, multi-GPU, multi-node, TPU, or Apple MPS. Covers device placement, FP16\u002FBF16\u002FFP8 mixed precision, gradient accumulation, distributed checkpointing, and switching between DDP, DeepSpeed ZeRO, FSDP, and Megatron backends. Use when converting a single-GPU script to multi-GPU, enabling mixed precision, configuring DeepSpeed ZeRO or FSDP, or setting up gradient accumulation. Use when one script must run on different hardware. Not for callback-heavy training loops (use PyTorch Lightning) or multi-node orchestration with hyperparameter tuning (use Ray Train).","Tashima-Tarsh\u002FDisha6.6","Tashima-Tarsh","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F269906498?v=4","01M6PBH98X03XERH0HFT4PFB1P","NITISH KUMAR","01M6PBH9DA0M6KM3Z12P848F0K",0,"MIT","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fkal-huggingface-accelerate\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38],"HuggingFace","huggingface accelerate","Accelerate","megatron","Existing","gradient","for","callback","Unified API","DDP","FSDP","CPU","config","train","use","pytorch lightning","Ray","run","training","distributed","2026-11-10 10:12:30",false,"---\nname: huggingface-accelerate\ndescription: Wraps existing PyTorch training scripts with HuggingFace Accelerate (Accelerator class, accelerate config, accelerate launch) so the same code runs on CPU, single GPU, multi-GPU, multi-node, TPU, or Apple MPS. Covers device placement, FP16\u002FBF16\u002FFP8 mixed precision, gradient accumulation, distributed checkpointing, and switching between DDP, DeepSpeed ZeRO, FSDP, and Megatron backends. Use when converting a single-GPU script to multi-GPU, enabling mixed precision, configuring DeepSpeed ZeRO or FSDP, or setting up gradient accumulation. Use when one script must run on different hardware. Not for callback-heavy training loops (use PyTorch Lightning) or multi-node orchestration with hyperparameter tuning (use Ray Train).\nlicense: MIT\nmetadata:\n  version: 1.0.0\n  category: ml-training\n  maintainer: Kalaris Labs\n  tags: Distributed Training, HuggingFace, Accelerate, DeepSpeed, FSDP, Mixed Precision, PyTorch, DDP, Unified API, Simple\n  dependencies: accelerate, torch, transformers\n---\n\n# HuggingFace Accelerate - Unified Distributed Training\n\n## Quick start\n\nAccelerate simplifies distributed training to 4 lines of code.\n\n**Installation**:\n```bash\npip install accelerate\n```\n\n**Convert PyTorch script** (4 lines):\n```python\nimport torch\n+ from accelerate import Accelerator\n\n+ accelerator = Accelerator()\n\n  model = torch.nn.Transformer()\n  optimizer = torch.optim.Adam(model.parameters())\n  dataloader = torch.utils.data.DataLoader(dataset)\n\n+ model, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n\n  for batch in dataloader:\n      optimizer.zero_grad()\n      loss = model(batch)\n-     loss.backward()\n+     accelerator.backward(loss)\n      optimizer.step()\n```\n\n**Run** (single command):\n```bash\naccelerate launch train.py\n```\n\n## Common workflows\n\n### Workflow 1: From single GPU to multi-GPU\n\n**Original script**:\n```python\n# train.py\nimport torch\n\nmodel = torch.nn.Linear(10, 2).to('cuda')\noptimizer = torch.optim.Adam(model.parameters())\ndataloader = torch.utils.data.DataLoader(dataset, batch_size=32)\n\nfor epoch in range(10):\n    for batch in dataloader:\n        batch = batch.to('cuda')\n        optimizer.zero_grad()\n        loss = model(batch).mean()\n        loss.backward()\n        optimizer.step()\n```\n\n**With Accelerate** (4 lines added):\n```python\n# train.py\nimport torch\nfrom accelerate import Accelerator  # +1\n\naccelerator = Accelerator()  # +2\n\nmodel = torch.nn.Linear(10, 2)\noptimizer = torch.optim.Adam(model.parameters())\ndataloader = torch.utils.data.DataLoader(dataset, batch_size=32)\n\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)  # +3\n\nfor epoch in range(10):\n    for batch in dataloader:\n        # No .to('cuda') needed - automatic!\n        optimizer.zero_grad()\n        loss = model(batch).mean()\n        accelerator.backward(loss)  # +4\n        optimizer.step()\n```\n\n**Configure** (interactive):\n```bash\naccelerate config\n```\n\n**Questions**:\n- Which machine? (single\u002Fmulti GPU\u002FTPU\u002FCPU)\n- How many machines? (1)\n- Mixed precision? (no\u002Ffp16\u002Fbf16\u002Ffp8)\n- DeepSpeed? (no\u002Fyes)\n\n**Launch** (works on any setup):\n```bash\n# Single GPU\naccelerate launch train.py\n\n# Multi-GPU (8 GPUs)\naccelerate launch --multi_gpu --num_processes 8 train.py\n\n# Multi-node\naccelerate launch --multi_gpu --num_processes 16 \\\n  --num_machines 2 --machine_rank 0 \\\n  --main_process_ip $MASTER_ADDR \\\n  train.py\n```\n\n### Workflow 2: Mixed precision training\n\n**Enable FP16\u002FBF16**:\n```python\nfrom accelerate import Accelerator\n\n# FP16 (with gradient scaling)\naccelerator = Accelerator(mixed_precision='fp16')\n\n# BF16 (no scaling, more stable)\naccelerator = Accelerator(mixed_precision='bf16')\n\n# FP8 (H100+)\naccelerator = Accelerator(mixed_precision='fp8')\n\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n\n# Everything else is automatic!\nfor batch in dataloader:\n    with accelerator.autocast():  # Optional, done automatically\n        loss = model(batch)\n    accelerator.backward(loss)\n```\n\n### Workflow 3: DeepSpeed ZeRO integration\n\n**Enable DeepSpeed ZeRO-2**:\n```python\nfrom accelerate import Accelerator\n\naccelerator = Accelerator(\n    mixed_precision='bf16',\n    deepspeed_plugin={\n        \"zero_stage\": 2,  # ZeRO-2\n        \"offload_optimizer\": False,\n        \"gradient_accumulation_steps\": 4\n    }\n)\n\n# Same code as before!\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n```\n\n**Or via config**:\n```bash\naccelerate config\n# Select: DeepSpeed → ZeRO-2\n```\n\n**deepspeed_config.json**:\n```json\n{\n    \"fp16\": {\"enabled\": false},\n    \"bf16\": {\"enabled\": true},\n    \"zero_optimization\": {\n        \"stage\": 2,\n        \"offload_optimizer\": {\"device\": \"cpu\"},\n        \"allgather_bucket_size\": 5e8,\n        \"reduce_bucket_size\": 5e8\n    }\n}\n```\n\n**Launch**:\n```bash\naccelerate launch --config_file deepspeed_config.json train.py\n```\n\n### Workflow 4: FSDP (Fully Sharded Data Parallel)\n\n**Enable FSDP**:\n```python\nfrom accelerate import Accelerator, FullyShardedDataParallelPlugin\n\nfsdp_plugin = FullyShardedDataParallelPlugin(\n    sharding_strategy=\"FULL_SHARD\",  # ZeRO-3 equivalent\n    auto_wrap_policy=\"TRANSFORMER_AUTO_WRAP\",\n    cpu_offload=False\n)\n\naccelerator = Accelerator(\n    mixed_precision='bf16',\n    fsdp_plugin=fsdp_plugin\n)\n\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n```\n\n**Or via config**:\n```bash\naccelerate config\n# Select: FSDP → Full Shard → No CPU Offload\n```\n\n### Workflow 5: Gradient accumulation\n\n**Accumulate gradients**:\n```python\nfrom accelerate import Accelerator\n\naccelerator = Accelerator(gradient_accumulation_steps=4)\n\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n\nfor batch in dataloader:\n    with accelerator.accumulate(model):  # Handles accumulation\n        optimizer.zero_grad()\n        loss = model(batch)\n        accelerator.backward(loss)\n        optimizer.step()\n```\n\n**Effective batch size**: `batch_size * num_gpus * gradient_accumulation_steps`\n\n## When to use vs alternatives\n\n**Use Accelerate when**:\n- Want simplest distributed training\n- Need single script for any hardware\n- Use HuggingFace ecosystem\n- Want flexibility (DDP\u002FDeepSpeed\u002FFSDP\u002FMegatron)\n- Need quick prototyping\n\n**Key advantages**:\n- **4 lines**: Minimal code changes\n- **Unified API**: Same code for DDP, DeepSpeed, FSDP, Megatron\n- **Automatic**: Device placement, mixed precision, sharding\n- **Interactive config**: No manual launcher setup\n- **Single launch**: Works everywhere\n\n**Use alternatives instead**:\n- **PyTorch Lightning**: Need callbacks, high-level abstractions\n- **Ray Train**: Multi-node orchestration, hyperparameter tuning\n- **DeepSpeed**: Direct API control, advanced features\n- **Raw DDP**: Maximum control, minimal abstraction\n\n## Common issues\n\n**Issue: Wrong device placement**\n\nDon't manually move to device:\n```python\n# WRONG\nbatch = batch.to('cuda')\n\n# CORRECT\n# Accelerate handles it automatically after prepare()\n```\n\n**Issue: Gradient accumulation not working**\n\nUse context manager:\n```python\n# CORRECT\nwith accelerator.accumulate(model):\n    optimizer.zero_grad()\n    accelerator.backward(loss)\n    optimizer.step()\n```\n\n**Issue: Checkpointing in distributed**\n\nUse accelerator methods:\n```python\n# Save only on main process\nif accelerator.is_main_process:\n    accelerator.save_state('checkpoint\u002F')\n\n# Load on all processes\naccelerator.load_state('checkpoint\u002F')\n```\n\n**Issue: Different results with FSDP**\n\nEnsure same random seed:\n```python\nfrom accelerate.utils import set_seed\nset_seed(42)\n```\n\n## Advanced topics\n\n**Megatron integration**: See [references\u002Fmegatron-integration.md](references\u002Fmegatron-integration.md) for tensor parallelism, pipeline parallelism, and sequence parallelism setup.\n\n**Custom plugins**: See [references\u002Fcustom-plugins.md](references\u002Fcustom-plugins.md) for creating custom distributed plugins and advanced configuration.\n\n**Performance tuning**: See [references\u002Fperformance.md](references\u002Fperformance.md) for profiling, memory optimization, and best practices.\n\n## Hardware requirements\n\n- **CPU**: Works (slow)\n- **Single GPU**: Works\n- **Multi-GPU**: DDP (default), DeepSpeed, or FSDP\n- **Multi-node**: DDP, DeepSpeed, FSDP, Megatron\n- **TPU**: Supported\n- **Apple MPS**: Supported\n\n**Launcher requirements**:\n- **DDP**: `torch.distributed.run` (built-in)\n- **DeepSpeed**: `deepspeed` (pip install deepspeed)\n- **FSDP**: PyTorch 1.12+ (built-in)\n- **Megatron**: Custom setup\n\n## Resources\n\n- Docs: https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Faccelerate\n- GitHub: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Faccelerate\n- Version: 1.11.0+\n- Tutorial: \"Accelerate your scripts\"\n- Examples: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Faccelerate\u002Ftree\u002Fmain\u002Fexamples\n- Used by: HuggingFace Transformers, TRL, PEFT, all HF libraries\n\n## Agent operating procedure\n\n1. **Check the environment.** Check GPU type, memory and driver\u002FCUDA versions (`nvidia-smi`), framework versions, and dataset location and size.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Do a smoke run: tiny model or subset, few steps, and confirm loss decreases and checkpoints save.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Track metrics on held-out data, compare against a baseline, and record seeds, configs and hardware.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| CUDA out-of-memory | Reduce batch size, enable gradient accumulation\u002Fcheckpointing or mixed precision, or shard the model. |\n| Loss is NaN or diverges | Lower the learning rate, check data for invalid values, and enable gradient clipping. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Never claim training results without logs; estimate compute cost before launching large jobs.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `pytorch-lightning`: Organizes PyTorch training code with the lightning package (PyTorch Lightning): LightningModule, LightningDataModule, Trainer, callbacks su…\n- `pytorch-lightning-distributed`: High-level PyTorch framework with Trainer class, automatic distributed training (DDP\u002FFSDP\u002FDeepSpeed), callbacks system, and minimal boilerp…\n- `deepspeed`: Covers DeepSpeed for distributed deep learning training and I\u002FO: ZeRO optimization stages, pipeline parallelism, FP16\u002FBF16\u002FFP8 training, 1-…\n","# HuggingFace Accelerate - Unified Distributed Training\n\n## Quick start\n\nAccelerate simplifies distributed training to 4 lines of code.\n\n**Installation**:\n```bash\npip install accelerate\n```\n\n**Convert PyTorch script** (4 lines):\n```python\nimport torch\n+ from accelerate import Accelerator\n\n+ accelerator = Accelerator()\n\n  model = torch.nn.Transformer()\n  optimizer = torch.optim.Adam(model.parameters())\n  dataloader = torch.utils.data.DataLoader(dataset)\n\n+ model, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n\n  for batch in dataloader:\n      optimizer.zero_grad()\n      loss = model(batch)\n-     loss.backward()\n+     accelerator.backward(loss)\n      optimizer.step()\n```\n\n**Run** (single command):\n```bash\naccelerate launch train.py\n```\n\n## Common workflows\n\n### Workflow 1: From single GPU to multi-GPU\n\n**Original script**:\n```python\n# train.py\nimport torch\n\nmodel = torch.nn.Linear(10, 2).to('cuda')\noptimizer = torch.optim.Adam(model.parameters())\ndataloader = torch.utils.data.DataLoader(dataset, batch_size=32)\n\nfor epoch in range(10):\n    for batch in dataloader:\n        batch = batch.to('cuda')\n        optimizer.zero_grad()\n        loss = model(batch).mean()\n        loss.backward()\n        optimizer.step()\n```\n\n**With Accelerate** (4 lines added):\n```python\n# train.py\nimport torch\nfrom accelerate import Accelerator  # +1\n\naccelerator = Accelerator()  # +2\n\nmodel = torch.nn.Linear(10, 2)\noptimizer = torch.optim.Adam(model.parameters())\ndataloader = torch.utils.data.DataLoader(dataset, batch_size=32)\n\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)  # +3\n\nfor epoch in range(10):\n    for batch in dataloader:\n        # No .to('cuda') needed - automatic!\n        optimizer.zero_grad()\n        loss = model(batch).mean()\n        accelerator.backward(loss)  # +4\n        optimizer.step()\n```\n\n**Configure** (interactive):\n```bash\naccelerate config\n```\n\n**Questions**:\n- Which machine? (single\u002Fmulti GPU\u002FTPU\u002FCPU)\n- How many machines? (1)\n- Mixed precision? (no\u002Ffp16\u002Fbf16\u002Ffp8)\n- DeepSpeed? (no\u002Fyes)\n\n**Launch** (works on any setup):\n```bash\n# Single GPU\naccelerate launch train.py\n\n# Multi-GPU (8 GPUs)\naccelerate launch --multi_gpu --num_processes 8 train.py\n\n# Multi-node\naccelerate launch --multi_gpu --num_processes 16 \\\n  --num_machines 2 --machine_rank 0 \\\n  --main_process_ip $MASTER_ADDR \\\n  train.py\n```\n\n### Workflow 2: Mixed precision training\n\n**Enable FP16\u002FBF16**:\n```python\nfrom accelerate import Accelerator\n\n# FP16 (with gradient scaling)\naccelerator = Accelerator(mixed_precision='fp16')\n\n# BF16 (no scaling, more stable)\naccelerator = Accelerator(mixed_precision='bf16')\n\n# FP8 (H100+)\naccelerator = Accelerator(mixed_precision='fp8')\n\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n\n# Everything else is automatic!\nfor batch in dataloader:\n    with accelerator.autocast():  # Optional, done automatically\n        loss = model(batch)\n    accelerator.backward(loss)\n```\n\n### Workflow 3: DeepSpeed ZeRO integration\n\n**Enable DeepSpeed ZeRO-2**:\n```python\nfrom accelerate import Accelerator\n\naccelerator = Accelerator(\n    mixed_precision='bf16',\n    deepspeed_plugin={\n        \"zero_stage\": 2,  # ZeRO-2\n        \"offload_optimizer\": False,\n        \"gradient_accumulation_steps\": 4\n    }\n)\n\n# Same code as before!\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n```\n\n**Or via config**:\n```bash\naccelerate config\n# Select: DeepSpeed → ZeRO-2\n```\n\n**deepspeed_config.json**:\n```json\n{\n    \"fp16\": {\"enabled\": false},\n    \"bf16\": {\"enabled\": true},\n    \"zero_optimization\": {\n        \"stage\": 2,\n        \"offload_optimizer\": {\"device\": \"cpu\"},\n        \"allgather_bucket_size\": 5e8,\n        \"reduce_bucket_size\": 5e8\n    }\n}\n```\n\n**Launch**:\n```bash\naccelerate launch --config_file deepspeed_config.json train.py\n```\n\n### Workflow 4: FSDP (Fully Sharded Data Parallel)\n\n**Enable FSDP**:\n```python\nfrom accelerate import Accelerator, FullyShardedDataParallelPlugin\n\nfsdp_plugin = FullyShardedDataParallelPlugin(\n    sharding_strategy=\"FULL_SHARD\",  # ZeRO-3 equivalent\n    auto_wrap_policy=\"TRANSFORMER_AUTO_WRAP\",\n    cpu_offload=False\n)\n\naccelerator = Accelerator(\n    mixed_precision='bf16',\n    fsdp_plugin=fsdp_plugin\n)\n\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n```\n\n**Or via config**:\n```bash\naccelerate config\n# Select: FSDP → Full Shard → No CPU Offload\n```\n\n### Workflow 5: Gradient accumulation\n\n**Accumulate gradients**:\n```python\nfrom accelerate import Accelerator\n\naccelerator = Accelerator(gradient_accumulation_steps=4)\n\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n\nfor batch in dataloader:\n    with accelerator.accumulate(model):  # Handles accumulation\n        optimizer.zero_grad()\n        loss = model(batch)\n        accelerator.backward(loss)\n        optimizer.step()\n```\n\n**Effective batch size**: `batch_size * num_gpus * gradient_accumulation_steps`\n\n## When to use vs alternatives\n\n**Use Accelerate when**:\n- Want simplest distributed training\n- Need single script for any hardware\n- Use HuggingFace ecosystem\n- Want flexibility (DDP\u002FDeepSpeed\u002FFSDP\u002FMegatron)\n- Need quick prototyping\n\n**Key advantages**:\n- **4 lines**: Minimal code changes\n- **Unified API**: Same code for DDP, DeepSpeed, FSDP, Megatron\n- **Automatic**: Device placement, mixed precision, sharding\n- **Interactive config**: No manual launcher setup\n- **Single launch**: Works everywhere\n\n**Use alternatives instead**:\n- **PyTorch Lightning**: Need callbacks, high-level abstractions\n- **Ray Train**: Multi-node orchestration, hyperparameter tuning\n- **DeepSpeed**: Direct API control, advanced features\n- **Raw DDP**: Maximum control, minimal abstraction\n\n## Common issues\n\n**Issue: Wrong device placement**\n\nDon't manually move to device:\n```python\n# WRONG\nbatch = batch.to('cuda')\n\n# CORRECT\n# Accelerate handles it automatically after prepare()\n```\n\n**Issue: Gradient accumulation not working**\n\nUse context manager:\n```python\n# CORRECT\nwith accelerator.accumulate(model):\n    optimizer.zero_grad()\n    accelerator.backward(loss)\n    optimizer.step()\n```\n\n**Issue: Checkpointing in distributed**\n\nUse accelerator methods:\n```python\n# Save only on main process\nif accelerator.is_main_process:\n    accelerator.save_state('checkpoint\u002F')\n\n# Load on all processes\naccelerator.load_state('checkpoint\u002F')\n```\n\n**Issue: Different results with FSDP**\n\nEnsure same random seed:\n```python\nfrom accelerate.utils import set_seed\nset_seed(42)\n```\n\n## Advanced topics\n\n**Megatron integration**: See [references\u002Fmegatron-integration.md](references\u002Fmegatron-integration.md) for tensor parallelism, pipeline parallelism, and sequence parallelism setup.\n\n**Custom plugins**: See [references\u002Fcustom-plugins.md](references\u002Fcustom-plugins.md) for creating custom distributed plugins and advanced configuration.\n\n**Performance tuning**: See [references\u002Fperformance.md](references\u002Fperformance.md) for profiling, memory optimization, and best practices.\n\n## Hardware requirements\n\n- **CPU**: Works (slow)\n- **Single GPU**: Works\n- **Multi-GPU**: DDP (default), DeepSpeed, or FSDP\n- **Multi-node**: DDP, DeepSpeed, FSDP, Megatron\n- **TPU**: Supported\n- **Apple MPS**: Supported\n\n**Launcher requirements**:\n- **DDP**: `torch.distributed.run` (built-in)\n- **DeepSpeed**: `deepspeed` (pip install deepspeed)\n- **FSDP**: PyTorch 1.12+ (built-in)\n- **Megatron**: Custom setup\n\n## Resources\n\n- Docs: https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Faccelerate\n- GitHub: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Faccelerate\n- Version: 1.11.0+\n- Tutorial: \"Accelerate your scripts\"\n- Examples: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Faccelerate\u002Ftree\u002Fmain\u002Fexamples\n- Used by: HuggingFace Transformers, TRL, PEFT, all HF libraries\n\n## Agent operating procedure\n\n1. **Check the environment.** Check GPU type, memory and driver\u002FCUDA versions (`nvidia-smi`), framework versions, and dataset location and size.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Do a smoke run: tiny model or subset, few steps, and confirm loss decreases and checkpoints save.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Track metrics on held-out data, compare against a baseline, and record seeds, configs and hardware.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| CUDA out-of-memory | Reduce batch size, enable gradient accumulation\u002Fcheckpointing or mixed precision, or shard the model. |\n| Loss is NaN or diverges | Lower the learning rate, check data for invalid values, and enable gradient clipping. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Never claim training results without logs; estimate compute cost before launching large jobs.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `pytorch-lightning`: Organizes PyTorch training code with the lightning package (PyTorch Lightning): LightningModule, LightningDataModule, Trainer, callbacks su…\n- `pytorch-lightning-distributed`: High-level PyTorch framework with Trainer class, automatic distributed training (DDP\u002FFSDP\u002FDeepSpeed), callbacks system, and minimal boilerp…\n- `deepspeed`: Covers DeepSpeed for distributed deep learning training and I\u002FO: ZeRO optimization stages, pipeline parallelism, FP16\u002FBF16\u002FFP8 training, 1-…\n",{"name":5,"license":15,"metadata":44,"description":7},{"tags":45,"version":46,"category":47,"maintainer":48,"dependencies":49},"Distributed Training, HuggingFace, Accelerate, DeepSpeed, FSDP, Mixed Precision, PyTorch, DDP, Unified API, Simple","1.0.0","ml-training","Kalaris Labs","accelerate, torch, transformers","main","0da4485785ee599388900be50db1635489d53056",".agents\u002Fskills\u002Fkal-huggingface-accelerate","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6","https:\u002F\u002Fdisha6.6.thenitishkr.in\u002F","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh","www.thenitishkr","thenitishkr | AI • Cyber-security • Research\r\n","2026-11-08 03:33:25",{"source":8,"skillName":5,"commands":60,"prompt":70,"zipAvailable":71},[61,64,67],{"tool":62,"command":63},"npx","npx skills add Tashima-Tarsh\u002FDisha6.6 --skill huggingface-accelerate",{"tool":65,"command":66},"bunx","bunx skills add Tashima-Tarsh\u002FDisha6.6 --skill huggingface-accelerate",{"tool":68,"command":69},"pnpm","pnpm dlx skills add Tashima-Tarsh\u002FDisha6.6 --skill huggingface-accelerate","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fkal-huggingface-accelerate\u002FSKILL.md",true,[73,79,83],{"path":74,"resourceType":75,"resourceTypeDesc":76,"sizeBytes":77,"sourceHtmlUrl":78},"references\u002Fcustom-plugins.md","REFERENCE","参考文档",11781,"https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fkal-huggingface-accelerate\u002Freferences\u002Fcustom-plugins.md",{"path":80,"resourceType":75,"resourceTypeDesc":76,"sizeBytes":81,"sourceHtmlUrl":82},"references\u002Fmegatron-integration.md",11252,"https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fkal-huggingface-accelerate\u002Freferences\u002Fmegatron-integration.md",{"path":84,"resourceType":75,"resourceTypeDesc":76,"sizeBytes":85,"sourceHtmlUrl":86},"references\u002Fperformance.md",12569,"https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fkal-huggingface-accelerate\u002Freferences\u002Fperformance.md",[88,119,143,170,194,218],{"skillNo":89,"name":90,"nameZh":6,"description":91,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":92,"categories":93,"keywords":99,"publishedAt":39},"01M6QRD07D4TGNDK6VJ4EHF8BV","youtube-full","Use when the user needs YouTube transcripts, video search, channel browsing, playlist extraction, or content monitoring. Trigger phrases: 'get the transcript for', 'search YouTube for', 'what are the latest videos on', 'list this playlist', 'monitor this channel', or any request involving a YouTube URL, video ID, or @handle. Do NOT use for downloading video or audio files, YouTube engagement data (likes, comments), or private\u002Fage-restricted videos.","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fyoutube-full\u002FSKILL.md",[94],{"categoryNo":95,"categoryName":96,"taxonomyType":97,"taxonomyTypeDesc":98},"01JBSEEDCAT000000000000016","图像与多媒体","SCENARIO","使用场景",[100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,25,116,117,118],"youtube full","youtube","full","transcript","search","monitoring","data","Video","Audio","list","files","do","get","age","trigger","URL","browsing","request","youtube transcripts",{"skillNo":120,"name":121,"nameZh":6,"description":122,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":123,"categories":124,"keywords":125,"publishedAt":39},"01M6QRCJ3112VYPA97YG7KGMC2","x-twitter-growth","X\u002FTwitter growth engine for building audience, crafting viral content, and analyzing engagement. Use when the user wants to grow on X\u002FTwitter, write tweets or threads, analyze their X profile, research competitors on X, plan a posting strategy, or optimize engagement. Complements social-content (generic multi-platform) with X-specific depth: algorithm mechanics, thread engineering, reply strategy, profile optimization, and competitive intelligence via web search.","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fx-twitter-growth\u002FSKILL.md",[],[126,127,128,129,104,130,131,132,133,134,135,136,137,138,33,139,140,141,142,25],"twitter","x twitter growth","growth","research","content","engineering","plan","web search","competitors","analyze","engine","write","strategy","platform","grow","posting","competitive",{"skillNo":144,"name":145,"nameZh":6,"description":146,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":147,"categories":148,"keywords":152,"publishedAt":39},"01M6QRBN3EHQNJY42RERP7A149","write-a-skill","Create new agent skills with proper structure, progressive disclosure, and bundled resources. Use when user wants to create, write, build, or author a new skill.","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fwrite-a-skill\u002FSKILL.md",[149],{"categoryNo":150,"categoryName":151,"taxonomyType":97,"taxonomyTypeDesc":98},"01JBSEEDCAT000000000000023","Skill与智能体开发",[153,137,154,155,156,157,158,159,33,160,161,50,162,163,164,165,166,167,168,169],"write a skill","github","create","https","productivity","matt pocock","direct","example","build","new skill","structure","new agent","resources","with","tree","progressive disclosure","author",{"skillNo":171,"name":172,"nameZh":6,"description":173,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":174,"categories":175,"keywords":176,"publishedAt":39},"01M6QRB6G8P33PNGAF9Y69EJ9T","wrap-up","Close out a launched Claude Managed Agent — recap every primitive the founder now owns, regenerate the single-file overview page, and suggest the next 1-2 upgrades. Use when the user says \"wrap up\", \"close this out\", \"what do I own now\", \"give me the summary\", \"recap the agent\", or when the orchestrator routes phase=wrap-up. primitives_inventory.py tables everything owned (agent, environment, session, memory, outcome, deployment); overview_page.py regenerates a self-contained .\u002Fmy-agent\u002Fagent-overview.html; upgrade_suggester.py ranks the next moves from recorded deferrals plus standing hardening steps. Companion to run-without-you; the last stop before phase=done.","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fwrap-up\u002FSKILL.md",[],[177,178,179,180,181,36,182,183,184,33,185,111,186,187,188,189,190,191,192,193],"wrap up","up","wrap","gemini","deployment","file","hardening","closeout","stop","upgrade","founder","close","companion","you","tables","directions","phase",{"skillNo":195,"name":196,"nameZh":6,"description":197,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":198,"categories":199,"keywords":201,"publishedAt":39},"01M6QRA5AYC1RBFZ39GTWGS3N8","workflow-builder","Design and write deterministic multi-agent workflow scripts (.js files in .claude\u002Fworkflows\u002F) for Claude Code's Workflow tool. Use when a user wants to build, create, author, scaffold, or run a custom Claude Code workflow, orchestrate sub-agents (fan-out, pipeline, loop, judge-panel), or automate a repeatable multi-step task across fresh-context agents.","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fworkflow-builder\u002FSKILL.md",[200],{"categoryNo":150,"categoryName":151,"taxonomyType":97,"taxonomyTypeDesc":98},[202,203,204,205,206,155,36,207,208,209,210,211,137,212,110,213,214,215,216,217],"workflow","workflow builder","builder","design","workflows","code","orchestrate","pipeline","loop","multi-agent","context","across","fresh","scripts","judge","claude code workflow",{"skillNo":219,"name":220,"nameZh":6,"description":221,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":222,"categories":223,"keywords":224,"publishedAt":39},"01M6QR7QMCZESKQA0ERW1E657M","weekly-review","Use when someone wants to run a weekly review, close open loops, audit stalled projects and commitments, get their system back to trusted, restart a lapsed review habit, or says \"\u002Fcs:weekly-review\". Walks David Allen's three-phase loop — GET CLEAR, GET CURRENT, GET CREATIVE — with deterministic scripts that inventory open loops, gate the checklist with named gaps, and score commitment health 0-100.","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fweekly-review\u002FSKILL.md",[],[225,226,227,228,229,230,231,232,233,210,234,235,236,237,238,239,112,240,241,193],"review","weekly review","weekly","deterministic scripts","reflect","feeds","inventory","open","team","projects","project-management","scanner","conversation","back","project","sprint","cadence"]