[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6MA874FMS84R5YQP32VACKW":3,"skill-same:01M6MA874FMS84R5YQP32VACKW":89},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":40,"linkOnly":41,"markdownContent":42,"markdownBody":43,"frontMatter":44,"refName":51,"commitSha":52,"skillDirectory":53,"forksCount":54,"repositoryHtmlUrl":55,"repositoryHomepageUrl":56,"ownerHtmlUrl":57,"ownerBlogUrl":58,"ownerDescription":59,"updatedAt":60,"install":61,"resources":74},"01M6MA874FMS84R5YQP32VACKW","huggingface-accelerate",null,"Wraps existing PyTorch training scripts with HuggingFace Accelerate (Accelerator class, accelerate config, accelerate launch) so the same code runs on CPU, single GPU, multi-GPU, multi-node, TPU, or Apple MPS. Covers device placement, FP16\u002FBF16\u002FFP8 mixed precision, gradient accumulation, distributed checkpointing, and switching between DDP, DeepSpeed ZeRO, FSDP, and Megatron backends. Use when converting a single-GPU script to multi-GPU, enabling mixed precision, configuring DeepSpeed ZeRO or FSDP, or setting up gradient accumulation. Use when one script must run on different hardware. Not for callback-heavy training loops (use PyTorch Lightning) or multi-node orchestration with hyperparameter tuning (use Ray Train).","hashgraph-online\u002Fawesome-codex-plugins","hashgraph-online","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F160371962?v=4","01M6CNB6NRJVNNGN14TR01Q6D3","HOL (Hashgraph Online)","01M6JT1HSKKG9V68KQAQGBA7XP",1559,"MIT","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fhuggingface-accelerate\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39],"HuggingFace","Accelerate","huggingface accelerate","when","orchestration","loops","Unified API","DDP","FSDP","CPU","megatron","switching","apple","up","api","pytorch lightning","Transformers","code","training","pytorch","tpu","2026-11-10 10:12:30",false,"---\nname: huggingface-accelerate\ndescription: Wraps existing PyTorch training scripts with HuggingFace Accelerate (Accelerator class, accelerate config, accelerate launch) so the same code runs on CPU, single GPU, multi-GPU, multi-node, TPU, or Apple MPS. Covers device placement, FP16\u002FBF16\u002FFP8 mixed precision, gradient accumulation, distributed checkpointing, and switching between DDP, DeepSpeed ZeRO, FSDP, and Megatron backends. Use when converting a single-GPU script to multi-GPU, enabling mixed precision, configuring DeepSpeed ZeRO or FSDP, or setting up gradient accumulation. Use when one script must run on different hardware. Not for callback-heavy training loops (use PyTorch Lightning) or multi-node orchestration with hyperparameter tuning (use Ray Train).\nlicense: MIT\nmetadata:\n  version: 1.0.0\n  category: ml-training\n  maintainer: Kalaris Labs\n  tags: Distributed Training, HuggingFace, Accelerate, DeepSpeed, FSDP, Mixed Precision, PyTorch, DDP, Unified API, Simple\n  dependencies: accelerate, torch, transformers\n---\n\n# HuggingFace Accelerate - Unified Distributed Training\n\n## Quick start\n\nAccelerate simplifies distributed training to 4 lines of code.\n\n**Installation**:\n```bash\npip install accelerate\n```\n\n**Convert PyTorch script** (4 lines):\n```python\nimport torch\n+ from accelerate import Accelerator\n\n+ accelerator = Accelerator()\n\n  model = torch.nn.Transformer()\n  optimizer = torch.optim.Adam(model.parameters())\n  dataloader = torch.utils.data.DataLoader(dataset)\n\n+ model, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n\n  for batch in dataloader:\n      optimizer.zero_grad()\n      loss = model(batch)\n-     loss.backward()\n+     accelerator.backward(loss)\n      optimizer.step()\n```\n\n**Run** (single command):\n```bash\naccelerate launch train.py\n```\n\n## Common workflows\n\n### Workflow 1: From single GPU to multi-GPU\n\n**Original script**:\n```python\n# train.py\nimport torch\n\nmodel = torch.nn.Linear(10, 2).to('cuda')\noptimizer = torch.optim.Adam(model.parameters())\ndataloader = torch.utils.data.DataLoader(dataset, batch_size=32)\n\nfor epoch in range(10):\n    for batch in dataloader:\n        batch = batch.to('cuda')\n        optimizer.zero_grad()\n        loss = model(batch).mean()\n        loss.backward()\n        optimizer.step()\n```\n\n**With Accelerate** (4 lines added):\n```python\n# train.py\nimport torch\nfrom accelerate import Accelerator  # +1\n\naccelerator = Accelerator()  # +2\n\nmodel = torch.nn.Linear(10, 2)\noptimizer = torch.optim.Adam(model.parameters())\ndataloader = torch.utils.data.DataLoader(dataset, batch_size=32)\n\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)  # +3\n\nfor epoch in range(10):\n    for batch in dataloader:\n        # No .to('cuda') needed - automatic!\n        optimizer.zero_grad()\n        loss = model(batch).mean()\n        accelerator.backward(loss)  # +4\n        optimizer.step()\n```\n\n**Configure** (interactive):\n```bash\naccelerate config\n```\n\n**Questions**:\n- Which machine? (single\u002Fmulti GPU\u002FTPU\u002FCPU)\n- How many machines? (1)\n- Mixed precision? (no\u002Ffp16\u002Fbf16\u002Ffp8)\n- DeepSpeed? (no\u002Fyes)\n\n**Launch** (works on any setup):\n```bash\n# Single GPU\naccelerate launch train.py\n\n# Multi-GPU (8 GPUs)\naccelerate launch --multi_gpu --num_processes 8 train.py\n\n# Multi-node\naccelerate launch --multi_gpu --num_processes 16 \\\n  --num_machines 2 --machine_rank 0 \\\n  --main_process_ip $MASTER_ADDR \\\n  train.py\n```\n\n### Workflow 2: Mixed precision training\n\n**Enable FP16\u002FBF16**:\n```python\nfrom accelerate import Accelerator\n\n# FP16 (with gradient scaling)\naccelerator = Accelerator(mixed_precision='fp16')\n\n# BF16 (no scaling, more stable)\naccelerator = Accelerator(mixed_precision='bf16')\n\n# FP8 (H100+)\naccelerator = Accelerator(mixed_precision='fp8')\n\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n\n# Everything else is automatic!\nfor batch in dataloader:\n    with accelerator.autocast():  # Optional, done automatically\n        loss = model(batch)\n    accelerator.backward(loss)\n```\n\n### Workflow 3: DeepSpeed ZeRO integration\n\n**Enable DeepSpeed ZeRO-2**:\n```python\nfrom accelerate import Accelerator\n\naccelerator = Accelerator(\n    mixed_precision='bf16',\n    deepspeed_plugin={\n        \"zero_stage\": 2,  # ZeRO-2\n        \"offload_optimizer\": False,\n        \"gradient_accumulation_steps\": 4\n    }\n)\n\n# Same code as before!\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n```\n\n**Or via config**:\n```bash\naccelerate config\n# Select: DeepSpeed → ZeRO-2\n```\n\n**deepspeed_config.json**:\n```json\n{\n    \"fp16\": {\"enabled\": false},\n    \"bf16\": {\"enabled\": true},\n    \"zero_optimization\": {\n        \"stage\": 2,\n        \"offload_optimizer\": {\"device\": \"cpu\"},\n        \"allgather_bucket_size\": 5e8,\n        \"reduce_bucket_size\": 5e8\n    }\n}\n```\n\n**Launch**:\n```bash\naccelerate launch --config_file deepspeed_config.json train.py\n```\n\n### Workflow 4: FSDP (Fully Sharded Data Parallel)\n\n**Enable FSDP**:\n```python\nfrom accelerate import Accelerator, FullyShardedDataParallelPlugin\n\nfsdp_plugin = FullyShardedDataParallelPlugin(\n    sharding_strategy=\"FULL_SHARD\",  # ZeRO-3 equivalent\n    auto_wrap_policy=\"TRANSFORMER_AUTO_WRAP\",\n    cpu_offload=False\n)\n\naccelerator = Accelerator(\n    mixed_precision='bf16',\n    fsdp_plugin=fsdp_plugin\n)\n\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n```\n\n**Or via config**:\n```bash\naccelerate config\n# Select: FSDP → Full Shard → No CPU Offload\n```\n\n### Workflow 5: Gradient accumulation\n\n**Accumulate gradients**:\n```python\nfrom accelerate import Accelerator\n\naccelerator = Accelerator(gradient_accumulation_steps=4)\n\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n\nfor batch in dataloader:\n    with accelerator.accumulate(model):  # Handles accumulation\n        optimizer.zero_grad()\n        loss = model(batch)\n        accelerator.backward(loss)\n        optimizer.step()\n```\n\n**Effective batch size**: `batch_size * num_gpus * gradient_accumulation_steps`\n\n## When to use vs alternatives\n\n**Use Accelerate when**:\n- Want simplest distributed training\n- Need single script for any hardware\n- Use HuggingFace ecosystem\n- Want flexibility (DDP\u002FDeepSpeed\u002FFSDP\u002FMegatron)\n- Need quick prototyping\n\n**Key advantages**:\n- **4 lines**: Minimal code changes\n- **Unified API**: Same code for DDP, DeepSpeed, FSDP, Megatron\n- **Automatic**: Device placement, mixed precision, sharding\n- **Interactive config**: No manual launcher setup\n- **Single launch**: Works everywhere\n\n**Use alternatives instead**:\n- **PyTorch Lightning**: Need callbacks, high-level abstractions\n- **Ray Train**: Multi-node orchestration, hyperparameter tuning\n- **DeepSpeed**: Direct API control, advanced features\n- **Raw DDP**: Maximum control, minimal abstraction\n\n## Common issues\n\n**Issue: Wrong device placement**\n\nDon't manually move to device:\n```python\n# WRONG\nbatch = batch.to('cuda')\n\n# CORRECT\n# Accelerate handles it automatically after prepare()\n```\n\n**Issue: Gradient accumulation not working**\n\nUse context manager:\n```python\n# CORRECT\nwith accelerator.accumulate(model):\n    optimizer.zero_grad()\n    accelerator.backward(loss)\n    optimizer.step()\n```\n\n**Issue: Checkpointing in distributed**\n\nUse accelerator methods:\n```python\n# Save only on main process\nif accelerator.is_main_process:\n    accelerator.save_state('checkpoint\u002F')\n\n# Load on all processes\naccelerator.load_state('checkpoint\u002F')\n```\n\n**Issue: Different results with FSDP**\n\nEnsure same random seed:\n```python\nfrom accelerate.utils import set_seed\nset_seed(42)\n```\n\n## Advanced topics\n\n**Megatron integration**: See [references\u002Fmegatron-integration.md](references\u002Fmegatron-integration.md) for tensor parallelism, pipeline parallelism, and sequence parallelism setup.\n\n**Custom plugins**: See [references\u002Fcustom-plugins.md](references\u002Fcustom-plugins.md) for creating custom distributed plugins and advanced configuration.\n\n**Performance tuning**: See [references\u002Fperformance.md](references\u002Fperformance.md) for profiling, memory optimization, and best practices.\n\n## Hardware requirements\n\n- **CPU**: Works (slow)\n- **Single GPU**: Works\n- **Multi-GPU**: DDP (default), DeepSpeed, or FSDP\n- **Multi-node**: DDP, DeepSpeed, FSDP, Megatron\n- **TPU**: Supported\n- **Apple MPS**: Supported\n\n**Launcher requirements**:\n- **DDP**: `torch.distributed.run` (built-in)\n- **DeepSpeed**: `deepspeed` (pip install deepspeed)\n- **FSDP**: PyTorch 1.12+ (built-in)\n- **Megatron**: Custom setup\n\n## Resources\n\n- Docs: https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Faccelerate\n- GitHub: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Faccelerate\n- Version: 1.11.0+\n- Tutorial: \"Accelerate your scripts\"\n- Examples: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Faccelerate\u002Ftree\u002Fmain\u002Fexamples\n- Used by: HuggingFace Transformers, TRL, PEFT, all HF libraries\n\n## Agent operating procedure\n\n1. **Check the environment.** Check GPU type, memory and driver\u002FCUDA versions (`nvidia-smi`), framework versions, and dataset location and size.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Do a smoke run: tiny model or subset, few steps, and confirm loss decreases and checkpoints save.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Track metrics on held-out data, compare against a baseline, and record seeds, configs and hardware.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| CUDA out-of-memory | Reduce batch size, enable gradient accumulation\u002Fcheckpointing or mixed precision, or shard the model. |\n| Loss is NaN or diverges | Lower the learning rate, check data for invalid values, and enable gradient clipping. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Never claim training results without logs; estimate compute cost before launching large jobs.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `pytorch-lightning`: Organizes PyTorch training code with the lightning package (PyTorch Lightning): LightningModule, LightningDataModule, Trainer, callbacks su…\n- `pytorch-lightning-distributed`: High-level PyTorch framework with Trainer class, automatic distributed training (DDP\u002FFSDP\u002FDeepSpeed), callbacks system, and minimal boilerp…\n- `deepspeed`: Covers DeepSpeed for distributed deep learning training and I\u002FO: ZeRO optimization stages, pipeline parallelism, FP16\u002FBF16\u002FFP8 training, 1-…\n","# HuggingFace Accelerate - Unified Distributed Training\n\n## Quick start\n\nAccelerate simplifies distributed training to 4 lines of code.\n\n**Installation**:\n```bash\npip install accelerate\n```\n\n**Convert PyTorch script** (4 lines):\n```python\nimport torch\n+ from accelerate import Accelerator\n\n+ accelerator = Accelerator()\n\n  model = torch.nn.Transformer()\n  optimizer = torch.optim.Adam(model.parameters())\n  dataloader = torch.utils.data.DataLoader(dataset)\n\n+ model, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n\n  for batch in dataloader:\n      optimizer.zero_grad()\n      loss = model(batch)\n-     loss.backward()\n+     accelerator.backward(loss)\n      optimizer.step()\n```\n\n**Run** (single command):\n```bash\naccelerate launch train.py\n```\n\n## Common workflows\n\n### Workflow 1: From single GPU to multi-GPU\n\n**Original script**:\n```python\n# train.py\nimport torch\n\nmodel = torch.nn.Linear(10, 2).to('cuda')\noptimizer = torch.optim.Adam(model.parameters())\ndataloader = torch.utils.data.DataLoader(dataset, batch_size=32)\n\nfor epoch in range(10):\n    for batch in dataloader:\n        batch = batch.to('cuda')\n        optimizer.zero_grad()\n        loss = model(batch).mean()\n        loss.backward()\n        optimizer.step()\n```\n\n**With Accelerate** (4 lines added):\n```python\n# train.py\nimport torch\nfrom accelerate import Accelerator  # +1\n\naccelerator = Accelerator()  # +2\n\nmodel = torch.nn.Linear(10, 2)\noptimizer = torch.optim.Adam(model.parameters())\ndataloader = torch.utils.data.DataLoader(dataset, batch_size=32)\n\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)  # +3\n\nfor epoch in range(10):\n    for batch in dataloader:\n        # No .to('cuda') needed - automatic!\n        optimizer.zero_grad()\n        loss = model(batch).mean()\n        accelerator.backward(loss)  # +4\n        optimizer.step()\n```\n\n**Configure** (interactive):\n```bash\naccelerate config\n```\n\n**Questions**:\n- Which machine? (single\u002Fmulti GPU\u002FTPU\u002FCPU)\n- How many machines? (1)\n- Mixed precision? (no\u002Ffp16\u002Fbf16\u002Ffp8)\n- DeepSpeed? (no\u002Fyes)\n\n**Launch** (works on any setup):\n```bash\n# Single GPU\naccelerate launch train.py\n\n# Multi-GPU (8 GPUs)\naccelerate launch --multi_gpu --num_processes 8 train.py\n\n# Multi-node\naccelerate launch --multi_gpu --num_processes 16 \\\n  --num_machines 2 --machine_rank 0 \\\n  --main_process_ip $MASTER_ADDR \\\n  train.py\n```\n\n### Workflow 2: Mixed precision training\n\n**Enable FP16\u002FBF16**:\n```python\nfrom accelerate import Accelerator\n\n# FP16 (with gradient scaling)\naccelerator = Accelerator(mixed_precision='fp16')\n\n# BF16 (no scaling, more stable)\naccelerator = Accelerator(mixed_precision='bf16')\n\n# FP8 (H100+)\naccelerator = Accelerator(mixed_precision='fp8')\n\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n\n# Everything else is automatic!\nfor batch in dataloader:\n    with accelerator.autocast():  # Optional, done automatically\n        loss = model(batch)\n    accelerator.backward(loss)\n```\n\n### Workflow 3: DeepSpeed ZeRO integration\n\n**Enable DeepSpeed ZeRO-2**:\n```python\nfrom accelerate import Accelerator\n\naccelerator = Accelerator(\n    mixed_precision='bf16',\n    deepspeed_plugin={\n        \"zero_stage\": 2,  # ZeRO-2\n        \"offload_optimizer\": False,\n        \"gradient_accumulation_steps\": 4\n    }\n)\n\n# Same code as before!\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n```\n\n**Or via config**:\n```bash\naccelerate config\n# Select: DeepSpeed → ZeRO-2\n```\n\n**deepspeed_config.json**:\n```json\n{\n    \"fp16\": {\"enabled\": false},\n    \"bf16\": {\"enabled\": true},\n    \"zero_optimization\": {\n        \"stage\": 2,\n        \"offload_optimizer\": {\"device\": \"cpu\"},\n        \"allgather_bucket_size\": 5e8,\n        \"reduce_bucket_size\": 5e8\n    }\n}\n```\n\n**Launch**:\n```bash\naccelerate launch --config_file deepspeed_config.json train.py\n```\n\n### Workflow 4: FSDP (Fully Sharded Data Parallel)\n\n**Enable FSDP**:\n```python\nfrom accelerate import Accelerator, FullyShardedDataParallelPlugin\n\nfsdp_plugin = FullyShardedDataParallelPlugin(\n    sharding_strategy=\"FULL_SHARD\",  # ZeRO-3 equivalent\n    auto_wrap_policy=\"TRANSFORMER_AUTO_WRAP\",\n    cpu_offload=False\n)\n\naccelerator = Accelerator(\n    mixed_precision='bf16',\n    fsdp_plugin=fsdp_plugin\n)\n\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n```\n\n**Or via config**:\n```bash\naccelerate config\n# Select: FSDP → Full Shard → No CPU Offload\n```\n\n### Workflow 5: Gradient accumulation\n\n**Accumulate gradients**:\n```python\nfrom accelerate import Accelerator\n\naccelerator = Accelerator(gradient_accumulation_steps=4)\n\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\n\nfor batch in dataloader:\n    with accelerator.accumulate(model):  # Handles accumulation\n        optimizer.zero_grad()\n        loss = model(batch)\n        accelerator.backward(loss)\n        optimizer.step()\n```\n\n**Effective batch size**: `batch_size * num_gpus * gradient_accumulation_steps`\n\n## When to use vs alternatives\n\n**Use Accelerate when**:\n- Want simplest distributed training\n- Need single script for any hardware\n- Use HuggingFace ecosystem\n- Want flexibility (DDP\u002FDeepSpeed\u002FFSDP\u002FMegatron)\n- Need quick prototyping\n\n**Key advantages**:\n- **4 lines**: Minimal code changes\n- **Unified API**: Same code for DDP, DeepSpeed, FSDP, Megatron\n- **Automatic**: Device placement, mixed precision, sharding\n- **Interactive config**: No manual launcher setup\n- **Single launch**: Works everywhere\n\n**Use alternatives instead**:\n- **PyTorch Lightning**: Need callbacks, high-level abstractions\n- **Ray Train**: Multi-node orchestration, hyperparameter tuning\n- **DeepSpeed**: Direct API control, advanced features\n- **Raw DDP**: Maximum control, minimal abstraction\n\n## Common issues\n\n**Issue: Wrong device placement**\n\nDon't manually move to device:\n```python\n# WRONG\nbatch = batch.to('cuda')\n\n# CORRECT\n# Accelerate handles it automatically after prepare()\n```\n\n**Issue: Gradient accumulation not working**\n\nUse context manager:\n```python\n# CORRECT\nwith accelerator.accumulate(model):\n    optimizer.zero_grad()\n    accelerator.backward(loss)\n    optimizer.step()\n```\n\n**Issue: Checkpointing in distributed**\n\nUse accelerator methods:\n```python\n# Save only on main process\nif accelerator.is_main_process:\n    accelerator.save_state('checkpoint\u002F')\n\n# Load on all processes\naccelerator.load_state('checkpoint\u002F')\n```\n\n**Issue: Different results with FSDP**\n\nEnsure same random seed:\n```python\nfrom accelerate.utils import set_seed\nset_seed(42)\n```\n\n## Advanced topics\n\n**Megatron integration**: See [references\u002Fmegatron-integration.md](references\u002Fmegatron-integration.md) for tensor parallelism, pipeline parallelism, and sequence parallelism setup.\n\n**Custom plugins**: See [references\u002Fcustom-plugins.md](references\u002Fcustom-plugins.md) for creating custom distributed plugins and advanced configuration.\n\n**Performance tuning**: See [references\u002Fperformance.md](references\u002Fperformance.md) for profiling, memory optimization, and best practices.\n\n## Hardware requirements\n\n- **CPU**: Works (slow)\n- **Single GPU**: Works\n- **Multi-GPU**: DDP (default), DeepSpeed, or FSDP\n- **Multi-node**: DDP, DeepSpeed, FSDP, Megatron\n- **TPU**: Supported\n- **Apple MPS**: Supported\n\n**Launcher requirements**:\n- **DDP**: `torch.distributed.run` (built-in)\n- **DeepSpeed**: `deepspeed` (pip install deepspeed)\n- **FSDP**: PyTorch 1.12+ (built-in)\n- **Megatron**: Custom setup\n\n## Resources\n\n- Docs: https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Faccelerate\n- GitHub: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Faccelerate\n- Version: 1.11.0+\n- Tutorial: \"Accelerate your scripts\"\n- Examples: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Faccelerate\u002Ftree\u002Fmain\u002Fexamples\n- Used by: HuggingFace Transformers, TRL, PEFT, all HF libraries\n\n## Agent operating procedure\n\n1. **Check the environment.** Check GPU type, memory and driver\u002FCUDA versions (`nvidia-smi`), framework versions, and dataset location and size.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Do a smoke run: tiny model or subset, few steps, and confirm loss decreases and checkpoints save.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Track metrics on held-out data, compare against a baseline, and record seeds, configs and hardware.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| CUDA out-of-memory | Reduce batch size, enable gradient accumulation\u002Fcheckpointing or mixed precision, or shard the model. |\n| Loss is NaN or diverges | Lower the learning rate, check data for invalid values, and enable gradient clipping. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Never claim training results without logs; estimate compute cost before launching large jobs.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `pytorch-lightning`: Organizes PyTorch training code with the lightning package (PyTorch Lightning): LightningModule, LightningDataModule, Trainer, callbacks su…\n- `pytorch-lightning-distributed`: High-level PyTorch framework with Trainer class, automatic distributed training (DDP\u002FFSDP\u002FDeepSpeed), callbacks system, and minimal boilerp…\n- `deepspeed`: Covers DeepSpeed for distributed deep learning training and I\u002FO: ZeRO optimization stages, pipeline parallelism, FP16\u002FBF16\u002FFP8 training, 1-…\n",{"name":5,"license":15,"metadata":45,"description":7},{"tags":46,"version":47,"category":48,"maintainer":49,"dependencies":50},"Distributed Training, HuggingFace, Accelerate, DeepSpeed, FSDP, Mixed Precision, PyTorch, DDP, Unified API, Simple","1.0.0","ml-training","Kalaris Labs","accelerate, torch, transformers","main","2b63f856ce19c2a797929cacabd9e0fc936812fa","plugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fhuggingface-accelerate",474,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins","https:\u002F\u002Fhol.org\u002Fplugins\u002Fbest-codex-plugins","https:\u002F\u002Fgithub.com\u002Fhashgraph-online","https:\u002F\u002Fhol.org","Open-source standards and developer tools for interoperable AI agents: discovery, identity, communication, payments, privacy and security.","2026-11-11 14:40:14",{"source":8,"skillName":5,"commands":62,"prompt":72,"zipAvailable":73},[63,66,69],{"tool":64,"command":65},"npx","npx skills add hashgraph-online\u002Fawesome-codex-plugins --skill huggingface-accelerate",{"tool":67,"command":68},"bunx","bunx skills add hashgraph-online\u002Fawesome-codex-plugins --skill huggingface-accelerate",{"tool":70,"command":71},"pnpm","pnpm dlx skills add hashgraph-online\u002Fawesome-codex-plugins --skill huggingface-accelerate","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fhuggingface-accelerate\u002FSKILL.md",true,[75,81,85],{"path":76,"resourceType":77,"resourceTypeDesc":78,"sizeBytes":79,"sourceHtmlUrl":80},"references\u002Fcustom-plugins.md","REFERENCE","参考文档",11781,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fhuggingface-accelerate\u002Freferences\u002Fcustom-plugins.md",{"path":82,"resourceType":77,"resourceTypeDesc":78,"sizeBytes":83,"sourceHtmlUrl":84},"references\u002Fmegatron-integration.md",11252,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fhuggingface-accelerate\u002Freferences\u002Fmegatron-integration.md",{"path":86,"resourceType":77,"resourceTypeDesc":78,"sizeBytes":87,"sourceHtmlUrl":88},"references\u002Fperformance.md",12569,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fhuggingface-accelerate\u002Freferences\u002Fperformance.md",[90,99,106,113,139,164],{"skillNo":91,"name":92,"nameZh":6,"description":93,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":94,"licenseFileUrl":6,"sourceHtmlUrl":95,"categories":96,"keywords":97,"publishedAt":98},"01M746ZWACVKMZ1BJXJZHYJSSX","migrate-to-e2e","Moves an existing Playwright, Cypress, Selenium, Appium, Detox, Maestro, or TesterArmy suite to e2e one slice at a time, with the old suite running beside the new one until each ported test is proven. Keeps a MIGRATION.md progress file so any later session resumes where the last stopped. Optionally plants a defect to show both suites catch it. Use for \u002Fmigrate-to-e2e, \"migrate our tests to e2e\", \"port these Maestro flows\", \"move our TesterArmy tests\", or when a project has an e2e config and an older suite still running.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Ftester-army\u002Fe2e\u002Fskills\u002Fmigrate-to-e2e\u002FSKILL.md",[],[],"2026-11-11 15:11:14",{"skillNo":100,"name":101,"nameZh":6,"description":102,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":94,"licenseFileUrl":6,"sourceHtmlUrl":103,"categories":104,"keywords":105,"publishedAt":98},"01M746ZPTAX1Q0FBKAB1CYND7E","e2e","Writes, runs, and debugs end-to-end tests for web and mobile apps with e2e, the agentic test runner, which mixes agent.act goals with exact locator assertions on a Playwright browser or an iOS or Android device. Also drives a project's running app live over the e2e MCP server to look at a screen, check a UI change, take a screenshot or record a video, and try a locator; and runs bug bashes, parallel e2e explore runs with each bug proven by a repro test. Use when a project depends on e2e or has an e2e.config.ts, when asked for end-to-end, browser, mobile, or agentic UI tests, to check a change in the running app, to QA or bug bash an app or branch, or when an e2e run fails.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Ftester-army\u002Fe2e\u002Fskills\u002Fe2e\u002FSKILL.md",[],[],{"skillNo":107,"name":108,"nameZh":6,"description":109,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":94,"licenseFileUrl":6,"sourceHtmlUrl":110,"categories":111,"keywords":112,"publishedAt":98},"01M746ZJN784HVHMDP9MQGCSGG","create-verification-skill","Generate a project-local verify-\u003Capp> skill on top of e2e, so any coding agent can launch the app, check it is worth driving, exercise a feature the way a user does, keep evidence, and bug bash it. Writes the skill, a feature map with one explore charter per feature, and proves it once. Use for \u002Fcreate-verification-skill, \"make a verification skill for this repo\", or when a project has no scripted way to prove UI behavior.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Ftester-army\u002Fe2e\u002Fskills\u002Fcreate-verification-skill\u002FSKILL.md",[],[],{"skillNo":114,"name":115,"nameZh":6,"description":116,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":94,"licenseFileUrl":6,"sourceHtmlUrl":117,"categories":118,"keywords":119,"publishedAt":40},"01M716A6ETNB9B1SQ3GXZC75VR","deja-search","deja-vu memory — search the user's past AI coding sessions with the deja CLI. Holds what they already settled and never wrote into the repo (naming rules, banned or already-tried tools, required checks before merge, preferred values, who owns what). Use before your first edit in a task, before you call a change done or ready to merge, before you recommend a tool, library, design or reviewer, before re-debugging an error that may already be solved, when they say 'didn't we fix this before', 'what did we decide about X' or 'I already have that', and before telling them something on this machine does not exist.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fvshulcz\u002Fdeja-vu\u002Fskills\u002Fdeja-search\u002FSKILL.md",[],[120,121,122,123,124,125,126,127,128,129,130,22,131,132,133,134,135,136,137,138],"search","deja search","never","error","ready","memory","re","decide","fix","design","debugging","you","done","reviewer","io","change","cli","merge","past",{"skillNo":140,"name":141,"nameZh":6,"description":142,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":94,"licenseFileUrl":6,"sourceHtmlUrl":143,"categories":144,"keywords":145,"publishedAt":40},"01M716A3J2W5SENSE9VGMXB375","deja-reflect","End-of-session reflection and memory curation with deja-vu. Review what was decided, what commands broke and what repaired them, and record durable project rules or decisions with deja remember so the next session starts with them.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fvshulcz\u002Fdeja-vu\u002Fskills\u002Fdeja-reflect\u002FSKILL.md",[],[146,147,148,134,149,150,151,152,153,154,155,156,157,158,125,159,160,161,162,163],"deja reflect","reflect","review","with","record","github","next","https","node","Rules","decisions","brew","remember","project rules","project","session","end","reflection",{"skillNo":165,"name":166,"nameZh":6,"description":167,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":94,"licenseFileUrl":6,"sourceHtmlUrl":168,"categories":169,"keywords":181,"publishedAt":40},"01M715T3RR4BAYSR57HPPTJ0PF","revid","Make, edit, export and post AI videos with Revid (revid.ai) through the Revid MCP server. Use whenever the user wants a short-form or long-form video made (from an idea, a script, a URL, an article, a PDF, a tweet, a product page, their own clips, or a song), wants to find what is going viral on TikTok, YouTube, Instagram or X before making something, wants to edit or re-export an existing Revid project, generate a standalone AI image or clip, make a series episode, or publish and schedule a finished video to connected social accounts. Covers setup (hosted MCP with OAuth, or an API key), the 36 tools, credits, polling and the rules for publishing. Routes to the more specific revid-* skills when one fits.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Ftmakersas\u002Frevid-skills\u002Fskills\u002Frevid\u002FSKILL.md",[170,175,178],{"categoryNo":171,"categoryName":172,"taxonomyType":173,"taxonomyTypeDesc":174},"01JBSEEDCAT000000000000016","图像与多媒体","SCENARIO","使用场景",{"categoryNo":176,"categoryName":177,"taxonomyType":173,"taxonomyTypeDesc":174},"01JBSEEDCAT000000000000023","Skill与智能体开发",{"categoryNo":179,"categoryName":180,"taxonomyType":173,"taxonomyTypeDesc":174},"01JBSEEDCAT000000000000010","后端与接口",[166,182,183,184,185,186,187,160,188,126,189,190,191,192,22,193,194,195,33,196],"URL","Video","short-form","standalone","PDF","viral","tiktok","short","social","MCP","song","image","mcp server","schedule","publish"]