[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6DG3370EEEQZ938R0JWF7SS":3,"skill-same:01M6DG3370EEEQZ938R0JWF7SS":86},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":44,"linkOnly":45,"markdownContent":46,"markdownBody":47,"frontMatter":48,"refName":64,"commitSha":65,"skillDirectory":66,"forksCount":67,"repositoryHtmlUrl":68,"repositoryHomepageUrl":6,"ownerHtmlUrl":69,"ownerBlogUrl":6,"ownerDescription":70,"updatedAt":71,"install":72,"resources":85},"01M6DG3370EEEQZ938R0JWF7SS","huggingface-accelerate",null,"Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed\u002FFSDP\u002FMegatron\u002FDDP. Automatic device placement, mixed precision (FP16\u002FBF16\u002FFP8). Interactive config, single launch command. HuggingFace ecosystem standard.","zapabob\u002FSkills","zapabob","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F130830376?v=4","01M6DBMYZAGRTFQDSQYW3W764P","峯岸 亮","01M6DBMZ3GKDSJ01NZ9XTZD7JD",2,"MIT","https:\u002F\u002Fgithub.com\u002Fzapabob\u002FSkills\u002Fblob\u002Fmain\u002Farchive\u002Fskill-md-reviewed\u002Fsha256\u002F58\u002F5817bf829c0412f67de7e0fb111a9106bcb834137e82dfacd258b11538f41afa\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000010","后端与接口","SCENARIO","使用场景",[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43],"huggingface accelerate","HuggingFace","FP16","device","simple","interactive","command","api","Transformers","FP8","research","Mixed Precision","Distributed Training","DeepSpeed","training","pytorch","distributed","config","launch","support","2026-11-10 10:12:30",false,"---\r\nname: huggingface-accelerate\r\ndescription: Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed\u002FFSDP\u002FMegatron\u002FDDP. Automatic device placement, mixed precision (FP16\u002FBF16\u002FFP8). Interactive config, single launch command. HuggingFace ecosystem standard.\r\nversion: 1.0.0\r\nauthor: Orchestra Research\r\nlicense: MIT\r\ndependencies: [accelerate, torch, transformers]\r\nmetadata:\r\n  hermes:\r\n    tags: [Distributed Training, HuggingFace, Accelerate, DeepSpeed, FSDP, Mixed Precision, PyTorch, DDP, Unified API, Simple]\r\n\r\n---\r\n\r\n# HuggingFace Accelerate - Unified Distributed Training\r\n\r\n## Quick start\r\n\r\nAccelerate simplifies distributed training to 4 lines of code.\r\n\r\n**Installation**:\r\n```bash\r\npip install accelerate\r\n```\r\n\r\n**Convert PyTorch script** (4 lines):\r\n```python\r\nimport torch\r\n+ from accelerate import Accelerator\r\n\r\n+ accelerator = Accelerator()\r\n\r\n  model = torch.nn.Transformer()\r\n  optimizer = torch.optim.Adam(model.parameters())\r\n  dataloader = torch.utils.data.DataLoader(dataset)\r\n\r\n+ model, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\r\n\r\n  for batch in dataloader:\r\n      optimizer.zero_grad()\r\n      loss = model(batch)\r\n-     loss.backward()\r\n+     accelerator.backward(loss)\r\n      optimizer.step()\r\n```\r\n\r\n**Run** (single command):\r\n```bash\r\naccelerate launch train.py\r\n```\r\n\r\n## Common workflows\r\n\r\n### Workflow 1: From single GPU to multi-GPU\r\n\r\n**Original script**:\r\n```python\r\n# train.py\r\nimport torch\r\n\r\nmodel = torch.nn.Linear(10, 2).to('cuda')\r\noptimizer = torch.optim.Adam(model.parameters())\r\ndataloader = torch.utils.data.DataLoader(dataset, batch_size=32)\r\n\r\nfor epoch in range(10):\r\n    for batch in dataloader:\r\n        batch = batch.to('cuda')\r\n        optimizer.zero_grad()\r\n        loss = model(batch).mean()\r\n        loss.backward()\r\n        optimizer.step()\r\n```\r\n\r\n**With Accelerate** (4 lines added):\r\n```python\r\n# train.py\r\nimport torch\r\nfrom accelerate import Accelerator  # +1\r\n\r\naccelerator = Accelerator()  # +2\r\n\r\nmodel = torch.nn.Linear(10, 2)\r\noptimizer = torch.optim.Adam(model.parameters())\r\ndataloader = torch.utils.data.DataLoader(dataset, batch_size=32)\r\n\r\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)  # +3\r\n\r\nfor epoch in range(10):\r\n    for batch in dataloader:\r\n        # No .to('cuda') needed - automatic!\r\n        optimizer.zero_grad()\r\n        loss = model(batch).mean()\r\n        accelerator.backward(loss)  # +4\r\n        optimizer.step()\r\n```\r\n\r\n**Configure** (interactive):\r\n```bash\r\naccelerate config\r\n```\r\n\r\n**Questions**:\r\n- Which machine? (single\u002Fmulti GPU\u002FTPU\u002FCPU)\r\n- How many machines? (1)\r\n- Mixed precision? (no\u002Ffp16\u002Fbf16\u002Ffp8)\r\n- DeepSpeed? (no\u002Fyes)\r\n\r\n**Launch** (works on any setup):\r\n```bash\r\n# Single GPU\r\naccelerate launch train.py\r\n\r\n# Multi-GPU (8 GPUs)\r\naccelerate launch --multi_gpu --num_processes 8 train.py\r\n\r\n# Multi-node\r\naccelerate launch --multi_gpu --num_processes 16 \\\r\n  --num_machines 2 --machine_rank 0 \\\r\n  --main_process_ip $MASTER_ADDR \\\r\n  train.py\r\n```\r\n\r\n### Workflow 2: Mixed precision training\r\n\r\n**Enable FP16\u002FBF16**:\r\n```python\r\nfrom accelerate import Accelerator\r\n\r\n# FP16 (with gradient scaling)\r\naccelerator = Accelerator(mixed_precision='fp16')\r\n\r\n# BF16 (no scaling, more stable)\r\naccelerator = Accelerator(mixed_precision='bf16')\r\n\r\n# FP8 (H100+)\r\naccelerator = Accelerator(mixed_precision='fp8')\r\n\r\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\r\n\r\n# Everything else is automatic!\r\nfor batch in dataloader:\r\n    with accelerator.autocast():  # Optional, done automatically\r\n        loss = model(batch)\r\n    accelerator.backward(loss)\r\n```\r\n\r\n### Workflow 3: DeepSpeed ZeRO integration\r\n\r\n**Enable DeepSpeed ZeRO-2**:\r\n```python\r\nfrom accelerate import Accelerator\r\n\r\naccelerator = Accelerator(\r\n    mixed_precision='bf16',\r\n    deepspeed_plugin={\r\n        \"zero_stage\": 2,  # ZeRO-2\r\n        \"offload_optimizer\": False,\r\n        \"gradient_accumulation_steps\": 4\r\n    }\r\n)\r\n\r\n# Same code as before!\r\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\r\n```\r\n\r\n**Or via config**:\r\n```bash\r\naccelerate config\r\n# Select: DeepSpeed → ZeRO-2\r\n```\r\n\r\n**deepspeed_config.json**:\r\n```json\r\n{\r\n    \"fp16\": {\"enabled\": false},\r\n    \"bf16\": {\"enabled\": true},\r\n    \"zero_optimization\": {\r\n        \"stage\": 2,\r\n        \"offload_optimizer\": {\"device\": \"cpu\"},\r\n        \"allgather_bucket_size\": 5e8,\r\n        \"reduce_bucket_size\": 5e8\r\n    }\r\n}\r\n```\r\n\r\n**Launch**:\r\n```bash\r\naccelerate launch --config_file deepspeed_config.json train.py\r\n```\r\n\r\n### Workflow 4: FSDP (Fully Sharded Data Parallel)\r\n\r\n**Enable FSDP**:\r\n```python\r\nfrom accelerate import Accelerator, FullyShardedDataParallelPlugin\r\n\r\nfsdp_plugin = FullyShardedDataParallelPlugin(\r\n    sharding_strategy=\"FULL_SHARD\",  # ZeRO-3 equivalent\r\n    auto_wrap_policy=\"TRANSFORMER_AUTO_WRAP\",\r\n    cpu_offload=False\r\n)\r\n\r\naccelerator = Accelerator(\r\n    mixed_precision='bf16',\r\n    fsdp_plugin=fsdp_plugin\r\n)\r\n\r\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\r\n```\r\n\r\n**Or via config**:\r\n```bash\r\naccelerate config\r\n# Select: FSDP → Full Shard → No CPU Offload\r\n```\r\n\r\n### Workflow 5: Gradient accumulation\r\n\r\n**Accumulate gradients**:\r\n```python\r\nfrom accelerate import Accelerator\r\n\r\naccelerator = Accelerator(gradient_accumulation_steps=4)\r\n\r\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\r\n\r\nfor batch in dataloader:\r\n    with accelerator.accumulate(model):  # Handles accumulation\r\n        optimizer.zero_grad()\r\n        loss = model(batch)\r\n        accelerator.backward(loss)\r\n        optimizer.step()\r\n```\r\n\r\n**Effective batch size**: `batch_size * num_gpus * gradient_accumulation_steps`\r\n\r\n## When to use vs alternatives\r\n\r\n**Use Accelerate when**:\r\n- Want simplest distributed training\r\n- Need single script for any hardware\r\n- Use HuggingFace ecosystem\r\n- Want flexibility (DDP\u002FDeepSpeed\u002FFSDP\u002FMegatron)\r\n- Need quick prototyping\r\n\r\n**Key advantages**:\r\n- **4 lines**: Minimal code changes\r\n- **Unified API**: Same code for DDP, DeepSpeed, FSDP, Megatron\r\n- **Automatic**: Device placement, mixed precision, sharding\r\n- **Interactive config**: No manual launcher setup\r\n- **Single launch**: Works everywhere\r\n\r\n**Use alternatives instead**:\r\n- **PyTorch Lightning**: Need callbacks, high-level abstractions\r\n- **Ray Train**: Multi-node orchestration, hyperparameter tuning\r\n- **DeepSpeed**: Direct API control, advanced features\r\n- **Raw DDP**: Maximum control, minimal abstraction\r\n\r\n## Common issues\r\n\r\n**Issue: Wrong device placement**\r\n\r\nDon't manually move to device:\r\n```python\r\n# WRONG\r\nbatch = batch.to('cuda')\r\n\r\n# CORRECT\r\n# Accelerate handles it automatically after prepare()\r\n```\r\n\r\n**Issue: Gradient accumulation not working**\r\n\r\nUse context manager:\r\n```python\r\n# CORRECT\r\nwith accelerator.accumulate(model):\r\n    optimizer.zero_grad()\r\n    accelerator.backward(loss)\r\n    optimizer.step()\r\n```\r\n\r\n**Issue: Checkpointing in distributed**\r\n\r\nUse accelerator methods:\r\n```python\r\n# Save only on main process\r\nif accelerator.is_main_process:\r\n    accelerator.save_state('checkpoint\u002F')\r\n\r\n# Load on all processes\r\naccelerator.load_state('checkpoint\u002F')\r\n```\r\n\r\n**Issue: Different results with FSDP**\r\n\r\nEnsure same random seed:\r\n```python\r\nfrom accelerate.utils import set_seed\r\nset_seed(42)\r\n```\r\n\r\n## Advanced topics\r\n\r\n**Megatron integration**: See [references\u002Fmegatron-integration.md](references\u002Fmegatron-integration.md) for tensor parallelism, pipeline parallelism, and sequence parallelism setup.\r\n\r\n**Custom plugins**: See [references\u002Fcustom-plugins.md](references\u002Fcustom-plugins.md) for creating custom distributed plugins and advanced configuration.\r\n\r\n**Performance tuning**: See [references\u002Fperformance.md](references\u002Fperformance.md) for profiling, memory optimization, and best practices.\r\n\r\n## Hardware requirements\r\n\r\n- **CPU**: Works (slow)\r\n- **Single GPU**: Works\r\n- **Multi-GPU**: DDP (default), DeepSpeed, or FSDP\r\n- **Multi-node**: DDP, DeepSpeed, FSDP, Megatron\r\n- **TPU**: Supported\r\n- **Apple MPS**: Supported\r\n\r\n**Launcher requirements**:\r\n- **DDP**: `torch.distributed.run` (built-in)\r\n- **DeepSpeed**: `deepspeed` (pip install deepspeed)\r\n- **FSDP**: PyTorch 1.12+ (built-in)\r\n- **Megatron**: Custom setup\r\n\r\n## Resources\r\n\r\n- Docs: https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Faccelerate\r\n- GitHub: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Faccelerate\r\n- Version: 1.11.0+\r\n- Tutorial: \"Accelerate your scripts\"\r\n- Examples: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Faccelerate\u002Ftree\u002Fmain\u002Fexamples\r\n- Used by: HuggingFace Transformers, TRL, PEFT, all HF libraries\r\n","# HuggingFace Accelerate - Unified Distributed Training\r\n\r\n## Quick start\r\n\r\nAccelerate simplifies distributed training to 4 lines of code.\r\n\r\n**Installation**:\r\n```bash\r\npip install accelerate\r\n```\r\n\r\n**Convert PyTorch script** (4 lines):\r\n```python\r\nimport torch\r\n+ from accelerate import Accelerator\r\n\r\n+ accelerator = Accelerator()\r\n\r\n  model = torch.nn.Transformer()\r\n  optimizer = torch.optim.Adam(model.parameters())\r\n  dataloader = torch.utils.data.DataLoader(dataset)\r\n\r\n+ model, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\r\n\r\n  for batch in dataloader:\r\n      optimizer.zero_grad()\r\n      loss = model(batch)\r\n-     loss.backward()\r\n+     accelerator.backward(loss)\r\n      optimizer.step()\r\n```\r\n\r\n**Run** (single command):\r\n```bash\r\naccelerate launch train.py\r\n```\r\n\r\n## Common workflows\r\n\r\n### Workflow 1: From single GPU to multi-GPU\r\n\r\n**Original script**:\r\n```python\r\n# train.py\r\nimport torch\r\n\r\nmodel = torch.nn.Linear(10, 2).to('cuda')\r\noptimizer = torch.optim.Adam(model.parameters())\r\ndataloader = torch.utils.data.DataLoader(dataset, batch_size=32)\r\n\r\nfor epoch in range(10):\r\n    for batch in dataloader:\r\n        batch = batch.to('cuda')\r\n        optimizer.zero_grad()\r\n        loss = model(batch).mean()\r\n        loss.backward()\r\n        optimizer.step()\r\n```\r\n\r\n**With Accelerate** (4 lines added):\r\n```python\r\n# train.py\r\nimport torch\r\nfrom accelerate import Accelerator  # +1\r\n\r\naccelerator = Accelerator()  # +2\r\n\r\nmodel = torch.nn.Linear(10, 2)\r\noptimizer = torch.optim.Adam(model.parameters())\r\ndataloader = torch.utils.data.DataLoader(dataset, batch_size=32)\r\n\r\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)  # +3\r\n\r\nfor epoch in range(10):\r\n    for batch in dataloader:\r\n        # No .to('cuda') needed - automatic!\r\n        optimizer.zero_grad()\r\n        loss = model(batch).mean()\r\n        accelerator.backward(loss)  # +4\r\n        optimizer.step()\r\n```\r\n\r\n**Configure** (interactive):\r\n```bash\r\naccelerate config\r\n```\r\n\r\n**Questions**:\r\n- Which machine? (single\u002Fmulti GPU\u002FTPU\u002FCPU)\r\n- How many machines? (1)\r\n- Mixed precision? (no\u002Ffp16\u002Fbf16\u002Ffp8)\r\n- DeepSpeed? (no\u002Fyes)\r\n\r\n**Launch** (works on any setup):\r\n```bash\r\n# Single GPU\r\naccelerate launch train.py\r\n\r\n# Multi-GPU (8 GPUs)\r\naccelerate launch --multi_gpu --num_processes 8 train.py\r\n\r\n# Multi-node\r\naccelerate launch --multi_gpu --num_processes 16 \\\r\n  --num_machines 2 --machine_rank 0 \\\r\n  --main_process_ip $MASTER_ADDR \\\r\n  train.py\r\n```\r\n\r\n### Workflow 2: Mixed precision training\r\n\r\n**Enable FP16\u002FBF16**:\r\n```python\r\nfrom accelerate import Accelerator\r\n\r\n# FP16 (with gradient scaling)\r\naccelerator = Accelerator(mixed_precision='fp16')\r\n\r\n# BF16 (no scaling, more stable)\r\naccelerator = Accelerator(mixed_precision='bf16')\r\n\r\n# FP8 (H100+)\r\naccelerator = Accelerator(mixed_precision='fp8')\r\n\r\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\r\n\r\n# Everything else is automatic!\r\nfor batch in dataloader:\r\n    with accelerator.autocast():  # Optional, done automatically\r\n        loss = model(batch)\r\n    accelerator.backward(loss)\r\n```\r\n\r\n### Workflow 3: DeepSpeed ZeRO integration\r\n\r\n**Enable DeepSpeed ZeRO-2**:\r\n```python\r\nfrom accelerate import Accelerator\r\n\r\naccelerator = Accelerator(\r\n    mixed_precision='bf16',\r\n    deepspeed_plugin={\r\n        \"zero_stage\": 2,  # ZeRO-2\r\n        \"offload_optimizer\": False,\r\n        \"gradient_accumulation_steps\": 4\r\n    }\r\n)\r\n\r\n# Same code as before!\r\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\r\n```\r\n\r\n**Or via config**:\r\n```bash\r\naccelerate config\r\n# Select: DeepSpeed → ZeRO-2\r\n```\r\n\r\n**deepspeed_config.json**:\r\n```json\r\n{\r\n    \"fp16\": {\"enabled\": false},\r\n    \"bf16\": {\"enabled\": true},\r\n    \"zero_optimization\": {\r\n        \"stage\": 2,\r\n        \"offload_optimizer\": {\"device\": \"cpu\"},\r\n        \"allgather_bucket_size\": 5e8,\r\n        \"reduce_bucket_size\": 5e8\r\n    }\r\n}\r\n```\r\n\r\n**Launch**:\r\n```bash\r\naccelerate launch --config_file deepspeed_config.json train.py\r\n```\r\n\r\n### Workflow 4: FSDP (Fully Sharded Data Parallel)\r\n\r\n**Enable FSDP**:\r\n```python\r\nfrom accelerate import Accelerator, FullyShardedDataParallelPlugin\r\n\r\nfsdp_plugin = FullyShardedDataParallelPlugin(\r\n    sharding_strategy=\"FULL_SHARD\",  # ZeRO-3 equivalent\r\n    auto_wrap_policy=\"TRANSFORMER_AUTO_WRAP\",\r\n    cpu_offload=False\r\n)\r\n\r\naccelerator = Accelerator(\r\n    mixed_precision='bf16',\r\n    fsdp_plugin=fsdp_plugin\r\n)\r\n\r\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\r\n```\r\n\r\n**Or via config**:\r\n```bash\r\naccelerate config\r\n# Select: FSDP → Full Shard → No CPU Offload\r\n```\r\n\r\n### Workflow 5: Gradient accumulation\r\n\r\n**Accumulate gradients**:\r\n```python\r\nfrom accelerate import Accelerator\r\n\r\naccelerator = Accelerator(gradient_accumulation_steps=4)\r\n\r\nmodel, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)\r\n\r\nfor batch in dataloader:\r\n    with accelerator.accumulate(model):  # Handles accumulation\r\n        optimizer.zero_grad()\r\n        loss = model(batch)\r\n        accelerator.backward(loss)\r\n        optimizer.step()\r\n```\r\n\r\n**Effective batch size**: `batch_size * num_gpus * gradient_accumulation_steps`\r\n\r\n## When to use vs alternatives\r\n\r\n**Use Accelerate when**:\r\n- Want simplest distributed training\r\n- Need single script for any hardware\r\n- Use HuggingFace ecosystem\r\n- Want flexibility (DDP\u002FDeepSpeed\u002FFSDP\u002FMegatron)\r\n- Need quick prototyping\r\n\r\n**Key advantages**:\r\n- **4 lines**: Minimal code changes\r\n- **Unified API**: Same code for DDP, DeepSpeed, FSDP, Megatron\r\n- **Automatic**: Device placement, mixed precision, sharding\r\n- **Interactive config**: No manual launcher setup\r\n- **Single launch**: Works everywhere\r\n\r\n**Use alternatives instead**:\r\n- **PyTorch Lightning**: Need callbacks, high-level abstractions\r\n- **Ray Train**: Multi-node orchestration, hyperparameter tuning\r\n- **DeepSpeed**: Direct API control, advanced features\r\n- **Raw DDP**: Maximum control, minimal abstraction\r\n\r\n## Common issues\r\n\r\n**Issue: Wrong device placement**\r\n\r\nDon't manually move to device:\r\n```python\r\n# WRONG\r\nbatch = batch.to('cuda')\r\n\r\n# CORRECT\r\n# Accelerate handles it automatically after prepare()\r\n```\r\n\r\n**Issue: Gradient accumulation not working**\r\n\r\nUse context manager:\r\n```python\r\n# CORRECT\r\nwith accelerator.accumulate(model):\r\n    optimizer.zero_grad()\r\n    accelerator.backward(loss)\r\n    optimizer.step()\r\n```\r\n\r\n**Issue: Checkpointing in distributed**\r\n\r\nUse accelerator methods:\r\n```python\r\n# Save only on main process\r\nif accelerator.is_main_process:\r\n    accelerator.save_state('checkpoint\u002F')\r\n\r\n# Load on all processes\r\naccelerator.load_state('checkpoint\u002F')\r\n```\r\n\r\n**Issue: Different results with FSDP**\r\n\r\nEnsure same random seed:\r\n```python\r\nfrom accelerate.utils import set_seed\r\nset_seed(42)\r\n```\r\n\r\n## Advanced topics\r\n\r\n**Megatron integration**: See [references\u002Fmegatron-integration.md](references\u002Fmegatron-integration.md) for tensor parallelism, pipeline parallelism, and sequence parallelism setup.\r\n\r\n**Custom plugins**: See [references\u002Fcustom-plugins.md](references\u002Fcustom-plugins.md) for creating custom distributed plugins and advanced configuration.\r\n\r\n**Performance tuning**: See [references\u002Fperformance.md](references\u002Fperformance.md) for profiling, memory optimization, and best practices.\r\n\r\n## Hardware requirements\r\n\r\n- **CPU**: Works (slow)\r\n- **Single GPU**: Works\r\n- **Multi-GPU**: DDP (default), DeepSpeed, or FSDP\r\n- **Multi-node**: DDP, DeepSpeed, FSDP, Megatron\r\n- **TPU**: Supported\r\n- **Apple MPS**: Supported\r\n\r\n**Launcher requirements**:\r\n- **DDP**: `torch.distributed.run` (built-in)\r\n- **DeepSpeed**: `deepspeed` (pip install deepspeed)\r\n- **FSDP**: PyTorch 1.12+ (built-in)\r\n- **Megatron**: Custom setup\r\n\r\n## Resources\r\n\r\n- Docs: https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Faccelerate\r\n- GitHub: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Faccelerate\r\n- Version: 1.11.0+\r\n- Tutorial: \"Accelerate your scripts\"\r\n- Examples: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Faccelerate\u002Ftree\u002Fmain\u002Fexamples\r\n- Used by: HuggingFace Transformers, TRL, PEFT, all HF libraries\r\n",{"name":5,"author":49,"license":15,"version":50,"metadata":51,"description":7,"dependencies":60},"Orchestra Research","1.0.0",{"hermes":52},{"tags":53},[36,25,54,37,55,35,56,57,58,59],"Accelerate","FSDP","PyTorch","DDP","Unified API","Simple",[61,62,63],"accelerate","torch","transformers","main","0d1434f89f4bc6b07e9d272accd8c1013768820f","archive\u002Fskill-md-reviewed\u002Fsha256\u002F58\u002F5817bf829c0412f67de7e0fb111a9106bcb834137e82dfacd258b11538f41afa",0,"https:\u002F\u002Fgithub.com\u002Fzapabob\u002FSkills","https:\u002F\u002Fgithub.com\u002Fzapabob","The Open University of JAPAN　2019-now\r\n\r\nTokyo University of Science　2007-2010","2026-10-26 16:29:01",{"source":8,"skillName":5,"commands":73,"prompt":83,"zipAvailable":84},[74,77,80],{"tool":75,"command":76},"npx","npx skills add zapabob\u002FSkills --skill huggingface-accelerate",{"tool":78,"command":79},"bunx","bunx skills add zapabob\u002FSkills --skill huggingface-accelerate",{"tool":81,"command":82},"pnpm","pnpm dlx skills add zapabob\u002FSkills --skill huggingface-accelerate","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fzapabob\u002FSkills\u002Fblob\u002Fmain\u002Farchive\u002Fskill-md-reviewed\u002Fsha256\u002F58\u002F5817bf829c0412f67de7e0fb111a9106bcb834137e82dfacd258b11538f41afa\u002FSKILL.md",true,[],[87,120,155,181,207,224],{"skillNo":88,"name":89,"nameZh":6,"description":90,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":91,"categories":92,"keywords":102,"publishedAt":44},"01M6GJTE1GESGGF9K028Y7753E","xurl","X\u002FTwitter via xurl CLI: post, search, DM, media, v2 API.","https:\u002F\u002Fgithub.com\u002Fzapabob\u002FSkills\u002Fblob\u002Fmain\u002Fregistry\u002Fskills\u002Fxurl\u002Fvariants\u002Fhermes-agent\u002FSKILL.md",[93,94,97],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":95,"categoryName":96,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",{"categoryNo":98,"categoryName":99,"taxonomyType":100,"taxonomyTypeDesc":101},"01JBSEEDCAT000000000000030","开发工程师","PROFESSION","适用职业",[89,103,104,105,106,107,108,109,110,111,112,31,113,114,115,116,64,117,118,119],"openclaw","macos","cli","github","linux","windows","official","search","https","social-media","Media","hermes","hermes agent","twitter","social","via","post",{"skillNo":121,"name":122,"nameZh":6,"description":123,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":124,"categories":125,"keywords":137,"publishedAt":44},"01M6GJT8S1PXQ8WPXT1HSNBWH2","writing-plans","Write implementation plans: bite-sized tasks, paths, code.","https:\u002F\u002Fgithub.com\u002Fzapabob\u002FSkills\u002Fblob\u002Fmain\u002Fregistry\u002Fskills\u002Fwriting-plans\u002Fvariants\u002Fhermes-agent\u002FSKILL.md",[126,129,132,133,134],{"categoryNo":127,"categoryName":128,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000015","视觉设计",{"categoryNo":130,"categoryName":131,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000018","写作与文案",{"categoryNo":95,"categoryName":96,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":98,"categoryName":99,"taxonomyType":100,"taxonomyTypeDesc":101},{"categoryNo":135,"categoryName":136,"taxonomyType":100,"taxonomyTypeDesc":101},"01JBSEEDCAT000000000000036","内容创作者",[138,139,140,141,104,142,107,143,108,144,145,146,147,148,149,150,151,152,153,154],"writing","writing plans","plans","design","review","workflow","documentation","tasks","development","code","planning","code-review","subagent","write","superpowers","paths","test-driven-development",{"skillNo":156,"name":157,"nameZh":6,"description":158,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":159,"categories":160,"keywords":164,"publishedAt":44},"01M6GJST0H5X7WHBJ3BZK0DVKH","whisper","OpenAI's general-purpose speech recognition model. Supports 99 languages, transcription, translation to English, and language identification. Six model sizes from tiny (39M params) to large (1550M params). Use for speech-to-text, podcast transcription, or multilingual audio processing. Best for robust, multilingual ASR.","https:\u002F\u002Fgithub.com\u002Fzapabob\u002FSkills\u002Fblob\u002Fmain\u002Fregistry\u002Fskills\u002Fwhisper\u002Fvariants\u002Fhermes-agent-optional\u002FSKILL.md",[161],{"categoryNo":162,"categoryName":163,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000016","图像与多媒体",[157,104,107,165,166,167,32,168,169,170,171,172,173,174,175,176,177,178,179,180],"openai","speech","multimodal","ASR","Audio","text","model","speech recognition","use","general","multilingual","translation","english","identification","processing","Audio Processing",{"skillNo":182,"name":183,"nameZh":6,"description":184,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":185,"categories":186,"keywords":190,"publishedAt":44},"01M6GJSMRQ8EETZTYMG72P70EA","weights-and-biases","W&B: log ML experiments, sweeps, model registry, dashboards.","https:\u002F\u002Fgithub.com\u002Fzapabob\u002FSkills\u002Fblob\u002Fmain\u002Fregistry\u002Fskills\u002Fweights-and-biases\u002Fvariants\u002Fhermes-agent\u002FSKILL.md",[187],{"categoryNo":188,"categoryName":189,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000013","分析与可视化",[191,104,192,193,108,39,194,195,196,197,198,171,199,200,201,202,203,204,205,206],"weights and biases","tensorflow","experiments","visualization","registry","real-time","tuning","dashboards","experiment","log","time","Experiment Tracking","Model Registry","Hyperparameter Tuning","WandB","Real-Time Visualization",{"skillNo":208,"name":209,"nameZh":6,"description":210,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":211,"categories":212,"keywords":214,"publishedAt":44},"01M6GJS19KDPRY112JTR326S06","watchers","Poll RSS, JSON APIs, and GitHub with watermark dedup.","https:\u002F\u002Fgithub.com\u002Fzapabob\u002FSkills\u002Fblob\u002Fmain\u002Fregistry\u002Fskills\u002Fwatchers\u002Fvariants\u002Fhermes-agent-optional\u002FSKILL.md",[213],{"categoryNo":95,"categoryName":96,"taxonomyType":21,"taxonomyTypeDesc":22},[209,104,215,106,107,216,217,218,219,220,221,114,115,222,223],"terminal","automation","http","monitoring","cron","json","rss","devops","with",{"skillNo":225,"name":226,"nameZh":6,"description":227,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":228,"categories":229,"keywords":232,"publishedAt":44},"01M6GJRHJD2A6Q830SWF9W48BC","serving-llms-vllm","vLLM: high-throughput LLM serving, OpenAI API, quantization.","https:\u002F\u002Fgithub.com\u002Fzapabob\u002FSkills\u002Fblob\u002Fmain\u002Fregistry\u002Fskills\u002Fvllm\u002Fvariants\u002Fhermes-agent\u002FSKILL.md",[230,231],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":98,"categoryName":99,"taxonomyType":100,"taxonomyTypeDesc":101},[233,234,235,107,34,165,236,237,238,32,239,31,240,241,242,243,244,245,246,247],"llms","vLLM","serving llms vllm","openai api","tensor","Inference Serving","Production","llm serving","High Throughput","PagedAttention","Continuous Batching","Quantization","Tensor Parallelism","high-throughput","throughput"]