[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7D2MPEK9KPC4PDWYDDZWJ5D":3,"skill-same:01M7D2MPEK9KPC4PDWYDDZWJ5D":67},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":18,"linkOnly":19,"markdownContent":20,"markdownBody":21,"frontMatter":22,"refName":37,"commitSha":38,"skillDirectory":39,"forksCount":13,"repositoryHtmlUrl":40,"repositoryHomepageUrl":6,"ownerHtmlUrl":41,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":42,"install":43,"resources":56},"01M7D2MPEK9KPC4PDWYDDZWJ5D","slime-rl-training",null,"Provides guidance for LLM post-training with RL using slime, a Megatron+SGLang framework. Use when training GLM models, implementing custom data generation workflows, or needing tight Megatron-LM integration for RL scaling.","beyondai\u002Fsandbox","beyondai","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F2845958?v=4","01M7D24SBNM47BJGJ9J2HB13BP","01M7D24SCDC9DYE005XH6R553S",0,"MIT","https:\u002F\u002Fgithub.com\u002Fbeyondai\u002Fsandbox\u002Fblob\u002Fmain\u002Flabs\u002F.claude\u002Fskills\u002Fslime\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: slime-rl-training\ndescription: Provides guidance for LLM post-training with RL using slime, a Megatron+SGLang framework. Use when training GLM models, implementing custom data generation workflows, or needing tight Megatron-LM integration for RL scaling.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Reinforcement Learning, Megatron-LM, SGLang, GRPO, Post-Training, GLM]\ndependencies: [sglang-router>=0.2.3, ray, torch>=2.0.0, transformers>=4.40.0]\n---\n\n# slime: LLM Post-Training Framework for RL Scaling\n\nslime is an LLM post-training framework from Tsinghua's THUDM team, powering GLM-4.5, GLM-4.6, and GLM-4.7. It connects Megatron-LM for training with SGLang for high-throughput rollout generation.\n\n## When to Use slime\n\n**Choose slime when you need:**\n- Megatron-LM native training with SGLang inference\n- Custom data generation workflows with flexible data buffers\n- Training GLM, Qwen3, DeepSeek V3, or Llama 3 models\n- Research-grade framework with production backing (Z.ai)\n\n**Consider alternatives when:**\n- You need enterprise-grade stability features → use **miles**\n- You want flexible backend swapping → use **verl**\n- You need PyTorch-native abstractions → use **torchforge**\n\n## Key Features\n\n- **Training**: Megatron-LM with full parallelism support (TP, PP, DP, SP)\n- **Rollout**: SGLang-based high-throughput generation with router\n- **Data Buffer**: Flexible prompt management and sample storage\n- **Models**: GLM-4.x, Qwen3, DeepSeek V3\u002FR1, Llama 3\n\n## Architecture Overview\n\n```\n┌─────────────────────────────────────────────────────────┐\n│                    Data Buffer                          │\n│ - Prompt initialization and management                  │\n│ - Custom data generation and filtering                  │\n│ - Rollout sample storage                                │\n└─────────────┬───────────────────────────┬───────────────┘\n              │                           │\n┌─────────────▼───────────┐ ┌─────────────▼───────────────┐\n│ Training (Megatron-LM)  │ │ Rollout (SGLang + Router)   │\n│ - Actor model training  │ │ - Response generation       │\n│ - Critic (optional)     │ │ - Reward\u002Fverifier output    │\n│ - Weight sync to rollout│ │ - Multi-turn support        │\n└─────────────────────────┘ └─────────────────────────────┘\n```\n\n## Installation\n\n```bash\n# Recommended: Docker\ndocker pull slimerl\u002Fslime:latest\ndocker run --rm --gpus all --ipc=host --shm-size=16g \\\n  -it slimerl\u002Fslime:latest \u002Fbin\u002Fbash\n\n# Inside container\ncd \u002Froot\u002Fslime && pip install -e . --no-deps\n```\n\n### From Source\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FTHUDM\u002Fslime.git\ncd slime\npip install -r requirements.txt\npip install -e .\n```\n\n## Quick Start: GRPO Training\n\n```bash\n# Source model configuration\nsource scripts\u002Fmodels\u002Fqwen3-4B.sh\n\n# Launch training\npython train.py \\\n    --actor-num-nodes 1 \\\n    --actor-num-gpus-per-node 4 \\\n    --rollout-num-gpus 4 \\\n    --advantage-estimator grpo \\\n    --use-kl-loss --kl-loss-coef 0.001 \\\n    --rollout-batch-size 32 \\\n    --n-samples-per-prompt 8 \\\n    --global-batch-size 256 \\\n    --num-rollout 3000 \\\n    --prompt-data \u002Fpath\u002Fto\u002Fdata.jsonl \\\n    ${MODEL_ARGS[@]} ${CKPT_ARGS[@]}\n```\n\n---\n\n## Workflow 1: Standard GRPO Training\n\nUse this workflow for training reasoning models with group-relative advantages.\n\n### Prerequisites Checklist\n- [ ] Docker environment or Megatron-LM + SGLang installed\n- [ ] Model checkpoint (HuggingFace or Megatron format)\n- [ ] Training data in JSONL format\n\n### Step 1: Prepare Data\n\n```python\n# data.jsonl format\n{\"prompt\": \"What is 2 + 2?\", \"label\": \"4\"}\n{\"prompt\": \"Solve: 3x = 12\", \"label\": \"x = 4\"}\n```\n\nOr with chat format:\n```python\n{\n    \"prompt\": [\n        {\"role\": \"system\", \"content\": \"You are a math tutor.\"},\n        {\"role\": \"user\", \"content\": \"What is 15 + 27?\"}\n    ],\n    \"label\": \"42\"\n}\n```\n\n### Step 2: Configure Model\n\nChoose a pre-configured model script:\n\n```bash\n# List available models\nls scripts\u002Fmodels\u002F\n# glm4-9B.sh, qwen3-4B.sh, qwen3-30B-A3B.sh, deepseek-v3.sh, llama3-8B.sh, ...\n\n# Source your model\nsource scripts\u002Fmodels\u002Fqwen3-4B.sh\n```\n\n### Step 3: Launch Training\n\n```bash\npython train.py \\\n    --actor-num-nodes 1 \\\n    --actor-num-gpus-per-node 8 \\\n    --rollout-num-gpus 8 \\\n    --advantage-estimator grpo \\\n    --use-kl-loss \\\n    --kl-loss-coef 0.001 \\\n    --prompt-data \u002Fpath\u002Fto\u002Ftrain.jsonl \\\n    --input-key prompt \\\n    --label-key label \\\n    --apply-chat-template \\\n    --rollout-batch-size 32 \\\n    --n-samples-per-prompt 8 \\\n    --global-batch-size 256 \\\n    --num-rollout 3000 \\\n    --save-interval 100 \\\n    --eval-interval 50 \\\n    ${MODEL_ARGS[@]}\n```\n\n### Step 4: Monitor Training\n- [ ] Check TensorBoard: `tensorboard --logdir outputs\u002F`\n- [ ] Verify reward curves are increasing\n- [ ] Monitor GPU utilization across nodes\n\n---\n\n## Workflow 2: Asynchronous Training\n\nUse async mode for higher throughput by overlapping rollout and training.\n\n### When to Use Async\n- Large models with long generation times\n- High GPU idle time in synchronous mode\n- Sufficient memory for buffering\n\n### Launch Async Training\n\n```bash\npython train_async.py \\\n    --actor-num-nodes 1 \\\n    --actor-num-gpus-per-node 8 \\\n    --rollout-num-gpus 8 \\\n    --advantage-estimator grpo \\\n    --async-buffer-size 4 \\\n    --prompt-data \u002Fpath\u002Fto\u002Ftrain.jsonl \\\n    ${MODEL_ARGS[@]}\n```\n\n### Async-Specific Parameters\n\n```bash\n--async-buffer-size 4        # Number of rollouts to buffer\n--update-weights-interval 2  # Sync weights every N rollouts\n```\n\n---\n\n## Workflow 3: Multi-Turn Agentic Training\n\nUse this workflow for training agents with tool use or multi-step reasoning.\n\n### Prerequisites\n- [ ] Custom generate function for multi-turn logic\n- [ ] Tool\u002Fenvironment interface\n\n### Step 1: Define Custom Generate Function\n\n```python\n# custom_generate.py\nasync def custom_generate(args, samples, evaluation=False):\n    \"\"\"Multi-turn generation with tool calling.\"\"\"\n    for sample in samples:\n        conversation = sample.prompt\n\n        for turn in range(args.max_turns):\n            # Generate response\n            response = await generate_single(conversation)\n\n            # Check for tool call\n            tool_call = extract_tool_call(response)\n            if tool_call:\n                tool_result = execute_tool(tool_call)\n                conversation.append({\"role\": \"assistant\", \"content\": response})\n                conversation.append({\"role\": \"tool\", \"content\": tool_result})\n            else:\n                break\n\n        sample.response = response\n        sample.reward = compute_reward(sample)\n\n    return samples\n```\n\n### Step 2: Launch with Custom Function\n\n```bash\npython train.py \\\n    --custom-generate-function-path custom_generate.py \\\n    --max-turns 5 \\\n    --prompt-data \u002Fpath\u002Fto\u002Fagent_data.jsonl \\\n    ${MODEL_ARGS[@]}\n```\n\nSee `examples\u002Fsearch-r1\u002F` for a complete multi-turn search example.\n\n---\n\n## Configuration Reference\n\n### Three Argument Categories\n\nslime uses three types of arguments:\n\n**1. Megatron Arguments** (passed directly):\n```bash\n--tensor-model-parallel-size 2\n--pipeline-model-parallel-size 1\n--num-layers 32\n--hidden-size 4096\n```\n\n**2. SGLang Arguments** (prefixed with `--sglang-`):\n```bash\n--sglang-mem-fraction-static 0.8\n--sglang-context-length 8192\n--sglang-log-level INFO\n```\n\n**3. slime Arguments**:\n```bash\n# Resource allocation\n--actor-num-nodes 1\n--actor-num-gpus-per-node 8\n--rollout-num-gpus 8\n--colocate  # Share GPUs between training\u002Finference\n\n# Data\n--prompt-data \u002Fpath\u002Fto\u002Fdata.jsonl\n--input-key prompt\n--label-key label\n\n# Training loop\n--num-rollout 3000\n--rollout-batch-size 32\n--n-samples-per-prompt 8\n--global-batch-size 256\n\n# Algorithm\n--advantage-estimator grpo  # or: gspo, ppo, reinforce_plus_plus\n--use-kl-loss\n--kl-loss-coef 0.001\n```\n\n### Key Constraints\n\n```\nrollout_batch_size × n_samples_per_prompt = global_batch_size × num_steps_per_rollout\n```\n\nExample: 32 × 8 = 256 × 1\n\n---\n\n## Data Buffer System\n\nslime's data buffer enables flexible data management:\n\n### Basic Data Source\n\n```python\nclass RolloutDataSource:\n    def get_samples(self, num_samples):\n        \"\"\"Fetch prompts from dataset.\"\"\"\n        return self.dataset.sample(num_samples)\n\n    def add_samples(self, samples):\n        \"\"\"Called after generation (no-op by default).\"\"\"\n        pass\n```\n\n### Buffered Data Source (Off-Policy)\n\n```python\nclass RolloutDataSourceWithBuffer(RolloutDataSource):\n    def __init__(self):\n        self.buffer = []\n\n    def add_samples(self, samples):\n        \"\"\"Store generated samples for reuse.\"\"\"\n        self.buffer.extend(samples)\n\n    def buffer_filter(self, args, buffer, num_samples):\n        \"\"\"Custom selection logic (prioritized, stratified, etc.).\"\"\"\n        return select_best(buffer, num_samples)\n```\n\n---\n\n## Common Issues and Solutions\n\n### Issue: SGLang Engine Crash\n\n**Symptoms**: Inference engine dies mid-training\n\n**Solutions**:\n```bash\n# Enable fault tolerance\n--use-fault-tolerance\n\n# Increase memory allocation\n--sglang-mem-fraction-static 0.85\n\n# Reduce batch size\n--rollout-batch-size 16\n```\n\n### Issue: Weight Sync Timeout\n\n**Symptoms**: Training hangs after rollout\n\n**Solutions**:\n```bash\n# Increase sync interval\n--update-weights-interval 5\n\n# Use colocated mode (no network transfer)\n--colocate\n```\n\n### Issue: OOM During Training\n\n**Symptoms**: CUDA OOM in backward pass\n\n**Solutions**:\n```bash\n# Enable gradient checkpointing\n--recompute-activations\n\n# Reduce micro-batch size\n--micro-batch-size 1\n\n# Enable sequence parallelism\n--sequence-parallel\n```\n\n### Issue: Slow Data Loading\n\n**Symptoms**: GPU idle during data fetch\n\n**Solutions**:\n```bash\n# Increase data workers\n--num-data-workers 4\n\n# Use streaming dataset\n--streaming-data\n```\n\n---\n\n## Supported Models\n\n| Model Family | Configurations |\n|--------------|----------------|\n| GLM | GLM-4.5, GLM-4.6, GLM-4.7, GLM-Z1-9B |\n| Qwen | Qwen3 (4B, 8B, 30B-A3B), Qwen3-MoE, Qwen2.5 |\n| DeepSeek | V3, V3.1, R1 |\n| Llama | Llama 3 (8B, 70B) |\n| Others | Kimi K2, Moonlight-16B |\n\nEach model has pre-configured scripts in `scripts\u002Fmodels\u002F`.\n\n---\n\n## Advanced Topics\n\n### Co-location Mode\n\nShare GPUs between training and inference to reduce memory:\n\n```bash\npython train.py \\\n    --colocate \\\n    --actor-num-gpus-per-node 8 \\\n    --sglang-mem-fraction-static 0.4 \\\n    ${MODEL_ARGS[@]}\n```\n\n### Custom Reward Model\n\n```python\n# custom_rm.py\nclass CustomRewardModel:\n    def __init__(self, model_path):\n        self.model = load_model(model_path)\n\n    def compute_reward(self, prompts, responses):\n        inputs = self.tokenize(prompts, responses)\n        scores = self.model(inputs)\n        return scores.tolist()\n```\n\n```bash\n--custom-rm-path custom_rm.py\n```\n\n### Evaluation Multi-Task\n\n```bash\n--eval-prompt-data aime \u002Fpath\u002Fto\u002Faime.jsonl \\\n--eval-prompt-data gsm8k \u002Fpath\u002Fto\u002Fgsm8k.jsonl \\\n--n-samples-per-eval-prompt 16\n```\n\n---\n\n## Resources\n\n- **Documentation**: https:\u002F\u002Fthudm.github.io\u002Fslime\u002F\n- **GitHub**: https:\u002F\u002Fgithub.com\u002FTHUDM\u002Fslime\n- **Blog**: https:\u002F\u002Flmsys.org\u002Fblog\u002F2025-07-09-slime\u002F\n- **Examples**: See `examples\u002F` directory for 14+ worked examples\n\n","# slime: LLM Post-Training Framework for RL Scaling\n\nslime is an LLM post-training framework from Tsinghua's THUDM team, powering GLM-4.5, GLM-4.6, and GLM-4.7. It connects Megatron-LM for training with SGLang for high-throughput rollout generation.\n\n## When to Use slime\n\n**Choose slime when you need:**\n- Megatron-LM native training with SGLang inference\n- Custom data generation workflows with flexible data buffers\n- Training GLM, Qwen3, DeepSeek V3, or Llama 3 models\n- Research-grade framework with production backing (Z.ai)\n\n**Consider alternatives when:**\n- You need enterprise-grade stability features → use **miles**\n- You want flexible backend swapping → use **verl**\n- You need PyTorch-native abstractions → use **torchforge**\n\n## Key Features\n\n- **Training**: Megatron-LM with full parallelism support (TP, PP, DP, SP)\n- **Rollout**: SGLang-based high-throughput generation with router\n- **Data Buffer**: Flexible prompt management and sample storage\n- **Models**: GLM-4.x, Qwen3, DeepSeek V3\u002FR1, Llama 3\n\n## Architecture Overview\n\n```\n┌─────────────────────────────────────────────────────────┐\n│                    Data Buffer                          │\n│ - Prompt initialization and management                  │\n│ - Custom data generation and filtering                  │\n│ - Rollout sample storage                                │\n└─────────────┬───────────────────────────┬───────────────┘\n              │                           │\n┌─────────────▼───────────┐ ┌─────────────▼───────────────┐\n│ Training (Megatron-LM)  │ │ Rollout (SGLang + Router)   │\n│ - Actor model training  │ │ - Response generation       │\n│ - Critic (optional)     │ │ - Reward\u002Fverifier output    │\n│ - Weight sync to rollout│ │ - Multi-turn support        │\n└─────────────────────────┘ └─────────────────────────────┘\n```\n\n## Installation\n\n```bash\n# Recommended: Docker\ndocker pull slimerl\u002Fslime:latest\ndocker run --rm --gpus all --ipc=host --shm-size=16g \\\n  -it slimerl\u002Fslime:latest \u002Fbin\u002Fbash\n\n# Inside container\ncd \u002Froot\u002Fslime && pip install -e . --no-deps\n```\n\n### From Source\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FTHUDM\u002Fslime.git\ncd slime\npip install -r requirements.txt\npip install -e .\n```\n\n## Quick Start: GRPO Training\n\n```bash\n# Source model configuration\nsource scripts\u002Fmodels\u002Fqwen3-4B.sh\n\n# Launch training\npython train.py \\\n    --actor-num-nodes 1 \\\n    --actor-num-gpus-per-node 4 \\\n    --rollout-num-gpus 4 \\\n    --advantage-estimator grpo \\\n    --use-kl-loss --kl-loss-coef 0.001 \\\n    --rollout-batch-size 32 \\\n    --n-samples-per-prompt 8 \\\n    --global-batch-size 256 \\\n    --num-rollout 3000 \\\n    --prompt-data \u002Fpath\u002Fto\u002Fdata.jsonl \\\n    ${MODEL_ARGS[@]} ${CKPT_ARGS[@]}\n```\n\n---\n\n## Workflow 1: Standard GRPO Training\n\nUse this workflow for training reasoning models with group-relative advantages.\n\n### Prerequisites Checklist\n- [ ] Docker environment or Megatron-LM + SGLang installed\n- [ ] Model checkpoint (HuggingFace or Megatron format)\n- [ ] Training data in JSONL format\n\n### Step 1: Prepare Data\n\n```python\n# data.jsonl format\n{\"prompt\": \"What is 2 + 2?\", \"label\": \"4\"}\n{\"prompt\": \"Solve: 3x = 12\", \"label\": \"x = 4\"}\n```\n\nOr with chat format:\n```python\n{\n    \"prompt\": [\n        {\"role\": \"system\", \"content\": \"You are a math tutor.\"},\n        {\"role\": \"user\", \"content\": \"What is 15 + 27?\"}\n    ],\n    \"label\": \"42\"\n}\n```\n\n### Step 2: Configure Model\n\nChoose a pre-configured model script:\n\n```bash\n# List available models\nls scripts\u002Fmodels\u002F\n# glm4-9B.sh, qwen3-4B.sh, qwen3-30B-A3B.sh, deepseek-v3.sh, llama3-8B.sh, ...\n\n# Source your model\nsource scripts\u002Fmodels\u002Fqwen3-4B.sh\n```\n\n### Step 3: Launch Training\n\n```bash\npython train.py \\\n    --actor-num-nodes 1 \\\n    --actor-num-gpus-per-node 8 \\\n    --rollout-num-gpus 8 \\\n    --advantage-estimator grpo \\\n    --use-kl-loss \\\n    --kl-loss-coef 0.001 \\\n    --prompt-data \u002Fpath\u002Fto\u002Ftrain.jsonl \\\n    --input-key prompt \\\n    --label-key label \\\n    --apply-chat-template \\\n    --rollout-batch-size 32 \\\n    --n-samples-per-prompt 8 \\\n    --global-batch-size 256 \\\n    --num-rollout 3000 \\\n    --save-interval 100 \\\n    --eval-interval 50 \\\n    ${MODEL_ARGS[@]}\n```\n\n### Step 4: Monitor Training\n- [ ] Check TensorBoard: `tensorboard --logdir outputs\u002F`\n- [ ] Verify reward curves are increasing\n- [ ] Monitor GPU utilization across nodes\n\n---\n\n## Workflow 2: Asynchronous Training\n\nUse async mode for higher throughput by overlapping rollout and training.\n\n### When to Use Async\n- Large models with long generation times\n- High GPU idle time in synchronous mode\n- Sufficient memory for buffering\n\n### Launch Async Training\n\n```bash\npython train_async.py \\\n    --actor-num-nodes 1 \\\n    --actor-num-gpus-per-node 8 \\\n    --rollout-num-gpus 8 \\\n    --advantage-estimator grpo \\\n    --async-buffer-size 4 \\\n    --prompt-data \u002Fpath\u002Fto\u002Ftrain.jsonl \\\n    ${MODEL_ARGS[@]}\n```\n\n### Async-Specific Parameters\n\n```bash\n--async-buffer-size 4        # Number of rollouts to buffer\n--update-weights-interval 2  # Sync weights every N rollouts\n```\n\n---\n\n## Workflow 3: Multi-Turn Agentic Training\n\nUse this workflow for training agents with tool use or multi-step reasoning.\n\n### Prerequisites\n- [ ] Custom generate function for multi-turn logic\n- [ ] Tool\u002Fenvironment interface\n\n### Step 1: Define Custom Generate Function\n\n```python\n# custom_generate.py\nasync def custom_generate(args, samples, evaluation=False):\n    \"\"\"Multi-turn generation with tool calling.\"\"\"\n    for sample in samples:\n        conversation = sample.prompt\n\n        for turn in range(args.max_turns):\n            # Generate response\n            response = await generate_single(conversation)\n\n            # Check for tool call\n            tool_call = extract_tool_call(response)\n            if tool_call:\n                tool_result = execute_tool(tool_call)\n                conversation.append({\"role\": \"assistant\", \"content\": response})\n                conversation.append({\"role\": \"tool\", \"content\": tool_result})\n            else:\n                break\n\n        sample.response = response\n        sample.reward = compute_reward(sample)\n\n    return samples\n```\n\n### Step 2: Launch with Custom Function\n\n```bash\npython train.py \\\n    --custom-generate-function-path custom_generate.py \\\n    --max-turns 5 \\\n    --prompt-data \u002Fpath\u002Fto\u002Fagent_data.jsonl \\\n    ${MODEL_ARGS[@]}\n```\n\nSee `examples\u002Fsearch-r1\u002F` for a complete multi-turn search example.\n\n---\n\n## Configuration Reference\n\n### Three Argument Categories\n\nslime uses three types of arguments:\n\n**1. Megatron Arguments** (passed directly):\n```bash\n--tensor-model-parallel-size 2\n--pipeline-model-parallel-size 1\n--num-layers 32\n--hidden-size 4096\n```\n\n**2. SGLang Arguments** (prefixed with `--sglang-`):\n```bash\n--sglang-mem-fraction-static 0.8\n--sglang-context-length 8192\n--sglang-log-level INFO\n```\n\n**3. slime Arguments**:\n```bash\n# Resource allocation\n--actor-num-nodes 1\n--actor-num-gpus-per-node 8\n--rollout-num-gpus 8\n--colocate  # Share GPUs between training\u002Finference\n\n# Data\n--prompt-data \u002Fpath\u002Fto\u002Fdata.jsonl\n--input-key prompt\n--label-key label\n\n# Training loop\n--num-rollout 3000\n--rollout-batch-size 32\n--n-samples-per-prompt 8\n--global-batch-size 256\n\n# Algorithm\n--advantage-estimator grpo  # or: gspo, ppo, reinforce_plus_plus\n--use-kl-loss\n--kl-loss-coef 0.001\n```\n\n### Key Constraints\n\n```\nrollout_batch_size × n_samples_per_prompt = global_batch_size × num_steps_per_rollout\n```\n\nExample: 32 × 8 = 256 × 1\n\n---\n\n## Data Buffer System\n\nslime's data buffer enables flexible data management:\n\n### Basic Data Source\n\n```python\nclass RolloutDataSource:\n    def get_samples(self, num_samples):\n        \"\"\"Fetch prompts from dataset.\"\"\"\n        return self.dataset.sample(num_samples)\n\n    def add_samples(self, samples):\n        \"\"\"Called after generation (no-op by default).\"\"\"\n        pass\n```\n\n### Buffered Data Source (Off-Policy)\n\n```python\nclass RolloutDataSourceWithBuffer(RolloutDataSource):\n    def __init__(self):\n        self.buffer = []\n\n    def add_samples(self, samples):\n        \"\"\"Store generated samples for reuse.\"\"\"\n        self.buffer.extend(samples)\n\n    def buffer_filter(self, args, buffer, num_samples):\n        \"\"\"Custom selection logic (prioritized, stratified, etc.).\"\"\"\n        return select_best(buffer, num_samples)\n```\n\n---\n\n## Common Issues and Solutions\n\n### Issue: SGLang Engine Crash\n\n**Symptoms**: Inference engine dies mid-training\n\n**Solutions**:\n```bash\n# Enable fault tolerance\n--use-fault-tolerance\n\n# Increase memory allocation\n--sglang-mem-fraction-static 0.85\n\n# Reduce batch size\n--rollout-batch-size 16\n```\n\n### Issue: Weight Sync Timeout\n\n**Symptoms**: Training hangs after rollout\n\n**Solutions**:\n```bash\n# Increase sync interval\n--update-weights-interval 5\n\n# Use colocated mode (no network transfer)\n--colocate\n```\n\n### Issue: OOM During Training\n\n**Symptoms**: CUDA OOM in backward pass\n\n**Solutions**:\n```bash\n# Enable gradient checkpointing\n--recompute-activations\n\n# Reduce micro-batch size\n--micro-batch-size 1\n\n# Enable sequence parallelism\n--sequence-parallel\n```\n\n### Issue: Slow Data Loading\n\n**Symptoms**: GPU idle during data fetch\n\n**Solutions**:\n```bash\n# Increase data workers\n--num-data-workers 4\n\n# Use streaming dataset\n--streaming-data\n```\n\n---\n\n## Supported Models\n\n| Model Family | Configurations |\n|--------------|----------------|\n| GLM | GLM-4.5, GLM-4.6, GLM-4.7, GLM-Z1-9B |\n| Qwen | Qwen3 (4B, 8B, 30B-A3B), Qwen3-MoE, Qwen2.5 |\n| DeepSeek | V3, V3.1, R1 |\n| Llama | Llama 3 (8B, 70B) |\n| Others | Kimi K2, Moonlight-16B |\n\nEach model has pre-configured scripts in `scripts\u002Fmodels\u002F`.\n\n---\n\n## Advanced Topics\n\n### Co-location Mode\n\nShare GPUs between training and inference to reduce memory:\n\n```bash\npython train.py \\\n    --colocate \\\n    --actor-num-gpus-per-node 8 \\\n    --sglang-mem-fraction-static 0.4 \\\n    ${MODEL_ARGS[@]}\n```\n\n### Custom Reward Model\n\n```python\n# custom_rm.py\nclass CustomRewardModel:\n    def __init__(self, model_path):\n        self.model = load_model(model_path)\n\n    def compute_reward(self, prompts, responses):\n        inputs = self.tokenize(prompts, responses)\n        scores = self.model(inputs)\n        return scores.tolist()\n```\n\n```bash\n--custom-rm-path custom_rm.py\n```\n\n### Evaluation Multi-Task\n\n```bash\n--eval-prompt-data aime \u002Fpath\u002Fto\u002Faime.jsonl \\\n--eval-prompt-data gsm8k \u002Fpath\u002Fto\u002Fgsm8k.jsonl \\\n--n-samples-per-eval-prompt 16\n```\n\n---\n\n## Resources\n\n- **Documentation**: https:\u002F\u002Fthudm.github.io\u002Fslime\u002F\n- **GitHub**: https:\u002F\u002Fgithub.com\u002FTHUDM\u002Fslime\n- **Blog**: https:\u002F\u002Flmsys.org\u002Fblog\u002F2025-07-09-slime\u002F\n- **Examples**: See `examples\u002F` directory for 14+ worked examples\n\n",{"name":5,"tags":23,"author":30,"license":14,"version":31,"description":7,"dependencies":32},[24,25,26,27,28,29],"Reinforcement Learning","Megatron-LM","SGLang","GRPO","Post-Training","GLM","Orchestra Research","1.0.0",[33,34,35,36],"sglang-router>=0.2.3","ray","torch>=2.0.0","transformers>=4.40.0","main","46e03791269608c5ac886a64c08f5d94147e8786","labs\u002F.claude\u002Fskills\u002Fslime","https:\u002F\u002Fgithub.com\u002Fbeyondai\u002Fsandbox","https:\u002F\u002Fgithub.com\u002Fbeyondai","2026-11-08 04:18:42",{"source":8,"skillName":5,"commands":44,"prompt":54,"zipAvailable":55},[45,48,51],{"tool":46,"command":47},"npx","npx skills add beyondai\u002Fsandbox --skill slime-rl-training",{"tool":49,"command":50},"bunx","bunx skills add beyondai\u002Fsandbox --skill slime-rl-training",{"tool":52,"command":53},"pnpm","pnpm dlx skills add beyondai\u002Fsandbox --skill slime-rl-training","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fbeyondai\u002Fsandbox\u002Fblob\u002Fmain\u002Flabs\u002F.claude\u002Fskills\u002Fslime\u002FSKILL.md",true,[57,63],{"path":58,"resourceType":59,"resourceTypeDesc":60,"sizeBytes":61,"sourceHtmlUrl":62},"references\u002Fapi-reference.md","REFERENCE","参考文档",11907,"https:\u002F\u002Fgithub.com\u002Fbeyondai\u002Fsandbox\u002Fblob\u002Fmain\u002Flabs\u002F.claude\u002Fskills\u002Fslime\u002Freferences\u002Fapi-reference.md",{"path":64,"resourceType":59,"resourceTypeDesc":60,"sizeBytes":65,"sourceHtmlUrl":66},"references\u002Ftroubleshooting.md",7221,"https:\u002F\u002Fgithub.com\u002Fbeyondai\u002Fsandbox\u002Fblob\u002Fmain\u002Flabs\u002F.claude\u002Fskills\u002Fslime\u002Freferences\u002Ftroubleshooting.md",[68,75,82,89,96,103],{"skillNo":69,"name":70,"nameZh":6,"description":71,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":72,"categories":73,"keywords":74,"publishedAt":18},"01M7D3GMPGJZRG8EPY404YF8TR","weights-and-biases","Track ML experiments with automatic logging, visualize training in real-time, optimize hyperparameters with sweeps, and manage model registry with W&B - collaborative MLOps platform","https:\u002F\u002Fgithub.com\u002Fbeyondai\u002Fsandbox\u002Fblob\u002Fmain\u002Flabs\u002F.opencode\u002Fskills\u002Fweights-and-biases\u002FSKILL.md",[],[],{"skillNo":76,"name":77,"nameZh":6,"description":78,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":79,"categories":80,"keywords":81,"publishedAt":18},"01M7D3GFZH440V2J532BSZMPVP","serving-llms-vllm","Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency\u002Fthroughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ\u002FAWQ\u002FFP8), and tensor parallelism.","https:\u002F\u002Fgithub.com\u002Fbeyondai\u002Fsandbox\u002Fblob\u002Fmain\u002Flabs\u002F.opencode\u002Fskills\u002Fvllm\u002FSKILL.md",[],[],{"skillNo":83,"name":84,"nameZh":6,"description":85,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":86,"categories":87,"keywords":88,"publishedAt":18},"01M7D3GAE8MQCPGF72EP9G9GKY","verl-rl-training","Provides guidance for training LLMs with reinforcement learning using verl (Volcano Engine RL). Use when implementing RLHF, GRPO, PPO, or other RL algorithms for LLM post-training at scale with flexible infrastructure backends.","https:\u002F\u002Fgithub.com\u002Fbeyondai\u002Fsandbox\u002Fblob\u002Fmain\u002Flabs\u002F.opencode\u002Fskills\u002Fverl\u002FSKILL.md",[],[],{"skillNo":90,"name":91,"nameZh":6,"description":92,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":93,"categories":94,"keywords":95,"publishedAt":18},"01M7D3G7KKNE5RGNYJNXV4F0MM","unsloth","Expert guidance for fast fine-tuning with Unsloth - 2-5x faster training, 50-80% less memory, LoRA\u002FQLoRA optimization","https:\u002F\u002Fgithub.com\u002Fbeyondai\u002Fsandbox\u002Fblob\u002Fmain\u002Flabs\u002F.opencode\u002Fskills\u002Funsloth\u002FSKILL.md",[],[],{"skillNo":97,"name":98,"nameZh":6,"description":99,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":100,"categories":101,"keywords":102,"publishedAt":18},"01M7D3G2RCZQ08GS0KAW5CBWSC","fine-tuning-with-trl","Fine-tune LLMs using reinforcement learning with TRL - SFT for instruction tuning, DPO for preference alignment, PPO\u002FGRPO for reward optimization, and reward model training. Use when need RLHF, align model with preferences, or train from human feedback. Works with HuggingFace Transformers.","https:\u002F\u002Fgithub.com\u002Fbeyondai\u002Fsandbox\u002Fblob\u002Fmain\u002Flabs\u002F.opencode\u002Fskills\u002Ftrl-fine-tuning\u002FSKILL.md",[],[],{"skillNo":104,"name":105,"nameZh":6,"description":106,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":107,"categories":108,"keywords":109,"publishedAt":18},"01M7D3FX08BPB172CYGM597NZF","torchforge-rl-training","Provides guidance for PyTorch-native agentic RL using torchforge, Meta's library separating infra from algorithms. Use when you want clean RL abstractions, easy algorithm experimentation, or scalable training with Monarch and TorchTitan.","https:\u002F\u002Fgithub.com\u002Fbeyondai\u002Fsandbox\u002Fblob\u002Fmain\u002Flabs\u002F.opencode\u002Fskills\u002Ftorchforge\u002FSKILL.md",[],[]]