[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M73AAMCMW4TA47MBYMF09890":3,"skill-same:01M73AAMCMW4TA47MBYMF09890":70},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":38,"commitSha":39,"skillDirectory":40,"forksCount":41,"repositoryHtmlUrl":42,"repositoryHomepageUrl":6,"ownerHtmlUrl":43,"ownerBlogUrl":44,"ownerDescription":6,"updatedAt":45,"install":46,"resources":59},"01M73AAMCMW4TA47MBYMF09890","slime-rl-training",null,"Provides guidance for LLM post-training with RL using slime, a Megatron+SGLang framework. Use when training GLM models, implementing custom data generation workflows, or needing tight Megatron-LM integration for RL scaling.","qhjqhj00\u002Fresearch-skills-pool","qhjqhj00","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F29684092?v=4","01M72AE0ZGXVM5C57N512DGQG8","Tommy Chien","01M72AE107ESJMHT7QC5SXX144",3,"MIT","https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Fskills_collected\u002FOrchestra-Research--AI-research-SKILLs--06-post-training__slime\u002FSKILL.md",[],[],"2026-11-11 15:11:14",false,"---\nname: slime-rl-training\ndescription: Provides guidance for LLM post-training with RL using slime, a Megatron+SGLang framework. Use when training GLM models, implementing custom data generation workflows, or needing tight Megatron-LM integration for RL scaling.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Reinforcement Learning, Megatron-LM, SGLang, GRPO, Post-Training, GLM]\ndependencies: [sglang-router>=0.2.3, ray, torch>=2.0.0, transformers>=4.40.0]\n---\n\n# slime: LLM Post-Training Framework for RL Scaling\n\nslime is an LLM post-training framework from Tsinghua's THUDM team, powering GLM-4.5, GLM-4.6, and GLM-4.7. It connects Megatron-LM for training with SGLang for high-throughput rollout generation.\n\n## When to Use slime\n\n**Choose slime when you need:**\n- Megatron-LM native training with SGLang inference\n- Custom data generation workflows with flexible data buffers\n- Training GLM, Qwen3, DeepSeek V3, or Llama 3 models\n- Research-grade framework with production backing (Z.ai)\n\n**Consider alternatives when:**\n- You need enterprise-grade stability features → use **miles**\n- You want flexible backend swapping → use **verl**\n- You need PyTorch-native abstractions → use **torchforge**\n\n## Key Features\n\n- **Training**: Megatron-LM with full parallelism support (TP, PP, DP, SP)\n- **Rollout**: SGLang-based high-throughput generation with router\n- **Data Buffer**: Flexible prompt management and sample storage\n- **Models**: GLM-4.x, Qwen3, DeepSeek V3\u002FR1, Llama 3\n\n## Architecture Overview\n\n```\n┌─────────────────────────────────────────────────────────┐\n│                    Data Buffer                          │\n│ - Prompt initialization and management                  │\n│ - Custom data generation and filtering                  │\n│ - Rollout sample storage                                │\n└─────────────┬───────────────────────────┬───────────────┘\n              │                           │\n┌─────────────▼───────────┐ ┌─────────────▼───────────────┐\n│ Training (Megatron-LM)  │ │ Rollout (SGLang + Router)   │\n│ - Actor model training  │ │ - Response generation       │\n│ - Critic (optional)     │ │ - Reward\u002Fverifier output    │\n│ - Weight sync to rollout│ │ - Multi-turn support        │\n└─────────────────────────┘ └─────────────────────────────┘\n```\n\n## Installation\n\n```bash\n# Recommended: Docker\ndocker pull slimerl\u002Fslime:latest\ndocker run --rm --gpus all --ipc=host --shm-size=16g \\\n  -it slimerl\u002Fslime:latest \u002Fbin\u002Fbash\n\n# Inside container\ncd \u002Froot\u002Fslime && pip install -e . --no-deps\n```\n\n### From Source\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FTHUDM\u002Fslime.git\ncd slime\npip install -r requirements.txt\npip install -e .\n```\n\n## Quick Start: GRPO Training\n\n```bash\n# Source model configuration\nsource scripts\u002Fmodels\u002Fqwen3-4B.sh\n\n# Launch training\npython train.py \\\n    --actor-num-nodes 1 \\\n    --actor-num-gpus-per-node 4 \\\n    --rollout-num-gpus 4 \\\n    --advantage-estimator grpo \\\n    --use-kl-loss --kl-loss-coef 0.001 \\\n    --rollout-batch-size 32 \\\n    --n-samples-per-prompt 8 \\\n    --global-batch-size 256 \\\n    --num-rollout 3000 \\\n    --prompt-data \u002Fpath\u002Fto\u002Fdata.jsonl \\\n    ${MODEL_ARGS[@]} ${CKPT_ARGS[@]}\n```\n\n---\n\n## Workflow 1: Standard GRPO Training\n\nUse this workflow for training reasoning models with group-relative advantages.\n\n### Prerequisites Checklist\n- [ ] Docker environment or Megatron-LM + SGLang installed\n- [ ] Model checkpoint (HuggingFace or Megatron format)\n- [ ] Training data in JSONL format\n\n### Step 1: Prepare Data\n\n```python\n# data.jsonl format\n{\"prompt\": \"What is 2 + 2?\", \"label\": \"4\"}\n{\"prompt\": \"Solve: 3x = 12\", \"label\": \"x = 4\"}\n```\n\nOr with chat format:\n```python\n{\n    \"prompt\": [\n        {\"role\": \"system\", \"content\": \"You are a math tutor.\"},\n        {\"role\": \"user\", \"content\": \"What is 15 + 27?\"}\n    ],\n    \"label\": \"42\"\n}\n```\n\n### Step 2: Configure Model\n\nChoose a pre-configured model script:\n\n```bash\n# List available models\nls scripts\u002Fmodels\u002F\n# glm4-9B.sh, qwen3-4B.sh, qwen3-30B-A3B.sh, deepseek-v3.sh, llama3-8B.sh, ...\n\n# Source your model\nsource scripts\u002Fmodels\u002Fqwen3-4B.sh\n```\n\n### Step 3: Launch Training\n\n```bash\npython train.py \\\n    --actor-num-nodes 1 \\\n    --actor-num-gpus-per-node 8 \\\n    --rollout-num-gpus 8 \\\n    --advantage-estimator grpo \\\n    --use-kl-loss \\\n    --kl-loss-coef 0.001 \\\n    --prompt-data \u002Fpath\u002Fto\u002Ftrain.jsonl \\\n    --input-key prompt \\\n    --label-key label \\\n    --apply-chat-template \\\n    --rollout-batch-size 32 \\\n    --n-samples-per-prompt 8 \\\n    --global-batch-size 256 \\\n    --num-rollout 3000 \\\n    --save-interval 100 \\\n    --eval-interval 50 \\\n    ${MODEL_ARGS[@]}\n```\n\n### Step 4: Monitor Training\n- [ ] Check TensorBoard: `tensorboard --logdir outputs\u002F`\n- [ ] Verify reward curves are increasing\n- [ ] Monitor GPU utilization across nodes\n\n---\n\n## Workflow 2: Asynchronous Training\n\nUse async mode for higher throughput by overlapping rollout and training.\n\n### When to Use Async\n- Large models with long generation times\n- High GPU idle time in synchronous mode\n- Sufficient memory for buffering\n\n### Launch Async Training\n\n```bash\npython train_async.py \\\n    --actor-num-nodes 1 \\\n    --actor-num-gpus-per-node 8 \\\n    --rollout-num-gpus 8 \\\n    --advantage-estimator grpo \\\n    --async-buffer-size 4 \\\n    --prompt-data \u002Fpath\u002Fto\u002Ftrain.jsonl \\\n    ${MODEL_ARGS[@]}\n```\n\n### Async-Specific Parameters\n\n```bash\n--async-buffer-size 4        # Number of rollouts to buffer\n--update-weights-interval 2  # Sync weights every N rollouts\n```\n\n---\n\n## Workflow 3: Multi-Turn Agentic Training\n\nUse this workflow for training agents with tool use or multi-step reasoning.\n\n### Prerequisites\n- [ ] Custom generate function for multi-turn logic\n- [ ] Tool\u002Fenvironment interface\n\n### Step 1: Define Custom Generate Function\n\n```python\n# custom_generate.py\nasync def custom_generate(args, samples, evaluation=False):\n    \"\"\"Multi-turn generation with tool calling.\"\"\"\n    for sample in samples:\n        conversation = sample.prompt\n\n        for turn in range(args.max_turns):\n            # Generate response\n            response = await generate_single(conversation)\n\n            # Check for tool call\n            tool_call = extract_tool_call(response)\n            if tool_call:\n                tool_result = execute_tool(tool_call)\n                conversation.append({\"role\": \"assistant\", \"content\": response})\n                conversation.append({\"role\": \"tool\", \"content\": tool_result})\n            else:\n                break\n\n        sample.response = response\n        sample.reward = compute_reward(sample)\n\n    return samples\n```\n\n### Step 2: Launch with Custom Function\n\n```bash\npython train.py \\\n    --custom-generate-function-path custom_generate.py \\\n    --max-turns 5 \\\n    --prompt-data \u002Fpath\u002Fto\u002Fagent_data.jsonl \\\n    ${MODEL_ARGS[@]}\n```\n\nSee `examples\u002Fsearch-r1\u002F` for a complete multi-turn search example.\n\n---\n\n## Configuration Reference\n\n### Three Argument Categories\n\nslime uses three types of arguments:\n\n**1. Megatron Arguments** (passed directly):\n```bash\n--tensor-model-parallel-size 2\n--pipeline-model-parallel-size 1\n--num-layers 32\n--hidden-size 4096\n```\n\n**2. SGLang Arguments** (prefixed with `--sglang-`):\n```bash\n--sglang-mem-fraction-static 0.8\n--sglang-context-length 8192\n--sglang-log-level INFO\n```\n\n**3. slime Arguments**:\n```bash\n# Resource allocation\n--actor-num-nodes 1\n--actor-num-gpus-per-node 8\n--rollout-num-gpus 8\n--colocate  # Share GPUs between training\u002Finference\n\n# Data\n--prompt-data \u002Fpath\u002Fto\u002Fdata.jsonl\n--input-key prompt\n--label-key label\n\n# Training loop\n--num-rollout 3000\n--rollout-batch-size 32\n--n-samples-per-prompt 8\n--global-batch-size 256\n\n# Algorithm\n--advantage-estimator grpo  # or: gspo, ppo, reinforce_plus_plus\n--use-kl-loss\n--kl-loss-coef 0.001\n```\n\n### Key Constraints\n\n```\nrollout_batch_size × n_samples_per_prompt = global_batch_size × num_steps_per_rollout\n```\n\nExample: 32 × 8 = 256 × 1\n\n---\n\n## Data Buffer System\n\nslime's data buffer enables flexible data management:\n\n### Basic Data Source\n\n```python\nclass RolloutDataSource:\n    def get_samples(self, num_samples):\n        \"\"\"Fetch prompts from dataset.\"\"\"\n        return self.dataset.sample(num_samples)\n\n    def add_samples(self, samples):\n        \"\"\"Called after generation (no-op by default).\"\"\"\n        pass\n```\n\n### Buffered Data Source (Off-Policy)\n\n```python\nclass RolloutDataSourceWithBuffer(RolloutDataSource):\n    def __init__(self):\n        self.buffer = []\n\n    def add_samples(self, samples):\n        \"\"\"Store generated samples for reuse.\"\"\"\n        self.buffer.extend(samples)\n\n    def buffer_filter(self, args, buffer, num_samples):\n        \"\"\"Custom selection logic (prioritized, stratified, etc.).\"\"\"\n        return select_best(buffer, num_samples)\n```\n\n---\n\n## Common Issues and Solutions\n\n### Issue: SGLang Engine Crash\n\n**Symptoms**: Inference engine dies mid-training\n\n**Solutions**:\n```bash\n# Enable fault tolerance\n--use-fault-tolerance\n\n# Increase memory allocation\n--sglang-mem-fraction-static 0.85\n\n# Reduce batch size\n--rollout-batch-size 16\n```\n\n### Issue: Weight Sync Timeout\n\n**Symptoms**: Training hangs after rollout\n\n**Solutions**:\n```bash\n# Increase sync interval\n--update-weights-interval 5\n\n# Use colocated mode (no network transfer)\n--colocate\n```\n\n### Issue: OOM During Training\n\n**Symptoms**: CUDA OOM in backward pass\n\n**Solutions**:\n```bash\n# Enable gradient checkpointing\n--recompute-activations\n\n# Reduce micro-batch size\n--micro-batch-size 1\n\n# Enable sequence parallelism\n--sequence-parallel\n```\n\n### Issue: Slow Data Loading\n\n**Symptoms**: GPU idle during data fetch\n\n**Solutions**:\n```bash\n# Increase data workers\n--num-data-workers 4\n\n# Use streaming dataset\n--streaming-data\n```\n\n---\n\n## Supported Models\n\n| Model Family | Configurations |\n|--------------|----------------|\n| GLM | GLM-4.5, GLM-4.6, GLM-4.7, GLM-Z1-9B |\n| Qwen | Qwen3 (4B, 8B, 30B-A3B), Qwen3-MoE, Qwen2.5 |\n| DeepSeek | V3, V3.1, R1 |\n| Llama | Llama 3 (8B, 70B) |\n| Others | Kimi K2, Moonlight-16B |\n\nEach model has pre-configured scripts in `scripts\u002Fmodels\u002F`.\n\n---\n\n## Advanced Topics\n\n### Co-location Mode\n\nShare GPUs between training and inference to reduce memory:\n\n```bash\npython train.py \\\n    --colocate \\\n    --actor-num-gpus-per-node 8 \\\n    --sglang-mem-fraction-static 0.4 \\\n    ${MODEL_ARGS[@]}\n```\n\n### Custom Reward Model\n\n```python\n# custom_rm.py\nclass CustomRewardModel:\n    def __init__(self, model_path):\n        self.model = load_model(model_path)\n\n    def compute_reward(self, prompts, responses):\n        inputs = self.tokenize(prompts, responses)\n        scores = self.model(inputs)\n        return scores.tolist()\n```\n\n```bash\n--custom-rm-path custom_rm.py\n```\n\n### Evaluation Multi-Task\n\n```bash\n--eval-prompt-data aime \u002Fpath\u002Fto\u002Faime.jsonl \\\n--eval-prompt-data gsm8k \u002Fpath\u002Fto\u002Fgsm8k.jsonl \\\n--n-samples-per-eval-prompt 16\n```\n\n---\n\n## Resources\n\n- **Documentation**: https:\u002F\u002Fthudm.github.io\u002Fslime\u002F\n- **GitHub**: https:\u002F\u002Fgithub.com\u002FTHUDM\u002Fslime\n- **Blog**: https:\u002F\u002Flmsys.org\u002Fblog\u002F2025-07-09-slime\u002F\n- **Examples**: See `examples\u002F` directory for 14+ worked examples\n\n","# slime: LLM Post-Training Framework for RL Scaling\n\nslime is an LLM post-training framework from Tsinghua's THUDM team, powering GLM-4.5, GLM-4.6, and GLM-4.7. It connects Megatron-LM for training with SGLang for high-throughput rollout generation.\n\n## When to Use slime\n\n**Choose slime when you need:**\n- Megatron-LM native training with SGLang inference\n- Custom data generation workflows with flexible data buffers\n- Training GLM, Qwen3, DeepSeek V3, or Llama 3 models\n- Research-grade framework with production backing (Z.ai)\n\n**Consider alternatives when:**\n- You need enterprise-grade stability features → use **miles**\n- You want flexible backend swapping → use **verl**\n- You need PyTorch-native abstractions → use **torchforge**\n\n## Key Features\n\n- **Training**: Megatron-LM with full parallelism support (TP, PP, DP, SP)\n- **Rollout**: SGLang-based high-throughput generation with router\n- **Data Buffer**: Flexible prompt management and sample storage\n- **Models**: GLM-4.x, Qwen3, DeepSeek V3\u002FR1, Llama 3\n\n## Architecture Overview\n\n```\n┌─────────────────────────────────────────────────────────┐\n│                    Data Buffer                          │\n│ - Prompt initialization and management                  │\n│ - Custom data generation and filtering                  │\n│ - Rollout sample storage                                │\n└─────────────┬───────────────────────────┬───────────────┘\n              │                           │\n┌─────────────▼───────────┐ ┌─────────────▼───────────────┐\n│ Training (Megatron-LM)  │ │ Rollout (SGLang + Router)   │\n│ - Actor model training  │ │ - Response generation       │\n│ - Critic (optional)     │ │ - Reward\u002Fverifier output    │\n│ - Weight sync to rollout│ │ - Multi-turn support        │\n└─────────────────────────┘ └─────────────────────────────┘\n```\n\n## Installation\n\n```bash\n# Recommended: Docker\ndocker pull slimerl\u002Fslime:latest\ndocker run --rm --gpus all --ipc=host --shm-size=16g \\\n  -it slimerl\u002Fslime:latest \u002Fbin\u002Fbash\n\n# Inside container\ncd \u002Froot\u002Fslime && pip install -e . --no-deps\n```\n\n### From Source\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FTHUDM\u002Fslime.git\ncd slime\npip install -r requirements.txt\npip install -e .\n```\n\n## Quick Start: GRPO Training\n\n```bash\n# Source model configuration\nsource scripts\u002Fmodels\u002Fqwen3-4B.sh\n\n# Launch training\npython train.py \\\n    --actor-num-nodes 1 \\\n    --actor-num-gpus-per-node 4 \\\n    --rollout-num-gpus 4 \\\n    --advantage-estimator grpo \\\n    --use-kl-loss --kl-loss-coef 0.001 \\\n    --rollout-batch-size 32 \\\n    --n-samples-per-prompt 8 \\\n    --global-batch-size 256 \\\n    --num-rollout 3000 \\\n    --prompt-data \u002Fpath\u002Fto\u002Fdata.jsonl \\\n    ${MODEL_ARGS[@]} ${CKPT_ARGS[@]}\n```\n\n---\n\n## Workflow 1: Standard GRPO Training\n\nUse this workflow for training reasoning models with group-relative advantages.\n\n### Prerequisites Checklist\n- [ ] Docker environment or Megatron-LM + SGLang installed\n- [ ] Model checkpoint (HuggingFace or Megatron format)\n- [ ] Training data in JSONL format\n\n### Step 1: Prepare Data\n\n```python\n# data.jsonl format\n{\"prompt\": \"What is 2 + 2?\", \"label\": \"4\"}\n{\"prompt\": \"Solve: 3x = 12\", \"label\": \"x = 4\"}\n```\n\nOr with chat format:\n```python\n{\n    \"prompt\": [\n        {\"role\": \"system\", \"content\": \"You are a math tutor.\"},\n        {\"role\": \"user\", \"content\": \"What is 15 + 27?\"}\n    ],\n    \"label\": \"42\"\n}\n```\n\n### Step 2: Configure Model\n\nChoose a pre-configured model script:\n\n```bash\n# List available models\nls scripts\u002Fmodels\u002F\n# glm4-9B.sh, qwen3-4B.sh, qwen3-30B-A3B.sh, deepseek-v3.sh, llama3-8B.sh, ...\n\n# Source your model\nsource scripts\u002Fmodels\u002Fqwen3-4B.sh\n```\n\n### Step 3: Launch Training\n\n```bash\npython train.py \\\n    --actor-num-nodes 1 \\\n    --actor-num-gpus-per-node 8 \\\n    --rollout-num-gpus 8 \\\n    --advantage-estimator grpo \\\n    --use-kl-loss \\\n    --kl-loss-coef 0.001 \\\n    --prompt-data \u002Fpath\u002Fto\u002Ftrain.jsonl \\\n    --input-key prompt \\\n    --label-key label \\\n    --apply-chat-template \\\n    --rollout-batch-size 32 \\\n    --n-samples-per-prompt 8 \\\n    --global-batch-size 256 \\\n    --num-rollout 3000 \\\n    --save-interval 100 \\\n    --eval-interval 50 \\\n    ${MODEL_ARGS[@]}\n```\n\n### Step 4: Monitor Training\n- [ ] Check TensorBoard: `tensorboard --logdir outputs\u002F`\n- [ ] Verify reward curves are increasing\n- [ ] Monitor GPU utilization across nodes\n\n---\n\n## Workflow 2: Asynchronous Training\n\nUse async mode for higher throughput by overlapping rollout and training.\n\n### When to Use Async\n- Large models with long generation times\n- High GPU idle time in synchronous mode\n- Sufficient memory for buffering\n\n### Launch Async Training\n\n```bash\npython train_async.py \\\n    --actor-num-nodes 1 \\\n    --actor-num-gpus-per-node 8 \\\n    --rollout-num-gpus 8 \\\n    --advantage-estimator grpo \\\n    --async-buffer-size 4 \\\n    --prompt-data \u002Fpath\u002Fto\u002Ftrain.jsonl \\\n    ${MODEL_ARGS[@]}\n```\n\n### Async-Specific Parameters\n\n```bash\n--async-buffer-size 4        # Number of rollouts to buffer\n--update-weights-interval 2  # Sync weights every N rollouts\n```\n\n---\n\n## Workflow 3: Multi-Turn Agentic Training\n\nUse this workflow for training agents with tool use or multi-step reasoning.\n\n### Prerequisites\n- [ ] Custom generate function for multi-turn logic\n- [ ] Tool\u002Fenvironment interface\n\n### Step 1: Define Custom Generate Function\n\n```python\n# custom_generate.py\nasync def custom_generate(args, samples, evaluation=False):\n    \"\"\"Multi-turn generation with tool calling.\"\"\"\n    for sample in samples:\n        conversation = sample.prompt\n\n        for turn in range(args.max_turns):\n            # Generate response\n            response = await generate_single(conversation)\n\n            # Check for tool call\n            tool_call = extract_tool_call(response)\n            if tool_call:\n                tool_result = execute_tool(tool_call)\n                conversation.append({\"role\": \"assistant\", \"content\": response})\n                conversation.append({\"role\": \"tool\", \"content\": tool_result})\n            else:\n                break\n\n        sample.response = response\n        sample.reward = compute_reward(sample)\n\n    return samples\n```\n\n### Step 2: Launch with Custom Function\n\n```bash\npython train.py \\\n    --custom-generate-function-path custom_generate.py \\\n    --max-turns 5 \\\n    --prompt-data \u002Fpath\u002Fto\u002Fagent_data.jsonl \\\n    ${MODEL_ARGS[@]}\n```\n\nSee `examples\u002Fsearch-r1\u002F` for a complete multi-turn search example.\n\n---\n\n## Configuration Reference\n\n### Three Argument Categories\n\nslime uses three types of arguments:\n\n**1. Megatron Arguments** (passed directly):\n```bash\n--tensor-model-parallel-size 2\n--pipeline-model-parallel-size 1\n--num-layers 32\n--hidden-size 4096\n```\n\n**2. SGLang Arguments** (prefixed with `--sglang-`):\n```bash\n--sglang-mem-fraction-static 0.8\n--sglang-context-length 8192\n--sglang-log-level INFO\n```\n\n**3. slime Arguments**:\n```bash\n# Resource allocation\n--actor-num-nodes 1\n--actor-num-gpus-per-node 8\n--rollout-num-gpus 8\n--colocate  # Share GPUs between training\u002Finference\n\n# Data\n--prompt-data \u002Fpath\u002Fto\u002Fdata.jsonl\n--input-key prompt\n--label-key label\n\n# Training loop\n--num-rollout 3000\n--rollout-batch-size 32\n--n-samples-per-prompt 8\n--global-batch-size 256\n\n# Algorithm\n--advantage-estimator grpo  # or: gspo, ppo, reinforce_plus_plus\n--use-kl-loss\n--kl-loss-coef 0.001\n```\n\n### Key Constraints\n\n```\nrollout_batch_size × n_samples_per_prompt = global_batch_size × num_steps_per_rollout\n```\n\nExample: 32 × 8 = 256 × 1\n\n---\n\n## Data Buffer System\n\nslime's data buffer enables flexible data management:\n\n### Basic Data Source\n\n```python\nclass RolloutDataSource:\n    def get_samples(self, num_samples):\n        \"\"\"Fetch prompts from dataset.\"\"\"\n        return self.dataset.sample(num_samples)\n\n    def add_samples(self, samples):\n        \"\"\"Called after generation (no-op by default).\"\"\"\n        pass\n```\n\n### Buffered Data Source (Off-Policy)\n\n```python\nclass RolloutDataSourceWithBuffer(RolloutDataSource):\n    def __init__(self):\n        self.buffer = []\n\n    def add_samples(self, samples):\n        \"\"\"Store generated samples for reuse.\"\"\"\n        self.buffer.extend(samples)\n\n    def buffer_filter(self, args, buffer, num_samples):\n        \"\"\"Custom selection logic (prioritized, stratified, etc.).\"\"\"\n        return select_best(buffer, num_samples)\n```\n\n---\n\n## Common Issues and Solutions\n\n### Issue: SGLang Engine Crash\n\n**Symptoms**: Inference engine dies mid-training\n\n**Solutions**:\n```bash\n# Enable fault tolerance\n--use-fault-tolerance\n\n# Increase memory allocation\n--sglang-mem-fraction-static 0.85\n\n# Reduce batch size\n--rollout-batch-size 16\n```\n\n### Issue: Weight Sync Timeout\n\n**Symptoms**: Training hangs after rollout\n\n**Solutions**:\n```bash\n# Increase sync interval\n--update-weights-interval 5\n\n# Use colocated mode (no network transfer)\n--colocate\n```\n\n### Issue: OOM During Training\n\n**Symptoms**: CUDA OOM in backward pass\n\n**Solutions**:\n```bash\n# Enable gradient checkpointing\n--recompute-activations\n\n# Reduce micro-batch size\n--micro-batch-size 1\n\n# Enable sequence parallelism\n--sequence-parallel\n```\n\n### Issue: Slow Data Loading\n\n**Symptoms**: GPU idle during data fetch\n\n**Solutions**:\n```bash\n# Increase data workers\n--num-data-workers 4\n\n# Use streaming dataset\n--streaming-data\n```\n\n---\n\n## Supported Models\n\n| Model Family | Configurations |\n|--------------|----------------|\n| GLM | GLM-4.5, GLM-4.6, GLM-4.7, GLM-Z1-9B |\n| Qwen | Qwen3 (4B, 8B, 30B-A3B), Qwen3-MoE, Qwen2.5 |\n| DeepSeek | V3, V3.1, R1 |\n| Llama | Llama 3 (8B, 70B) |\n| Others | Kimi K2, Moonlight-16B |\n\nEach model has pre-configured scripts in `scripts\u002Fmodels\u002F`.\n\n---\n\n## Advanced Topics\n\n### Co-location Mode\n\nShare GPUs between training and inference to reduce memory:\n\n```bash\npython train.py \\\n    --colocate \\\n    --actor-num-gpus-per-node 8 \\\n    --sglang-mem-fraction-static 0.4 \\\n    ${MODEL_ARGS[@]}\n```\n\n### Custom Reward Model\n\n```python\n# custom_rm.py\nclass CustomRewardModel:\n    def __init__(self, model_path):\n        self.model = load_model(model_path)\n\n    def compute_reward(self, prompts, responses):\n        inputs = self.tokenize(prompts, responses)\n        scores = self.model(inputs)\n        return scores.tolist()\n```\n\n```bash\n--custom-rm-path custom_rm.py\n```\n\n### Evaluation Multi-Task\n\n```bash\n--eval-prompt-data aime \u002Fpath\u002Fto\u002Faime.jsonl \\\n--eval-prompt-data gsm8k \u002Fpath\u002Fto\u002Fgsm8k.jsonl \\\n--n-samples-per-eval-prompt 16\n```\n\n---\n\n## Resources\n\n- **Documentation**: https:\u002F\u002Fthudm.github.io\u002Fslime\u002F\n- **GitHub**: https:\u002F\u002Fgithub.com\u002FTHUDM\u002Fslime\n- **Blog**: https:\u002F\u002Flmsys.org\u002Fblog\u002F2025-07-09-slime\u002F\n- **Examples**: See `examples\u002F` directory for 14+ worked examples\n\n",{"name":5,"tags":24,"author":31,"license":15,"version":32,"description":7,"dependencies":33},[25,26,27,28,29,30],"Reinforcement Learning","Megatron-LM","SGLang","GRPO","Post-Training","GLM","Orchestra Research","1.0.0",[34,35,36,37],"sglang-router>=0.2.3","ray","torch>=2.0.0","transformers>=4.40.0","master","c247302c836de6438d746b45670990eb71691166","skills_collected\u002FOrchestra-Research--AI-research-SKILLs--06-post-training__slime",1,"https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool","https:\u002F\u002Fgithub.com\u002Fqhjqhj00","chien.io","2026-05-11 15:07:59",{"source":8,"skillName":5,"commands":47,"prompt":57,"zipAvailable":58},[48,51,54],{"tool":49,"command":50},"npx","npx skills add qhjqhj00\u002Fresearch-skills-pool --skill slime-rl-training",{"tool":52,"command":53},"bunx","bunx skills add qhjqhj00\u002Fresearch-skills-pool --skill slime-rl-training",{"tool":55,"command":56},"pnpm","pnpm dlx skills add qhjqhj00\u002Fresearch-skills-pool --skill slime-rl-training","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Fskills_collected\u002FOrchestra-Research--AI-research-SKILLs--06-post-training__slime\u002FSKILL.md",true,[60,66],{"path":61,"resourceType":62,"resourceTypeDesc":63,"sizeBytes":64,"sourceHtmlUrl":65},"references\u002Fapi-reference.md","REFERENCE","参考文档",11907,"https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Fskills_collected\u002FOrchestra-Research--AI-research-SKILLs--06-post-training__slime\u002Freferences\u002Fapi-reference.md",{"path":67,"resourceType":62,"resourceTypeDesc":63,"sizeBytes":68,"sourceHtmlUrl":69},"references\u002Ftroubleshooting.md",7221,"https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Fskills_collected\u002FOrchestra-Research--AI-research-SKILLs--06-post-training__slime\u002Freferences\u002Ftroubleshooting.md",[71,78,85,92,99,106],{"skillNo":72,"name":73,"nameZh":6,"description":74,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":75,"categories":76,"keywords":77,"publishedAt":19},"01M73JJB6EQPFPPVQK2Y9N67Q0","transformer-lens-interpretability","Provides guidance for mechanistic interpretability research using TransformerLens to inspect and manipulate transformer internals via HookPoints and activation caching. Use when reverse-engineering model algorithms, studying attention patterns, or performing activation patching experiments.","https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Ftop100-runs\u002F130-transformer-lens\u002FSKILL.md",[],[],{"skillNo":79,"name":80,"nameZh":6,"description":81,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":82,"categories":83,"keywords":84,"publishedAt":19},"01M73JHTANRRWEE9HP98TPEHAY","wiki-vitepress","Packages generated wiki Markdown into a VitePress static site with dark theme, dark-mode Mermaid diagrams with click-to-zoom, and production build output. Use when the user wants to create a browsable website from generated wiki pages.","https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Ftop100-runs\u002F126-wiki-vitepress\u002FSKILL.md",[],[],{"skillNo":86,"name":87,"nameZh":6,"description":88,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":89,"categories":90,"keywords":91,"publishedAt":19},"01M73JHJBMVFW46F1Y8K3SBF43","phoenix-observability","Open-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.","https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Ftop100-runs\u002F124-phoenix\u002FSKILL.md",[],[],{"skillNo":93,"name":94,"nameZh":6,"description":95,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":96,"categories":97,"keywords":98,"publishedAt":19},"01M73JHFPA3A2806PMP42ZN86Q","frontend-ui-dark-ts","Build dark-themed React applications using Tailwind CSS with custom theming, glassmorphism effects, and Framer Motion animations. Use when creating dashboards, admin panels, or data-rich interfaces with a refined dark aesthetic.","https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Ftop100-runs\u002F123-frontend-ui-dark-ts\u002FSKILL.md",[],[],{"skillNo":100,"name":101,"nameZh":6,"description":102,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":103,"categories":104,"keywords":105,"publishedAt":19},"01M73JFVBVK57NRJS49TT2VDCT","tensorboard","Visualize training metrics, debug models with histograms, compare experiments, visualize model graphs, and profile performance with TensorBoard - Google's ML visualization toolkit","https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Ftop100-runs\u002F113-tensorboard\u002FSKILL.md",[],[],{"skillNo":107,"name":108,"nameZh":6,"description":109,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":110,"licenseFileUrl":6,"sourceHtmlUrl":111,"categories":112,"keywords":113,"publishedAt":19},"01M73JFF6PK6ZCN41C89J7HS2B","markdown-mermaid-writing","Comprehensive markdown and Mermaid diagram writing skill. Use when creating any scientific document, report, analysis, or visualization. Establishes text-based diagrams as the default documentation standard with full style guides (markdown + mermaid), 24 diagram type references, and 9 document templates.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Ftop100-runs\u002F110-markdown-mermaid-writing\u002FSKILL.md",[],[]]