[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M758X3MW1ZGAEBTFD2DWGRVJ":3,"skill-same:01M758X3MW1ZGAEBTFD2DWGRVJ":81},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":41,"commitSha":42,"skillDirectory":43,"forksCount":14,"repositoryHtmlUrl":44,"repositoryHomepageUrl":6,"ownerHtmlUrl":45,"ownerBlogUrl":46,"ownerDescription":47,"updatedAt":48,"install":49,"resources":62},"01M758X3MW1ZGAEBTFD2DWGRVJ","simpo-training",null,"Simple Preference Optimization for LLM alignment. Reference-free alternative to DPO with better performance (+6.4 points on AlpacaEval 2.0). No reference model needed, more efficient than DPO. Use for preference alignment when want simpler, faster training than DPO\u002FPPO.","zengrong233\u002Fdevin-skills","zengrong233","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F55529920?v=4","01M758DNYKQQCQX87XFVAGX5MW","songzengrong","01M758DNZVEJJQNM71DSRVHFEB",0,"MIT","https:\u002F\u002Fgithub.com\u002Fzengrong233\u002Fdevin-skills\u002Fblob\u002Fskills\u002F.agents\u002Fskills\u002Fsimpo\u002FSKILL.md",[],[],"2026-11-12 13:55:43",false,"---\nname: simpo-training\ndescription: Simple Preference Optimization for LLM alignment. Reference-free alternative to DPO with better performance (+6.4 points on AlpacaEval 2.0). No reference model needed, more efficient than DPO. Use for preference alignment when want simpler, faster training than DPO\u002FPPO.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Post-Training, SimPO, Preference Optimization, Alignment, DPO Alternative, Reference-Free, LLM Alignment, Efficient Training]\ndependencies: [torch, transformers, datasets, trl, accelerate]\n---\n\n# SimPO - Simple Preference Optimization\n\n## Quick start\n\nSimPO is a reference-free preference optimization method that outperforms DPO without needing a reference model.\n\n**Installation**:\n```bash\n# Create environment\nconda create -n simpo python=3.10 && conda activate simpo\n\n# Install PyTorch 2.2.2\n# Visit: https:\u002F\u002Fpytorch.org\u002Fget-started\u002Flocally\u002F\n\n# Install alignment-handbook\ngit clone https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Falignment-handbook.git\ncd alignment-handbook\npython -m pip install .\n\n# Install Flash Attention 2\npython -m pip install flash-attn --no-build-isolation\n```\n\n**Training** (Mistral 7B):\n```bash\nACCELERATE_LOG_LEVEL=info accelerate launch \\\n  --config_file accelerate_configs\u002Fdeepspeed_zero3.yaml \\\n  scripts\u002Frun_simpo.py \\\n  training_configs\u002Fmistral-7b-base-simpo.yaml\n```\n\n## Common workflows\n\n### Workflow 1: Train from base model (Mistral 7B)\n\n**Config** (`mistral-7b-base-simpo.yaml`):\n```yaml\n# Model\nmodel_name_or_path: mistralai\u002FMistral-7B-v0.1\ntorch_dtype: bfloat16\n\n# Dataset\ndataset_mixer:\n  HuggingFaceH4\u002Fultrafeedback_binarized: 1.0\ndataset_splits:\n  - train_prefs\n  - test_prefs\n\n# SimPO hyperparameters\nbeta: 2.0                  # Reward scaling (2.0-10.0)\ngamma_beta_ratio: 0.5       # Target margin (0-1)\nloss_type: sigmoid          # sigmoid or hinge\nsft_weight: 0.0             # Optional SFT regularization\n\n# Training\nlearning_rate: 5e-7         # Critical: 3e-7 to 1e-6\nnum_train_epochs: 1\nper_device_train_batch_size: 1\ngradient_accumulation_steps: 8\n\n# Output\noutput_dir: .\u002Foutputs\u002Fmistral-7b-simpo\n```\n\n**Launch training**:\n```bash\naccelerate launch --config_file accelerate_configs\u002Fdeepspeed_zero3.yaml \\\n  scripts\u002Frun_simpo.py training_configs\u002Fmistral-7b-base-simpo.yaml\n```\n\n### Workflow 2: Fine-tune instruct model (Llama 3 8B)\n\n**Config** (`llama3-8b-instruct-simpo.yaml`):\n```yaml\nmodel_name_or_path: meta-llama\u002FMeta-Llama-3-8B-Instruct\n\ndataset_mixer:\n  argilla\u002Fultrafeedback-binarized-preferences-cleaned: 1.0\n\nbeta: 2.5\ngamma_beta_ratio: 0.5\nlearning_rate: 5e-7\nsft_weight: 0.1             # Add SFT loss to preserve capabilities\n\nnum_train_epochs: 1\nper_device_train_batch_size: 2\ngradient_accumulation_steps: 4\noutput_dir: .\u002Foutputs\u002Fllama3-8b-simpo\n```\n\n**Launch**:\n```bash\naccelerate launch --config_file accelerate_configs\u002Fdeepspeed_zero3.yaml \\\n  scripts\u002Frun_simpo.py training_configs\u002Fllama3-8b-instruct-simpo.yaml\n```\n\n### Workflow 3: Reasoning-intensive tasks (lower LR)\n\n**For math\u002Fcode tasks**:\n```yaml\nmodel_name_or_path: deepseek-ai\u002Fdeepseek-math-7b-base\n\ndataset_mixer:\n  argilla\u002Fdistilabel-math-preference-dpo: 1.0\n\nbeta: 5.0                   # Higher for stronger signal\ngamma_beta_ratio: 0.7       # Larger margin\nlearning_rate: 3e-7         # Lower LR for reasoning\nsft_weight: 0.0\n\nnum_train_epochs: 1\nper_device_train_batch_size: 1\ngradient_accumulation_steps: 16\n```\n\n## When to use vs alternatives\n\n**Use SimPO when**:\n- Want simpler training than DPO (no reference model)\n- Have preference data (chosen\u002Frejected pairs)\n- Need better performance than DPO\n- Limited compute resources\n- Single-node training sufficient\n\n**Algorithm selection**:\n- **SimPO**: Simplest, best performance, no reference model\n- **DPO**: Need reference model baseline, more conservative\n- **PPO**: Maximum control, need reward model, complex setup\n- **GRPO**: Memory-efficient RL, no critic\n\n**Use alternatives instead**:\n- **OpenRLHF**: Multi-node distributed training, PPO\u002FGRPO\n- **TRL**: Need multiple methods in one framework\n- **DPO**: Established baseline comparison\n\n## Common issues\n\n**Issue: Loss divergence**\n\nReduce learning rate:\n```yaml\nlearning_rate: 3e-7  # Reduce from 5e-7\n```\n\nReduce beta:\n```yaml\nbeta: 1.0  # Reduce from 2.0\n```\n\n**Issue: Model forgets capabilities**\n\nAdd SFT regularization:\n```yaml\nsft_weight: 0.1  # Add SFT loss component\n```\n\n**Issue: Poor preference separation**\n\nIncrease beta and margin:\n```yaml\nbeta: 5.0            # Increase from 2.0\ngamma_beta_ratio: 0.8  # Increase from 0.5\n```\n\n**Issue: OOM during training**\n\nReduce batch size:\n```yaml\nper_device_train_batch_size: 1\ngradient_accumulation_steps: 16  # Maintain effective batch\n```\n\nEnable gradient checkpointing:\n```yaml\ngradient_checkpointing: true\n```\n\n## Advanced topics\n\n**Loss functions**: See [references\u002Floss-functions.md](references\u002Floss-functions.md) for sigmoid vs hinge loss, mathematical formulations, and when to use each.\n\n**Hyperparameter tuning**: See [references\u002Fhyperparameters.md](references\u002Fhyperparameters.md) for beta, gamma, learning rate selection guide, and model-size-specific recommendations.\n\n**Dataset preparation**: See [references\u002Fdatasets.md](references\u002Fdatasets.md) for preference data formats, quality filtering, and custom dataset creation.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA A100\u002FH100 recommended\n- **VRAM**:\n  - 7B model: 1× A100 40GB (DeepSpeed ZeRO-3)\n  - 8B model: 2× A100 40GB\n  - 70B model: 8× A100 80GB\n- **Single-node**: DeepSpeed ZeRO-3 sufficient\n- **Mixed precision**: BF16 recommended\n\n**Memory optimization**:\n- DeepSpeed ZeRO-3 (default config)\n- Gradient checkpointing\n- Flash Attention 2\n\n## Resources\n\n- Paper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2405.14734 (NeurIPS 2024)\n- GitHub: https:\u002F\u002Fgithub.com\u002Fprinceton-nlp\u002FSimPO\n- Models: https:\u002F\u002Fhuggingface.co\u002Fprinceton-nlp\n- Alignment Handbook: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Falignment-handbook\n\n\n\n","# SimPO - Simple Preference Optimization\n\n## Quick start\n\nSimPO is a reference-free preference optimization method that outperforms DPO without needing a reference model.\n\n**Installation**:\n```bash\n# Create environment\nconda create -n simpo python=3.10 && conda activate simpo\n\n# Install PyTorch 2.2.2\n# Visit: https:\u002F\u002Fpytorch.org\u002Fget-started\u002Flocally\u002F\n\n# Install alignment-handbook\ngit clone https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Falignment-handbook.git\ncd alignment-handbook\npython -m pip install .\n\n# Install Flash Attention 2\npython -m pip install flash-attn --no-build-isolation\n```\n\n**Training** (Mistral 7B):\n```bash\nACCELERATE_LOG_LEVEL=info accelerate launch \\\n  --config_file accelerate_configs\u002Fdeepspeed_zero3.yaml \\\n  scripts\u002Frun_simpo.py \\\n  training_configs\u002Fmistral-7b-base-simpo.yaml\n```\n\n## Common workflows\n\n### Workflow 1: Train from base model (Mistral 7B)\n\n**Config** (`mistral-7b-base-simpo.yaml`):\n```yaml\n# Model\nmodel_name_or_path: mistralai\u002FMistral-7B-v0.1\ntorch_dtype: bfloat16\n\n# Dataset\ndataset_mixer:\n  HuggingFaceH4\u002Fultrafeedback_binarized: 1.0\ndataset_splits:\n  - train_prefs\n  - test_prefs\n\n# SimPO hyperparameters\nbeta: 2.0                  # Reward scaling (2.0-10.0)\ngamma_beta_ratio: 0.5       # Target margin (0-1)\nloss_type: sigmoid          # sigmoid or hinge\nsft_weight: 0.0             # Optional SFT regularization\n\n# Training\nlearning_rate: 5e-7         # Critical: 3e-7 to 1e-6\nnum_train_epochs: 1\nper_device_train_batch_size: 1\ngradient_accumulation_steps: 8\n\n# Output\noutput_dir: .\u002Foutputs\u002Fmistral-7b-simpo\n```\n\n**Launch training**:\n```bash\naccelerate launch --config_file accelerate_configs\u002Fdeepspeed_zero3.yaml \\\n  scripts\u002Frun_simpo.py training_configs\u002Fmistral-7b-base-simpo.yaml\n```\n\n### Workflow 2: Fine-tune instruct model (Llama 3 8B)\n\n**Config** (`llama3-8b-instruct-simpo.yaml`):\n```yaml\nmodel_name_or_path: meta-llama\u002FMeta-Llama-3-8B-Instruct\n\ndataset_mixer:\n  argilla\u002Fultrafeedback-binarized-preferences-cleaned: 1.0\n\nbeta: 2.5\ngamma_beta_ratio: 0.5\nlearning_rate: 5e-7\nsft_weight: 0.1             # Add SFT loss to preserve capabilities\n\nnum_train_epochs: 1\nper_device_train_batch_size: 2\ngradient_accumulation_steps: 4\noutput_dir: .\u002Foutputs\u002Fllama3-8b-simpo\n```\n\n**Launch**:\n```bash\naccelerate launch --config_file accelerate_configs\u002Fdeepspeed_zero3.yaml \\\n  scripts\u002Frun_simpo.py training_configs\u002Fllama3-8b-instruct-simpo.yaml\n```\n\n### Workflow 3: Reasoning-intensive tasks (lower LR)\n\n**For math\u002Fcode tasks**:\n```yaml\nmodel_name_or_path: deepseek-ai\u002Fdeepseek-math-7b-base\n\ndataset_mixer:\n  argilla\u002Fdistilabel-math-preference-dpo: 1.0\n\nbeta: 5.0                   # Higher for stronger signal\ngamma_beta_ratio: 0.7       # Larger margin\nlearning_rate: 3e-7         # Lower LR for reasoning\nsft_weight: 0.0\n\nnum_train_epochs: 1\nper_device_train_batch_size: 1\ngradient_accumulation_steps: 16\n```\n\n## When to use vs alternatives\n\n**Use SimPO when**:\n- Want simpler training than DPO (no reference model)\n- Have preference data (chosen\u002Frejected pairs)\n- Need better performance than DPO\n- Limited compute resources\n- Single-node training sufficient\n\n**Algorithm selection**:\n- **SimPO**: Simplest, best performance, no reference model\n- **DPO**: Need reference model baseline, more conservative\n- **PPO**: Maximum control, need reward model, complex setup\n- **GRPO**: Memory-efficient RL, no critic\n\n**Use alternatives instead**:\n- **OpenRLHF**: Multi-node distributed training, PPO\u002FGRPO\n- **TRL**: Need multiple methods in one framework\n- **DPO**: Established baseline comparison\n\n## Common issues\n\n**Issue: Loss divergence**\n\nReduce learning rate:\n```yaml\nlearning_rate: 3e-7  # Reduce from 5e-7\n```\n\nReduce beta:\n```yaml\nbeta: 1.0  # Reduce from 2.0\n```\n\n**Issue: Model forgets capabilities**\n\nAdd SFT regularization:\n```yaml\nsft_weight: 0.1  # Add SFT loss component\n```\n\n**Issue: Poor preference separation**\n\nIncrease beta and margin:\n```yaml\nbeta: 5.0            # Increase from 2.0\ngamma_beta_ratio: 0.8  # Increase from 0.5\n```\n\n**Issue: OOM during training**\n\nReduce batch size:\n```yaml\nper_device_train_batch_size: 1\ngradient_accumulation_steps: 16  # Maintain effective batch\n```\n\nEnable gradient checkpointing:\n```yaml\ngradient_checkpointing: true\n```\n\n## Advanced topics\n\n**Loss functions**: See [references\u002Floss-functions.md](references\u002Floss-functions.md) for sigmoid vs hinge loss, mathematical formulations, and when to use each.\n\n**Hyperparameter tuning**: See [references\u002Fhyperparameters.md](references\u002Fhyperparameters.md) for beta, gamma, learning rate selection guide, and model-size-specific recommendations.\n\n**Dataset preparation**: See [references\u002Fdatasets.md](references\u002Fdatasets.md) for preference data formats, quality filtering, and custom dataset creation.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA A100\u002FH100 recommended\n- **VRAM**:\n  - 7B model: 1× A100 40GB (DeepSpeed ZeRO-3)\n  - 8B model: 2× A100 40GB\n  - 70B model: 8× A100 80GB\n- **Single-node**: DeepSpeed ZeRO-3 sufficient\n- **Mixed precision**: BF16 recommended\n\n**Memory optimization**:\n- DeepSpeed ZeRO-3 (default config)\n- Gradient checkpointing\n- Flash Attention 2\n\n## Resources\n\n- Paper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2405.14734 (NeurIPS 2024)\n- GitHub: https:\u002F\u002Fgithub.com\u002Fprinceton-nlp\u002FSimPO\n- Models: https:\u002F\u002Fhuggingface.co\u002Fprinceton-nlp\n- Alignment Handbook: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Falignment-handbook\n\n\n\n",{"name":5,"tags":24,"author":33,"license":15,"version":34,"description":7,"dependencies":35},[25,26,27,28,29,30,31,32],"Post-Training","SimPO","Preference Optimization","Alignment","DPO Alternative","Reference-Free","LLM Alignment","Efficient Training","Orchestra Research","1.0.0",[36,37,38,39,40],"torch","transformers","datasets","trl","accelerate","skills","4fb202b97f2676132ac8713dbd2370cefacacb81",".agents\u002Fskills\u002Fsimpo","https:\u002F\u002Fgithub.com\u002Fzengrong233\u002Fdevin-skills","https:\u002F\u002Fgithub.com\u002Fzengrong233","https:\u002F\u002Fwtu.edu.cn\u002F","my name songzengrong\r\n  research interest's:NLP","2026-06-04 10:48:22",{"source":8,"skillName":5,"commands":50,"prompt":60,"zipAvailable":61},[51,54,57],{"tool":52,"command":53},"npx","npx skills add zengrong233\u002Fdevin-skills --skill simpo-training",{"tool":55,"command":56},"bunx","bunx skills add zengrong233\u002Fdevin-skills --skill simpo-training",{"tool":58,"command":59},"pnpm","pnpm dlx skills add zengrong233\u002Fdevin-skills --skill simpo-training","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fzengrong233\u002Fdevin-skills\u002Fblob\u002Fskills\u002F.agents\u002Fskills\u002Fsimpo\u002FSKILL.md",true,[63,69,73,77],{"path":64,"resourceType":65,"resourceTypeDesc":66,"sizeBytes":67,"sourceHtmlUrl":68},".openskills.json","REFERENCE","参考文档",228,"https:\u002F\u002Fgithub.com\u002Fzengrong233\u002Fdevin-skills\u002Fblob\u002Fskills\u002F.agents\u002Fskills\u002Fsimpo\u002F.openskills.json",{"path":70,"resourceType":65,"resourceTypeDesc":66,"sizeBytes":71,"sourceHtmlUrl":72},"references\u002Fdatasets.md",10889,"https:\u002F\u002Fgithub.com\u002Fzengrong233\u002Fdevin-skills\u002Fblob\u002Fskills\u002F.agents\u002Fskills\u002Fsimpo\u002Freferences\u002Fdatasets.md",{"path":74,"resourceType":65,"resourceTypeDesc":66,"sizeBytes":75,"sourceHtmlUrl":76},"references\u002Fhyperparameters.md",8502,"https:\u002F\u002Fgithub.com\u002Fzengrong233\u002Fdevin-skills\u002Fblob\u002Fskills\u002F.agents\u002Fskills\u002Fsimpo\u002Freferences\u002Fhyperparameters.md",{"path":78,"resourceType":65,"resourceTypeDesc":66,"sizeBytes":79,"sourceHtmlUrl":80},"references\u002Floss-functions.md",7067,"https:\u002F\u002Fgithub.com\u002Fzengrong233\u002Fdevin-skills\u002Fblob\u002Fskills\u002F.agents\u002Fskills\u002Fsimpo\u002Freferences\u002Floss-functions.md",[82,89,96,105,113,120],{"skillNo":83,"name":84,"nameZh":6,"description":85,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":86,"categories":87,"keywords":88,"publishedAt":19},"01M759188DV0S9ARY1ETHCEY45","whisper","OpenAI's general-purpose speech recognition model. Supports 99 languages, transcription, translation to English, and language identification. Six model sizes from tiny (39M params) to large (1550M params). Use for speech-to-text, podcast transcription, or multilingual audio processing. Best for robust, multilingual ASR.","https:\u002F\u002Fgithub.com\u002Fzengrong233\u002Fdevin-skills\u002Fblob\u002Fskills\u002F.agents\u002Fskills\u002Fwhisper\u002FSKILL.md",[],[],{"skillNo":90,"name":91,"nameZh":6,"description":92,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":93,"categories":94,"keywords":95,"publishedAt":19},"01M75913ZB4EDVWYGYRTRWTKD1","weights-and-biases","Track ML experiments with automatic logging, visualize training in real-time, optimize hyperparameters with sweeps, and manage model registry with W&B - collaborative MLOps platform","https:\u002F\u002Fgithub.com\u002Fzengrong233\u002Fdevin-skills\u002Fblob\u002Fskills\u002F.agents\u002Fskills\u002Fweights-and-biases\u002FSKILL.md",[],[],{"skillNo":97,"name":98,"nameZh":6,"description":99,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":100,"licenseFileUrl":101,"sourceHtmlUrl":102,"categories":103,"keywords":104,"publishedAt":19},"01M75910YNW7H0KZQDR96TGWZ9","webapp-testing","Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Fzengrong233\u002Fdevin-skills\u002Fblob\u002Fskills\u002F.agents\u002Fskills\u002Fwebapp-testing\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002Fzengrong233\u002Fdevin-skills\u002Fblob\u002Fskills\u002F.agents\u002Fskills\u002Fwebapp-testing\u002FSKILL.md",[],[],{"skillNo":106,"name":107,"nameZh":6,"description":108,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":100,"licenseFileUrl":109,"sourceHtmlUrl":110,"categories":111,"keywords":112,"publishedAt":19},"01M7590X6XJR78KVSHCVFYB59A","web-artifacts-builder","Suite of tools for creating elaborate, multi-component claude.ai HTML artifacts using modern frontend web technologies (React, Tailwind CSS, shadcn\u002Fui). Use for complex artifacts requiring state management, routing, or shadcn\u002Fui components - not for simple single-file HTML\u002FJSX artifacts.","https:\u002F\u002Fgithub.com\u002Fzengrong233\u002Fdevin-skills\u002Fblob\u002Fskills\u002F.agents\u002Fskills\u002Fweb-artifacts-builder\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002Fzengrong233\u002Fdevin-skills\u002Fblob\u002Fskills\u002F.agents\u002Fskills\u002Fweb-artifacts-builder\u002FSKILL.md",[],[],{"skillNo":114,"name":115,"nameZh":6,"description":116,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":117,"categories":118,"keywords":119,"publishedAt":19},"01M7590RENMJJTXV44P8G553ZQ","serving-llms-vllm","Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency\u002Fthroughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ\u002FAWQ\u002FFP8), and tensor parallelism.","https:\u002F\u002Fgithub.com\u002Fzengrong233\u002Fdevin-skills\u002Fblob\u002Fskills\u002F.agents\u002Fskills\u002Fvllm\u002FSKILL.md",[],[],{"skillNo":121,"name":122,"nameZh":6,"description":123,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":124,"categories":125,"keywords":126,"publishedAt":19},"01M7590GA6EZKJA8GAAMMPAXBB","verl-rl-training","Provides guidance for training LLMs with reinforcement learning using verl (Volcano Engine RL). Use when implementing RLHF, GRPO, PPO, or other RL algorithms for LLM post-training at scale with flexible infrastructure backends.","https:\u002F\u002Fgithub.com\u002Fzengrong233\u002Fdevin-skills\u002Fblob\u002Fskills\u002F.agents\u002Fskills\u002Fverl\u002FSKILL.md",[],[]]