[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6EDTM422D31RZK4WXG0CS67":3,"skill-same:01M6EDTM422D31RZK4WXG0CS67":81},{"skillNo":4,"name":5,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":18,"linkOnly":19,"markdownContent":20,"markdownBody":20,"frontMatter":21,"refName":22,"commitSha":23,"skillDirectory":24,"forksCount":25,"repositoryHtmlUrl":26,"repositoryHomepageUrl":27,"ownerHtmlUrl":28,"ownerBlogUrl":6,"ownerDescription":29,"updatedAt":30,"install":31,"resources":44},"01M6EDTM422D31RZK4WXG0CS67","simpo",null,"ComeOnOliver\u002Fskillshub","ComeOnOliver","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F43215937?v=4","01M6E5JNFECJAAVFE0W7SMWWD2","Haoran Shu","01M6E5JNNDQS7AJZDYFQ8TPZCK",66,"MIT","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002FOrchestra-Research\u002FAI-Research-SKILLs\u002Fsimpo\u002FSKILL.md",[],[5],"2026-11-10 10:12:30",false,"# SimPO - Simple Preference Optimization\n\n## Quick start\n\nSimPO is a reference-free preference optimization method that outperforms DPO without needing a reference model.\n\n**Installation**:\n```bash\n# Create environment\nconda create -n simpo python=3.10 && conda activate simpo\n\n# Install PyTorch 2.2.2\n# Visit: https:\u002F\u002Fpytorch.org\u002Fget-started\u002Flocally\u002F\n\n# Install alignment-handbook\ngit clone https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Falignment-handbook.git\ncd alignment-handbook\npython -m pip install .\n\n# Install Flash Attention 2\npython -m pip install flash-attn --no-build-isolation\n```\n\n**Training** (Mistral 7B):\n```bash\nACCELERATE_LOG_LEVEL=info accelerate launch \\\n  --config_file accelerate_configs\u002Fdeepspeed_zero3.yaml \\\n  scripts\u002Frun_simpo.py \\\n  training_configs\u002Fmistral-7b-base-simpo.yaml\n```\n\n## Common workflows\n\n### Workflow 1: Train from base model (Mistral 7B)\n\n**Config** (`mistral-7b-base-simpo.yaml`):\n```yaml\n# Model\nmodel_name_or_path: mistralai\u002FMistral-7B-v0.1\ntorch_dtype: bfloat16\n\n# Dataset\ndataset_mixer:\n  HuggingFaceH4\u002Fultrafeedback_binarized: 1.0\ndataset_splits:\n  - train_prefs\n  - test_prefs\n\n# SimPO hyperparameters\nbeta: 2.0                  # Reward scaling (2.0-10.0)\ngamma_beta_ratio: 0.5       # Target margin (0-1)\nloss_type: sigmoid          # sigmoid or hinge\nsft_weight: 0.0             # Optional SFT regularization\n\n# Training\nlearning_rate: 5e-7         # Critical: 3e-7 to 1e-6\nnum_train_epochs: 1\nper_device_train_batch_size: 1\ngradient_accumulation_steps: 8\n\n# Output\noutput_dir: .\u002Foutputs\u002Fmistral-7b-simpo\n```\n\n**Launch training**:\n```bash\naccelerate launch --config_file accelerate_configs\u002Fdeepspeed_zero3.yaml \\\n  scripts\u002Frun_simpo.py training_configs\u002Fmistral-7b-base-simpo.yaml\n```\n\n### Workflow 2: Fine-tune instruct model (Llama 3 8B)\n\n**Config** (`llama3-8b-instruct-simpo.yaml`):\n```yaml\nmodel_name_or_path: meta-llama\u002FMeta-Llama-3-8B-Instruct\n\ndataset_mixer:\n  argilla\u002Fultrafeedback-binarized-preferences-cleaned: 1.0\n\nbeta: 2.5\ngamma_beta_ratio: 0.5\nlearning_rate: 5e-7\nsft_weight: 0.1             # Add SFT loss to preserve capabilities\n\nnum_train_epochs: 1\nper_device_train_batch_size: 2\ngradient_accumulation_steps: 4\noutput_dir: .\u002Foutputs\u002Fllama3-8b-simpo\n```\n\n**Launch**:\n```bash\naccelerate launch --config_file accelerate_configs\u002Fdeepspeed_zero3.yaml \\\n  scripts\u002Frun_simpo.py training_configs\u002Fllama3-8b-instruct-simpo.yaml\n```\n\n### Workflow 3: Reasoning-intensive tasks (lower LR)\n\n**For math\u002Fcode tasks**:\n```yaml\nmodel_name_or_path: deepseek-ai\u002Fdeepseek-math-7b-base\n\ndataset_mixer:\n  argilla\u002Fdistilabel-math-preference-dpo: 1.0\n\nbeta: 5.0                   # Higher for stronger signal\ngamma_beta_ratio: 0.7       # Larger margin\nlearning_rate: 3e-7         # Lower LR for reasoning\nsft_weight: 0.0\n\nnum_train_epochs: 1\nper_device_train_batch_size: 1\ngradient_accumulation_steps: 16\n```\n\n## When to use vs alternatives\n\n**Use SimPO when**:\n- Want simpler training than DPO (no reference model)\n- Have preference data (chosen\u002Frejected pairs)\n- Need better performance than DPO\n- Limited compute resources\n- Single-node training sufficient\n\n**Algorithm selection**:\n- **SimPO**: Simplest, best performance, no reference model\n- **DPO**: Need reference model baseline, more conservative\n- **PPO**: Maximum control, need reward model, complex setup\n- **GRPO**: Memory-efficient RL, no critic\n\n**Use alternatives instead**:\n- **OpenRLHF**: Multi-node distributed training, PPO\u002FGRPO\n- **TRL**: Need multiple methods in one framework\n- **DPO**: Established baseline comparison\n\n## Common issues\n\n**Issue: Loss divergence**\n\nReduce learning rate:\n```yaml\nlearning_rate: 3e-7  # Reduce from 5e-7\n```\n\nReduce beta:\n```yaml\nbeta: 1.0  # Reduce from 2.0\n```\n\n**Issue: Model forgets capabilities**\n\nAdd SFT regularization:\n```yaml\nsft_weight: 0.1  # Add SFT loss component\n```\n\n**Issue: Poor preference separation**\n\nIncrease beta and margin:\n```yaml\nbeta: 5.0            # Increase from 2.0\ngamma_beta_ratio: 0.8  # Increase from 0.5\n```\n\n**Issue: OOM during training**\n\nReduce batch size:\n```yaml\nper_device_train_batch_size: 1\ngradient_accumulation_steps: 16  # Maintain effective batch\n```\n\nEnable gradient checkpointing:\n```yaml\ngradient_checkpointing: true\n```\n\n## Advanced topics\n\n**Loss functions**: See [references\u002Floss-functions.md](references\u002Floss-functions.md) for sigmoid vs hinge loss, mathematical formulations, and when to use each.\n\n**Hyperparameter tuning**: See [references\u002Fhyperparameters.md](references\u002Fhyperparameters.md) for beta, gamma, learning rate selection guide, and model-size-specific recommendations.\n\n**Dataset preparation**: See [references\u002Fdatasets.md](references\u002Fdatasets.md) for preference data formats, quality filtering, and custom dataset creation.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA A100\u002FH100 recommended\n- **VRAM**:\n  - 7B model: 1× A100 40GB (DeepSpeed ZeRO-3)\n  - 8B model: 2× A100 40GB\n  - 70B model: 8× A100 80GB\n- **Single-node**: DeepSpeed ZeRO-3 sufficient\n- **Mixed precision**: BF16 recommended\n\n**Memory optimization**:\n- DeepSpeed ZeRO-3 (default config)\n- Gradient checkpointing\n- Flash Attention 2\n\n## Resources\n\n- Paper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2405.14734 (NeurIPS 2024)\n- GitHub: https:\u002F\u002Fgithub.com\u002Fprinceton-nlp\u002FSimPO\n- Models: https:\u002F\u002Fhuggingface.co\u002Fprinceton-nlp\n- Alignment Handbook: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Falignment-handbook\n",{},"main","def8531e65114c0fca8fb8551c1871ee0eed705c","skills\u002FOrchestra-Research\u002FAI-Research-SKILLs\u002Fsimpo",24,"https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub","https:\u002F\u002Fskillshub.wtf","https:\u002F\u002Fgithub.com\u002FComeOnOliver","AI & Blockchain Engineer\r\n","2026-06-24 22:07:19",{"source":7,"skillName":5,"commands":32,"prompt":42,"zipAvailable":43},[33,36,39],{"tool":34,"command":35},"npx","npx skills add ComeOnOliver\u002Fskillshub --skill simpo",{"tool":37,"command":38},"bunx","bunx skills add ComeOnOliver\u002Fskillshub --skill simpo",{"tool":40,"command":41},"pnpm","pnpm dlx skills add ComeOnOliver\u002Fskillshub --skill simpo","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002FOrchestra-Research\u002FAI-Research-SKILLs\u002Fsimpo\u002FSKILL.md",true,[45,51,55,61,65,69,73,77],{"path":46,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":49,"sourceHtmlUrl":50},"agents\u002Fartifact-analyzer.md","REFERENCE","参考文档",2646,"https:\u002F\u002Fgithub.com\u002FdevelopAgent\u002Fbmad-start\u002Fblob\u002Fmain\u002F.opencode\u002Fskills\u002Fbmad-prfaq\u002Fagents\u002Fartifact-analyzer.md",{"path":52,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":53,"sourceHtmlUrl":54},"agents\u002Fweb-researcher.md",1756,"https:\u002F\u002Fgithub.com\u002FdevelopAgent\u002Fbmad-start\u002Fblob\u002Fmain\u002F.opencode\u002Fskills\u002Fbmad-prfaq\u002Fagents\u002Fweb-researcher.md",{"path":56,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":59,"sourceHtmlUrl":60},"assets\u002Fprfaq-template.md","ASSET","资产文件",1434,"https:\u002F\u002Fgithub.com\u002FdevelopAgent\u002Fbmad-start\u002Fblob\u002Fmain\u002F.opencode\u002Fskills\u002Fbmad-prfaq\u002Fassets\u002Fprfaq-template.md",{"path":62,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":63,"sourceHtmlUrl":64},"bmad-manifest.json",458,"https:\u002F\u002Fgithub.com\u002FdevelopAgent\u002Fbmad-start\u002Fblob\u002Fmain\u002F.opencode\u002Fskills\u002Fbmad-prfaq\u002Fbmad-manifest.json",{"path":66,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":67,"sourceHtmlUrl":68},"references\u002Fcustomer-faq.md",3697,"https:\u002F\u002Fgithub.com\u002FdevelopAgent\u002Fbmad-start\u002Fblob\u002Fmain\u002F.opencode\u002Fskills\u002Fbmad-prfaq\u002Freferences\u002Fcustomer-faq.md",{"path":70,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":71,"sourceHtmlUrl":72},"references\u002Finternal-faq.md",3928,"https:\u002F\u002Fgithub.com\u002FdevelopAgent\u002Fbmad-start\u002Fblob\u002Fmain\u002F.opencode\u002Fskills\u002Fbmad-prfaq\u002Freferences\u002Finternal-faq.md",{"path":74,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":75,"sourceHtmlUrl":76},"references\u002Fpress-release.md",4308,"https:\u002F\u002Fgithub.com\u002FdevelopAgent\u002Fbmad-start\u002Fblob\u002Fmain\u002F.opencode\u002Fskills\u002Fbmad-prfaq\u002Freferences\u002Fpress-release.md",{"path":78,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":79,"sourceHtmlUrl":80},"references\u002Fverdict.md",4262,"https:\u002F\u002Fgithub.com\u002FdevelopAgent\u002Fbmad-start\u002Fblob\u002Fmain\u002F.opencode\u002Fskills\u002Fbmad-prfaq\u002Freferences\u002Fverdict.md",[82,96,106,115,121,128],{"skillNo":83,"name":84,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":85,"categories":86,"keywords":92,"publishedAt":18},"01M6FKJN13GMG4KY57J3QD8GY5","model-hierarchy-skill","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzscole\u002Fmodel-hierarchy-skill\u002Fmodel-hierarchy-skill\u002FSKILL.md",[87],{"categoryNo":88,"categoryName":89,"taxonomyType":90,"taxonomyTypeDesc":91},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[93,94,95],"model","model hierarchy","model hierarchy skill",{"skillNo":97,"name":98,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":99,"categories":100,"keywords":102,"publishedAt":18},"01M6FKJJJ6GCAMR4PPPRD12M5X","zoom-mcp","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fzoom-mcp\u002FSKILL.md",[101],{"categoryNo":88,"categoryName":89,"taxonomyType":90,"taxonomyTypeDesc":91},[103,104,105],"MCP","zoom","zoom mcp",{"skillNo":107,"name":108,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":109,"categories":110,"keywords":111,"publishedAt":18},"01M6FKJE3KYT3KPEZMEEHYKA6K","zoom-apps-sdk","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fzoom-apps-sdk\u002FSKILL.md",[],[112,113,114,104],"sdk","zoom apps sdk","apps",{"skillNo":116,"name":117,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":118,"categories":119,"keywords":120,"publishedAt":18},"01M6FKJ9EDH9Q5VWDF8HAJRSP0","windows","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fwindows\u002FSKILL.md",[],[117],{"skillNo":122,"name":123,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":124,"categories":125,"keywords":127,"publishedAt":18},"01M6FKJ666BEEFC3AE9GJ72Q93","whiteboard","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fwhiteboard\u002FSKILL.md",[126],{"categoryNo":88,"categoryName":89,"taxonomyType":90,"taxonomyTypeDesc":91},[123],{"skillNo":129,"name":130,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":131,"categories":132,"keywords":133,"publishedAt":18},"01M6FKJ21WFCDGTJQ2Q91T9DVW","websockets","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fwebsockets\u002FSKILL.md",[],[130]]