[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6KD581T07RT1RJZAD4Z8HHT":3,"skill-same:01M6KD581T07RT1RJZAD4Z8HHT":50},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":23,"linkOnly":24,"markdownContent":25,"markdownBody":26,"frontMatter":27,"refName":28,"commitSha":29,"skillDirectory":30,"forksCount":31,"repositoryHtmlUrl":32,"repositoryHomepageUrl":33,"ownerHtmlUrl":34,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":35,"install":36,"resources":49},"01M6KD581T07RT1RJZAD4Z8HHT","simpo-loss-function",null,"Implement SimPO loss with length-normalized rewards and target margin.","cxcscmu\u002FSkillLearnBench","cxcscmu","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F142838272?v=4","01M6KCQDFQWPDSN0MKZY96MSQP","01M6KCQDM7G70TKJ28NTSQC0RZ",93,"MIT","https:\u002F\u002Fgithub.com\u002Fcxcscmu\u002FSkillLearnBench\u002Fblob\u002Fmain\u002Fskills\u002Fb1-one-shot-claude-haiku-4-5\u002Fnlp-paper-reproduction\u002Fsimpo-loss-function\u002FSKILL.md",[],[18,19,20,21,22],"simpo","simpo loss function","implement","margin","with","2026-11-10 10:12:30",false,"---\nname: simpo-loss-function\ndescription: Implement SimPO loss with length-normalized rewards and target margin.\n---\n\n# SimPO Loss Function Implementation\n\n## Overview\nSimPO (Simple Preference Optimization) implements a preference optimization objective that uses length-normalized average log probability as an implicit reward, with a target reward margin component.\n\n## Key Formula\n\n```\nL_SimPO(πθ) = -E_(x,yw,yl)~D log σ(β\u002F|yw| log πθ(yw|x) - β\u002F|yl| log πθ(yl|x) - γ)\n```\n\n## Components\n\n### 1. Length-Normalized Reward\n- **Formula**: `r_SimPO(x, y) = β\u002F|y| * log πθ(y|x)`\n- **Purpose**: Average log probability per token, prevents length bias\n- **Why**: Aligns training with generation metric (which uses average log likelihood for beam search)\n\n### 2. Bradley-Terry Objective\n- **Formula**: `p(yw ≻ yl | x) = σ(r(x, yw) - r(x, yl) - γ)`\n- **Purpose**: Probabilistic ranking between winning and losing responses\n- **Function**: σ is sigmoid function\n\n### 3. Target Reward Margin (γ)\n- **Purpose**: Ensure reward difference exceeds a target threshold\n- **Effect**: Improves generalization by enforcing margin between classes\n- **Typical range**: 0.5 to 1.5\n\n## Implementation Details\n\n### Computing Log Probabilities\n```python\n# log_probs shape: (batch_size, seq_len)\n# Sum across sequence dimension to get total log probability\nlog_prob_sum = log_probs.sum(dim=1)  # (batch_size,)\n\n# Divide by sequence length for normalization\nseq_lengths = (input_ids != pad_token_id).sum(dim=1)  # (batch_size,)\navg_log_prob = log_prob_sum \u002F seq_lengths.float()  # (batch_size,)\n```\n\n### Computing Reward Differences\n```python\n# Batch structure: pairs of (winning, losing) responses\nbatch_size = avg_log_probs.shape[0]\nwinning_rewards = avg_log_probs[:batch_size\u002F\u002F2]\nlosing_rewards = avg_log_probs[batch_size\u002F\u002F2:]\n\n# Reward difference with margin\nreward_diff = beta * winning_rewards - beta * losing_rewards - gamma\n```\n\n### Computing Loss\n```python\n# Bradley-Terry with sigmoid\nimport torch.nn.functional as F\n\nsigmoid_term = torch.sigmoid(reward_diff)\nloss = -torch.log(sigmoid_term).mean()\n```\n\n## Common Pitfalls\n\n1. **Not using length normalization**: Creates bias toward longer sequences\n2. **Wrong batch structure**: Ensure paired winning\u002Flosing responses\n3. **Missing average in log probabilities**: Use sum\u002Flength, not just sum\n4. **Gradient flow**: Ensure no detach() breaks gradients to model\n\n## Hyperparameters\n- **β**: Temperature parameter, typically 2.0-2.5\n- **γ**: Target margin, typically 0.3-1.6, depends on setting\n- **learning_rate**: Usually small, 1e-6 to 5e-7\n\n## References\n- SimPO Paper: Section 2.3 \"The SimPO Objective\"\n- Length normalization: Equation (3) in paper\n- Gradient analysis: Appendix F\n","# SimPO Loss Function Implementation\n\n## Overview\nSimPO (Simple Preference Optimization) implements a preference optimization objective that uses length-normalized average log probability as an implicit reward, with a target reward margin component.\n\n## Key Formula\n\n```\nL_SimPO(πθ) = -E_(x,yw,yl)~D log σ(β\u002F|yw| log πθ(yw|x) - β\u002F|yl| log πθ(yl|x) - γ)\n```\n\n## Components\n\n### 1. Length-Normalized Reward\n- **Formula**: `r_SimPO(x, y) = β\u002F|y| * log πθ(y|x)`\n- **Purpose**: Average log probability per token, prevents length bias\n- **Why**: Aligns training with generation metric (which uses average log likelihood for beam search)\n\n### 2. Bradley-Terry Objective\n- **Formula**: `p(yw ≻ yl | x) = σ(r(x, yw) - r(x, yl) - γ)`\n- **Purpose**: Probabilistic ranking between winning and losing responses\n- **Function**: σ is sigmoid function\n\n### 3. Target Reward Margin (γ)\n- **Purpose**: Ensure reward difference exceeds a target threshold\n- **Effect**: Improves generalization by enforcing margin between classes\n- **Typical range**: 0.5 to 1.5\n\n## Implementation Details\n\n### Computing Log Probabilities\n```python\n# log_probs shape: (batch_size, seq_len)\n# Sum across sequence dimension to get total log probability\nlog_prob_sum = log_probs.sum(dim=1)  # (batch_size,)\n\n# Divide by sequence length for normalization\nseq_lengths = (input_ids != pad_token_id).sum(dim=1)  # (batch_size,)\navg_log_prob = log_prob_sum \u002F seq_lengths.float()  # (batch_size,)\n```\n\n### Computing Reward Differences\n```python\n# Batch structure: pairs of (winning, losing) responses\nbatch_size = avg_log_probs.shape[0]\nwinning_rewards = avg_log_probs[:batch_size\u002F\u002F2]\nlosing_rewards = avg_log_probs[batch_size\u002F\u002F2:]\n\n# Reward difference with margin\nreward_diff = beta * winning_rewards - beta * losing_rewards - gamma\n```\n\n### Computing Loss\n```python\n# Bradley-Terry with sigmoid\nimport torch.nn.functional as F\n\nsigmoid_term = torch.sigmoid(reward_diff)\nloss = -torch.log(sigmoid_term).mean()\n```\n\n## Common Pitfalls\n\n1. **Not using length normalization**: Creates bias toward longer sequences\n2. **Wrong batch structure**: Ensure paired winning\u002Flosing responses\n3. **Missing average in log probabilities**: Use sum\u002Flength, not just sum\n4. **Gradient flow**: Ensure no detach() breaks gradients to model\n\n## Hyperparameters\n- **β**: Temperature parameter, typically 2.0-2.5\n- **γ**: Target margin, typically 0.3-1.6, depends on setting\n- **learning_rate**: Usually small, 1e-6 to 5e-7\n\n## References\n- SimPO Paper: Section 2.3 \"The SimPO Objective\"\n- Length normalization: Equation (3) in paper\n- Gradient analysis: Appendix F\n",{"name":5,"description":7},"main","726e1f79b96b0cb3d729d0e55a1c847e0b5353ae","skills\u002Fb1-one-shot-claude-haiku-4-5\u002Fnlp-paper-reproduction\u002Fsimpo-loss-function",7,"https:\u002F\u002Fgithub.com\u002Fcxcscmu\u002FSkillLearnBench","https:\u002F\u002Fcxcscmu.github.io\u002FSkillLearnBench","https:\u002F\u002Fgithub.com\u002Fcxcscmu","2026-10-02 22:55:36",{"source":8,"skillName":5,"commands":37,"prompt":47,"zipAvailable":48},[38,41,44],{"tool":39,"command":40},"npx","npx skills add cxcscmu\u002FSkillLearnBench --skill simpo-loss-function",{"tool":42,"command":43},"bunx","bunx skills add cxcscmu\u002FSkillLearnBench --skill simpo-loss-function",{"tool":45,"command":46},"pnpm","pnpm dlx skills add cxcscmu\u002FSkillLearnBench --skill simpo-loss-function","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fcxcscmu\u002FSkillLearnBench\u002Fblob\u002Fmain\u002Fskills\u002Fb1-one-shot-claude-haiku-4-5\u002Fnlp-paper-reproduction\u002Fsimpo-loss-function\u002FSKILL.md",true,[],[51,69,94,112,132,144],{"skillNo":52,"name":53,"nameZh":6,"description":54,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":55,"categories":56,"keywords":62,"publishedAt":23},"01M6KM5KH175P5ABP5JMX2SBDQ","object_counter","Count occurrences of an object in the image using computer vision algorithm.","https:\u002F\u002Fgithub.com\u002Fcxcscmu\u002FSkillLearnBench\u002Fblob\u002Fmain\u002Fskills\u002Fhuman_authored\u002Fvideo-object-counting\u002Fobject_counter\u002FSKILL.md",[57],{"categoryNo":58,"categoryName":59,"taxonomyType":60,"taxonomyTypeDesc":61},"01JBSEEDCAT000000000000016","图像与多媒体","SCENARIO","使用场景",[63,64,65,66,67,68],"object counter","object","vision","computer","Computer Vision","image",{"skillNo":70,"name":71,"nameZh":6,"description":72,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":73,"categories":74,"keywords":78,"publishedAt":23},"01M6KM51TMVXG79MMPCS6Y8G86","travel-planning-workflow","Invoke this skill when performing travel planning. It defines a structured workflow for generating itineraries. Following this workflow is essential to ensure the final plan satisfies user requirements.","https:\u002F\u002Fgithub.com\u002Fcxcscmu\u002FSkillLearnBench\u002Fblob\u002Fmain\u002Fskills\u002Fhuman_authored\u002Ftravel-planning\u002Ftravel-planning-workflow\u002FSKILL.md",[75],{"categoryNo":76,"categoryName":77,"taxonomyType":60,"taxonomyTypeDesc":61},"01JBSEEDCAT000000000000023","Skill与智能体开发",[79,80,81,82,83,84,85,86,87,88,89,90,91,92,93],"workflow","planning","planning workflow","travel","travel planning","travel planning workflow","structured","performing","plan","for","user","it","requirements","essential","itineraries",{"skillNo":95,"name":96,"nameZh":6,"description":97,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":98,"categories":99,"keywords":100,"publishedAt":23},"01M6KM4DY820XHS9YTSK5QP5HD","bundled-dataset-search-tool","invoke this skill when you need to perform database search for travel planning. This skill provides some useful pre-packaged tools to look up accommodations, attractions, cities, driving distance, flights, and restaurants from the bundled dataset.","https:\u002F\u002Fgithub.com\u002Fcxcscmu\u002FSkillLearnBench\u002Fblob\u002Fmain\u002Fskills\u002Fhuman_authored\u002Ftravel-planning\u002Fbundled-dataset-search-tool\u002FSKILL.md",[],[101,102,103,104,105,106,107,82,108,109,83,110,88,111,80],"search","tool","dataset","bundled dataset search tool","restaurants","Tools","up","look","flights","you","database",{"skillNo":113,"name":114,"nameZh":6,"description":115,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":116,"categories":117,"keywords":118,"publishedAt":23},"01M6KM465BB9PQ0B548HQSVAWZ","glm-output","Output-processing guidance for GLM tasks. Especially useful after glm-basics and glm-calibration when you need verifier-matching metrics from output.nc and a final \u002Froot\u002Fmetrics.json.","https:\u002F\u002Fgithub.com\u002Fcxcscmu\u002FSkillLearnBench\u002Fblob\u002Fmain\u002Fskills\u002Fhuman_authored\u002Ftemperature-simulation\u002Fglm-output\u002FSKILL.md",[],[119,120,121,88,122,123,124,125,126,127,128,129,110,130,131],"glm output","output","glm","tasks","metrics","json","basics","guidance","matching","verifier","processing","root","calibration",{"skillNo":133,"name":134,"nameZh":6,"description":135,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":136,"categories":137,"keywords":139,"publishedAt":23},"01M6KM40KY3QWDW5W97S39R2BC","glm-calibration","Calibration guidance for GLM tasks. Often most effective after glm-basics has clarified the setup; glm-output is the companion skill for exact final metric computation.","https:\u002F\u002Fgithub.com\u002Fcxcscmu\u002FSkillLearnBench\u002Fblob\u002Fmain\u002Fskills\u002Fhuman_authored\u002Ftemperature-simulation\u002Fglm-calibration\u002FSKILL.md",[138],{"categoryNo":76,"categoryName":77,"taxonomyType":60,"taxonomyTypeDesc":61},[140,131,121,120,126,125,122,141,142,88,143],"glm calibration","setup","companion","metric",{"skillNo":145,"name":146,"nameZh":6,"description":147,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":148,"categories":149,"keywords":150,"publishedAt":23},"01M6KM3VHHVSGMGHJQB3PPT6YG","glm-basics","A good starting point for GLM calibration tasks. Use to inspect glm3.nml, confirm how GLM runs, and identify the relevant files before moving on to calibration and output evaluation.","https:\u002F\u002Fgithub.com\u002Fcxcscmu\u002FSkillLearnBench\u002Fblob\u002Fmain\u002Fskills\u002Fhuman_authored\u002Ftemperature-simulation\u002Fglm-basics\u002FSKILL.md",[],[151,121,125,152,140,131,88,120,153,154,155,156,157,158,122],"glm basics","inspect","relevant","files","good","use","how","Evaluation"]