[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6KFSYRKZSSDJ8MEJ3N7M7WP":3,"skill-same:01M6KFSYRKZSSDJ8MEJ3N7M7WP":60},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":33,"linkOnly":34,"markdownContent":35,"markdownBody":36,"frontMatter":37,"refName":38,"commitSha":39,"skillDirectory":40,"forksCount":41,"repositoryHtmlUrl":42,"repositoryHomepageUrl":43,"ownerHtmlUrl":44,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":45,"install":46,"resources":59},"01M6KFSYRKZSSDJ8MEJ3N7M7WP","run2_simpo-loss",null,"Complete implementation of the SimPO (NeurIPS 2024) loss function with all variants and edge cases - use when reproducing SimPO or implementing reference-free preference optimization.","cxcscmu\u002FSkillLearnBench","cxcscmu","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F142838272?v=4","01M6KCQDFQWPDSN0MKZY96MSQP","01M6KCQDM7G70TKJ28NTSQC0RZ",93,"MIT","https:\u002F\u002Fgithub.com\u002Fcxcscmu\u002FSkillLearnBench\u002Fblob\u002Fmain\u002Fskills\u002Fb2-self-feedback-claude-sonnet-4-6\u002Fnlp-paper-reproduction\u002Frun2_simpo-loss\u002FSKILL.md",[],[18,19,20,21,22,23,24,25,26,27,28,29,30,31,32],"simpo","simpo loss","run2 simpo loss","edge","Optimization","NeurIPS","use","implementation","reference","free","complete","Preference Optimization","Reference-Free","with","implementing","2026-11-10 10:12:30",false,"---\nname: run2_simpo-loss\ndescription: Complete implementation of the SimPO (NeurIPS 2024) loss function with all variants and edge cases - use when reproducing SimPO or implementing reference-free preference optimization.\n---\n\n# SimPO Loss Function - Complete Implementation Guide\n\n## Paper Reference\nSimPO: Simple Preference Optimization with a Reference-Free Reward (NeurIPS 2024)\narXiv: 2405.14734v3 — Yu Meng, Mengzhou Xia, Danqi Chen\n\n## Mathematical Formulation\n\n### Length-Normalized Reward (Eq. 3-4)\n```\np_θ(y|x) = 1\u002F|y| * log π_θ(y|x) = 1\u002F|y| * Σ log π_θ(yi|x, y\u003Ci)\n\nr_SimPO(x, y) = β * p_θ(y|x) = β\u002F|y| * log π_θ(y|x)\n```\n- `β` scales the reward difference\n- Length normalization prevents bias toward longer sequences\n\n### Bradley-Terry Objective with Target Margin (Eq. 5-6)\n```\np(yw ≻ yl | x) = σ(r(x, yw) - r(x, yl) - γ)\n\nL_SimPO(π_θ) = -E_{(x,yw,yl)} [log σ(β\u002F|yw| * log π_θ(yw|x) - β\u002F|yl| * log π_θ(yl|x) - γ)]\n```\n- `γ > 0` is the target reward margin\n- `γ = β * gamma_beta_ratio` (recommended ratio: 0.0–1.0)\n\n## Full Implementation\n\n```python\nimport torch\nimport torch.nn.functional as F\n\ndef simpo_loss(self, policy_chosen_logps, policy_rejected_logps):\n    \"\"\"\n    Compute SimPO loss.\n\n    Inputs are ALREADY length-normalized (average log probability per token),\n    i.e., policy_chosen_logps = 1\u002F|yw| * log π_θ(yw|x)\n\n    This is because get_batch_logps() is called with average_log_prob=True.\n    \"\"\"\n    # r(x, y) = β * avg_log_prob(y)\n    chosen_rewards = self.beta * policy_chosen_logps.detach()\n    rejected_rewards = self.beta * policy_rejected_logps.detach()\n\n    # logit = β*(chosen - rejected) - γ\n    # Factor out β: β*(chosen - rejected - γ\u002Fβ) = β*(chosen - rejected - gamma_beta_ratio)\n    pi_logratios = policy_chosen_logps - policy_rejected_logps\n    gamma_logratios = self.gamma_beta_ratio  # = γ\u002Fβ\n    logits = pi_logratios - gamma_logratios\n\n    if self.loss_type == \"sigmoid\":\n        # L = -log σ(β*logits) with optional label smoothing\n        losses = (\n            -F.logsigmoid(self.beta * logits) * (1 - self.label_smoothing)\n            - F.logsigmoid(-self.beta * logits) * self.label_smoothing\n        )\n    elif self.loss_type == \"hinge\":\n        losses = torch.relu(1 - self.beta * logits)\n    else:\n        raise ValueError(f\"Unknown loss type: {self.loss_type}\")\n\n    return losses, chosen_rewards, rejected_rewards\n```\n\n## Config Defaults\n| Parameter | Default | Description |\n|-----------|---------|-------------|\n| beta | 2.0 | Reward scaling constant |\n| gamma_beta_ratio | 0.25 | γ\u002Fβ ratio → γ = 0.5 |\n| loss_type | \"sigmoid\" | \"sigmoid\" or \"hinge\" |\n| label_smoothing | 0.0 | Smoothing for sigmoid loss |\n\n## Key Insight: Input Normalization\nThe trainer calls `get_batch_logps(..., average_log_prob=True)`, so inputs are:\n- `policy_chosen_logps[i]` = Σ log π(token) \u002F num_tokens for chosen sequence i\n- NOT the raw log probability of the full sequence\n\nTherefore `β * policy_chosen_logps` directly gives `r_SimPO(x, yw)` from Eq. 4.\n\n## Numerical Check (default config)\nWith beta=2.0, gamma_beta_ratio=0.25, sigmoid loss, no smoothing:\n- effective_logit = 2.0 * (chosen_logp - rejected_logp - 0.25)\n- loss = -log(σ(effective_logit))\n- When chosen=rejected: loss = -log(σ(-0.5)) ≈ 0.974\n- When chosen >> rejected: loss → 0\n","# SimPO Loss Function - Complete Implementation Guide\n\n## Paper Reference\nSimPO: Simple Preference Optimization with a Reference-Free Reward (NeurIPS 2024)\narXiv: 2405.14734v3 — Yu Meng, Mengzhou Xia, Danqi Chen\n\n## Mathematical Formulation\n\n### Length-Normalized Reward (Eq. 3-4)\n```\np_θ(y|x) = 1\u002F|y| * log π_θ(y|x) = 1\u002F|y| * Σ log π_θ(yi|x, y\u003Ci)\n\nr_SimPO(x, y) = β * p_θ(y|x) = β\u002F|y| * log π_θ(y|x)\n```\n- `β` scales the reward difference\n- Length normalization prevents bias toward longer sequences\n\n### Bradley-Terry Objective with Target Margin (Eq. 5-6)\n```\np(yw ≻ yl | x) = σ(r(x, yw) - r(x, yl) - γ)\n\nL_SimPO(π_θ) = -E_{(x,yw,yl)} [log σ(β\u002F|yw| * log π_θ(yw|x) - β\u002F|yl| * log π_θ(yl|x) - γ)]\n```\n- `γ > 0` is the target reward margin\n- `γ = β * gamma_beta_ratio` (recommended ratio: 0.0–1.0)\n\n## Full Implementation\n\n```python\nimport torch\nimport torch.nn.functional as F\n\ndef simpo_loss(self, policy_chosen_logps, policy_rejected_logps):\n    \"\"\"\n    Compute SimPO loss.\n\n    Inputs are ALREADY length-normalized (average log probability per token),\n    i.e., policy_chosen_logps = 1\u002F|yw| * log π_θ(yw|x)\n\n    This is because get_batch_logps() is called with average_log_prob=True.\n    \"\"\"\n    # r(x, y) = β * avg_log_prob(y)\n    chosen_rewards = self.beta * policy_chosen_logps.detach()\n    rejected_rewards = self.beta * policy_rejected_logps.detach()\n\n    # logit = β*(chosen - rejected) - γ\n    # Factor out β: β*(chosen - rejected - γ\u002Fβ) = β*(chosen - rejected - gamma_beta_ratio)\n    pi_logratios = policy_chosen_logps - policy_rejected_logps\n    gamma_logratios = self.gamma_beta_ratio  # = γ\u002Fβ\n    logits = pi_logratios - gamma_logratios\n\n    if self.loss_type == \"sigmoid\":\n        # L = -log σ(β*logits) with optional label smoothing\n        losses = (\n            -F.logsigmoid(self.beta * logits) * (1 - self.label_smoothing)\n            - F.logsigmoid(-self.beta * logits) * self.label_smoothing\n        )\n    elif self.loss_type == \"hinge\":\n        losses = torch.relu(1 - self.beta * logits)\n    else:\n        raise ValueError(f\"Unknown loss type: {self.loss_type}\")\n\n    return losses, chosen_rewards, rejected_rewards\n```\n\n## Config Defaults\n| Parameter | Default | Description |\n|-----------|---------|-------------|\n| beta | 2.0 | Reward scaling constant |\n| gamma_beta_ratio | 0.25 | γ\u002Fβ ratio → γ = 0.5 |\n| loss_type | \"sigmoid\" | \"sigmoid\" or \"hinge\" |\n| label_smoothing | 0.0 | Smoothing for sigmoid loss |\n\n## Key Insight: Input Normalization\nThe trainer calls `get_batch_logps(..., average_log_prob=True)`, so inputs are:\n- `policy_chosen_logps[i]` = Σ log π(token) \u002F num_tokens for chosen sequence i\n- NOT the raw log probability of the full sequence\n\nTherefore `β * policy_chosen_logps` directly gives `r_SimPO(x, yw)` from Eq. 4.\n\n## Numerical Check (default config)\nWith beta=2.0, gamma_beta_ratio=0.25, sigmoid loss, no smoothing:\n- effective_logit = 2.0 * (chosen_logp - rejected_logp - 0.25)\n- loss = -log(σ(effective_logit))\n- When chosen=rejected: loss = -log(σ(-0.5)) ≈ 0.974\n- When chosen >> rejected: loss → 0\n",{"name":5,"description":7},"main","726e1f79b96b0cb3d729d0e55a1c847e0b5353ae","skills\u002Fb2-self-feedback-claude-sonnet-4-6\u002Fnlp-paper-reproduction\u002Frun2_simpo-loss",8,"https:\u002F\u002Fgithub.com\u002Fcxcscmu\u002FSkillLearnBench","https:\u002F\u002Fcxcscmu.github.io\u002FSkillLearnBench","https:\u002F\u002Fgithub.com\u002Fcxcscmu","2026-10-02 22:55:36",{"source":8,"skillName":5,"commands":47,"prompt":57,"zipAvailable":58},[48,51,54],{"tool":49,"command":50},"npx","npx skills add cxcscmu\u002FSkillLearnBench --skill run2_simpo-loss",{"tool":52,"command":53},"bunx","bunx skills add cxcscmu\u002FSkillLearnBench --skill run2_simpo-loss",{"tool":55,"command":56},"pnpm","pnpm dlx skills add cxcscmu\u002FSkillLearnBench --skill run2_simpo-loss","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fcxcscmu\u002FSkillLearnBench\u002Fblob\u002Fmain\u002Fskills\u002Fb2-self-feedback-claude-sonnet-4-6\u002Fnlp-paper-reproduction\u002Frun2_simpo-loss\u002FSKILL.md",true,[],[61,79,104,122,142,154],{"skillNo":62,"name":63,"nameZh":6,"description":64,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":65,"categories":66,"keywords":72,"publishedAt":33},"01M6KM5KH175P5ABP5JMX2SBDQ","object_counter","Count occurrences of an object in the image using computer vision algorithm.","https:\u002F\u002Fgithub.com\u002Fcxcscmu\u002FSkillLearnBench\u002Fblob\u002Fmain\u002Fskills\u002Fhuman_authored\u002Fvideo-object-counting\u002Fobject_counter\u002FSKILL.md",[67],{"categoryNo":68,"categoryName":69,"taxonomyType":70,"taxonomyTypeDesc":71},"01JBSEEDCAT000000000000016","图像与多媒体","SCENARIO","使用场景",[73,74,75,76,77,78],"object counter","object","vision","computer","Computer Vision","image",{"skillNo":80,"name":81,"nameZh":6,"description":82,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":83,"categories":84,"keywords":88,"publishedAt":33},"01M6KM51TMVXG79MMPCS6Y8G86","travel-planning-workflow","Invoke this skill when performing travel planning. It defines a structured workflow for generating itineraries. Following this workflow is essential to ensure the final plan satisfies user requirements.","https:\u002F\u002Fgithub.com\u002Fcxcscmu\u002FSkillLearnBench\u002Fblob\u002Fmain\u002Fskills\u002Fhuman_authored\u002Ftravel-planning\u002Ftravel-planning-workflow\u002FSKILL.md",[85],{"categoryNo":86,"categoryName":87,"taxonomyType":70,"taxonomyTypeDesc":71},"01JBSEEDCAT000000000000023","Skill与智能体开发",[89,90,91,92,93,94,95,96,97,98,99,100,101,102,103],"workflow","planning","planning workflow","travel","travel planning","travel planning workflow","structured","performing","plan","for","user","it","requirements","essential","itineraries",{"skillNo":105,"name":106,"nameZh":6,"description":107,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":108,"categories":109,"keywords":110,"publishedAt":33},"01M6KM4DY820XHS9YTSK5QP5HD","bundled-dataset-search-tool","invoke this skill when you need to perform database search for travel planning. This skill provides some useful pre-packaged tools to look up accommodations, attractions, cities, driving distance, flights, and restaurants from the bundled dataset.","https:\u002F\u002Fgithub.com\u002Fcxcscmu\u002FSkillLearnBench\u002Fblob\u002Fmain\u002Fskills\u002Fhuman_authored\u002Ftravel-planning\u002Fbundled-dataset-search-tool\u002FSKILL.md",[],[111,112,113,114,115,116,117,92,118,119,93,120,98,121,90],"search","tool","dataset","bundled dataset search tool","restaurants","Tools","up","look","flights","you","database",{"skillNo":123,"name":124,"nameZh":6,"description":125,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":126,"categories":127,"keywords":128,"publishedAt":33},"01M6KM465BB9PQ0B548HQSVAWZ","glm-output","Output-processing guidance for GLM tasks. Especially useful after glm-basics and glm-calibration when you need verifier-matching metrics from output.nc and a final \u002Froot\u002Fmetrics.json.","https:\u002F\u002Fgithub.com\u002Fcxcscmu\u002FSkillLearnBench\u002Fblob\u002Fmain\u002Fskills\u002Fhuman_authored\u002Ftemperature-simulation\u002Fglm-output\u002FSKILL.md",[],[129,130,131,98,132,133,134,135,136,137,138,139,120,140,141],"glm output","output","glm","tasks","metrics","json","basics","guidance","matching","verifier","processing","root","calibration",{"skillNo":143,"name":144,"nameZh":6,"description":145,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":146,"categories":147,"keywords":149,"publishedAt":33},"01M6KM40KY3QWDW5W97S39R2BC","glm-calibration","Calibration guidance for GLM tasks. Often most effective after glm-basics has clarified the setup; glm-output is the companion skill for exact final metric computation.","https:\u002F\u002Fgithub.com\u002Fcxcscmu\u002FSkillLearnBench\u002Fblob\u002Fmain\u002Fskills\u002Fhuman_authored\u002Ftemperature-simulation\u002Fglm-calibration\u002FSKILL.md",[148],{"categoryNo":86,"categoryName":87,"taxonomyType":70,"taxonomyTypeDesc":71},[150,141,131,130,136,135,132,151,152,98,153],"glm calibration","setup","companion","metric",{"skillNo":155,"name":156,"nameZh":6,"description":157,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":158,"categories":159,"keywords":160,"publishedAt":33},"01M6KM3VHHVSGMGHJQB3PPT6YG","glm-basics","A good starting point for GLM calibration tasks. Use to inspect glm3.nml, confirm how GLM runs, and identify the relevant files before moving on to calibration and output evaluation.","https:\u002F\u002Fgithub.com\u002Fcxcscmu\u002FSkillLearnBench\u002Fblob\u002Fmain\u002Fskills\u002Fhuman_authored\u002Ftemperature-simulation\u002Fglm-basics\u002FSKILL.md",[],[161,131,135,162,150,141,98,130,163,164,165,24,166,167,132],"glm basics","inspect","relevant","files","good","how","Evaluation"]