trl-setup
Setup and installation of TRL (Transformer Reinforcement Learning) library with compatible torch/transformers versions. Use when setting up preference optimization training environments.
- License 许可
- MIT
- 最近更新
- 2026-10-02
- Stars
- ★ 93
- Forks
- 8
01 前言 Front Matter
- name
- trl-setup
- description
- Setup and installation of TRL (Transformer Reinforcement Learning) library with compatible torch/transformers versions. Use when setting up preference optimization training environments.
03 SKILL.md 正文
TRL Setup
Installation
pip install torch transformers trl accelerate peft datasets --break-system-packages
Compatibility Notes
- TRL imports from
trl.trainer.utils:DPODataCollatorWithPadding,pad_to_length, etc. - Some imports may change across TRL versions (e.g.,
trl_sanitze_kwargs_for_taggingvstrl_sanitize_kwargs_for_tagging) - If running in externally-managed Python (Debian), use
--break-system-packagesflag
Common Import Errors
trl_sanitze_kwargs_for_taggingnot found → check trl version, may be renamed or removed- Missing
CPOTrainer→ upgrade trl:pip install --upgrade trl
Running Tests from Project Root
cd /root/SimPO && python -m pytest unit_test/unit_test_1.py -v
# or
cd /root/SimPO && python unit_test/unit_test_1.py
Key TRL Trainer Utilities
DPODataCollatorWithPadding: handles padding for preference pairspad_to_length: pads tensor to specified lengthdisable_dropout_in_model: disables dropout during eval
04 同仓库更多 Skills
object_counter
Count occurrences of an object in the image using computer vision algorithm.
travel-planning-workflow
Invoke this skill when performing travel planning. It defines a structured workflow for generating itineraries. Following this workflow is essential to ensure the final plan satisfies user requirements.
bundled-dataset-search-tool
invoke this skill when you need to perform database search for travel planning. This skill provides some useful pre-packaged tools to look up accommodations, attractions, cities, driving distance, flights, and restaurants from the bundled dataset.
glm-output
Output-processing guidance for GLM tasks. Especially useful after glm-basics and glm-calibration when you need verifier-matching metrics from output.nc and a final /root/metrics.json.
glm-calibration
Calibration guidance for GLM tasks. Often most effective after glm-basics has clarified the setup; glm-output is the companion skill for exact final metric computation.
glm-basics
A good starting point for GLM calibration tasks. Use to inspect glm3.nml, confirm how GLM runs, and identify the relevant files before moving on to calibration and output evaluation.