AGENT SKILL

trl

Use and modify TRL, the Hugging Face Transformers Reinforcement Learning library for post-training, CLI workflows, data/reward utilities, scaling backends, experimental environments, and repo development.

License 许可
Apache-2.0
最近更新
2026-09-03
Stars
★ 351
Forks
27

01 前言 Front Matter

name
trl
license
Apache 2.0
metadata
{ "disco-role": "operating" }
description
Use and modify TRL, the Hugging Face Transformers Reinforcement Learning library for post-training, CLI workflows, data/reward utilities, scaling backends, experimental environments, and repo development.
disable-model-invocation
true

02 资源文件 5

03 SKILL.md 正文

TRL Repo Skill

Use this skill for TRL (Transformers Reinforcement Learning), a Hugging Face library for post-training foundation models with SFT, DPO, GRPO, reward modeling, RLOO, experimental RLHF methods, CLI launchers, dataset utilities, reward functions, and scaling integrations.

Start Here

  • Read repo provenance before deciding whether this skill matches a current checkout or should be refreshed.
  • Read installation and extras when a task involves installing TRL, optional extras, Python versions, or import checks.
  • Read troubleshooting for cross-cutting install/import, optional dependency, CLI, data, backend, and repo-development failures.
  • Run scripts/check_trl_environment.py --help when you need a safe local diagnostic helper; it checks imports, metadata, CLI help, and optional packages without training or downloads.

Route by Task

  • Use core-training for stable trainer selection and API usage: SFTTrainer, DPOTrainer, GRPOTrainer, RewardTrainer, RLOOTrainer, and their configs.
  • Use data-and-rewards for dataset schemas, chat templates, multimodal messages, packing, prompt extraction, tool-call parsing, and built-in reward functions.
  • Use cli-and-configs for trl commands, YAML configs, dataset mixtures, trl env, trl skills, and command/config parser troubleshooting.
  • Use scaling-and-backends for Accelerate, FSDP, DeepSpeed, PEFT, quantization, Liger, Unsloth, Kernels Hub, vLLM, memory planning, and optional backend diagnostics.
  • Use experimental-and-environments for trl.experimental, experimental trainers, GRPOTrainer(environment_factory=...), OpenEnv, OpenReward, Harbor, and tool-calling environment training.
  • Use repo-development when modifying this repository: duplicated trainer logic, tests, docs, paper index updates, packaged skills, and contribution policy.

Minimal Install and Import Check

Install the public package for normal use:

pip install trl
python - <<'PY'
import trl
from trl import SFTTrainer, DPOTrainer, GRPOTrainer, RewardTrainer
print('trl import ok')
PY

For source development, use an editable install from the checkout:

pip install -e .
trl --help
trl env

Install optional extras only when the selected workflow needs them. Examples include trl[peft] for adapter training, trl[vllm] for vLLM generation/server support, trl[deepspeed] for DeepSpeed, trl[vlm] for vision-language examples, trl[openreward] for OpenReward integration, and trl[harbor] for Harbor tasks. Avoid broad dev installs unless doing repository development.

Common Workflow Routing

  • "Fine-tune a model with SFT/DPO/GRPO": start in core-training, then use data-and-rewards for dataset shape and cli-and-configs if the user wants a command or YAML file.
  • "My dataset columns or chat template are wrong": start in data-and-rewards; use its validator script before changing trainer code.
  • "Generate a trl sft or trl grpo command": start in cli-and-configs, then cross-check trainer semantics in core-training.
  • "Use vLLM, DeepSpeed, FSDP, LoRA, quantization, or Unsloth": start in scaling-and-backends; do not start servers or training jobs unless the user asked and prerequisites are available.
  • "Train with tools, OpenEnv, OpenReward, or Harbor": start in experimental-and-environments; verify extras, services, credentials, and stability warnings.
  • "Edit TRL internals": start in repo-development; maintain duplicated trainer logic consistency and update docs/source/paper_index.md for paper-backed methods.

Safety and Validation

  • Prefer read-only checks first: imports, trl --help, config rendering, dataset validation, and optional backend diagnostics.
  • Treat examples, notebooks, long training runs, vLLM servers, Docker/sandbox tasks, and distributed jobs as expensive or environment-dependent unless explicitly requested.
  • Do not assume optional dependencies are installed because the base package imports.
  • Keep trainer changes consistent across duplicated blocks when editing the repository.
  • Use focused tests for changed areas; skip invariant, distributed, service, GPU, or long-running candidates unless they match the change and the environment supports them.

04 同仓库更多 Skills

VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-triton-opt

Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-solve

END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-skill

PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-pattern-fusion

Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-orchestrate

Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-feedback

Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).

★ 351