AGENT SKILL

simulation-and-rl

Route LeRobot Gymnasium simulation, benchmark evaluation, and bounded RL or HIL-SERL configuration work with explicit dependency, asset, backend, and safety gates.

License 许可
Apache-2.0
最近更新
2026-09-03
Stars
★ 351
Forks
27

01 前言 Front Matter

name
simulation-and-rl
license
Apache 2.0
metadata
{ "disco-role": "operating" }
description
Route LeRobot Gymnasium simulation, benchmark evaluation, and bounded RL or HIL-SERL configuration work with explicit dependency, asset, backend, and safety gates.
disable-model-invocation
true

02 资源文件 7

03 SKILL.md 正文

Simulation and RL

Use this skill when a task mentions env.type, Gymnasium, PushT, LIBERO, LIBERO-plus, MetaWorld, RoboTwin/RobotWin, VLABench, RoboCasa, RoboMME, IsaacLab Arena, RL, HIL-SERL, actor/learner, or a reward model. This skill routes environment construction and evaluation; it does not silently turn a configuration check into a simulator rollout or a real-robot action.

Route first

  • Route generic policy training, processors, checkpoints, and inference to policy-training-inference.
  • Route physical robot control, teleoperation, recording, and hardware safety to robot-control-data-collection.
  • Route remote jobs, hosted services, gRPC service deployment, and extension authoring to extensions-and-services.
  • Keep this skill responsible for simulation dispatch, benchmark evaluation, RL data flow, and reward integration.

Inputs and gates

Collect before changing a command or config:

  1. env.type, task/suite, observation mode, action mode, episode horizon, environment count, async preference, policy checkpoint, and evaluation episode count.
  2. Whether the request is a config/registry check, a CPU smoke, or an actual rollout. A config check must not call make_env() or reset().
  3. Platform, Python version, Gymnasium/NumPy versions, simulator package, asset-pack status, rendering mode, CUDA/Vulkan/MuJoCo backend, and any Hub credential or remote-code consent.
  4. For RL: actor/learner topology, algorithm (currently sac), policy action/state features, replay capacity, batch size, online/offline ratio, warmup threshold, device, and whether human intervention is intended.

Use scripts/environment_probe.py for dispatch-only inspection and scripts/rl_config_check.py for bounded RL config validation. Both are network-free, asset-free, non-actuating, and non-training.

Environment dispatch contract

make_env_config(type, **kwargs) resolves the registered EnvConfig choice. The config owns create_envs(n_envs, use_async_envs) and get_env_processors(). make_env(config) returns:

{suite_name: {task_id: gymnasium.vector.VectorEnv}}

The base config uses gym_<type>/<task> and imports the package only if that Gym ID is absent. Benchmark configs override creation for multi-task or third-party simulators. SyncVectorEnv is the conservative default; async workers are opt-in and are only selected for more than one environment. GPU, EGL, Vulkan, MuJoCo, SAPIEN, and asset initialization are not proved by config construction or registry inspection.

Raw environment observations use pixels, nested camera dictionaries, agent_pos, environment_state, or robot_state. preprocess_observation normalizes these to LeRobot observation keys; env_to_policy_features and the config features_map must agree with the policy. Evaluation depends on task, task_description, _max_episode_steps, and info["is_success"].

Benchmark routing

Trigger Config route Required gate
PushT pusht, PushT-v0 gym-pusht and Pymunk; CPU smoke is reasonable
Aloha aloha gym-aloha; verify simulator import before rollout
LIBERO libero Linux, hf-libero, MuJoCo assets, render backend
LIBERO-plus libero_plus Plus fork/assets; do not mix with vanilla LIBERO
MetaWorld metaworld MetaWorld 3.0.0 and compatible Gymnasium
RobotWin/RoboTwin robotwin external RoboTwin tree, SAPIEN/CuRobo assets, NVIDIA GPU
VLABench vlabench external package, MuJoCo/dm-control, mesh assets, Linux
RoboCasa robocasa editable external packages, MuJoCo, kitchen/object assets
RoboMME robomme isolated Linux ManiSkill/SAPIEN/Vulkan environment
IsaacLab Arena isaaclab_arena Hub access plus explicit trusted remote-code consent
HIL simulation gym_manipulator hilserl extra, gym_hil, input device, usually NVIDIA GPU

Use the detailed matrix and known conflicts in environment-overview.md and compatibility.md. Do not claim a runnable rollout from a present Python module alone.

Configuration decisions

Read configuration.md before editing nested fields. Validate task names, observation/action dimensions, camera keys, n_envs >= 1, positive FPS/horizon, and policy feature alignment. Keep hard_reset=false for LIBERO only when fixed initial states are enabled and reproducibility trade-offs are accepted. Use use_async_envs=false for the first smoke when a simulator has fragile rendering or fork behavior.

Hub environment strings or HubEnvConfig.hub_path are a separate path: make_env downloads/imports remote env.py only with trust_remote_code=true. Treat this as code execution and a credential/network boundary, never as a local registry result.

Evaluation workflow

  1. Run environment_probe.py --env-type ... and save its JSON output.
  2. Resolve missing extras, version conflicts, backend, credentials, and assets; stop on a required gap rather than substituting an unrelated simulator.
  3. For a CPU dispatch smoke, call make_env only for a known installed, lightweight candidate; reset at most one bounded episode and close it.
  4. For a real benchmark evaluation, use lerobot-eval with the matching policy feature names, --eval.batch_size=1 first, explicit task and episode count, and the benchmark's render/backend settings.
  5. Verify success information and observation/action shapes before increasing parallelism. Record whether the result is config-only, partial smoke, or full rollout; do not label substitution as benchmark recovery.

See workflows.md for safe command patterns and simulation-specific evaluation notes.

RL and reward boundaries

The gRPC-free RL core exposes ReplayBuffer, OnlineOfflineMixer, RLTrainer, RLAlgorithmConfig, and SAC. RLTrainer.training_step() asks the algorithm for data, and SAC consumes batches containing state, action, reward, next state, done, and truncation flags. online_ratio must be in [0, 1]; SAC's policy_config must be populated before make_algorithm.

HIL-SERL actor/learner mode is different from a local RL smoke: the actor collects transitions and interventions, queues them, receives weights, and requires a learner service; the learner fills replay, delays updates until online_step_before_learning, trains, checkpoints, and periodically pushes weights. It requires the hilserl extra and hardware or gym_hil simulation. Do not start actor/learner processes merely to validate JSON.

Reward configs route through make_reward_model_config, make_reward_model, and make_reward_pre_post_processors. Supported built-in names are reward_classifier, sarm, robometer, and topreward. Reward models consume preprocessed tensor batches and emit per-sample rewards; classifier threshold/success-reward settings affect HIL transition processing. Transformer/VLM-backed models may need model weights, credentials, network, and GPU memory. Zero-shot reward inference is not the same as trainable reward model fitting.

Recovery and handoff

On failure, classify it as missing package, version conflict, missing asset, backend/display failure, credential/remote-code refusal, policy feature mismatch, or actual simulator/runtime failure. Follow troubleshooting.md; do not hide the class by switching benchmarks. Report the exact command, status (config, dispatch, CPU smoke, GPU rollout, or none), unresolved gates, and whether any substitution was used.

Bundled tools

  • environment_probe.py: registry/config and package-presence inspection; never downloads, creates, resets, or closes an external simulator.
  • rl_config_check.py: bounded JSON schema/value checks; never trains, contacts an actor/learner, downloads weights, or initializes a simulator.

04 同仓库更多 Skills

VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-triton-opt

Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-solve

END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-skill

PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-pattern-fusion

Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-orchestrate

Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-feedback

Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).

★ 351