AGENT SKILL

simulation-rendering

Use for MuJoCo model loading, simulation state and time inspection, headless or onscreen rendering, camera/output configuration, and viewer/display diagnostics; route environment catalog, XML editing/IK, MJX acceleration, and training elsewhere.

License 许可
Apache-2.0
最近更新
2026-09-03
Stars
★ 351
Forks
27

01 前言 Front Matter

name
simulation-rendering
license
Apache 2.0
metadata
{ "disco-role": "operating" }
description
Use for MuJoCo model loading, simulation state and time inspection, headless or onscreen rendering, camera/output configuration, and viewer/display diagnostics; route environment catalog, XML editing/IK, MJX acceleration, and training elsewhere.
disable-model-invocation
true

02 资源文件 3

03 SKILL.md 正文

Simulation and rendering

Use this sub-skill when the task concerns the MuJoCo model/data handles behind a MyoSuite environment, rendering pixels or depth, selecting cameras, preserving or restoring simulator state, or diagnosing a viewer/display failure. It covers the base MuJoCo backend in MyoSuite 2.x. It does not teach task selection, XML mutation or IK, MJX/JAX acceleration, or RL training.

Route first

  • Choose environment workflows for registry lookup, task configuration, reset/step semantics, or environment-specific observation and reward behavior.
  • Choose model editing and kinematics for XML changes, MjSpec editing, site/body edits as a modeling operation, or IK.
  • Choose MJX acceleration for JAX, MJX, CUDA, batching, or accelerator performance.
  • Choose training integration for policies, learners, checkpoints, or long-running experiments.

Fast decision: which rendering path?

  1. Headless or CI: load mujoco.MjModel/mujoco.MjData, step with mujoco.mj_step, and use MJRenderer.render_offscreen(...), or run the bundled safe checker: python scripts/check_mujoco_xml.py --help.
  2. A window is explicitly wanted: env.mj_render() delegates to env.mj_renderer.render_to_window(), which creates a native passive viewer on first use. This is display-dependent and is not verification ground truth.
  3. Raw XML diagnosis: run python scripts/check_mujoco_xml.py --xml MODEL.xml --render none first. It loads and steps without creating a viewer. Add --render offscreen only when pixel output is required.

Never use myosuite.utils.examine_sim as a headless check: its documented workflow calls mujoco.viewer.launch(...) and is intentionally onscreen. Its safe native candidate is python -m myosuite.utils.examine_sim --help only.

Environment rendering contract

from myosuite.utils import gym

env = gym.make("myoElbowPose1D6MRandom-v0")
env.reset(seed=1234)
frame = env.mj_renderer.render_offscreen(
    width=320, height=240, camera_id=-1, rgb=True
)
env.close()
  • env.mj_model is the compiled mujoco.MjModel; env.mj_data is its live mujoco.MjData. The base environment constructs env.mj_renderer as an MJRenderer for those handles.
  • camera_id=-1 means MuJoCo's free camera. A named camera string or numeric camera id selects a model camera. Use a model camera name only after checking that it exists; a missing name is a model/rendering error, not a display fix.
  • render_offscreen returns an RGB numpy array for the default call. The detailed return matrix for RGB/depth/segmentation is in rendering-api.md.
  • env.mj_render() is deliberately different: it opens/synchronizes a window and returns no frame. Do not call it in a server, CI job, or headless synthetic case.
  • env.viewer_setup(distance=..., azimuth=..., elevation=..., lookat=..., render_actuator=..., render_tendon=...) stores free-camera and visualization settings for the next window or offscreen scene. Camera/output details and the distance adjustment are documented in rendering-api.md.

Deterministic headless rollout

For a raw model, the safe contract is:

python scripts/check_mujoco_xml.py \
  --xml MODEL.xml --render offscreen --frames 8 \
  --width 320 --height 240 --output-dir ./mujoco-check

The command never calls mujoco.viewer or launch_passive. It prints model sizes, steps the requested number of frames, and writes one binary PPM image per rendered frame under the supplied output directory (for example, mujoco-check/frame-0000.ppm). --render none is the default and creates no output directory. The script accepts optional comma-separated --qpos and --ctrl; it validates their lengths before applying them. See the script's --help output for the complete input/output contract.

For a policy/environment rollout, use MujocoEnv.examine_policy_new only when its policy API is already available. Its supported render values are "onscreen", "offscreen", and "none"; offscreen mode accumulates RGB frames and writes an MP4 per episode through imageio. Set an explicit output_dir, filename, frame_size, and finite horizon. The safe raw-model script is preferred for a small, dependency-light rendering check.

State and time workflow

  • env.dt is env.mj_model.opt.timestep * env.frame_skip.
  • env.time reads the observed simulation data time (env.obsd_mj_data.time), so compare it with env.mj_data.time only when the environment uses the same observed and ground-truth model.
  • env.get_env_state() returns copied time, qpos, qvel, optional act, optional mocap position/quaternion, optional site position/quaternion, and body position/quaternion arrays.
  • Save the returned dictionary before a branch, then call env.set_env_state(state) to restore the base state. The method updates both ground-truth and observed data where present and performs a MuJoCo step to refresh derived state; treat restoration as a simulator-state operation, not as a byte-for-byte snapshot of every internal buffer.
  • env.reset(...) establishes the task's initial state. After changing raw qpos/qvel, use mujoco.mj_forward(model, data) before inspecting derived positions or rendering. Do not edit model arrays merely to move a body unless the task is explicitly a model-editing task.

Visual observations

Visual observations are opt-in. Call env.get_obs(update_exteroception=True) or env.get_visuals(...) when the environment has configured visual_keys. Supported key forms are rgb:CAMERA:HxW:1d, rgb:CAMERA:HxW:2d, and optional encoder forms such as r3m18, r3m34, r3m50, rrl, or vc1 when their optional encoder dependencies are installed. A matching d: key requests depth alongside the RGB key. get_obs() does not refresh exteroception by default, and env_info["visual_dict"] may therefore be empty or stale unless visuals were explicitly updated at the current simulation time.

Backend and safety boundary

Base MuJoCo model loading, stepping, state inspection, and the MJRenderer offscreen path are CPU/base-package capabilities, subject to a usable graphics backend for pixel rendering. Onscreen viewing additionally requires a display and native viewer support. MJX/JAX/CUDA is optional and is not established by an offscreen CPU check; route it to the MJX sub-skill and report it separately.

Before reporting success, check the actual artifact: model load/step succeeded, rendered arrays have the requested dimensions, and every requested output file exists and is non-empty. For failure symptoms and recovery branches, use troubleshooting.md.

04 同仓库更多 Skills

VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-triton-opt

Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-solve

END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-skill

PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-pattern-fusion

Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-orchestrate

Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-feedback

Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).

★ 351