[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M74YKH3BQJHJ05PR0XWSTPDF":3,"skill-same:01M74YKH3BQJHJ05PR0XWSTPDF":64},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":30,"commitSha":31,"skillDirectory":32,"forksCount":33,"repositoryHtmlUrl":34,"repositoryHomepageUrl":35,"ownerHtmlUrl":36,"ownerBlogUrl":37,"ownerDescription":38,"updatedAt":39,"install":40,"resources":53},"01M74YKH3BQJHJ05PR0XWSTPDF","evaluating-cosmos-policy",null,"Evaluates NVIDIA Cosmos Policy on LIBERO and RoboCasa simulation environments. Use when setting up cosmos-policy for robot manipulation evaluation, running headless GPU evaluations with EGL rendering, or profiling inference latency on cluster or local GPU machines.","KalarisLabs\u002Fresearch-agent-skills","KalarisLabs","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F300878568?v=4","01M74Y7MT23GXXC56VM6TVSXZY","Kalarislabs","01M74Y7MTQM9REXZX4SNXWX95C",132,"MIT","https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fevaluating-cosmos-policy\u002FSKILL.md",[],[],"2026-11-12 13:55:43",false,"---\nname: evaluating-cosmos-policy\ndescription: Evaluates NVIDIA Cosmos Policy on LIBERO and RoboCasa simulation environments. Use when setting up cosmos-policy for robot manipulation evaluation, running headless GPU evaluations with EGL rendering, or profiling inference latency on cluster or local GPU machines.\nlicense: MIT\nmetadata:\n  version: 1.0.0\n  category: multimodal-and-emerging\n  maintainer: Kalaris Labs\n  tags: Cosmos Policy, VLA, Robotics, LIBERO, RoboCasa, Simulation, Evaluation, Profiling, EGL Rendering\n  dependencies: torch>=2.1.0, mujoco>=3.0.0, robosuite>=1.4.0, robocasa @ git+https:\u002F\u002Fgithub.com\u002Fmoojink\u002Frobocasa-cosmos-policy.git, transformers>=4.40.0, cosmos-policy @ git+https:\u002F\u002Fgithub.com\u002FNVlabs\u002Fcosmos-policy.git\n---\n\n# Cosmos Policy Evaluation\n\nEvaluation workflows for NVIDIA Cosmos Policy on LIBERO and RoboCasa simulation environments from the public `cosmos-policy` repository. Covers blank-machine setup, headless GPU evaluation, and inference profiling.\n\n## Quick start\n\nRun a minimal LIBERO evaluation using the official public eval module:\n\n```bash\nuv run --extra cu128 --group libero --python 3.10 \\\n  python -m cosmos_policy.experiments.robot.libero.run_libero_eval \\\n    --config cosmos_predict2_2b_480p_libero__inference_only \\\n    --ckpt_path nvidia\u002FCosmos-Policy-LIBERO-Predict2-2B \\\n    --config_file cosmos_policy\u002Fconfig\u002Fconfig.py \\\n    --use_wrist_image True \\\n    --use_proprio True \\\n    --normalize_proprio True \\\n    --unnormalize_actions True \\\n    --dataset_stats_path nvidia\u002FCosmos-Policy-LIBERO-Predict2-2B\u002Flibero_dataset_statistics.json \\\n    --t5_text_embeddings_path nvidia\u002FCosmos-Policy-LIBERO-Predict2-2B\u002Flibero_t5_embeddings.pkl \\\n    --trained_with_image_aug True \\\n    --chunk_size 16 \\\n    --num_open_loop_steps 16 \\\n    --task_suite_name libero_10 \\\n    --num_trials_per_task 1 \\\n    --local_log_dir cosmos_policy\u002Fexperiments\u002Frobot\u002Flibero\u002Flogs\u002F \\\n    --seed 195 \\\n    --randomize_seed False \\\n    --deterministic True \\\n    --run_id_note smoke \\\n    --ar_future_prediction False \\\n    --ar_value_prediction False \\\n    --use_jpeg_compression True \\\n    --flip_images True \\\n    --num_denoising_steps_action 5 \\\n    --num_denoising_steps_future_state 1 \\\n    --num_denoising_steps_value 1 \\\n    --data_collection False\n```\n\n## Core concepts\n\n**What Cosmos Policy is**: NVIDIA Cosmos Policy is a vision-language-action (VLA) model that uses Cosmos Tokenizer to encode visual observations into discrete tokens, then predicts robot actions conditioned on language instructions and visual context.\n\n**Key architecture choices**:\n\n| Component | Design |\n|-----------|--------|\n| Visual encoder | Cosmos Tokenizer (discrete tokens) |\n| Language conditioning | Cross-attention to language embeddings |\n| Action prediction | Autoregressive action token generation |\n\n**Public command surface**: The supported evaluation entrypoints are `cosmos_policy.experiments.robot.libero.run_libero_eval` and `cosmos_policy.experiments.robot.robocasa.run_robocasa_eval`. Keep reproduction notes anchored to these public modules and their documented flags.\n\n## Compute requirements\n\n| Task | GPU | VRAM | Typical wall time |\n|------|-----|------|-------------------|\n| LIBERO smoke eval (1 trial) | 1x A40\u002FA100 | ~16 GB | 5-10 min |\n| LIBERO full eval (50 trials) | 1x A40\u002FA100 | ~16 GB | 2-4 hours |\n| RoboCasa single-task (2 trials) | 1x A40\u002FA100 | ~18 GB | 10-15 min |\n| RoboCasa all-tasks | 1x A40\u002FA100 | ~18 GB | 4-8 hours |\n\n## When to use vs alternatives\n\n**Use this skill when:**\n- Evaluating NVIDIA Cosmos Policy on LIBERO or RoboCasa benchmarks\n- Profiling inference latency and throughput for Cosmos Policy\n- Setting up headless EGL rendering for robot simulation on GPU clusters\n\n**Use alternatives when:**\n- Training or fine-tuning Cosmos Policy from scratch (use official Cosmos training docs)\n- Working with OpenVLA-based policies (use `fine-tuning-openvla-oft`)\n- Working with Physical Intelligence pi0 models (use `fine-tuning-serving-openpi`)\n- Running real-robot evaluation rather than simulation\n\n---\n\n## Workflow 1: LIBERO evaluation\n\nCopy this checklist and track progress:\n\n```text\nLIBERO Eval Progress:\n- [ ] Step 1: Install environment and dependencies\n- [ ] Step 2: Configure headless EGL rendering\n- [ ] Step 3: Run smoke evaluation\n- [ ] Step 4: Validate outputs and parse results\n- [ ] Step 5: Run full benchmark if smoke passes\n```\n\n**Step 1: Install environment**\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FNVlabs\u002Fcosmos-policy.git\ncd cosmos-policy\n# Follow SETUP.md to build and enter the supported Docker container.\n# Then, inside the container:\nuv sync --extra cu128 --group libero --python 3.10\n```\n\n**Step 2: Configure headless rendering**\n\n```bash\nexport CUDA_VISIBLE_DEVICES=0\nexport MUJOCO_EGL_DEVICE_ID=0\nexport MUJOCO_GL=egl\nexport PYOPENGL_PLATFORM=egl\n```\n\n**Step 3: Run smoke evaluation**\n\n```bash\nuv run --extra cu128 --group libero --python 3.10 \\\n  python -m cosmos_policy.experiments.robot.libero.run_libero_eval \\\n    --config cosmos_predict2_2b_480p_libero__inference_only \\\n    --ckpt_path nvidia\u002FCosmos-Policy-LIBERO-Predict2-2B \\\n    --config_file cosmos_policy\u002Fconfig\u002Fconfig.py \\\n    --use_wrist_image True \\\n    --use_proprio True \\\n    --normalize_proprio True \\\n    --unnormalize_actions True \\\n    --dataset_stats_path nvidia\u002FCosmos-Policy-LIBERO-Predict2-2B\u002Flibero_dataset_statistics.json \\\n    --t5_text_embeddings_path nvidia\u002FCosmos-Policy-LIBERO-Predict2-2B\u002Flibero_t5_embeddings.pkl \\\n    --trained_with_image_aug True \\\n    --chunk_size 16 \\\n    --num_open_loop_steps 16 \\\n    --task_suite_name libero_10 \\\n    --num_trials_per_task 1 \\\n    --local_log_dir cosmos_policy\u002Fexperiments\u002Frobot\u002Flibero\u002Flogs\u002F \\\n    --seed 195 \\\n    --randomize_seed False \\\n    --deterministic True \\\n    --run_id_note smoke \\\n    --ar_future_prediction False \\\n    --ar_value_prediction False \\\n    --use_jpeg_compression True \\\n    --flip_images True \\\n    --num_denoising_steps_action 5 \\\n    --num_denoising_steps_future_state 1 \\\n    --num_denoising_steps_value 1 \\\n    --data_collection False\n```\n\n**Step 4: Validate and parse results**\n\n```python\nimport json\nimport glob\n\n# Find latest evaluation result from the official log directory\nlog_files = sorted(glob.glob(\"cosmos_policy\u002Fexperiments\u002Frobot\u002Flibero\u002Flogs\u002F**\u002F*.json\", recursive=True))\nwith open(log_files[-1]) as f:\n    results = json.load(f)\n\nprint(results)\n```\n\n**Step 5: Scale up**\n\nRun across all four LIBERO task suites with 50 trials:\n\n```bash\nfor suite in libero_spatial libero_object libero_goal libero_10; do\n  uv run --extra cu128 --group libero --python 3.10 \\\n    python -m cosmos_policy.experiments.robot.libero.run_libero_eval \\\n      --config cosmos_predict2_2b_480p_libero__inference_only \\\n      --ckpt_path nvidia\u002FCosmos-Policy-LIBERO-Predict2-2B \\\n      --config_file cosmos_policy\u002Fconfig\u002Fconfig.py \\\n      --use_wrist_image True \\\n      --use_proprio True \\\n      --normalize_proprio True \\\n      --unnormalize_actions True \\\n      --dataset_stats_path nvidia\u002FCosmos-Policy-LIBERO-Predict2-2B\u002Flibero_dataset_statistics.json \\\n      --t5_text_embeddings_path nvidia\u002FCosmos-Policy-LIBERO-Predict2-2B\u002Flibero_t5_embeddings.pkl \\\n      --trained_with_image_aug True \\\n      --chunk_size 16 \\\n      --num_open_loop_steps 16 \\\n      --task_suite_name \"$suite\" \\\n      --num_trials_per_task 50 \\\n      --local_log_dir cosmos_policy\u002Fexperiments\u002Frobot\u002Flibero\u002Flogs\u002F \\\n      --seed 195 \\\n      --randomize_seed False \\\n      --deterministic True \\\n      --run_id_note \"suite_${suite}\" \\\n      --ar_future_prediction False \\\n      --ar_value_prediction False \\\n      --use_jpeg_compression True \\\n      --flip_images True \\\n      --num_denoising_steps_action 5 \\\n      --num_denoising_steps_future_state 1 \\\n      --num_denoising_steps_value 1 \\\n      --data_collection False\ndone\n```\n\n---\n\n## Workflow 2: RoboCasa evaluation\n\nCopy this checklist and track progress:\n\n```text\nRoboCasa Eval Progress:\n- [ ] Step 1: Install RoboCasa assets and verify macros\n- [ ] Step 2: Run single-task smoke evaluation\n- [ ] Step 3: Validate outputs\n- [ ] Step 4: Expand to multi-task runs\n```\n\n**Step 1: Install RoboCasa**\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002Fmoojink\u002Frobocasa-cosmos-policy.git\nuv pip install -e robocasa-cosmos-policy\npython -m robocasa.scripts.setup_macros\npython -m robocasa.scripts.download_kitchen_assets\n```\n\nThis fork installs the `robocasa` Python package expected by Cosmos Policy while preserving the patched environment changes used in the public RoboCasa eval path. Verify `macros_private.py` exists and paths are correct.\n\n**Step 2: Single-task smoke evaluation**\n\n```bash\nuv run --extra cu128 --group robocasa --python 3.10 \\\n  python -m cosmos_policy.experiments.robot.robocasa.run_robocasa_eval \\\n    --config cosmos_predict2_2b_480p_robocasa_50_demos_per_task__inference \\\n    --ckpt_path nvidia\u002FCosmos-Policy-RoboCasa-Predict2-2B \\\n    --config_file cosmos_policy\u002Fconfig\u002Fconfig.py \\\n    --use_wrist_image True \\\n    --num_wrist_images 1 \\\n    --use_proprio True \\\n    --normalize_proprio True \\\n    --unnormalize_actions True \\\n    --dataset_stats_path nvidia\u002FCosmos-Policy-RoboCasa-Predict2-2B\u002Frobocasa_dataset_statistics.json \\\n    --t5_text_embeddings_path nvidia\u002FCosmos-Policy-RoboCasa-Predict2-2B\u002Frobocasa_t5_embeddings.pkl \\\n    --trained_with_image_aug True \\\n    --chunk_size 32 \\\n    --num_open_loop_steps 16 \\\n    --task_name TurnOffMicrowave \\\n    --obj_instance_split A \\\n    --num_trials_per_task 2 \\\n    --local_log_dir cosmos_policy\u002Fexperiments\u002Frobot\u002Frobocasa\u002Flogs\u002F \\\n    --seed 195 \\\n    --randomize_seed False \\\n    --deterministic True \\\n    --run_id_note smoke \\\n    --use_variance_scale False \\\n    --use_jpeg_compression True \\\n    --flip_images True \\\n    --num_denoising_steps_action 5 \\\n    --num_denoising_steps_future_state 1 \\\n    --num_denoising_steps_value 1 \\\n    --data_collection False\n```\n\n**Step 3: Validate outputs**\n\n- Confirm the eval log prints the expected task name, object split, and checkpoint\u002Fconfig values.\n- Inspect the final `Success rate:` line in the log.\n\n**Step 4: Expand scope**\n\nIncrease `--num_trials_per_task` or add more tasks. Keep `--obj_instance_split` fixed across repeated runs for comparability.\n\n---\n\n## Workflow 3: Blank-machine cluster launch\n\n```text\nCluster Launch Progress:\n- [ ] Step 1: Clone the public repo and enter the supported runtime\n- [ ] Step 2: Sync the benchmark-specific dependency group\n- [ ] Step 3: Export rendering and cache environment variables before eval\n```\n\n**Step 1: Clone and enter the supported runtime**\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FNVlabs\u002Fcosmos-policy.git\ncd cosmos-policy\n# Follow SETUP.md, start the Docker container, and enter it before continuing.\n```\n\n**Step 2: Sync dependencies**\n\n```bash\nuv sync --extra cu128 --group libero --python 3.10\n# or, for RoboCasa:\nuv sync --extra cu128 --group robocasa --python 3.10\n# then install the Cosmos-compatible RoboCasa fork:\ngit clone https:\u002F\u002Fgithub.com\u002Fmoojink\u002Frobocasa-cosmos-policy.git\nuv pip install -e robocasa-cosmos-policy\n```\n\n**Step 3: Export runtime environment**\n\n```bash\nexport CUDA_VISIBLE_DEVICES=0\nexport MUJOCO_EGL_DEVICE_ID=0\nexport MUJOCO_GL=egl\nexport PYOPENGL_PLATFORM=egl\nexport HF_HOME=${HF_HOME:-$HOME\u002F.cache\u002Fhuggingface}\nexport TRANSFORMERS_CACHE=${TRANSFORMERS_CACHE:-$HF_HOME}\n```\n\n---\n\n## Expected performance benchmarks\n\nReference values from official evaluation (tied to specific setup and seeds):\n\n| Task Suite | Success Rate | Notes |\n|-----------|-------------|-------|\n| LIBERO-Spatial | 98.1% | Official LIBERO spatial result |\n| LIBERO-Object | 100.0% | Official LIBERO object result |\n| LIBERO-Goal | 98.2% | Official LIBERO goal result |\n| LIBERO-Long | 97.6% | Official LIBERO long-horizon result |\n| LIBERO-Average | 98.5% | Official average across LIBERO suites |\n| RoboCasa | 67.1% | Official RoboCasa average result |\n\n**Reproduction note**: Published success rates still depend on checkpoint choice, task suite, seeds, and simulator setup. Record the exact command and environment alongside any reported number.\n\n---\n\n## Non-negotiable rules\n\n- **EGL alignment**: Always set `CUDA_VISIBLE_DEVICES`, `MUJOCO_EGL_DEVICE_ID`, `MUJOCO_GL=egl`, and `PYOPENGL_PLATFORM=egl` together on headless GPU nodes.\n- **Official runtime first**: If host-Python installs hit binary compatibility issues, fall back to the supported container workflow from `SETUP.md` before debugging package internals.\n- **Cache consistency**: Use the same cache directory across setup and eval so Hugging Face and dependency caches are reused.\n- **Run comparability**: Keep task name, object split, seed, and trial count fixed across repeated runs.\n\n---\n\n## Common issues\n\n**Issue: binary compatibility or loader failures on host Python**\n\nFix: rerun inside the official container\u002Fruntime from `SETUP.md`. Do not assume host-package rebuilds will match the public release environment.\n\n**Issue: LIBERO prompts for config path in a non-interactive shell**\n\nFix: pre-create `LIBERO_CONFIG_PATH\u002Fconfig.yaml`:\n\n```python\nimport os, yaml\n\nconfig_dir = os.path.expanduser(\"~\u002F.libero\")\nos.makedirs(config_dir, exist_ok=True)\nwith open(os.path.join(config_dir, \"config.yaml\"), \"w\") as f:\n    yaml.dump({\"benchmark_root\": \"\u002Fpath\u002Fto\u002Flibero\u002Fdatasets\"}, f)\n```\n\n**Issue: EGL initialization or shutdown noise**\n\nFix: align EGL environment variables first. Treat teardown-only `EGL_NOT_INITIALIZED` warnings as low-signal unless the job exits non-zero.\n\n**Issue: Kitchen object sampling NaNs or asset lookup failures in RoboCasa**\n\nFix: rerun asset setup and confirm the patched robocasa install is intact:\n\n```bash\npython -m robocasa.scripts.download_kitchen_assets\npython -c \"import robocasa; print(robocasa.__file__)\"\n```\n\n**Issue: MuJoCo rendering mismatch**\n\nFix: verify GPU device alignment:\n\n```python\nimport os\ncuda_dev = os.environ.get(\"CUDA_VISIBLE_DEVICES\", \"not set\")\negl_dev = os.environ.get(\"MUJOCO_EGL_DEVICE_ID\", \"not set\")\nassert cuda_dev == egl_dev, f\"GPU mismatch: CUDA={cuda_dev}, EGL={egl_dev}\"\nprint(f\"Rendering on GPU {cuda_dev}\")\n```\n\n---\n\n## Advanced topics\n\n**LIBERO command matrix**: See [references\u002Flibero-commands.md](references\u002Flibero-commands.md)\n**RoboCasa command matrix**: See [references\u002Frobocasa-commands.md](references\u002Frobocasa-commands.md)\n\n## Resources\n\n- Cosmos Policy repository: https:\u002F\u002Fgithub.com\u002FNVlabs\u002Fcosmos-policy\n- LIBERO benchmark: https:\u002F\u002Fgithub.com\u002FLifelong-Robot-Learning\u002FLIBERO\n- Cosmos-compatible RoboCasa fork: https:\u002F\u002Fgithub.com\u002Fmoojink\u002Frobocasa-cosmos-policy\n- Upstream RoboCasa project: https:\u002F\u002Fgithub.com\u002Frobocasa\u002Frobocasa\n- MuJoCo documentation: https:\u002F\u002Fmujoco.readthedocs.io\u002F\n\n## Agent operating procedure\n\n1. **Check the environment.** Confirm GPU memory, model checkpoint and licence, framework versions and input formats.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Run inference on a single sample at low resolution or short length first.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Inspect outputs qualitatively and with task metrics; check licence restrictions for generated content.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| Out-of-memory on large inputs | Reduce resolution or sequence length, use half precision, or tile the input. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Check model and dataset licences before commercial or published use.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `fine-tuning-serving-openpi`: Fine-tune and serve Physical Intelligence OpenPI models (pi0, pi0-fast, pi0.5) using JAX or PyTorch backends for robot policy inference acr…\n- `fine-tuning-openvla-oft`: Fine-tunes and evaluates OpenVLA-OFT and OpenVLA-OFT+ policies for robot action generation with continuous action heads, LoRA adaptation, a…\n- `phoenix-observability`: Open-source AI observability platform for LLM tracing, evaluation, and monitoring.\n","# Cosmos Policy Evaluation\n\nEvaluation workflows for NVIDIA Cosmos Policy on LIBERO and RoboCasa simulation environments from the public `cosmos-policy` repository. Covers blank-machine setup, headless GPU evaluation, and inference profiling.\n\n## Quick start\n\nRun a minimal LIBERO evaluation using the official public eval module:\n\n```bash\nuv run --extra cu128 --group libero --python 3.10 \\\n  python -m cosmos_policy.experiments.robot.libero.run_libero_eval \\\n    --config cosmos_predict2_2b_480p_libero__inference_only \\\n    --ckpt_path nvidia\u002FCosmos-Policy-LIBERO-Predict2-2B \\\n    --config_file cosmos_policy\u002Fconfig\u002Fconfig.py \\\n    --use_wrist_image True \\\n    --use_proprio True \\\n    --normalize_proprio True \\\n    --unnormalize_actions True \\\n    --dataset_stats_path nvidia\u002FCosmos-Policy-LIBERO-Predict2-2B\u002Flibero_dataset_statistics.json \\\n    --t5_text_embeddings_path nvidia\u002FCosmos-Policy-LIBERO-Predict2-2B\u002Flibero_t5_embeddings.pkl \\\n    --trained_with_image_aug True \\\n    --chunk_size 16 \\\n    --num_open_loop_steps 16 \\\n    --task_suite_name libero_10 \\\n    --num_trials_per_task 1 \\\n    --local_log_dir cosmos_policy\u002Fexperiments\u002Frobot\u002Flibero\u002Flogs\u002F \\\n    --seed 195 \\\n    --randomize_seed False \\\n    --deterministic True \\\n    --run_id_note smoke \\\n    --ar_future_prediction False \\\n    --ar_value_prediction False \\\n    --use_jpeg_compression True \\\n    --flip_images True \\\n    --num_denoising_steps_action 5 \\\n    --num_denoising_steps_future_state 1 \\\n    --num_denoising_steps_value 1 \\\n    --data_collection False\n```\n\n## Core concepts\n\n**What Cosmos Policy is**: NVIDIA Cosmos Policy is a vision-language-action (VLA) model that uses Cosmos Tokenizer to encode visual observations into discrete tokens, then predicts robot actions conditioned on language instructions and visual context.\n\n**Key architecture choices**:\n\n| Component | Design |\n|-----------|--------|\n| Visual encoder | Cosmos Tokenizer (discrete tokens) |\n| Language conditioning | Cross-attention to language embeddings |\n| Action prediction | Autoregressive action token generation |\n\n**Public command surface**: The supported evaluation entrypoints are `cosmos_policy.experiments.robot.libero.run_libero_eval` and `cosmos_policy.experiments.robot.robocasa.run_robocasa_eval`. Keep reproduction notes anchored to these public modules and their documented flags.\n\n## Compute requirements\n\n| Task | GPU | VRAM | Typical wall time |\n|------|-----|------|-------------------|\n| LIBERO smoke eval (1 trial) | 1x A40\u002FA100 | ~16 GB | 5-10 min |\n| LIBERO full eval (50 trials) | 1x A40\u002FA100 | ~16 GB | 2-4 hours |\n| RoboCasa single-task (2 trials) | 1x A40\u002FA100 | ~18 GB | 10-15 min |\n| RoboCasa all-tasks | 1x A40\u002FA100 | ~18 GB | 4-8 hours |\n\n## When to use vs alternatives\n\n**Use this skill when:**\n- Evaluating NVIDIA Cosmos Policy on LIBERO or RoboCasa benchmarks\n- Profiling inference latency and throughput for Cosmos Policy\n- Setting up headless EGL rendering for robot simulation on GPU clusters\n\n**Use alternatives when:**\n- Training or fine-tuning Cosmos Policy from scratch (use official Cosmos training docs)\n- Working with OpenVLA-based policies (use `fine-tuning-openvla-oft`)\n- Working with Physical Intelligence pi0 models (use `fine-tuning-serving-openpi`)\n- Running real-robot evaluation rather than simulation\n\n---\n\n## Workflow 1: LIBERO evaluation\n\nCopy this checklist and track progress:\n\n```text\nLIBERO Eval Progress:\n- [ ] Step 1: Install environment and dependencies\n- [ ] Step 2: Configure headless EGL rendering\n- [ ] Step 3: Run smoke evaluation\n- [ ] Step 4: Validate outputs and parse results\n- [ ] Step 5: Run full benchmark if smoke passes\n```\n\n**Step 1: Install environment**\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FNVlabs\u002Fcosmos-policy.git\ncd cosmos-policy\n# Follow SETUP.md to build and enter the supported Docker container.\n# Then, inside the container:\nuv sync --extra cu128 --group libero --python 3.10\n```\n\n**Step 2: Configure headless rendering**\n\n```bash\nexport CUDA_VISIBLE_DEVICES=0\nexport MUJOCO_EGL_DEVICE_ID=0\nexport MUJOCO_GL=egl\nexport PYOPENGL_PLATFORM=egl\n```\n\n**Step 3: Run smoke evaluation**\n\n```bash\nuv run --extra cu128 --group libero --python 3.10 \\\n  python -m cosmos_policy.experiments.robot.libero.run_libero_eval \\\n    --config cosmos_predict2_2b_480p_libero__inference_only \\\n    --ckpt_path nvidia\u002FCosmos-Policy-LIBERO-Predict2-2B \\\n    --config_file cosmos_policy\u002Fconfig\u002Fconfig.py \\\n    --use_wrist_image True \\\n    --use_proprio True \\\n    --normalize_proprio True \\\n    --unnormalize_actions True \\\n    --dataset_stats_path nvidia\u002FCosmos-Policy-LIBERO-Predict2-2B\u002Flibero_dataset_statistics.json \\\n    --t5_text_embeddings_path nvidia\u002FCosmos-Policy-LIBERO-Predict2-2B\u002Flibero_t5_embeddings.pkl \\\n    --trained_with_image_aug True \\\n    --chunk_size 16 \\\n    --num_open_loop_steps 16 \\\n    --task_suite_name libero_10 \\\n    --num_trials_per_task 1 \\\n    --local_log_dir cosmos_policy\u002Fexperiments\u002Frobot\u002Flibero\u002Flogs\u002F \\\n    --seed 195 \\\n    --randomize_seed False \\\n    --deterministic True \\\n    --run_id_note smoke \\\n    --ar_future_prediction False \\\n    --ar_value_prediction False \\\n    --use_jpeg_compression True \\\n    --flip_images True \\\n    --num_denoising_steps_action 5 \\\n    --num_denoising_steps_future_state 1 \\\n    --num_denoising_steps_value 1 \\\n    --data_collection False\n```\n\n**Step 4: Validate and parse results**\n\n```python\nimport json\nimport glob\n\n# Find latest evaluation result from the official log directory\nlog_files = sorted(glob.glob(\"cosmos_policy\u002Fexperiments\u002Frobot\u002Flibero\u002Flogs\u002F**\u002F*.json\", recursive=True))\nwith open(log_files[-1]) as f:\n    results = json.load(f)\n\nprint(results)\n```\n\n**Step 5: Scale up**\n\nRun across all four LIBERO task suites with 50 trials:\n\n```bash\nfor suite in libero_spatial libero_object libero_goal libero_10; do\n  uv run --extra cu128 --group libero --python 3.10 \\\n    python -m cosmos_policy.experiments.robot.libero.run_libero_eval \\\n      --config cosmos_predict2_2b_480p_libero__inference_only \\\n      --ckpt_path nvidia\u002FCosmos-Policy-LIBERO-Predict2-2B \\\n      --config_file cosmos_policy\u002Fconfig\u002Fconfig.py \\\n      --use_wrist_image True \\\n      --use_proprio True \\\n      --normalize_proprio True \\\n      --unnormalize_actions True \\\n      --dataset_stats_path nvidia\u002FCosmos-Policy-LIBERO-Predict2-2B\u002Flibero_dataset_statistics.json \\\n      --t5_text_embeddings_path nvidia\u002FCosmos-Policy-LIBERO-Predict2-2B\u002Flibero_t5_embeddings.pkl \\\n      --trained_with_image_aug True \\\n      --chunk_size 16 \\\n      --num_open_loop_steps 16 \\\n      --task_suite_name \"$suite\" \\\n      --num_trials_per_task 50 \\\n      --local_log_dir cosmos_policy\u002Fexperiments\u002Frobot\u002Flibero\u002Flogs\u002F \\\n      --seed 195 \\\n      --randomize_seed False \\\n      --deterministic True \\\n      --run_id_note \"suite_${suite}\" \\\n      --ar_future_prediction False \\\n      --ar_value_prediction False \\\n      --use_jpeg_compression True \\\n      --flip_images True \\\n      --num_denoising_steps_action 5 \\\n      --num_denoising_steps_future_state 1 \\\n      --num_denoising_steps_value 1 \\\n      --data_collection False\ndone\n```\n\n---\n\n## Workflow 2: RoboCasa evaluation\n\nCopy this checklist and track progress:\n\n```text\nRoboCasa Eval Progress:\n- [ ] Step 1: Install RoboCasa assets and verify macros\n- [ ] Step 2: Run single-task smoke evaluation\n- [ ] Step 3: Validate outputs\n- [ ] Step 4: Expand to multi-task runs\n```\n\n**Step 1: Install RoboCasa**\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002Fmoojink\u002Frobocasa-cosmos-policy.git\nuv pip install -e robocasa-cosmos-policy\npython -m robocasa.scripts.setup_macros\npython -m robocasa.scripts.download_kitchen_assets\n```\n\nThis fork installs the `robocasa` Python package expected by Cosmos Policy while preserving the patched environment changes used in the public RoboCasa eval path. Verify `macros_private.py` exists and paths are correct.\n\n**Step 2: Single-task smoke evaluation**\n\n```bash\nuv run --extra cu128 --group robocasa --python 3.10 \\\n  python -m cosmos_policy.experiments.robot.robocasa.run_robocasa_eval \\\n    --config cosmos_predict2_2b_480p_robocasa_50_demos_per_task__inference \\\n    --ckpt_path nvidia\u002FCosmos-Policy-RoboCasa-Predict2-2B \\\n    --config_file cosmos_policy\u002Fconfig\u002Fconfig.py \\\n    --use_wrist_image True \\\n    --num_wrist_images 1 \\\n    --use_proprio True \\\n    --normalize_proprio True \\\n    --unnormalize_actions True \\\n    --dataset_stats_path nvidia\u002FCosmos-Policy-RoboCasa-Predict2-2B\u002Frobocasa_dataset_statistics.json \\\n    --t5_text_embeddings_path nvidia\u002FCosmos-Policy-RoboCasa-Predict2-2B\u002Frobocasa_t5_embeddings.pkl \\\n    --trained_with_image_aug True \\\n    --chunk_size 32 \\\n    --num_open_loop_steps 16 \\\n    --task_name TurnOffMicrowave \\\n    --obj_instance_split A \\\n    --num_trials_per_task 2 \\\n    --local_log_dir cosmos_policy\u002Fexperiments\u002Frobot\u002Frobocasa\u002Flogs\u002F \\\n    --seed 195 \\\n    --randomize_seed False \\\n    --deterministic True \\\n    --run_id_note smoke \\\n    --use_variance_scale False \\\n    --use_jpeg_compression True \\\n    --flip_images True \\\n    --num_denoising_steps_action 5 \\\n    --num_denoising_steps_future_state 1 \\\n    --num_denoising_steps_value 1 \\\n    --data_collection False\n```\n\n**Step 3: Validate outputs**\n\n- Confirm the eval log prints the expected task name, object split, and checkpoint\u002Fconfig values.\n- Inspect the final `Success rate:` line in the log.\n\n**Step 4: Expand scope**\n\nIncrease `--num_trials_per_task` or add more tasks. Keep `--obj_instance_split` fixed across repeated runs for comparability.\n\n---\n\n## Workflow 3: Blank-machine cluster launch\n\n```text\nCluster Launch Progress:\n- [ ] Step 1: Clone the public repo and enter the supported runtime\n- [ ] Step 2: Sync the benchmark-specific dependency group\n- [ ] Step 3: Export rendering and cache environment variables before eval\n```\n\n**Step 1: Clone and enter the supported runtime**\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FNVlabs\u002Fcosmos-policy.git\ncd cosmos-policy\n# Follow SETUP.md, start the Docker container, and enter it before continuing.\n```\n\n**Step 2: Sync dependencies**\n\n```bash\nuv sync --extra cu128 --group libero --python 3.10\n# or, for RoboCasa:\nuv sync --extra cu128 --group robocasa --python 3.10\n# then install the Cosmos-compatible RoboCasa fork:\ngit clone https:\u002F\u002Fgithub.com\u002Fmoojink\u002Frobocasa-cosmos-policy.git\nuv pip install -e robocasa-cosmos-policy\n```\n\n**Step 3: Export runtime environment**\n\n```bash\nexport CUDA_VISIBLE_DEVICES=0\nexport MUJOCO_EGL_DEVICE_ID=0\nexport MUJOCO_GL=egl\nexport PYOPENGL_PLATFORM=egl\nexport HF_HOME=${HF_HOME:-$HOME\u002F.cache\u002Fhuggingface}\nexport TRANSFORMERS_CACHE=${TRANSFORMERS_CACHE:-$HF_HOME}\n```\n\n---\n\n## Expected performance benchmarks\n\nReference values from official evaluation (tied to specific setup and seeds):\n\n| Task Suite | Success Rate | Notes |\n|-----------|-------------|-------|\n| LIBERO-Spatial | 98.1% | Official LIBERO spatial result |\n| LIBERO-Object | 100.0% | Official LIBERO object result |\n| LIBERO-Goal | 98.2% | Official LIBERO goal result |\n| LIBERO-Long | 97.6% | Official LIBERO long-horizon result |\n| LIBERO-Average | 98.5% | Official average across LIBERO suites |\n| RoboCasa | 67.1% | Official RoboCasa average result |\n\n**Reproduction note**: Published success rates still depend on checkpoint choice, task suite, seeds, and simulator setup. Record the exact command and environment alongside any reported number.\n\n---\n\n## Non-negotiable rules\n\n- **EGL alignment**: Always set `CUDA_VISIBLE_DEVICES`, `MUJOCO_EGL_DEVICE_ID`, `MUJOCO_GL=egl`, and `PYOPENGL_PLATFORM=egl` together on headless GPU nodes.\n- **Official runtime first**: If host-Python installs hit binary compatibility issues, fall back to the supported container workflow from `SETUP.md` before debugging package internals.\n- **Cache consistency**: Use the same cache directory across setup and eval so Hugging Face and dependency caches are reused.\n- **Run comparability**: Keep task name, object split, seed, and trial count fixed across repeated runs.\n\n---\n\n## Common issues\n\n**Issue: binary compatibility or loader failures on host Python**\n\nFix: rerun inside the official container\u002Fruntime from `SETUP.md`. Do not assume host-package rebuilds will match the public release environment.\n\n**Issue: LIBERO prompts for config path in a non-interactive shell**\n\nFix: pre-create `LIBERO_CONFIG_PATH\u002Fconfig.yaml`:\n\n```python\nimport os, yaml\n\nconfig_dir = os.path.expanduser(\"~\u002F.libero\")\nos.makedirs(config_dir, exist_ok=True)\nwith open(os.path.join(config_dir, \"config.yaml\"), \"w\") as f:\n    yaml.dump({\"benchmark_root\": \"\u002Fpath\u002Fto\u002Flibero\u002Fdatasets\"}, f)\n```\n\n**Issue: EGL initialization or shutdown noise**\n\nFix: align EGL environment variables first. Treat teardown-only `EGL_NOT_INITIALIZED` warnings as low-signal unless the job exits non-zero.\n\n**Issue: Kitchen object sampling NaNs or asset lookup failures in RoboCasa**\n\nFix: rerun asset setup and confirm the patched robocasa install is intact:\n\n```bash\npython -m robocasa.scripts.download_kitchen_assets\npython -c \"import robocasa; print(robocasa.__file__)\"\n```\n\n**Issue: MuJoCo rendering mismatch**\n\nFix: verify GPU device alignment:\n\n```python\nimport os\ncuda_dev = os.environ.get(\"CUDA_VISIBLE_DEVICES\", \"not set\")\negl_dev = os.environ.get(\"MUJOCO_EGL_DEVICE_ID\", \"not set\")\nassert cuda_dev == egl_dev, f\"GPU mismatch: CUDA={cuda_dev}, EGL={egl_dev}\"\nprint(f\"Rendering on GPU {cuda_dev}\")\n```\n\n---\n\n## Advanced topics\n\n**LIBERO command matrix**: See [references\u002Flibero-commands.md](references\u002Flibero-commands.md)\n**RoboCasa command matrix**: See [references\u002Frobocasa-commands.md](references\u002Frobocasa-commands.md)\n\n## Resources\n\n- Cosmos Policy repository: https:\u002F\u002Fgithub.com\u002FNVlabs\u002Fcosmos-policy\n- LIBERO benchmark: https:\u002F\u002Fgithub.com\u002FLifelong-Robot-Learning\u002FLIBERO\n- Cosmos-compatible RoboCasa fork: https:\u002F\u002Fgithub.com\u002Fmoojink\u002Frobocasa-cosmos-policy\n- Upstream RoboCasa project: https:\u002F\u002Fgithub.com\u002Frobocasa\u002Frobocasa\n- MuJoCo documentation: https:\u002F\u002Fmujoco.readthedocs.io\u002F\n\n## Agent operating procedure\n\n1. **Check the environment.** Confirm GPU memory, model checkpoint and licence, framework versions and input formats.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Run inference on a single sample at low resolution or short length first.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Inspect outputs qualitatively and with task metrics; check licence restrictions for generated content.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| Out-of-memory on large inputs | Reduce resolution or sequence length, use half precision, or tile the input. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Check model and dataset licences before commercial or published use.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `fine-tuning-serving-openpi`: Fine-tune and serve Physical Intelligence OpenPI models (pi0, pi0-fast, pi0.5) using JAX or PyTorch backends for robot policy inference acr…\n- `fine-tuning-openvla-oft`: Fine-tunes and evaluates OpenVLA-OFT and OpenVLA-OFT+ policies for robot action generation with continuous action heads, LoRA adaptation, a…\n- `phoenix-observability`: Open-source AI observability platform for LLM tracing, evaluation, and monitoring.\n",{"name":5,"license":15,"metadata":24,"description":7},{"tags":25,"version":26,"category":27,"maintainer":28,"dependencies":29},"Cosmos Policy, VLA, Robotics, LIBERO, RoboCasa, Simulation, Evaluation, Profiling, EGL Rendering","1.0.0","multimodal-and-emerging","Kalaris Labs","torch>=2.1.0, mujoco>=3.0.0, robosuite>=1.4.0, robocasa @ git+https:\u002F\u002Fgithub.com\u002Fmoojink\u002Frobocasa-cosmos-policy.git, transformers>=4.40.0, cosmos-policy @ git+https:\u002F\u002Fgithub.com\u002FNVlabs\u002Fcosmos-policy.git","main","29fe36d26127205057d5e0ed8425f6b0e9a8c13a","skills\u002Fevaluating-cosmos-policy",16,"https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills","https:\u002F\u002Fdocs.kalarislabs.com\u002Fresearch-agent-skills","https:\u002F\u002Fgithub.com\u002FKalarisLabs","kalarislabs.com","Applied AI lab building the infrastructure for the agentic era","2026-11-14 17:49:08",{"source":8,"skillName":5,"commands":41,"prompt":51,"zipAvailable":52},[42,45,48],{"tool":43,"command":44},"npx","npx skills add KalarisLabs\u002Fresearch-agent-skills --skill evaluating-cosmos-policy",{"tool":46,"command":47},"bunx","bunx skills add KalarisLabs\u002Fresearch-agent-skills --skill evaluating-cosmos-policy",{"tool":49,"command":50},"pnpm","pnpm dlx skills add KalarisLabs\u002Fresearch-agent-skills --skill evaluating-cosmos-policy","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fevaluating-cosmos-policy\u002FSKILL.md",true,[54,60],{"path":55,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":58,"sourceHtmlUrl":59},"references\u002Flibero-commands.md","REFERENCE","参考文档",6863,"https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fevaluating-cosmos-policy\u002Freferences\u002Flibero-commands.md",{"path":61,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":62,"sourceHtmlUrl":63},"references\u002Frobocasa-commands.md",5894,"https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fevaluating-cosmos-policy\u002Freferences\u002Frobocasa-commands.md",[65,72,79,86,93,100],{"skillNo":66,"name":67,"nameZh":6,"description":68,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":69,"categories":70,"keywords":71,"publishedAt":19},"01M74Z8MGT305R11SNY5G28QH5","plos","Prepare manuscripts for PLOS journals (PLOS ONE, PLOS Biology, PLOS Computational Biology, PLOS Genetics, PLOS Medicine, PLOS Pathogens, PLOS Neglected Tropical Diseases, PLOS Global Public Health and others), covering structure, abstract and author summary, Vancouver-style references, the mandatory data availability policy, ethics and financial disclosure statements, figure requirements (PACE), the PLOS LaTeX template, reporting guidelines and PLOS ONE's publication criteria. Use when targeting any PLOS journal or checking compliance with PLOS policies.","https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fplos\u002FSKILL.md",[],[],{"skillNo":73,"name":74,"nameZh":6,"description":75,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":76,"categories":77,"keywords":78,"publishedAt":19},"01M74Z8CCN9APXYHNQ95V1EBGT","pkpd-modeling","Pharmacokinetic and pharmacodynamic modelling and simulation - non-compartmental analysis, compartmental and population PK, PK\u002FPD and exposure-response, TMDD, PBPK orientation, bioequivalence, allometric scaling and first-in-human dose, drug interaction prediction, and Bayesian therapeutic drug monitoring. Use when analysing concentration-time data, deriving exposure metrics, fitting PK or PD models, or evaluating dosing regimens. Triggers include \"pharmacokinetics\", \"pharmacodynamics\", \"PK\u002FPD\", \"NCA\", \"non-compartmental\", \"AUC\", \"Cmax\", \"lambda z\", \"half-life\", \"clearance\", \"volume of distribution\", \"compartmental model\", \"population PK\", \"popPK\", \"NONMEM\", \"nlmixr2\", \"Pharmpy\", \"Monolix\", \"exposure-response\", \"Emax\", \"EC50\", \"indirect response\", \"effect compartment\", \"TMDD\", \"PBPK\", \"bioequivalence\", \"RSABE\", \"ABEL\", \"allometric scaling\", \"first-in-human\", \"MABEL\", \"drug-drug interaction\", \"DDI\", \"ICH M12\", \"concentration-QTc\", \"therapeutic drug monitoring\", \"MIPD\", and \"dosing regimen\".","https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fpkpd-modeling\u002FSKILL.md",[],[],{"skillNo":80,"name":81,"nameZh":6,"description":82,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":83,"categories":84,"keywords":85,"publishedAt":19},"01M74Z86YHJJ1FM3X87BHN76KZ","pinecone","Guides use of Pinecone, a managed serverless vector database, through its Python client and the LangChain and LlamaIndex integrations. Covers creating indexes, upserting and querying vectors, metadata filtering, namespaces, hybrid dense and sparse search, index management, and deleting vectors. Use when building a production RAG system on a hosted vector store, adding semantic search or recommendations without running infrastructure, isolating per-user or per-tenant data with namespaces, combining dense and sparse vectors in one query, or filtering results by metadata. Do not use for self-hosted or local stores (Chroma, Weaviate) or offline similarity search (FAISS).","https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fpinecone\u002FSKILL.md",[],[],{"skillNo":87,"name":88,"nameZh":6,"description":89,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":90,"categories":91,"keywords":92,"publishedAt":19},"01M74Z80T6E89SAJ8MYDD0767Q","pi-agent","Build with and use Pi, the minimal terminal coding harness. Use for installing Pi, configuring providers\u002Fmodels\u002Fsettings\u002Fenvironment variables, creating Pi skills\u002Fextensions\u002Fpackages\u002Fthemes\u002Fprompt templates, embedding Pi through the SDK, integrating over RPC or JSON event streams, parsing sessions, running local models through the llama.cpp router, developing custom Pi providers and TUI components, or using ecosystem packages such as pi-subagents (delegation\u002Forchestration), pi-mcp-adapter (MCP servers), pi-interview (interactive forms), and pi-web-access (web search, fetching, video understanding).","https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fpi-agent\u002FSKILL.md",[],[],{"skillNo":94,"name":95,"nameZh":6,"description":96,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":97,"categories":98,"keywords":99,"publishedAt":19},"01M74Z7WF1Y8Q3VFAQPPPRAFMV","phoenix-observability","Open-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.","https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fphoenix-observability\u002FSKILL.md",[],[],{"skillNo":101,"name":102,"nameZh":6,"description":103,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":104,"categories":105,"keywords":106,"publishedAt":19},"01M74Z7HNEH1A2AVYR3G1X2QV6","peft-fine-tuning","Fine-tunes LLMs with Hugging Face PEFT, using LoRA, QLoRA, IA3, AdaLoRA, prefix tuning, and prompt tuning so that under 1% of parameters are trained. Covers rank, alpha, and target module selection, loading and merging adapters, multi-adapter serving, and integration with TRL SFTTrainer, Axolotl, and vLLM. Use when fine-tuning 7B-70B models on limited GPU memory, when running QLoRA on a single 24GB GPU, when choosing LoRA rank and alpha, when merging or swapping adapters on one base model, or when debugging CUDA OOM or an adapter that does not apply. Not for full fine-tuning of small models under 1B parameters or cases needing all weights updated.","https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fpeft-fine-tuning\u002FSKILL.md",[],[]]