AGENT SKILL

pytorch

Operate transformer_engine.pytorch for PyTorch layers, precision control, distributed overlap, and debug workflows.

License 许可
Apache-2.0
最近更新
2026-09-03
Stars
★ 351
Forks
27

01 前言 Front Matter

name
pytorch
license
Apache 2.0
metadata
{ "disco-role": "operating" }
description
Operate transformer_engine.pytorch for PyTorch layers, precision control, distributed overlap, and debug workflows.
disable-model-invocation
true

02 资源文件 5

03 SKILL.md 正文

PyTorch

Use this sub-skill when the task is to apply Transformer Engine's PyTorch surface inside a model, training loop, export path, or debugging session.

Route here for

  • Replacing torch.nn layers with transformer_engine.pytorch.Linear, GroupedLinear, LayerNorm, RMSNorm, LayerNormLinear, LayerNormMLP, DotProductAttention, MultiheadAttention, or TransformerLayer.
  • BF16/FP16 training or inference with matching params_dtype and torch.autocast.
  • FP8, MXFP8, or NVFP4 questions, recipe selection, and hardware gating with te.autocast(...).
  • Quantized parameter initialization, quantized tensor inspection, and quantizer choices.
  • Distributed execution, checkpointing, userbuffers, comm/GEMM overlap, CUDA graphs, CPU offload, or op-fuser work.
  • Debugging with Nvidia-DL-Framework-Inspect and Transformer Engine debug features.
  • Import, version, backend, and precision-availability troubleshooting.

Required references

Read these bundled references before changing code or explaining usage:

  1. references/api-reference.md for constructors, helpers, tensor classes, and op-fuser surfaces.
  2. references/workflows.md for minimal BF16/FP16 flows and guarded low-precision recipes.
  3. references/distributed-and-advanced.md for FSDP, checkpointing, userbuffers, CPU offload, export, and advanced fusion.
  4. references/troubleshooting.md for import order, runtime mismatches, and precision failures.

Smoke helper

Run the bundled smoke script to validate a small BF16 te.Linear path without depending on the source repository:

python scripts/pytorch_bf16_smoke.py --help
python scripts/pytorch_bf16_smoke.py --device cuda --in-features 16 --out-features 32 --batch-size 4

The helper prints version, device, and precision-availability facts, then runs a tiny BF16 forward/backward pass. It is intentionally narrower than the full TransformerLayer examples.

Operating cautions

  • Use torch.autocast for BF16/FP16 compute control; use te.autocast for FP8/MXFP8/NVFP4 quantized compute. They are not interchangeable.
  • Do not promise FP8 on A100-class hardware. Always check availability first and surface the reason string when a recipe is unavailable.
  • When quantized_model_init(enabled=True) is used with master-weight optimizers, preserve or copy high-precision init values before clearing them if you need FP32 seeding.
  • Prefer the bundled smoke path for environment checks when larger TransformerLayer examples are unstable.

04 同仓库更多 Skills

VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-triton-opt

Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-solve

END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-skill

PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-pattern-fusion

Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-orchestrate

Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-feedback

Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).

★ 351