pytorch
Operate transformer_engine.pytorch for PyTorch layers, precision control, distributed overlap, and debug workflows.
- License 许可
- Apache-2.0
- 最近更新
- 2026-09-03
- Stars
- ★ 351
- Forks
- 27
01 前言 Front Matter
- name
- pytorch
- license
- Apache 2.0
- metadata
- { "disco-role": "operating" }
- description
- Operate transformer_engine.pytorch for PyTorch layers, precision control, distributed overlap, and debug workflows.
- disable-model-invocation
- true
02 资源文件 5
- 参考文档references/api-reference.md12.2 KB
- 参考文档references/distributed-and-advanced.md6.5 KB
- 参考文档references/troubleshooting.md4.0 KB
- 参考文档references/workflows.md4.1 KB
- 脚本scripts/pytorch_bf16_smoke.py4.6 KB
03 SKILL.md 正文
PyTorch
Use this sub-skill when the task is to apply Transformer Engine's PyTorch surface inside a model, training loop, export path, or debugging session.
Route here for
- Replacing
torch.nnlayers withtransformer_engine.pytorch.Linear,GroupedLinear,LayerNorm,RMSNorm,LayerNormLinear,LayerNormMLP,DotProductAttention,MultiheadAttention, orTransformerLayer. - BF16/FP16 training or inference with matching
params_dtypeandtorch.autocast. - FP8, MXFP8, or NVFP4 questions, recipe selection, and hardware gating with
te.autocast(...). - Quantized parameter initialization, quantized tensor inspection, and quantizer choices.
- Distributed execution, checkpointing, userbuffers, comm/GEMM overlap, CUDA graphs, CPU offload, or op-fuser work.
- Debugging with Nvidia-DL-Framework-Inspect and Transformer Engine debug features.
- Import, version, backend, and precision-availability troubleshooting.
Required references
Read these bundled references before changing code or explaining usage:
references/api-reference.mdfor constructors, helpers, tensor classes, and op-fuser surfaces.references/workflows.mdfor minimal BF16/FP16 flows and guarded low-precision recipes.references/distributed-and-advanced.mdfor FSDP, checkpointing, userbuffers, CPU offload, export, and advanced fusion.references/troubleshooting.mdfor import order, runtime mismatches, and precision failures.
Smoke helper
Run the bundled smoke script to validate a small BF16 te.Linear path without depending on the source repository:
python scripts/pytorch_bf16_smoke.py --help
python scripts/pytorch_bf16_smoke.py --device cuda --in-features 16 --out-features 32 --batch-size 4
The helper prints version, device, and precision-availability facts, then runs a tiny BF16 forward/backward pass. It is intentionally narrower than the full TransformerLayer examples.
Operating cautions
- Use
torch.autocastfor BF16/FP16 compute control; usete.autocastfor FP8/MXFP8/NVFP4 quantized compute. They are not interchangeable. - Do not promise FP8 on A100-class hardware. Always check availability first and surface the reason string when a recipe is unavailable.
- When
quantized_model_init(enabled=True)is used with master-weight optimizers, preserve or copy high-precision init values before clearing them if you need FP32 seeding. - Prefer the bundled smoke path for environment checks when larger TransformerLayer examples are unstable.
04 同仓库更多 Skills
passnet-triton-opt
Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.
passnet-solve
END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.
passnet-skill
PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.
passnet-pattern-fusion
Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.
passnet-orchestrate
Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.
passnet-feedback
Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).