training-evaluation
Construct and troubleshoot MMDetection3D training, testing, evaluation, visualization-hook, TTA, distributed, and Slurm commands.
- License 许可
- Apache-2.0
- 最近更新
- 2026-09-03
- Stars
- ★ 351
- Forks
- 27
01 前言 Front Matter
- name
- training-evaluation
- license
- Apache 2.0
- metadata
- { "disco-role": "operating" }
- description
- Construct and troubleshoot MMDetection3D training, testing, evaluation, visualization-hook, TTA, distributed, and Slurm commands.
- disable-model-invocation
- true
02 资源文件 4
- 参考文档references/evaluation.md6.2 KB
- 参考文档references/troubleshooting.md6.7 KB
- 参考文档references/workflows.md6.3 KB
- 脚本scripts/build_train_test_command.py12.8 KB
03 SKILL.md 正文
MMDetection3D training-evaluation sub-skill
Use this sub-skill when the user needs to train, test, evaluate, visualize evaluation outputs, enable test-time augmentation, or launch distributed/Slurm MMDetection3D jobs from an already chosen config and prepared dataset.
Best-fit tasks
- Build safe single-process, distributed, or Slurm training commands for
tools/train.pysemantics. - Build safe single-process, distributed, or Slurm test/evaluation commands for
tools/test.pysemantics. - Decide how to pass
--work-dir,--amp,--sync_bn,--auto-scale-lr,--resume, and nested--cfg-optionsoverrides. - Add evaluation output prefixes for validation artifacts or benchmark submissions.
- Add visualization-hook flags (
--show,--show-dir,--task, score threshold, wait time) without assuming a GUI is available. - Add
--ttaonly when the selected segmentation config contains the required TTA sections. - Diagnose launch, evaluator, visualization, TTA, CPU/GPU, port, and Slurm failures.
Route elsewhere
- Dataset download, info-file generation, dataset layout validation, and stale annotation migrations belong in
data-preparation. - Choosing model families, editing config inheritance, changing dataset roots/classes, and validating config syntax belong in
configuration-model-zoo. - Standalone inference demos, inferencer APIs, and prediction serialization without evaluation belong in
inference. - Geometry API use and offline interpretation of saved OBJ/visualizer artifacts belong in
structures-visualization. - TorchServe, model publishing/conversion, log plotting, FLOPs, and benchmark helper boundaries belong in
serving-tools.
Operating procedure
-
Confirm the caller has a config path, dataset prepared for that config, and for testing a checkpoint path. If these are missing, route before generating commands.
-
Choose the launcher pattern in references/workflows.md: single process, distributed shell launcher, or Slurm launcher.
-
Generate command strings with the bundled dry-run helper:
python scripts/build_train_test_command.py --helpThe helper prints shell commands only; it never starts training, testing, distributed workers, or Slurm jobs.
-
For training, decide whether the job needs
--work-dir,--amp,--sync_bn,--auto-scale-lr,--resume, and config overrides before adding launch resources. -
For testing/evaluation, decide whether the task is ordinary metric evaluation, benchmark formatting, saved visualization, or segmentation TTA. Use references/evaluation.md for evaluator-specific output keys.
-
For distributed or Slurm jobs, set GPU count and ports explicitly when multiple jobs may share a host. Avoid the default port collision described in references/troubleshooting.md.
-
Before recommending an expensive run, check the high-signal guardrails below and the failure patterns in references/troubleshooting.md.
High-signal guardrails
- Do not run long training/evaluation as a verification step. Prefer parser/help checks, config checks, and command rendering unless the user explicitly authorizes the expensive job.
--work-diroverrides configwork_dir; if neither is set, MMEngine uses./work_dirs/<config-basename>.--amponly converts a config whose optimizer wrapper type isOptimWrapper; configs already using AMP warn, and unrelated wrappers fail.--auto-scale-lrrequires the config to defineauto_scale_lr.enableandauto_scale_lr.base_batch_size.--resumewith no value means auto-resume from the latest checkpoint in the work directory;--resume path/to/checkpoint.pthresumes from that checkpoint.--sync_bnacceptsnone,torch, ormmcv; it is mainly useful for multi-GPU training and should not be used as a generic CPU fix.- Evaluation output prefixes are usually nested under
test_evaluator.*; do not pass a top-levelsubmission_prefixunless the active config explicitly consumes it. --show/--show-diractivate the visualization hook and require a valid--taskchoice. Saved visualization works headlessly through output files, but documented use is for single-GPU debugging.--ttain this release is for 3D segmentation configs that include bothtta_modelandtta_pipeline; do not enable it for ordinary 3D detection configs.- CPU train/test is experimental and narrow. Most point-cloud models depend on CUDA 3D ops; CPU testing is documented only for SMOKE-style debugging.
04 同仓库更多 Skills
passnet-triton-opt
Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.
passnet-solve
END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.
passnet-skill
PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.
passnet-pattern-fusion
Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.
passnet-orchestrate
Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.
passnet-feedback
Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).