AGENT SKILL

pipelines-and-benchmarks

Use for Anomalib benchmark pipelines, tiled ensemble workflows, and advanced pipeline orchestration helpers while keeping experimental execution paths explicit.

License 许可
Apache-2.0
最近更新
2026-09-03
Stars
★ 351
Forks
27

01 前言 Front Matter

name
pipelines-and-benchmarks
license
Apache 2.0
metadata
{ "disco-role": "operating" }
description
Use for Anomalib benchmark pipelines, tiled ensemble workflows, and advanced pipeline orchestration helpers while keeping experimental execution paths explicit.
disable-model-invocation
true

02 资源文件 4

03 SKILL.md 正文

Pipelines and Benchmarks

Use this sub-skill when the user asks about Anomalib benchmark configs, pipeline runners/jobs/generators, tiled ensemble training or evaluation, or safe preflight checks for pipeline configuration.

Read first

Safe operating boundary

Prefer read-only parsing and config repair before execution. Pipeline execution can train models, load datasets, spawn processes, create results directories, and run for a long time. Treat these as expensive unless the user explicitly provides dataset paths, runtime budget, and backend/hardware intent.

Route elsewhere when the request is primarily about:

  • model or datamodule selection details beyond the pipeline config shell;
  • core Engine.fit, Engine.test, metrics, post-processing, loggers, callbacks, or visualization semantics;
  • export formats, inferencers, OpenVINO/Torch deployment, or Studio application code.

Quick routing

User need Do this
“How do I benchmark several models/categories?” Explain the benchmark YAML shape, grid keys, and CPU/CUDA runner selection from pipelines-and-benchmarks.
“Switch a benchmark from CUDA to CPU without losing the grid.” Change only the top-level accelerator to cpu or [cpu]; keep the benchmark tree and all grid leaves unchanged. Run the smoke helper before execution.
“Create a custom pipeline/job/runner.” Explain the Pipeline → Runner → JobGenerator → Job contract; keep custom code independent of execution strategy.
“Use tiled ensemble on high-resolution images.” Read the tiled ensemble reference, warn that it is experimental, validate the config, and require a results-root decision before evaluation.
“Tiled eval cannot find checkpoints/results.” Check whether EvalTiledEnsemble(root_dir=...) points at the exact versioned run directory containing the ensemble weights and stats, not the parent default results directory.
“Run the MEBin post-processing benchmark.” Treat it as benchmark-scale and reference-only by default; require explicit dataset path, output path, model/category budget, and runtime approval.

Minimal preflight commands

From a project where Anomalib is importable:

python sub-skills/pipelines-and-benchmarks/scripts/pipeline_config_smoke.py --import-only
python sub-skills/pipelines-and-benchmarks/scripts/pipeline_config_smoke.py --benchmark-config benchmark.yaml
python sub-skills/pipelines-and-benchmarks/scripts/pipeline_config_smoke.py --tiled-config ensemble.yaml --eval-root results/Padim/MVTecAD/bottle/v0

The helper only imports public entrypoints and validates config shape/paths. It does not call Pipeline.run, train models, evaluate models, or create result directories.

04 同仓库更多 Skills

VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-triton-opt

Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-solve

END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-skill

PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-pattern-fusion

Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-orchestrate

Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-feedback

Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).

★ 351