data-validation
Inspect class membership and ground-truth/detection-result file-set intersections for mAP-style inputs with safe report-first repair guidance.
- License 许可
- Apache-2.0
- 最近更新
- 2026-09-03
- Stars
- ★ 351
- Forks
- 27
01 前言 Front Matter
- name
- data-validation
- license
- Apache 2.0
- metadata
- { "disco-role": "operating" }
- description
- Inspect class membership and ground-truth/detection-result file-set intersections for mAP-style inputs with safe report-first repair guidance.
- disable-model-invocation
- true
02 资源文件 3
- 参考文档references/troubleshooting.md7.0 KB
- 参考文档references/workflows.md6.3 KB
- 脚本scripts/check_map_inputs.py14.4 KB
03 SKILL.md 正文
data-validation
Use this sub-skill when a future agent must decide whether mAP-style ground-truth and detection-result folders are ready for evaluation, find which files contain a class, or safely repair mismatched .txt file sets before running metrics.
Route here for
- Checking that
ground-truth/anddetection-results/contain top-level.txtfiles with matching filenames. - Diagnosing evaluator errors that imply a missing ground-truth or detection-result pair.
- Finding every GT and DR file whose first token on any annotation line equals a requested class name.
- Reporting GT-only and DR-only files before evaluation.
- Moving non-intersecting files into an explicit backup folder after a report-only dry run and explicit confirmation.
Do not handle here
- Computing AP, mAP, IoU matching, ignored classes, custom IoU thresholds, plots, animations, or evaluator output files: use
../evaluation/SKILL.md. - Converting PASCAL VOC XML, YOLO, darkflow JSON, darknet result text, or keras-yolo3 annotations into mAP text files: use
../conversion/SKILL.md. - Deep validation of bounding-box coordinates, confidence values, optional
difficultbehavior, or metric semantics beyond the file-set and first-token class checks owned here.
Evidence-backed assumptions
- The evaluator expects one ground-truth
.txtfile and one detection-result.txtfile per image, with matching filenames such asimage_1.txton both sides. - Ground-truth lines start with the class name and then contain box coordinates plus optional
difficult. - Detection-result lines start with the class name, then confidence and box coordinates.
- The legacy class finder searched both folders for a class by comparing the first whitespace-separated token of each line.
- The legacy intersection helper moved non-intersecting files immediately; this generated skill uses a bundled report-first helper instead.
First checks
-
Identify the actual GT and DR folders the user wants evaluated. Do not assume implicit repository-relative folders; pass explicit paths to the helper.
-
Run a non-mutating readiness report:
python sub-skills/data-validation/scripts/check_map_inputs.py \ --gt-dir /path/to/ground-truth \ --dr-dir /path/to/detection-results -
If the user asks about a class, add
--class-name CLASSand read both GT and DR results. A class absent from both folders usually means the class label is misspelled, conversion mapped it differently, or the wrong folders were selected. -
If the report shows GT-only or DR-only files, inspect the listed basenames before moving anything. Same counts can still be invalid when basenames differ.
-
Repair only when the user accepts moving the non-intersecting
.txtfiles out of evaluator inputs. Use a backup root outside the GT/DR folders and require explicit confirmation:python sub-skills/data-validation/scripts/check_map_inputs.py \ --gt-dir /path/to/ground-truth \ --dr-dir /path/to/detection-results \ --move-extra-to /path/to/map-input-backup \ --confirm-move -
Rerun the report after any repair. Route to evaluation only when the helper reports matching file sets and non-empty intersections.
Reference map
- Read
references/workflows.mdfor step-by-step class lookup, file-set readiness, dry-run interpretation, and explicit backup repair workflows. - Read
references/troubleshooting.mdfor empty folders, same-count/different-basename failures, non-intersecting sets, missing classes, unsafe legacy moves, wrong filenames, and backup conflicts. - Run
scripts/check_map_inputs.py --helpwhen you need exact helper options and exit-code behavior.
Safety rules
- The bundled helper is report-only by default. Supplying
--move-extra-towithout--confirm-moveprints a plan but still does not move files. - Never delete GT or DR files as a repair step. The only bundled repair moves non-intersecting
.txtfiles to a named backup root. - Do not move files when the GT/DR intersection is empty; that usually means wrong paths, wrong extensions, or incompatible naming rather than a safe partial mismatch.
- Keep conversion fixes in the conversion sub-skill and metric/debug interpretation in the evaluation sub-skill so this sub-skill stays focused on class lookup and file-set intersection readiness.
04 同仓库更多 Skills
passnet-triton-opt
Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.
passnet-solve
END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.
passnet-skill
PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.
passnet-pattern-fusion
Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.
passnet-orchestrate
Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.
passnet-feedback
Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).