structured-label-issues
Route cleanlab token classification, object detection, and semantic segmentation label-issue workflows.
- License 许可
- Apache-2.0
- 最近更新
- 2026-09-03
- Stars
- ★ 351
- Forks
- 27
01 前言 Front Matter
- name
- structured-label-issues
- license
- Apache 2.0
- metadata
- { "disco-role": "operating", "root-skill": "cleanlab" }
- description
- Route cleanlab token classification, object detection, and semantic segmentation label-issue workflows.
- disable-model-invocation
- true
02 资源文件 7
- 参考文档references/api-reference.md7.7 KB
- 参考文档references/data-formats.md5.1 KB
- 参考文档references/object-detection.md6.3 KB
- 参考文档references/segmentation.md5.3 KB
- 参考文档references/token-classification.md4.9 KB
- 参考文档references/troubleshooting.md6.7 KB
- 脚本scripts/smoke_structured_label_issues.py11.0 KB
03 SKILL.md 正文
Structured Label Issues
Use this sub-skill for cleanlab workflows where each dataset example has structured labels rather than one scalar label:
- Token classification: each sentence/document is a list of tokens with one class label and one probability vector per token.
- Object detection: each image has annotated bounding boxes/classes and detector predictions grouped by class.
- Semantic segmentation: each image has a pixel mask and per-pixel class probabilities.
Keep this file as a router. Read the bundled references for schemas, API calls, and troubleshooting before writing code.
Route quickly
| User asks for | Route here? | First reference |
|---|---|---|
| Named-entity recognition, POS tagging, or other per-token labels | Yes | references/token-classification.md |
| Finding/ranking/displaying token label issues | Yes | references/token-classification.md |
| Object-detection label issues, bounding-box ranking, or box visualization | Yes | references/object-detection.md |
| Semantic-segmentation mask label issues or pixel/image ranking | Yes | references/segmentation.md |
| Exact input schemas for all three families | Yes | references/data-formats.md |
| Function signatures and return types | Yes | references/api-reference.md |
| Shape/schema/backend failures | Yes | references/troubleshooting.md |
Route elsewhere
- If the user asks about span classification specifically, route to the sibling
experimentalsub-skill (../experimental/). Span classification is intentionally not owned here. - If the user wants a broad audit across one dataset with many issue types or a single audit/report wrapper, route to the sibling
datalabsub-skill (../datalab/). Use this sub-skill only when the user wants the direct task-specific token/object/segmentation APIs. - For standard multiclass/binary classification, use
../classification/SKILL.md. - For multilabel classification or regression labels, use
../tabular-label-issues/SKILL.md. - For multi-annotator consensus/quality, use
../multiannotator/SKILL.md. - For out-of-distribution or outlier scoring, use
../outlier/SKILL.md.
Operating workflow
- Identify the structured-output family: token classification, object detection, or semantic segmentation.
- Validate data format first. Most failures are nested-list length mismatches, class-order mismatches, wrong bounding-box schema, or mask/probability shape errors.
- Prefer out-of-sample model predictions/probabilities when available. Cleanlab scores are most meaningful when predictions were not obtained from a model fit on the same labels without cross-validation/holdout safeguards.
- Use the family-specific
filter.find_label_issuesfunction when the user wants estimated issues; userank.get_label_quality_scoresplusissues_from_scoreswhen the user wants ranking/thresholding control. - Use summary/visualization helpers only after confirming optional plotting dependencies are available or setting a non-interactive matplotlib backend for scripts and CI.
- Interpret outputs at the right level: token tuples for token classification, image-level masks/indices plus per-box helper scores for object detection, and pixel masks plus image-level summaries for segmentation.
Bundled helper
Run scripts/smoke_structured_label_issues.py to exercise deterministic tiny fixtures for all three families. It is distilled from the public tutorials and native tests rather than copied from a source script: tutorial notebooks require external downloads/interactive visualization, while native pytest files are test harnesses rather than safe reusable runtime helpers.
04 同仓库更多 Skills
passnet-triton-opt
Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.
passnet-solve
END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.
passnet-skill
PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.
passnet-pattern-fusion
Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.
passnet-orchestrate
Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.
passnet-feedback
Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).