[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M78AMSD8NFQH8TQQDCSAKDQY":3,"skill-same:01M78AMSD8NFQH8TQQDCSAKDQY":91},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":34,"commitSha":35,"skillDirectory":36,"forksCount":14,"repositoryHtmlUrl":37,"repositoryHomepageUrl":38,"ownerHtmlUrl":39,"ownerBlogUrl":40,"ownerDescription":41,"updatedAt":42,"install":43,"resources":56},"01M78AMSD8NFQH8TQQDCSAKDQY","tao-analyze-gaps-visual-changenet",null,"Performs gap analysis on NVIDIA TAO VCN Classify (Visual Component Net) experiments by invoking the data-services container (`tao_toolkit.data_services` from `versions.yaml`) directly via `docker run … gap_analysis vcn_aoi …` — picks the optimal decision threshold, ranks per-sample weakness, and emits a top-K weakest parquet expanded per-lighting for downstream augmentation. Use when analyzing VCN classification failures, picking SDA augmentation targets, or auditing PASS\u002FNO_PASS boundary cases.","Solizardking\u002Fskillhub-main","Solizardking","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F108096810?v=4","01M786KDB10AYV042N0F9BD5B5","Clawd","01M786NYYXSZK1D05WM8JQPC48",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-analyze-gaps-visual-changenet\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: tao-analyze-gaps-visual-changenet\ndescription: Performs gap analysis on NVIDIA TAO VCN Classify (Visual Component Net) experiments by invoking the data-services container (`tao_toolkit.data_services` from `versions.yaml`) directly via `docker run … gap_analysis vcn_aoi …` — picks the optimal decision threshold, ranks per-sample weakness, and emits a top-K weakest parquet expanded per-lighting for downstream augmentation. Use when analyzing VCN classification failures, picking SDA augmentation targets, or auditing PASS\u002FNO_PASS boundary cases.\nlicense: Apache-2.0\ncompatibility: Requires docker + nvidia-container-toolkit and a CUDA GPU. Pulls the `tao_toolkit.data_services` image declared in `versions.yaml` at the skill bank root.\nmetadata:\n  author: NVIDIA Corporation\n  version: \"0.1.0\"\nallowed-tools: Read Bash\ntags:\n- data\n- rca\n- vcn\n- aoi\n---\n\n# TAO VCN Classify Gap Analysis Skill\n\nYou are an analyst for NVIDIA TAO VCN Classify (Visual Component Net) inference results. Your job is to identify the **weakest samples per ground-truth label** by measuring signed distance from the decision threshold *in the wrong direction*, then surface them for downstream augmentation or relabeling.\n\nThis skill is intentionally lightweight. VCN's classify head is a single-score binary boundary (PASS vs NO_PASS by `siamese_score`), so the analysis is computational, not investigative. The whole computation lives behind one direct `docker run` invocation against the `tao_toolkit.data_services` image declared in `versions.yaml` (resolved at runtime — see Setup). The container's entrypoint takes `\u003Ccategory> \u003Caction> [hydra overrides...]`; we pass `gap_analysis vcn_aoi key=value …`. Each override is a bare Hydra `key=value` that selectively overrides the script's `GapAnalysisConfig` schema (defaults are baked into the container; introspect with `docker run ... gap_analysis vcn_aoi --cfg=job`). (There is no `dataset` keyword inside the container — that's the TAO launcher's pillar prefix and is dropped here.) You do **not** need delegated analysis, multi-phase image audits, or component-type clustering — VCN does not expose those dimensions. View only a small set of representative weak samples to qualify the gaps after the container returns.\n\nCLI surface can shift between data-services container builds. If a `gap_analysis vcn_aoi` invocation fails on argument parsing, introspect the actual schema once per image with `docker run --rm \"$DS_IMAGE\" gap_analysis vcn_aoi --cfg=job` and reconcile any renamed keys (e.g. `inference_csv` vs `inference_results_dir`, `output_dir` vs `results_dir`) before retrying. Output parquet name is `kpi_gaps.parquet`.\n\n---\n\n## Inputs\n\n1. **Experiment result directory** — contains `inference\u002Finference.csv` from TAO VCN Classify inference. Required columns: `input_path`, `object_name`, `label`, `siamese_score`. Pass the **directory** (e.g. `inference\u002Flatest\u002F`), not the CSV file — the container reads `inference_results_dir\u002Finference.csv`.\n2. **Training code\u002Fconfig directory** — contains the VCN train YAML. The container reads `dataset.classify.input_map` (lighting condition list) and `dataset.classify.image_ext` from it to expand each weak sample into one row per lighting.\n3. **Dataset directory** — image root prepended to the relative `input_path` from each row (`kpi_media_path`).\n4. **Schema overrides** — `min_recall`, `top_k_per_label`, and optionally a hard-pinned `threshold` are passed as Hydra overrides (defaults: `min_recall=1.0`, `top_k_per_label=50`, `threshold=-1.0` meaning sweep). **`top_k_per_label` must be a positive integer** — omitting it flips the container into \"below-threshold filter\" mode, which at `min_recall=1.0` returns only PASS misclassifications and zero NO_PASS rows. See Common pitfalls.\n\n---\n\n## Setup\n\nThe threshold sweep, weakness ranking, and per-lighting expansion all run inside the `tao_toolkit.data_services` image declared in `versions.yaml`. Resolve the concrete URI once at the top of the run, then confirm Docker, the NVIDIA container toolkit, and a GPU are present and ensure the image is cached:\n\n```bash\n# Resolve tao_toolkit.data_services → concrete nvcr.io\u002F... URI from versions.yaml\nDS_IMAGE=$(python3 -c \"import yaml,os; print(yaml.safe_load(open(os.environ['TAO_SKILL_BANK_PATH']+'\u002Fversions.yaml'))['images']['tao_toolkit']['data_services'])\")\necho \"DS_IMAGE=$DS_IMAGE\"\n\ndocker info > \u002Fdev\u002Fnull && echo \"OK: docker\"\nnvidia-smi > \u002Fdev\u002Fnull && echo \"OK: GPU\"\ndocker image inspect \"$DS_IMAGE\" > \u002Fdev\u002Fnull \\\n  || docker pull \"$DS_IMAGE\"\n```\n\n`TAO_SKILL_BANK_PATH` is usually exported by the installed skill bank. If it is unset, point it at the skill-bank repo root before resolving. A GPU is required; aborting early on a GPU-less host saves a confusing late error.\n\nThree setup rules are load-bearing and easy to get wrong:\n\n- **Path mounting** — every host path the container reads or writes (`inference.csv`, train YAML, dataset image root, output dir) must be bind-mounted, simplest with `-v $WORKSPACE:$WORKSPACE -w $WORKSPACE` so absolute paths resolve identically on both sides.\n- **Do not pass `--user $(id -u):$(id -g)`** — it triggers `KeyError: 'getpwuid(): uid not found: \u003Cuid>'` during the container's `transformers` import; `chown` outputs back to the host UID afterwards instead.\n- **`-e \u003Cspec>` is required, not optional** — current images hard-require it and exit with `ValueError: The subtask vcn_aoi requires the following argument: -e\u002F--experiment_spec_file` before parsing CLI overrides.\n\nSee `references\u002Fcontainer-setup.md` for the full path-mounting pattern, the `--user`\u002F`chown` rationale and `alpine` chown command, multi-`-v` guidance, and the `-e \u003Cspec>` requirement detail.\n\n---\n\n## Method\n\nThe whole skill is a single `docker run` invocation followed by a small visual spot-check. The container does Steps 1–4 internally (threshold sweep, weakness scoring, top-K selection, per-lighting expansion). You handle Step 5 (visual spot-check) directly with the Read tool.\n\n### Step 1–4 — Run the container\n\n```bash\n$DOCKER gap_analysis vcn_aoi \\\n    inference_results_dir=\u003Cexp_dir>\u002Finference\u002F\u003Clabel>\u002F \\\n    train_config=\u003Cexp_dir>\u002Ftrain.yaml \\\n    kpi_media_path=\u003Cdataset_root> \\\n    results_dir=\u003Crca_results_dir> \\\n    top_k_per_label=50\n```\n\n> **Always pass `top_k_per_label`.** This is the argument that switches the container\n> from the default \"samples below threshold\" filter into proper top-K-per-label\n> ranking. At `min_recall=1.0` the threshold is by construction at-or-below every\n> NO_PASS score, so the below-threshold filter returns ONLY misclassified PASS rows\n> and zero NO_PASS rows — useless as an augmentation queue. With `top_k_per_label`\n> set to a positive integer (either in the spec or as a Hydra override), the\n> container computes signed weakness against the threshold for every row and\n> surfaces the K weakest **per ground-truth label**, which is the per-label ranked\n> output downstream steps consume.\n\nReads `inference.csv`, sweeps every unique `siamese_score` plus one value just below the minimum, keeps the candidates with NO_PASS-class recall ≥ `min_recall` (with `1e-12` tolerance), then picks the threshold with the best F1 (tie-break: precision, then threshold value). For every row, computes signed weakness from that threshold (positive = misclassified, negative = correct, magnitude = margin). Sorts by weakness descending and takes the top `top_k_per_label` per ground-truth label, then expands each weak row into one row per lighting condition using `dataset.classify.input_map` and `dataset.classify.image_ext` from the train YAML.\n\nIf **no** candidate threshold meets the recall target, the container exits non-zero and writes `unreachable_kpi.txt` into `results_dir` explaining which recall the model can actually achieve. In that case, stop the analysis after the docker call, write a one-section report explaining the model fundamentally cannot reach the KPI at any operating point, and recommend retraining or relabeling — skip the visual spot-check.\n\n**Container writes into `results_dir`:**\n\n| Artifact | Contents |\n|----------|----------|\n| `kpi_gaps.parquet` | Top-K weakest per label, expanded per lighting. Columns: `filepath`, `label`, `siamese_score`, `weakness`. |\n| `threshold.txt` | Chosen decision threshold (single float, plain text). |\n| `metrics.json` | At the chosen threshold: `precision`, `recall`, `f1`, confusion matrix `{tp, fp, tn, fn}`, plus per-label `{total, mean_weakness, median_weakness, max_weakness, n_misclassified}`. |\n| `weak_samples_breakdown.txt` | Per-label kept-row breakdown: `\u003Ccount>` total, `\u003C%>` of all kept rows, `N` misclassified (weakness > 0), `N` marginal (weakness ≤ 0). |\n| `unreachable_kpi.txt` | Only written when the recall target is unreachable. Presence of this file means: skip Step 5, write the abridged report, recommend retrain. |\n\nPrint the container's stdout summary (chosen threshold, kept-row counts, per-label breakdown) to your own stdout so the script-check hook can verify the run produced output.\n\n### Step 5 — Visual spot check (small, fixed)\n\nSkip this step if `unreachable_kpi.txt` exists. Otherwise use the Read tool to **view** the 5 weakest PASS samples and the 5 weakest NO_PASS samples from `kpi_gaps.parquet` (deduplicated to one row per sample, using the FIRST-lighting `filepath`), classify each as exactly one of **mislabeled** \u002F **edge case** \u002F **data quality** \u002F **systematic**, and copy each viewed image (resized to 128×128 if PIL is available, otherwise just copy) into `\u003Cresults_dir>\u002Frca_images\u002F`. This is the only image inspection required — do not view dozens of images, run failure mode clustering, or audit goldens (VCN has no golden images).\n\nSee `references\u002Fvisual-spot-check.md` for the exact sample-selection sort, the per-lighting deduplication rule, the full definition of each verdict category, and the image-copy detail.\n\n---\n\n## Reference invocation\n\nPaste-and-edit the workspace, the four paths, and the two numeric knobs; this runs end-to-end. Capture stdout so the script-check hook sees row counts.\n\n```bash\nWORKSPACE=\u003Cabsolute path>            # mounted identically inside the container\nEXP_DIR=\u003Cexperiment_result_dir>      # contains inference\u002Finference.csv and train.yaml; must be inside $WORKSPACE\nDATASET_ROOT=\u003Cdataset_root>          # image root for inference.csv input_path entries; must be inside $WORKSPACE\nMIN_RECALL=1.0                       # zero-miss default; lower if KPI relaxes\nTOP_K=50                             # per-label augmentation budget\nOUT=\"$EXP_DIR\u002Frca_results\u002F$(date +%Y-%m-%d_%H%M%S)\"\nSPEC=\"$OUT\u002Fvcn_aoi_spec.yaml\"\nIMG=$(python3 -c \"import yaml,os; print(yaml.safe_load(open(os.environ['TAO_SKILL_BANK_PATH']+'\u002Fversions.yaml'))['images']['tao_toolkit']['data_services'])\")\n\nmkdir -p \"$OUT\"\n\n# Write the gap-analysis spec for this run\ncat > \"$SPEC\" \u003C\u003CEOF\nmin_recall: $MIN_RECALL\ntop_k_per_label: $TOP_K\nEOF\n\ndocker run --gpus all --rm --ipc=host \\\n    -v \"$WORKSPACE:$WORKSPACE\" -w \"$WORKSPACE\" \\\n    \"$IMG\" gap_analysis vcn_aoi \\\n    -e \"$SPEC\" \\\n    inference_results_dir=\"$EXP_DIR\u002Finference\u002Flatest\u002F\" \\\n    train_config=\"$EXP_DIR\u002Ftrain.yaml\" \\\n    kpi_media_path=\"$DATASET_ROOT\" \\\n    results_dir=\"$OUT\"\n\n# Container writes as root with --user dropped; chown back to host UID if needed.\ndocker run --rm -v \"$WORKSPACE:\u002Fw\" alpine chown -R \"$(id -u):$(id -g)\" \"\u002Fw\u002F$(realpath --relative-to=\"$WORKSPACE\" \"$OUT\")\"\n\n# Sanity print so the script-check hook sees real numbers\npython3 - \"$OUT\" \u003C\u003C 'PYEOF'\nimport json, os, sys\nout = sys.argv[1]\nunreachable = os.path.join(out, \"unreachable_kpi.txt\")\nif os.path.isfile(unreachable):\n    print(\"KPI UNREACHABLE — see\", unreachable)\n    sys.exit(0)\nwith open(os.path.join(out, \"threshold.txt\")) as f:\n    print(\"threshold:\", f.read().strip())\nwith open(os.path.join(out, \"metrics.json\")) as f:\n    m = json.load(f)\nprint(f\"precision={m['precision']:.4f} recall={m['recall']:.4f} f1={m['f1']:.4f}\")\nimport pandas as pd\ndf = pd.read_parquet(os.path.join(out, \"kpi_gaps.parquet\"))\nprint(f\"kpi_gaps.parquet: rows={len(df)}, cols={list(df.columns)}\")\nprint(df['label'].value_counts())\nPYEOF\n```\n\n---\n\n## Outputs\n\nWrite everything into a timestamped folder under the experiment result directory. The container's outputs go straight there; the visual spot-check writes `rca_images\u002F`; any runtime packaging hook may add session\u002Fconfig capture artifacts after `RCA_Report.md` is written.\n\n```\n\u003Cexperiment_result_dir>\u002Frca_results\u002FYYYY-MM-DD_HHMMSS\u002F\n├── RCA_Report.md              # Full gap analysis report (you write this)\n├── kpi_gaps.parquet           # Container: top-K weakest per label, expanded per lighting\n├── threshold.txt              # Container: chosen decision threshold (single float)\n├── metrics.json               # Container: confusion matrix + per-label distribution stats\n├── weak_samples_breakdown.txt # Container: per-label count\u002Fmisclassified\u002Fmarginal counts\n├── unreachable_kpi.txt        # Container: ONLY when no threshold meets min_recall\n├── rca_images\u002F                # You: thumbnails of the 10 viewed weak samples\n├── rca_config\u002F                # Auto-copied by hook\n└── session log\u002Fartifacts      # Optional, runtime-dependent packaging capture\n```\n\nAt the start of the run, get the real timestamp by running `date +%Y-%m-%d_%H%M%S` in Bash. Do NOT hardcode or guess. If the user specifies a custom output path, use that instead but maintain the same internal structure.\n\n---\n\n## Common pitfalls\n\nThe single most consequential failure mode is **forgetting `top_k_per_label` when `min_recall=1.0`**: at that recall the chosen threshold sits at or below every NO_PASS score, so without `top_k_per_label` the container falls back to a \"samples below threshold\" filter that returns ONLY misclassified PASS rows and zero NO_PASS rows, breaking the augmentation queue. Always include an explicit positive `top_k_per_label` (default 50) in the spec or as a Hydra override.\n\nSee `references\u002Fpitfalls.md` for the complete checklist, covering: forgetting `top_k_per_label`; passing `--user`; calling with only Hydra overrides (no `-e \u003Cspec>`); spec file outside `$WORKSPACE`; spec file with unresolved `???` sentinels; image not pulled \u002F wrong tag; path-mount mismatch; `unreachable_kpi.txt` written; `inference.csv` missing required columns; train YAML missing `dataset.classify.input_map` or `image_ext`; `kpi_media_path` not matching `input_path` prefixes; and no GPU detected from inside the container.\n\n---\n\n## Report Structure\n\nWrite `RCA_Report.md` as a tight (1000–1800 word) computational gap analysis — depth comes from accurate numbers and a clear action list, not narrative. The full report template (7 sections: Verdict, Threshold Selection, Weakness Distribution, Top-K Weakest Samples, Visual Spot Check, Per-Label Breakdown, Recommended Actions — with the confusion-matrix and table layouts) is in `references\u002Foutput-template.md`. When `unreachable_kpi.txt` exists, replace sections 3–6 with a single short section quoting that file's contents and collapse section 7 to one recommendation: retrain or relabel.\n\n---\n\n## Execution Order\n\n1. Resolve `DS_IMAGE` from `versions.yaml` (`images.tao_toolkit.data_services`), then run `docker info`, `nvidia-smi`, and `docker image inspect \"$DS_IMAGE\"` (pulling if missing) once to confirm the environment. Abort with a clear message if any fail.\n2. Run `date +%Y-%m-%d_%H%M%S` to get the timestamp; create `\u003Cexperiment_result_dir>\u002Frca_results\u002F\u003Ctimestamp>\u002F`.\n3. Write `vcn_aoi_spec.yaml` into the timestamped dir with `min_recall` and `top_k_per_label` filled in. Keep it under `$WORKSPACE` so the `-e` path resolves inside the container.\n4. Run `docker run … \"$DS_IMAGE\" gap_analysis vcn_aoi -e vcn_aoi_spec.yaml inference_results_dir=… train_config=… kpi_media_path=… output_dir=…`. The container writes `kpi_gaps.parquet`, `threshold.txt`, `metrics.json`, `weak_samples_breakdown.txt` into `results_dir`. Print the chosen threshold and kept-row counts to stdout so the script-check hook can verify the run produced output.\n5. If `unreachable_kpi.txt` exists, skip Step 6 and write the abridged report. Otherwise continue.\n6. Pick 10 weak samples (5 weakest PASS + 5 weakest NO_PASS) from `kpi_gaps.parquet`, view each test image with Read, classify, and copy each into `rca_images\u002F`.\n7. Write `RCA_Report.md` last — writing it triggers the packaging hook, which copies session logs and skill config alongside.\n","# TAO VCN Classify Gap Analysis Skill\n\nYou are an analyst for NVIDIA TAO VCN Classify (Visual Component Net) inference results. Your job is to identify the **weakest samples per ground-truth label** by measuring signed distance from the decision threshold *in the wrong direction*, then surface them for downstream augmentation or relabeling.\n\nThis skill is intentionally lightweight. VCN's classify head is a single-score binary boundary (PASS vs NO_PASS by `siamese_score`), so the analysis is computational, not investigative. The whole computation lives behind one direct `docker run` invocation against the `tao_toolkit.data_services` image declared in `versions.yaml` (resolved at runtime — see Setup). The container's entrypoint takes `\u003Ccategory> \u003Caction> [hydra overrides...]`; we pass `gap_analysis vcn_aoi key=value …`. Each override is a bare Hydra `key=value` that selectively overrides the script's `GapAnalysisConfig` schema (defaults are baked into the container; introspect with `docker run ... gap_analysis vcn_aoi --cfg=job`). (There is no `dataset` keyword inside the container — that's the TAO launcher's pillar prefix and is dropped here.) You do **not** need delegated analysis, multi-phase image audits, or component-type clustering — VCN does not expose those dimensions. View only a small set of representative weak samples to qualify the gaps after the container returns.\n\nCLI surface can shift between data-services container builds. If a `gap_analysis vcn_aoi` invocation fails on argument parsing, introspect the actual schema once per image with `docker run --rm \"$DS_IMAGE\" gap_analysis vcn_aoi --cfg=job` and reconcile any renamed keys (e.g. `inference_csv` vs `inference_results_dir`, `output_dir` vs `results_dir`) before retrying. Output parquet name is `kpi_gaps.parquet`.\n\n---\n\n## Inputs\n\n1. **Experiment result directory** — contains `inference\u002Finference.csv` from TAO VCN Classify inference. Required columns: `input_path`, `object_name`, `label`, `siamese_score`. Pass the **directory** (e.g. `inference\u002Flatest\u002F`), not the CSV file — the container reads `inference_results_dir\u002Finference.csv`.\n2. **Training code\u002Fconfig directory** — contains the VCN train YAML. The container reads `dataset.classify.input_map` (lighting condition list) and `dataset.classify.image_ext` from it to expand each weak sample into one row per lighting.\n3. **Dataset directory** — image root prepended to the relative `input_path` from each row (`kpi_media_path`).\n4. **Schema overrides** — `min_recall`, `top_k_per_label`, and optionally a hard-pinned `threshold` are passed as Hydra overrides (defaults: `min_recall=1.0`, `top_k_per_label=50`, `threshold=-1.0` meaning sweep). **`top_k_per_label` must be a positive integer** — omitting it flips the container into \"below-threshold filter\" mode, which at `min_recall=1.0` returns only PASS misclassifications and zero NO_PASS rows. See Common pitfalls.\n\n---\n\n## Setup\n\nThe threshold sweep, weakness ranking, and per-lighting expansion all run inside the `tao_toolkit.data_services` image declared in `versions.yaml`. Resolve the concrete URI once at the top of the run, then confirm Docker, the NVIDIA container toolkit, and a GPU are present and ensure the image is cached:\n\n```bash\n# Resolve tao_toolkit.data_services → concrete nvcr.io\u002F... URI from versions.yaml\nDS_IMAGE=$(python3 -c \"import yaml,os; print(yaml.safe_load(open(os.environ['TAO_SKILL_BANK_PATH']+'\u002Fversions.yaml'))['images']['tao_toolkit']['data_services'])\")\necho \"DS_IMAGE=$DS_IMAGE\"\n\ndocker info > \u002Fdev\u002Fnull && echo \"OK: docker\"\nnvidia-smi > \u002Fdev\u002Fnull && echo \"OK: GPU\"\ndocker image inspect \"$DS_IMAGE\" > \u002Fdev\u002Fnull \\\n  || docker pull \"$DS_IMAGE\"\n```\n\n`TAO_SKILL_BANK_PATH` is usually exported by the installed skill bank. If it is unset, point it at the skill-bank repo root before resolving. A GPU is required; aborting early on a GPU-less host saves a confusing late error.\n\nThree setup rules are load-bearing and easy to get wrong:\n\n- **Path mounting** — every host path the container reads or writes (`inference.csv`, train YAML, dataset image root, output dir) must be bind-mounted, simplest with `-v $WORKSPACE:$WORKSPACE -w $WORKSPACE` so absolute paths resolve identically on both sides.\n- **Do not pass `--user $(id -u):$(id -g)`** — it triggers `KeyError: 'getpwuid(): uid not found: \u003Cuid>'` during the container's `transformers` import; `chown` outputs back to the host UID afterwards instead.\n- **`-e \u003Cspec>` is required, not optional** — current images hard-require it and exit with `ValueError: The subtask vcn_aoi requires the following argument: -e\u002F--experiment_spec_file` before parsing CLI overrides.\n\nSee `references\u002Fcontainer-setup.md` for the full path-mounting pattern, the `--user`\u002F`chown` rationale and `alpine` chown command, multi-`-v` guidance, and the `-e \u003Cspec>` requirement detail.\n\n---\n\n## Method\n\nThe whole skill is a single `docker run` invocation followed by a small visual spot-check. The container does Steps 1–4 internally (threshold sweep, weakness scoring, top-K selection, per-lighting expansion). You handle Step 5 (visual spot-check) directly with the Read tool.\n\n### Step 1–4 — Run the container\n\n```bash\n$DOCKER gap_analysis vcn_aoi \\\n    inference_results_dir=\u003Cexp_dir>\u002Finference\u002F\u003Clabel>\u002F \\\n    train_config=\u003Cexp_dir>\u002Ftrain.yaml \\\n    kpi_media_path=\u003Cdataset_root> \\\n    results_dir=\u003Crca_results_dir> \\\n    top_k_per_label=50\n```\n\n> **Always pass `top_k_per_label`.** This is the argument that switches the container\n> from the default \"samples below threshold\" filter into proper top-K-per-label\n> ranking. At `min_recall=1.0` the threshold is by construction at-or-below every\n> NO_PASS score, so the below-threshold filter returns ONLY misclassified PASS rows\n> and zero NO_PASS rows — useless as an augmentation queue. With `top_k_per_label`\n> set to a positive integer (either in the spec or as a Hydra override), the\n> container computes signed weakness against the threshold for every row and\n> surfaces the K weakest **per ground-truth label**, which is the per-label ranked\n> output downstream steps consume.\n\nReads `inference.csv`, sweeps every unique `siamese_score` plus one value just below the minimum, keeps the candidates with NO_PASS-class recall ≥ `min_recall` (with `1e-12` tolerance), then picks the threshold with the best F1 (tie-break: precision, then threshold value). For every row, computes signed weakness from that threshold (positive = misclassified, negative = correct, magnitude = margin). Sorts by weakness descending and takes the top `top_k_per_label` per ground-truth label, then expands each weak row into one row per lighting condition using `dataset.classify.input_map` and `dataset.classify.image_ext` from the train YAML.\n\nIf **no** candidate threshold meets the recall target, the container exits non-zero and writes `unreachable_kpi.txt` into `results_dir` explaining which recall the model can actually achieve. In that case, stop the analysis after the docker call, write a one-section report explaining the model fundamentally cannot reach the KPI at any operating point, and recommend retraining or relabeling — skip the visual spot-check.\n\n**Container writes into `results_dir`:**\n\n| Artifact | Contents |\n|----------|----------|\n| `kpi_gaps.parquet` | Top-K weakest per label, expanded per lighting. Columns: `filepath`, `label`, `siamese_score`, `weakness`. |\n| `threshold.txt` | Chosen decision threshold (single float, plain text). |\n| `metrics.json` | At the chosen threshold: `precision`, `recall`, `f1`, confusion matrix `{tp, fp, tn, fn}`, plus per-label `{total, mean_weakness, median_weakness, max_weakness, n_misclassified}`. |\n| `weak_samples_breakdown.txt` | Per-label kept-row breakdown: `\u003Ccount>` total, `\u003C%>` of all kept rows, `N` misclassified (weakness > 0), `N` marginal (weakness ≤ 0). |\n| `unreachable_kpi.txt` | Only written when the recall target is unreachable. Presence of this file means: skip Step 5, write the abridged report, recommend retrain. |\n\nPrint the container's stdout summary (chosen threshold, kept-row counts, per-label breakdown) to your own stdout so the script-check hook can verify the run produced output.\n\n### Step 5 — Visual spot check (small, fixed)\n\nSkip this step if `unreachable_kpi.txt` exists. Otherwise use the Read tool to **view** the 5 weakest PASS samples and the 5 weakest NO_PASS samples from `kpi_gaps.parquet` (deduplicated to one row per sample, using the FIRST-lighting `filepath`), classify each as exactly one of **mislabeled** \u002F **edge case** \u002F **data quality** \u002F **systematic**, and copy each viewed image (resized to 128×128 if PIL is available, otherwise just copy) into `\u003Cresults_dir>\u002Frca_images\u002F`. This is the only image inspection required — do not view dozens of images, run failure mode clustering, or audit goldens (VCN has no golden images).\n\nSee `references\u002Fvisual-spot-check.md` for the exact sample-selection sort, the per-lighting deduplication rule, the full definition of each verdict category, and the image-copy detail.\n\n---\n\n## Reference invocation\n\nPaste-and-edit the workspace, the four paths, and the two numeric knobs; this runs end-to-end. Capture stdout so the script-check hook sees row counts.\n\n```bash\nWORKSPACE=\u003Cabsolute path>            # mounted identically inside the container\nEXP_DIR=\u003Cexperiment_result_dir>      # contains inference\u002Finference.csv and train.yaml; must be inside $WORKSPACE\nDATASET_ROOT=\u003Cdataset_root>          # image root for inference.csv input_path entries; must be inside $WORKSPACE\nMIN_RECALL=1.0                       # zero-miss default; lower if KPI relaxes\nTOP_K=50                             # per-label augmentation budget\nOUT=\"$EXP_DIR\u002Frca_results\u002F$(date +%Y-%m-%d_%H%M%S)\"\nSPEC=\"$OUT\u002Fvcn_aoi_spec.yaml\"\nIMG=$(python3 -c \"import yaml,os; print(yaml.safe_load(open(os.environ['TAO_SKILL_BANK_PATH']+'\u002Fversions.yaml'))['images']['tao_toolkit']['data_services'])\")\n\nmkdir -p \"$OUT\"\n\n# Write the gap-analysis spec for this run\ncat > \"$SPEC\" \u003C\u003CEOF\nmin_recall: $MIN_RECALL\ntop_k_per_label: $TOP_K\nEOF\n\ndocker run --gpus all --rm --ipc=host \\\n    -v \"$WORKSPACE:$WORKSPACE\" -w \"$WORKSPACE\" \\\n    \"$IMG\" gap_analysis vcn_aoi \\\n    -e \"$SPEC\" \\\n    inference_results_dir=\"$EXP_DIR\u002Finference\u002Flatest\u002F\" \\\n    train_config=\"$EXP_DIR\u002Ftrain.yaml\" \\\n    kpi_media_path=\"$DATASET_ROOT\" \\\n    results_dir=\"$OUT\"\n\n# Container writes as root with --user dropped; chown back to host UID if needed.\ndocker run --rm -v \"$WORKSPACE:\u002Fw\" alpine chown -R \"$(id -u):$(id -g)\" \"\u002Fw\u002F$(realpath --relative-to=\"$WORKSPACE\" \"$OUT\")\"\n\n# Sanity print so the script-check hook sees real numbers\npython3 - \"$OUT\" \u003C\u003C 'PYEOF'\nimport json, os, sys\nout = sys.argv[1]\nunreachable = os.path.join(out, \"unreachable_kpi.txt\")\nif os.path.isfile(unreachable):\n    print(\"KPI UNREACHABLE — see\", unreachable)\n    sys.exit(0)\nwith open(os.path.join(out, \"threshold.txt\")) as f:\n    print(\"threshold:\", f.read().strip())\nwith open(os.path.join(out, \"metrics.json\")) as f:\n    m = json.load(f)\nprint(f\"precision={m['precision']:.4f} recall={m['recall']:.4f} f1={m['f1']:.4f}\")\nimport pandas as pd\ndf = pd.read_parquet(os.path.join(out, \"kpi_gaps.parquet\"))\nprint(f\"kpi_gaps.parquet: rows={len(df)}, cols={list(df.columns)}\")\nprint(df['label'].value_counts())\nPYEOF\n```\n\n---\n\n## Outputs\n\nWrite everything into a timestamped folder under the experiment result directory. The container's outputs go straight there; the visual spot-check writes `rca_images\u002F`; any runtime packaging hook may add session\u002Fconfig capture artifacts after `RCA_Report.md` is written.\n\n```\n\u003Cexperiment_result_dir>\u002Frca_results\u002FYYYY-MM-DD_HHMMSS\u002F\n├── RCA_Report.md              # Full gap analysis report (you write this)\n├── kpi_gaps.parquet           # Container: top-K weakest per label, expanded per lighting\n├── threshold.txt              # Container: chosen decision threshold (single float)\n├── metrics.json               # Container: confusion matrix + per-label distribution stats\n├── weak_samples_breakdown.txt # Container: per-label count\u002Fmisclassified\u002Fmarginal counts\n├── unreachable_kpi.txt        # Container: ONLY when no threshold meets min_recall\n├── rca_images\u002F                # You: thumbnails of the 10 viewed weak samples\n├── rca_config\u002F                # Auto-copied by hook\n└── session log\u002Fartifacts      # Optional, runtime-dependent packaging capture\n```\n\nAt the start of the run, get the real timestamp by running `date +%Y-%m-%d_%H%M%S` in Bash. Do NOT hardcode or guess. If the user specifies a custom output path, use that instead but maintain the same internal structure.\n\n---\n\n## Common pitfalls\n\nThe single most consequential failure mode is **forgetting `top_k_per_label` when `min_recall=1.0`**: at that recall the chosen threshold sits at or below every NO_PASS score, so without `top_k_per_label` the container falls back to a \"samples below threshold\" filter that returns ONLY misclassified PASS rows and zero NO_PASS rows, breaking the augmentation queue. Always include an explicit positive `top_k_per_label` (default 50) in the spec or as a Hydra override.\n\nSee `references\u002Fpitfalls.md` for the complete checklist, covering: forgetting `top_k_per_label`; passing `--user`; calling with only Hydra overrides (no `-e \u003Cspec>`); spec file outside `$WORKSPACE`; spec file with unresolved `???` sentinels; image not pulled \u002F wrong tag; path-mount mismatch; `unreachable_kpi.txt` written; `inference.csv` missing required columns; train YAML missing `dataset.classify.input_map` or `image_ext`; `kpi_media_path` not matching `input_path` prefixes; and no GPU detected from inside the container.\n\n---\n\n## Report Structure\n\nWrite `RCA_Report.md` as a tight (1000–1800 word) computational gap analysis — depth comes from accurate numbers and a clear action list, not narrative. The full report template (7 sections: Verdict, Threshold Selection, Weakness Distribution, Top-K Weakest Samples, Visual Spot Check, Per-Label Breakdown, Recommended Actions — with the confusion-matrix and table layouts) is in `references\u002Foutput-template.md`. When `unreachable_kpi.txt` exists, replace sections 3–6 with a single short section quoting that file's contents and collapse section 7 to one recommendation: retrain or relabel.\n\n---\n\n## Execution Order\n\n1. Resolve `DS_IMAGE` from `versions.yaml` (`images.tao_toolkit.data_services`), then run `docker info`, `nvidia-smi`, and `docker image inspect \"$DS_IMAGE\"` (pulling if missing) once to confirm the environment. Abort with a clear message if any fail.\n2. Run `date +%Y-%m-%d_%H%M%S` to get the timestamp; create `\u003Cexperiment_result_dir>\u002Frca_results\u002F\u003Ctimestamp>\u002F`.\n3. Write `vcn_aoi_spec.yaml` into the timestamped dir with `min_recall` and `top_k_per_label` filled in. Keep it under `$WORKSPACE` so the `-e` path resolves inside the container.\n4. Run `docker run … \"$DS_IMAGE\" gap_analysis vcn_aoi -e vcn_aoi_spec.yaml inference_results_dir=… train_config=… kpi_media_path=… output_dir=…`. The container writes `kpi_gaps.parquet`, `threshold.txt`, `metrics.json`, `weak_samples_breakdown.txt` into `results_dir`. Print the chosen threshold and kept-row counts to stdout so the script-check hook can verify the run produced output.\n5. If `unreachable_kpi.txt` exists, skip Step 6 and write the abridged report. Otherwise continue.\n6. Pick 10 weak samples (5 weakest PASS + 5 weakest NO_PASS) from `kpi_gaps.parquet`, view each test image with Read, classify, and copy each into `rca_images\u002F`.\n7. Write `RCA_Report.md` last — writing it triggers the packaging hook, which copies session logs and skill config alongside.\n",{"name":5,"tags":24,"license":15,"metadata":29,"description":7,"allowed-tools":32,"compatibility":33},[25,26,27,28],"data","rca","vcn","aoi",{"author":30,"version":31},"NVIDIA Corporation","0.1.0","Read Bash","Requires docker + nvidia-container-toolkit and a CUDA GPU. Pulls the `tao_toolkit.data_services` image declared in `versions.yaml` at the skill bank root.","main","be3de0244dc1a57e323d8160787ebb40fb64930e","public\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-analyze-gaps-visual-changenet","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main","https:\u002F\u002Fskills.x402.wtf","https:\u002F\u002Fgithub.com\u002FSolizardking","https:\u002F\u002Fmusebook.trade","Got a mile high and never came down","2026-10-08 23:20:13",{"source":8,"skillName":5,"commands":44,"prompt":54,"zipAvailable":55},[45,48,51],{"tool":46,"command":47},"npx","npx skills add Solizardking\u002Fskillhub-main --skill tao-analyze-gaps-visual-changenet",{"tool":49,"command":50},"bunx","bunx skills add Solizardking\u002Fskillhub-main --skill tao-analyze-gaps-visual-changenet",{"tool":52,"command":53},"pnpm","pnpm dlx skills add Solizardking\u002Fskillhub-main --skill tao-analyze-gaps-visual-changenet","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-analyze-gaps-visual-changenet\u002FSKILL.md",true,[57,63,67,71,75,79,83,87],{"path":58,"resourceType":59,"resourceTypeDesc":60,"sizeBytes":61,"sourceHtmlUrl":62},"BENCHMARK.md","REFERENCE","参考文档",4151,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-analyze-gaps-visual-changenet\u002FBENCHMARK.md",{"path":64,"resourceType":59,"resourceTypeDesc":60,"sizeBytes":65,"sourceHtmlUrl":66},"metadata.json",755,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-analyze-gaps-visual-changenet\u002Fmetadata.json",{"path":68,"resourceType":59,"resourceTypeDesc":60,"sizeBytes":69,"sourceHtmlUrl":70},"references\u002Fcontainer-setup.md",3084,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-analyze-gaps-visual-changenet\u002Freferences\u002Fcontainer-setup.md",{"path":72,"resourceType":59,"resourceTypeDesc":60,"sizeBytes":73,"sourceHtmlUrl":74},"references\u002Foutput-template.md",3308,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-analyze-gaps-visual-changenet\u002Freferences\u002Foutput-template.md",{"path":76,"resourceType":59,"resourceTypeDesc":60,"sizeBytes":77,"sourceHtmlUrl":78},"references\u002Fpitfalls.md",3943,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-analyze-gaps-visual-changenet\u002Freferences\u002Fpitfalls.md",{"path":80,"resourceType":59,"resourceTypeDesc":60,"sizeBytes":81,"sourceHtmlUrl":82},"references\u002Fvisual-spot-check.md",1602,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-analyze-gaps-visual-changenet\u002Freferences\u002Fvisual-spot-check.md",{"path":84,"resourceType":59,"resourceTypeDesc":60,"sizeBytes":85,"sourceHtmlUrl":86},"skill-card.md",4046,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-analyze-gaps-visual-changenet\u002Fskill-card.md",{"path":88,"resourceType":59,"resourceTypeDesc":60,"sizeBytes":89,"sourceHtmlUrl":90},"verification.json",2436,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-analyze-gaps-visual-changenet\u002Fverification.json",[92,100,107,114,121,129],{"skillNo":93,"name":94,"nameZh":6,"description":95,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":96,"licenseFileUrl":6,"sourceHtmlUrl":97,"categories":98,"keywords":99,"publishedAt":19},"01M78E0TGCAMNPS207BA3G07S7","zkrouter","Self-hosted OpenAI-compatible LLM router — smart tier-based routing, ZK-stamped routing decisions, mode overrides, free OpenRouter at install via the Birth bot. Save 60-80% on inference costs by routing to the cheapest capable model across 12+ OpenRouter models.","MIT","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzkrouter\u002FSKILL.md",[],[],{"skillNo":101,"name":102,"nameZh":6,"description":103,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":96,"licenseFileUrl":6,"sourceHtmlUrl":104,"categories":105,"keywords":106,"publishedAt":19},"01M78E0R1Z8BDT6EDTHY17MTNB","zk","For custom ZK Solana programs and privacy-preserving applications to prevent double spending. Guide to integrate rent-free nullifier PDAs for double-spend prevention.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzk\u002FSKILL.md",[],[],{"skillNo":108,"name":109,"nameZh":6,"description":110,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":96,"licenseFileUrl":6,"sourceHtmlUrl":111,"categories":112,"keywords":113,"publishedAt":19},"01M78E0P3ZTZ1YWF8MD5MM23QE","zk-omni-messaging","Zero-knowledge omnichain messaging between Robinhood Chain and Solana using CheshireZkOmniMessenger (msgType 4) with nullifier anti-replay and the zk-omni-relayer service.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzk-omni-messaging\u002FSKILL.md",[],[],{"skillNo":115,"name":116,"nameZh":6,"description":117,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":96,"licenseFileUrl":6,"sourceHtmlUrl":118,"categories":119,"keywords":120,"publishedAt":19},"01M78E0JXZ2PNB1896VXAGDBWD","zerion-wallet-data","Use when the user asks to check a wallet's portfolio, token balances, DeFi positions, PnL, transaction history, or NFTs. Provides interpreted on-chain data across 40+ EVM chains and Solana via x402 pay-per-request or API key — no raw RPC decoding needed.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzerion-wallet-data\u002FSKILL.md",[],[],{"skillNo":122,"name":123,"nameZh":6,"description":124,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":96,"licenseFileUrl":125,"sourceHtmlUrl":126,"categories":127,"keywords":128,"publishedAt":19},"01M78E0G14FH9GRRTQYMKD5E59","youtube-clipper","YouTube 视频智能剪辑工具。下载视频和字幕，AI 分析生成精细章节（几分钟级别）， 用户选择片段后自动剪辑、翻译字幕为中英双语、烧录字幕到视频，并生成总结文案。 使用场景：当用户需要剪辑 YouTube 视频、生成短视频片段、制作双语字幕版本时。 关键词：视频剪辑、YouTube、字幕翻译、双语字幕、视频下载、clip video","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fyoutube-clipper\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fyoutube-clipper\u002FSKILL.md",[],[],{"skillNo":130,"name":131,"nameZh":6,"description":132,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":96,"licenseFileUrl":6,"sourceHtmlUrl":133,"categories":134,"keywords":135,"publishedAt":19},"01M78E0C2ZYA99CFFDNB7ZMK5S","yield-optimization","Multi-chain yield optimization via yield.xyz (StakeKit). Discover 2,988+ yield opportunities across 75+ blockchains, build deposit\u002Fwithdrawal transactions with shell scripts, and sign them with a MoonPay wallet. Use when the user wants to earn yield, find the best lending or staking rate, enter\u002Fexit a position, or check portfolio balances.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fyield-optimization\u002FSKILL.md",[],[]]