[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M79FVDMHBPCAPVAYP75HNQCT":3,"skill-same:01M79FVDMHBPCAPVAYP75HNQCT":127},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":35,"commitSha":36,"skillDirectory":37,"forksCount":14,"repositoryHtmlUrl":38,"repositoryHomepageUrl":6,"ownerHtmlUrl":39,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":40,"install":41,"resources":54},"01M79FVDMHBPCAPVAYP75HNQCT","tao-train-fast-foundation-stereo",null,"Real-time stereo depth estimation using FastFoundationStereo (FFS), the distilled bp2 commercial variant of FoundationStereo. Predicts disparity maps from stereo image pairs with ~10× lower latency than full FoundationStereo. Use when training, evaluating, exporting, or running inference for a TAO FastFoundationStereo (FFS) model. Trigger phrases include \"train fast stereo\", \"real-time stereo disparity\", \"FastFoundationStereo\", \"distilled stereo depth\".","Elogic360\u002Fsoftware-factory","Elogic360","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F156535519?v=4","01M74NH3F0RB13PWB7JZDE2V80","elogic360","01M74NH3FPC65082MDZVMFHDM3",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: tao-train-fast-foundation-stereo\ndescription: Real-time stereo depth estimation using FastFoundationStereo (FFS), the distilled bp2 commercial variant of\n  FoundationStereo. Predicts disparity maps from stereo image pairs with ~10× lower latency than full FoundationStereo. Use\n  when training, evaluating, exporting, or running inference for a TAO FastFoundationStereo (FFS) model. Trigger phrases\n  include \"train fast stereo\", \"real-time stereo disparity\", \"FastFoundationStereo\", \"distilled stereo depth\".\nlicense: Apache-2.0\ncompatibility: Requires docker + nvidia-container-toolkit.\nmetadata:\n  version: \"0.1.0\"\n  author: NVIDIA Corporation\nallowed-tools: Read Bash\ntags:\n- stereo\n- depth\n- estimation\n- realtime\n- distilled\n---\n\n# Depth Net Fast Stereo\n\nReal-time stereo depth estimation using **FastFoundationStereo (FFS)** — the bp2 commercial distilled variant of FoundationStereo. Predicts disparity maps from rectified stereo image pairs with per-layer pruned widths for real-time inference.\n\nThe mono \u002F stereo \u002F fast-stereo skills share the unified TAO `depth_net` CLI; FFS is selected via `model.model_type: FastFoundationStereo`. FFS differs from `FoundationStereo` only in pruned per-layer widths and a serialized forward path; everything else (entrypoint, action verbs, dataset classes, deploy chain) is identical to `depth-net-stereo`.\n\nFor TAO Deploy TensorRT actions (`gen_trt_engine`, TensorRT `evaluate`, TensorRT `inference`), read `references\u002Ftao-deploy-fast-foundation-stereo.md` first. The deploy spec template lives at `references\u002Fspec_template_deploy.yaml`.\n\n## Train Action Policy\n\nThis model is AutoML-enabled at the model layer. Before handling any train-stage request, read `references\u002Fskill_info.yaml` and resolve the run override from either an explicit `automl_policy` value or the user's workflow request. Treat phrases like \"turn off AutoML\", \"disable AutoML\", \"no HPO\", or \"plain training\" as `automl_policy: off` for this run only; otherwise default to `auto`. When `automl_policy: auto`, `automl_enabled: true`, and both `schemas\u002Ftrain.schema.json` and `references\u002Fspec_template_train.yaml` are packaged, route the train action through `tao-skill-bank:tao-run-automl` by default with this model's `skill_dir`. Preserve workflow\u002Fapplication overrides for datasets, specs, output directories, GPU\u002Fplatform settings, parent checkpoints, and `automl_policy`. Use direct model training only when `automl_policy: off` or the packaged train schema\u002Ftemplate is missing; in the missing-schema case, report that AutoML is enabled but not runnable for this model until schemas are generated.\n\nNon-train actions such as `evaluate`, `inference`, `export`, and deploy flows stay in this model skill. The per-run `automl_policy` override does not change model metadata.\n\n## Two Use Cases\n\nFFS ships with a pre-trained bp2 commercial checkpoint (`model_best_bp2_serialize.pth`).\n\n1. **Raw deploy** — use the bp2 ckpt as-is. Skip `train`; run `inference` \u002F `evaluate` \u002F `export` \u002F `gen_trt_engine` directly with the bp2 file as the action's checkpoint.\n2. **Finetune on user data** — set `train.pretrained_model_path` to the bp2 file, train on user data, then verify + deploy on the resulting ckpt. The full 7-action sequence (train → evaluate pyt → inference pyt → export → gen_trt_engine → inference deploy → evaluate deploy) is supported.\n\n## Workflow\n\n### Prerequisites — data accessibility\n\nYour dataset (left + right images + GT disparity for train \u002F evaluate, left + right only for inference) must be reachable from inside the container:\n- **SDK runner**: place files at the S3 paths the runner resolves (`S3_TRAIN` \u002F `S3_EVAL` placeholders shown in the spec overrides).\n- **Direct `docker run`** (e.g. local testing): mount the host dataset root read-only at the same in-container path:\n\n```\ndocker run ... -v \u003Chost_data_root>:\u003Chost_data_root>:ro \u003Ccontainer> ...\n```\n\nThe same accessibility requirement applies to the `\u003Coutput_dir>` written by all actions, and to the bp2 checkpoint path.\n\n### Step 1 — Annotation file\n\nPer-line annotation file referenced by `data_sources[*].data_file`. Schema is identical to `depth-net-stereo`:\n\n| Columns | Format | Use |\n|---|---|---|\n| 2 | `\u003Cleft> \u003Cright>` | Stereo inference (no GT) |\n| 3 | `\u003Cleft> \u003Cright> \u003Cdisparity>` | Stereo with GT |\n| 4 | `\u003Cleft> \u003Cright> \u003Cdisparity> \u003Cocclusion_mask>` | Stereo with GT and occlusion mask |\n\nGenerate via `depth_net convert` if needed; see the `depth-net-stereo` skill for `convert_spec.yaml` template.\n\n### Step 2 — Pair `model_type` and `dataset_name` based on your data\n\nUse `model_type: FastFoundationStereo` for FFS. The `dataset_name` choice mirrors the stereo skill — pick the dataset-specific class when your layout matches a registered one, otherwise `GenericDataset`.\n\n| Data category | `model_type` | `dataset_name` |\n|---|---|---|\n| Middlebury | `FastFoundationStereo` | `Middlebury` |\n| KITTI | `FastFoundationStereo` | `Kitti` |\n| ETH3D | `FastFoundationStereo` | `Eth3d` |\n| FSD synthetic | `FastFoundationStereo` | `FSD` |\n| IsaacReal synthetic | `FastFoundationStereo` | `IsaacRealDataset` |\n| Crestereo synthetic | `FastFoundationStereo` | `Crestereo` |\n| Other \u002F non-canonical | `FastFoundationStereo` | `GenericDataset` |\n\nFor inference with 2-column annotations (left + right, no GT), use `dataset_name: GenericDataset` regardless of layout.\n\n### Step 3 — Set the bp2 distilled width overrides\n\nFFS requires 15 model-section width override fields whose values match the bp2 commercial checkpoint exactly. Omitting any field falls back to TAO defaults that do **not** match the bp2 ckpt and produce shape-mismatch errors at forward time.\n\n```yaml\nmodel:\n  model_type: FastFoundationStereo\n  encoder: vitl\n  hidden_dims: [128]                    # 1-layer GRU; NOT [128,128,128]\n  n_gru_layers: 1                       # bp2 single-GRU\n  corr_radius: 4\n  corr_levels: 2\n  n_downsample: 2\n  valid_iters: 8\n  max_disparity: 192                    # bp2 commercial; NOT 416 (full FS default)\n  volume_dim: 28                       # bp2 ckpt invariant; NOT 32 (full FS default)\n  mixed_precision: false                # see references\u002Fparameters.md\n  gwc_feature_normalize: true           # see references\u002Fparameters.md\n\n  # 15 bp2 distilled width overrides — copy as-is\n  motion_encoder_widths: [56, 96, 16, 12]\n  motion_encoder_final: 48\n  gru_hidden: 60\n  gru_gating_conv_widths: [100, 168]\n  disp_head_input_dim: 60\n  disp_head_intermediate: 36\n  disp_head_pwconv1_widths: [212, 244]\n  mask_widths: [32, 16]\n  stem_2_widths: [12, 16]\n  spx_2_gru_widths: [16, 12, 16, 24]\n  spx_gru_out: 9\n  classifier_mid: 14\n  cnet_conv04_widths: [60, 48]\n  cam_mid_channels: 8\n  cost_agg_conv_patch_padding: [0, 0, 0]\n```\n\nThe spec templates at `references\u002Fspec_template_*.yaml` carry this block as the canonical source.\n\n### Step 4 — Write spec yaml from the spec overrides\n\nCopy the action block from `references\u002Fspec-overrides.md` (per-action Python override dicts plus the shared `FFS_MODEL_BLOCK`). Replace:\n- `model.model_type: FastFoundationStereo` (already set)\n- `dataset.\u003C...>.data_sources[*].dataset_name` from Step 2\n- `dataset.\u003C...>.data_sources[*].data_file` with the path from Step 1\n- For raw deploy use cases (no train): set `\u003Caction>.checkpoint` to the bp2 file path\n- For finetune use cases: set `train.pretrained_model_path` to the bp2 file path\n\n**Chained train → next action checkpoint path**: For local Docker chaining (no SDK runner), the trained checkpoint lives at `\u003Ctrain.results_dir>\u002F\u003Ctask>\u002Fdn_model_latest.pth` — Lightning `ModelCheckpoint` nests under the task name. Example: `train.results_dir: \u002Fworkspace\u002Fresults\u002Ffinetune\u002Ftrain` produces `\u002Fworkspace\u002Fresults\u002Ffinetune\u002Ftrain\u002Ftrain\u002Fdn_model_latest.pth`. Use that nested path for the next action's `\u003Caction>.checkpoint`. SDK-runner deploys resolve this automatically via `parent_job_id` — see `references\u002Fparent-model-inference.md`.\n\nShape consistency: `crop_size` in `dataset.test_dataset.augmentation.crop_size` should match `export.input_height` \u002F `input_width` for end-to-end pyt-vs-deploy comparability — see `references\u002Ftao-deploy-fast-foundation-stereo.md`'s shape table.\n\n### Step 5 — Run\n\n```\ndocker run --gpus 'device=0' --shm-size 16G --ipc=host \\\n  --user $(id -u):$(id -g) \\\n  -v \u003Cdata_root>:\u003Cdata_root>:ro \\\n  -v \u003Coutput_dir>:\u003Coutput_dir> \\\n  -v \u003Cbp2_ckpt_dir>:\u003Cbp2_ckpt_dir>:ro \\\n  \u003Ccontainer> \\\n  depth_net \u003Caction> -e \u003Cspec.yaml>\n```\n\nWithout `--user $(id -u):$(id -g)` the container writes outputs as `nobody:nogroup`, blocking host-side cleanup \u002F retry.\n\nFor the local bind-mount `__pycache__` caveat (QA \u002F development only — clearing stale `.pyc` files that shadow patched source), see `references\u002Ftroubleshooting.md` → \"Local bind-mount tip\".\n\n### Step 6 — Verify\n\n- Container exit code 0\n- `status.json` `kpi` block populated\n- For `train`: inspect per-step `train_loss` directly (the entrypoint reports `Execution status: PASS` even when loss is NaN)\n- For `evaluate`: rely on `epe` \u002F `bp1` \u002F `bp2` \u002F `bp3` \u002F `d1` \u002F `rmse` (the evaluator also emits `abs_rel` \u002F `sq_rel` \u002F `rmse_log` which are non-meaningful for stereo)\n- For `inference`: artifacts under `results_dir`\n- **KPI namespace difference between pyt and deploy**: pyt `evaluate` writes the metric set under `kpi.val\u002Fepe`, `kpi.val\u002Fbp1`, etc. (namespaced by Lightning's `val\u002F` prefix). Deploy `evaluate` (TRT engine path) writes the same metric set under `kpi.epe`, `kpi.bp1`, etc. (no `val\u002F` prefix). Downstream verification scripts that read `status.json` need to handle both shapes.\n- **Validate drift on your own dataset**: if you compare TAO FFS deploy (`gen_trt_engine` + TRT `evaluate`) against the upstream FFS deploy path on the same input, expect a small residual mean_abs disparity drift (TAO export graph + TRT 10.13 interaction; not improvable at the source-code level). The exact magnitude is dataset and hardware dependent — measure on your own data and decide whether the drift is acceptable for your downstream task.\n\n### 7-action deploy flow\n\n```\ntrain (optional)            → finetuned ckpt\nevaluate (pyt)              → PyT eager EPE \u002F bp on val GT\ninference (pyt)             → PyT eager disparity samples (visual sanity)\nexport                      → static fp32 ONNX (recommended at 480×736 or 320×736)\ngen_trt_engine             → fp16 TRT engine on static ONNX path\ninference (deploy)         → TRT disparity samples\nevaluate (deploy)          → TRT EPE \u002F bp drift vs PyT eager fp32\n```\n\nSkip `train` for raw-bp2 deploy. The remaining 6 actions (or the 4 deploy-only verbs starting from `export`) cover both use cases.\n\nFull TAO Deploy reference: [tao-deploy-fast-foundation-stereo](references\u002Ftao-deploy-fast-foundation-stereo.md).\n\n## Training Requirements\n\n- **Valid `dataset_name` values for stereo `data_sources`** (case-insensitive): `FSD`, `IsaacRealDataset`, `Crestereo`, `Middlebury`, `Eth3d`, `Kitti`, `GenericDataset`\n- **Monitoring metric:** val\u002Floss\n\n### Per-Action Dataset Requirements\n\n| Action | Spec Key | Source | Files | List? |\n|---|---|---|---|---|\n| evaluate | dataset.test_dataset.data_sources | eval_dataset | data_file: annotations.txt + dataset_name | Yes |\n| inference | dataset.infer_dataset.data_sources | inference_dataset | data_file: annotations.txt + dataset_name | Yes |\n| train | dataset.train_dataset.data_sources | train_datasets | data_file: annotations.txt + dataset_name | Yes |\n| train | dataset.val_dataset.data_sources | eval_dataset | data_file: annotations.txt + dataset_name | Yes |\n\nData source overrides are **mandatory for every action**. Each `data_sources` entry needs both `data_file` and `dataset_name`. The `model.*` width fields from Step 3 are also mandatory. See `references\u002Fspec-overrides.md` for the complete per-action override dicts (train finetune, raw-bp2 evaluate \u002F inference \u002F export) and the shared `FFS_MODEL_BLOCK`.\n\n## Eval Dataset\n\nOptional. Val dataset configured via `dataset.val_dataset.data_sources` (each entry needs `data_file` and `dataset_name`).\n\n## Parameters, Metrics, Hardware\n\nSee `references\u002Fparameters.md` for the full parameter glossary (`model.*` \u002F `dataset.*` \u002F `train.*` knobs including `max_disparity: 192`, `gwc_feature_normalize: true`, `mixed_precision: false`, `volume_dim: 28`, `valid_iters`, `save_raw_pfm`), the evaluation-metric table (`epe` \u002F `bp1` \u002F `bp2` \u002F `bp3` \u002F `d1` \u002F `rmse` are meaningful; `abs_rel` \u002F `sq_rel` \u002F `rmse_log` are not), multi-GPU \u002F multi-node spec keys, and hardware requirements.\n\n## Export \u002F TRT Defaults\n\n`export` always emits a **fp32 ONNX** regardless of `model.mixed_precision`; the fp16 vs fp32 selection happens at `gen_trt_engine` via `gen_trt_engine.tensorrt.data_type`. Recommended TRT precision for FFS-bp2 is `fp16` on the static-shape ONNX path (lowest drift). The dynamic-shape path supports both `fp32` (default; static-fp32 parity) and `fp16` (latency-critical multi-resolution; higher drift, may NaN under some checkpoint states — fall back to fp32 if observed).\n\nSee `references\u002Fexport-trt-defaults.md` for the full TRT\u002FONNX defaults and the four-way export use-case matrix (`export.batch_size` × `export.dynamic_hw`; dynamic H\u002FW is FFS-only). See `references\u002Ftao-deploy-fast-foundation-stereo.md` for the deployment matrix and static-vs-dynamic shape guidance.\n\n## Troubleshooting\n\nSee `references\u002Ftroubleshooting.md` for error patterns and fixes, including `shape mismatch` at forward (missing width override), missing `gwc_feature_normalize` (TAO Core too old), `dynamic_hw: true` warning on FS \u002F mono export, `Key 'encoder' not in 'StereoBackBone'`, missing `dataset_name` in `data_sources`, negative disparity, larger-than-expected disparity drift (missing `max_disparity: 192`), `depth_net_stereo: not found`, decorative pyt-eval `crop_size`, the cosmetic `Failed to import SAM3` warning, and silent dynamic-deploy stride-incompatibility.\n\n## Spec Param \u002F Parent Model Inference\n\nModel-specific inference mappings belong in this skill, not in `config.json`. Generated runners should apply the mappings with SDK helpers before `create_job()`. See `references\u002Fparent-model-inference.md` for the full per-action spec-field → inference-function mapping table.\n\nFor `parent_model` or `parent_model_folder`, pass the upstream train \u002F export \u002F AutoML child job id as `parent_job_id`. The SDK lists the parent result folder, filters checkpoint artifacts, and returns the selected model file or folder. For raw-bp2 use cases without a parent train job, set the `\u003Caction>.checkpoint` field explicitly to the bp2 file path. Do not patch generated runner scripts to guess checkpoint paths.\n","# Depth Net Fast Stereo\n\nReal-time stereo depth estimation using **FastFoundationStereo (FFS)** — the bp2 commercial distilled variant of FoundationStereo. Predicts disparity maps from rectified stereo image pairs with per-layer pruned widths for real-time inference.\n\nThe mono \u002F stereo \u002F fast-stereo skills share the unified TAO `depth_net` CLI; FFS is selected via `model.model_type: FastFoundationStereo`. FFS differs from `FoundationStereo` only in pruned per-layer widths and a serialized forward path; everything else (entrypoint, action verbs, dataset classes, deploy chain) is identical to `depth-net-stereo`.\n\nFor TAO Deploy TensorRT actions (`gen_trt_engine`, TensorRT `evaluate`, TensorRT `inference`), read `references\u002Ftao-deploy-fast-foundation-stereo.md` first. The deploy spec template lives at `references\u002Fspec_template_deploy.yaml`.\n\n## Train Action Policy\n\nThis model is AutoML-enabled at the model layer. Before handling any train-stage request, read `references\u002Fskill_info.yaml` and resolve the run override from either an explicit `automl_policy` value or the user's workflow request. Treat phrases like \"turn off AutoML\", \"disable AutoML\", \"no HPO\", or \"plain training\" as `automl_policy: off` for this run only; otherwise default to `auto`. When `automl_policy: auto`, `automl_enabled: true`, and both `schemas\u002Ftrain.schema.json` and `references\u002Fspec_template_train.yaml` are packaged, route the train action through `tao-skill-bank:tao-run-automl` by default with this model's `skill_dir`. Preserve workflow\u002Fapplication overrides for datasets, specs, output directories, GPU\u002Fplatform settings, parent checkpoints, and `automl_policy`. Use direct model training only when `automl_policy: off` or the packaged train schema\u002Ftemplate is missing; in the missing-schema case, report that AutoML is enabled but not runnable for this model until schemas are generated.\n\nNon-train actions such as `evaluate`, `inference`, `export`, and deploy flows stay in this model skill. The per-run `automl_policy` override does not change model metadata.\n\n## Two Use Cases\n\nFFS ships with a pre-trained bp2 commercial checkpoint (`model_best_bp2_serialize.pth`).\n\n1. **Raw deploy** — use the bp2 ckpt as-is. Skip `train`; run `inference` \u002F `evaluate` \u002F `export` \u002F `gen_trt_engine` directly with the bp2 file as the action's checkpoint.\n2. **Finetune on user data** — set `train.pretrained_model_path` to the bp2 file, train on user data, then verify + deploy on the resulting ckpt. The full 7-action sequence (train → evaluate pyt → inference pyt → export → gen_trt_engine → inference deploy → evaluate deploy) is supported.\n\n## Workflow\n\n### Prerequisites — data accessibility\n\nYour dataset (left + right images + GT disparity for train \u002F evaluate, left + right only for inference) must be reachable from inside the container:\n- **SDK runner**: place files at the S3 paths the runner resolves (`S3_TRAIN` \u002F `S3_EVAL` placeholders shown in the spec overrides).\n- **Direct `docker run`** (e.g. local testing): mount the host dataset root read-only at the same in-container path:\n\n```\ndocker run ... -v \u003Chost_data_root>:\u003Chost_data_root>:ro \u003Ccontainer> ...\n```\n\nThe same accessibility requirement applies to the `\u003Coutput_dir>` written by all actions, and to the bp2 checkpoint path.\n\n### Step 1 — Annotation file\n\nPer-line annotation file referenced by `data_sources[*].data_file`. Schema is identical to `depth-net-stereo`:\n\n| Columns | Format | Use |\n|---|---|---|\n| 2 | `\u003Cleft> \u003Cright>` | Stereo inference (no GT) |\n| 3 | `\u003Cleft> \u003Cright> \u003Cdisparity>` | Stereo with GT |\n| 4 | `\u003Cleft> \u003Cright> \u003Cdisparity> \u003Cocclusion_mask>` | Stereo with GT and occlusion mask |\n\nGenerate via `depth_net convert` if needed; see the `depth-net-stereo` skill for `convert_spec.yaml` template.\n\n### Step 2 — Pair `model_type` and `dataset_name` based on your data\n\nUse `model_type: FastFoundationStereo` for FFS. The `dataset_name` choice mirrors the stereo skill — pick the dataset-specific class when your layout matches a registered one, otherwise `GenericDataset`.\n\n| Data category | `model_type` | `dataset_name` |\n|---|---|---|\n| Middlebury | `FastFoundationStereo` | `Middlebury` |\n| KITTI | `FastFoundationStereo` | `Kitti` |\n| ETH3D | `FastFoundationStereo` | `Eth3d` |\n| FSD synthetic | `FastFoundationStereo` | `FSD` |\n| IsaacReal synthetic | `FastFoundationStereo` | `IsaacRealDataset` |\n| Crestereo synthetic | `FastFoundationStereo` | `Crestereo` |\n| Other \u002F non-canonical | `FastFoundationStereo` | `GenericDataset` |\n\nFor inference with 2-column annotations (left + right, no GT), use `dataset_name: GenericDataset` regardless of layout.\n\n### Step 3 — Set the bp2 distilled width overrides\n\nFFS requires 15 model-section width override fields whose values match the bp2 commercial checkpoint exactly. Omitting any field falls back to TAO defaults that do **not** match the bp2 ckpt and produce shape-mismatch errors at forward time.\n\n```yaml\nmodel:\n  model_type: FastFoundationStereo\n  encoder: vitl\n  hidden_dims: [128]                    # 1-layer GRU; NOT [128,128,128]\n  n_gru_layers: 1                       # bp2 single-GRU\n  corr_radius: 4\n  corr_levels: 2\n  n_downsample: 2\n  valid_iters: 8\n  max_disparity: 192                    # bp2 commercial; NOT 416 (full FS default)\n  volume_dim: 28                       # bp2 ckpt invariant; NOT 32 (full FS default)\n  mixed_precision: false                # see references\u002Fparameters.md\n  gwc_feature_normalize: true           # see references\u002Fparameters.md\n\n  # 15 bp2 distilled width overrides — copy as-is\n  motion_encoder_widths: [56, 96, 16, 12]\n  motion_encoder_final: 48\n  gru_hidden: 60\n  gru_gating_conv_widths: [100, 168]\n  disp_head_input_dim: 60\n  disp_head_intermediate: 36\n  disp_head_pwconv1_widths: [212, 244]\n  mask_widths: [32, 16]\n  stem_2_widths: [12, 16]\n  spx_2_gru_widths: [16, 12, 16, 24]\n  spx_gru_out: 9\n  classifier_mid: 14\n  cnet_conv04_widths: [60, 48]\n  cam_mid_channels: 8\n  cost_agg_conv_patch_padding: [0, 0, 0]\n```\n\nThe spec templates at `references\u002Fspec_template_*.yaml` carry this block as the canonical source.\n\n### Step 4 — Write spec yaml from the spec overrides\n\nCopy the action block from `references\u002Fspec-overrides.md` (per-action Python override dicts plus the shared `FFS_MODEL_BLOCK`). Replace:\n- `model.model_type: FastFoundationStereo` (already set)\n- `dataset.\u003C...>.data_sources[*].dataset_name` from Step 2\n- `dataset.\u003C...>.data_sources[*].data_file` with the path from Step 1\n- For raw deploy use cases (no train): set `\u003Caction>.checkpoint` to the bp2 file path\n- For finetune use cases: set `train.pretrained_model_path` to the bp2 file path\n\n**Chained train → next action checkpoint path**: For local Docker chaining (no SDK runner), the trained checkpoint lives at `\u003Ctrain.results_dir>\u002F\u003Ctask>\u002Fdn_model_latest.pth` — Lightning `ModelCheckpoint` nests under the task name. Example: `train.results_dir: \u002Fworkspace\u002Fresults\u002Ffinetune\u002Ftrain` produces `\u002Fworkspace\u002Fresults\u002Ffinetune\u002Ftrain\u002Ftrain\u002Fdn_model_latest.pth`. Use that nested path for the next action's `\u003Caction>.checkpoint`. SDK-runner deploys resolve this automatically via `parent_job_id` — see `references\u002Fparent-model-inference.md`.\n\nShape consistency: `crop_size` in `dataset.test_dataset.augmentation.crop_size` should match `export.input_height` \u002F `input_width` for end-to-end pyt-vs-deploy comparability — see `references\u002Ftao-deploy-fast-foundation-stereo.md`'s shape table.\n\n### Step 5 — Run\n\n```\ndocker run --gpus 'device=0' --shm-size 16G --ipc=host \\\n  --user $(id -u):$(id -g) \\\n  -v \u003Cdata_root>:\u003Cdata_root>:ro \\\n  -v \u003Coutput_dir>:\u003Coutput_dir> \\\n  -v \u003Cbp2_ckpt_dir>:\u003Cbp2_ckpt_dir>:ro \\\n  \u003Ccontainer> \\\n  depth_net \u003Caction> -e \u003Cspec.yaml>\n```\n\nWithout `--user $(id -u):$(id -g)` the container writes outputs as `nobody:nogroup`, blocking host-side cleanup \u002F retry.\n\nFor the local bind-mount `__pycache__` caveat (QA \u002F development only — clearing stale `.pyc` files that shadow patched source), see `references\u002Ftroubleshooting.md` → \"Local bind-mount tip\".\n\n### Step 6 — Verify\n\n- Container exit code 0\n- `status.json` `kpi` block populated\n- For `train`: inspect per-step `train_loss` directly (the entrypoint reports `Execution status: PASS` even when loss is NaN)\n- For `evaluate`: rely on `epe` \u002F `bp1` \u002F `bp2` \u002F `bp3` \u002F `d1` \u002F `rmse` (the evaluator also emits `abs_rel` \u002F `sq_rel` \u002F `rmse_log` which are non-meaningful for stereo)\n- For `inference`: artifacts under `results_dir`\n- **KPI namespace difference between pyt and deploy**: pyt `evaluate` writes the metric set under `kpi.val\u002Fepe`, `kpi.val\u002Fbp1`, etc. (namespaced by Lightning's `val\u002F` prefix). Deploy `evaluate` (TRT engine path) writes the same metric set under `kpi.epe`, `kpi.bp1`, etc. (no `val\u002F` prefix). Downstream verification scripts that read `status.json` need to handle both shapes.\n- **Validate drift on your own dataset**: if you compare TAO FFS deploy (`gen_trt_engine` + TRT `evaluate`) against the upstream FFS deploy path on the same input, expect a small residual mean_abs disparity drift (TAO export graph + TRT 10.13 interaction; not improvable at the source-code level). The exact magnitude is dataset and hardware dependent — measure on your own data and decide whether the drift is acceptable for your downstream task.\n\n### 7-action deploy flow\n\n```\ntrain (optional)            → finetuned ckpt\nevaluate (pyt)              → PyT eager EPE \u002F bp on val GT\ninference (pyt)             → PyT eager disparity samples (visual sanity)\nexport                      → static fp32 ONNX (recommended at 480×736 or 320×736)\ngen_trt_engine             → fp16 TRT engine on static ONNX path\ninference (deploy)         → TRT disparity samples\nevaluate (deploy)          → TRT EPE \u002F bp drift vs PyT eager fp32\n```\n\nSkip `train` for raw-bp2 deploy. The remaining 6 actions (or the 4 deploy-only verbs starting from `export`) cover both use cases.\n\nFull TAO Deploy reference: [tao-deploy-fast-foundation-stereo](references\u002Ftao-deploy-fast-foundation-stereo.md).\n\n## Training Requirements\n\n- **Valid `dataset_name` values for stereo `data_sources`** (case-insensitive): `FSD`, `IsaacRealDataset`, `Crestereo`, `Middlebury`, `Eth3d`, `Kitti`, `GenericDataset`\n- **Monitoring metric:** val\u002Floss\n\n### Per-Action Dataset Requirements\n\n| Action | Spec Key | Source | Files | List? |\n|---|---|---|---|---|\n| evaluate | dataset.test_dataset.data_sources | eval_dataset | data_file: annotations.txt + dataset_name | Yes |\n| inference | dataset.infer_dataset.data_sources | inference_dataset | data_file: annotations.txt + dataset_name | Yes |\n| train | dataset.train_dataset.data_sources | train_datasets | data_file: annotations.txt + dataset_name | Yes |\n| train | dataset.val_dataset.data_sources | eval_dataset | data_file: annotations.txt + dataset_name | Yes |\n\nData source overrides are **mandatory for every action**. Each `data_sources` entry needs both `data_file` and `dataset_name`. The `model.*` width fields from Step 3 are also mandatory. See `references\u002Fspec-overrides.md` for the complete per-action override dicts (train finetune, raw-bp2 evaluate \u002F inference \u002F export) and the shared `FFS_MODEL_BLOCK`.\n\n## Eval Dataset\n\nOptional. Val dataset configured via `dataset.val_dataset.data_sources` (each entry needs `data_file` and `dataset_name`).\n\n## Parameters, Metrics, Hardware\n\nSee `references\u002Fparameters.md` for the full parameter glossary (`model.*` \u002F `dataset.*` \u002F `train.*` knobs including `max_disparity: 192`, `gwc_feature_normalize: true`, `mixed_precision: false`, `volume_dim: 28`, `valid_iters`, `save_raw_pfm`), the evaluation-metric table (`epe` \u002F `bp1` \u002F `bp2` \u002F `bp3` \u002F `d1` \u002F `rmse` are meaningful; `abs_rel` \u002F `sq_rel` \u002F `rmse_log` are not), multi-GPU \u002F multi-node spec keys, and hardware requirements.\n\n## Export \u002F TRT Defaults\n\n`export` always emits a **fp32 ONNX** regardless of `model.mixed_precision`; the fp16 vs fp32 selection happens at `gen_trt_engine` via `gen_trt_engine.tensorrt.data_type`. Recommended TRT precision for FFS-bp2 is `fp16` on the static-shape ONNX path (lowest drift). The dynamic-shape path supports both `fp32` (default; static-fp32 parity) and `fp16` (latency-critical multi-resolution; higher drift, may NaN under some checkpoint states — fall back to fp32 if observed).\n\nSee `references\u002Fexport-trt-defaults.md` for the full TRT\u002FONNX defaults and the four-way export use-case matrix (`export.batch_size` × `export.dynamic_hw`; dynamic H\u002FW is FFS-only). See `references\u002Ftao-deploy-fast-foundation-stereo.md` for the deployment matrix and static-vs-dynamic shape guidance.\n\n## Troubleshooting\n\nSee `references\u002Ftroubleshooting.md` for error patterns and fixes, including `shape mismatch` at forward (missing width override), missing `gwc_feature_normalize` (TAO Core too old), `dynamic_hw: true` warning on FS \u002F mono export, `Key 'encoder' not in 'StereoBackBone'`, missing `dataset_name` in `data_sources`, negative disparity, larger-than-expected disparity drift (missing `max_disparity: 192`), `depth_net_stereo: not found`, decorative pyt-eval `crop_size`, the cosmetic `Failed to import SAM3` warning, and silent dynamic-deploy stride-incompatibility.\n\n## Spec Param \u002F Parent Model Inference\n\nModel-specific inference mappings belong in this skill, not in `config.json`. Generated runners should apply the mappings with SDK helpers before `create_job()`. See `references\u002Fparent-model-inference.md` for the full per-action spec-field → inference-function mapping table.\n\nFor `parent_model` or `parent_model_folder`, pass the upstream train \u002F export \u002F AutoML child job id as `parent_job_id`. The SDK lists the parent result folder, filters checkpoint artifacts, and returns the selected model file or folder. For raw-bp2 use cases without a parent train job, set the `\u003Caction>.checkpoint` field explicitly to the bp2 file path. Do not patch generated runner scripts to guess checkpoint paths.\n",{"name":5,"tags":24,"license":15,"metadata":30,"description":7,"allowed-tools":33,"compatibility":34},[25,26,27,28,29],"stereo","depth","estimation","realtime","distilled",{"author":31,"version":32},"NVIDIA Corporation","0.1.0","Read Bash","Requires docker + nvidia-container-toolkit.","main","c475fc64e94e803b50d14a791dfee8853fbb2c5c","skills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo","https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory","https:\u002F\u002Fgithub.com\u002FElogic360","2026-11-08 22:21:22",{"source":8,"skillName":5,"commands":42,"prompt":52,"zipAvailable":53},[43,46,49],{"tool":44,"command":45},"npx","npx skills add Elogic360\u002Fsoftware-factory --skill tao-train-fast-foundation-stereo",{"tool":47,"command":48},"bunx","bunx skills add Elogic360\u002Fsoftware-factory --skill tao-train-fast-foundation-stereo",{"tool":50,"command":51},"pnpm","pnpm dlx skills add Elogic360\u002Fsoftware-factory --skill tao-train-fast-foundation-stereo","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo\u002FSKILL.md",true,[55,61,65,69,73,77,81,85,89,93,97,101,105,109,113,117,121],{"path":56,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":59,"sourceHtmlUrl":60},"BENCHMARK.md","REFERENCE","参考文档",5915,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo\u002FBENCHMARK.md",{"path":62,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":63,"sourceHtmlUrl":64},"evals\u002Fevals.json",809,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo\u002Fevals\u002Fevals.json",{"path":66,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":67,"sourceHtmlUrl":68},"references\u002Fexport-trt-defaults.md",2247,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo\u002Freferences\u002Fexport-trt-defaults.md",{"path":70,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":71,"sourceHtmlUrl":72},"references\u002Fparameters.md",5581,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo\u002Freferences\u002Fparameters.md",{"path":74,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":75,"sourceHtmlUrl":76},"references\u002Fparent-model-inference.md",3059,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo\u002Freferences\u002Fparent-model-inference.md",{"path":78,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":79,"sourceHtmlUrl":80},"references\u002Fskill_info.yaml",1927,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo\u002Freferences\u002Fskill_info.yaml",{"path":82,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":83,"sourceHtmlUrl":84},"references\u002Fspec_template_deploy.yaml",2617,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo\u002Freferences\u002Fspec_template_deploy.yaml",{"path":86,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":87,"sourceHtmlUrl":88},"references\u002Fspec_template_evaluate.yaml",1890,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo\u002Freferences\u002Fspec_template_evaluate.yaml",{"path":90,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":91,"sourceHtmlUrl":92},"references\u002Fspec_template_export.yaml",1870,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo\u002Freferences\u002Fspec_template_export.yaml",{"path":94,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":95,"sourceHtmlUrl":96},"references\u002Fspec_template_inference.yaml",1835,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo\u002Freferences\u002Fspec_template_inference.yaml",{"path":98,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":99,"sourceHtmlUrl":100},"references\u002Fspec_template_train.yaml",2707,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo\u002Freferences\u002Fspec_template_train.yaml",{"path":102,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":103,"sourceHtmlUrl":104},"references\u002Fspec-overrides.md",3691,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo\u002Freferences\u002Fspec-overrides.md",{"path":106,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":107,"sourceHtmlUrl":108},"references\u002Ftao-deploy-fast-foundation-stereo.md",18996,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo\u002Freferences\u002Ftao-deploy-fast-foundation-stereo.md",{"path":110,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":111,"sourceHtmlUrl":112},"references\u002Ftao-deploy-fast-foundation-stereo.skill_info.yaml",2746,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo\u002Freferences\u002Ftao-deploy-fast-foundation-stereo.skill_info.yaml",{"path":114,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":115,"sourceHtmlUrl":116},"references\u002Ftroubleshooting.md",4328,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo\u002Freferences\u002Ftroubleshooting.md",{"path":118,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":119,"sourceHtmlUrl":120},"skill-card.md",4248,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo\u002Fskill-card.md",{"path":122,"resourceType":123,"resourceTypeDesc":124,"sizeBytes":125,"sourceHtmlUrl":126},"skill.oms.sig","OTHER","其他",7885,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Ftao-train-fast-foundation-stereo\u002Fskill.oms.sig",[128,135,142,149,156,163],{"skillNo":129,"name":130,"nameZh":6,"description":131,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":132,"categories":133,"keywords":134,"publishedAt":19},"01M79G206MT3J6S0GY3C45P2KV","vss-summarize-video","Use to summarize a recorded video via the LVS summarization microservice (HITL-gated) with a VLM fallback. Not for report generation or live RTSP captioning.","https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fvss-summarize-video\u002FSKILL.md",[],[],{"skillNo":136,"name":137,"nameZh":6,"description":138,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":139,"categories":140,"keywords":141,"publishedAt":19},"01M79G1WN63PGRE8CKD47WEBQ5","vss-setup-video-analytics-api","Use to deploy the vss-video-analytics-api REST service standalone (config-source, data-log bind, Elasticsearch, optional Kafka). Not for full warehouse deploy.","https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fvss-setup-video-analytics-api\u002FSKILL.md",[],[],{"skillNo":143,"name":144,"nameZh":6,"description":145,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":146,"categories":147,"keywords":148,"publishedAt":19},"01M79G1S9F7VY3EF6H53PRP5AN","vss-setup-behavior-analytics","Use to deploy the vss-behavior-analytics service standalone (entrypoint, config-source, optional calibration). Not for the full warehouse deploy.","https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fvss-setup-behavior-analytics\u002FSKILL.md",[],[],{"skillNo":150,"name":151,"nameZh":6,"description":152,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":153,"categories":154,"keywords":155,"publishedAt":19},"01M79G1N1S1Y27PFZPYMNK1MTQ","vss-search-archive","Use this skill to run top-level VSS fusion search on archived video, or to ingest video files \u002F RTSP streams for search. Do NOT use for ad-hoc visual Q&A (use vss-ask-video), live captioning (use vss-deploy-dense-captioning), or video summarization and reports (use vss-summarize-video).","https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fvss-search-archive\u002FSKILL.md",[],[],{"skillNo":157,"name":158,"nameZh":6,"description":159,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":160,"categories":161,"keywords":162,"publishedAt":19},"01M79G1H6WN0Q6C0VCAXCZX6B7","vss-query-analytics","Use this skill when reading video-analytics metrics, incidents, alerts, and sensor data via the VA-MCP server (port 9901). Not for live VLM or incident-range narrative reports.","https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fvss-query-analytics\u002FSKILL.md",[],[],{"skillNo":164,"name":165,"nameZh":6,"description":166,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":167,"categories":168,"keywords":169,"publishedAt":19},"01M79G1DYPNXQS7QHZ0AAF5FX5","vss-manage-video-io-storage","Use to call the VIOS REST API (sensor list, timelines, clip extraction, snapshots, add\u002Fdelete sensors and streams). Not for VLM inference or search.","https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fvss-manage-video-io-storage\u002FSKILL.md",[],[]]