[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M78JFAE6HEPKDKATMPNMB0DV":3,"skill-same:01M78JFAE6HEPKDKATMPNMB0DV":147},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":18,"linkOnly":19,"markdownContent":20,"markdownBody":21,"frontMatter":22,"refName":30,"commitSha":31,"skillDirectory":32,"forksCount":13,"repositoryHtmlUrl":33,"repositoryHomepageUrl":34,"ownerHtmlUrl":35,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":36,"install":37,"resources":50},"01M78JFAE6HEPKDKATMPNMB0DV","tao-train-mask-grounding-dino",null,"Mask Grounding DINO for grounded instance segmentation. Extends Grounding DINO with a mask-prediction head for open-set segmentation guided by text prompts. Use when training, evaluating, exporting, quantizing, or running inference for a TAO Mask-Grounding-DINO model. Trigger phrases include \"train Mask Grounding DINO\", \"open-vocabulary segmentation\", \"text-prompted instance segmentation\", \"grounded mask DETR\".","matheussluzz\u002FZeuZ-Agent","matheussluzz","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F209255758?v=4","01M78HKHH1D1ASC9KC9BG583DH","01M78HKHHP6W7XRXQM2SBCAM3R",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: tao-train-mask-grounding-dino\ndescription: Mask Grounding DINO for grounded instance segmentation. Extends Grounding DINO with a mask-prediction head for\n  open-set segmentation guided by text prompts. Use when training, evaluating, exporting, quantizing, or running inference for\n  a TAO Mask-Grounding-DINO model. Trigger phrases include \"train Mask Grounding DINO\", \"open-vocabulary segmentation\",\n  \"text-prompted instance segmentation\", \"grounded mask DETR\".\nlicense: Apache-2.0\ncompatibility: Requires docker + nvidia-container-toolkit.\nmetadata:\n  version: \"0.1.0\"\n  author: NVIDIA Corporation\nallowed-tools: Read Bash\ntags:\n- segmentation\n---\n\n# Mask Grounding DINO\n\nMask Grounding DINO for grounded instance segmentation. Extends Grounding DINO with mask prediction head for open-set segmentation guided by text prompts.\n\nSet train.pretrained_model_path for full model weights.\n\nFor TAO Deploy TensorRT actions (`gen_trt_engine`, TensorRT `evaluate`, and TensorRT `inference`), read `references\u002Ftao-deploy-mask-grounding-dino.md` first. Deploy spec templates live in this skill's `references\u002F` folder with the `spec_template_deploy_*.yaml` prefix.\n\n## Dataclass Schemas\n\nGenerated TAO Core schemas are packaged in `schemas\u002F\u003Caction>.schema.json`, with `schemas\u002Fmanifest.json` listing available actions. Each generated schema also emits `references\u002Fspec_template_\u003Caction>.yaml` from the schema top-level `default` field. AutoML enablement is declared at the model layer in `references\u002Fskill_info.yaml` via `automl_enabled`. Runnable AutoML still requires `schemas\u002Ftrain.schema.json` and `references\u002Fspec_template_train.yaml` to exist and parse. Use the packaged train schema for `automl_default_parameters`, `automl_disabled_parameters`, defaults, min\u002Fmax bounds, enums, option weights, math conditions, dependencies, and popular parameters. Do not expect `~\u002Ftao-core` at runtime; maintainers regenerate schemas\u002Ftemplates before packaging the skill bank.\n\n## Train Action Policy\n\nThis model is AutoML-enabled at the model layer. Before handling any train-stage request, read `references\u002Fskill_info.yaml` and resolve the run override from either an explicit `automl_policy` value or the user's workflow request. Use `automl_policy: on` by default and only expose `on` \u002F `off` in new launch prompts. Treat phrases like \"turn off AutoML\", \"disable AutoML\", \"no HPO\", or \"plain training\" as `automl_policy: off` for this run only. When `automl_policy: on`, `automl_enabled: true`, and both `schemas\u002Ftrain.schema.json` and `references\u002Fspec_template_train.yaml` are packaged, route the train action through `tao-skill-bank:tao-run-automl` by default with this model's `skill_dir`. Preserve workflow\u002Fapplication overrides for datasets, specs, output directories, GPU\u002Fplatform settings, parent checkpoints, and `automl_policy`. Use direct model training only when `automl_policy: off` or the packaged train schema\u002Ftemplate is missing; in the missing-schema case, report that AutoML is enabled but not runnable for this model until schemas are generated.\n\nNon-train actions such as `evaluate`, `inference`, `export`, and deploy flows stay in this model skill. The per-run `automl_policy` override does not change model metadata.\n\n## Training Requirements\n\n- **Dataset type:** segmentation\n- **Formats:** odvg, coco, coco_raw\n- **Monitoring metric:** val_loss\n\n### Per-Action Dataset Requirements\n\n| Action | Spec Key | Source | Files | List? |\n|---|---|---|---|---|\n| evaluate | dataset.test_data_sources | eval_dataset | image_dir: images.tar.gz, json_file: annotations.json | No |\n| evaluate | dataset.test_data_sources.data_type | eval_dataset | OD | No |\n| inference | dataset.infer_data_sources | inference_dataset | image_dir: images.tar.gz, captions: text prompts | No |\n| inference | dataset.infer_data_sources.data_type | inference_dataset | OD | No |\n| quantize | dataset.train_data_sources | train_datasets | image_dir: images.tar.gz, json_file: annotations_odvg.jsonl, label_map: annotations_odvg_labelmap.json | Yes |\n| quantize | dataset.val_data_sources | eval_dataset | image_dir: images.tar.gz, json_file: annotations.json | No |\n| quantize | dataset.val_data_sources.data_type | eval_dataset | OD | No |\n| quantize | dataset.quant_calibration_data_sources | train_datasets | image_dir: images.tar.gz, json_file: annotations_odvg.jsonl, label_map: annotations_odvg_labelmap.json | No |\n| train | dataset.train_data_sources | train_datasets | image_dir: images.tar.gz, json_file: annotations_odvg.jsonl, label_map: annotations_odvg_labelmap.json | Yes |\n| train | dataset.val_data_sources | eval_dataset | image_dir: images.tar.gz, json_file: annotations.json | No |\n| train | dataset.val_data_sources.data_type | eval_dataset | OD | No |\n\n### Typical Spec Overrides\n\nData source overrides are **mandatory for every action** — the agent MUST construct data source paths from the Per-Action Dataset Requirements table above and include them in `spec_overrides`.\n\n```python\nS3_TRAIN = \"s3:\u002F\u002Fbucket\u002Fdata\u002Ftrain\"\nS3_EVAL = \"s3:\u002F\u002Fbucket\u002Fdata\u002Feval\"\n```\n\n**train (mandatory data sources):**\n```python\n{\n    \"train.num_gpus\": 1,\n    \"train.num_epochs\": 10,\n    \"train.checkpoint_interval\": 10,\n    \"train.validation_interval\": 10,\n    \"dataset.val_data_sources.data_type\": \"OD\",\n    \"model.num_region_queries\": 100,\n    \"dataset.train_data_sources\": [{\"image_dir\": f\"{S3_TRAIN}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_TRAIN}\u002Fannotations_odvg.jsonl\", \"label_map\": f\"{S3_TRAIN}\u002Fannotations_odvg_labelmap.json\"}],\n    \"dataset.val_data_sources\": {\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_EVAL}\u002Fannotations.json\"},\n}\n```\n\n**evaluate (mandatory data sources):**\n```python\n{\n    \"evaluate.checkpoint\": \"\u003Cselected train\u002FAutoML checkpoint>\",\n    \"dataset.test_data_sources.data_type\": \"OD\",\n    \"dataset.test_data_sources\": {\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_EVAL}\u002Fannotations.json\"},\n}\n```\n\n**inference (mandatory data sources):**\n```python\n{\n    \"inference.checkpoint\": \"\u003Cselected train\u002FAutoML checkpoint>\",\n    \"dataset.infer_data_sources.data_type\": \"OD\",\n    \"dataset.infer_data_sources\": {\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"captions\": [\"person\", \"bicycle\", \"car\"]},\n}\n```\n\n**quantize (mandatory data sources):**\n```python\n{\n    \"dataset.train_data_sources\": [{\"image_dir\": f\"{S3_TRAIN}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_TRAIN}\u002Fannotations_odvg.jsonl\", \"label_map\": f\"{S3_TRAIN}\u002Fannotations_odvg_labelmap.json\"}],\n    \"dataset.val_data_sources\": {\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_EVAL}\u002Fannotations.json\"},\n    \"dataset.quant_calibration_data_sources\": {\"image_dir\": f\"{S3_TRAIN}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_TRAIN}\u002Fannotations_odvg.jsonl\", \"label_map\": f\"{S3_TRAIN}\u002Fannotations_odvg_labelmap.json\"},\n}\n```\n## Eval Dataset\n\nOptional. Validation uses COCO-format annotations even when training uses ODVG.\n\n## Important Parameters\n\n- **model.backbone**: Default swin_tiny_224_1k. Same backbone options as Grounding DINO.\n- **train.optim.lr**: Learning rate. Default 2e-4. lr_backbone 2e-5. Reuses GDINOTrainExpConfig — same training setup as Grounding DINO.\n- **model.num_queries**: Object queries. Default 900.\n- **model.enc_layers \u002F model.dec_layers**: Keep both at 6 for train\u002FAutoML\n  runs. The mask head asserts six decoder outputs during validation, so\n  copying Grounding DINO smoke overrides that reduce transformer layers causes\n  an immediate failure.\n- **AutoML metric note**: Use `metric=\"val_loss\"` with\n  `direction=\"minimize\"` for train-stage AutoML. The packaged train loop logs\n  validation loss scalars; it does not emit `[bbox] val_mAP@50` during the\n  train job.\n- **model.has_mask**: Enables mask prediction head. Default True. Adds mask\u002Fdice\u002Frela loss coefficients.\n- **model.num_region_queries**: Number of region queries for mask prediction. Default 100.\n- **model.loss_types**: Loss components. Default [labels, boxes, masks]. Includes mask_loss_coef, dice_loss_coef, rela_loss_coef.\n- **evaluate.ioi_threshold**: IoI threshold for mask evaluation. Default 0.5.\n- **evaluate.nms_threshold**: NMS threshold. Default 0.2.\n- **evaluate.text_threshold**: Text matching threshold. Default 0.3.\n- **dataset.has_mask**: Dataset includes mask annotations. Default True. val_data_sources default data_type is \"VG\".\n\n## Multi-GPU \u002F Multi-Node\n\n**Launch method:** Lightning-managed. Same DDP\u002FFSDP behavior as Grounding DINO.\n\n| Spec Key | Description | Default |\n|----------|-------------|---------|\n| `train.num_gpus` | Number of GPUs | 1 |\n| `train.gpu_ids` | GPU device indices | [0] |\n| `train.num_nodes` | Number of nodes | 1 |\n| `train.distributed_strategy` | `ddp` or `fsdp` | `ddp` |\n\n## Hardware\n\nMinimum 1 GPU(s), recommended 4 GPU(s). 24GB+ (A100 recommended) VRAM per GPU. Heavier than Grounding DINO due to mask prediction head. 24GB+ GPU memory recommended.\n\n## Error Patterns\n\n**CUDA out of memory**: Reduce batch_size. Mask prediction adds overhead on top of Grounding DINO.\n\n**Deploy schema error for `test_threshold`**: TAO Deploy uses\n`evaluate.text_threshold` and `inference.text_threshold`. Do not use\n`test_threshold` in deploy specs.\n\n**Deploy model shape mismatch**: Carry transformer and mask structure fields\nfrom export into deploy evaluate\u002Finference specs, including `model.num_queries`,\n`model.num_select`, `model.max_text_len`, `model.num_region_queries`, and\n`model.has_mask`. These values must match the ONNX model used to build the\nTensorRT engine.\n\n## Spec Param \u002F Parent Model Inference\n\nModel-specific inference mappings belong in this MD file, not in `config.json`. Generated runners should read this section and apply the mappings with SDK helpers before `create_job()`. This mirrors the old microservices `infer_params.py` flow.\n\nInference mappings from TAO Core `mask_grounding_dino.config.json`:\n\n| Action | Spec Field | Inference Function | Meaning |\n|---|---|---|---|\n| evaluate | `encryption_key` | `key` | encryption key |\n| evaluate | `evaluate.checkpoint` | `parent_model` | model file inferred from the parent job results folder |\n| evaluate | `evaluate.trt_engine` | `parent_model` | model file inferred from the parent job results folder |\n| evaluate | `results_dir` | `output_dir` | current job results directory |\n| export | `encryption_key` | `key` | encryption key |\n| export | `export.checkpoint` | `parent_model` | model file inferred from the parent job results folder |\n| export | `export.onnx_file` | `create_onnx_file` | output ONNX path |\n| export | `results_dir` | `output_dir` | current job results directory |\n| gen_trt_engine | `encryption_key` | `key` | encryption key |\n| gen_trt_engine | `gen_trt_engine.onnx_file` | `parent_model` | model file inferred from the parent job results folder |\n| gen_trt_engine | `gen_trt_engine.trt_engine` | `create_engine_file` | output TensorRT engine path |\n| gen_trt_engine | `results_dir` | `output_dir` | current job results directory |\n| inference | `encryption_key` | `key` | encryption key |\n| inference | `inference.checkpoint` | `parent_model` | model file inferred from the parent job results folder |\n| inference | `inference.trt_engine` | `parent_model` | model file inferred from the parent job results folder |\n| inference | `results_dir` | `output_dir` | current job results directory |\n| quantize | `encryption_key` | `key` | encryption key |\n| quantize | `quantize.model_path` | `parent_model` | model file inferred from the parent job results folder |\n| quantize | `results_dir` | `output_dir` | current job results directory |\n| train | `encryption_key` | `key` | encryption key |\n| train | `model.pretrained_backbone_path` | `ptm_if_no_resume_model` | PTM when no resume checkpoint exists |\n| train | `results_dir` | `output_dir` | current job results directory |\n| train | `train.pretrained_model_path` | `ptm_if_no_resume_model` | PTM when no resume checkpoint exists |\n| train | `train.resume_training_checkpoint_path` | `resume_model` | model file inferred from the current job results folder |\n\nFor `parent_model` or `parent_model_folder`, pass the upstream train\u002Fexport\u002FAutoML child job id as `parent_job_id`. The SDK lists the parent result folder, filters checkpoint artifacts, and returns the selected model file or folder. Do not add these mappings back to `config.json` and do not patch generated runner scripts to guess checkpoint paths.\n\nWhen selecting a Mask Grounding DINO checkpoint outside the SDK resolver, match\nthe intended epoch\u002Fstep artifact exactly, for example\n`model_epoch_000_step_00049.pth`. The `mask_gdino_model_latest.pth` symlink is\nvalid only when latest is explicitly requested. The parent PyTorch\n`mask_grounding_dino` CLI supports `train`, `evaluate`, `inference`, `export`,\nand `quantize`; run TensorRT engine generation, TensorRT inference, and\nTensorRT evaluation through `references\u002Ftao-deploy-mask-grounding-dino.md`.\n\n## Deployment\n\n- [tao-deploy-mask-grounding-dino](references\u002Ftao-deploy-mask-grounding-dino.md)\n","# Mask Grounding DINO\n\nMask Grounding DINO for grounded instance segmentation. Extends Grounding DINO with mask prediction head for open-set segmentation guided by text prompts.\n\nSet train.pretrained_model_path for full model weights.\n\nFor TAO Deploy TensorRT actions (`gen_trt_engine`, TensorRT `evaluate`, and TensorRT `inference`), read `references\u002Ftao-deploy-mask-grounding-dino.md` first. Deploy spec templates live in this skill's `references\u002F` folder with the `spec_template_deploy_*.yaml` prefix.\n\n## Dataclass Schemas\n\nGenerated TAO Core schemas are packaged in `schemas\u002F\u003Caction>.schema.json`, with `schemas\u002Fmanifest.json` listing available actions. Each generated schema also emits `references\u002Fspec_template_\u003Caction>.yaml` from the schema top-level `default` field. AutoML enablement is declared at the model layer in `references\u002Fskill_info.yaml` via `automl_enabled`. Runnable AutoML still requires `schemas\u002Ftrain.schema.json` and `references\u002Fspec_template_train.yaml` to exist and parse. Use the packaged train schema for `automl_default_parameters`, `automl_disabled_parameters`, defaults, min\u002Fmax bounds, enums, option weights, math conditions, dependencies, and popular parameters. Do not expect `~\u002Ftao-core` at runtime; maintainers regenerate schemas\u002Ftemplates before packaging the skill bank.\n\n## Train Action Policy\n\nThis model is AutoML-enabled at the model layer. Before handling any train-stage request, read `references\u002Fskill_info.yaml` and resolve the run override from either an explicit `automl_policy` value or the user's workflow request. Use `automl_policy: on` by default and only expose `on` \u002F `off` in new launch prompts. Treat phrases like \"turn off AutoML\", \"disable AutoML\", \"no HPO\", or \"plain training\" as `automl_policy: off` for this run only. When `automl_policy: on`, `automl_enabled: true`, and both `schemas\u002Ftrain.schema.json` and `references\u002Fspec_template_train.yaml` are packaged, route the train action through `tao-skill-bank:tao-run-automl` by default with this model's `skill_dir`. Preserve workflow\u002Fapplication overrides for datasets, specs, output directories, GPU\u002Fplatform settings, parent checkpoints, and `automl_policy`. Use direct model training only when `automl_policy: off` or the packaged train schema\u002Ftemplate is missing; in the missing-schema case, report that AutoML is enabled but not runnable for this model until schemas are generated.\n\nNon-train actions such as `evaluate`, `inference`, `export`, and deploy flows stay in this model skill. The per-run `automl_policy` override does not change model metadata.\n\n## Training Requirements\n\n- **Dataset type:** segmentation\n- **Formats:** odvg, coco, coco_raw\n- **Monitoring metric:** val_loss\n\n### Per-Action Dataset Requirements\n\n| Action | Spec Key | Source | Files | List? |\n|---|---|---|---|---|\n| evaluate | dataset.test_data_sources | eval_dataset | image_dir: images.tar.gz, json_file: annotations.json | No |\n| evaluate | dataset.test_data_sources.data_type | eval_dataset | OD | No |\n| inference | dataset.infer_data_sources | inference_dataset | image_dir: images.tar.gz, captions: text prompts | No |\n| inference | dataset.infer_data_sources.data_type | inference_dataset | OD | No |\n| quantize | dataset.train_data_sources | train_datasets | image_dir: images.tar.gz, json_file: annotations_odvg.jsonl, label_map: annotations_odvg_labelmap.json | Yes |\n| quantize | dataset.val_data_sources | eval_dataset | image_dir: images.tar.gz, json_file: annotations.json | No |\n| quantize | dataset.val_data_sources.data_type | eval_dataset | OD | No |\n| quantize | dataset.quant_calibration_data_sources | train_datasets | image_dir: images.tar.gz, json_file: annotations_odvg.jsonl, label_map: annotations_odvg_labelmap.json | No |\n| train | dataset.train_data_sources | train_datasets | image_dir: images.tar.gz, json_file: annotations_odvg.jsonl, label_map: annotations_odvg_labelmap.json | Yes |\n| train | dataset.val_data_sources | eval_dataset | image_dir: images.tar.gz, json_file: annotations.json | No |\n| train | dataset.val_data_sources.data_type | eval_dataset | OD | No |\n\n### Typical Spec Overrides\n\nData source overrides are **mandatory for every action** — the agent MUST construct data source paths from the Per-Action Dataset Requirements table above and include them in `spec_overrides`.\n\n```python\nS3_TRAIN = \"s3:\u002F\u002Fbucket\u002Fdata\u002Ftrain\"\nS3_EVAL = \"s3:\u002F\u002Fbucket\u002Fdata\u002Feval\"\n```\n\n**train (mandatory data sources):**\n```python\n{\n    \"train.num_gpus\": 1,\n    \"train.num_epochs\": 10,\n    \"train.checkpoint_interval\": 10,\n    \"train.validation_interval\": 10,\n    \"dataset.val_data_sources.data_type\": \"OD\",\n    \"model.num_region_queries\": 100,\n    \"dataset.train_data_sources\": [{\"image_dir\": f\"{S3_TRAIN}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_TRAIN}\u002Fannotations_odvg.jsonl\", \"label_map\": f\"{S3_TRAIN}\u002Fannotations_odvg_labelmap.json\"}],\n    \"dataset.val_data_sources\": {\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_EVAL}\u002Fannotations.json\"},\n}\n```\n\n**evaluate (mandatory data sources):**\n```python\n{\n    \"evaluate.checkpoint\": \"\u003Cselected train\u002FAutoML checkpoint>\",\n    \"dataset.test_data_sources.data_type\": \"OD\",\n    \"dataset.test_data_sources\": {\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_EVAL}\u002Fannotations.json\"},\n}\n```\n\n**inference (mandatory data sources):**\n```python\n{\n    \"inference.checkpoint\": \"\u003Cselected train\u002FAutoML checkpoint>\",\n    \"dataset.infer_data_sources.data_type\": \"OD\",\n    \"dataset.infer_data_sources\": {\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"captions\": [\"person\", \"bicycle\", \"car\"]},\n}\n```\n\n**quantize (mandatory data sources):**\n```python\n{\n    \"dataset.train_data_sources\": [{\"image_dir\": f\"{S3_TRAIN}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_TRAIN}\u002Fannotations_odvg.jsonl\", \"label_map\": f\"{S3_TRAIN}\u002Fannotations_odvg_labelmap.json\"}],\n    \"dataset.val_data_sources\": {\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_EVAL}\u002Fannotations.json\"},\n    \"dataset.quant_calibration_data_sources\": {\"image_dir\": f\"{S3_TRAIN}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_TRAIN}\u002Fannotations_odvg.jsonl\", \"label_map\": f\"{S3_TRAIN}\u002Fannotations_odvg_labelmap.json\"},\n}\n```\n## Eval Dataset\n\nOptional. Validation uses COCO-format annotations even when training uses ODVG.\n\n## Important Parameters\n\n- **model.backbone**: Default swin_tiny_224_1k. Same backbone options as Grounding DINO.\n- **train.optim.lr**: Learning rate. Default 2e-4. lr_backbone 2e-5. Reuses GDINOTrainExpConfig — same training setup as Grounding DINO.\n- **model.num_queries**: Object queries. Default 900.\n- **model.enc_layers \u002F model.dec_layers**: Keep both at 6 for train\u002FAutoML\n  runs. The mask head asserts six decoder outputs during validation, so\n  copying Grounding DINO smoke overrides that reduce transformer layers causes\n  an immediate failure.\n- **AutoML metric note**: Use `metric=\"val_loss\"` with\n  `direction=\"minimize\"` for train-stage AutoML. The packaged train loop logs\n  validation loss scalars; it does not emit `[bbox] val_mAP@50` during the\n  train job.\n- **model.has_mask**: Enables mask prediction head. Default True. Adds mask\u002Fdice\u002Frela loss coefficients.\n- **model.num_region_queries**: Number of region queries for mask prediction. Default 100.\n- **model.loss_types**: Loss components. Default [labels, boxes, masks]. Includes mask_loss_coef, dice_loss_coef, rela_loss_coef.\n- **evaluate.ioi_threshold**: IoI threshold for mask evaluation. Default 0.5.\n- **evaluate.nms_threshold**: NMS threshold. Default 0.2.\n- **evaluate.text_threshold**: Text matching threshold. Default 0.3.\n- **dataset.has_mask**: Dataset includes mask annotations. Default True. val_data_sources default data_type is \"VG\".\n\n## Multi-GPU \u002F Multi-Node\n\n**Launch method:** Lightning-managed. Same DDP\u002FFSDP behavior as Grounding DINO.\n\n| Spec Key | Description | Default |\n|----------|-------------|---------|\n| `train.num_gpus` | Number of GPUs | 1 |\n| `train.gpu_ids` | GPU device indices | [0] |\n| `train.num_nodes` | Number of nodes | 1 |\n| `train.distributed_strategy` | `ddp` or `fsdp` | `ddp` |\n\n## Hardware\n\nMinimum 1 GPU(s), recommended 4 GPU(s). 24GB+ (A100 recommended) VRAM per GPU. Heavier than Grounding DINO due to mask prediction head. 24GB+ GPU memory recommended.\n\n## Error Patterns\n\n**CUDA out of memory**: Reduce batch_size. Mask prediction adds overhead on top of Grounding DINO.\n\n**Deploy schema error for `test_threshold`**: TAO Deploy uses\n`evaluate.text_threshold` and `inference.text_threshold`. Do not use\n`test_threshold` in deploy specs.\n\n**Deploy model shape mismatch**: Carry transformer and mask structure fields\nfrom export into deploy evaluate\u002Finference specs, including `model.num_queries`,\n`model.num_select`, `model.max_text_len`, `model.num_region_queries`, and\n`model.has_mask`. These values must match the ONNX model used to build the\nTensorRT engine.\n\n## Spec Param \u002F Parent Model Inference\n\nModel-specific inference mappings belong in this MD file, not in `config.json`. Generated runners should read this section and apply the mappings with SDK helpers before `create_job()`. This mirrors the old microservices `infer_params.py` flow.\n\nInference mappings from TAO Core `mask_grounding_dino.config.json`:\n\n| Action | Spec Field | Inference Function | Meaning |\n|---|---|---|---|\n| evaluate | `encryption_key` | `key` | encryption key |\n| evaluate | `evaluate.checkpoint` | `parent_model` | model file inferred from the parent job results folder |\n| evaluate | `evaluate.trt_engine` | `parent_model` | model file inferred from the parent job results folder |\n| evaluate | `results_dir` | `output_dir` | current job results directory |\n| export | `encryption_key` | `key` | encryption key |\n| export | `export.checkpoint` | `parent_model` | model file inferred from the parent job results folder |\n| export | `export.onnx_file` | `create_onnx_file` | output ONNX path |\n| export | `results_dir` | `output_dir` | current job results directory |\n| gen_trt_engine | `encryption_key` | `key` | encryption key |\n| gen_trt_engine | `gen_trt_engine.onnx_file` | `parent_model` | model file inferred from the parent job results folder |\n| gen_trt_engine | `gen_trt_engine.trt_engine` | `create_engine_file` | output TensorRT engine path |\n| gen_trt_engine | `results_dir` | `output_dir` | current job results directory |\n| inference | `encryption_key` | `key` | encryption key |\n| inference | `inference.checkpoint` | `parent_model` | model file inferred from the parent job results folder |\n| inference | `inference.trt_engine` | `parent_model` | model file inferred from the parent job results folder |\n| inference | `results_dir` | `output_dir` | current job results directory |\n| quantize | `encryption_key` | `key` | encryption key |\n| quantize | `quantize.model_path` | `parent_model` | model file inferred from the parent job results folder |\n| quantize | `results_dir` | `output_dir` | current job results directory |\n| train | `encryption_key` | `key` | encryption key |\n| train | `model.pretrained_backbone_path` | `ptm_if_no_resume_model` | PTM when no resume checkpoint exists |\n| train | `results_dir` | `output_dir` | current job results directory |\n| train | `train.pretrained_model_path` | `ptm_if_no_resume_model` | PTM when no resume checkpoint exists |\n| train | `train.resume_training_checkpoint_path` | `resume_model` | model file inferred from the current job results folder |\n\nFor `parent_model` or `parent_model_folder`, pass the upstream train\u002Fexport\u002FAutoML child job id as `parent_job_id`. The SDK lists the parent result folder, filters checkpoint artifacts, and returns the selected model file or folder. Do not add these mappings back to `config.json` and do not patch generated runner scripts to guess checkpoint paths.\n\nWhen selecting a Mask Grounding DINO checkpoint outside the SDK resolver, match\nthe intended epoch\u002Fstep artifact exactly, for example\n`model_epoch_000_step_00049.pth`. The `mask_gdino_model_latest.pth` symlink is\nvalid only when latest is explicitly requested. The parent PyTorch\n`mask_grounding_dino` CLI supports `train`, `evaluate`, `inference`, `export`,\nand `quantize`; run TensorRT engine generation, TensorRT inference, and\nTensorRT evaluation through `references\u002Ftao-deploy-mask-grounding-dino.md`.\n\n## Deployment\n\n- [tao-deploy-mask-grounding-dino](references\u002Ftao-deploy-mask-grounding-dino.md)\n",{"name":5,"tags":23,"license":14,"metadata":25,"description":7,"allowed-tools":28,"compatibility":29},[24],"segmentation",{"author":26,"version":27},"NVIDIA Corporation","0.1.0","Read Bash","Requires docker + nvidia-container-toolkit.","main","fc4f2c7759ae400c66cd38656db70ead882a0d08","catalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino","https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent","https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent#readme","https:\u002F\u002Fgithub.com\u002Fmatheussluzz","2026-08-27 05:21:51",{"source":8,"skillName":5,"commands":38,"prompt":48,"zipAvailable":49},[39,42,45],{"tool":40,"command":41},"npx","npx skills add matheussluzz\u002FZeuZ-Agent --skill tao-train-mask-grounding-dino",{"tool":43,"command":44},"bunx","bunx skills add matheussluzz\u002FZeuZ-Agent --skill tao-train-mask-grounding-dino",{"tool":46,"command":47},"pnpm","pnpm dlx skills add matheussluzz\u002FZeuZ-Agent --skill tao-train-mask-grounding-dino","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002FSKILL.md",true,[51,57,61,65,69,73,77,81,85,89,93,97,101,105,109,113,117,121,125,129,133,137,143],{"path":52,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":55,"sourceHtmlUrl":56},"BENCHMARK.md","REFERENCE","参考文档",4136,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002FBENCHMARK.md",{"path":58,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":59,"sourceHtmlUrl":60},"references\u002Fskill_info.yaml",6309,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fskill_info.yaml",{"path":62,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":63,"sourceHtmlUrl":64},"references\u002Fspec_template_deploy_evaluate.yaml",567,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fspec_template_deploy_evaluate.yaml",{"path":66,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":67,"sourceHtmlUrl":68},"references\u002Fspec_template_deploy_gen_trt_engine.yaml",365,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fspec_template_deploy_gen_trt_engine.yaml",{"path":70,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":71,"sourceHtmlUrl":72},"references\u002Fspec_template_deploy_inference.yaml",1651,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fspec_template_deploy_inference.yaml",{"path":74,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":75,"sourceHtmlUrl":76},"references\u002Fspec_template_evaluate.yaml",3612,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fspec_template_evaluate.yaml",{"path":78,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":79,"sourceHtmlUrl":80},"references\u002Fspec_template_export.yaml",3645,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fspec_template_export.yaml",{"path":82,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":83,"sourceHtmlUrl":84},"references\u002Fspec_template_gen_trt_engine.yaml",3682,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fspec_template_gen_trt_engine.yaml",{"path":86,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":87,"sourceHtmlUrl":88},"references\u002Fspec_template_inference.yaml",3692,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fspec_template_inference.yaml",{"path":90,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":91,"sourceHtmlUrl":92},"references\u002Fspec_template_quantize.yaml",3418,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fspec_template_quantize.yaml",{"path":94,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":95,"sourceHtmlUrl":96},"references\u002Fspec_template_train.yaml",3400,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fspec_template_train.yaml",{"path":98,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":99,"sourceHtmlUrl":100},"references\u002Ftao-deploy-mask-grounding-dino.md",5683,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Ftao-deploy-mask-grounding-dino.md",{"path":102,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":103,"sourceHtmlUrl":104},"references\u002Ftao-deploy-mask-grounding-dino.skill_info.yaml",3395,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Ftao-deploy-mask-grounding-dino.skill_info.yaml",{"path":106,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":107,"sourceHtmlUrl":108},"schemas\u002Fevaluate.schema.json",61617,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Fschemas\u002Fevaluate.schema.json",{"path":110,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":111,"sourceHtmlUrl":112},"schemas\u002Fexport.schema.json",61110,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Fschemas\u002Fexport.schema.json",{"path":114,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":115,"sourceHtmlUrl":116},"schemas\u002Fgen_trt_engine.schema.json",62875,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Fschemas\u002Fgen_trt_engine.schema.json",{"path":118,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":119,"sourceHtmlUrl":120},"schemas\u002Finference.schema.json",62557,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Fschemas\u002Finference.schema.json",{"path":122,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":123,"sourceHtmlUrl":124},"schemas\u002Fmanifest.json",16388,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Fschemas\u002Fmanifest.json",{"path":126,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":127,"sourceHtmlUrl":128},"schemas\u002Fquantize.schema.json",57170,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Fschemas\u002Fquantize.schema.json",{"path":130,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":131,"sourceHtmlUrl":132},"schemas\u002Ftrain.schema.json",57164,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Fschemas\u002Ftrain.schema.json",{"path":134,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":135,"sourceHtmlUrl":136},"skill-card.md",3794,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Fskill-card.md",{"path":138,"resourceType":139,"resourceTypeDesc":140,"sizeBytes":141,"sourceHtmlUrl":142},"skill.oms.sig","OTHER","其他",9401,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Fskill.oms.sig",{"path":144,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":145,"sourceHtmlUrl":146},"zeuz.manifest.yaml",172,"https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fcatalog\u002Fbundles\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Fzeuz.manifest.yaml",[148,156,163,170,177,184],{"skillNo":149,"name":150,"nameZh":6,"description":151,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":152,"licenseFileUrl":6,"sourceHtmlUrl":153,"categories":154,"keywords":155,"publishedAt":18},"01M78JQ2R0QR8H2Z6662RFX804","gamma","Fixture skill gamma for discovery benchmarks.","MIT","https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Ftest\u002Ffixtures\u002Fskill-registry-mini\u002Fskills\u002Fgamma\u002FSKILL.md",[],[],{"skillNo":157,"name":158,"nameZh":6,"description":159,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":152,"licenseFileUrl":6,"sourceHtmlUrl":160,"categories":161,"keywords":162,"publishedAt":18},"01M78JQ1KCVZ15APZG3FFF29K0","beta","Fixture skill beta for discovery benchmarks.","https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Ftest\u002Ffixtures\u002Fskill-registry-mini\u002Fskills\u002Fbeta\u002FSKILL.md",[],[],{"skillNo":164,"name":165,"nameZh":6,"description":166,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":152,"licenseFileUrl":6,"sourceHtmlUrl":167,"categories":168,"keywords":169,"publishedAt":18},"01M78JQ0FJTK4Q9JKZMX9SVSEN","alpha","Fixture skill alpha for discovery benchmarks.","https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Ftest\u002Ffixtures\u002Fskill-registry-mini\u002Fskills\u002Falpha\u002FSKILL.md",[],[],{"skillNo":171,"name":172,"nameZh":6,"description":173,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":152,"licenseFileUrl":6,"sourceHtmlUrl":174,"categories":175,"keywords":176,"publishedAt":18},"01M78JPYVYPY7R8QS10KADQPWE","writing-for-agents","Writing documents for agents. Use when creating or editing skills, or modifying AGENTS.md or CLAUDE.md.","https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fskills\u002Fwriting-for-agents\u002FSKILL.md",[],[],{"skillNo":178,"name":179,"nameZh":6,"description":180,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":152,"licenseFileUrl":6,"sourceHtmlUrl":181,"categories":182,"keywords":183,"publishedAt":18},"01M78JPVYHB88RF5ATZVGX84RR","wizard","Generate an interactive bash wizard that walks a human through steps only they can perform. Use when provisioning infrastructure, setting up credentials or CI secrets, walking an unfamiliar third-party dashboard, or running a one-off migration or cutover. Don't invoke this for steps the agent can perform itself.","https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fskills\u002Fwizard\u002FSKILL.md",[],[],{"skillNo":185,"name":186,"nameZh":6,"description":187,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":152,"licenseFileUrl":6,"sourceHtmlUrl":188,"categories":189,"keywords":190,"publishedAt":18},"01M78JPSSWNTZW89DWGJTX7716","wayfinder","Plan a huge chunk of work (more than one agent session can hold) as a shared map of decision tickets on your issue tracker, and resolve them one at a time until the way to the destination is clear.","https:\u002F\u002Fgithub.com\u002Fmatheussluzz\u002FZeuZ-Agent\u002Fblob\u002Fmain\u002Fskills\u002Fwayfinder\u002FSKILL.md",[],[]]