[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M78AX0G0C58WJG2G15N3FDG2":3,"skill-same:01M78AX0G0C58WJG2G15N3FDG2":120},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":31,"commitSha":32,"skillDirectory":33,"forksCount":14,"repositoryHtmlUrl":34,"repositoryHomepageUrl":35,"ownerHtmlUrl":36,"ownerBlogUrl":37,"ownerDescription":38,"updatedAt":39,"install":40,"resources":53},"01M78AX0G0C58WJG2G15N3FDG2","tao-train-mask-grounding-dino",null,"Mask Grounding DINO for grounded instance segmentation. Extends Grounding DINO with a mask-prediction head for open-set segmentation guided by text prompts. Use when training, evaluating, exporting, quantizing, or running inference for a TAO Mask-Grounding-DINO model. Trigger phrases include \"train Mask Grounding DINO\", \"open-vocabulary segmentation\", \"text-prompted instance segmentation\", \"grounded mask DETR\".","Solizardking\u002Fskillhub-main","Solizardking","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F108096810?v=4","01M786KDB10AYV042N0F9BD5B5","Clawd","01M786NYYXSZK1D05WM8JQPC48",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: tao-train-mask-grounding-dino\ndescription: Mask Grounding DINO for grounded instance segmentation. Extends Grounding DINO with a mask-prediction head for\n  open-set segmentation guided by text prompts. Use when training, evaluating, exporting, quantizing, or running inference for\n  a TAO Mask-Grounding-DINO model. Trigger phrases include \"train Mask Grounding DINO\", \"open-vocabulary segmentation\",\n  \"text-prompted instance segmentation\", \"grounded mask DETR\".\nlicense: Apache-2.0\ncompatibility: Requires docker + nvidia-container-toolkit.\nmetadata:\n  version: \"0.1.0\"\n  author: NVIDIA Corporation\nallowed-tools: Read Bash\ntags:\n- segmentation\n---\n\n# Mask Grounding DINO\n\nMask Grounding DINO for grounded instance segmentation. Extends Grounding DINO with mask prediction head for open-set segmentation guided by text prompts.\n\nSet train.pretrained_model_path for full model weights.\n\nFor TAO Deploy TensorRT actions (`gen_trt_engine`, TensorRT `evaluate`, and TensorRT `inference`), read `references\u002Ftao-deploy-mask-grounding-dino.md` first. Deploy spec templates live in this skill's `references\u002F` folder with the `spec_template_deploy_*.yaml` prefix.\n\n## Dataclass Schemas\n\nGenerated TAO Core schemas are packaged in `schemas\u002F\u003Caction>.schema.json`, with `schemas\u002Fmanifest.json` listing available actions. Each generated schema also emits `references\u002Fspec_template_\u003Caction>.yaml` from the schema top-level `default` field. AutoML enablement is declared at the model layer in `references\u002Fskill_info.yaml` via `automl_enabled`. Runnable AutoML still requires `schemas\u002Ftrain.schema.json` and `references\u002Fspec_template_train.yaml` to exist and parse. Use the packaged train schema for `automl_default_parameters`, `automl_disabled_parameters`, defaults, min\u002Fmax bounds, enums, option weights, math conditions, dependencies, and popular parameters. Do not expect `~\u002Ftao-core` at runtime; maintainers regenerate schemas\u002Ftemplates before packaging the skill bank.\n\n## Train Action Policy\n\nThis model is AutoML-enabled at the model layer. Before handling any train-stage request, read `references\u002Fskill_info.yaml` and resolve the run override from either an explicit `automl_policy` value or the user's workflow request. Use `automl_policy: on` by default and only expose `on` \u002F `off` in new launch prompts. Treat phrases like \"turn off AutoML\", \"disable AutoML\", \"no HPO\", or \"plain training\" as `automl_policy: off` for this run only. When `automl_policy: on`, `automl_enabled: true`, and both `schemas\u002Ftrain.schema.json` and `references\u002Fspec_template_train.yaml` are packaged, route the train action through `tao-skill-bank:tao-run-automl` by default with this model's `skill_dir`. Preserve workflow\u002Fapplication overrides for datasets, specs, output directories, GPU\u002Fplatform settings, parent checkpoints, and `automl_policy`. Use direct model training only when `automl_policy: off` or the packaged train schema\u002Ftemplate is missing; in the missing-schema case, report that AutoML is enabled but not runnable for this model until schemas are generated.\n\nNon-train actions such as `evaluate`, `inference`, `export`, and deploy flows stay in this model skill. The per-run `automl_policy` override does not change model metadata.\n\n## Training Requirements\n\n- **Dataset type:** segmentation\n- **Formats:** odvg, coco, coco_raw\n- **Monitoring metric:** val_loss\n\n### Per-Action Dataset Requirements\n\n| Action | Spec Key | Source | Files | List? |\n|---|---|---|---|---|\n| evaluate | dataset.test_data_sources | eval_dataset | image_dir: images.tar.gz, json_file: annotations.json | No |\n| evaluate | dataset.test_data_sources.data_type | eval_dataset | OD | No |\n| inference | dataset.infer_data_sources | inference_dataset | image_dir: images.tar.gz, captions: text prompts | No |\n| inference | dataset.infer_data_sources.data_type | inference_dataset | OD | No |\n| quantize | dataset.train_data_sources | train_datasets | image_dir: images.tar.gz, json_file: annotations_odvg.jsonl, label_map: annotations_odvg_labelmap.json | Yes |\n| quantize | dataset.val_data_sources | eval_dataset | image_dir: images.tar.gz, json_file: annotations.json | No |\n| quantize | dataset.val_data_sources.data_type | eval_dataset | OD | No |\n| quantize | dataset.quant_calibration_data_sources | train_datasets | image_dir: images.tar.gz, json_file: annotations_odvg.jsonl, label_map: annotations_odvg_labelmap.json | No |\n| train | dataset.train_data_sources | train_datasets | image_dir: images.tar.gz, json_file: annotations_odvg.jsonl, label_map: annotations_odvg_labelmap.json | Yes |\n| train | dataset.val_data_sources | eval_dataset | image_dir: images.tar.gz, json_file: annotations.json | No |\n| train | dataset.val_data_sources.data_type | eval_dataset | OD | No |\n\n### Typical Spec Overrides\n\nData source overrides are **mandatory for every action** — the agent MUST construct data source paths from the Per-Action Dataset Requirements table above and include them in `spec_overrides`.\n\n```python\nS3_TRAIN = \"s3:\u002F\u002Fbucket\u002Fdata\u002Ftrain\"\nS3_EVAL = \"s3:\u002F\u002Fbucket\u002Fdata\u002Feval\"\n```\n\n**train (mandatory data sources):**\n```python\n{\n    \"train.num_gpus\": 1,\n    \"train.num_epochs\": 10,\n    \"train.checkpoint_interval\": 10,\n    \"train.validation_interval\": 10,\n    \"dataset.val_data_sources.data_type\": \"OD\",\n    \"model.num_region_queries\": 100,\n    \"dataset.train_data_sources\": [{\"image_dir\": f\"{S3_TRAIN}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_TRAIN}\u002Fannotations_odvg.jsonl\", \"label_map\": f\"{S3_TRAIN}\u002Fannotations_odvg_labelmap.json\"}],\n    \"dataset.val_data_sources\": {\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_EVAL}\u002Fannotations.json\"},\n}\n```\n\n**evaluate (mandatory data sources):**\n```python\n{\n    \"evaluate.checkpoint\": \"\u003Cselected train\u002FAutoML checkpoint>\",\n    \"dataset.test_data_sources.data_type\": \"OD\",\n    \"dataset.test_data_sources\": {\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_EVAL}\u002Fannotations.json\"},\n}\n```\n\n**inference (mandatory data sources):**\n```python\n{\n    \"inference.checkpoint\": \"\u003Cselected train\u002FAutoML checkpoint>\",\n    \"dataset.infer_data_sources.data_type\": \"OD\",\n    \"dataset.infer_data_sources\": {\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"captions\": [\"person\", \"bicycle\", \"car\"]},\n}\n```\n\n**quantize (mandatory data sources):**\n```python\n{\n    \"dataset.train_data_sources\": [{\"image_dir\": f\"{S3_TRAIN}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_TRAIN}\u002Fannotations_odvg.jsonl\", \"label_map\": f\"{S3_TRAIN}\u002Fannotations_odvg_labelmap.json\"}],\n    \"dataset.val_data_sources\": {\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_EVAL}\u002Fannotations.json\"},\n    \"dataset.quant_calibration_data_sources\": {\"image_dir\": f\"{S3_TRAIN}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_TRAIN}\u002Fannotations_odvg.jsonl\", \"label_map\": f\"{S3_TRAIN}\u002Fannotations_odvg_labelmap.json\"},\n}\n```\n## Eval Dataset\n\nOptional. Validation uses COCO-format annotations even when training uses ODVG.\n\n## Important Parameters\n\n- **model.backbone**: Default swin_tiny_224_1k. Same backbone options as Grounding DINO.\n- **train.optim.lr**: Learning rate. Default 2e-4. lr_backbone 2e-5. Reuses GDINOTrainExpConfig — same training setup as Grounding DINO.\n- **model.num_queries**: Object queries. Default 900.\n- **model.enc_layers \u002F model.dec_layers**: Keep both at 6 for train\u002FAutoML\n  runs. The mask head asserts six decoder outputs during validation, so\n  copying Grounding DINO smoke overrides that reduce transformer layers causes\n  an immediate failure.\n- **AutoML metric note**: Use `metric=\"val_loss\"` with\n  `direction=\"minimize\"` for train-stage AutoML. The packaged train loop logs\n  validation loss scalars; it does not emit `[bbox] val_mAP@50` during the\n  train job.\n- **model.has_mask**: Enables mask prediction head. Default True. Adds mask\u002Fdice\u002Frela loss coefficients.\n- **model.num_region_queries**: Number of region queries for mask prediction. Default 100.\n- **model.loss_types**: Loss components. Default [labels, boxes, masks]. Includes mask_loss_coef, dice_loss_coef, rela_loss_coef.\n- **evaluate.ioi_threshold**: IoI threshold for mask evaluation. Default 0.5.\n- **evaluate.nms_threshold**: NMS threshold. Default 0.2.\n- **evaluate.text_threshold**: Text matching threshold. Default 0.3.\n- **dataset.has_mask**: Dataset includes mask annotations. Default True. val_data_sources default data_type is \"VG\".\n\n## Multi-GPU \u002F Multi-Node\n\n**Launch method:** Lightning-managed. Same DDP\u002FFSDP behavior as Grounding DINO.\n\n| Spec Key | Description | Default |\n|----------|-------------|---------|\n| `train.num_gpus` | Number of GPUs | 1 |\n| `train.gpu_ids` | GPU device indices | [0] |\n| `train.num_nodes` | Number of nodes | 1 |\n| `train.distributed_strategy` | `ddp` or `fsdp` | `ddp` |\n\n## Hardware\n\nMinimum 1 GPU(s), recommended 4 GPU(s). 24GB+ (A100 recommended) VRAM per GPU. Heavier than Grounding DINO due to mask prediction head. 24GB+ GPU memory recommended.\n\n## Error Patterns\n\n**CUDA out of memory**: Reduce batch_size. Mask prediction adds overhead on top of Grounding DINO.\n\n**Deploy schema error for `test_threshold`**: TAO Deploy uses\n`evaluate.text_threshold` and `inference.text_threshold`. Do not use\n`test_threshold` in deploy specs.\n\n**Deploy model shape mismatch**: Carry transformer and mask structure fields\nfrom export into deploy evaluate\u002Finference specs, including `model.num_queries`,\n`model.num_select`, `model.max_text_len`, `model.num_region_queries`, and\n`model.has_mask`. These values must match the ONNX model used to build the\nTensorRT engine.\n\n## Spec Param \u002F Parent Model Inference\n\nModel-specific inference mappings belong in this MD file, not in `config.json`. Generated runners should read this section and apply the mappings with SDK helpers before `create_job()`. This mirrors the old microservices `infer_params.py` flow.\n\nInference mappings from TAO Core `mask_grounding_dino.config.json`:\n\n| Action | Spec Field | Inference Function | Meaning |\n|---|---|---|---|\n| evaluate | `encryption_key` | `key` | encryption key |\n| evaluate | `evaluate.checkpoint` | `parent_model` | model file inferred from the parent job results folder |\n| evaluate | `evaluate.trt_engine` | `parent_model` | model file inferred from the parent job results folder |\n| evaluate | `results_dir` | `output_dir` | current job results directory |\n| export | `encryption_key` | `key` | encryption key |\n| export | `export.checkpoint` | `parent_model` | model file inferred from the parent job results folder |\n| export | `export.onnx_file` | `create_onnx_file` | output ONNX path |\n| export | `results_dir` | `output_dir` | current job results directory |\n| gen_trt_engine | `encryption_key` | `key` | encryption key |\n| gen_trt_engine | `gen_trt_engine.onnx_file` | `parent_model` | model file inferred from the parent job results folder |\n| gen_trt_engine | `gen_trt_engine.trt_engine` | `create_engine_file` | output TensorRT engine path |\n| gen_trt_engine | `results_dir` | `output_dir` | current job results directory |\n| inference | `encryption_key` | `key` | encryption key |\n| inference | `inference.checkpoint` | `parent_model` | model file inferred from the parent job results folder |\n| inference | `inference.trt_engine` | `parent_model` | model file inferred from the parent job results folder |\n| inference | `results_dir` | `output_dir` | current job results directory |\n| quantize | `encryption_key` | `key` | encryption key |\n| quantize | `quantize.model_path` | `parent_model` | model file inferred from the parent job results folder |\n| quantize | `results_dir` | `output_dir` | current job results directory |\n| train | `encryption_key` | `key` | encryption key |\n| train | `model.pretrained_backbone_path` | `ptm_if_no_resume_model` | PTM when no resume checkpoint exists |\n| train | `results_dir` | `output_dir` | current job results directory |\n| train | `train.pretrained_model_path` | `ptm_if_no_resume_model` | PTM when no resume checkpoint exists |\n| train | `train.resume_training_checkpoint_path` | `resume_model` | model file inferred from the current job results folder |\n\nFor `parent_model` or `parent_model_folder`, pass the upstream train\u002Fexport\u002FAutoML child job id as `parent_job_id`. The SDK lists the parent result folder, filters checkpoint artifacts, and returns the selected model file or folder. Do not add these mappings back to `config.json` and do not patch generated runner scripts to guess checkpoint paths.\n\nWhen selecting a Mask Grounding DINO checkpoint outside the SDK resolver, match\nthe intended epoch\u002Fstep artifact exactly, for example\n`model_epoch_000_step_00049.pth`. The `mask_gdino_model_latest.pth` symlink is\nvalid only when latest is explicitly requested. The parent PyTorch\n`mask_grounding_dino` CLI supports `train`, `evaluate`, `inference`, `export`,\nand `quantize`; run TensorRT engine generation, TensorRT inference, and\nTensorRT evaluation through `references\u002Ftao-deploy-mask-grounding-dino.md`.\n\n## Deployment\n\n- [tao-deploy-mask-grounding-dino](references\u002Ftao-deploy-mask-grounding-dino.md)\n","# Mask Grounding DINO\n\nMask Grounding DINO for grounded instance segmentation. Extends Grounding DINO with mask prediction head for open-set segmentation guided by text prompts.\n\nSet train.pretrained_model_path for full model weights.\n\nFor TAO Deploy TensorRT actions (`gen_trt_engine`, TensorRT `evaluate`, and TensorRT `inference`), read `references\u002Ftao-deploy-mask-grounding-dino.md` first. Deploy spec templates live in this skill's `references\u002F` folder with the `spec_template_deploy_*.yaml` prefix.\n\n## Dataclass Schemas\n\nGenerated TAO Core schemas are packaged in `schemas\u002F\u003Caction>.schema.json`, with `schemas\u002Fmanifest.json` listing available actions. Each generated schema also emits `references\u002Fspec_template_\u003Caction>.yaml` from the schema top-level `default` field. AutoML enablement is declared at the model layer in `references\u002Fskill_info.yaml` via `automl_enabled`. Runnable AutoML still requires `schemas\u002Ftrain.schema.json` and `references\u002Fspec_template_train.yaml` to exist and parse. Use the packaged train schema for `automl_default_parameters`, `automl_disabled_parameters`, defaults, min\u002Fmax bounds, enums, option weights, math conditions, dependencies, and popular parameters. Do not expect `~\u002Ftao-core` at runtime; maintainers regenerate schemas\u002Ftemplates before packaging the skill bank.\n\n## Train Action Policy\n\nThis model is AutoML-enabled at the model layer. Before handling any train-stage request, read `references\u002Fskill_info.yaml` and resolve the run override from either an explicit `automl_policy` value or the user's workflow request. Use `automl_policy: on` by default and only expose `on` \u002F `off` in new launch prompts. Treat phrases like \"turn off AutoML\", \"disable AutoML\", \"no HPO\", or \"plain training\" as `automl_policy: off` for this run only. When `automl_policy: on`, `automl_enabled: true`, and both `schemas\u002Ftrain.schema.json` and `references\u002Fspec_template_train.yaml` are packaged, route the train action through `tao-skill-bank:tao-run-automl` by default with this model's `skill_dir`. Preserve workflow\u002Fapplication overrides for datasets, specs, output directories, GPU\u002Fplatform settings, parent checkpoints, and `automl_policy`. Use direct model training only when `automl_policy: off` or the packaged train schema\u002Ftemplate is missing; in the missing-schema case, report that AutoML is enabled but not runnable for this model until schemas are generated.\n\nNon-train actions such as `evaluate`, `inference`, `export`, and deploy flows stay in this model skill. The per-run `automl_policy` override does not change model metadata.\n\n## Training Requirements\n\n- **Dataset type:** segmentation\n- **Formats:** odvg, coco, coco_raw\n- **Monitoring metric:** val_loss\n\n### Per-Action Dataset Requirements\n\n| Action | Spec Key | Source | Files | List? |\n|---|---|---|---|---|\n| evaluate | dataset.test_data_sources | eval_dataset | image_dir: images.tar.gz, json_file: annotations.json | No |\n| evaluate | dataset.test_data_sources.data_type | eval_dataset | OD | No |\n| inference | dataset.infer_data_sources | inference_dataset | image_dir: images.tar.gz, captions: text prompts | No |\n| inference | dataset.infer_data_sources.data_type | inference_dataset | OD | No |\n| quantize | dataset.train_data_sources | train_datasets | image_dir: images.tar.gz, json_file: annotations_odvg.jsonl, label_map: annotations_odvg_labelmap.json | Yes |\n| quantize | dataset.val_data_sources | eval_dataset | image_dir: images.tar.gz, json_file: annotations.json | No |\n| quantize | dataset.val_data_sources.data_type | eval_dataset | OD | No |\n| quantize | dataset.quant_calibration_data_sources | train_datasets | image_dir: images.tar.gz, json_file: annotations_odvg.jsonl, label_map: annotations_odvg_labelmap.json | No |\n| train | dataset.train_data_sources | train_datasets | image_dir: images.tar.gz, json_file: annotations_odvg.jsonl, label_map: annotations_odvg_labelmap.json | Yes |\n| train | dataset.val_data_sources | eval_dataset | image_dir: images.tar.gz, json_file: annotations.json | No |\n| train | dataset.val_data_sources.data_type | eval_dataset | OD | No |\n\n### Typical Spec Overrides\n\nData source overrides are **mandatory for every action** — the agent MUST construct data source paths from the Per-Action Dataset Requirements table above and include them in `spec_overrides`.\n\n```python\nS3_TRAIN = \"s3:\u002F\u002Fbucket\u002Fdata\u002Ftrain\"\nS3_EVAL = \"s3:\u002F\u002Fbucket\u002Fdata\u002Feval\"\n```\n\n**train (mandatory data sources):**\n```python\n{\n    \"train.num_gpus\": 1,\n    \"train.num_epochs\": 10,\n    \"train.checkpoint_interval\": 10,\n    \"train.validation_interval\": 10,\n    \"dataset.val_data_sources.data_type\": \"OD\",\n    \"model.num_region_queries\": 100,\n    \"dataset.train_data_sources\": [{\"image_dir\": f\"{S3_TRAIN}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_TRAIN}\u002Fannotations_odvg.jsonl\", \"label_map\": f\"{S3_TRAIN}\u002Fannotations_odvg_labelmap.json\"}],\n    \"dataset.val_data_sources\": {\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_EVAL}\u002Fannotations.json\"},\n}\n```\n\n**evaluate (mandatory data sources):**\n```python\n{\n    \"evaluate.checkpoint\": \"\u003Cselected train\u002FAutoML checkpoint>\",\n    \"dataset.test_data_sources.data_type\": \"OD\",\n    \"dataset.test_data_sources\": {\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_EVAL}\u002Fannotations.json\"},\n}\n```\n\n**inference (mandatory data sources):**\n```python\n{\n    \"inference.checkpoint\": \"\u003Cselected train\u002FAutoML checkpoint>\",\n    \"dataset.infer_data_sources.data_type\": \"OD\",\n    \"dataset.infer_data_sources\": {\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"captions\": [\"person\", \"bicycle\", \"car\"]},\n}\n```\n\n**quantize (mandatory data sources):**\n```python\n{\n    \"dataset.train_data_sources\": [{\"image_dir\": f\"{S3_TRAIN}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_TRAIN}\u002Fannotations_odvg.jsonl\", \"label_map\": f\"{S3_TRAIN}\u002Fannotations_odvg_labelmap.json\"}],\n    \"dataset.val_data_sources\": {\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_EVAL}\u002Fannotations.json\"},\n    \"dataset.quant_calibration_data_sources\": {\"image_dir\": f\"{S3_TRAIN}\u002Fimages.tar.gz\", \"json_file\": f\"{S3_TRAIN}\u002Fannotations_odvg.jsonl\", \"label_map\": f\"{S3_TRAIN}\u002Fannotations_odvg_labelmap.json\"},\n}\n```\n## Eval Dataset\n\nOptional. Validation uses COCO-format annotations even when training uses ODVG.\n\n## Important Parameters\n\n- **model.backbone**: Default swin_tiny_224_1k. Same backbone options as Grounding DINO.\n- **train.optim.lr**: Learning rate. Default 2e-4. lr_backbone 2e-5. Reuses GDINOTrainExpConfig — same training setup as Grounding DINO.\n- **model.num_queries**: Object queries. Default 900.\n- **model.enc_layers \u002F model.dec_layers**: Keep both at 6 for train\u002FAutoML\n  runs. The mask head asserts six decoder outputs during validation, so\n  copying Grounding DINO smoke overrides that reduce transformer layers causes\n  an immediate failure.\n- **AutoML metric note**: Use `metric=\"val_loss\"` with\n  `direction=\"minimize\"` for train-stage AutoML. The packaged train loop logs\n  validation loss scalars; it does not emit `[bbox] val_mAP@50` during the\n  train job.\n- **model.has_mask**: Enables mask prediction head. Default True. Adds mask\u002Fdice\u002Frela loss coefficients.\n- **model.num_region_queries**: Number of region queries for mask prediction. Default 100.\n- **model.loss_types**: Loss components. Default [labels, boxes, masks]. Includes mask_loss_coef, dice_loss_coef, rela_loss_coef.\n- **evaluate.ioi_threshold**: IoI threshold for mask evaluation. Default 0.5.\n- **evaluate.nms_threshold**: NMS threshold. Default 0.2.\n- **evaluate.text_threshold**: Text matching threshold. Default 0.3.\n- **dataset.has_mask**: Dataset includes mask annotations. Default True. val_data_sources default data_type is \"VG\".\n\n## Multi-GPU \u002F Multi-Node\n\n**Launch method:** Lightning-managed. Same DDP\u002FFSDP behavior as Grounding DINO.\n\n| Spec Key | Description | Default |\n|----------|-------------|---------|\n| `train.num_gpus` | Number of GPUs | 1 |\n| `train.gpu_ids` | GPU device indices | [0] |\n| `train.num_nodes` | Number of nodes | 1 |\n| `train.distributed_strategy` | `ddp` or `fsdp` | `ddp` |\n\n## Hardware\n\nMinimum 1 GPU(s), recommended 4 GPU(s). 24GB+ (A100 recommended) VRAM per GPU. Heavier than Grounding DINO due to mask prediction head. 24GB+ GPU memory recommended.\n\n## Error Patterns\n\n**CUDA out of memory**: Reduce batch_size. Mask prediction adds overhead on top of Grounding DINO.\n\n**Deploy schema error for `test_threshold`**: TAO Deploy uses\n`evaluate.text_threshold` and `inference.text_threshold`. Do not use\n`test_threshold` in deploy specs.\n\n**Deploy model shape mismatch**: Carry transformer and mask structure fields\nfrom export into deploy evaluate\u002Finference specs, including `model.num_queries`,\n`model.num_select`, `model.max_text_len`, `model.num_region_queries`, and\n`model.has_mask`. These values must match the ONNX model used to build the\nTensorRT engine.\n\n## Spec Param \u002F Parent Model Inference\n\nModel-specific inference mappings belong in this MD file, not in `config.json`. Generated runners should read this section and apply the mappings with SDK helpers before `create_job()`. This mirrors the old microservices `infer_params.py` flow.\n\nInference mappings from TAO Core `mask_grounding_dino.config.json`:\n\n| Action | Spec Field | Inference Function | Meaning |\n|---|---|---|---|\n| evaluate | `encryption_key` | `key` | encryption key |\n| evaluate | `evaluate.checkpoint` | `parent_model` | model file inferred from the parent job results folder |\n| evaluate | `evaluate.trt_engine` | `parent_model` | model file inferred from the parent job results folder |\n| evaluate | `results_dir` | `output_dir` | current job results directory |\n| export | `encryption_key` | `key` | encryption key |\n| export | `export.checkpoint` | `parent_model` | model file inferred from the parent job results folder |\n| export | `export.onnx_file` | `create_onnx_file` | output ONNX path |\n| export | `results_dir` | `output_dir` | current job results directory |\n| gen_trt_engine | `encryption_key` | `key` | encryption key |\n| gen_trt_engine | `gen_trt_engine.onnx_file` | `parent_model` | model file inferred from the parent job results folder |\n| gen_trt_engine | `gen_trt_engine.trt_engine` | `create_engine_file` | output TensorRT engine path |\n| gen_trt_engine | `results_dir` | `output_dir` | current job results directory |\n| inference | `encryption_key` | `key` | encryption key |\n| inference | `inference.checkpoint` | `parent_model` | model file inferred from the parent job results folder |\n| inference | `inference.trt_engine` | `parent_model` | model file inferred from the parent job results folder |\n| inference | `results_dir` | `output_dir` | current job results directory |\n| quantize | `encryption_key` | `key` | encryption key |\n| quantize | `quantize.model_path` | `parent_model` | model file inferred from the parent job results folder |\n| quantize | `results_dir` | `output_dir` | current job results directory |\n| train | `encryption_key` | `key` | encryption key |\n| train | `model.pretrained_backbone_path` | `ptm_if_no_resume_model` | PTM when no resume checkpoint exists |\n| train | `results_dir` | `output_dir` | current job results directory |\n| train | `train.pretrained_model_path` | `ptm_if_no_resume_model` | PTM when no resume checkpoint exists |\n| train | `train.resume_training_checkpoint_path` | `resume_model` | model file inferred from the current job results folder |\n\nFor `parent_model` or `parent_model_folder`, pass the upstream train\u002Fexport\u002FAutoML child job id as `parent_job_id`. The SDK lists the parent result folder, filters checkpoint artifacts, and returns the selected model file or folder. Do not add these mappings back to `config.json` and do not patch generated runner scripts to guess checkpoint paths.\n\nWhen selecting a Mask Grounding DINO checkpoint outside the SDK resolver, match\nthe intended epoch\u002Fstep artifact exactly, for example\n`model_epoch_000_step_00049.pth`. The `mask_gdino_model_latest.pth` symlink is\nvalid only when latest is explicitly requested. The parent PyTorch\n`mask_grounding_dino` CLI supports `train`, `evaluate`, `inference`, `export`,\nand `quantize`; run TensorRT engine generation, TensorRT inference, and\nTensorRT evaluation through `references\u002Ftao-deploy-mask-grounding-dino.md`.\n\n## Deployment\n\n- [tao-deploy-mask-grounding-dino](references\u002Ftao-deploy-mask-grounding-dino.md)\n",{"name":5,"tags":24,"license":15,"metadata":26,"description":7,"allowed-tools":29,"compatibility":30},[25],"segmentation",{"author":27,"version":28},"NVIDIA Corporation","0.1.0","Read Bash","Requires docker + nvidia-container-toolkit.","main","be3de0244dc1a57e323d8160787ebb40fb64930e","public\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-train-mask-grounding-dino","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main","https:\u002F\u002Fskills.x402.wtf","https:\u002F\u002Fgithub.com\u002FSolizardking","https:\u002F\u002Fmusebook.trade","Got a mile high and never came down","2026-10-08 23:20:13",{"source":8,"skillName":5,"commands":41,"prompt":51,"zipAvailable":52},[42,45,48],{"tool":43,"command":44},"npx","npx skills add Solizardking\u002Fskillhub-main --skill tao-train-mask-grounding-dino",{"tool":46,"command":47},"bunx","bunx skills add Solizardking\u002Fskillhub-main --skill tao-train-mask-grounding-dino",{"tool":49,"command":50},"pnpm","pnpm dlx skills add Solizardking\u002Fskillhub-main --skill tao-train-mask-grounding-dino","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002FSKILL.md",true,[54,60,64,68,72,76,80,84,88,92,96,100,104,108,112,116],{"path":55,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":58,"sourceHtmlUrl":59},"BENCHMARK.md","REFERENCE","参考文档",4136,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002FBENCHMARK.md",{"path":61,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":62,"sourceHtmlUrl":63},"metadata.json",659,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Fmetadata.json",{"path":65,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":66,"sourceHtmlUrl":67},"references\u002Fskill_info.yaml",6309,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fskill_info.yaml",{"path":69,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":70,"sourceHtmlUrl":71},"references\u002Fspec_template_deploy_evaluate.yaml",567,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fspec_template_deploy_evaluate.yaml",{"path":73,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":74,"sourceHtmlUrl":75},"references\u002Fspec_template_deploy_gen_trt_engine.yaml",365,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fspec_template_deploy_gen_trt_engine.yaml",{"path":77,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":78,"sourceHtmlUrl":79},"references\u002Fspec_template_deploy_inference.yaml",1651,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fspec_template_deploy_inference.yaml",{"path":81,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":82,"sourceHtmlUrl":83},"references\u002Fspec_template_evaluate.yaml",3612,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fspec_template_evaluate.yaml",{"path":85,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":86,"sourceHtmlUrl":87},"references\u002Fspec_template_export.yaml",3645,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fspec_template_export.yaml",{"path":89,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":90,"sourceHtmlUrl":91},"references\u002Fspec_template_gen_trt_engine.yaml",3682,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fspec_template_gen_trt_engine.yaml",{"path":93,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":94,"sourceHtmlUrl":95},"references\u002Fspec_template_inference.yaml",3692,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fspec_template_inference.yaml",{"path":97,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":98,"sourceHtmlUrl":99},"references\u002Fspec_template_quantize.yaml",3418,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fspec_template_quantize.yaml",{"path":101,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":102,"sourceHtmlUrl":103},"references\u002Fspec_template_train.yaml",3400,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Fspec_template_train.yaml",{"path":105,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":106,"sourceHtmlUrl":107},"references\u002Ftao-deploy-mask-grounding-dino.md",5683,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Ftao-deploy-mask-grounding-dino.md",{"path":109,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":110,"sourceHtmlUrl":111},"references\u002Ftao-deploy-mask-grounding-dino.skill_info.yaml",3395,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Freferences\u002Ftao-deploy-mask-grounding-dino.skill_info.yaml",{"path":113,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":114,"sourceHtmlUrl":115},"skill-card.md",3794,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Fskill-card.md",{"path":117,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":118,"sourceHtmlUrl":119},"verification.json",3858,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-train-mask-grounding-dino\u002Fverification.json",[121,129,136,143,150,158],{"skillNo":122,"name":123,"nameZh":6,"description":124,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":125,"licenseFileUrl":6,"sourceHtmlUrl":126,"categories":127,"keywords":128,"publishedAt":19},"01M78E0TGCAMNPS207BA3G07S7","zkrouter","Self-hosted OpenAI-compatible LLM router — smart tier-based routing, ZK-stamped routing decisions, mode overrides, free OpenRouter at install via the Birth bot. Save 60-80% on inference costs by routing to the cheapest capable model across 12+ OpenRouter models.","MIT","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzkrouter\u002FSKILL.md",[],[],{"skillNo":130,"name":131,"nameZh":6,"description":132,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":125,"licenseFileUrl":6,"sourceHtmlUrl":133,"categories":134,"keywords":135,"publishedAt":19},"01M78E0R1Z8BDT6EDTHY17MTNB","zk","For custom ZK Solana programs and privacy-preserving applications to prevent double spending. Guide to integrate rent-free nullifier PDAs for double-spend prevention.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzk\u002FSKILL.md",[],[],{"skillNo":137,"name":138,"nameZh":6,"description":139,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":125,"licenseFileUrl":6,"sourceHtmlUrl":140,"categories":141,"keywords":142,"publishedAt":19},"01M78E0P3ZTZ1YWF8MD5MM23QE","zk-omni-messaging","Zero-knowledge omnichain messaging between Robinhood Chain and Solana using CheshireZkOmniMessenger (msgType 4) with nullifier anti-replay and the zk-omni-relayer service.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzk-omni-messaging\u002FSKILL.md",[],[],{"skillNo":144,"name":145,"nameZh":6,"description":146,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":125,"licenseFileUrl":6,"sourceHtmlUrl":147,"categories":148,"keywords":149,"publishedAt":19},"01M78E0JXZ2PNB1896VXAGDBWD","zerion-wallet-data","Use when the user asks to check a wallet's portfolio, token balances, DeFi positions, PnL, transaction history, or NFTs. Provides interpreted on-chain data across 40+ EVM chains and Solana via x402 pay-per-request or API key — no raw RPC decoding needed.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzerion-wallet-data\u002FSKILL.md",[],[],{"skillNo":151,"name":152,"nameZh":6,"description":153,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":125,"licenseFileUrl":154,"sourceHtmlUrl":155,"categories":156,"keywords":157,"publishedAt":19},"01M78E0G14FH9GRRTQYMKD5E59","youtube-clipper","YouTube 视频智能剪辑工具。下载视频和字幕，AI 分析生成精细章节（几分钟级别）， 用户选择片段后自动剪辑、翻译字幕为中英双语、烧录字幕到视频，并生成总结文案。 使用场景：当用户需要剪辑 YouTube 视频、生成短视频片段、制作双语字幕版本时。 关键词：视频剪辑、YouTube、字幕翻译、双语字幕、视频下载、clip video","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fyoutube-clipper\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fyoutube-clipper\u002FSKILL.md",[],[],{"skillNo":159,"name":160,"nameZh":6,"description":161,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":125,"licenseFileUrl":6,"sourceHtmlUrl":162,"categories":163,"keywords":164,"publishedAt":19},"01M78E0C2ZYA99CFFDNB7ZMK5S","yield-optimization","Multi-chain yield optimization via yield.xyz (StakeKit). Discover 2,988+ yield opportunities across 75+ blockchains, build deposit\u002Fwithdrawal transactions with shell scripts, and sign them with a MoonPay wallet. Use when the user wants to earn yield, find the best lending or staking rate, enter\u002Fexit a position, or check portfolio balances.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fyield-optimization\u002FSKILL.md",[],[]]