[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7BK6YBJCQQ1BWB611MBFTMD":3,"skill-same:01M7BK6YBJCQQ1BWB611MBFTMD":106},{"skillNo":4,"name":5,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":18,"linkOnly":19,"markdownContent":20,"markdownBody":21,"frontMatter":22,"refName":40,"commitSha":41,"skillDirectory":42,"forksCount":13,"repositoryHtmlUrl":43,"repositoryHomepageUrl":44,"ownerHtmlUrl":45,"ownerBlogUrl":6,"ownerDescription":46,"updatedAt":47,"install":48,"resources":61},"01M7BK6YBJCQQ1BWB611MBFTMD","tao-finetune-cosmos-embed",null,"YYC-Cube\u002FYYC3-AI-Agent-Archive","YYC-Cube","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F235006543?v=4","01M79J2PH4SWD2X0825X6PN748","yyc3","01M79K9J2WD44BBY4662XKNZRB",2,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive\u002Fblob\u002Fmain\u002Fskills-hub\u002Fai-ml\u002Fnvidia-skills\u002Fskills\u002Ftao-finetune-cosmos-embed\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: tao-finetune-cosmos-embed\ndescription: >-\ncategory: development-code\nversion: 1.0.0\n  Cosmos-Embed1 video-text embedding for text-to-video retrieval, video-to-video search, semantic deduplication, and\n  fine-tuning. Use when the user asks to \"fine-tune Cosmos-Embed1\", \"run cosmos-embed inference\", \"export Cosmos-Embed1\",\n  \"embed videos\", or \"search videos with text\".\nlicense: Apache-2.0\ncompatibility: Requires docker + nvidia-container-toolkit, the published Cosmos-Embed TAO container from versions.yaml, and a HuggingFace token when downloading pretrained `nvidia\u002FCosmos-Embed1-*` weights.\nmetadata:\n  author: NVIDIA Corporation\n  version: \"0.1.0\"\nallowed-tools: Read Bash\ntags:\n- video\n- vision-language\n- vlm\n- multimodal\n- retrieval\n- embedding\n- cosmos\n- fine-tuning\n---\n\n# Cosmos-Embed\n\nCosmos-Embed1 is a joint video-text embedder for text-to-video retrieval, video-to-video search, zero-shot\u002FkNN classification, and semantic deduplication. The packaged CLI is `cosmos-embed1` and supports `train`, `evaluate`, `inference`, and `export`.\n\nContainer image and per-action commands are in `references\u002Fskill_info.yaml`. Compact starting specs are in `references\u002Fspec_template_*.yaml`.\n\n## Train Action Policy\n\nThis model is AutoML-enabled at the model layer. Before handling any train-stage request, read `references\u002Fskill_info.yaml` and resolve the run override from either an explicit `automl_policy` value or the user's workflow request. Treat phrases like \"turn off AutoML\", \"disable AutoML\", \"no HPO\", or \"plain training\" as `automl_policy: off` for this run only; otherwise default to `auto`. When `automl_policy: auto`, `automl_enabled: true`, and both `schemas\u002Ftrain.schema.json` and `references\u002Fspec_template_train.yaml` are packaged, route the train action through `tao-skill-bank:tao-run-automl` by default with this model's `skill_dir`. Preserve workflow\u002Fapplication overrides for datasets, specs, output directories, GPU\u002Fplatform settings, parent checkpoints, and `automl_policy`. Use direct model training only when `automl_policy: off` or the packaged train schema\u002Ftemplate is missing; in the missing-schema case, report that AutoML is enabled but not runnable for this model until schemas are generated.\n\nNon-train actions such as `evaluate`, `inference`, `export`, and deploy flows stay in this model skill. The per-run `automl_policy` override does not change model metadata.\n\n## Quick Start\n\nUse the published Cosmos-Embed container declared by `references\u002Fskill_info.yaml`\nand resolved through `versions.yaml`. Do not build from the private\nCosmos-Embed1 source tree for normal skill use; build from source only when\ndeveloping the container itself.\n\n```bash\nTAO_SKILL_BANK_PATH=\"${TAO_SKILL_BANK_PATH:-$PWD}\"\nCOSMOS_EMBED_IMAGE=\"${COSMOS_EMBED_IMAGE:-$(\n  python \"$TAO_SKILL_BANK_PATH\u002Fscripts\u002Fresolve_tao_image.py\" \\\n    --skill-bank \"$TAO_SKILL_BANK_PATH\" \\\n    --model cosmos-embed \\\n    --action train \\\n    --format json |\n  python -c 'import json,sys; print(json.load(sys.stdin)[\"image\"])'\n)}\"\ndocker pull \"$COSMOS_EMBED_IMAGE\"\n```\n\nExpected local workspace layout:\n\n```text\nworkspace\u002F\n├── data\u002F\n│   ├── msrvtt_test_1k.json\n│   └── video\u002F\n│       ├── video7020.mp4\n│       └── ...\n├── model\u002F\n│   └── Cosmos-Embed1-224p\u002F        # optional if using HF repo id\n├── specs\u002F\n│   ├── train.yaml\n│   ├── evaluate.yaml\n│   ├── inference.yaml\n│   ├── export_onnx.yaml\n│   └── export_hf.yaml\n└── results\u002F\n```\n\nUse these Docker options for all actions unless the local Docker\u002Fplatform skill gives a stricter environment-specific command:\n\n```bash\nTAO_SKILL_BANK_PATH=\"${TAO_SKILL_BANK_PATH:-$PWD}\"\nCOSMOS_EMBED_IMAGE=\"${COSMOS_EMBED_IMAGE:-$(\n  python \"$TAO_SKILL_BANK_PATH\u002Fscripts\u002Fresolve_tao_image.py\" \\\n    --skill-bank \"$TAO_SKILL_BANK_PATH\" \\\n    --model cosmos-embed \\\n    --action train \\\n    --format json |\n  python -c 'import json,sys; print(json.load(sys.stdin)[\"image\"])'\n)}\"\nRUN_ROOT=\"${RUN_ROOT:-$PWD}\"\nDOCKER_COMMON=(\n  --rm --gpus all --ipc=host --network=host\n  --shm-size=64g\n  --ulimit memlock=-1\n  --ulimit stack=67108864\n  -e HF_TOKEN\n  -v \"$RUN_ROOT\u002Fdata:\u002Fdata:ro\"\n  -v \"$RUN_ROOT\u002Fmodel:\u002Fmodel\"\n  -v \"$RUN_ROOT\u002Fspecs:\u002Fspecs:ro\"\n  -v \"$RUN_ROOT\u002Fresults:\u002Fresults\"\n)\n```\n\nTrain:\n\n```bash\ndocker run \"${DOCKER_COMMON[@]}\" \"$COSMOS_EMBED_IMAGE\" \\\n  cosmos-embed1 train -e \u002Fspecs\u002Ftrain.yaml results_dir=\u002Fresults\n```\n\nEvaluate:\n\n```bash\ndocker run \"${DOCKER_COMMON[@]}\" \"$COSMOS_EMBED_IMAGE\" \\\n  cosmos-embed1 evaluate -e \u002Fspecs\u002Fevaluate.yaml results_dir=\u002Fresults\n```\n\nInference:\n\n```bash\ndocker run \"${DOCKER_COMMON[@]}\" \"$COSMOS_EMBED_IMAGE\" \\\n  cosmos-embed1 inference -e \u002Fspecs\u002Finference.yaml \\\n  'inference.query.input_texts=[\"a man is singing on stage\"]' \\\n  inference.k=5 \\\n  results_dir=\u002Fresults\n```\n\nExport ONNX:\n\n```bash\ndocker run \"${DOCKER_COMMON[@]}\" \"$COSMOS_EMBED_IMAGE\" \\\n  cosmos-embed1 export -e \u002Fspecs\u002Fexport_onnx.yaml \\\n  export.checkpoint=\u002Fresults\u002Ftrain\u002Fcosmos_embed1_model_latest.pth \\\n  export.onnx_file=\u002Fresults\u002Fexport\u002Fcosmos_embed1_combined.onnx \\\n  results_dir=\u002Fresults\n```\n\nExport HuggingFace format:\n\n```bash\ndocker run \"${DOCKER_COMMON[@]}\" \"$COSMOS_EMBED_IMAGE\" \\\n  cosmos-embed1 export -e \u002Fspecs\u002Fexport_hf.yaml \\\n  export.checkpoint=\u002Fresults\u002Ftrain\u002Fcosmos_embed1_model_latest.pth \\\n  export.hf_output_dir=\u002Fresults\u002Fexport_hf\u002Fcosmos_embed1_hf \\\n  results_dir=\u002Fresults\n```\n\n## Smoke Overrides\n\nFor a small functional check, keep the same specs and override the expensive knobs:\n\n```bash\ntrain.max_iter=1\ntrain.validation_iter=2\ntrain.checkpoint_iter=1\ntrain.optim.optim=adamw\ndataset.train_dataset.batch_size=1\ndataset.val_dataset.batch_size=1\ndataset.train_dataset.workers=0\ndataset.val_dataset.workers=0\n```\n\nIf no local Cosmos-Embed1 pretrained checkpoint or HuggingFace token is available, set `model.pretrained_model_path=null` for a plumbing-only smoke train. The model quality is meaningless in that mode, but the train\u002Fevaluate\u002Finference\u002Fexport action paths can still be exercised.\n\nFor evaluation and inference smoke tests on a tiny subset:\n\n```bash\nevaluate.callbacks.embedding_visualization=false\nevaluate.callbacks.max_eval_samples=8\ndataset.test_dataset.batch_size=1\ndataset.test_dataset.workers=0\ninference.k=2\ndataset.inference_dataset.batch_size=1\ndataset.inference_dataset.workers=0\n```\n\n## Data Format\n\nThe MSR-VTT path expects a local video glob and a JSON metadata file:\n\n```yaml\ndataset:\n  train_dataset:\n    dataset_type: msrvtt\n    mp4_urls: \u002Fdata\u002Fvideo\u002F*.mp4\n    metadata: \u002Fdata\u002Fmsrvtt_test_1k.json\n```\n\nList-format metadata rows must include at least `video` and `caption`:\n\n```json\n{\"video_id\": \"video7020\", \"video\": \"video7020.mp4\", \"caption\": \"a woman creating a fondant baby and flower\"}\n```\n\nThe dataset loader derives the video id from the local `.mp4` filename and filters to videos present in the metadata. If a run finds zero videos, check that `mp4_urls` points to a container-local glob and that metadata `video` names match the filenames.\n\n## Model Weights\n\n- Local HF directory: mount it under `\u002Fmodel` and set `model.pretrained_model_path=\u002Fmodel\u002FCosmos-Embed1-224p`.\n- HuggingFace repo: set `model.pretrained_model_path=nvidia\u002FCosmos-Embed1-224p` and pass `HF_TOKEN` if access is gated.\n- Fine-tuned checkpoint: downstream actions default to `\u002Fresults\u002Ftrain\u002Fcosmos_embed1_model_latest.pth`.\n\nVariants:\n\n| Variant | Resolution | Frames | Embedding dim |\n|---|---:|---:|---:|\n| `Cosmos-Embed1-224p` | 224 x 224 | 8 | 256 |\n| `Cosmos-Embed1-336p` | 336 x 336 | 8 | 768 |\n| `Cosmos-Embed1-448p` | 448 x 448 | 8 | 768 |\n\nKeep `model.network.embed_dim`, `model.input_hw`, and `model.network.spatial_resolution` aligned with the selected variant.\n\n## Important Parameters\n\n| Parameter | Notes |\n|---|---|\n| `train.num_gpus` | `1` for single GPU, `>1` auto-launches `torchrun`, `-1` auto-detects visible GPUs. |\n| `train.max_iter` | Main training length. Use `1` only for smoke testing. |\n| `train.optim.optim` | `fused_adamw` is faster when available; `adamw` is safer for smoke and portability. |\n| `model.lora.enabled` | Enables LoRA. Set `model.network.visual_encoder.transformer_engine=false` when LoRA is on. |\n| `model.lora.lora_rank` | LoRA rank. Start with `8`; try `4`, `8`, or `16` for manual or AutoML-style sweeps. |\n| `model.lora.lora_alpha` | LoRA scaling factor. Start with `16`; keep near `2 * lora_rank` unless experiments show otherwise. |\n| `model.lora.lora_dropout` | LoRA dropout. Start with `0.1`; sweep `0.0`, `0.05`, and `0.1` for small datasets. |\n| `model.lora.bias` | Bias policy: `none`, `all`, or `lora_only`. Keep `none` unless intentionally training biases. |\n| `model.lora.use_rslora` \u002F `use_dora` | Optional LoRA variants. Enable one at a time and record the setting with the checkpoint. |\n| `model.lora.target_modules` | Optional module-name patterns for LoRA injection. Leave empty for the default ViT + Q-Former attention\u002FMLP targets. |\n| `model.lora.modules_to_save` | Optional modules to keep fully trainable alongside LoRA. Leave empty unless preserving a task-specific head. |\n| `evaluate.load_dataset_pkl` \u002F `save_dataset_pkl` | Cache evaluation embeddings. |\n| `inference.load_dataset_pkl` \u002F `save_dataset_pkl` | Cache the search database for repeated retrieval. |\n| `export.mode` | `video`, `text`, `combined`, or `huggingface`. |\n| `export.on_cpu` | Recommended for export to avoid device mismatch issues. |\n\n### LoRA and AutoML Notes\n\nFor parameter-efficient fine-tuning, set `model.lora.enabled=true` and keep\n`model.network.visual_encoder.transformer_engine=false`; TAO Core's\nCosmos-Embed1 config notes that PEFT cannot inject adapters into Transformer\nEngine layers. Treat the LoRA fields above as the first candidate parameters\nfor manual tuning or AutoML-style search before unfreezing larger model blocks.\nAvoid changing `target_modules` or `modules_to_save` unless the user explicitly\nneeds custom adapter placement.\n\n## S3 Staging\n\nThe Cosmos-Embed1 CLI consumes local paths and Python globs, not raw `s3:\u002F\u002F...\u002F*.mp4` URIs. For S3-backed runs, first stage a subset or full dataset to the execution host\u002Fcontainer filesystem, then use local paths such as `\u002Fdata\u002Fvideo\u002F*.mp4` in the spec.\n\nRecommended S3 layout for staged MSR-VTT data:\n\n```text\ns3:\u002F\u002Fbucket\u002Fpath\u002Fcosmos-embed\u002Fmsrvtt-subset\u002F\n├── msrvtt_test_1k.json\n└── video\u002F\n    ├── video7020.mp4\n    └── ...\n```\n\nAfter downloading\u002Fsyncing that prefix into the mounted `data\u002F` directory, use the same Docker commands above.\n\n## Outputs\n\n```text\nresults\u002F\n├── train\u002F\n│   ├── cosmos_embed1_model_latest.pth\n│   ├── cosmos_embed1_model_\u003Citer>.pth\n│   └── experiment.yaml\n├── evaluate\u002F\n│   ├── metrics.json\n│   └── experiment.yaml\n├── inference\u002F\n│   ├── results.json\n│   └── experiment.yaml\n├── export\u002F\n│   ├── cosmos_embed1_combined.onnx\n│   └── export_config.yaml\n└── export_hf\u002F\n    └── cosmos_embed1_hf\u002F\n```\n\n## Known Pitfalls\n\n| Symptom | Cause | Fix |\n|---|---|---|\n| `MSRVTTDataset: 0 videos found` | `mp4_urls` is not a local glob or metadata filenames do not match videos. | Mount data into the container and set `mp4_urls=\u002Fdata\u002Fvideo\u002F*.mp4`. |\n| HF download\u002Fauth failure | Missing or invalid `HF_TOKEN`, or model agreement not accepted. | Accept the model terms and pass `-e HF_TOKEN`. |\n| LoRA injection failure | Transformer Engine visual encoder is enabled. | Set `model.network.visual_encoder.transformer_engine=false`. |\n| ONNX\u002FHF export complains about missing components | Export checkpoint is partial or adapter-only. | Use a full checkpoint or configure pretrained visual\u002Ftext sources before export. |\n| CUDA OOM | Batch\u002Fresolution too high for the GPU. | Reduce batch size, use 224p, enable LoRA, or use more GPUs. |\n","# Cosmos-Embed\n\nCosmos-Embed1 is a joint video-text embedder for text-to-video retrieval, video-to-video search, zero-shot\u002FkNN classification, and semantic deduplication. The packaged CLI is `cosmos-embed1` and supports `train`, `evaluate`, `inference`, and `export`.\n\nContainer image and per-action commands are in `references\u002Fskill_info.yaml`. Compact starting specs are in `references\u002Fspec_template_*.yaml`.\n\n## Train Action Policy\n\nThis model is AutoML-enabled at the model layer. Before handling any train-stage request, read `references\u002Fskill_info.yaml` and resolve the run override from either an explicit `automl_policy` value or the user's workflow request. Treat phrases like \"turn off AutoML\", \"disable AutoML\", \"no HPO\", or \"plain training\" as `automl_policy: off` for this run only; otherwise default to `auto`. When `automl_policy: auto`, `automl_enabled: true`, and both `schemas\u002Ftrain.schema.json` and `references\u002Fspec_template_train.yaml` are packaged, route the train action through `tao-skill-bank:tao-run-automl` by default with this model's `skill_dir`. Preserve workflow\u002Fapplication overrides for datasets, specs, output directories, GPU\u002Fplatform settings, parent checkpoints, and `automl_policy`. Use direct model training only when `automl_policy: off` or the packaged train schema\u002Ftemplate is missing; in the missing-schema case, report that AutoML is enabled but not runnable for this model until schemas are generated.\n\nNon-train actions such as `evaluate`, `inference`, `export`, and deploy flows stay in this model skill. The per-run `automl_policy` override does not change model metadata.\n\n## Quick Start\n\nUse the published Cosmos-Embed container declared by `references\u002Fskill_info.yaml`\nand resolved through `versions.yaml`. Do not build from the private\nCosmos-Embed1 source tree for normal skill use; build from source only when\ndeveloping the container itself.\n\n```bash\nTAO_SKILL_BANK_PATH=\"${TAO_SKILL_BANK_PATH:-$PWD}\"\nCOSMOS_EMBED_IMAGE=\"${COSMOS_EMBED_IMAGE:-$(\n  python \"$TAO_SKILL_BANK_PATH\u002Fscripts\u002Fresolve_tao_image.py\" \\\n    --skill-bank \"$TAO_SKILL_BANK_PATH\" \\\n    --model cosmos-embed \\\n    --action train \\\n    --format json |\n  python -c 'import json,sys; print(json.load(sys.stdin)[\"image\"])'\n)}\"\ndocker pull \"$COSMOS_EMBED_IMAGE\"\n```\n\nExpected local workspace layout:\n\n```text\nworkspace\u002F\n├── data\u002F\n│   ├── msrvtt_test_1k.json\n│   └── video\u002F\n│       ├── video7020.mp4\n│       └── ...\n├── model\u002F\n│   └── Cosmos-Embed1-224p\u002F        # optional if using HF repo id\n├── specs\u002F\n│   ├── train.yaml\n│   ├── evaluate.yaml\n│   ├── inference.yaml\n│   ├── export_onnx.yaml\n│   └── export_hf.yaml\n└── results\u002F\n```\n\nUse these Docker options for all actions unless the local Docker\u002Fplatform skill gives a stricter environment-specific command:\n\n```bash\nTAO_SKILL_BANK_PATH=\"${TAO_SKILL_BANK_PATH:-$PWD}\"\nCOSMOS_EMBED_IMAGE=\"${COSMOS_EMBED_IMAGE:-$(\n  python \"$TAO_SKILL_BANK_PATH\u002Fscripts\u002Fresolve_tao_image.py\" \\\n    --skill-bank \"$TAO_SKILL_BANK_PATH\" \\\n    --model cosmos-embed \\\n    --action train \\\n    --format json |\n  python -c 'import json,sys; print(json.load(sys.stdin)[\"image\"])'\n)}\"\nRUN_ROOT=\"${RUN_ROOT:-$PWD}\"\nDOCKER_COMMON=(\n  --rm --gpus all --ipc=host --network=host\n  --shm-size=64g\n  --ulimit memlock=-1\n  --ulimit stack=67108864\n  -e HF_TOKEN\n  -v \"$RUN_ROOT\u002Fdata:\u002Fdata:ro\"\n  -v \"$RUN_ROOT\u002Fmodel:\u002Fmodel\"\n  -v \"$RUN_ROOT\u002Fspecs:\u002Fspecs:ro\"\n  -v \"$RUN_ROOT\u002Fresults:\u002Fresults\"\n)\n```\n\nTrain:\n\n```bash\ndocker run \"${DOCKER_COMMON[@]}\" \"$COSMOS_EMBED_IMAGE\" \\\n  cosmos-embed1 train -e \u002Fspecs\u002Ftrain.yaml results_dir=\u002Fresults\n```\n\nEvaluate:\n\n```bash\ndocker run \"${DOCKER_COMMON[@]}\" \"$COSMOS_EMBED_IMAGE\" \\\n  cosmos-embed1 evaluate -e \u002Fspecs\u002Fevaluate.yaml results_dir=\u002Fresults\n```\n\nInference:\n\n```bash\ndocker run \"${DOCKER_COMMON[@]}\" \"$COSMOS_EMBED_IMAGE\" \\\n  cosmos-embed1 inference -e \u002Fspecs\u002Finference.yaml \\\n  'inference.query.input_texts=[\"a man is singing on stage\"]' \\\n  inference.k=5 \\\n  results_dir=\u002Fresults\n```\n\nExport ONNX:\n\n```bash\ndocker run \"${DOCKER_COMMON[@]}\" \"$COSMOS_EMBED_IMAGE\" \\\n  cosmos-embed1 export -e \u002Fspecs\u002Fexport_onnx.yaml \\\n  export.checkpoint=\u002Fresults\u002Ftrain\u002Fcosmos_embed1_model_latest.pth \\\n  export.onnx_file=\u002Fresults\u002Fexport\u002Fcosmos_embed1_combined.onnx \\\n  results_dir=\u002Fresults\n```\n\nExport HuggingFace format:\n\n```bash\ndocker run \"${DOCKER_COMMON[@]}\" \"$COSMOS_EMBED_IMAGE\" \\\n  cosmos-embed1 export -e \u002Fspecs\u002Fexport_hf.yaml \\\n  export.checkpoint=\u002Fresults\u002Ftrain\u002Fcosmos_embed1_model_latest.pth \\\n  export.hf_output_dir=\u002Fresults\u002Fexport_hf\u002Fcosmos_embed1_hf \\\n  results_dir=\u002Fresults\n```\n\n## Smoke Overrides\n\nFor a small functional check, keep the same specs and override the expensive knobs:\n\n```bash\ntrain.max_iter=1\ntrain.validation_iter=2\ntrain.checkpoint_iter=1\ntrain.optim.optim=adamw\ndataset.train_dataset.batch_size=1\ndataset.val_dataset.batch_size=1\ndataset.train_dataset.workers=0\ndataset.val_dataset.workers=0\n```\n\nIf no local Cosmos-Embed1 pretrained checkpoint or HuggingFace token is available, set `model.pretrained_model_path=null` for a plumbing-only smoke train. The model quality is meaningless in that mode, but the train\u002Fevaluate\u002Finference\u002Fexport action paths can still be exercised.\n\nFor evaluation and inference smoke tests on a tiny subset:\n\n```bash\nevaluate.callbacks.embedding_visualization=false\nevaluate.callbacks.max_eval_samples=8\ndataset.test_dataset.batch_size=1\ndataset.test_dataset.workers=0\ninference.k=2\ndataset.inference_dataset.batch_size=1\ndataset.inference_dataset.workers=0\n```\n\n## Data Format\n\nThe MSR-VTT path expects a local video glob and a JSON metadata file:\n\n```yaml\ndataset:\n  train_dataset:\n    dataset_type: msrvtt\n    mp4_urls: \u002Fdata\u002Fvideo\u002F*.mp4\n    metadata: \u002Fdata\u002Fmsrvtt_test_1k.json\n```\n\nList-format metadata rows must include at least `video` and `caption`:\n\n```json\n{\"video_id\": \"video7020\", \"video\": \"video7020.mp4\", \"caption\": \"a woman creating a fondant baby and flower\"}\n```\n\nThe dataset loader derives the video id from the local `.mp4` filename and filters to videos present in the metadata. If a run finds zero videos, check that `mp4_urls` points to a container-local glob and that metadata `video` names match the filenames.\n\n## Model Weights\n\n- Local HF directory: mount it under `\u002Fmodel` and set `model.pretrained_model_path=\u002Fmodel\u002FCosmos-Embed1-224p`.\n- HuggingFace repo: set `model.pretrained_model_path=nvidia\u002FCosmos-Embed1-224p` and pass `HF_TOKEN` if access is gated.\n- Fine-tuned checkpoint: downstream actions default to `\u002Fresults\u002Ftrain\u002Fcosmos_embed1_model_latest.pth`.\n\nVariants:\n\n| Variant | Resolution | Frames | Embedding dim |\n|---|---:|---:|---:|\n| `Cosmos-Embed1-224p` | 224 x 224 | 8 | 256 |\n| `Cosmos-Embed1-336p` | 336 x 336 | 8 | 768 |\n| `Cosmos-Embed1-448p` | 448 x 448 | 8 | 768 |\n\nKeep `model.network.embed_dim`, `model.input_hw`, and `model.network.spatial_resolution` aligned with the selected variant.\n\n## Important Parameters\n\n| Parameter | Notes |\n|---|---|\n| `train.num_gpus` | `1` for single GPU, `>1` auto-launches `torchrun`, `-1` auto-detects visible GPUs. |\n| `train.max_iter` | Main training length. Use `1` only for smoke testing. |\n| `train.optim.optim` | `fused_adamw` is faster when available; `adamw` is safer for smoke and portability. |\n| `model.lora.enabled` | Enables LoRA. Set `model.network.visual_encoder.transformer_engine=false` when LoRA is on. |\n| `model.lora.lora_rank` | LoRA rank. Start with `8`; try `4`, `8`, or `16` for manual or AutoML-style sweeps. |\n| `model.lora.lora_alpha` | LoRA scaling factor. Start with `16`; keep near `2 * lora_rank` unless experiments show otherwise. |\n| `model.lora.lora_dropout` | LoRA dropout. Start with `0.1`; sweep `0.0`, `0.05`, and `0.1` for small datasets. |\n| `model.lora.bias` | Bias policy: `none`, `all`, or `lora_only`. Keep `none` unless intentionally training biases. |\n| `model.lora.use_rslora` \u002F `use_dora` | Optional LoRA variants. Enable one at a time and record the setting with the checkpoint. |\n| `model.lora.target_modules` | Optional module-name patterns for LoRA injection. Leave empty for the default ViT + Q-Former attention\u002FMLP targets. |\n| `model.lora.modules_to_save` | Optional modules to keep fully trainable alongside LoRA. Leave empty unless preserving a task-specific head. |\n| `evaluate.load_dataset_pkl` \u002F `save_dataset_pkl` | Cache evaluation embeddings. |\n| `inference.load_dataset_pkl` \u002F `save_dataset_pkl` | Cache the search database for repeated retrieval. |\n| `export.mode` | `video`, `text`, `combined`, or `huggingface`. |\n| `export.on_cpu` | Recommended for export to avoid device mismatch issues. |\n\n### LoRA and AutoML Notes\n\nFor parameter-efficient fine-tuning, set `model.lora.enabled=true` and keep\n`model.network.visual_encoder.transformer_engine=false`; TAO Core's\nCosmos-Embed1 config notes that PEFT cannot inject adapters into Transformer\nEngine layers. Treat the LoRA fields above as the first candidate parameters\nfor manual tuning or AutoML-style search before unfreezing larger model blocks.\nAvoid changing `target_modules` or `modules_to_save` unless the user explicitly\nneeds custom adapter placement.\n\n## S3 Staging\n\nThe Cosmos-Embed1 CLI consumes local paths and Python globs, not raw `s3:\u002F\u002F...\u002F*.mp4` URIs. For S3-backed runs, first stage a subset or full dataset to the execution host\u002Fcontainer filesystem, then use local paths such as `\u002Fdata\u002Fvideo\u002F*.mp4` in the spec.\n\nRecommended S3 layout for staged MSR-VTT data:\n\n```text\ns3:\u002F\u002Fbucket\u002Fpath\u002Fcosmos-embed\u002Fmsrvtt-subset\u002F\n├── msrvtt_test_1k.json\n└── video\u002F\n    ├── video7020.mp4\n    └── ...\n```\n\nAfter downloading\u002Fsyncing that prefix into the mounted `data\u002F` directory, use the same Docker commands above.\n\n## Outputs\n\n```text\nresults\u002F\n├── train\u002F\n│   ├── cosmos_embed1_model_latest.pth\n│   ├── cosmos_embed1_model_\u003Citer>.pth\n│   └── experiment.yaml\n├── evaluate\u002F\n│   ├── metrics.json\n│   └── experiment.yaml\n├── inference\u002F\n│   ├── results.json\n│   └── experiment.yaml\n├── export\u002F\n│   ├── cosmos_embed1_combined.onnx\n│   └── export_config.yaml\n└── export_hf\u002F\n    └── cosmos_embed1_hf\u002F\n```\n\n## Known Pitfalls\n\n| Symptom | Cause | Fix |\n|---|---|---|\n| `MSRVTTDataset: 0 videos found` | `mp4_urls` is not a local glob or metadata filenames do not match videos. | Mount data into the container and set `mp4_urls=\u002Fdata\u002Fvideo\u002F*.mp4`. |\n| HF download\u002Fauth failure | Missing or invalid `HF_TOKEN`, or model agreement not accepted. | Accept the model terms and pass `-e HF_TOKEN`. |\n| LoRA injection failure | Transformer Engine visual encoder is enabled. | Set `model.network.visual_encoder.transformer_engine=false`. |\n| ONNX\u002FHF export complains about missing components | Export checkpoint is partial or adapter-only. | Use a full checkpoint or configure pretrained visual\u002Ftext sources before export. |\n| CUDA OOM | Batch\u002Fresolution too high for the GPU. | Reduce batch size, use 224p, enable LoRA, or use more GPUs. |\n",{"name":5,"tags":23,"license":14,"version":32,"category":33,"metadata":34,"description":37,"allowed-tools":38,"compatibility":39},[24,25,26,27,28,29,30,31],"video","vision-language","vlm","multimodal","retrieval","embedding","cosmos","fine-tuning","1.0.0 Cosmos-Embed1 video-text embedding for text-to-video retrieval, video-to-video search, semantic deduplication, and fine-tuning. Use when the user asks to \"fine-tune Cosmos-Embed1\", \"run cosmos-embed inference\", \"export Cosmos-Embed1\", \"embed videos\", or \"search videos with text\".","development-code",{"author":35,"version":36},"NVIDIA Corporation","0.1.0","","Read Bash","Requires docker + nvidia-container-toolkit, the published Cosmos-Embed TAO container from versions.yaml, and a HuggingFace token when downloading pretrained `nvidia\u002FCosmos-Embed1-*` weights.","main","d591030ef9fb1bbcebed3454aa5c2dcb8d608f69","skills-hub\u002Fai-ml\u002Fnvidia-skills\u002Fskills\u002Ftao-finetune-cosmos-embed","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive","https:\u002F\u002Fai-agent.yyc3.vip\u002F","https:\u002F\u002Fgithub.com\u002FYYC-Cube","万象归元于云枢 | 深栈智启新纪元","2026-11-13 22:26:14",{"source":7,"skillName":5,"commands":49,"prompt":59,"zipAvailable":60},[50,53,56],{"tool":51,"command":52},"npx","npx skills add YYC-Cube\u002FYYC3-AI-Agent-Archive --skill tao-finetune-cosmos-embed",{"tool":54,"command":55},"bunx","bunx skills add YYC-Cube\u002FYYC3-AI-Agent-Archive --skill tao-finetune-cosmos-embed",{"tool":57,"command":58},"pnpm","pnpm dlx skills add YYC-Cube\u002FYYC3-AI-Agent-Archive --skill tao-finetune-cosmos-embed","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive\u002Fblob\u002Fmain\u002Fskills-hub\u002Fai-ml\u002Fnvidia-skills\u002Fskills\u002Ftao-finetune-cosmos-embed\u002FSKILL.md",true,[62,68,72,76,80,84,88,92,96,100],{"path":63,"resourceType":64,"resourceTypeDesc":65,"sizeBytes":66,"sourceHtmlUrl":67},"BENCHMARK.md","REFERENCE","参考文档",4387,"https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive\u002Fblob\u002Fmain\u002Fskills-hub\u002Fai-ml\u002Fnvidia-skills\u002Fskills\u002Ftao-finetune-cosmos-embed\u002FBENCHMARK.md",{"path":69,"resourceType":64,"resourceTypeDesc":65,"sizeBytes":70,"sourceHtmlUrl":71},"evals\u002Fevals.json",787,"https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive\u002Fblob\u002Fmain\u002Fskills-hub\u002Fai-ml\u002Fnvidia-skills\u002Fskills\u002Ftao-finetune-cosmos-embed\u002Fevals\u002Fevals.json",{"path":73,"resourceType":64,"resourceTypeDesc":65,"sizeBytes":74,"sourceHtmlUrl":75},"references\u002Fskill_info.yaml",2515,"https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive\u002Fblob\u002Fmain\u002Fskills-hub\u002Fai-ml\u002Fnvidia-skills\u002Fskills\u002Ftao-finetune-cosmos-embed\u002Freferences\u002Fskill_info.yaml",{"path":77,"resourceType":64,"resourceTypeDesc":65,"sizeBytes":78,"sourceHtmlUrl":79},"references\u002Fspec_template_evaluate.yaml",1201,"https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive\u002Fblob\u002Fmain\u002Fskills-hub\u002Fai-ml\u002Fnvidia-skills\u002Fskills\u002Ftao-finetune-cosmos-embed\u002Freferences\u002Fspec_template_evaluate.yaml",{"path":81,"resourceType":64,"resourceTypeDesc":65,"sizeBytes":82,"sourceHtmlUrl":83},"references\u002Fspec_template_export_hf.yaml",775,"https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive\u002Fblob\u002Fmain\u002Fskills-hub\u002Fai-ml\u002Fnvidia-skills\u002Fskills\u002Ftao-finetune-cosmos-embed\u002Freferences\u002Fspec_template_export_hf.yaml",{"path":85,"resourceType":64,"resourceTypeDesc":65,"sizeBytes":86,"sourceHtmlUrl":87},"references\u002Fspec_template_export_onnx.yaml",847,"https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive\u002Fblob\u002Fmain\u002Fskills-hub\u002Fai-ml\u002Fnvidia-skills\u002Fskills\u002Ftao-finetune-cosmos-embed\u002Freferences\u002Fspec_template_export_onnx.yaml",{"path":89,"resourceType":64,"resourceTypeDesc":65,"sizeBytes":90,"sourceHtmlUrl":91},"references\u002Fspec_template_inference.yaml",1156,"https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive\u002Fblob\u002Fmain\u002Fskills-hub\u002Fai-ml\u002Fnvidia-skills\u002Fskills\u002Ftao-finetune-cosmos-embed\u002Freferences\u002Fspec_template_inference.yaml",{"path":93,"resourceType":64,"resourceTypeDesc":65,"sizeBytes":94,"sourceHtmlUrl":95},"references\u002Fspec_template_train.yaml",2261,"https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive\u002Fblob\u002Fmain\u002Fskills-hub\u002Fai-ml\u002Fnvidia-skills\u002Fskills\u002Ftao-finetune-cosmos-embed\u002Freferences\u002Fspec_template_train.yaml",{"path":97,"resourceType":64,"resourceTypeDesc":65,"sizeBytes":98,"sourceHtmlUrl":99},"skill-card.md",4089,"https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive\u002Fblob\u002Fmain\u002Fskills-hub\u002Fai-ml\u002Fnvidia-skills\u002Fskills\u002Ftao-finetune-cosmos-embed\u002Fskill-card.md",{"path":101,"resourceType":102,"resourceTypeDesc":103,"sizeBytes":104,"sourceHtmlUrl":105},"skill.oms.sig","OTHER","其他",6113,"https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive\u002Fblob\u002Fmain\u002Fskills-hub\u002Fai-ml\u002Fnvidia-skills\u002Fskills\u002Ftao-finetune-cosmos-embed\u002Fskill.oms.sig",[107,115,122,129,136,144],{"skillNo":108,"name":109,"nameZh":6,"description":110,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":111,"licenseFileUrl":6,"sourceHtmlUrl":112,"categories":113,"keywords":114,"publishedAt":18},"01M7BTB92RE8MT5YE706QBNDTQ","yingmi-skill","当用户需要查询基金、策略、公告、财经资讯，做资产配置、组合诊断、风险回测、现金流分析，或生成图表、PDF 时，优先使用本 Skill 获取真实数据与可执行能力。","MIT","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive\u002Fblob\u002Fmain\u002Ftools-hub\u002Fcode-ide\u002Fskills-marketplace\u002Fskills\u002Fyingmi-skill\u002FSKILL.md",[],[],{"skillNo":116,"name":117,"nameZh":6,"description":118,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":111,"licenseFileUrl":6,"sourceHtmlUrl":119,"categories":120,"keywords":121,"publishedAt":18},"01M7BT7ZYR091HPJYEFNWVTEJJ","web-access","所有联网操作必须通过此 skill 处理，包括：搜索、网页抓取、登录后操作、网络交互等。","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive\u002Fblob\u002Fmain\u002Ftools-hub\u002Fcode-ide\u002Fskills-marketplace\u002Fskills\u002Fweb-access\u002FSKILL.md",[],[],{"skillNo":123,"name":124,"nameZh":6,"description":125,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":111,"licenseFileUrl":6,"sourceHtmlUrl":126,"categories":127,"keywords":128,"publishedAt":18},"01M7BT4NKS0DNP8A53M1BGGZR2","tencentmap-map-assistant","腾讯位置服务·地图助手 Skill，一句自然语言调用腾讯地图全套能力，无需开发者账号、开箱即用。提供 AI 旅游攻略、地点搜索（含评分\u002F人均\u002F营业时间）、关键词提示、路线规划（驾车\u002F步行\u002F公交\u002F骑行）、地址解析与逆解析、行政区划、IP 定位、距离计算、天气查询，并可将行程或多 POI 渲染成网页地图。涉及找地点、规划路线、旅游行程、查天气、坐标转换等出行场景时使用。","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive\u002Fblob\u002Fmain\u002Ftools-hub\u002Fcode-ide\u002Fskills-marketplace\u002Fskills\u002Ftencentmap-map-assistant\u002FSKILL.md",[],[],{"skillNo":130,"name":131,"nameZh":6,"description":132,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":111,"licenseFileUrl":6,"sourceHtmlUrl":133,"categories":134,"keywords":135,"publishedAt":18},"01M7BSWKV1T5XYYRGV4W4JP88C","skill-scanner","Scan any agent skill for security risks before you install or use it. Powered by Tencent Zhuque Lab A.I.G (AI-Infra-Guard). 100% local static analysis — no file contents or credentials leave your device. Compatible with CodeBuddy, Cursor, Windsurf, Claude Code, OpenClaw and more. Triggers on: 这个 skill 安全吗, skill 安全扫描, 检查 skill 安全, audit skill, scan skill, check skill safety, analyze skill, inspect skill, verify skill, skill security, skill supply chain. Do NOT trigger for general agent usage, full system health checks, project debugging, or normal development.","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive\u002Fblob\u002Fmain\u002Ftools-hub\u002Fcode-ide\u002Fskills-marketplace\u002Fskills\u002Fskill-scanner\u002FSKILL.md",[],[],{"skillNo":137,"name":138,"nameZh":6,"description":139,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":140,"sourceHtmlUrl":141,"categories":142,"keywords":143,"publishedAt":18},"01M7BSWGTCFW7F123S4BM4XY6W","skill-creator","Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends agent capabilities with specialized knowledge, workflows, or tool integrations.","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive\u002Fblob\u002Fmain\u002Ftools-hub\u002Fcode-ide\u002Fskills-marketplace\u002Fskills\u002Fskill-creator\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive\u002Fblob\u002Fmain\u002Ftools-hub\u002Fcode-ide\u002Fskills-marketplace\u002Fskills\u002Fskill-creator\u002FSKILL.md",[],[],{"skillNo":145,"name":146,"nameZh":6,"description":147,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":111,"licenseFileUrl":6,"sourceHtmlUrl":148,"categories":149,"keywords":150,"publishedAt":18},"01M7BSVV4DH8JK2ZMQSCW25VCD","shader-dev","Comprehensive GLSL shader techniques for creating stunning visual effects — ray marching, SDF modeling, fluid simulation, particle systems, procedural generation, lighting, post-processing, and more.","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-AI-Agent-Archive\u002Fblob\u002Fmain\u002Ftools-hub\u002Fcode-ide\u002Fskills-marketplace\u002Fskills\u002Fshader-dev\u002FSKILL.md",[],[]]