[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M78APV5C2RJ0QNEJXGXMBDBZ":3,"skill-same:01M78APV5C2RJ0QNEJXGXMBDBZ":101},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":38,"commitSha":39,"skillDirectory":40,"forksCount":14,"repositoryHtmlUrl":41,"repositoryHomepageUrl":42,"ownerHtmlUrl":43,"ownerBlogUrl":44,"ownerDescription":45,"updatedAt":46,"install":47,"resources":60},"01M78APV5C2RJ0QNEJXGXMBDBZ","tao-generate-video-reasoning-annotations",null,"Multi-step video annotation pipeline that turns raw videos into Chain-of-Thought training data — multi-level captions, structured descriptions, and QA pairs (MCQ, binary, open-ended) with reasoning traces, via VLM\u002FLLM distillation. Use when the user wants to \"create video training data\", \"generate video QA datasets\", \"build CoT reasoning traces from videos\", \"auto-label videos\", or run the video_reasoning_annotation pipeline. Triggers include \"video annotation\", \"video CoT\", \"video QA\", \"chain-of-thought\", \"video captioning pipeline\", \"video distillation\".","Solizardking\u002Fskillhub-main","Solizardking","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F108096810?v=4","01M786KDB10AYV042N0F9BD5B5","Clawd","01M786NYYXSZK1D05WM8JQPC48",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-generate-video-reasoning-annotations\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: tao-generate-video-reasoning-annotations\ndescription: >-\n  Multi-step video annotation pipeline that turns raw videos into\n  Chain-of-Thought training data — multi-level captions, structured\n  descriptions, and QA pairs (MCQ, binary, open-ended) with reasoning\n  traces, via VLM\u002FLLM distillation. Use when the user wants to \"create\n  video training data\", \"generate video QA datasets\", \"build CoT\n  reasoning traces from videos\", \"auto-label videos\", or run the\n  video_reasoning_annotation pipeline. Triggers include \"video\n  annotation\", \"video CoT\", \"video QA\", \"chain-of-thought\",\n  \"video captioning pipeline\", \"video distillation\".\nlicense: Apache-2.0\ncompatibility: Requires docker + nvidia-container-toolkit + at least one VLM endpoint (Gemini API key or OpenAI-compatible).\nmetadata:\n  author: NVIDIA Corporation\n  version: \"0.1.0\"\nallowed-tools: Read Bash Write\ntags:\n  - video\n  - annotation\n  - chain-of-thought\n  - captioning\n  - qa-generation\n  - vlm\n  - llm\n  - auto-label\n---\n\n# Video Reasoning Annotation Pipeline\n\nGenerate Chain-of-Thought training datasets from videos by producing multi-level captions, structured descriptions, and QA pairs (MCQ, binary, open-ended) with step-by-step reasoning traces. Domain-agnostic by default — customize prompts for any video domain.\n\n## Purpose\n\nTransform raw videos into CoT Q&A training data for video understanding models. VLMs (e.g., Gemini, Qwen) act as \"teacher\" annotators: Steps 0–1 require the model to see the video (VLM calls); Steps 2–3 are text-to-text (cheaper LLM calls).\n\n## Pipeline architecture\n\n```\nStep 0:  [Optional] Filter & classify videos  → Keep domain-relevant, classify anomaly vs normal\nStep 1a: Global + dense captions               → VLM: narrative summary + timestamped events\nStep 1b: Chunk captions                         → VLM: fixed-duration segment micro-captions\nStep 1c: [Optional, anomaly only] Highlight     → LLM extracts anomaly timestamp, VLM captions clip\nStep 2:  Description synthesis                  → LLM: synthesize captions into structured narrative\nStep 3:  QA generation                          → LLM: MCQ, binary, open-ended with reasoning\nStep 4:  Parse outputs                          → Per-task `tao-vl-reason-v1.0` JSON files\n```\n\nSteps are individually selectable via `workflow.steps`. The pipeline has built-in resume — each step skips already-processed videos, so re-running after a prompt tweak is safe.\n\n## Initial consultation\n\nWhen the user invokes this skill, walk through these questions in order. Don't skip — getting domain and VLM access right up front prevents wasted runs.\n\n### 1. Videos\n\n- Path to the video directory and\u002For a JSONL with `{\"video_path\": \"...\"}` per line.\n- Confirm format (`.mp4` preferred; `.avi`, `.mov`, `.mkv` also walked).\n\n### 2. Domain — drives prompt selection\n\nAsk the user: *\"What domain are these videos from?\"* Choose one of the following branches:\n\n| Domain | What to do |\n|---|---|\n| **general** | Use the default prompts. Set `prompts_module: \"\"` (or omit). The built-in `nvidia_tao_ds.auto_label.video_reasoning_annotation.prompts` covers domain-agnostic content. |\n| **traffic** (CCTV intersections, highways; dashcam excluded) | Use the reference module. Set `prompts_module: \"nvidia_tao_ds.auto_label.video_reasoning_annotation.prompts_traffic\"`, **or** copy `references\u002Fprompts_traffic.py` into the user's project and tune for their specific camera angles, then point `prompts_module` at the copy. |\n| **warehouse** (industrial site CCTV — safety, operations, security) | Same pattern. Set `prompts_module: \"nvidia_tao_ds.auto_label.video_reasoning_annotation.prompts_warehouse\"`, or copy `references\u002Fprompts_warehouse.py` and tune. |\n| **custom** (any other domain) | **Run the workshop in [references\u002Fdomain_adaptation.md](references\u002Fdomain_adaptation.md)**. It walks through: Phase 1 — question types the user wants the model to answer; Phase 2 — caption-requirements checklist; Phase 3 — fill the `[PLACEHOLDER]` markers in `nvidia_tao_ds.auto_label.video_reasoning_annotation.prompt_template`. The two reference modules above are working examples to model after. Do this **before** any pipeline runs. |\n\n### 3. Anomaly \u002F normal \u002F mixed\n\n- Mixed dataset → `workflow.mode: \"auto\"` (Step 0 classifies each video).\n- Pre-split anomaly only → `workflow.mode: \"anomaly\"`, drop Step 0.\n- Pre-split normal only → `workflow.mode: \"normal\"`, drop Steps 0 and 1c.\n\n### 4. VLM \u002F LLM endpoint — confirm access **before** running\n\n- **Gemini** (default for both `vlm.backend` and `llm.backend`): user needs `GOOGLE_API_KEY` set, or to put the key in the YAML.\n- **OpenAI-compatible** (Qwen via vLLM, NIM endpoint, etc.): user provides `base_url`, `model_name`, and `api_key`.\n- Steps 2–3 are text-only — a smaller\u002Fcheaper LLM is fine for `llm.backend` even when `vlm.backend` is a frontier video model.\n\nIf the user has **no endpoint at all** and wants to self-host, point them at the `skills\u002Fapplications\u002Ftao-run-inference-service` skill — a workflow that stands up a network-specific TAO inference microservice locally and exposes an OpenAI-compatible endpoint. Should support Cosmos, Qwen, and Gemma. Check `skills\u002Fapplications\u002Ftao-run-inference-service\u002Freferences\u002Fservice.yaml` for the current `valid_network_arch_config_basenames` list before relying on a specific model.\n\nIf the user doesn't have endpoint access ready and isn't ready to set one up, stop here and help them figure it out first.\n\n### 5. Pilot vs full run\n\n- **Recommend a 5–10 video pilot** when domain is `custom`, when any prompt was edited, or when this is the user's first run.\n- **Full-run is fine** for `general` \u002F `traffic` \u002F `warehouse` once the user has previously verified output quality on the same data type.\n- The pipeline has built-in resume, so a pilot followed by a full run does not re-process the pilot videos.\n\n## Quick start\n\nThe pipeline runs inside the TAO Toolkit container via the `auto_label` CLI:\n\n```bash\nauto_label generate -e \u002Fpath\u002Fto\u002Fspec.yaml \\\n    results_dir=\u002Fresults \\\n    video_reasoning_annotation.data.video_root=\u002Fvideos \\\n    video_reasoning_annotation.vlm.gemini.api_key=$GOOGLE_API_KEY \\\n    video_reasoning_annotation.workflow.mode=auto\n```\n\nGenerate a default spec to start from:\n\n```bash\nauto_label default_specs results_dir=\u002Fresults module_name=auto_label\n# then set:  autolabel_type: \"video_reasoning_annotation\"\n```\n\nAll fields support Hydra dot-notation overrides on the command line. For the full YAML reference (every field, model\u002Fendpoint setup, error patterns), see [references\u002Fconfiguration.md](references\u002Fconfiguration.md).\n\n## Pilot workflow\n\nUse this when running a 5–10 video pilot:\n\n1. Run the pipeline on the pilot subset with the chosen `prompts_module` and `workflow.mode`.\n2. Inspect `results_dir\u002Fstep_1a_caption\u002Fcaptions.jsonl` — captions accurate, capturing the right level of detail?\n3. Inspect `results_dir\u002Fstep_3_qa\u002Fqa_output.jsonl` — questions meaningful, answers correct, reasoning logical?\n4. If quality is insufficient: adjust the prompts (in `prompts_module` if domain-customized, or fall back to `general` if a domain module is over-tuned), and re-run. The pipeline auto-skips already-processed videos.\n5. Once satisfied, scale to the full dataset by pointing `data.video_root` (or `data.input_jsonl_files`) at the full set and re-running with the same `results_dir` (resume) or a fresh one (full re-run).\n\nQuality compounds downstream — bad captions produce bad descriptions which produce bad QA. Focus iteration on Step 1a\u002F1b output first; descriptions and QA usually improve once captions are right.\n\n## Configuration summary\n\nKey fields (full reference in [references\u002Fconfiguration.md](references\u002Fconfiguration.md)):\n\n| Field | Default | Description |\n|---|---|---|\n| `workflow.steps` | `[\"0\",\"1a\",\"1b\",\"1c\",\"2\",\"3\",\"4\"]` | Which pipeline steps to execute |\n| `workflow.mode` | `\"auto\"` | `\"auto\"`, `\"anomaly\"`, or `\"normal\"` |\n| `vlm.backend` | `\"gemini\"` | `\"gemini\"` or `\"openai\"` (OpenAI-compatible) |\n| `llm.backend` | `\"gemini\"` | Same options; text-only, cheaper model works |\n| `workflow.max_workers` | `4` | Parallel threads per step (watch API rate limits) |\n| `license` | `\"\"` | Optional: written to `metadata.license` in step 4 outputs (e.g. `\"CC-BY-4.0\"`) |\n| `description_extra` | `\"\"` | Optional: extra text appended to per-task descriptions in step 4 metadata |\n| `prompts_module` | `\"\"` | Dotted import path to custom prompts module |\n\n## Prompts\n\n- **Built-in (general)**: `nvidia_tao_ds.auto_label.video_reasoning_annotation.prompts` — domain-agnostic, used by default.\n- **Template**: `nvidia_tao_ds.auto_label.video_reasoning_annotation.prompt_template` — same 26 keys with `[PLACEHOLDER]` markers for domain customization.\n- **Reference modules** (working examples for the consultation's `traffic` \u002F `warehouse` branches): [references\u002Fprompts_traffic.py](references\u002Fprompts_traffic.py), [references\u002Fprompts_warehouse.py](references\u002Fprompts_warehouse.py).\n- **Custom domains**: see [references\u002Fdomain_adaptation.md](references\u002Fdomain_adaptation.md) for the full workshop and placeholder reference.\n\n## Inputs\n\n- **`video_root`**: Directory of videos (walked recursively for `.mp4`, `.avi`, `.mov`, `.mkv`).\n- **`input_jsonl_files`**: List of JSONL files with `{\"video_path\": \"...\"}` per line. The `video` key is also accepted; extra fields are allowed.\n- **`filter_field`**: Optional boolean field to filter JSONL entries.\n\nProvide `video_root`, `input_jsonl_files`, or both (lists merge).\n\n## Outputs\n\nAll outputs go to `results_dir\u002F` with per-step subdirectories (`step_0_filter\u002F`, `step_1a_caption\u002F`, …, `step_4_output\u002F`):\n\n- **Steps 0–3**: JSONL — one JSON object per video per line.\n- **Step 4**: One `\u003Ctask>.json` per non-empty task type, in the **`tao-vl-reason-v1.0`** envelope. Up to 10 files: `mcq.json`, `mcq_openended.json`, `bcq.json`, `bcq_openended.json`, `open_qa.json`, `causal_linkage.json`, `temporal_localization.json`, `temporal_description.json`, `scene_description.json`, `video_summarization.json`.\n\nEach step 4 file looks like:\n\n```json\n{\n  \"format\": \"tao-vl-reason-v1.0\",\n  \"metadata\": {\"type\": \"annotation\", \"task\": \"\u003Ctask>\", \"date\": \"YYYY-MM-DD\",\n               \"description\": \"\u003Cper-task + description_extra>\", \"license\": \"\u003Cfrom config>\"},\n  \"media_root\": \"\u003Cdata.video_root>\" | null,\n  \"items\": [{\"video_id\": \"...\", \"question\": \"...\", \"answer\": \"...\", \"reasoning\": \"...\"}, ...]\n}\n```\n\n`media_root` mirrors `data.video_root` (or `null` when unset); each item's `video_id` is the entry's video path with the `video_root` prefix stripped. Set `license` and `description_extra` in the spec to populate the metadata.\n\n## Prerequisites\n\n- **Container**: `tao_toolkit.pyt` (resolves to `nvcr.io\u002Fnvidia\u002Ftao\u002Ftao-toolkit:6.26.3-pyt` via `versions.yaml`).\n- **ffmpeg \u002F ffprobe**: required for chunk captioning (Step 1b) and highlight extraction (Step 1c).\n- **VLM endpoint**: at least one — Gemini API key or OpenAI-compatible endpoint.\n","# Video Reasoning Annotation Pipeline\n\nGenerate Chain-of-Thought training datasets from videos by producing multi-level captions, structured descriptions, and QA pairs (MCQ, binary, open-ended) with step-by-step reasoning traces. Domain-agnostic by default — customize prompts for any video domain.\n\n## Purpose\n\nTransform raw videos into CoT Q&A training data for video understanding models. VLMs (e.g., Gemini, Qwen) act as \"teacher\" annotators: Steps 0–1 require the model to see the video (VLM calls); Steps 2–3 are text-to-text (cheaper LLM calls).\n\n## Pipeline architecture\n\n```\nStep 0:  [Optional] Filter & classify videos  → Keep domain-relevant, classify anomaly vs normal\nStep 1a: Global + dense captions               → VLM: narrative summary + timestamped events\nStep 1b: Chunk captions                         → VLM: fixed-duration segment micro-captions\nStep 1c: [Optional, anomaly only] Highlight     → LLM extracts anomaly timestamp, VLM captions clip\nStep 2:  Description synthesis                  → LLM: synthesize captions into structured narrative\nStep 3:  QA generation                          → LLM: MCQ, binary, open-ended with reasoning\nStep 4:  Parse outputs                          → Per-task `tao-vl-reason-v1.0` JSON files\n```\n\nSteps are individually selectable via `workflow.steps`. The pipeline has built-in resume — each step skips already-processed videos, so re-running after a prompt tweak is safe.\n\n## Initial consultation\n\nWhen the user invokes this skill, walk through these questions in order. Don't skip — getting domain and VLM access right up front prevents wasted runs.\n\n### 1. Videos\n\n- Path to the video directory and\u002For a JSONL with `{\"video_path\": \"...\"}` per line.\n- Confirm format (`.mp4` preferred; `.avi`, `.mov`, `.mkv` also walked).\n\n### 2. Domain — drives prompt selection\n\nAsk the user: *\"What domain are these videos from?\"* Choose one of the following branches:\n\n| Domain | What to do |\n|---|---|\n| **general** | Use the default prompts. Set `prompts_module: \"\"` (or omit). The built-in `nvidia_tao_ds.auto_label.video_reasoning_annotation.prompts` covers domain-agnostic content. |\n| **traffic** (CCTV intersections, highways; dashcam excluded) | Use the reference module. Set `prompts_module: \"nvidia_tao_ds.auto_label.video_reasoning_annotation.prompts_traffic\"`, **or** copy `references\u002Fprompts_traffic.py` into the user's project and tune for their specific camera angles, then point `prompts_module` at the copy. |\n| **warehouse** (industrial site CCTV — safety, operations, security) | Same pattern. Set `prompts_module: \"nvidia_tao_ds.auto_label.video_reasoning_annotation.prompts_warehouse\"`, or copy `references\u002Fprompts_warehouse.py` and tune. |\n| **custom** (any other domain) | **Run the workshop in [references\u002Fdomain_adaptation.md](references\u002Fdomain_adaptation.md)**. It walks through: Phase 1 — question types the user wants the model to answer; Phase 2 — caption-requirements checklist; Phase 3 — fill the `[PLACEHOLDER]` markers in `nvidia_tao_ds.auto_label.video_reasoning_annotation.prompt_template`. The two reference modules above are working examples to model after. Do this **before** any pipeline runs. |\n\n### 3. Anomaly \u002F normal \u002F mixed\n\n- Mixed dataset → `workflow.mode: \"auto\"` (Step 0 classifies each video).\n- Pre-split anomaly only → `workflow.mode: \"anomaly\"`, drop Step 0.\n- Pre-split normal only → `workflow.mode: \"normal\"`, drop Steps 0 and 1c.\n\n### 4. VLM \u002F LLM endpoint — confirm access **before** running\n\n- **Gemini** (default for both `vlm.backend` and `llm.backend`): user needs `GOOGLE_API_KEY` set, or to put the key in the YAML.\n- **OpenAI-compatible** (Qwen via vLLM, NIM endpoint, etc.): user provides `base_url`, `model_name`, and `api_key`.\n- Steps 2–3 are text-only — a smaller\u002Fcheaper LLM is fine for `llm.backend` even when `vlm.backend` is a frontier video model.\n\nIf the user has **no endpoint at all** and wants to self-host, point them at the `skills\u002Fapplications\u002Ftao-run-inference-service` skill — a workflow that stands up a network-specific TAO inference microservice locally and exposes an OpenAI-compatible endpoint. Should support Cosmos, Qwen, and Gemma. Check `skills\u002Fapplications\u002Ftao-run-inference-service\u002Freferences\u002Fservice.yaml` for the current `valid_network_arch_config_basenames` list before relying on a specific model.\n\nIf the user doesn't have endpoint access ready and isn't ready to set one up, stop here and help them figure it out first.\n\n### 5. Pilot vs full run\n\n- **Recommend a 5–10 video pilot** when domain is `custom`, when any prompt was edited, or when this is the user's first run.\n- **Full-run is fine** for `general` \u002F `traffic` \u002F `warehouse` once the user has previously verified output quality on the same data type.\n- The pipeline has built-in resume, so a pilot followed by a full run does not re-process the pilot videos.\n\n## Quick start\n\nThe pipeline runs inside the TAO Toolkit container via the `auto_label` CLI:\n\n```bash\nauto_label generate -e \u002Fpath\u002Fto\u002Fspec.yaml \\\n    results_dir=\u002Fresults \\\n    video_reasoning_annotation.data.video_root=\u002Fvideos \\\n    video_reasoning_annotation.vlm.gemini.api_key=$GOOGLE_API_KEY \\\n    video_reasoning_annotation.workflow.mode=auto\n```\n\nGenerate a default spec to start from:\n\n```bash\nauto_label default_specs results_dir=\u002Fresults module_name=auto_label\n# then set:  autolabel_type: \"video_reasoning_annotation\"\n```\n\nAll fields support Hydra dot-notation overrides on the command line. For the full YAML reference (every field, model\u002Fendpoint setup, error patterns), see [references\u002Fconfiguration.md](references\u002Fconfiguration.md).\n\n## Pilot workflow\n\nUse this when running a 5–10 video pilot:\n\n1. Run the pipeline on the pilot subset with the chosen `prompts_module` and `workflow.mode`.\n2. Inspect `results_dir\u002Fstep_1a_caption\u002Fcaptions.jsonl` — captions accurate, capturing the right level of detail?\n3. Inspect `results_dir\u002Fstep_3_qa\u002Fqa_output.jsonl` — questions meaningful, answers correct, reasoning logical?\n4. If quality is insufficient: adjust the prompts (in `prompts_module` if domain-customized, or fall back to `general` if a domain module is over-tuned), and re-run. The pipeline auto-skips already-processed videos.\n5. Once satisfied, scale to the full dataset by pointing `data.video_root` (or `data.input_jsonl_files`) at the full set and re-running with the same `results_dir` (resume) or a fresh one (full re-run).\n\nQuality compounds downstream — bad captions produce bad descriptions which produce bad QA. Focus iteration on Step 1a\u002F1b output first; descriptions and QA usually improve once captions are right.\n\n## Configuration summary\n\nKey fields (full reference in [references\u002Fconfiguration.md](references\u002Fconfiguration.md)):\n\n| Field | Default | Description |\n|---|---|---|\n| `workflow.steps` | `[\"0\",\"1a\",\"1b\",\"1c\",\"2\",\"3\",\"4\"]` | Which pipeline steps to execute |\n| `workflow.mode` | `\"auto\"` | `\"auto\"`, `\"anomaly\"`, or `\"normal\"` |\n| `vlm.backend` | `\"gemini\"` | `\"gemini\"` or `\"openai\"` (OpenAI-compatible) |\n| `llm.backend` | `\"gemini\"` | Same options; text-only, cheaper model works |\n| `workflow.max_workers` | `4` | Parallel threads per step (watch API rate limits) |\n| `license` | `\"\"` | Optional: written to `metadata.license` in step 4 outputs (e.g. `\"CC-BY-4.0\"`) |\n| `description_extra` | `\"\"` | Optional: extra text appended to per-task descriptions in step 4 metadata |\n| `prompts_module` | `\"\"` | Dotted import path to custom prompts module |\n\n## Prompts\n\n- **Built-in (general)**: `nvidia_tao_ds.auto_label.video_reasoning_annotation.prompts` — domain-agnostic, used by default.\n- **Template**: `nvidia_tao_ds.auto_label.video_reasoning_annotation.prompt_template` — same 26 keys with `[PLACEHOLDER]` markers for domain customization.\n- **Reference modules** (working examples for the consultation's `traffic` \u002F `warehouse` branches): [references\u002Fprompts_traffic.py](references\u002Fprompts_traffic.py), [references\u002Fprompts_warehouse.py](references\u002Fprompts_warehouse.py).\n- **Custom domains**: see [references\u002Fdomain_adaptation.md](references\u002Fdomain_adaptation.md) for the full workshop and placeholder reference.\n\n## Inputs\n\n- **`video_root`**: Directory of videos (walked recursively for `.mp4`, `.avi`, `.mov`, `.mkv`).\n- **`input_jsonl_files`**: List of JSONL files with `{\"video_path\": \"...\"}` per line. The `video` key is also accepted; extra fields are allowed.\n- **`filter_field`**: Optional boolean field to filter JSONL entries.\n\nProvide `video_root`, `input_jsonl_files`, or both (lists merge).\n\n## Outputs\n\nAll outputs go to `results_dir\u002F` with per-step subdirectories (`step_0_filter\u002F`, `step_1a_caption\u002F`, …, `step_4_output\u002F`):\n\n- **Steps 0–3**: JSONL — one JSON object per video per line.\n- **Step 4**: One `\u003Ctask>.json` per non-empty task type, in the **`tao-vl-reason-v1.0`** envelope. Up to 10 files: `mcq.json`, `mcq_openended.json`, `bcq.json`, `bcq_openended.json`, `open_qa.json`, `causal_linkage.json`, `temporal_localization.json`, `temporal_description.json`, `scene_description.json`, `video_summarization.json`.\n\nEach step 4 file looks like:\n\n```json\n{\n  \"format\": \"tao-vl-reason-v1.0\",\n  \"metadata\": {\"type\": \"annotation\", \"task\": \"\u003Ctask>\", \"date\": \"YYYY-MM-DD\",\n               \"description\": \"\u003Cper-task + description_extra>\", \"license\": \"\u003Cfrom config>\"},\n  \"media_root\": \"\u003Cdata.video_root>\" | null,\n  \"items\": [{\"video_id\": \"...\", \"question\": \"...\", \"answer\": \"...\", \"reasoning\": \"...\"}, ...]\n}\n```\n\n`media_root` mirrors `data.video_root` (or `null` when unset); each item's `video_id` is the entry's video path with the `video_root` prefix stripped. Set `license` and `description_extra` in the spec to populate the metadata.\n\n## Prerequisites\n\n- **Container**: `tao_toolkit.pyt` (resolves to `nvcr.io\u002Fnvidia\u002Ftao\u002Ftao-toolkit:6.26.3-pyt` via `versions.yaml`).\n- **ffmpeg \u002F ffprobe**: required for chunk captioning (Step 1b) and highlight extraction (Step 1c).\n- **VLM endpoint**: at least one — Gemini API key or OpenAI-compatible endpoint.\n",{"name":5,"tags":24,"license":15,"metadata":33,"description":7,"allowed-tools":36,"compatibility":37},[25,26,27,28,29,30,31,32],"video","annotation","chain-of-thought","captioning","qa-generation","vlm","llm","auto-label",{"author":34,"version":35},"NVIDIA Corporation","0.1.0","Read Bash Write","Requires docker + nvidia-container-toolkit + at least one VLM endpoint (Gemini API key or OpenAI-compatible).","main","be3de0244dc1a57e323d8160787ebb40fb64930e","public\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-generate-video-reasoning-annotations","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main","https:\u002F\u002Fskills.x402.wtf","https:\u002F\u002Fgithub.com\u002FSolizardking","https:\u002F\u002Fmusebook.trade","Got a mile high and never came down","2026-10-08 23:20:13",{"source":8,"skillName":5,"commands":48,"prompt":58,"zipAvailable":59},[49,52,55],{"tool":50,"command":51},"npx","npx skills add Solizardking\u002Fskillhub-main --skill tao-generate-video-reasoning-annotations",{"tool":53,"command":54},"bunx","bunx skills add Solizardking\u002Fskillhub-main --skill tao-generate-video-reasoning-annotations",{"tool":56,"command":57},"pnpm","pnpm dlx skills add Solizardking\u002Fskillhub-main --skill tao-generate-video-reasoning-annotations","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-generate-video-reasoning-annotations\u002FSKILL.md",true,[61,67,71,75,79,85,89,93,97],{"path":62,"resourceType":63,"resourceTypeDesc":64,"sizeBytes":65,"sourceHtmlUrl":66},"BENCHMARK.md","REFERENCE","参考文档",4079,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-generate-video-reasoning-annotations\u002FBENCHMARK.md",{"path":68,"resourceType":63,"resourceTypeDesc":64,"sizeBytes":69,"sourceHtmlUrl":70},"metadata.json",854,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-generate-video-reasoning-annotations\u002Fmetadata.json",{"path":72,"resourceType":63,"resourceTypeDesc":64,"sizeBytes":73,"sourceHtmlUrl":74},"references\u002Fconfiguration.md",8678,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-generate-video-reasoning-annotations\u002Freferences\u002Fconfiguration.md",{"path":76,"resourceType":63,"resourceTypeDesc":64,"sizeBytes":77,"sourceHtmlUrl":78},"references\u002Fdomain_adaptation.md",8317,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-generate-video-reasoning-annotations\u002Freferences\u002Fdomain_adaptation.md",{"path":80,"resourceType":81,"resourceTypeDesc":82,"sizeBytes":83,"sourceHtmlUrl":84},"references\u002Fprompts_traffic.py","SCRIPT","脚本",80578,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-generate-video-reasoning-annotations\u002Freferences\u002Fprompts_traffic.py",{"path":86,"resourceType":81,"resourceTypeDesc":82,"sizeBytes":87,"sourceHtmlUrl":88},"references\u002Fprompts_warehouse.py",87002,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-generate-video-reasoning-annotations\u002Freferences\u002Fprompts_warehouse.py",{"path":90,"resourceType":63,"resourceTypeDesc":64,"sizeBytes":91,"sourceHtmlUrl":92},"references\u002Fskill_info.yaml",1479,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-generate-video-reasoning-annotations\u002Freferences\u002Fskill_info.yaml",{"path":94,"resourceType":63,"resourceTypeDesc":64,"sizeBytes":95,"sourceHtmlUrl":96},"skill-card.md",4224,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-generate-video-reasoning-annotations\u002Fskill-card.md",{"path":98,"resourceType":63,"resourceTypeDesc":64,"sizeBytes":99,"sourceHtmlUrl":100},"verification.json",2719,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-generate-video-reasoning-annotations\u002Fverification.json",[102,110,117,124,131,139],{"skillNo":103,"name":104,"nameZh":6,"description":105,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":106,"licenseFileUrl":6,"sourceHtmlUrl":107,"categories":108,"keywords":109,"publishedAt":19},"01M78E0TGCAMNPS207BA3G07S7","zkrouter","Self-hosted OpenAI-compatible LLM router — smart tier-based routing, ZK-stamped routing decisions, mode overrides, free OpenRouter at install via the Birth bot. Save 60-80% on inference costs by routing to the cheapest capable model across 12+ OpenRouter models.","MIT","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzkrouter\u002FSKILL.md",[],[],{"skillNo":111,"name":112,"nameZh":6,"description":113,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":106,"licenseFileUrl":6,"sourceHtmlUrl":114,"categories":115,"keywords":116,"publishedAt":19},"01M78E0R1Z8BDT6EDTHY17MTNB","zk","For custom ZK Solana programs and privacy-preserving applications to prevent double spending. Guide to integrate rent-free nullifier PDAs for double-spend prevention.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzk\u002FSKILL.md",[],[],{"skillNo":118,"name":119,"nameZh":6,"description":120,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":106,"licenseFileUrl":6,"sourceHtmlUrl":121,"categories":122,"keywords":123,"publishedAt":19},"01M78E0P3ZTZ1YWF8MD5MM23QE","zk-omni-messaging","Zero-knowledge omnichain messaging between Robinhood Chain and Solana using CheshireZkOmniMessenger (msgType 4) with nullifier anti-replay and the zk-omni-relayer service.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzk-omni-messaging\u002FSKILL.md",[],[],{"skillNo":125,"name":126,"nameZh":6,"description":127,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":106,"licenseFileUrl":6,"sourceHtmlUrl":128,"categories":129,"keywords":130,"publishedAt":19},"01M78E0JXZ2PNB1896VXAGDBWD","zerion-wallet-data","Use when the user asks to check a wallet's portfolio, token balances, DeFi positions, PnL, transaction history, or NFTs. Provides interpreted on-chain data across 40+ EVM chains and Solana via x402 pay-per-request or API key — no raw RPC decoding needed.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzerion-wallet-data\u002FSKILL.md",[],[],{"skillNo":132,"name":133,"nameZh":6,"description":134,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":106,"licenseFileUrl":135,"sourceHtmlUrl":136,"categories":137,"keywords":138,"publishedAt":19},"01M78E0G14FH9GRRTQYMKD5E59","youtube-clipper","YouTube 视频智能剪辑工具。下载视频和字幕，AI 分析生成精细章节（几分钟级别）， 用户选择片段后自动剪辑、翻译字幕为中英双语、烧录字幕到视频，并生成总结文案。 使用场景：当用户需要剪辑 YouTube 视频、生成短视频片段、制作双语字幕版本时。 关键词：视频剪辑、YouTube、字幕翻译、双语字幕、视频下载、clip video","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fyoutube-clipper\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fyoutube-clipper\u002FSKILL.md",[],[],{"skillNo":140,"name":141,"nameZh":6,"description":142,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":106,"licenseFileUrl":6,"sourceHtmlUrl":143,"categories":144,"keywords":145,"publishedAt":19},"01M78E0C2ZYA99CFFDNB7ZMK5S","yield-optimization","Multi-chain yield optimization via yield.xyz (StakeKit). Discover 2,988+ yield opportunities across 75+ blockchains, build deposit\u002Fwithdrawal transactions with shell scripts, and sign them with a MoonPay wallet. Use when the user wants to earn yield, find the best lending or staking rate, enter\u002Fexit a position, or check portfolio balances.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fyield-optimization\u002FSKILL.md",[],[]]