vss-generate-video-report
Use this skill when producing a VSS analysis report — Mode A per-clip VLM, Mode B incident-range via video-analytics. Not for real-time alerts or ad-hoc Q&A.
- License 许可
- Apache-2.0
- 最近更新
- 2026-11-10
- Stars
- ★ 1
- Forks
- 0
01 前言 Front Matter
- name
- vss-generate-video-report
- license
- Apache-2.0
- metadata
- { "tags": "nvidia blueprint operational", "version": "3.2.0", "github-url": "https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization" }
- description
- Use this skill when producing a VSS analysis report — Mode A per-clip VLM, Mode B incident-range via video-analytics. Not for real-time alerts or ad-hoc Q&A.
02 资源文件 4
- 参考文档BENCHMARK.md2.7 KB
- 参考文档evals/base_profile_report.json8.4 KB
- 参考文档skill-card.md2.7 KB
- 其他skill.oms.sig4.5 KB
03 SKILL.md 正文
Report
Generate a video analysis report by routing to one of two backends — never via POST /generate on the VSS agent.
| Mode | Trigger | Backend |
|---|---|---|
| A. Video clip | "report on <sensor>", "report on this video", "analyze warehouse_01.mp4" |
/vss-manage-video-io-storage → clip URL → VLM chat/completions |
| B. Incident range | "report on incidents from <t1> to <t2>", "report on alerts today", "what incidents happened on <sensor> last hour" |
/vss-query-analytics → incident list → narrative report |
If the request is ambiguous (e.g. "report on <sensor>" with no time range and no incident wording), default to Mode A. Ask only if the user mentions both a sensor and a time range.
When to Use
- "Generate a report for this video" / "for
<sensor-id>" — Mode A - "Create an analysis report on the uploaded video" — Mode A
- "Report on incidents from 12:31Z to 12:32Z" — Mode B
- "Summarize alerts on
<sensor>between<t1>and<t2>" — Mode B
Deployment prerequisite
Mode A needs the VSS base profile (VST + VLM NIM). Mode B needs the VSS alerts profile (VA-MCP + Elasticsearch).
Probe:
# Mode A — VST + VLM reachability
curl -sf --max-time 5 "http://${HOST_IP}:30888/vst/api/v1/sensor/version" >/dev/null
# Mode B — VA-MCP
curl -sf --max-time 5 "http://${HOST_IP}:9901/" >/dev/null
If the probe fails, hand off to /vss-deploy-profile with -p base (Mode A) or -p alerts (Mode B). Always confirm the deploy with the user first.
Browser-playable clip URL (always do this before embedding any clip in the report)
VST returns clip URLs using the agent-internal ${HOST_IP}:30888 host:port. Those work in-cluster (VLM frame pulls, agent backend) but the user's browser cannot reach them. The deploy layer already exports the browser-facing host:port as $VSS_PUBLIC_HOST / $VSS_PUBLIC_PORT (and scheme as $VSS_PUBLIC_HTTP_PROTOCOL) in every profile .env — Brev or bare-metal — so the rewrite is a one-liner:
BROWSER_CLIP_URL=$(echo "$RAW_URL" | sed -E "s|^https?://[^/]+|${VSS_PUBLIC_HTTP_PROTOCOL}://${VSS_PUBLIC_HOST}:${VSS_PUBLIC_PORT}|")
Apply it to every clip URL surfaced in the rendered report (Mode A Step 4 Clip URL row; Mode B per-incident clip sub-bullet). Leave the VLM video_url content block in Mode A Step 3 on the original internal URL — the VLM is in-cluster.
Mode A — Report on a recorded video clip
If the VSS lvs profile is deployed — curl -sf --max-time 5 "http://${HOST_IP}:38111/v1/ready" returns HTTP 200 — run /vss-summarize-video to produce the summary, then paste its output into the report template in Step 4 and skip Steps 1–3 (the VLM-direct path). Run Steps 1–3 only when /v1/ready is non-200.
Step 1 — Resolve the clip URL
Hand off to /vss-manage-video-io-storage to:
-
List sensors and confirm the named
<sensor-id>exists (upload first if not). -
Fetch
/storage/<streamId>/timelinesfor the recorded range when the user did not supplystartTime/endTime. -
Request a clip URL:
curl -s "http://${HOST_IP}:30888/vst/api/v1/storage/file/<streamId>/url?startTime=<startTime>&endTime=<endTime>&container=mp4&disableAudio=true" | jq -r .videoUrlThat gives a direct
mp4URL that the VLM can pull frames from. Bind it toVIDEO_URL(used in-cluster by the VLM in Step 3) and rewrite toBROWSER_CLIP_URLfor the Step 4 report template using the one-liner from Browser-playable clip URL above — the user's browser cannot reach$VIDEO_URLdirectly. Mode A requires the selected VLM endpoint to be able to fetchVIDEO_URL. Local NIM/RT-VLM deployments normally can; remote endpoints generally cannot fetchlocalhost, privateHOST_IP, or VST-internal URLs. If the liveVLM_ENDPOINTis remote, surface that reachability requirement instead of making a chat request that will fail after/v1/modelssucceeds.
Step 2 — Resolve VLM endpoint and model
The deploy may serve the VLM through either of two stacks. Both expose an OpenAI-compatible chat/completions API — pick whichever is live:
| Backend | Env vars | Typical host endpoint | Picked when |
|---|---|---|---|
| NIM Cosmos | VLM_BASE_URL, VLM_NAME |
${VLM_BASE_URL}/v1 (no trailing /v1 on the env var; the agent appends it) |
VLM_MODE ∈ {local, local_shared, remote} and VLM_BASE_URL is non-empty |
| RT-VLM Cosmos | RTVI_VLM_BASE_URL, RTVI_VLM_MODEL_TO_USE (model identifier on the RT-VLM side, e.g. cosmos-reason2) |
${RTVI_VLM_BASE_URL}/v1 — alerts default http://${HOST_IP}:8018/v1, base default http://${HOST_IP}:30082/v1 (RTVI_VLM_ENDPOINT) |
VLM_MODE=none or VLM_BASE_URL empty; also the only path for warehouse |
Read the live values off the running agent container — do not guess:
docker exec vss-agent env | grep -E '^(VLM_BASE_URL|VLM_NAME|VLM_MODE|RTVI_VLM_BASE_URL|RTVI_VLM_ENDPOINT|RTVI_VLM_MODEL_TO_USE)='
Selection rule:
if [ -n "${VLM_BASE_URL}" ] && [ "${VLM_MODE}" != "none" ]; then
VLM_ENDPOINT="${VLM_BASE_URL%/}/v1"
VLM_MODEL="${VLM_NAME}"
else
VLM_ENDPOINT="${RTVI_VLM_ENDPOINT:-${RTVI_VLM_BASE_URL%/}/v1}"
VLM_MODEL="${RTVI_VLM_MODEL_TO_USE}"
fi
Probe /v1/models before sending a chat request to confirm the chosen endpoint is alive and the model is loaded:
curl -sf --max-time 5 "${VLM_ENDPOINT}/models" | jq -r '.data[].id'
If the probe fails or the listed ids don't include ${VLM_MODEL}, fall back to the other backend (or surface the error — never silently pick a model that isn't on the server).
Step 3 — Call the VLM directly
Use the OpenAI-compatible chat/completions endpoint with a video_url content block — the same payload shape video_understanding builds in src/vss_agents/tools/video_understanding.py (_build_vlm_messages):
PROMPT='Describe in detail what happens in the video, with timestamps (start–end in seconds from clip start) for each segment or event. Cover scenes, objects, people, vehicles, and notable actions.'
# Cosmos Reason 2 reasoning prompt suffix — matches video_understanding.py for is_cosmos_reason2 + reasoning=true.
# Drop this suffix for non-cosmos-reason2 VLMs.
PROMPT="${PROMPT}
Answer the question using the following format:
<think>
Your reasoning.
</think>
Write your final answer immediately after the </think> tag."
curl -s -X POST "${VLM_ENDPOINT}/chat/completions" \
-H "Content-Type: application/json" \
-d @- <<EOF | jq -r '.choices[0].message.content'
{
"model": "${VLM_MODEL}",
"messages": [
{
"role": "user",
"content": [
{"type": "text", "text": $(jq -Rs . <<< "${PROMPT}")},
{"type": "video_url", "video_url": {"url": "${VIDEO_URL}"}}
]
}
],
"max_tokens": 1024,
"temperature": 0.0
}
EOF
If the VLM returns a <think>…</think> block (Cosmos Reason reasoning mode), keep only the text after </think> as the report body.
Step 4 — Fill the Video Analysis Report template
# Video Analysis Report
## Basic Information
| Field | Value |
|-------|-------|
| **Report Identifier** | vss_report_<YYYYMMDD_HHMMSS> |
| **Date of Analysis** | <YYYY-MM-DD> |
| **Time of Analysis** | <HH:MM:SS> |
| **Video Source** | <sensor_id or filename> |
| **Clip Range** | <startTime> – <endTime> |
| **Clip URL** | `<BROWSER_CLIP_URL>` (apply the `$VSS_PUBLIC_HOST:$VSS_PUBLIC_PORT` rewrite — NEVER paste the raw `HOST_IP:30888` URL here) |
| **VLM** | <VLM_MODEL (NIM or RT-VLM)> |
| **Analysis Request** | <user's request> |
## Analysis Results
<VLM output: timestamped caption / summary>
Return the rendered markdown to the user.
Mode B — Report on incidents in a time range
Step 1 — Resolve the time range and (optionally) sensor
start_time/end_timemust be ISO 8601 UTC (YYYY-MM-DDTHH:MM:SS.sssZ). Resolve relative phrases ("last hour", "today") against the current host clock.- If the user names a sensor, capture it as
source+source_type=sensor. Otherwise leave both unset for an all-sensors query.
Step 2 — Fetch incidents via /vss-query-analytics
Hand off to /vss-query-analytics (initialize → tools/call) with:
{
"name": "video_analytics__get_incidents",
"arguments": {
"source": "<sensor-id-or-omit>",
"source_type": "sensor",
"start_time": "<ISO>",
"end_time": "<ISO>",
"max_count": 100,
"includes": ["objectIds", "info"]
}
}
For each incident keep: id, sensorId, timestamp, end, category, place.name, info.verdict, info.reasoning, objectIds, and the clip URL (commonly info.clip_url, clip_url, or whichever clip-pointer field the response carries). Apply the $VSS_PUBLIC_HOST:$VSS_PUBLIC_PORT rewrite (see Browser-playable clip URL above) to every clip URL before pasting it into the report — the raw value is a HOST_IP:30888 URL the user's browser cannot reach.
Step 3 — Fill the Incident Range Report template
Group by sensor (or by category if no sensor scope), tally verdicts, list each incident as a bullet with timestamp / category / verdict / reasoning.
# Incident Range Report
## Basic Information
| Field | Value |
|-------|-------|
| **Report Identifier** | vss_report_<YYYYMMDD_HHMMSS> |
| **Range** | <start_time> – <end_time> |
| **Scope** | <sensor_id> | all sensors |
| **Total Incidents** | <N> |
| **Confirmed / Rejected / Unverified** | <c> / <r> / <u> |
## Incidents
### <sensor_id_or_category>
- **<timestamp>** — <category> — verdict: **<confirmed|rejected|unverified>**
- <info.reasoning (1–2 lines)>
- clip: `<rewritten URL>` (omit row when the incident carries no clip URL — never paste a raw `HOST_IP:30888` URL)
- objects: <objectIds joined>
- …
## Summary
<2–4 sentences synthesizing what dominates the range — top categories, sensors with the most confirmed incidents, any clusters in time.>
If get_incidents returns zero results, return a one-line report stating the range and scope produced no incidents — do not invent content and do not fall back to Mode A.
Cross-Reference
/vss-manage-video-io-storage— sensor list, timelines, and clip URL for Mode A Step 1./vss-query-analytics— incident retrieval (and verdict / reasoning enrichment) for Mode B Step 2./vss-ask-video— ad-hoc VLM Q&A on a single clip (not a structured report)./vss-summarize-video— used by Mode A to produce the summary body when thelvsprofile is deployed; the report template (Step 4) is still filled here.
04 同仓库更多 Skills
vercel-sandbox
Run agent-browser + Chrome inside Vercel Sandbox microVMs for browser automation from any Vercel-deployed app. Use when the user needs browser automation in a Vercel app (Next.js, SvelteKit, Nuxt, Remix, Astro, etc.), wants to run headless Chrome without binary size limits, needs persistent browser sessions across commands, or wants ephemeral isolated browser environments. Triggers include "Vercel Sandbox browser", "microVM Chrome", "agent-browser in sandbox", "browser automation on Vercel", or any task requiring Chrome in a Vercel Sandbox.
slack
Interact with Slack workspaces using browser automation. Use when the user needs to check unread channels, navigate Slack, send messages, extract data, find information, search conversations, or automate any Slack task. Triggers include "check my Slack", "what channels have unreads", "send a message to", "search Slack for", "extract from Slack", "find who said", or any task requiring programmatic Slack interaction.
electron
Automate Electron desktop apps (VS Code, Slack, Discord, Figma, Notion, Spotify, etc.) using agent-browser via Chrome DevTools Protocol. Use when the user needs to interact with an Electron app, automate a desktop app, connect to a running app, control a native app, or test an Electron application. Triggers include "automate Slack app", "control VS Code", "interact with Discord app", "test this Electron app", "connect to desktop app", or any task requiring automation of a native Electron application.
dogfood
Systematically explore and test a web application to find bugs, UX issues, and other problems. Use when asked to "dogfood", "QA", "exploratory test", "find issues", "bug hunt", "test this app/site/platform", or review the quality of a web application. Produces a structured report with full reproduction evidence -- step-by-step screenshots, repro videos, and detailed repro steps for every issue -- so findings can be handed directly to the responsible teams.
agent-browser
Browser automation CLI for AI agents. Use when the user needs to interact with websites, including navigating pages, filling forms, clicking buttons, taking screenshots, extracting data, testing web apps, or automating any browser task. Triggers include requests to "open a website", "fill out a form", "click a button", "take a screenshot", "scrape data from a page", "test this web app", "login to a site", "automate browser actions", or any task requiring programmatic web interaction.
shadcn
Manages shadcn components and projects — adding, searching, fixing, debugging, styling, and composing UI, including chat interfaces. Provides project context, component docs, and usage examples. Applies when working with shadcn/ui, component registries, presets, --preset codes, or any project with a components.json file. Also triggers for "shadcn init", "create an app with --preset", "switch to --preset", and for building a design system or applying a DESIGN.md or brand spec to shadcn/ui.