[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M79FN7YR9ENC8WTCYRH8PB9J":3,"skill-same:01M79FN7YR9ENC8WTCYRH8PB9J":104},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":48,"commitSha":49,"skillDirectory":50,"forksCount":14,"repositoryHtmlUrl":51,"repositoryHomepageUrl":6,"ownerHtmlUrl":52,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":53,"install":54,"resources":67},"01M79FN7YR9ENC8WTCYRH8PB9J","rag-perf",null,"Performance benchmarking for a deployed NVIDIA RAG Blueprint server: profiling pass + aiperf load test driven by a single YAML config. Not for accuracy \u002F RAGAS scoring (use rag-eval) or for deploying \u002F repairing services (use rag-blueprint).","Elogic360\u002Fsoftware-factory","Elogic360","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F156535519?v=4","01M74NH3F0RB13PWB7JZDE2V80","elogic360","01M74NH3FPC65082MDZVMFHDM3",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Frag-perf\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: rag-perf\nversion: \"2.6.0\"\ndescription: >-\n  Performance benchmarking for a deployed NVIDIA RAG Blueprint server: profiling pass + aiperf\n  load test driven by a single YAML config. Not for accuracy \u002F RAGAS scoring (use rag-eval) or\n  for deploying \u002F repairing services (use rag-blueprint).\nlicense: Apache-2.0\ncompatibility: Repository checkout with uv; Python 3.11+; run from repo root; uv sync --project scripts\u002Frag-perf (perf deps live in scripts\u002Frag-perf\u002Fpyproject.toml); reachable RAG server (default http:\u002F\u002Flocalhost:8081); for synthetic queries an OpenAI-compatible chat-completions endpoint is required (default http:\u002F\u002Flocalhost:8999\u002Fv1\u002Fchat\u002Fcompletions); aiperf load-test phase uses the bundled nvidia_rag endpoint plugin, registered automatically when rag-perf is installed editable.\nmetadata:\n  tool-version: \"0.1.0\"\n  author: NVIDIA RAG \u003Cfoundational-rag-dev@exchange.nvidia.com>\n  github-url: \"https:\u002F\u002Fgithub.com\u002FNVIDIA-AI-Blueprints\u002Frag\"\n  endpoint-openapi-schemas:\n    - docs\u002Fapi_reference\u002Fopenapi_schema_rag_server.json\n  argument-hint: rag-perf | aiperf | TTFT | latency | throughput | concurrency sweep | bottleneck | retrieval \u002F reranker tuning | profile-only | synthetic queries | quick_profile.yaml | single_run.yaml | sweep.yaml | uv run --project scripts\u002Frag-perf\n  tags:\n    - nvidia\n    - blueprint\n    - rag\n    - performance\n    - benchmarking\n    - aiperf\n    - nvidia-rag-blueprint\n  languages:\n    - python\n    - shell\n  frameworks:\n    - aiperf\n    - fastapi\n  domain: ai-ml\nallowed-tools: Read Grep Glob Bash(ls *) Bash(python3 *) Bash(uv *) Bash(cat *) Bash(curl *) Write Edit\n---\n\n# RAG-Perf — config-driven perf benchmark CLI\n\n## Purpose\n\nDrive a deployed NVIDIA RAG Blueprint server with a YAML config, run a server-side **profiling pass** (per-stage timing, citation quality, bottleneck inference) and an optional **aiperf load test** (TTFT \u002F E2E \u002F token & request throughput \u002F error rate), and write a unified report. The CLI is intentionally minimal: `rag-perf -c \u003Cconfig>` plus `--help` \u002F `--version`. Behaviour is *fully* config-driven; field variations belong in YAML.\n\n## Scope\n\n- **Accuracy \u002F RAGAS** scoring of answer quality → use the **rag-eval** skill.\n- **Deploying, repairing, or configuring services** (compose, helm, NIM env vars) → use the **rag-blueprint** skill.\n- **Production monitoring \u002F alerting** — rag-perf is a one-shot benchmark tool.\n- **Runtime requirement:** a deployed RAG server reachable on the network.\n\n## Prerequisites\n\n- Repo cloned; **run commands from the repo root** (config paths in the presets are repo-root-relative).\n- Python **3.11+** and **uv** on PATH.\n- Install rag-perf into its own uv-managed venv: `uv sync --project scripts\u002Frag-perf`.\n- For unit tests: install dev extras as well — `uv sync --project scripts\u002Frag-perf --extra dev` (otherwise `pytest-asyncio` is missing and async tests error out at collection time).\n- A reachable RAG server (default `http:\u002F\u002Flocalhost:8081`). For the aiperf phase, the bundled `nvidia_rag` endpoint plugin must be installed — `pip install -e .\u002Fscripts\u002Frag-perf` registers it via the `aiperf.plugins` entry point.\n- For **synthetic** queries: an OpenAI-compatible chat-completions endpoint reachable at `synthetic.llm_url` (default `http:\u002F\u002Flocalhost:8999\u002Fv1\u002Fchat\u002Fcompletions`).\n- rag-perf itself runs without `NVIDIA_API_KEY` (unlike rag-eval). The synthetic LLM endpoint may require its own auth — that's the deployment's concern.\n\n## Instructions\n\n1. **Pick a preset.** The three under [`scripts\u002Frag-perf\u002Fconfigs\u002F`](..\u002F..\u002Fscripts\u002Frag-perf\u002Fconfigs) are:\n   - `quick_profile.yaml` — profile-only, ~30 s. Skips load test. For fast iteration on retrieval \u002F reranker tuning.\n   - `single_run.yaml` — one concurrency level, profiling + aiperf, ~2 min. Regression checks.\n   - `sweep.yaml` — multi-axis sweep. `load.concurrency`, `rag.vdb_top_k`, `rag.reranker_top_k` are all `int | list[int]`; any of them as a list becomes a sweep axis (Cartesian product).\n\n2. **Edit the preset.** **Required:** replace `rag.collection_names: [\"\u003Ccollection_name>\"]` with a real collection on the deployed ingestor server. Verify the collection exists via `GET \u002Fv1\u002Fcollections` on the ingestor. The placeholder `\u003Ccollection_name>` validates fine but every request will fail at retrieval. Use a copied YAML preset for variants; the CLI surface is intentionally config-only.\n\n3. **Run.** From repo root:\n   ```bash\n   uv run --project scripts\u002Frag-perf rag-perf -c scripts\u002Frag-perf\u002Fconfigs\u002Fsingle_run.yaml\n   ```\n   Same form for the other presets. The CLI accepts only `-c \u002F --config` (required), `--help`, `--version`.\n\n4. **Read stdout.** Every invocation prints, in order: a startup banner, a one-line summary, the **fully resolved config as YAML** (so the run is reproducible from terminal output), per-grid-point progress with the **shlex-joined aiperf command** in copy-pastable form, a **rich per-point summary table** (stage breakdown with bars, citation quality, bottleneck, load-test block), and finally a **side-by-side comparison table** auto-labelled by whichever axis varied. See [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md).\n\n5. **Inspect artifacts.** Layout depends on run shape — flat for single-point + `iterations=1`, nested under `iter_\u003Ci>\u002F\u003Cpoint>\u002F...` otherwise. See [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md) for the full directory tree, file purposes, and how to parse `results.json` \u002F `results.csv` \u002F `report.md`.\n\n6. **Summarise for the user.** When reporting back, follow the playbook in [`references\u002Foutput-and-analysis.md#summarising-results-to-the-user`](references\u002Foutput-and-analysis.md#summarising-results-to-the-user): pick the canonical result file for the run shape, build a headline table (concurrency × top-k axes × TTFT × throughput × bottleneck × citation quality), compute scaling efficiency on sweeps, **always flag** zero citations \u002F non-zero error rate \u002F suspect `llm_ttft_ms` \u002F small-sample p99, and propose a concrete next-experiment YAML.\n\n7. **Tune.** Schema is fully documented in [`docs\u002Fperformance-benchmarking.md`](..\u002F..\u002Fdocs\u002Fperformance-benchmarking.md) and the deeper-dive references below. Common knobs: turn `aiperf.enabled: false` for profile-only mode, increase `load.iterations` for variance estimation, set `load.sleep_between_points_s: 60` for overnight Cartesian sweeps.\n\n## Examples\n\n**Profile-only (quickest signal on retrieval \u002F reranker tuning):**\n\n```bash\nuv run --project scripts\u002Frag-perf rag-perf -c scripts\u002Frag-perf\u002Fconfigs\u002Fquick_profile.yaml\n```\n\nOutput: `rag-perf-results\u002Fquick_profile\u002Frun_\u003Cts>\u002F{profile_report.md, profile_results.json, profiling\u002F}`. The `aiperf_rag_on\u002F` directory is omitted. Filenames are `profile_*` because `aiperf.enabled: false`.\n\n**Single benchmark point with full report:**\n\n```bash\nuv run --project scripts\u002Frag-perf rag-perf -c scripts\u002Frag-perf\u002Fconfigs\u002Fsingle_run.yaml\n```\n\nOutput: flat `run_\u003Cts>\u002F{report.md, results.json, results.csv, profiling\u002F, aiperf_rag_on\u002F}`.\n\n**Concurrency sweep:**\n\n```bash\nuv run --project scripts\u002Frag-perf rag-perf -c scripts\u002Frag-perf\u002Fconfigs\u002Fsweep.yaml\n```\n\nOutput: nested `run_\u003Cts>\u002Fiter_1\u002F\u003CCR:_VDB-K:_RERANKER-K:_…>\u002F{profiling,aiperf_rag_on}\u002F` per point, plus aggregate `report.md` \u002F `results.json` \u002F `results.csv` at the run root.\n\n**Run unit tests:**\n\n```bash\nuv sync --project scripts\u002Frag-perf --extra dev   # one-time, installs pytest-asyncio\nuv run --project scripts\u002Frag-perf python -m pytest tests\u002Funit\u002Ftest_rag_perf\u002F\n```\n\n## Limitations\n\n- The CLI is **config-only**: author or copy YAML to vary a parameter.\n- `load.concurrency` \u002F `rag.vdb_top_k` \u002F `rag.reranker_top_k` accept `int | list[int]`; the validator requires unique list values because each value names a unique point dir.\n- `input.file` and `input.synthetic` follow an XOR rule — both set fails validation. When neither is set, `synthetic` auto-fills with defaults so a bare config still validates.\n- File-based input format is **inferred from extension only** (`.jsonl` or `.csv`); other extensions are rejected.\n- Synthetic generation streams each query to disk as it completes (failure-resilient) but **fails fast on the first LLM error** — partial JSONL is preserved. Re-run after fixing the endpoint.\n- Reasoning models (Nemotron Omni, Qwen-Reasoning) require `synthetic.disable_thinking: true` (the default). Without it the model exhausts the token budget on chain-of-thought and `content` returns empty — the generator now raises with a clear message instead of substituting `reasoning_content` for the answer.\n- aiperf-specific knobs outside the YAML surface (request rate distribution, GPU telemetry config, etc.) require editing `AiperfRunner._base_aiperf_cmd` in `scripts\u002Frag-perf\u002Frag_perf\u002Frunner.py`.\n- Procedural detail lives under **`references\u002F`** to keep this file concise.\n\n## Troubleshooting\n\n| Error \u002F signal | Likely cause | What to do |\n|---|---|---|\n| `Configuration errors in \u003Cyaml>:  •  input  —  ... XOR rule` | Both `input.file` and `input.synthetic` set | Pick one. The XOR validator runs at YAML load time. |\n| `input.file must end in .jsonl or .csv` | Extension other than `.jsonl` \u002F `.csv` | Rename or convert. |\n| `load.concurrency has duplicate values` | e.g. `[2, 2, 4]` | Each concurrency maps to a unique point dir; dedupe. |\n| `warmup_requests must be >= 1` | YAML had `warmup_requests: 0` | aiperf rejects warmup=0; minimum is 1. |\n| `LLM returned empty content (reasoning_content was populated — model exhausted its budget on chain-of-thought; raise min_query_tokens or set synthetic.disable_thinking=true).` | Reasoning model used CoT and ran out of tokens | Set `synthetic.disable_thinking: true` (the default) or raise `min_query_tokens`. |\n| `✗ All N profiling requests failed across M point(s).` + exit 1 | Bad URL, server down, wrong collection | Verify `target.url`, `rag.collection_names` (the `\u003Ccollection_name>` placeholder will hit this). |\n| Per-iteration `⚠ N profiling requests failed` warning, run continues | Some requests timed out \u002F errored mid-run | Check rag-server logs, raise `target.timeout_s`, drop concurrency. |\n| `RuntimeError: Random synthetic query generation failed at query N: ...` | LLM endpoint rejected a request mid-generation | Partial JSONL is at `synthetic.jsonl_output_path`; fix endpoint and re-run with reduced `num_queries`, or point `input.file` at the partial file. |\n| `Citation count (mean): 0` and `Citation relevance score: N\u002FA` for a non-empty deployment | Collection mismatch between `rag.collection_names` and what's actually ingested | Run `curl -s http:\u002F\u002F\u003Cingestor>:8082\u002Fv1\u002Fcollections` to list real collections. |\n| Tests error with `ModuleNotFoundError: No module named 'pytest_asyncio'` | Dev extras missing | `uv sync --project scripts\u002Frag-perf --extra dev`. |\n| CI: `ModuleNotFoundError: No module named 'ruamel'` from `tests\u002Funit\u002Ftest_rag_perf\u002F` | rag-perf package missing from CI venv | Add `uv pip install -e .\u002Fscripts\u002Frag-perf` after the top-level install in the unit-tests job. |\n\n## Gotchas\n\n- **Run from repo root.** Preset configs reference `scripts\u002Frag-perf\u002Fexamples\u002Fqueries.jsonl` and `scripts\u002Frag-perf\u002Fprompts\u002Fdefault_prompts.yaml` with repo-root-relative paths. Running from inside `scripts\u002Frag-perf\u002F` will fail those file lookups.\n- **CLI is config-only.** Edit the YAML or copy a preset for URL, concurrency, collection, and similar fields.\n- **Always edit `rag.collection_names` before the first run.** The presets ship with `[\"\u003Ccollection_name>\"]` as a deliberate placeholder. Validation passes, retrieval fails silently for every request — manifests as `Citation count (mean): 0` everywhere.\n- **`load.concurrency_list`, `rag.vdb_top_k_list`, `rag.reranker_top_k_list`** are read-only properties that normalise scalar-or-list to a list. Use them when reasoning about the grid; the underlying YAML field is whatever the user wrote.\n- **`aiperf.enabled: false` changes filenames.** The top-level outputs become `profile_report.md` \u002F `profile_results.json` \u002F `profile_results.csv`. The aggregate sweep table also suppresses load-test rows and the \"Optimal throughput\" footer.\n- **Resolved-config dump is verbose** (50+ lines) — expected. It's what makes terminal output a self-contained reproducer; don't filter it out in scripts.\n- **The aiperf shell command is logged before each subprocess.** Look for `\\n  $ python -m aiperf profile -m ... --endpoint-type nvidia_rag ...` in stdout — copy-paste runnable for reproducing a single point outside rag-perf.\n- **`--endpoint-type nvidia_rag`** comes from the bundled plugin at `scripts\u002Frag-perf\u002Frag_perf\u002Fplugin\u002Fnvidia_rag.py`. It teaches aiperf about the RAG `\u002Fv1\u002Fgenerate` request shape and parses citations + per-stage `metrics` out of the SSE stream. If aiperf can't resolve `nvidia_rag`, rag-perf needs editable installation in the venv — re-run `uv sync --project scripts\u002Frag-perf` (or `uv pip install -e .\u002Fscripts\u002Frag-perf`).\n- **Sweep-mode point-name collision.** When two points differ only in concurrency (e.g. `[1, 4]` × single `vdb_top_k`), the dir name encodes everything: `CR:1_ISL:50_OSL:512_VDB-K:20_RERANKER-K:4_Model:...`. Cluster \u002F GPU \u002F experiment_name (`output.cluster`, `output.gpu`, `output.experiment_name`) are appended too — useful for diff-friendly artifact paths across machines.\n- **`load.iterations > 1` repeats the entire grid**. Each repetition writes to its own `iter_\u003Ci>\u002F`. Aggregate CSV row count = `n_points × iterations`.\n\n## Source of truth\n\n| Piece | Location |\n|---|---|\n| Driver | [`scripts\u002Frag-perf\u002Frag_perf\u002Fcli.py`](..\u002F..\u002Fscripts\u002Frag-perf\u002Frag_perf\u002Fcli.py) (`main` is the single Click command) |\n| Schema | [`scripts\u002Frag-perf\u002Frag_perf\u002Fconfig.py`](..\u002F..\u002Fscripts\u002Frag-perf\u002Frag_perf\u002Fconfig.py) (`RunConfig` and sub-models) |\n| Orchestrator | [`scripts\u002Frag-perf\u002Frag_perf\u002Frunner.py`](..\u002F..\u002Fscripts\u002Frag-perf\u002Frag_perf\u002Frunner.py) (`BenchmarkRunner.run`, `RagProfiler`, `AiperfRunner`) |\n| aiperf plugin | [`scripts\u002Frag-perf\u002Frag_perf\u002Fplugin\u002Fnvidia_rag.py`](..\u002F..\u002Fscripts\u002Frag-perf\u002Frag_perf\u002Fplugin\u002Fnvidia_rag.py) |\n| User-facing doc | [`docs\u002Fperformance-benchmarking.md`](..\u002F..\u002Fdocs\u002Fperformance-benchmarking.md) |\n| Presets | [`scripts\u002Frag-perf\u002Fconfigs\u002F{quick_profile,single_run,sweep}.yaml`](..\u002F..\u002Fscripts\u002Frag-perf\u002Fconfigs\u002F) |\n| Sample queries | [`scripts\u002Frag-perf\u002Fexamples\u002Fqueries.jsonl`](..\u002F..\u002Fscripts\u002Frag-perf\u002Fexamples\u002Fqueries.jsonl) |\n| Synthetic prompts | [`scripts\u002Frag-perf\u002Fprompts\u002Fdefault_prompts.yaml`](..\u002F..\u002Fscripts\u002Frag-perf\u002Fprompts\u002Fdefault_prompts.yaml) |\n| Config schema details | [`references\u002Fconfig-schema.md`](references\u002Fconfig-schema.md) |\n| Synthetic-query generation | [`references\u002Fsynthetic-generation.md`](references\u002Fsynthetic-generation.md) |\n| Output layout & metric semantics | [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md) |\n\n## Agent playbook\n\n1. **Sync deps:** `uv sync --project scripts\u002Frag-perf` (one-time per checkout).\n2. **Pick & customise a preset:** copy `scripts\u002Frag-perf\u002Fconfigs\u002F\u003Cpreset>.yaml` if you want a variant; always set `rag.collection_names` to a real collection.\n3. **Run:** `uv run --project scripts\u002Frag-perf rag-perf -c \u003Cconfig>` from repo root.\n4. **Read the per-point + aggregate tables on stdout.** Bottleneck inference is in the per-point profiling section; comparison across points is the final aggregate table.\n5. **Parse artifacts** under `output.dir\u002Frun_\u003Cts>\u002F` — see [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md). For multi-point runs, `results.csv` has one row per (point × iteration).\n6. **Summarise for the user** using the playbook in [`references\u002Foutput-and-analysis.md#summarising-results-to-the-user`](references\u002Foutput-and-analysis.md#summarising-results-to-the-user) — headline table, scaling-efficiency math for sweeps, mandatory flags for zero citations \u002F non-zero errors \u002F suspect `llm_ttft_ms` \u002F low sample size, and a concrete next-experiment YAML.\n7. **Tune retrieval \u002F reranker:** flip to `quick_profile.yaml` or `aiperf.enabled: false` for fast iteration, then return to `single_run.yaml` \u002F `sweep.yaml` when characterising under load.\n8. **Triage failures:** see Troubleshooting above and [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md) for empty-citation \u002F bottleneck=N\u002FA patterns.\n","# RAG-Perf — config-driven perf benchmark CLI\n\n## Purpose\n\nDrive a deployed NVIDIA RAG Blueprint server with a YAML config, run a server-side **profiling pass** (per-stage timing, citation quality, bottleneck inference) and an optional **aiperf load test** (TTFT \u002F E2E \u002F token & request throughput \u002F error rate), and write a unified report. The CLI is intentionally minimal: `rag-perf -c \u003Cconfig>` plus `--help` \u002F `--version`. Behaviour is *fully* config-driven; field variations belong in YAML.\n\n## Scope\n\n- **Accuracy \u002F RAGAS** scoring of answer quality → use the **rag-eval** skill.\n- **Deploying, repairing, or configuring services** (compose, helm, NIM env vars) → use the **rag-blueprint** skill.\n- **Production monitoring \u002F alerting** — rag-perf is a one-shot benchmark tool.\n- **Runtime requirement:** a deployed RAG server reachable on the network.\n\n## Prerequisites\n\n- Repo cloned; **run commands from the repo root** (config paths in the presets are repo-root-relative).\n- Python **3.11+** and **uv** on PATH.\n- Install rag-perf into its own uv-managed venv: `uv sync --project scripts\u002Frag-perf`.\n- For unit tests: install dev extras as well — `uv sync --project scripts\u002Frag-perf --extra dev` (otherwise `pytest-asyncio` is missing and async tests error out at collection time).\n- A reachable RAG server (default `http:\u002F\u002Flocalhost:8081`). For the aiperf phase, the bundled `nvidia_rag` endpoint plugin must be installed — `pip install -e .\u002Fscripts\u002Frag-perf` registers it via the `aiperf.plugins` entry point.\n- For **synthetic** queries: an OpenAI-compatible chat-completions endpoint reachable at `synthetic.llm_url` (default `http:\u002F\u002Flocalhost:8999\u002Fv1\u002Fchat\u002Fcompletions`).\n- rag-perf itself runs without `NVIDIA_API_KEY` (unlike rag-eval). The synthetic LLM endpoint may require its own auth — that's the deployment's concern.\n\n## Instructions\n\n1. **Pick a preset.** The three under [`scripts\u002Frag-perf\u002Fconfigs\u002F`](..\u002F..\u002Fscripts\u002Frag-perf\u002Fconfigs) are:\n   - `quick_profile.yaml` — profile-only, ~30 s. Skips load test. For fast iteration on retrieval \u002F reranker tuning.\n   - `single_run.yaml` — one concurrency level, profiling + aiperf, ~2 min. Regression checks.\n   - `sweep.yaml` — multi-axis sweep. `load.concurrency`, `rag.vdb_top_k`, `rag.reranker_top_k` are all `int | list[int]`; any of them as a list becomes a sweep axis (Cartesian product).\n\n2. **Edit the preset.** **Required:** replace `rag.collection_names: [\"\u003Ccollection_name>\"]` with a real collection on the deployed ingestor server. Verify the collection exists via `GET \u002Fv1\u002Fcollections` on the ingestor. The placeholder `\u003Ccollection_name>` validates fine but every request will fail at retrieval. Use a copied YAML preset for variants; the CLI surface is intentionally config-only.\n\n3. **Run.** From repo root:\n   ```bash\n   uv run --project scripts\u002Frag-perf rag-perf -c scripts\u002Frag-perf\u002Fconfigs\u002Fsingle_run.yaml\n   ```\n   Same form for the other presets. The CLI accepts only `-c \u002F --config` (required), `--help`, `--version`.\n\n4. **Read stdout.** Every invocation prints, in order: a startup banner, a one-line summary, the **fully resolved config as YAML** (so the run is reproducible from terminal output), per-grid-point progress with the **shlex-joined aiperf command** in copy-pastable form, a **rich per-point summary table** (stage breakdown with bars, citation quality, bottleneck, load-test block), and finally a **side-by-side comparison table** auto-labelled by whichever axis varied. See [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md).\n\n5. **Inspect artifacts.** Layout depends on run shape — flat for single-point + `iterations=1`, nested under `iter_\u003Ci>\u002F\u003Cpoint>\u002F...` otherwise. See [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md) for the full directory tree, file purposes, and how to parse `results.json` \u002F `results.csv` \u002F `report.md`.\n\n6. **Summarise for the user.** When reporting back, follow the playbook in [`references\u002Foutput-and-analysis.md#summarising-results-to-the-user`](references\u002Foutput-and-analysis.md#summarising-results-to-the-user): pick the canonical result file for the run shape, build a headline table (concurrency × top-k axes × TTFT × throughput × bottleneck × citation quality), compute scaling efficiency on sweeps, **always flag** zero citations \u002F non-zero error rate \u002F suspect `llm_ttft_ms` \u002F small-sample p99, and propose a concrete next-experiment YAML.\n\n7. **Tune.** Schema is fully documented in [`docs\u002Fperformance-benchmarking.md`](..\u002F..\u002Fdocs\u002Fperformance-benchmarking.md) and the deeper-dive references below. Common knobs: turn `aiperf.enabled: false` for profile-only mode, increase `load.iterations` for variance estimation, set `load.sleep_between_points_s: 60` for overnight Cartesian sweeps.\n\n## Examples\n\n**Profile-only (quickest signal on retrieval \u002F reranker tuning):**\n\n```bash\nuv run --project scripts\u002Frag-perf rag-perf -c scripts\u002Frag-perf\u002Fconfigs\u002Fquick_profile.yaml\n```\n\nOutput: `rag-perf-results\u002Fquick_profile\u002Frun_\u003Cts>\u002F{profile_report.md, profile_results.json, profiling\u002F}`. The `aiperf_rag_on\u002F` directory is omitted. Filenames are `profile_*` because `aiperf.enabled: false`.\n\n**Single benchmark point with full report:**\n\n```bash\nuv run --project scripts\u002Frag-perf rag-perf -c scripts\u002Frag-perf\u002Fconfigs\u002Fsingle_run.yaml\n```\n\nOutput: flat `run_\u003Cts>\u002F{report.md, results.json, results.csv, profiling\u002F, aiperf_rag_on\u002F}`.\n\n**Concurrency sweep:**\n\n```bash\nuv run --project scripts\u002Frag-perf rag-perf -c scripts\u002Frag-perf\u002Fconfigs\u002Fsweep.yaml\n```\n\nOutput: nested `run_\u003Cts>\u002Fiter_1\u002F\u003CCR:_VDB-K:_RERANKER-K:_…>\u002F{profiling,aiperf_rag_on}\u002F` per point, plus aggregate `report.md` \u002F `results.json` \u002F `results.csv` at the run root.\n\n**Run unit tests:**\n\n```bash\nuv sync --project scripts\u002Frag-perf --extra dev   # one-time, installs pytest-asyncio\nuv run --project scripts\u002Frag-perf python -m pytest tests\u002Funit\u002Ftest_rag_perf\u002F\n```\n\n## Limitations\n\n- The CLI is **config-only**: author or copy YAML to vary a parameter.\n- `load.concurrency` \u002F `rag.vdb_top_k` \u002F `rag.reranker_top_k` accept `int | list[int]`; the validator requires unique list values because each value names a unique point dir.\n- `input.file` and `input.synthetic` follow an XOR rule — both set fails validation. When neither is set, `synthetic` auto-fills with defaults so a bare config still validates.\n- File-based input format is **inferred from extension only** (`.jsonl` or `.csv`); other extensions are rejected.\n- Synthetic generation streams each query to disk as it completes (failure-resilient) but **fails fast on the first LLM error** — partial JSONL is preserved. Re-run after fixing the endpoint.\n- Reasoning models (Nemotron Omni, Qwen-Reasoning) require `synthetic.disable_thinking: true` (the default). Without it the model exhausts the token budget on chain-of-thought and `content` returns empty — the generator now raises with a clear message instead of substituting `reasoning_content` for the answer.\n- aiperf-specific knobs outside the YAML surface (request rate distribution, GPU telemetry config, etc.) require editing `AiperfRunner._base_aiperf_cmd` in `scripts\u002Frag-perf\u002Frag_perf\u002Frunner.py`.\n- Procedural detail lives under **`references\u002F`** to keep this file concise.\n\n## Troubleshooting\n\n| Error \u002F signal | Likely cause | What to do |\n|---|---|---|\n| `Configuration errors in \u003Cyaml>:  •  input  —  ... XOR rule` | Both `input.file` and `input.synthetic` set | Pick one. The XOR validator runs at YAML load time. |\n| `input.file must end in .jsonl or .csv` | Extension other than `.jsonl` \u002F `.csv` | Rename or convert. |\n| `load.concurrency has duplicate values` | e.g. `[2, 2, 4]` | Each concurrency maps to a unique point dir; dedupe. |\n| `warmup_requests must be >= 1` | YAML had `warmup_requests: 0` | aiperf rejects warmup=0; minimum is 1. |\n| `LLM returned empty content (reasoning_content was populated — model exhausted its budget on chain-of-thought; raise min_query_tokens or set synthetic.disable_thinking=true).` | Reasoning model used CoT and ran out of tokens | Set `synthetic.disable_thinking: true` (the default) or raise `min_query_tokens`. |\n| `✗ All N profiling requests failed across M point(s).` + exit 1 | Bad URL, server down, wrong collection | Verify `target.url`, `rag.collection_names` (the `\u003Ccollection_name>` placeholder will hit this). |\n| Per-iteration `⚠ N profiling requests failed` warning, run continues | Some requests timed out \u002F errored mid-run | Check rag-server logs, raise `target.timeout_s`, drop concurrency. |\n| `RuntimeError: Random synthetic query generation failed at query N: ...` | LLM endpoint rejected a request mid-generation | Partial JSONL is at `synthetic.jsonl_output_path`; fix endpoint and re-run with reduced `num_queries`, or point `input.file` at the partial file. |\n| `Citation count (mean): 0` and `Citation relevance score: N\u002FA` for a non-empty deployment | Collection mismatch between `rag.collection_names` and what's actually ingested | Run `curl -s http:\u002F\u002F\u003Cingestor>:8082\u002Fv1\u002Fcollections` to list real collections. |\n| Tests error with `ModuleNotFoundError: No module named 'pytest_asyncio'` | Dev extras missing | `uv sync --project scripts\u002Frag-perf --extra dev`. |\n| CI: `ModuleNotFoundError: No module named 'ruamel'` from `tests\u002Funit\u002Ftest_rag_perf\u002F` | rag-perf package missing from CI venv | Add `uv pip install -e .\u002Fscripts\u002Frag-perf` after the top-level install in the unit-tests job. |\n\n## Gotchas\n\n- **Run from repo root.** Preset configs reference `scripts\u002Frag-perf\u002Fexamples\u002Fqueries.jsonl` and `scripts\u002Frag-perf\u002Fprompts\u002Fdefault_prompts.yaml` with repo-root-relative paths. Running from inside `scripts\u002Frag-perf\u002F` will fail those file lookups.\n- **CLI is config-only.** Edit the YAML or copy a preset for URL, concurrency, collection, and similar fields.\n- **Always edit `rag.collection_names` before the first run.** The presets ship with `[\"\u003Ccollection_name>\"]` as a deliberate placeholder. Validation passes, retrieval fails silently for every request — manifests as `Citation count (mean): 0` everywhere.\n- **`load.concurrency_list`, `rag.vdb_top_k_list`, `rag.reranker_top_k_list`** are read-only properties that normalise scalar-or-list to a list. Use them when reasoning about the grid; the underlying YAML field is whatever the user wrote.\n- **`aiperf.enabled: false` changes filenames.** The top-level outputs become `profile_report.md` \u002F `profile_results.json` \u002F `profile_results.csv`. The aggregate sweep table also suppresses load-test rows and the \"Optimal throughput\" footer.\n- **Resolved-config dump is verbose** (50+ lines) — expected. It's what makes terminal output a self-contained reproducer; don't filter it out in scripts.\n- **The aiperf shell command is logged before each subprocess.** Look for `\\n  $ python -m aiperf profile -m ... --endpoint-type nvidia_rag ...` in stdout — copy-paste runnable for reproducing a single point outside rag-perf.\n- **`--endpoint-type nvidia_rag`** comes from the bundled plugin at `scripts\u002Frag-perf\u002Frag_perf\u002Fplugin\u002Fnvidia_rag.py`. It teaches aiperf about the RAG `\u002Fv1\u002Fgenerate` request shape and parses citations + per-stage `metrics` out of the SSE stream. If aiperf can't resolve `nvidia_rag`, rag-perf needs editable installation in the venv — re-run `uv sync --project scripts\u002Frag-perf` (or `uv pip install -e .\u002Fscripts\u002Frag-perf`).\n- **Sweep-mode point-name collision.** When two points differ only in concurrency (e.g. `[1, 4]` × single `vdb_top_k`), the dir name encodes everything: `CR:1_ISL:50_OSL:512_VDB-K:20_RERANKER-K:4_Model:...`. Cluster \u002F GPU \u002F experiment_name (`output.cluster`, `output.gpu`, `output.experiment_name`) are appended too — useful for diff-friendly artifact paths across machines.\n- **`load.iterations > 1` repeats the entire grid**. Each repetition writes to its own `iter_\u003Ci>\u002F`. Aggregate CSV row count = `n_points × iterations`.\n\n## Source of truth\n\n| Piece | Location |\n|---|---|\n| Driver | [`scripts\u002Frag-perf\u002Frag_perf\u002Fcli.py`](..\u002F..\u002Fscripts\u002Frag-perf\u002Frag_perf\u002Fcli.py) (`main` is the single Click command) |\n| Schema | [`scripts\u002Frag-perf\u002Frag_perf\u002Fconfig.py`](..\u002F..\u002Fscripts\u002Frag-perf\u002Frag_perf\u002Fconfig.py) (`RunConfig` and sub-models) |\n| Orchestrator | [`scripts\u002Frag-perf\u002Frag_perf\u002Frunner.py`](..\u002F..\u002Fscripts\u002Frag-perf\u002Frag_perf\u002Frunner.py) (`BenchmarkRunner.run`, `RagProfiler`, `AiperfRunner`) |\n| aiperf plugin | [`scripts\u002Frag-perf\u002Frag_perf\u002Fplugin\u002Fnvidia_rag.py`](..\u002F..\u002Fscripts\u002Frag-perf\u002Frag_perf\u002Fplugin\u002Fnvidia_rag.py) |\n| User-facing doc | [`docs\u002Fperformance-benchmarking.md`](..\u002F..\u002Fdocs\u002Fperformance-benchmarking.md) |\n| Presets | [`scripts\u002Frag-perf\u002Fconfigs\u002F{quick_profile,single_run,sweep}.yaml`](..\u002F..\u002Fscripts\u002Frag-perf\u002Fconfigs\u002F) |\n| Sample queries | [`scripts\u002Frag-perf\u002Fexamples\u002Fqueries.jsonl`](..\u002F..\u002Fscripts\u002Frag-perf\u002Fexamples\u002Fqueries.jsonl) |\n| Synthetic prompts | [`scripts\u002Frag-perf\u002Fprompts\u002Fdefault_prompts.yaml`](..\u002F..\u002Fscripts\u002Frag-perf\u002Fprompts\u002Fdefault_prompts.yaml) |\n| Config schema details | [`references\u002Fconfig-schema.md`](references\u002Fconfig-schema.md) |\n| Synthetic-query generation | [`references\u002Fsynthetic-generation.md`](references\u002Fsynthetic-generation.md) |\n| Output layout & metric semantics | [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md) |\n\n## Agent playbook\n\n1. **Sync deps:** `uv sync --project scripts\u002Frag-perf` (one-time per checkout).\n2. **Pick & customise a preset:** copy `scripts\u002Frag-perf\u002Fconfigs\u002F\u003Cpreset>.yaml` if you want a variant; always set `rag.collection_names` to a real collection.\n3. **Run:** `uv run --project scripts\u002Frag-perf rag-perf -c \u003Cconfig>` from repo root.\n4. **Read the per-point + aggregate tables on stdout.** Bottleneck inference is in the per-point profiling section; comparison across points is the final aggregate table.\n5. **Parse artifacts** under `output.dir\u002Frun_\u003Cts>\u002F` — see [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md). For multi-point runs, `results.csv` has one row per (point × iteration).\n6. **Summarise for the user** using the playbook in [`references\u002Foutput-and-analysis.md#summarising-results-to-the-user`](references\u002Foutput-and-analysis.md#summarising-results-to-the-user) — headline table, scaling-efficiency math for sweeps, mandatory flags for zero citations \u002F non-zero errors \u002F suspect `llm_ttft_ms` \u002F low sample size, and a concrete next-experiment YAML.\n7. **Tune retrieval \u002F reranker:** flip to `quick_profile.yaml` or `aiperf.enabled: false` for fast iteration, then return to `single_run.yaml` \u002F `sweep.yaml` when characterising under load.\n8. **Triage failures:** see Troubleshooting above and [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md) for empty-citation \u002F bottleneck=N\u002FA patterns.\n",{"name":5,"license":15,"version":24,"metadata":25,"description":7,"allowed-tools":46,"compatibility":47},"2.6.0",{"tags":26,"author":34,"domain":35,"languages":36,"frameworks":39,"github-url":41,"tool-version":42,"argument-hint":43,"endpoint-openapi-schemas":44},[27,28,29,30,31,32,33],"nvidia","blueprint","rag","performance","benchmarking","aiperf","nvidia-rag-blueprint","NVIDIA RAG \u003Cfoundational-rag-dev@exchange.nvidia.com>","ai-ml",[37,38],"python","shell",[32,40],"fastapi","https:\u002F\u002Fgithub.com\u002FNVIDIA-AI-Blueprints\u002Frag","0.1.0","rag-perf | aiperf | TTFT | latency | throughput | concurrency sweep | bottleneck | retrieval \u002F reranker tuning | profile-only | synthetic queries | quick_profile.yaml | single_run.yaml | sweep.yaml | uv run --project scripts\u002Frag-perf",[45],"docs\u002Fapi_reference\u002Fopenapi_schema_rag_server.json","Read Grep Glob Bash(ls *) Bash(python3 *) Bash(uv *) Bash(cat *) Bash(curl *) Write Edit","Repository checkout with uv; Python 3.11+; run from repo root; uv sync --project scripts\u002Frag-perf (perf deps live in scripts\u002Frag-perf\u002Fpyproject.toml); reachable RAG server (default http:\u002F\u002Flocalhost:8081); for synthetic queries an OpenAI-compatible chat-completions endpoint is required (default http:\u002F\u002Flocalhost:8999\u002Fv1\u002Fchat\u002Fcompletions); aiperf load-test phase uses the bundled nvidia_rag endpoint plugin, registered automatically when rag-perf is installed editable.","main","c475fc64e94e803b50d14a791dfee8853fbb2c5c","skills\u002Fnvidia\u002Frag-perf","https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory","https:\u002F\u002Fgithub.com\u002FElogic360","2026-11-08 22:21:22",{"source":8,"skillName":5,"commands":55,"prompt":65,"zipAvailable":66},[56,59,62],{"tool":57,"command":58},"npx","npx skills add Elogic360\u002Fsoftware-factory --skill rag-perf",{"tool":60,"command":61},"bunx","bunx skills add Elogic360\u002Fsoftware-factory --skill rag-perf",{"tool":63,"command":64},"pnpm","pnpm dlx skills add Elogic360\u002Fsoftware-factory --skill rag-perf","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Frag-perf\u002FSKILL.md",true,[68,74,78,82,86,90,94,98],{"path":69,"resourceType":70,"resourceTypeDesc":71,"sizeBytes":72,"sourceHtmlUrl":73},"BENCHMARK.md","REFERENCE","参考文档",2672,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Frag-perf\u002FBENCHMARK.md",{"path":75,"resourceType":70,"resourceTypeDesc":71,"sizeBytes":76,"sourceHtmlUrl":77},"eval\u002Fh100.json",2440,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Frag-perf\u002Feval\u002Fh100.json",{"path":79,"resourceType":70,"resourceTypeDesc":71,"sizeBytes":80,"sourceHtmlUrl":81},"eval\u002Fnvidia_hosted.json",1887,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Frag-perf\u002Feval\u002Fnvidia_hosted.json",{"path":83,"resourceType":70,"resourceTypeDesc":71,"sizeBytes":84,"sourceHtmlUrl":85},"references\u002Fconfig-schema.md",7762,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Frag-perf\u002Freferences\u002Fconfig-schema.md",{"path":87,"resourceType":70,"resourceTypeDesc":71,"sizeBytes":88,"sourceHtmlUrl":89},"references\u002Foutput-and-analysis.md",13098,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Frag-perf\u002Freferences\u002Foutput-and-analysis.md",{"path":91,"resourceType":70,"resourceTypeDesc":71,"sizeBytes":92,"sourceHtmlUrl":93},"references\u002Fsynthetic-generation.md",7341,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Frag-perf\u002Freferences\u002Fsynthetic-generation.md",{"path":95,"resourceType":70,"resourceTypeDesc":71,"sizeBytes":96,"sourceHtmlUrl":97},"skill-card.md",2750,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Frag-perf\u002Fskill-card.md",{"path":99,"resourceType":100,"resourceTypeDesc":101,"sizeBytes":102,"sourceHtmlUrl":103},"skill.oms.sig","OTHER","其他",5549,"https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Frag-perf\u002Fskill.oms.sig",[105,112,119,126,133,140],{"skillNo":106,"name":107,"nameZh":6,"description":108,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":109,"categories":110,"keywords":111,"publishedAt":19},"01M79G206MT3J6S0GY3C45P2KV","vss-summarize-video","Use to summarize a recorded video via the LVS summarization microservice (HITL-gated) with a VLM fallback. Not for report generation or live RTSP captioning.","https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fvss-summarize-video\u002FSKILL.md",[],[],{"skillNo":113,"name":114,"nameZh":6,"description":115,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":116,"categories":117,"keywords":118,"publishedAt":19},"01M79G1WN63PGRE8CKD47WEBQ5","vss-setup-video-analytics-api","Use to deploy the vss-video-analytics-api REST service standalone (config-source, data-log bind, Elasticsearch, optional Kafka). Not for full warehouse deploy.","https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fvss-setup-video-analytics-api\u002FSKILL.md",[],[],{"skillNo":120,"name":121,"nameZh":6,"description":122,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":123,"categories":124,"keywords":125,"publishedAt":19},"01M79G1S9F7VY3EF6H53PRP5AN","vss-setup-behavior-analytics","Use to deploy the vss-behavior-analytics service standalone (entrypoint, config-source, optional calibration). Not for the full warehouse deploy.","https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fvss-setup-behavior-analytics\u002FSKILL.md",[],[],{"skillNo":127,"name":128,"nameZh":6,"description":129,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":130,"categories":131,"keywords":132,"publishedAt":19},"01M79G1N1S1Y27PFZPYMNK1MTQ","vss-search-archive","Use this skill to run top-level VSS fusion search on archived video, or to ingest video files \u002F RTSP streams for search. Do NOT use for ad-hoc visual Q&A (use vss-ask-video), live captioning (use vss-deploy-dense-captioning), or video summarization and reports (use vss-summarize-video).","https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fvss-search-archive\u002FSKILL.md",[],[],{"skillNo":134,"name":135,"nameZh":6,"description":136,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":137,"categories":138,"keywords":139,"publishedAt":19},"01M79G1H6WN0Q6C0VCAXCZX6B7","vss-query-analytics","Use this skill when reading video-analytics metrics, incidents, alerts, and sensor data via the VA-MCP server (port 9901). Not for live VLM or incident-range narrative reports.","https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fvss-query-analytics\u002FSKILL.md",[],[],{"skillNo":141,"name":142,"nameZh":6,"description":143,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":144,"categories":145,"keywords":146,"publishedAt":19},"01M79G1DYPNXQS7QHZ0AAF5FX5","vss-manage-video-io-storage","Use to call the VIOS REST API (sensor list, timelines, clip extraction, snapshots, add\u002Fdelete sensors and streams). Not for VLM inference or search.","https:\u002F\u002Fgithub.com\u002FElogic360\u002Fsoftware-factory\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fvss-manage-video-io-storage\u002FSKILL.md",[],[]]