[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6PJ8XE9CS5VTYT3G6GT34P4":3,"skill-same:01M6PJ8XE9CS5VTYT3G6GT34P4":130},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":26,"publishedAt":47,"linkOnly":48,"markdownContent":49,"markdownBody":50,"frontMatter":51,"refName":73,"commitSha":74,"skillDirectory":75,"forksCount":14,"repositoryHtmlUrl":76,"repositoryHomepageUrl":6,"ownerHtmlUrl":77,"ownerBlogUrl":78,"ownerDescription":6,"updatedAt":79,"install":80,"resources":93},"01M6PJ8XE9CS5VTYT3G6GT34P4","rag-perf",null,"Performance benchmarking for a deployed NVIDIA RAG Blueprint server: profiling pass + aiperf load test driven by a single YAML config. Not for accuracy \u002F RAGAS scoring (use rag-eval) or for deploying \u002F repairing services (use rag-blueprint).","headmktlucca-ui\u002Fneuroads-lp","headmktlucca-ui","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F247379838?v=4","01M6PDVAV6M1W96D1SR1PDF8CS","NeuroAds","01M6PDVAZ4P281GNDKD45CYTF5",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fheadmktlucca-ui\u002Fneuroads-lp\u002Fblob\u002Fmain\u002F.agent\u002Fskills\u002Frag-perf\u002FSKILL.md",[18,23],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000008","测试与质量","SCENARIO","使用场景",{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000010","后端与接口",[27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46],"rag","rag perf","perf","Blueprint","performance","ls","quick","ragas","checkout","project","scoring","default","uv","only","bottleneck","fastapi","eval","deps","Bash","api","2026-11-10 10:12:30",false,"---\nname: rag-perf\nversion: \"2.6.0\"\ndescription: >-\n  Performance benchmarking for a deployed NVIDIA RAG Blueprint server: profiling pass + aiperf\n  load test driven by a single YAML config. Not for accuracy \u002F RAGAS scoring (use rag-eval) or\n  for deploying \u002F repairing services (use rag-blueprint).\nlicense: Apache-2.0\ncompatibility: Repository checkout with uv; Python 3.11+; run from repo root; uv sync --project scripts\u002Frag-perf (perf deps live in scripts\u002Frag-perf\u002Fpyproject.toml); reachable RAG server (default http:\u002F\u002Flocalhost:8081); for synthetic queries an OpenAI-compatible chat-completions endpoint is required (default http:\u002F\u002Flocalhost:8999\u002Fv1\u002Fchat\u002Fcompletions); aiperf load-test phase uses the bundled nvidia_rag endpoint plugin, registered automatically when rag-perf is installed editable.\nmetadata:\n  tool-version: \"0.1.0\"\n  author: NVIDIA RAG \u003Cfoundational-rag-dev@exchange.nvidia.com>\n  github-url: \"https:\u002F\u002Fgithub.com\u002FNVIDIA-AI-Blueprints\u002Frag\"\n  endpoint-openapi-schemas:\n    - docs\u002Fapi_reference\u002Fopenapi_schema_rag_server.json\n  argument-hint: rag-perf | aiperf | TTFT | latency | throughput | concurrency sweep | bottleneck | retrieval \u002F reranker tuning | profile-only | synthetic queries | quick_profile.yaml | single_run.yaml | sweep.yaml | uv run --project scripts\u002Frag-perf\n  tags:\n    - nvidia\n    - blueprint\n    - rag\n    - performance\n    - benchmarking\n    - aiperf\n    - nvidia-rag-blueprint\n  languages:\n    - python\n    - shell\n  frameworks:\n    - aiperf\n    - fastapi\n  domain: ai-ml\nallowed-tools: Read Grep Glob Bash(ls *) Bash(python3 *) Bash(uv *) Bash(cat *) Bash(curl *) Write Edit\n---\n\n# RAG-Perf — config-driven perf benchmark CLI\n\n## Purpose\n\nDrive a deployed NVIDIA RAG Blueprint server with a YAML config, run a server-side **profiling pass** (per-stage timing, citation quality, bottleneck inference) and an optional **aiperf load test** (TTFT \u002F E2E \u002F token & request throughput \u002F error rate), and write a unified report. The CLI is intentionally minimal: `rag-perf -c \u003Cconfig>` plus `--help` \u002F `--version`. Behaviour is *fully* config-driven; field variations belong in YAML.\n\n## Scope\n\n- **Accuracy \u002F RAGAS** scoring of answer quality → use the **rag-eval** skill.\n- **Deploying, repairing, or configuring services** (compose, helm, NIM env vars) → use the **rag-blueprint** skill.\n- **Production monitoring \u002F alerting** — rag-perf is a one-shot benchmark tool.\n- **Runtime requirement:** a deployed RAG server reachable on the network.\n\n## Prerequisites\n\n- Repo cloned; **run commands from the repo root** (config paths in the presets are repo-root-relative).\n- Python **3.11+** and **uv** on PATH.\n- Install rag-perf into its own uv-managed venv: `uv sync --project scripts\u002Frag-perf`.\n- For unit tests: install dev extras as well — `uv sync --project scripts\u002Frag-perf --extra dev` (otherwise `pytest-asyncio` is missing and async tests error out at collection time).\n- A reachable RAG server (default `http:\u002F\u002Flocalhost:8081`). For the aiperf phase, the bundled `nvidia_rag` endpoint plugin must be installed — `pip install -e .\u002Fscripts\u002Frag-perf` registers it via the `aiperf.plugins` entry point.\n- For **synthetic** queries: an OpenAI-compatible chat-completions endpoint reachable at `synthetic.llm_url` (default `http:\u002F\u002Flocalhost:8999\u002Fv1\u002Fchat\u002Fcompletions`).\n- rag-perf itself runs without `NVIDIA_API_KEY` (unlike rag-eval). The synthetic LLM endpoint may require its own auth — that's the deployment's concern.\n\n## Instructions\n\n1. **Pick a preset.** The three under [`scripts\u002Frag-perf\u002Fconfigs\u002F`](..\u002F..\u002Fscripts\u002Frag-perf\u002Fconfigs) are:\n   - `quick_profile.yaml` — profile-only, ~30 s. Skips load test. For fast iteration on retrieval \u002F reranker tuning.\n   - `single_run.yaml` — one concurrency level, profiling + aiperf, ~2 min. Regression checks.\n   - `sweep.yaml` — multi-axis sweep. `load.concurrency`, `rag.vdb_top_k`, `rag.reranker_top_k` are all `int | list[int]`; any of them as a list becomes a sweep axis (Cartesian product).\n\n2. **Edit the preset.** **Required:** replace `rag.collection_names: [\"\u003Ccollection_name>\"]` with a real collection on the deployed ingestor server. Verify the collection exists via `GET \u002Fv1\u002Fcollections` on the ingestor. The placeholder `\u003Ccollection_name>` validates fine but every request will fail at retrieval. Use a copied YAML preset for variants; the CLI surface is intentionally config-only.\n\n3. **Run.** From repo root:\n   ```bash\n   uv run --project scripts\u002Frag-perf rag-perf -c scripts\u002Frag-perf\u002Fconfigs\u002Fsingle_run.yaml\n   ```\n   Same form for the other presets. The CLI accepts only `-c \u002F --config` (required), `--help`, `--version`.\n\n4. **Read stdout.** Every invocation prints, in order: a startup banner, a one-line summary, the **fully resolved config as YAML** (so the run is reproducible from terminal output), per-grid-point progress with the **shlex-joined aiperf command** in copy-pastable form, a **rich per-point summary table** (stage breakdown with bars, citation quality, bottleneck, load-test block), and finally a **side-by-side comparison table** auto-labelled by whichever axis varied. See [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md).\n\n5. **Inspect artifacts.** Layout depends on run shape — flat for single-point + `iterations=1`, nested under `iter_\u003Ci>\u002F\u003Cpoint>\u002F...` otherwise. See [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md) for the full directory tree, file purposes, and how to parse `results.json` \u002F `results.csv` \u002F `report.md`.\n\n6. **Summarise for the user.** When reporting back, follow the playbook in [`references\u002Foutput-and-analysis.md#summarising-results-to-the-user`](references\u002Foutput-and-analysis.md#summarising-results-to-the-user): pick the canonical result file for the run shape, build a headline table (concurrency × top-k axes × TTFT × throughput × bottleneck × citation quality), compute scaling efficiency on sweeps, **always flag** zero citations \u002F non-zero error rate \u002F suspect `llm_ttft_ms` \u002F small-sample p99, and propose a concrete next-experiment YAML.\n\n7. **Tune.** Schema is fully documented in [`docs\u002Fperformance-benchmarking.md`](..\u002F..\u002Fdocs\u002Fperformance-benchmarking.md) and the deeper-dive references below. Common knobs: turn `aiperf.enabled: false` for profile-only mode, increase `load.iterations` for variance estimation, set `load.sleep_between_points_s: 60` for overnight Cartesian sweeps.\n\n## Examples\n\n**Profile-only (quickest signal on retrieval \u002F reranker tuning):**\n\n```bash\nuv run --project scripts\u002Frag-perf rag-perf -c scripts\u002Frag-perf\u002Fconfigs\u002Fquick_profile.yaml\n```\n\nOutput: `rag-perf-results\u002Fquick_profile\u002Frun_\u003Cts>\u002F{profile_report.md, profile_results.json, profiling\u002F}`. The `aiperf_rag_on\u002F` directory is omitted. Filenames are `profile_*` because `aiperf.enabled: false`.\n\n**Single benchmark point with full report:**\n\n```bash\nuv run --project scripts\u002Frag-perf rag-perf -c scripts\u002Frag-perf\u002Fconfigs\u002Fsingle_run.yaml\n```\n\nOutput: flat `run_\u003Cts>\u002F{report.md, results.json, results.csv, profiling\u002F, aiperf_rag_on\u002F}`.\n\n**Concurrency sweep:**\n\n```bash\nuv run --project scripts\u002Frag-perf rag-perf -c scripts\u002Frag-perf\u002Fconfigs\u002Fsweep.yaml\n```\n\nOutput: nested `run_\u003Cts>\u002Fiter_1\u002F\u003CCR:_VDB-K:_RERANKER-K:_…>\u002F{profiling,aiperf_rag_on}\u002F` per point, plus aggregate `report.md` \u002F `results.json` \u002F `results.csv` at the run root.\n\n**Run unit tests:**\n\n```bash\nuv sync --project scripts\u002Frag-perf --extra dev   # one-time, installs pytest-asyncio\nuv run --project scripts\u002Frag-perf python -m pytest tests\u002Funit\u002Ftest_rag_perf\u002F\n```\n\n## Limitations\n\n- The CLI is **config-only**: author or copy YAML to vary a parameter.\n- `load.concurrency` \u002F `rag.vdb_top_k` \u002F `rag.reranker_top_k` accept `int | list[int]`; the validator requires unique list values because each value names a unique point dir.\n- `input.file` and `input.synthetic` follow an XOR rule — both set fails validation. When neither is set, `synthetic` auto-fills with defaults so a bare config still validates.\n- File-based input format is **inferred from extension only** (`.jsonl` or `.csv`); other extensions are rejected.\n- Synthetic generation streams each query to disk as it completes (failure-resilient) but **fails fast on the first LLM error** — partial JSONL is preserved. Re-run after fixing the endpoint.\n- Reasoning models (Nemotron Omni, Qwen-Reasoning) require `synthetic.disable_thinking: true` (the default). Without it the model exhausts the token budget on chain-of-thought and `content` returns empty — the generator now raises with a clear message instead of substituting `reasoning_content` for the answer.\n- aiperf-specific knobs outside the YAML surface (request rate distribution, GPU telemetry config, etc.) require editing `AiperfRunner._base_aiperf_cmd` in `scripts\u002Frag-perf\u002Frag_perf\u002Frunner.py`.\n- Procedural detail lives under **`references\u002F`** to keep this file concise.\n\n## Troubleshooting\n\n| Error \u002F signal | Likely cause | What to do |\n|---|---|---|\n| `Configuration errors in \u003Cyaml>:  •  input  —  ... XOR rule` | Both `input.file` and `input.synthetic` set | Pick one. The XOR validator runs at YAML load time. |\n| `input.file must end in .jsonl or .csv` | Extension other than `.jsonl` \u002F `.csv` | Rename or convert. |\n| `load.concurrency has duplicate values` | e.g. `[2, 2, 4]` | Each concurrency maps to a unique point dir; dedupe. |\n| `warmup_requests must be >= 1` | YAML had `warmup_requests: 0` | aiperf rejects warmup=0; minimum is 1. |\n| `LLM returned empty content (reasoning_content was populated — model exhausted its budget on chain-of-thought; raise min_query_tokens or set synthetic.disable_thinking=true).` | Reasoning model used CoT and ran out of tokens | Set `synthetic.disable_thinking: true` (the default) or raise `min_query_tokens`. |\n| `✗ All N profiling requests failed across M point(s).` + exit 1 | Bad URL, server down, wrong collection | Verify `target.url`, `rag.collection_names` (the `\u003Ccollection_name>` placeholder will hit this). |\n| Per-iteration `⚠ N profiling requests failed` warning, run continues | Some requests timed out \u002F errored mid-run | Check rag-server logs, raise `target.timeout_s`, drop concurrency. |\n| `RuntimeError: Random synthetic query generation failed at query N: ...` | LLM endpoint rejected a request mid-generation | Partial JSONL is at `synthetic.jsonl_output_path`; fix endpoint and re-run with reduced `num_queries`, or point `input.file` at the partial file. |\n| `Citation count (mean): 0` and `Citation relevance score: N\u002FA` for a non-empty deployment | Collection mismatch between `rag.collection_names` and what's actually ingested | Run `curl -s http:\u002F\u002F\u003Cingestor>:8082\u002Fv1\u002Fcollections` to list real collections. |\n| Tests error with `ModuleNotFoundError: No module named 'pytest_asyncio'` | Dev extras missing | `uv sync --project scripts\u002Frag-perf --extra dev`. |\n| CI: `ModuleNotFoundError: No module named 'ruamel'` from `tests\u002Funit\u002Ftest_rag_perf\u002F` | rag-perf package missing from CI venv | Add `uv pip install -e .\u002Fscripts\u002Frag-perf` after the top-level install in the unit-tests job. |\n\n## Gotchas\n\n- **Run from repo root.** Preset configs reference `scripts\u002Frag-perf\u002Fexamples\u002Fqueries.jsonl` and `scripts\u002Frag-perf\u002Fprompts\u002Fdefault_prompts.yaml` with repo-root-relative paths. Running from inside `scripts\u002Frag-perf\u002F` will fail those file lookups.\n- **CLI is config-only.** Edit the YAML or copy a preset for URL, concurrency, collection, and similar fields.\n- **Always edit `rag.collection_names` before the first run.** The presets ship with `[\"\u003Ccollection_name>\"]` as a deliberate placeholder. Validation passes, retrieval fails silently for every request — manifests as `Citation count (mean): 0` everywhere.\n- **`load.concurrency_list`, `rag.vdb_top_k_list`, `rag.reranker_top_k_list`** are read-only properties that normalise scalar-or-list to a list. Use them when reasoning about the grid; the underlying YAML field is whatever the user wrote.\n- **`aiperf.enabled: false` changes filenames.** The top-level outputs become `profile_report.md` \u002F `profile_results.json` \u002F `profile_results.csv`. The aggregate sweep table also suppresses load-test rows and the \"Optimal throughput\" footer.\n- **Resolved-config dump is verbose** (50+ lines) — expected. It's what makes terminal output a self-contained reproducer; don't filter it out in scripts.\n- **The aiperf shell command is logged before each subprocess.** Look for `\\n  $ python -m aiperf profile -m ... --endpoint-type nvidia_rag ...` in stdout — copy-paste runnable for reproducing a single point outside rag-perf.\n- **`--endpoint-type nvidia_rag`** comes from the bundled plugin at `scripts\u002Frag-perf\u002Frag_perf\u002Fplugin\u002Fnvidia_rag.py`. It teaches aiperf about the RAG `\u002Fv1\u002Fgenerate` request shape and parses citations + per-stage `metrics` out of the SSE stream. If aiperf can't resolve `nvidia_rag`, rag-perf needs editable installation in the venv — re-run `uv sync --project scripts\u002Frag-perf` (or `uv pip install -e .\u002Fscripts\u002Frag-perf`).\n- **Sweep-mode point-name collision.** When two points differ only in concurrency (e.g. `[1, 4]` × single `vdb_top_k`), the dir name encodes everything: `CR:1_ISL:50_OSL:512_VDB-K:20_RERANKER-K:4_Model:...`. Cluster \u002F GPU \u002F experiment_name (`output.cluster`, `output.gpu`, `output.experiment_name`) are appended too — useful for diff-friendly artifact paths across machines.\n- **`load.iterations > 1` repeats the entire grid**. Each repetition writes to its own `iter_\u003Ci>\u002F`. Aggregate CSV row count = `n_points × iterations`.\n\n## Source of truth\n\n| Piece | Location |\n|---|---|\n| Driver | [`scripts\u002Frag-perf\u002Frag_perf\u002Fcli.py`](..\u002F..\u002Fscripts\u002Frag-perf\u002Frag_perf\u002Fcli.py) (`main` is the single Click command) |\n| Schema | [`scripts\u002Frag-perf\u002Frag_perf\u002Fconfig.py`](..\u002F..\u002Fscripts\u002Frag-perf\u002Frag_perf\u002Fconfig.py) (`RunConfig` and sub-models) |\n| Orchestrator | [`scripts\u002Frag-perf\u002Frag_perf\u002Frunner.py`](..\u002F..\u002Fscripts\u002Frag-perf\u002Frag_perf\u002Frunner.py) (`BenchmarkRunner.run`, `RagProfiler`, `AiperfRunner`) |\n| aiperf plugin | [`scripts\u002Frag-perf\u002Frag_perf\u002Fplugin\u002Fnvidia_rag.py`](..\u002F..\u002Fscripts\u002Frag-perf\u002Frag_perf\u002Fplugin\u002Fnvidia_rag.py) |\n| User-facing doc | [`docs\u002Fperformance-benchmarking.md`](..\u002F..\u002Fdocs\u002Fperformance-benchmarking.md) |\n| Presets | [`scripts\u002Frag-perf\u002Fconfigs\u002F{quick_profile,single_run,sweep}.yaml`](..\u002F..\u002Fscripts\u002Frag-perf\u002Fconfigs\u002F) |\n| Sample queries | [`scripts\u002Frag-perf\u002Fexamples\u002Fqueries.jsonl`](..\u002F..\u002Fscripts\u002Frag-perf\u002Fexamples\u002Fqueries.jsonl) |\n| Synthetic prompts | [`scripts\u002Frag-perf\u002Fprompts\u002Fdefault_prompts.yaml`](..\u002F..\u002Fscripts\u002Frag-perf\u002Fprompts\u002Fdefault_prompts.yaml) |\n| Config schema details | [`references\u002Fconfig-schema.md`](references\u002Fconfig-schema.md) |\n| Synthetic-query generation | [`references\u002Fsynthetic-generation.md`](references\u002Fsynthetic-generation.md) |\n| Output layout & metric semantics | [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md) |\n\n## Agent playbook\n\n1. **Sync deps:** `uv sync --project scripts\u002Frag-perf` (one-time per checkout).\n2. **Pick & customise a preset:** copy `scripts\u002Frag-perf\u002Fconfigs\u002F\u003Cpreset>.yaml` if you want a variant; always set `rag.collection_names` to a real collection.\n3. **Run:** `uv run --project scripts\u002Frag-perf rag-perf -c \u003Cconfig>` from repo root.\n4. **Read the per-point + aggregate tables on stdout.** Bottleneck inference is in the per-point profiling section; comparison across points is the final aggregate table.\n5. **Parse artifacts** under `output.dir\u002Frun_\u003Cts>\u002F` — see [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md). For multi-point runs, `results.csv` has one row per (point × iteration).\n6. **Summarise for the user** using the playbook in [`references\u002Foutput-and-analysis.md#summarising-results-to-the-user`](references\u002Foutput-and-analysis.md#summarising-results-to-the-user) — headline table, scaling-efficiency math for sweeps, mandatory flags for zero citations \u002F non-zero errors \u002F suspect `llm_ttft_ms` \u002F low sample size, and a concrete next-experiment YAML.\n7. **Tune retrieval \u002F reranker:** flip to `quick_profile.yaml` or `aiperf.enabled: false` for fast iteration, then return to `single_run.yaml` \u002F `sweep.yaml` when characterising under load.\n8. **Triage failures:** see Troubleshooting above and [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md) for empty-citation \u002F bottleneck=N\u002FA patterns.\n","# RAG-Perf — config-driven perf benchmark CLI\n\n## Purpose\n\nDrive a deployed NVIDIA RAG Blueprint server with a YAML config, run a server-side **profiling pass** (per-stage timing, citation quality, bottleneck inference) and an optional **aiperf load test** (TTFT \u002F E2E \u002F token & request throughput \u002F error rate), and write a unified report. The CLI is intentionally minimal: `rag-perf -c \u003Cconfig>` plus `--help` \u002F `--version`. Behaviour is *fully* config-driven; field variations belong in YAML.\n\n## Scope\n\n- **Accuracy \u002F RAGAS** scoring of answer quality → use the **rag-eval** skill.\n- **Deploying, repairing, or configuring services** (compose, helm, NIM env vars) → use the **rag-blueprint** skill.\n- **Production monitoring \u002F alerting** — rag-perf is a one-shot benchmark tool.\n- **Runtime requirement:** a deployed RAG server reachable on the network.\n\n## Prerequisites\n\n- Repo cloned; **run commands from the repo root** (config paths in the presets are repo-root-relative).\n- Python **3.11+** and **uv** on PATH.\n- Install rag-perf into its own uv-managed venv: `uv sync --project scripts\u002Frag-perf`.\n- For unit tests: install dev extras as well — `uv sync --project scripts\u002Frag-perf --extra dev` (otherwise `pytest-asyncio` is missing and async tests error out at collection time).\n- A reachable RAG server (default `http:\u002F\u002Flocalhost:8081`). For the aiperf phase, the bundled `nvidia_rag` endpoint plugin must be installed — `pip install -e .\u002Fscripts\u002Frag-perf` registers it via the `aiperf.plugins` entry point.\n- For **synthetic** queries: an OpenAI-compatible chat-completions endpoint reachable at `synthetic.llm_url` (default `http:\u002F\u002Flocalhost:8999\u002Fv1\u002Fchat\u002Fcompletions`).\n- rag-perf itself runs without `NVIDIA_API_KEY` (unlike rag-eval). The synthetic LLM endpoint may require its own auth — that's the deployment's concern.\n\n## Instructions\n\n1. **Pick a preset.** The three under [`scripts\u002Frag-perf\u002Fconfigs\u002F`](..\u002F..\u002Fscripts\u002Frag-perf\u002Fconfigs) are:\n   - `quick_profile.yaml` — profile-only, ~30 s. Skips load test. For fast iteration on retrieval \u002F reranker tuning.\n   - `single_run.yaml` — one concurrency level, profiling + aiperf, ~2 min. Regression checks.\n   - `sweep.yaml` — multi-axis sweep. `load.concurrency`, `rag.vdb_top_k`, `rag.reranker_top_k` are all `int | list[int]`; any of them as a list becomes a sweep axis (Cartesian product).\n\n2. **Edit the preset.** **Required:** replace `rag.collection_names: [\"\u003Ccollection_name>\"]` with a real collection on the deployed ingestor server. Verify the collection exists via `GET \u002Fv1\u002Fcollections` on the ingestor. The placeholder `\u003Ccollection_name>` validates fine but every request will fail at retrieval. Use a copied YAML preset for variants; the CLI surface is intentionally config-only.\n\n3. **Run.** From repo root:\n   ```bash\n   uv run --project scripts\u002Frag-perf rag-perf -c scripts\u002Frag-perf\u002Fconfigs\u002Fsingle_run.yaml\n   ```\n   Same form for the other presets. The CLI accepts only `-c \u002F --config` (required), `--help`, `--version`.\n\n4. **Read stdout.** Every invocation prints, in order: a startup banner, a one-line summary, the **fully resolved config as YAML** (so the run is reproducible from terminal output), per-grid-point progress with the **shlex-joined aiperf command** in copy-pastable form, a **rich per-point summary table** (stage breakdown with bars, citation quality, bottleneck, load-test block), and finally a **side-by-side comparison table** auto-labelled by whichever axis varied. See [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md).\n\n5. **Inspect artifacts.** Layout depends on run shape — flat for single-point + `iterations=1`, nested under `iter_\u003Ci>\u002F\u003Cpoint>\u002F...` otherwise. See [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md) for the full directory tree, file purposes, and how to parse `results.json` \u002F `results.csv` \u002F `report.md`.\n\n6. **Summarise for the user.** When reporting back, follow the playbook in [`references\u002Foutput-and-analysis.md#summarising-results-to-the-user`](references\u002Foutput-and-analysis.md#summarising-results-to-the-user): pick the canonical result file for the run shape, build a headline table (concurrency × top-k axes × TTFT × throughput × bottleneck × citation quality), compute scaling efficiency on sweeps, **always flag** zero citations \u002F non-zero error rate \u002F suspect `llm_ttft_ms` \u002F small-sample p99, and propose a concrete next-experiment YAML.\n\n7. **Tune.** Schema is fully documented in [`docs\u002Fperformance-benchmarking.md`](..\u002F..\u002Fdocs\u002Fperformance-benchmarking.md) and the deeper-dive references below. Common knobs: turn `aiperf.enabled: false` for profile-only mode, increase `load.iterations` for variance estimation, set `load.sleep_between_points_s: 60` for overnight Cartesian sweeps.\n\n## Examples\n\n**Profile-only (quickest signal on retrieval \u002F reranker tuning):**\n\n```bash\nuv run --project scripts\u002Frag-perf rag-perf -c scripts\u002Frag-perf\u002Fconfigs\u002Fquick_profile.yaml\n```\n\nOutput: `rag-perf-results\u002Fquick_profile\u002Frun_\u003Cts>\u002F{profile_report.md, profile_results.json, profiling\u002F}`. The `aiperf_rag_on\u002F` directory is omitted. Filenames are `profile_*` because `aiperf.enabled: false`.\n\n**Single benchmark point with full report:**\n\n```bash\nuv run --project scripts\u002Frag-perf rag-perf -c scripts\u002Frag-perf\u002Fconfigs\u002Fsingle_run.yaml\n```\n\nOutput: flat `run_\u003Cts>\u002F{report.md, results.json, results.csv, profiling\u002F, aiperf_rag_on\u002F}`.\n\n**Concurrency sweep:**\n\n```bash\nuv run --project scripts\u002Frag-perf rag-perf -c scripts\u002Frag-perf\u002Fconfigs\u002Fsweep.yaml\n```\n\nOutput: nested `run_\u003Cts>\u002Fiter_1\u002F\u003CCR:_VDB-K:_RERANKER-K:_…>\u002F{profiling,aiperf_rag_on}\u002F` per point, plus aggregate `report.md` \u002F `results.json` \u002F `results.csv` at the run root.\n\n**Run unit tests:**\n\n```bash\nuv sync --project scripts\u002Frag-perf --extra dev   # one-time, installs pytest-asyncio\nuv run --project scripts\u002Frag-perf python -m pytest tests\u002Funit\u002Ftest_rag_perf\u002F\n```\n\n## Limitations\n\n- The CLI is **config-only**: author or copy YAML to vary a parameter.\n- `load.concurrency` \u002F `rag.vdb_top_k` \u002F `rag.reranker_top_k` accept `int | list[int]`; the validator requires unique list values because each value names a unique point dir.\n- `input.file` and `input.synthetic` follow an XOR rule — both set fails validation. When neither is set, `synthetic` auto-fills with defaults so a bare config still validates.\n- File-based input format is **inferred from extension only** (`.jsonl` or `.csv`); other extensions are rejected.\n- Synthetic generation streams each query to disk as it completes (failure-resilient) but **fails fast on the first LLM error** — partial JSONL is preserved. Re-run after fixing the endpoint.\n- Reasoning models (Nemotron Omni, Qwen-Reasoning) require `synthetic.disable_thinking: true` (the default). Without it the model exhausts the token budget on chain-of-thought and `content` returns empty — the generator now raises with a clear message instead of substituting `reasoning_content` for the answer.\n- aiperf-specific knobs outside the YAML surface (request rate distribution, GPU telemetry config, etc.) require editing `AiperfRunner._base_aiperf_cmd` in `scripts\u002Frag-perf\u002Frag_perf\u002Frunner.py`.\n- Procedural detail lives under **`references\u002F`** to keep this file concise.\n\n## Troubleshooting\n\n| Error \u002F signal | Likely cause | What to do |\n|---|---|---|\n| `Configuration errors in \u003Cyaml>:  •  input  —  ... XOR rule` | Both `input.file` and `input.synthetic` set | Pick one. The XOR validator runs at YAML load time. |\n| `input.file must end in .jsonl or .csv` | Extension other than `.jsonl` \u002F `.csv` | Rename or convert. |\n| `load.concurrency has duplicate values` | e.g. `[2, 2, 4]` | Each concurrency maps to a unique point dir; dedupe. |\n| `warmup_requests must be >= 1` | YAML had `warmup_requests: 0` | aiperf rejects warmup=0; minimum is 1. |\n| `LLM returned empty content (reasoning_content was populated — model exhausted its budget on chain-of-thought; raise min_query_tokens or set synthetic.disable_thinking=true).` | Reasoning model used CoT and ran out of tokens | Set `synthetic.disable_thinking: true` (the default) or raise `min_query_tokens`. |\n| `✗ All N profiling requests failed across M point(s).` + exit 1 | Bad URL, server down, wrong collection | Verify `target.url`, `rag.collection_names` (the `\u003Ccollection_name>` placeholder will hit this). |\n| Per-iteration `⚠ N profiling requests failed` warning, run continues | Some requests timed out \u002F errored mid-run | Check rag-server logs, raise `target.timeout_s`, drop concurrency. |\n| `RuntimeError: Random synthetic query generation failed at query N: ...` | LLM endpoint rejected a request mid-generation | Partial JSONL is at `synthetic.jsonl_output_path`; fix endpoint and re-run with reduced `num_queries`, or point `input.file` at the partial file. |\n| `Citation count (mean): 0` and `Citation relevance score: N\u002FA` for a non-empty deployment | Collection mismatch between `rag.collection_names` and what's actually ingested | Run `curl -s http:\u002F\u002F\u003Cingestor>:8082\u002Fv1\u002Fcollections` to list real collections. |\n| Tests error with `ModuleNotFoundError: No module named 'pytest_asyncio'` | Dev extras missing | `uv sync --project scripts\u002Frag-perf --extra dev`. |\n| CI: `ModuleNotFoundError: No module named 'ruamel'` from `tests\u002Funit\u002Ftest_rag_perf\u002F` | rag-perf package missing from CI venv | Add `uv pip install -e .\u002Fscripts\u002Frag-perf` after the top-level install in the unit-tests job. |\n\n## Gotchas\n\n- **Run from repo root.** Preset configs reference `scripts\u002Frag-perf\u002Fexamples\u002Fqueries.jsonl` and `scripts\u002Frag-perf\u002Fprompts\u002Fdefault_prompts.yaml` with repo-root-relative paths. Running from inside `scripts\u002Frag-perf\u002F` will fail those file lookups.\n- **CLI is config-only.** Edit the YAML or copy a preset for URL, concurrency, collection, and similar fields.\n- **Always edit `rag.collection_names` before the first run.** The presets ship with `[\"\u003Ccollection_name>\"]` as a deliberate placeholder. Validation passes, retrieval fails silently for every request — manifests as `Citation count (mean): 0` everywhere.\n- **`load.concurrency_list`, `rag.vdb_top_k_list`, `rag.reranker_top_k_list`** are read-only properties that normalise scalar-or-list to a list. Use them when reasoning about the grid; the underlying YAML field is whatever the user wrote.\n- **`aiperf.enabled: false` changes filenames.** The top-level outputs become `profile_report.md` \u002F `profile_results.json` \u002F `profile_results.csv`. The aggregate sweep table also suppresses load-test rows and the \"Optimal throughput\" footer.\n- **Resolved-config dump is verbose** (50+ lines) — expected. It's what makes terminal output a self-contained reproducer; don't filter it out in scripts.\n- **The aiperf shell command is logged before each subprocess.** Look for `\\n  $ python -m aiperf profile -m ... --endpoint-type nvidia_rag ...` in stdout — copy-paste runnable for reproducing a single point outside rag-perf.\n- **`--endpoint-type nvidia_rag`** comes from the bundled plugin at `scripts\u002Frag-perf\u002Frag_perf\u002Fplugin\u002Fnvidia_rag.py`. It teaches aiperf about the RAG `\u002Fv1\u002Fgenerate` request shape and parses citations + per-stage `metrics` out of the SSE stream. If aiperf can't resolve `nvidia_rag`, rag-perf needs editable installation in the venv — re-run `uv sync --project scripts\u002Frag-perf` (or `uv pip install -e .\u002Fscripts\u002Frag-perf`).\n- **Sweep-mode point-name collision.** When two points differ only in concurrency (e.g. `[1, 4]` × single `vdb_top_k`), the dir name encodes everything: `CR:1_ISL:50_OSL:512_VDB-K:20_RERANKER-K:4_Model:...`. Cluster \u002F GPU \u002F experiment_name (`output.cluster`, `output.gpu`, `output.experiment_name`) are appended too — useful for diff-friendly artifact paths across machines.\n- **`load.iterations > 1` repeats the entire grid**. Each repetition writes to its own `iter_\u003Ci>\u002F`. Aggregate CSV row count = `n_points × iterations`.\n\n## Source of truth\n\n| Piece | Location |\n|---|---|\n| Driver | [`scripts\u002Frag-perf\u002Frag_perf\u002Fcli.py`](..\u002F..\u002Fscripts\u002Frag-perf\u002Frag_perf\u002Fcli.py) (`main` is the single Click command) |\n| Schema | [`scripts\u002Frag-perf\u002Frag_perf\u002Fconfig.py`](..\u002F..\u002Fscripts\u002Frag-perf\u002Frag_perf\u002Fconfig.py) (`RunConfig` and sub-models) |\n| Orchestrator | [`scripts\u002Frag-perf\u002Frag_perf\u002Frunner.py`](..\u002F..\u002Fscripts\u002Frag-perf\u002Frag_perf\u002Frunner.py) (`BenchmarkRunner.run`, `RagProfiler`, `AiperfRunner`) |\n| aiperf plugin | [`scripts\u002Frag-perf\u002Frag_perf\u002Fplugin\u002Fnvidia_rag.py`](..\u002F..\u002Fscripts\u002Frag-perf\u002Frag_perf\u002Fplugin\u002Fnvidia_rag.py) |\n| User-facing doc | [`docs\u002Fperformance-benchmarking.md`](..\u002F..\u002Fdocs\u002Fperformance-benchmarking.md) |\n| Presets | [`scripts\u002Frag-perf\u002Fconfigs\u002F{quick_profile,single_run,sweep}.yaml`](..\u002F..\u002Fscripts\u002Frag-perf\u002Fconfigs\u002F) |\n| Sample queries | [`scripts\u002Frag-perf\u002Fexamples\u002Fqueries.jsonl`](..\u002F..\u002Fscripts\u002Frag-perf\u002Fexamples\u002Fqueries.jsonl) |\n| Synthetic prompts | [`scripts\u002Frag-perf\u002Fprompts\u002Fdefault_prompts.yaml`](..\u002F..\u002Fscripts\u002Frag-perf\u002Fprompts\u002Fdefault_prompts.yaml) |\n| Config schema details | [`references\u002Fconfig-schema.md`](references\u002Fconfig-schema.md) |\n| Synthetic-query generation | [`references\u002Fsynthetic-generation.md`](references\u002Fsynthetic-generation.md) |\n| Output layout & metric semantics | [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md) |\n\n## Agent playbook\n\n1. **Sync deps:** `uv sync --project scripts\u002Frag-perf` (one-time per checkout).\n2. **Pick & customise a preset:** copy `scripts\u002Frag-perf\u002Fconfigs\u002F\u003Cpreset>.yaml` if you want a variant; always set `rag.collection_names` to a real collection.\n3. **Run:** `uv run --project scripts\u002Frag-perf rag-perf -c \u003Cconfig>` from repo root.\n4. **Read the per-point + aggregate tables on stdout.** Bottleneck inference is in the per-point profiling section; comparison across points is the final aggregate table.\n5. **Parse artifacts** under `output.dir\u002Frun_\u003Cts>\u002F` — see [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md). For multi-point runs, `results.csv` has one row per (point × iteration).\n6. **Summarise for the user** using the playbook in [`references\u002Foutput-and-analysis.md#summarising-results-to-the-user`](references\u002Foutput-and-analysis.md#summarising-results-to-the-user) — headline table, scaling-efficiency math for sweeps, mandatory flags for zero citations \u002F non-zero errors \u002F suspect `llm_ttft_ms` \u002F low sample size, and a concrete next-experiment YAML.\n7. **Tune retrieval \u002F reranker:** flip to `quick_profile.yaml` or `aiperf.enabled: false` for fast iteration, then return to `single_run.yaml` \u002F `sweep.yaml` when characterising under load.\n8. **Triage failures:** see Troubleshooting above and [`references\u002Foutput-and-analysis.md`](references\u002Foutput-and-analysis.md) for empty-citation \u002F bottleneck=N\u002FA patterns.\n",{"name":5,"license":15,"version":52,"metadata":53,"description":7,"allowed-tools":71,"compatibility":72},"2.6.0",{"tags":54,"author":60,"domain":61,"languages":62,"frameworks":65,"github-url":66,"tool-version":67,"argument-hint":68,"endpoint-openapi-schemas":69},[55,56,27,31,57,58,59],"nvidia","blueprint","benchmarking","aiperf","nvidia-rag-blueprint","NVIDIA RAG \u003Cfoundational-rag-dev@exchange.nvidia.com>","ai-ml",[63,64],"python","shell",[58,42],"https:\u002F\u002Fgithub.com\u002FNVIDIA-AI-Blueprints\u002Frag","0.1.0","rag-perf | aiperf | TTFT | latency | throughput | concurrency sweep | bottleneck | retrieval \u002F reranker tuning | profile-only | synthetic queries | quick_profile.yaml | single_run.yaml | sweep.yaml | uv run --project scripts\u002Frag-perf",[70],"docs\u002Fapi_reference\u002Fopenapi_schema_rag_server.json","Read Grep Glob Bash(ls *) Bash(python3 *) Bash(uv *) Bash(cat *) Bash(curl *) Write Edit","Repository checkout with uv; Python 3.11+; run from repo root; uv sync --project scripts\u002Frag-perf (perf deps live in scripts\u002Frag-perf\u002Fpyproject.toml); reachable RAG server (default http:\u002F\u002Flocalhost:8081); for synthetic queries an OpenAI-compatible chat-completions endpoint is required (default http:\u002F\u002Flocalhost:8999\u002Fv1\u002Fchat\u002Fcompletions); aiperf load-test phase uses the bundled nvidia_rag endpoint plugin, registered automatically when rag-perf is installed editable.","main","7a940b8e5af16c95e5c190a524421d32fc7445a9",".agent\u002Fskills\u002Frag-perf","https:\u002F\u002Fgithub.com\u002Fheadmktlucca-ui\u002Fneuroads-lp","https:\u002F\u002Fgithub.com\u002Fheadmktlucca-ui","https:\u002F\u002Fneuroads.com.br","2026-08-12 00:42:44",{"source":8,"skillName":5,"commands":81,"prompt":91,"zipAvailable":92},[82,85,88],{"tool":83,"command":84},"npx","npx skills add headmktlucca-ui\u002Fneuroads-lp --skill rag-perf",{"tool":86,"command":87},"bunx","bunx skills add headmktlucca-ui\u002Fneuroads-lp --skill rag-perf",{"tool":89,"command":90},"pnpm","pnpm dlx skills add headmktlucca-ui\u002Fneuroads-lp --skill rag-perf","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fheadmktlucca-ui\u002Fneuroads-lp\u002Fblob\u002Fmain\u002F.agent\u002Fskills\u002Frag-perf\u002FSKILL.md",true,[94,100,104,108,112,116,120,124],{"path":95,"resourceType":96,"resourceTypeDesc":97,"sizeBytes":98,"sourceHtmlUrl":99},"BENCHMARK.md","REFERENCE","参考文档",2672,"https:\u002F\u002Fgithub.com\u002Fheadmktlucca-ui\u002Fneuroads-lp\u002Fblob\u002Fmain\u002F.agent\u002Fskills\u002Frag-perf\u002FBENCHMARK.md",{"path":101,"resourceType":96,"resourceTypeDesc":97,"sizeBytes":102,"sourceHtmlUrl":103},"eval\u002Fh100.json",2440,"https:\u002F\u002Fgithub.com\u002Fheadmktlucca-ui\u002Fneuroads-lp\u002Fblob\u002Fmain\u002F.agent\u002Fskills\u002Frag-perf\u002Feval\u002Fh100.json",{"path":105,"resourceType":96,"resourceTypeDesc":97,"sizeBytes":106,"sourceHtmlUrl":107},"eval\u002Fnvidia_hosted.json",1887,"https:\u002F\u002Fgithub.com\u002Fheadmktlucca-ui\u002Fneuroads-lp\u002Fblob\u002Fmain\u002F.agent\u002Fskills\u002Frag-perf\u002Feval\u002Fnvidia_hosted.json",{"path":109,"resourceType":96,"resourceTypeDesc":97,"sizeBytes":110,"sourceHtmlUrl":111},"references\u002Fconfig-schema.md",7762,"https:\u002F\u002Fgithub.com\u002Fheadmktlucca-ui\u002Fneuroads-lp\u002Fblob\u002Fmain\u002F.agent\u002Fskills\u002Frag-perf\u002Freferences\u002Fconfig-schema.md",{"path":113,"resourceType":96,"resourceTypeDesc":97,"sizeBytes":114,"sourceHtmlUrl":115},"references\u002Foutput-and-analysis.md",13098,"https:\u002F\u002Fgithub.com\u002Fheadmktlucca-ui\u002Fneuroads-lp\u002Fblob\u002Fmain\u002F.agent\u002Fskills\u002Frag-perf\u002Freferences\u002Foutput-and-analysis.md",{"path":117,"resourceType":96,"resourceTypeDesc":97,"sizeBytes":118,"sourceHtmlUrl":119},"references\u002Fsynthetic-generation.md",7341,"https:\u002F\u002Fgithub.com\u002Fheadmktlucca-ui\u002Fneuroads-lp\u002Fblob\u002Fmain\u002F.agent\u002Fskills\u002Frag-perf\u002Freferences\u002Fsynthetic-generation.md",{"path":121,"resourceType":96,"resourceTypeDesc":97,"sizeBytes":122,"sourceHtmlUrl":123},"skill-card.md",2750,"https:\u002F\u002Fgithub.com\u002Fheadmktlucca-ui\u002Fneuroads-lp\u002Fblob\u002Fmain\u002F.agent\u002Fskills\u002Frag-perf\u002Fskill-card.md",{"path":125,"resourceType":126,"resourceTypeDesc":127,"sizeBytes":128,"sourceHtmlUrl":129},"skill.oms.sig","OTHER","其他",5549,"https:\u002F\u002Fgithub.com\u002Fheadmktlucca-ui\u002Fneuroads-lp\u002Fblob\u002Fmain\u002F.agent\u002Fskills\u002Frag-perf\u002Fskill.oms.sig",[131,154,179,205,223,257],{"skillNo":132,"name":133,"nameZh":6,"description":134,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":135,"licenseFileUrl":6,"sourceHtmlUrl":136,"categories":137,"keywords":142,"publishedAt":47},"01M6PTDCZTXTGSTX4RFK901YSZ","xunit-skill","Generates xUnit.net tests in C#. Covers Fact\u002FTheory, constructor injection, IClassFixture, and FluentAssertions. Use when user mentions \"xUnit\", \"[Fact]\", \"[Theory]\", \"Assert.Equal\", \"C# xUnit\". Triggers on: \"xUnit\", \"[Fact]\", \"[Theory]\", \"Assert.Equal C#\", \"xUnit.net\".","MIT","https:\u002F\u002Fgithub.com\u002Fheadmktlucca-ui\u002Fneuroads-lp\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fxunit-skill\u002FSKILL.md",[138,139],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":140,"categoryName":141,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",[143,144,145,146,147,148,149,150,151,152,153],"xunit","xunit skill","testing","injection","user","use","tests","triggers","unit","unit-testing","C#",{"skillNo":155,"name":156,"nameZh":6,"description":157,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":135,"licenseFileUrl":6,"sourceHtmlUrl":158,"categories":159,"keywords":162,"publishedAt":47},"01M6PTCB59ZTG5FTMWGPYEMMBP","xcuitest-skill","Generates XCUITest UI tests for iOS\u002FiPadOS apps in Swift. Apple's native testing framework for reliable, fast UI automation. Supports local simulators and TestMu AI cloud real devices. Use when user mentions \"XCUITest\", \"XCTest\", \"iOS UI test\", \"Swift test\", \"XCUIApplication\". Triggers on: \"XCUITest\", \"XCTest UI\", \"iOS UI test\", \"Swift UI test\", \"XCUIApplication\", \"TestMu\".","https:\u002F\u002Fgithub.com\u002Fheadmktlucca-ui\u002Fneuroads-lp\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fxcuitest-skill\u002FSKILL.md",[160,161],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":140,"categoryName":141,"taxonomyType":21,"taxonomyTypeDesc":22},[163,164,145,165,166,167,168,169,148,170,171,172,173,174,149,150,175,176,177,178],"xcuitest","xcuitest skill","automation","test","testing framework","ios","devices","mobile","mobile-testing","ui test","local","apple","apps","native","for","objective",{"skillNo":180,"name":181,"nameZh":6,"description":182,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":135,"licenseFileUrl":6,"sourceHtmlUrl":183,"categories":184,"keywords":189,"publishedAt":47},"01M6PTATMNTY9NF4J7KSR76YK6","world-builder","Create and manage generative game zones for Dorothy's Pokemon-style world. Use this skill when creating, updating, or designing game worlds via MCP tools.","https:\u002F\u002Fgithub.com\u002Fheadmktlucca-ui\u002Fneuroads-lp\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fworld-builder\u002FSKILL.md",[185,188],{"categoryNo":186,"categoryName":187,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000015","视觉设计",{"categoryNo":140,"categoryName":141,"taxonomyType":21,"taxonomyTypeDesc":22},[190,191,192,193,194,195,196,197,148,198,199,200,201,202,203,177,204],"builder","world","world builder","MCP","create","mcp tools","Tools","game","pokemon","manage","style","via","theme","zone","generative",{"skillNo":206,"name":207,"nameZh":6,"description":208,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":135,"licenseFileUrl":6,"sourceHtmlUrl":209,"categories":210,"keywords":213,"publishedAt":47},"01M6PT9Z92JTSQEHE1QR0NNW5F","webdriverio-skill","Generates WebdriverIO (WDIO) automation tests in JavaScript or TypeScript. Supports local and TestMu AI cloud. Use when user mentions \"WebdriverIO\", \"WDIO\", \"wdio.conf\", \"browser.url\", \"$\", \"$$\". Triggers on: \"WebdriverIO\", \"WDIO\", \"wdio\", \"browser.$\".","https:\u002F\u002Fgithub.com\u002Fheadmktlucca-ui\u002Fneuroads-lp\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fwebdriverio-skill\u002FSKILL.md",[211,212],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":140,"categoryName":141,"taxonomyType":21,"taxonomyTypeDesc":22},[214,145,215,165,216,217,218,219,147,148,220,173,149,150,221,222],"webdriverio skill","typescript","javascript","browser","cloud","e2e","e2e-testing","URL","conf",{"skillNo":224,"name":225,"nameZh":6,"description":226,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":135,"licenseFileUrl":6,"sourceHtmlUrl":227,"categories":228,"keywords":238,"publishedAt":47},"01M6PT9NVK36ZWCQ69YXK3FR4K","web-design-guidelines","Web platform design and accessibility guidelines. Use when building web interfaces, auditing accessibility, implementing responsive layouts, or reviewing web UI code. Triggers on tasks involving HTML, CSS, web components, WCAG compliance, responsive design, or web performance.","https:\u002F\u002Fgithub.com\u002Fheadmktlucca-ui\u002Fneuroads-lp\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fweb\u002FSKILL.md",[229,232,233],{"categoryNo":230,"categoryName":231,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000009","前端开发",{"categoryNo":186,"categoryName":187,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":234,"categoryName":235,"taxonomyType":236,"taxonomyTypeDesc":237},"01JBSEEDCAT000000000000032","设计师","PROFESSION","适用职业",[239,240,241,242,243,244,31,245,246,247,248,249,250,251,252,253,254,255,150,256],"design","web","guidelines","web design guidelines","web design","css","components","web components","tasks","code","web performance","ui","platform design","html","responsive design","responsive","platform","implementing",{"skillNo":258,"name":259,"nameZh":6,"description":260,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":135,"licenseFileUrl":6,"sourceHtmlUrl":261,"categories":262,"keywords":265,"publishedAt":47},"01M6PT9A3JKRE3X34276F2FW5R","watchos-design-guidelines","Apple Human Interface Guidelines for Apple Watch. Use when building watchOS apps, complications, or workout features. Triggers on tasks involving Watch UI, Digital Crown, glanceable interfaces, or wrist-based interactions.","https:\u002F\u002Fgithub.com\u002Fheadmktlucca-ui\u002Fneuroads-lp\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fwatchos\u002FSKILL.md",[263,264],{"categoryNo":186,"categoryName":187,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":234,"categoryName":235,"taxonomyType":236,"taxonomyTypeDesc":237},[239,241,266,247,250,148,267,268,174,255,269,150,175,270,177,271,272],"watchos design guidelines","workout","human","watch","interface","platform-design","features"]