targets-scorers
Configure PyRIT prompt targets, target capabilities, scorers, score aggregation, and target/scorer troubleshooting.
- License 许可
- MIT
- 最近更新
- 2026-10-08
- Stars
- ★ 11
- Forks
- 3
01 前言 Front Matter
- name
- targets-scorers
- license
- MIT
- metadata
- { "disco-role": "operating" }
- description
- Configure PyRIT prompt targets, target capabilities, scorers, score aggregation, and target/scorer troubleshooting.
- disable-model-invocation
- true
02 资源文件 3
- 参考文档references/api-reference.md3.8 KB
- 参考文档references/targets-scorers-reference.md7.6 KB
- 参考文档references/troubleshooting.md3.0 KB
03 SKILL.md 正文
Targets and Scorers
Use this sub-skill when you need to:
- choose and configure a prompt target
- reason about target capabilities, modalities, auth, and rate limits
- choose or compose scorers
- aggregate, threshold, or batch score results
- debug credential, endpoint, browser, model, JSON, or modality failures
Start here
Fast routing
- Offline dry run, logging, or manual prompt capture:
TextTarget - OpenAI / Azure OpenAI / OpenAI-compatible services:
OpenAIChatTarget,OpenAIResponseTarget,OpenAICompletionTarget - Raw HTTP or API-style endpoints:
HTTPTarget,HTTPXAPITarget - Browser or web UI workflows:
PlaywrightTarget,PlaywrightCopilotTarget,WebSocketCopilotTarget - Provider fan-out or load balancing:
LiteLLMChatTarget,RoundRobinTarget - Local model / torch / download-heavy workflows:
HuggingFaceChatTarget - Exact or regex detectors:
SubStringScorer,RegexScorer - LLM-guided judgment:
SelfAsk*scorers,LlamaGuardScorer,ShieldGemmaScorer - Aggregation, thresholding, and memory-backed evaluation:
TrueFalseCompositeScorer,FloatScaleThresholdScorer,BatchScorer,pyrit.analytics
Boundaries
- For memory and configuration bootstrap, use setup-memory-core.
- For attacks and scenarios that consume targets and scorers, use attacks-scenarios.
- For converters and datasets that shape target input, use converters-datasets.
- For scanner/backend commands, use cli-backend-scanner.
Operating rule
- Pick the smallest target family that fits the endpoint or workflow.
- Check capabilities before assuming the target can accept a modality or conversation shape.
- Prefer local scorers when the signal is lexical or rule-based.
- Use LLM-backed scorers only when the judgment needs model reasoning.
- Keep live-service checks separate from the offline smoke script.
Bundled files
04 同仓库更多 Skills
obsidian-clipper-template-creator
Guide for creating templates for the Obsidian Web Clipper. Use when you want to create a new clipping template, understand available variables, or format clipped content.
obsidian-clipper-template-creator
Guide for creating templates for the Obsidian Web Clipper. Use when you want to create a new clipping template, understand available variables, or format clipped content.
python-api
Programmatic Object-Detection-Metrics API guidance for BoundingBox, BoundingBoxes, Evaluator, enums, coordinate conversion, AP/mAP semantics, and source-style import diagnostics.
file-evaluation
Folder-based PASCAL VOC AP/mAP evaluation from ground-truth and detection text files.
object-detection-metrics
Guides legacy Object-Detection-Metrics PASCAL VOC AP/mAP evaluation from detection text folders and direct Python API classes.
nx-workspace-patterns
Configure and optimize Nx monorepo workspaces. Use when setting up Nx, configuring project boundaries, optimizing build caching, or implementing affected commands.