AGENT SKILL

targets-scorers

Configure PyRIT prompt targets, target capabilities, scorers, score aggregation, and target/scorer troubleshooting.

License 许可
MIT
最近更新
2026-10-08
Stars
★ 11
Forks
3

01 前言 Front Matter

name
targets-scorers
license
MIT
metadata
{ "disco-role": "operating" }
description
Configure PyRIT prompt targets, target capabilities, scorers, score aggregation, and target/scorer troubleshooting.
disable-model-invocation
true

02 资源文件 3

03 SKILL.md 正文

Targets and Scorers

Use this sub-skill when you need to:

  • choose and configure a prompt target
  • reason about target capabilities, modalities, auth, and rate limits
  • choose or compose scorers
  • aggregate, threshold, or batch score results
  • debug credential, endpoint, browser, model, JSON, or modality failures

Start here

Fast routing

  • Offline dry run, logging, or manual prompt capture: TextTarget
  • OpenAI / Azure OpenAI / OpenAI-compatible services: OpenAIChatTarget, OpenAIResponseTarget, OpenAICompletionTarget
  • Raw HTTP or API-style endpoints: HTTPTarget, HTTPXAPITarget
  • Browser or web UI workflows: PlaywrightTarget, PlaywrightCopilotTarget, WebSocketCopilotTarget
  • Provider fan-out or load balancing: LiteLLMChatTarget, RoundRobinTarget
  • Local model / torch / download-heavy workflows: HuggingFaceChatTarget
  • Exact or regex detectors: SubStringScorer, RegexScorer
  • LLM-guided judgment: SelfAsk* scorers, LlamaGuardScorer, ShieldGemmaScorer
  • Aggregation, thresholding, and memory-backed evaluation: TrueFalseCompositeScorer, FloatScaleThresholdScorer, BatchScorer, pyrit.analytics

Boundaries

Operating rule

  1. Pick the smallest target family that fits the endpoint or workflow.
  2. Check capabilities before assuming the target can accept a modality or conversation shape.
  3. Prefer local scorers when the signal is lexical or rule-based.
  4. Use LLM-backed scorers only when the judgment needs model reasoning.
  5. Keep live-service checks separate from the offline smoke script.

Bundled files

04 同仓库更多 Skills