openai--plugin-eval--metric-pack-designer
Design custom metric packs for plugin-eval so teams can add local evaluation rubrics that emit schema-compatible checks and metrics. Use when the user wants their own evaluation criteria or visualizations.
- License 许可
- MIT
- 最近更新
- 2026-11-14
- Stars
- ★ 13
- Forks
- 2
01 前言 Front Matter
- name
- openai--plugin-eval--metric-pack-designer
- origin
- openai/plugins — plugin-eval/metric-pack-designer (MIT)
- license
- MIT
- version
- 0.1.0
- description
- Design custom metric packs for plugin-eval so teams can add local evaluation rubrics that emit schema-compatible checks and metrics. Use when the user wants their own evaluation criteria or visualizations.
- compatibility
- yana-ai >= 0.14.0
03 SKILL.md 正文
Metric Pack Designer
Use this skill when the user wants to extend plugin-eval with a local rubric.
Workflow
- Clarify the custom rubric categories and target kinds.
- Define the smallest useful
checks[]andmetrics[]payload. - Create a metric-pack manifest plus a script that prints JSON to stdout.
- Run the pack through
plugin-eval analyze <path> --metric-pack <manifest.json>.
Design Rules
- Keep IDs stable across runs so comparisons stay meaningful.
- Emit only
checks[],metrics[], and optionalartifacts[]. - Do not try to overwrite the core score or summary.
- Prefer deterministic local signals over subjective text generation.
Reference
../../references/metric-pack-manifest.md
04 同仓库更多 Skills
openai--sentry--sentry
Use when the user asks to inspect Sentry issues or events, summarize recent production errors, or pull basic Sentry health data via the Sentry API; perform read-only queries with the bundled script and require `SENTRY_AUTH_TOKEN`.
ito-basket-compare
Compare Itô prediction-market baskets against a user's knowledge base, portfolio notes, financial context, watchlist, or research thesis. Use for read-only basket comparison and gap analysis without investment advice or live trading.
implementing-network-traffic-baselining
Build network traffic baselines from NetFlow/IPFIX data using Python pandas for statistical analysis, z-score anomaly detection, and hourly/daily traffic pattern profiling
detecting-evasion-techniques-in-endpoint-logs
Detects defense evasion techniques used by adversaries in endpoint logs including log tampering, timestomping, process injection, and security tool disabling. Use when investigating suspicious endpoint behavior, building detection rules for evasion tactics, or conducting threat hunting for stealthy adversary activity. Activates for requests involving evasion detection, defense evasion analysis, log tampering detection, or MITRE ATT&CK TA0005.
implementing-microsegmentation-with-guardicore
Implementing microsegmentation using Akamai Guardicore Segmentation to map application dependencies, create granular network policies, visualize east-west traffic flows, and enforce least-privilege communication between workloads across data centers and cloud.
ramdisk-tmpfs-patterns
暂无简介