AGENT SKILL

openai--plugin-eval--metric-pack-designer

Design custom metric packs for plugin-eval so teams can add local evaluation rubrics that emit schema-compatible checks and metrics. Use when the user wants their own evaluation criteria or visualizations.

License 许可
MIT
最近更新
2026-11-14
Stars
★ 13
Forks
2

01 前言 Front Matter

name
openai--plugin-eval--metric-pack-designer
origin
openai/plugins — plugin-eval/metric-pack-designer (MIT)
license
MIT
version
0.1.0
description
Design custom metric packs for plugin-eval so teams can add local evaluation rubrics that emit schema-compatible checks and metrics. Use when the user wants their own evaluation criteria or visualizations.
compatibility
yana-ai >= 0.14.0

03 SKILL.md 正文

Metric Pack Designer

Use this skill when the user wants to extend plugin-eval with a local rubric.

Workflow

  1. Clarify the custom rubric categories and target kinds.
  2. Define the smallest useful checks[] and metrics[] payload.
  3. Create a metric-pack manifest plus a script that prints JSON to stdout.
  4. Run the pack through plugin-eval analyze <path> --metric-pack <manifest.json>.

Design Rules

  • Keep IDs stable across runs so comparisons stay meaningful.
  • Emit only checks[], metrics[], and optional artifacts[].
  • Do not try to overwrite the core score or summary.
  • Prefer deterministic local signals over subjective text generation.

Reference

  • ../../references/metric-pack-manifest.md

04 同仓库更多 Skills

Horonomos-laps
Horonomos-laps/Yana-AI

openai--sentry--sentry

Use when the user asks to inspect Sentry issues or events, summarize recent production errors, or pull basic Sentry health data via the Sentry API; perform read-only queries with the bundled script and require `SENTRY_AUTH_TOKEN`.

★ 13
Horonomos-laps
Horonomos-laps/Yana-AI

ito-basket-compare

Compare Itô prediction-market baskets against a user's knowledge base, portfolio notes, financial context, watchlist, or research thesis. Use for read-only basket comparison and gap analysis without investment advice or live trading.

★ 13
Horonomos-laps
Horonomos-laps/Yana-AI

implementing-network-traffic-baselining

Build network traffic baselines from NetFlow/IPFIX data using Python pandas for statistical analysis, z-score anomaly detection, and hourly/daily traffic pattern profiling

★ 13
Horonomos-laps
Horonomos-laps/Yana-AI

detecting-evasion-techniques-in-endpoint-logs

Detects defense evasion techniques used by adversaries in endpoint logs including log tampering, timestomping, process injection, and security tool disabling. Use when investigating suspicious endpoint behavior, building detection rules for evasion tactics, or conducting threat hunting for stealthy adversary activity. Activates for requests involving evasion detection, defense evasion analysis, log tampering detection, or MITRE ATT&CK TA0005.

★ 13
Horonomos-laps
Horonomos-laps/Yana-AI

implementing-microsegmentation-with-guardicore

Implementing microsegmentation using Akamai Guardicore Segmentation to map application dependencies, create granular network policies, visualize east-west traffic flows, and enforce least-privilege communication between workloads across data centers and cloud.

★ 13
Horonomos-laps
Horonomos-laps/Yana-AI

ramdisk-tmpfs-patterns

暂无简介

★ 13