AGENT SKILL

openai--plugin-eval--plugin-eval

plugin-eval — plugin-eval

License 许可
MIT
最近更新
2026-11-14
Stars
★ 13
Forks
2

01 前言 Front Matter

name
openai--plugin-eval--plugin-eval
origin
openai/plugins — plugin-eval/plugin-eval (MIT)
license
MIT
version
0.1.0
description
plugin-eval — plugin-eval
compatibility
yana-ai >= 0.14.0

03 SKILL.md 正文

Plugin Eval

Use this as the beginner-friendly umbrella entrypoint for local Codex skill and plugin evaluation.

Start Here

  1. Resolve whether the target path is a skill, a plugin, or another local folder.
  2. Prefer the chat-first router when the user speaks naturally or is not sure which command they need:
plugin-eval start <path> --request "<user request>" --format markdown
  1. Route natural chat requests to the matching workflow:
    • "Give me an analysis of the game dev skill." -> resolve the named skill path, run plugin-eval analyze <path> --format markdown, then initialize a benchmark and show the setup questions needed to tailor benchmark.json
    • "Evaluate this skill." -> plugin-eval analyze <path> --format markdown
    • "Why did this score that way?" -> plugin-eval analyze <path> --format markdown
    • "What should I fix first?" -> plugin-eval analyze <path> --format markdown
    • "Explain the token budget for this skill." -> plugin-eval explain-budget <path> --format markdown
    • "Measure the real token usage of this skill." -> benchmark flow, then plugin-eval measurement-plan
    • "Help me benchmark this plugin." -> starter benchmark flow
    • "What should I run next?" -> plugin-eval start <path> --request "What should I run next?" --format markdown
  2. If the user wants rewrite help after evaluation, route to ../improve-skill/SKILL.md.
  3. If the user wants a custom rubric, route to ../metric-pack-designer/SKILL.md.
  4. If the user names a skill instead of giving a path, resolve it locally before running commands:
    • check ~/.codex/skills/<skill-name> first
    • then check any repo-local skills/<skill-name> directory
    • if the name is still ambiguous, ask one short clarifying question before continuing
  5. When the request sounds like "analysis" rather than just "evaluate", do the fuller path:
    • run the report
    • initialize .plugin-eval/benchmark.json
    • surface the setup questions that will refine the starter scenarios
    • preview the dry-run command the user can execute next

Chat Requests To Recognize

  • Give me an analysis of the game dev skill.
  • Evaluate this skill.
  • Evaluate this plugin.
  • Why did this score that way?
  • What should I fix first?
  • Explain the token budget for this skill.
  • Measure the real token usage of this skill.
  • Help me benchmark this plugin.
  • What should I run next?

Matching Commands

plugin-eval start <path> --request "Evaluate this skill." --format markdown
plugin-eval start <path> --request "Give me a full analysis of this skill, including benchmark setup." --format markdown
plugin-eval analyze <path> --format markdown
plugin-eval explain-budget <path> --format markdown
plugin-eval measurement-plan <path> --format markdown
plugin-eval init-benchmark <path>
plugin-eval benchmark <path> --dry-run
plugin-eval benchmark <path>

Output Expectations

  • Prefer the JSON result as the source of truth.
  • Lead with At a Glance, Why It Matters, Fix First, and Recommended Next Step.
  • Keep the why content terse and easy to skim.
  • Call out whether budget numbers are static estimates or measured harness results.
  • Show the user the exact chat phrase they can reuse next, the plugin-eval start command that routes it, and the first local workflow command behind it.
  • When the user asks for an analysis of a named skill, do not stop at the report if benchmark setup is still missing.
  • When the user is asking about a skill specifically, hand off to ../evaluate-skill/SKILL.md.
  • When the user is asking about a plugin bundle, hand off to ../evaluate-plugin/SKILL.md.

References

  • ../../references/chat-first-workflows.md
  • ../../references/technical-design.md
  • ../../references/evaluation-result-schema.md

04 同仓库更多 Skills

Horonomos-laps
Horonomos-laps/Yana-AI

openai--sentry--sentry

Use when the user asks to inspect Sentry issues or events, summarize recent production errors, or pull basic Sentry health data via the Sentry API; perform read-only queries with the bundled script and require `SENTRY_AUTH_TOKEN`.

★ 13
Horonomos-laps
Horonomos-laps/Yana-AI

ito-basket-compare

Compare Itô prediction-market baskets against a user's knowledge base, portfolio notes, financial context, watchlist, or research thesis. Use for read-only basket comparison and gap analysis without investment advice or live trading.

★ 13
Horonomos-laps
Horonomos-laps/Yana-AI

implementing-network-traffic-baselining

Build network traffic baselines from NetFlow/IPFIX data using Python pandas for statistical analysis, z-score anomaly detection, and hourly/daily traffic pattern profiling

★ 13
Horonomos-laps
Horonomos-laps/Yana-AI

detecting-evasion-techniques-in-endpoint-logs

Detects defense evasion techniques used by adversaries in endpoint logs including log tampering, timestomping, process injection, and security tool disabling. Use when investigating suspicious endpoint behavior, building detection rules for evasion tactics, or conducting threat hunting for stealthy adversary activity. Activates for requests involving evasion detection, defense evasion analysis, log tampering detection, or MITRE ATT&CK TA0005.

★ 13
Horonomos-laps
Horonomos-laps/Yana-AI

implementing-microsegmentation-with-guardicore

Implementing microsegmentation using Akamai Guardicore Segmentation to map application dependencies, create granular network policies, visualize east-west traffic flows, and enforce least-privilege communication between workloads across data centers and cloud.

★ 13
Horonomos-laps
Horonomos-laps/Yana-AI

ramdisk-tmpfs-patterns

暂无简介

★ 13