ow-eval
Paired eval of a raw host, the control plane, and the routed skills. Three trials. A clean HOME for the raw arm. Promote a skill only when success is not negative and cost rises by 10 percent or less.
- License 许可
- MIT
- 最近更新
- 2026-11-13
- Stars
- ★ 1
- Forks
- 0
01 前言 Front Matter
- name
- ow-eval
- description
- Paired eval of a raw host, the control plane, and the routed skills. Three trials. A clean HOME for the raw arm. Promote a skill only when success is not negative and cost rises by 10 percent or less.
03 SKILL.md 正文
Eval
Use this when a skill, route, or release needs a measured comparison. It extends ow-benchmark. It does not invent a winner.
Preconditions
A fixture exists. Arm A gets a clean HOME. The same model label is used for A, B, and C.
Arms
- A. A clean
HOMEwith no Orchestra files and no global skills. The prompt says to skip Orchestra. - B. The Orchestra control plane only. No specialist skill under test.
- C. The control plane plus the skills the route selected.
Same model label, same fixture, same budget. Run A, B, and C in an order that rotates across trials. Keep failed and interrupted trials.
Procedure
- Read the fixture under
runtime/benchmarks/orchestra-47/fixtures/. - Record task success, tests, Lighthouse when the fixture is visual, axe, steps, tokens, and time.
- A blind judge scores visual fixtures from shuffled screenshots.
- Call
PromoteSkillinruntime/internal/benchmark/promote.go. Three trials is the minimum. - Write the trial JSON under
runtime/benchmarks/orchestra-47/<fixture>/. Statusnot_runstaysnot_runwhen a model, a browser, or a judge did not run.
Output
A trial JSON file. not_run stays not_run when a model, a browser, or a judge did not run.
Verify
PromoteSkill is called only with the recorded trials. A claim stays none until the rule passes.
Skip
Do not enable a comparison claim. Do not put a skill on a Load line from a single trial. Do not share the operator's skill folders with arm A.
04 同仓库更多 Skills
writing-skills
Use when creating new skills, editing existing skills, or verifying skills work before deployment
webapp-testing
Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.
web-design-guidelines
Review UI code for Web Interface Guidelines compliance. Use when asked to "review my UI", "check accessibility", "audit design", "review UX", or "check my site against best practices". Do not load on Plan-only or backend-only jobs.
verification-before-completion
Use when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always
using-git-worktrees
Use when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallback
unslop
Turns vibecoded, AI-looking UI into a designed interface with its own character. Finds a direction from the product itself, then fixes spacing, color, icons, copy, type, layout, details, visuals and motion in that order, and can generate a real hero visual with an image tool when one is available. Use when the user asks to unslop, polish, clean up or redesign a screen so it stops looking AI-generated, or when a UI shows the usual tells like purple gradients, glows, gradient text, emoji labels, "New ✨" badges, glossy blobs, uppercase tracked labels, pulsing elements or uneven spacing. Works on React, Tailwind, HTML and CSS, for dashboards, apps and landing pages.