AGENT SKILL
llm-evals
Build a small eval set before changing a prompt, model or retrieval step, and compare results before and after.
- License 许可
- MIT
- 所属仓库
- Zwin-ux/kit
- 最近更新
- 2026-11-02
- Stars
- ★ 0
- Forks
- 2
01 前言 Front Matter
- name
- llm-evals
- description
- Build a small eval set before changing a prompt, model or retrieval step, and compare results before and after.
03 SKILL.md 正文
LLM evals
Use when changing a prompt, switching models, or changing retrieval.
- Find existing evals (
evals/,tests/evals, a promptfoo or similar config). Use them if present. - If none exist, write 10 to 30 cases that cover the main task, known failures and edge cases. Store them as data (JSONL or YAML), not code.
- Pick a check per case: exact match, contains, JSON schema, or a rubric graded by a model. Prefer the cheapest check that catches the failure.
- Run the evals on the current version and record the score.
- Make the change. Run the evals again.
- Report both scores, the cases that changed, and the cost and latency difference.
- Never tune the prompt on the eval cases alone; keep a few held out.
04 同仓库更多 Skills
concepts
暂无简介
★ 0
write-tests
Add focused tests for important project behavior.
★ 0
workspace-setup
Set up monorepo and multi-package workspaces for agents and humans.
★ 0
ship-checklist
Run a practical pre-ship checklist for an app release.
★ 0
project-setup
Set up a clean project baseline for agents and humans.
★ 0
pr-ready
Prepare a clear pull request summary, test plan, and risk notes.
★ 0