prompt-engineer-toolkit
Analyzes and rewrites prompts for better AI output, creates reusable prompt templates for marketing use cases (ad copy, email campaigns, social media), and structures end-to-end AI content workflows. Use when the user wants to improve prompts for AI-assisted marketing, build prompt templates, or optimize AI content workflows. Also use when the user mentions 'prompt engineering,' 'improve my prompts,' 'AI writing quality,' 'prompt templates,' or 'AI content workflow.'
- License 许可
- MIT
- 最近更新
- 2026-05-17
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- prompt-engineer-toolkit
- license
- MIT
- metadata
- { "author": "Alireza Rezvani", "updated": 1772755200000, "version": "1.0.0", "category": "marketing" }
- description
- Analyzes and rewrites prompts for better AI output, creates reusable prompt templates for marketing use cases (ad copy, email campaigns, social media), and structures end-to-end AI content workflows. Use when the user wants to improve prompts for AI-assisted marketing, build prompt templates, or optimize AI content workflows. Also use when the user mentions 'prompt engineering,' 'improve my prompts,' 'AI writing quality,' 'prompt templates,' or 'AI content workflow.'
02 资源文件 6
- 参考文档README.md1.2 KB
- 参考文档references/evaluation-rubric.md328 B
- 参考文档references/prompt-templates.md1.5 KB
- 参考文档references/technique-guide.md675 B
- 脚本scripts/prompt_tester.py7.6 KB
- 脚本scripts/prompt_versioner.py7.9 KB
03 SKILL.md 正文
Prompt Engineer Toolkit
Overview
Use this skill to move prompts from ad-hoc drafts to production assets with repeatable testing, versioning, and regression safety. It emphasizes measurable quality over intuition. Apply it when launching a new LLM feature that needs reliable outputs, when prompt quality degrades after model or instruction changes, when multiple team members edit prompts and need history/diffs, when you need evidence-based prompt choice for production rollout, or when you want consistent prompt governance across environments.
Core Capabilities
- A/B prompt evaluation against structured test cases
- Quantitative scoring for adherence, relevance, and safety checks
- Prompt version tracking with immutable history and changelog
- Prompt diffs to review behavior-impacting edits
- Reusable prompt templates and selection guidance
- Regression-friendly workflows for model/prompt updates
Key Workflows
1. Run Prompt A/B Test
Prepare JSON test cases and run:
python3 scripts/prompt_tester.py \
--prompt-a-file prompts/a.txt \
--prompt-b-file prompts/b.txt \
--cases-file testcases.json \
--runner-cmd 'my-llm-cli --prompt {prompt} --input {input}' \
--format text
Input can also come from stdin/--input JSON payload.
2. Choose Winner With Evidence
The tester scores outputs per case and aggregates:
- expected content coverage
- forbidden content violations
- regex/format compliance
- output length sanity
Use the higher-scoring prompt as candidate baseline, then run regression suite.
3. Version Prompts
# Add version
python3 scripts/prompt_versioner.py add \
--name support_classifier \
--prompt-file prompts/support_v3.txt \
--author alice
# Diff versions
python3 scripts/prompt_versioner.py diff --name support_classifier --from-version 2 --to-version 3
# Changelog
python3 scripts/prompt_versioner.py changelog --name support_classifier
4. Regression Loop
- Store baseline version.
- Propose prompt edits.
- Re-run A/B test.
- Promote only if score and safety constraints improve.
Script Interfaces
python3 scripts/prompt_tester.py --help- Reads prompts/cases from stdin or
--input - Optional external runner command
- Emits text or JSON metrics
- Reads prompts/cases from stdin or
python3 scripts/prompt_versioner.py --help- Manages prompt history (
add,list,diff,changelog) - Stores metadata and content snapshots locally
- Manages prompt history (
Pitfalls, Best Practices & Review Checklist
Avoid these mistakes:
- Picking prompts from single-case outputs — use a realistic, edge-case-rich test suite.
- Changing prompt and model simultaneously — always isolate variables.
- Missing
must_not_contain(forbidden-content) checks in evaluation criteria. - Editing prompts without version metadata, author, or change rationale.
- Skipping semantic diffs before deploying a new prompt version.
- Optimizing one benchmark while harming edge cases — track the full suite.
- Model swap without rerunning the baseline A/B suite.
Before promoting any prompt, confirm:
- [ ] Task intent is explicit and unambiguous.
- [ ] Output schema/format is explicit.
- [ ] Safety and exclusion constraints are explicit.
- [ ] No contradictory instructions.
- [ ] No unnecessary verbosity tokens.
- [ ] A/B score improves and violation count stays at zero.
References
- references/prompt-templates.md
- references/technique-guide.md
- references/evaluation-rubric.md
- README.md
Evaluation Design
Each test case should define:
input: realistic production-like inputexpected_contains: required markers/contentforbidden_contains: disallowed phrases or unsafe contentexpected_regex: required structural patterns
This enables deterministic grading across prompt variants.
Versioning Policy
- Use semantic prompt identifiers per feature (
support_classifier,ad_copy_shortform). - Record author + change note for every revision.
- Never overwrite historical versions.
- Diff before promoting a new prompt to production.
Rollout Strategy
- Create baseline prompt version.
- Propose candidate prompt.
- Run A/B suite against same cases.
- Promote only if winner improves average and keeps violation count at zero.
- Track post-release feedback and feed new failure cases back into test suite.
04 同仓库更多 Skills
ra-qm-skills
12 regulatory & QM agent skills and plugins for Claude Code, Codex, Gemini CLI, Cursor, OpenClaw. ISO 13485 QMS, MDR 2017/745, FDA 510(k)/PMA, ISO 27001 ISMS, GDPR/DSGVO, risk management (ISO 14971), CAPA, document control, auditing. Python tools (stdlib-only).
iso42001-specialist
ISO/IEC 42001:2023 AI Management System (AIMS) specialist for compliance teams running internal audits. Three decisions: (1) Where are the gaps against Clauses 4-10 and what do we close first? (2) What goes in the AI risk register and which Annex A controls treat each risk? (3) What's the 12-month internal audit plan that satisfies Clause 9.2? Use when preparing for certification, scoping internal audit cycles, or onboarding AI systems into an existing ISMS (27001) / QMS (13485) program. NOT an executive AI strategy skill (see chief-ai-officer-advisor). NOT EU AI Act compliance (see compliance-team-eu-ai-act).
eu-ai-act-specialist
EU AI Act (Regulation (EU) 2024/1689) operational compliance for compliance teams. Three Article-level decisions: (1) What's the risk tier of this AI system — prohibited (Art. 5), high-risk (Art. 6 + Annex III), limited-risk (Art. 50), or minimal-risk? (2) For high-risk systems, what's the Article 43 conformity assessment route (Module A internal control vs Module H full QMS + notified body) and what goes in the Annex IV technical documentation? (3) Per organizational role (provider / deployer / importer / distributor / authorized representative), what are the active obligations and deadlines? Use during AI system intake review, when planning conformity assessment, or when scoping deployer obligations. Cites Articles + Annexes for every output. NOT executive AI strategy (see chief-ai-officer-advisor). NOT a legal substitute.
iso42001-specialist
ISO/IEC 42001:2023 AI Management System (AIMS) specialist for compliance teams running internal audits. Three decisions: (1) Where are the gaps against Clauses 4-10 and what do we close first? (2) What goes in the AI risk register and which Annex A controls treat each risk? (3) What's the 12-month internal audit plan that satisfies Clause 9.2? Use when preparing for certification, scoping internal audit cycles, or onboarding AI systems into an existing ISMS (27001) / QMS (13485) program. NOT an executive AI strategy skill (see chief-ai-officer-advisor). NOT EU AI Act compliance (see compliance-team-eu-ai-act).
eu-ai-act-specialist
EU AI Act (Regulation (EU) 2024/1689) operational compliance for compliance teams. Three Article-level decisions: (1) What's the risk tier of this AI system — prohibited (Art. 5), high-risk (Art. 6 + Annex III), limited-risk (Art. 50), or minimal-risk? (2) For high-risk systems, what's the Article 43 conformity assessment route (Module A internal control vs Module H full QMS + notified body) and what goes in the Annex IV technical documentation? (3) Per organizational role (provider / deployer / importer / distributor / authorized representative), what are the active obligations and deadlines? Use during AI system intake review, when planning conformity assessment, or when scoping deployer obligations. Cites Articles + Annexes for every output. NOT executive AI strategy (see chief-ai-officer-advisor). NOT a legal substitute.
scrum-master
Advanced Scrum Master skill for data-driven agile team analysis and coaching. Use when the user asks about sprint planning, velocity tracking, retrospectives, standup facilitation, backlog grooming, story points, burndown charts, blocker resolution, or agile team health. Runs Python scripts to analyse sprint JSON exports from Jira or similar tools: velocity_analyzer.py for Monte Carlo sprint forecasting, sprint_health_scorer.py for multi-dimension health scoring, and retrospective_analyzer.py for action-item and theme tracking. Produces confidence-interval forecasts, health grade reports, and improvement-velocity trends for high-performing Scrum teams.