AGENT SKILL

prompt-engineer-toolkit

Turns marketing prompts into tested, versioned production assets: A/B prompt evaluation against structured test cases, immutable prompt version history with diffs, ready-to-use marketing prompt templates (ad copy, email campaigns, social posts, landing pages, SEO meta), and an LLM-governance playbook for marketing teams (claim discipline, disclosure rules, human-review gates). Use when a marketing team relies on AI-generated content and needs prompt quality to be measurable and safe — or when the user mentions 'prompt engineering,' 'improve my prompts,' 'prompt templates,' 'prompt versioning,' 'AI content workflow,' or 'AI governance for marketing.'

License 许可
MIT
最近更新
2026-10-02
Stars
★ 0
Forks
0

01 前言 Front Matter

name
prompt-engineer-toolkit
license
MIT
metadata
{ "author": "Alireza Rezvani", "updated": 1772755200000, "version": "1.0.0", "category": "marketing" }
description
Turns marketing prompts into tested, versioned production assets: A/B prompt evaluation against structured test cases, immutable prompt version history with diffs, ready-to-use marketing prompt templates (ad copy, email campaigns, social posts, landing pages, SEO meta), and an LLM-governance playbook for marketing teams (claim discipline, disclosure rules, human-review gates). Use when a marketing team relies on AI-generated content and needs prompt quality to be measurable and safe — or when the user mentions 'prompt engineering,' 'improve my prompts,' 'prompt templates,' 'prompt versioning,' 'AI content workflow,' or 'AI governance for marketing.'

02 资源文件 6

03 SKILL.md 正文

Prompt Engineer Toolkit

Overview

Use this skill to move prompts from ad-hoc drafts to production assets with repeatable testing, versioning, and regression safety. It emphasizes measurable quality over intuition. Apply it when launching a new LLM feature that needs reliable outputs, when prompt quality degrades after model or instruction changes, when multiple team members edit prompts and need history/diffs, when you need evidence-based prompt choice for production rollout, or when you want consistent prompt governance across environments.

Core Capabilities

  • A/B prompt evaluation against structured test cases
  • Quantitative scoring for adherence, relevance, and safety checks
  • Prompt version tracking with immutable history and changelog
  • Prompt diffs to review behavior-impacting edits
  • Reusable prompt templates and selection guidance
  • Regression-friendly workflows for model/prompt updates

Key Workflows

1. Run Prompt A/B Test

Prepare JSON test cases and run:

python3 scripts/prompt_tester.py \
  --prompt-a-file prompts/a.txt \
  --prompt-b-file prompts/b.txt \
  --cases-file testcases.json \
  --runner-cmd 'my-llm-cli --prompt {prompt} --input {input}' \
  --format text

Input can also come from stdin/--input JSON payload.

2. Choose Winner With Evidence

The tester scores outputs per case and aggregates:

  • expected content coverage
  • forbidden content violations
  • regex/format compliance
  • output length sanity

Use the higher-scoring prompt as candidate baseline, then run regression suite.

3. Version Prompts

# Add version
python3 scripts/prompt_versioner.py add \
  --name support_classifier \
  --prompt-file prompts/support_v3.txt \
  --author alice

# Diff versions
python3 scripts/prompt_versioner.py diff --name support_classifier --from-version 2 --to-version 3

# Changelog
python3 scripts/prompt_versioner.py changelog --name support_classifier

4. Regression Loop

  1. Store baseline version.
  2. Propose prompt edits.
  3. Re-run A/B test.
  4. Promote only if score and safety constraints improve.

Script Interfaces

  • python3 scripts/prompt_tester.py --help
    • Reads prompts/cases from stdin or --input
    • Optional external runner command
    • Emits text or JSON metrics
  • python3 scripts/prompt_versioner.py --help
    • Manages prompt history (add, list, diff, changelog)
    • Stores metadata and content snapshots locally

Pitfalls, Best Practices & Review Checklist

Avoid these mistakes:

  1. Picking prompts from single-case outputs — use a realistic, edge-case-rich test suite.
  2. Changing prompt and model simultaneously — always isolate variables.
  3. Missing must_not_contain (forbidden-content) checks in evaluation criteria.
  4. Editing prompts without version metadata, author, or change rationale.
  5. Skipping semantic diffs before deploying a new prompt version.
  6. Optimizing one benchmark while harming edge cases — track the full suite.
  7. Model swap without rerunning the baseline A/B suite.

Before promoting any prompt, confirm:

  • [ ] Task intent is explicit and unambiguous.
  • [ ] Output schema/format is explicit.
  • [ ] Safety and exclusion constraints are explicit.
  • [ ] No contradictory instructions.
  • [ ] No unnecessary verbosity tokens.
  • [ ] A/B score improves and violation count stays at zero.

References

  • references/prompt-templates.md — 6 production marketing templates (ad copy, email sequence, social repurposing, landing sections, SEO meta, brand-voice rewrite) plus generic building blocks; each written to be graded by prompt_tester.py
  • references/technique-guide.md — technique-selection table for marketing tasks + the LLM-governance stack for marketing teams (claim discipline, disclosure rules, data boundaries, human-review gates)
  • references/evaluation-rubric.md — mechanical scoring weights, acceptance gates, marketing quality dimensions, test-suite design, and eval anti-patterns
  • README.md

Evaluation Design

Each test case should define:

  • input: realistic production-like input
  • expected_contains: required markers/content
  • forbidden_contains: disallowed phrases or unsafe content
  • expected_regex: required structural patterns

This enables deterministic grading across prompt variants.

Versioning Policy

  • Use semantic prompt identifiers per feature (support_classifier, ad_copy_shortform).
  • Record author + change note for every revision.
  • Never overwrite historical versions.
  • Diff before promoting a new prompt to production.

Rollout Strategy

  1. Create baseline prompt version.
  2. Propose candidate prompt.
  3. Run A/B suite against same cases.
  4. Promote only if winner improves average and keeps violation count at zero.
  5. Track post-release feedback and feed new failure cases back into test suite.

04 同仓库更多 Skills

Prudhviraj101
Prudhviraj101/X_Agent

webapp-testing

Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.

★ 0
Prudhviraj101
Prudhviraj101/X_Agent

web-artifacts-builder

Suite of tools for creating elaborate, multi-component claude.ai HTML artifacts using modern frontend web technologies (React, Tailwind CSS, shadcn/ui). Use for complex artifacts requiring state management, routing, or shadcn/ui components - not for simple single-file HTML/JSX artifacts.

★ 0
Prudhviraj101
Prudhviraj101/X_Agent

theme-factory

Toolkit for styling artifacts with a theme. These artifacts can be slides, docs, reportings, HTML landing pages, etc. There are 10 pre-set themes with colors/fonts that you can apply to any artifact that has been creating, or can generate a new theme on-the-fly.

★ 0
Prudhviraj101
Prudhviraj101/X_Agent

systematic-planning

Formulate crisp, executable multi-phase implementation plans for non-trivial features, refactors, and architectural migrations. Use before writing code on any task with multiple files, uncertain requirements, or breaking changes.

★ 0
Prudhviraj101
Prudhviraj101/X_Agent

slack-gif-creator

Knowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."

★ 0
Prudhviraj101
Prudhviraj101/X_Agent

skill-creator

Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.

★ 0