skill-comply
Visualize whether skills, rules, and agent definitions are actually followed — auto-generates scenarios at 3 prompt strictness levels, runs agents, classifies behavioral sequences, and reports compliance rates with full tool call timelines. Use when checking whether agents actually follow the skills, rules, and definitions they were given, rather than assuming they do.
- License 许可
- MIT
- 最近更新
- 2026-11-04
- Stars
- ★ 2
- Forks
- 0
01 前言 Front Matter
- name
- skill-comply
- tools
- Read, Bash
- metadata
- { "origin": "ECC" }
- description
- Visualize whether skills, rules, and agent definitions are actually followed — auto-generates scenarios at 3 prompt strictness levels, runs agents, classifies behavioral sequences, and reports compliance rates with full tool call timelines. Use when checking whether agents actually follow the skills, rules, and definitions they were given, rather than assuming they do.
02 资源文件 21
- 其他.gitignore76 B
- 其他fixtures/compliant_trace.jsonl1.0 KB
- 其他fixtures/noncompliant_trace.jsonl629 B
- 参考文档fixtures/tdd_spec.yaml1.2 KB
- 参考文档prompts/classifier.md1.0 KB
- 参考文档prompts/scenario_generator.md1.9 KB
- 参考文档prompts/spec_generator.md1.6 KB
- 其他pyproject.toml299 B
- 脚本scripts/__init__.py.txt—
- 脚本scripts/classifier.py.txt2.4 KB
- 脚本scripts/grader.py.txt4.3 KB
- 脚本scripts/parser.py.txt2.7 KB
- 脚本scripts/report.py.txt6.3 KB
- 脚本scripts/run.py.txt4.2 KB
- 脚本scripts/runner.py.txt6.9 KB
- 脚本scripts/scenario_generator.py.txt1.8 KB
- 脚本scripts/spec_generator.py.txt2.2 KB
- 脚本scripts/utils.py.txt411 B
- 参考文档tests/test_grader.py.txt8.0 KB
- 参考文档tests/test_parser.py.txt3.1 KB
- 参考文档tests/test_runner.py.txt6.5 KB
03 SKILL.md 正文
skill-comply: Automated Compliance Measurement
Measures whether coding agents actually follow skills, rules, or agent definitions by:
- Auto-generating expected behavioral sequences (specs) from any .md file
- Auto-generating scenarios with decreasing prompt strictness (supportive → neutral → competing)
- Running
claude -pand capturing tool call traces via stream-json - Classifying tool calls against spec steps using LLM (not regex)
- Checking temporal ordering deterministically
- Generating self-contained reports with spec, prompts, and timelines
Supported Targets
- Skills (
skills/*/SKILL.md): Workflow skills like search-first, TDD guides - Rules (
rules/common/*.md): Mandatory rules like testing.md, security.md, git-workflow.md - Agent definitions (
agents/*.md): Whether an agent gets invoked when expected (internal workflow verification not yet supported)
When to Activate
- User runs
/skill-comply <path> - User asks "is this rule actually being followed?"
- After adding new rules/skills, to verify agent compliance
- Periodically as part of quality maintenance
Usage
# Full run
uv run python -m scripts.run ~/.claude/rules/common/testing.md
# Dry run (no cost, spec + scenarios only)
uv run python -m scripts.run --dry-run ~/.claude/skills/search-first/SKILL.md
# Custom models
uv run python -m scripts.run --gen-model haiku --model sonnet <path>
Key Concept: Prompt Independence
Measures whether a skill/rule is followed even when the prompt doesn't explicitly support it.
Report Contents
Reports are self-contained and include:
- Expected behavioral sequence (auto-generated spec)
- Scenario prompts (what was asked at each strictness level)
- Compliance scores per scenario
- Tool call timelines with LLM classification labels
Advanced (optional)
For users familiar with hooks, reports also include hook promotion recommendations for steps with low compliance. This is informational — the main value is the compliance visibility itself.
04 同仓库更多 Skills
webapp-testing
Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.
youtube-downloader
Download YouTube videos with customizable quality and format options. Use this skill when the user asks to download, save, or grab YouTube videos. Supports various quality settings (best, 1080p, 720p, 480p, 360p), multiple formats (mp4, webm, mkv), and audio-only downloads as MP3.
theme-factory
Toolkit for styling artifacts with a theme. These artifacts can be slides, docs, reportings, HTML landing pages, etc. There are 10 pre-set themes with colors/fonts that you can apply to any artifact that has been creating, or can generate a new theme on-the-fly.
template-skill
Replace with description of the skill and when Claude should use it.
tailored-resume-generator
Analyzes job descriptions and generates tailored resumes that highlight relevant experience, skills, and achievements to maximize interview chances
slack-gif-creator
Toolkit for creating animated GIFs optimized for Slack, with validators for size constraints and composable animation primitives. This skill applies when users request animated GIFs or emoji animations for Slack from descriptions like "make me a GIF for Slack of X doing Y".