skill-harness
Create, test and improve Pi skills and optional ecosystem subagents. Use when the user describes a role they need, asks to create a skill, add evaluations, calibrate a judge, diagnose a skill score or compare models. Start from an idea or an existing skill; examples and specifications are not prerequisites.
- License 许可
- MIT
- 最近更新
- 2026-11-15
- Stars
- ★ 2
- Forks
- 0
01 前言 Front Matter
- name
- skill-harness
- version
- 0.35.6
- description
- Create, test and improve Pi skills and optional ecosystem subagents. Use when the user describes a role they need, asks to create a skill, add evaluations, calibrate a judge, diagnose a skill score or compare models. Start from an idea or an existing skill; examples and specifications are not prerequisites.
02 资源文件 485
- 其他.gitattributes292 B
- 参考文档.github/workflows/ci.yml6.9 KB
- 其他.gitignore822 B
- 其他.pi/pi-daddy/.gitignore137 B
- 参考文档.pi/pi-daddy/settings.json2.9 KB
- 参考文档action.yml835 B
- 参考文档AGENTS.md17.0 KB
- 资产文件assets/demo.gif481.4 KB
- 脚本assets/demo/demo.sh3.8 KB
- 资产文件assets/demo/README.md3.1 KB
- 脚本assets/demo/record.sh2.9 KB
- 脚本assets/demo/script2cast.mjs2.3 KB
- 脚本assets/report.grade.js7.2 KB
- 资产文件assets/report.template.html25.5 KB
- 脚本bin/skill-harness.js706 B
- 参考文档CHANGELOG.md26.7 KB
- 参考文档contracts/pi-daddy/ledger-record/v1/fixtures/capability-decision.json2.9 KB
- 参考文档contracts/pi-daddy/ledger-record/v1/fixtures/child-lifecycle.json2.5 KB
- 参考文档contracts/pi-daddy/ledger-record/v1/fixtures/episode-cost-gate.json363 B
- 参考文档contracts/pi-daddy/ledger-record/v1/fixtures/episode-outcome.json312 B
- 其他contracts/pi-daddy/ledger-record/v1/fixtures/ledger-record.jsonl8.0 KB
- 参考文档contracts/pi-daddy/ledger-record/v1/fixtures/session-config.json323 B
- 参考文档contracts/pi-daddy/ledger-record/v1/fixtures/workspace-lease.json1.5 KB
- 参考文档contracts/pi-daddy/ledger-record/v1/governance-event.schema.json29.7 KB
- 参考文档contracts/pi-daddy/ledger-record/v1/PINNED.json2.7 KB
- 参考文档contracts/pi-daddy/ledger-record/v1/README.md4.1 KB
- 参考文档contracts/pi-daddy/ledger-record/v1/record.schema.json2.0 KB
- 脚本contracts/pi-daddy/ledger-record/v1/record.ts12.7 KB
- 参考文档contracts/pi-daddy/ledger/v2/fixtures/capability-decision.json2.5 KB
- 参考文档contracts/pi-daddy/ledger/v2/fixtures/check-receipt.json1.5 KB
- 参考文档contracts/pi-daddy/ledger/v2/fixtures/child-lifecycle.json1.4 KB
- 参考文档contracts/pi-daddy/ledger/v2/fixtures/workspace-lease.json1.4 KB
- 参考文档contracts/pi-daddy/ledger/v2/ledger-event.schema.json13.9 KB
- 参考文档contracts/pi-daddy/ledger/v2/pi-daddy-README.md4.1 KB
- 参考文档contracts/pi-daddy/ledger/v2/PINNED.json2.2 KB
- 参考文档contracts/pi-daddy/ledger/v2/README.md6.0 KB
- 脚本contracts/pi-daddy/ledger/v2/refusals.ts2.1 KB
- 参考文档contracts/pi-daddy/ledger/v3/fixtures/capability-decision.json2.7 KB
- 参考文档contracts/pi-daddy/ledger/v3/fixtures/check-receipt.json1.6 KB
- 参考文档contracts/pi-daddy/ledger/v3/fixtures/child-lifecycle.json1.5 KB
- 参考文档contracts/pi-daddy/ledger/v3/fixtures/workflow-fact.json1.4 KB
- 参考文档contracts/pi-daddy/ledger/v3/fixtures/workspace-lease.json1.4 KB
- 参考文档contracts/pi-daddy/ledger/v3/ledger-event.schema.json21.9 KB
- 参考文档contracts/pi-daddy/ledger/v3/pi-daddy-README.md2.8 KB
- 参考文档contracts/pi-daddy/ledger/v3/PINNED.json2.9 KB
- 参考文档contracts/pi-daddy/ledger/v3/README.md1.9 KB
- 脚本contracts/pi-daddy/ledger/v3/refusals.ts2.2 KB
- 参考文档docs/ASSURANCE-WORKFLOWS.md1.8 KB
- 参考文档docs/DECISION-LEARNING.md34.1 KB
- 参考文档docs/EVAL-AUTHORING.md31.3 KB
- 参考文档docs/JUDGE-EVIDENCE.md3.2 KB
- 参考文档docs/MODEL-PROFILES.md5.9 KB
- 参考文档docs/NEXT-SESSION.md18.5 KB
- 参考文档docs/PI-DADDY-RECORD-COMPATIBILITY.md3.6 KB
- 参考文档docs/PI-OBSERVATIONS.md3.9 KB
- 参考文档docs/RESULTS-SCHEMA-V3.md2.3 KB
- 参考文档docs/ROADMAP.md12.1 KB
- 参考文档docs/USAGE.md31.8 KB
- 参考文档docs/VALIDATION-INDEX.md5.9 KB
- 参考文档examples/portable-roles/agents/artifact-review-agent.md348 B
- 参考文档examples/portable-roles/agents/interface-designer-agent.md441 B
- 参考文档examples/portable-roles/agents/meeting-secretary-agent.md438 B
- 参考文档examples/portable-roles/agents/sales-adviser-agent.md426 B
- 脚本examples/portable-roles/connectors/calendar-fixture.mjs2.9 KB
- 脚本examples/portable-roles/connectors/calendar-fixture.test.mjs2.7 KB
- 参考文档examples/portable-roles/connectors/README.md1.9 KB
- 参考文档examples/portable-roles/continuity-evals/artifact-review/tests/calibration-v2.json4.3 KB
- 参考文档examples/portable-roles/continuity-evals/artifact-review/tests/specification.yaml1.8 KB
- 参考文档examples/portable-roles/continuity-evals/meeting-secretary/tests/calibration-v2.json13.1 KB
- 参考文档examples/portable-roles/continuity-evals/meeting-secretary/tests/specification.yaml7.4 KB
- 参考文档examples/portable-roles/evals/artifact-review/tests/calibration.json8.3 KB
- 参考文档examples/portable-roles/evals/artifact-review/tests/specification.yaml5.6 KB
- 参考文档examples/portable-roles/evals/interface-designer/tests/calibration-v2.json7.1 KB
- 参考文档examples/portable-roles/evals/interface-designer/tests/specification.yaml3.2 KB
- 参考文档examples/portable-roles/evals/meeting-secretary/tests/calibration-v2.json6.2 KB
- 参考文档examples/portable-roles/evals/meeting-secretary/tests/specification.yaml3.3 KB
- 参考文档examples/portable-roles/evals/sales-adviser/tests/calibration-v3.json6.2 KB
- 参考文档examples/portable-roles/evals/sales-adviser/tests/specification.yaml3.3 KB
- 参考文档examples/portable-roles/package.json329 B
- 参考文档examples/portable-roles/README.md942 B
- 参考文档examples/product-development/service-desk/acceptance.json2.3 KB
- 参考文档examples/product-development/service-desk/brief.txt2.4 KB
- 参考文档examples/product-development/service-desk/follow-up.txt1.5 KB
- 脚本examples/product-development/service-desk/prepare.mjs4.7 KB
- 脚本examples/product-development/service-desk/prepare.test.mjs4.1 KB
- 参考文档examples/product-development/service-desk/README.md3.5 KB
- 参考文档examples/tests/arms.yaml706 B
- 脚本experiments/decision-shadow/cli.mjs365 B
- 脚本experiments/decision-shadow/cli.test.ts16.3 KB
- 脚本experiments/decision-shadow/collections.mjs6.9 KB
- 脚本experiments/decision-shadow/collections.test.ts7.1 KB
- 脚本experiments/decision-shadow/consent.mjs2.8 KB
- 脚本experiments/decision-shadow/consent.test.ts4.6 KB
- 脚本experiments/decision-shadow/data-root.mjs2.7 KB
- 脚本experiments/decision-shadow/data-root.test.ts3.7 KB
- 脚本experiments/decision-shadow/dataset.mjs9.2 KB
- 脚本experiments/decision-shadow/dataset.test.ts11.2 KB
- 参考文档experiments/decision-shadow/examples/cases.json6.6 KB
- 参考文档experiments/decision-shadow/examples/labels.json2.7 KB
- 脚本experiments/decision-shadow/learning-data.mjs19.1 KB
- 脚本experiments/decision-shadow/learning-data.test.ts21.5 KB
- 脚本experiments/decision-shadow/learning-fixtures.mjs8.9 KB
- 脚本experiments/decision-shadow/learning-fixtures.test.ts5.3 KB
- 脚本experiments/decision-shadow/learning-training.mjs17.8 KB
- 参考文档experiments/decision-shadow/LEARNING-WORKFLOW.md11.5 KB
- 脚本experiments/decision-shadow/main.mjs18.8 KB
- 脚本experiments/decision-shadow/native-cli.test.ts7.8 KB
- 脚本experiments/decision-shadow/native-jev.mjs7.2 KB
- 脚本experiments/decision-shadow/native-jev.test.ts5.3 KB
- 脚本experiments/decision-shadow/providers.mjs7.3 KB
- 脚本experiments/decision-shadow/providers.test.ts9.2 KB
- 脚本experiments/decision-shadow/public-evidence-cli.test.ts5.8 KB
- 脚本experiments/decision-shadow/public-evidence-fixture.mjs7.2 KB
- 脚本experiments/decision-shadow/public-evidence.mjs25.8 KB
- 脚本experiments/decision-shadow/public-evidence.test.ts17.1 KB
- 参考文档experiments/decision-shadow/README.md14.2 KB
- 脚本experiments/decision-shadow/research-commands.mjs5.6 KB
- 脚本experiments/decision-shadow/research.mjs14.1 KB
- 脚本experiments/decision-shadow/research.test.ts10.5 KB
- 脚本experiments/decision-shadow/workflow-fixtures.mjs5.6 KB
- 脚本experiments/decision-shadow/workflow-import-cli.test.ts9.4 KB
- 脚本experiments/decision-shadow/workflow-import.mjs13.7 KB
- 脚本experiments/decision-shadow/workflow-import.test.ts14.0 KB
- 许可证LICENSE1.0 KB
- 参考文档package-lock.json69.2 KB
- 参考文档package.json1.5 KB
- 参考文档packages/adapters/package.json921 B
- 参考文档packages/adapters/prices/model-prices.json1.7 KB
- 参考文档packages/adapters/README.md566 B
- 脚本packages/adapters/src/closed-schema.ts23.5 KB
- 脚本packages/adapters/src/index.ts730 B
- 脚本packages/adapters/src/model-pricing.ts2.9 KB
- 脚本packages/adapters/src/pi-daddy-ledger-v2.ts15.2 KB
- 脚本packages/adapters/src/pi-daddy-ledger-v3.ts22.4 KB
- 脚本packages/adapters/src/pi-daddy-record-v1-contract.ts32.0 KB
- 脚本packages/adapters/src/pi-daddy-record-v1.ts7.9 KB
- 脚本packages/adapters/src/pi-decision-worker.ts4.5 KB
- 脚本packages/adapters/src/pi-decision.ts9.9 KB
- 脚本packages/adapters/src/pi-json.ts6.3 KB
- 脚本packages/adapters/src/pi.ts16.5 KB
- 脚本packages/adapters/src/trajectory.ts76.4 KB
- 脚本packages/adapters/test/fixtures/fake-subagent-extension.ts2.1 KB
- 其他packages/adapters/test/fixtures/governance/pi-daddy-capability-escalation.jsonl526 B
- 其他packages/adapters/test/fixtures/governance/pi-daddy-gated-unapproved.jsonl499 B
- 其他packages/adapters/test/fixtures/governance/pi-daddy-undeclared-tools.jsonl465 B
- 其他packages/adapters/test/fixtures/governance/pi-daddy-unsupported-version.jsonl179 B
- 其他packages/adapters/test/fixtures/governance/pi-daddy-v2-missing-join.jsonl174 B
- 其他packages/adapters/test/fixtures/governance/pi-daddy-v2-positive.jsonl7.2 KB
- 参考文档packages/adapters/test/fixtures/governance/PI-DADDY-V2-PROVENANCE.md1.9 KB
- 其他packages/adapters/test/fixtures/governance/pi-daddy-v2-read-lease.jsonl342 B
- 其他packages/adapters/test/fixtures/governance/pi-daddy-v2-refusal-stale.jsonl1.7 KB
- 其他packages/adapters/test/fixtures/governance/pi-daddy-v2-workspace-not-authorized.jsonl1.8 KB
- 其他packages/adapters/test/fixtures/governance/principal-assurance-v1.jsonl2.0 KB
- 其他packages/adapters/test/fixtures/pi-1.0.4/error.jsonl8.5 KB
- 参考文档packages/adapters/test/fixtures/pi-1.0.4/final-conformance.json24.5 KB
- 其他packages/adapters/test/fixtures/pi-1.0.4/nested.jsonl12.8 KB
- 参考文档packages/adapters/test/fixtures/pi-1.0.4/provenance.json4.4 KB
- 其他packages/adapters/test/fixtures/pi-1.0.4/retry.jsonl10.4 KB
- 其他packages/adapters/test/fixtures/pi-1.0.4/stop-then-error.jsonl11.2 KB
- 其他packages/adapters/test/fixtures/pi-1.0.4/success.jsonl8.5 KB
- 参考文档packages/adapters/test/fixtures/pi-1.0.4/vendor-source.json474 B
- 脚本packages/adapters/test/fixtures/pi-1.1.0/capture.mjs14.6 KB
- 参考文档packages/adapters/test/fixtures/pi-1.1.0/pi104-current-qualification.json2.1 KB
- 参考文档packages/adapters/test/fixtures/pi-1.1.0/pi110-current-qualification.json2.1 KB
- 参考文档packages/adapters/test/fixtures/pi-1.1.0/provenance.json2.4 KB
- 参考文档packages/adapters/test/fixtures/pi-1.1.0/README.md2.5 KB
- 其他packages/adapters/test/fixtures/pi-1.1.0/success.jsonl25.0 KB
- 参考文档packages/adapters/test/fixtures/pi-1.4.0/adapter-multi-turn.result.json2.5 KB
- 参考文档packages/adapters/test/fixtures/pi-1.4.0/adapter-network.json5.2 KB
- 参考文档packages/adapters/test/fixtures/pi-1.4.0/adapter-pipeline.json358 B
- 参考文档packages/adapters/test/fixtures/pi-1.4.0/adapter-processes.json378 B
- 参考文档packages/adapters/test/fixtures/pi-1.4.0/adapter-qualification.json2.8 KB
- 脚本packages/adapters/test/fixtures/pi-1.4.0/capture.mjs18.3 KB
- 其他packages/adapters/test/fixtures/pi-1.4.0/force-transport.jsonl16.7 KB
- 其他packages/adapters/test/fixtures/pi-1.4.0/green-transport.jsonl18.0 KB
- 参考文档packages/adapters/test/fixtures/pi-1.4.0/native-network.json2.4 KB
- 参考文档packages/adapters/test/fixtures/pi-1.4.0/native-processes.json1.4 KB
- 参考文档packages/adapters/test/fixtures/pi-1.4.0/native-qualification.json6.1 KB
- 脚本packages/adapters/test/fixtures/pi-1.4.0/network-guard.mjs2.3 KB
- 参考文档packages/adapters/test/fixtures/pi-1.4.0/provenance.json5.8 KB
- 参考文档packages/adapters/test/fixtures/pi-1.4.0/README.md2.5 KB
- 其他packages/adapters/test/fixtures/pi-1.4.0/red-denied-tool.jsonl16.2 KB
- 其他packages/adapters/test/fixtures/pi-1.4.0/red-extension-filtered.jsonl16.0 KB
- 其他packages/adapters/test/fixtures/pi-1.4.0/red-extension-ok.jsonl16.3 KB
- 其他packages/adapters/test/fixtures/pi-1.4.0/red-length.jsonl12.1 KB
- 其他packages/adapters/test/fixtures/pi-1.4.0/red-mcp-denied.jsonl16.0 KB
- 其他packages/adapters/test/fixtures/pi-1.4.0/red-provider-error.jsonl11.4 KB
- 其他packages/adapters/test/fixtures/pi-1.4.0/red-tool-error.jsonl16.6 KB
- 其他packages/adapters/test/fixtures/pi-1.4.0/red-transport.jsonl16.1 KB
- 参考文档packages/adapters/test/fixtures/pi-1.5.0/adapter-multi-turn.result.json2.5 KB
- 参考文档packages/adapters/test/fixtures/pi-1.5.0/adapter-network.json6.0 KB
- 参考文档packages/adapters/test/fixtures/pi-1.5.0/adapter-pipeline.json358 B
- 参考文档packages/adapters/test/fixtures/pi-1.5.0/adapter-processes.json383 B
- 参考文档packages/adapters/test/fixtures/pi-1.5.0/adapter-qualification.json3.1 KB
- 脚本packages/adapters/test/fixtures/pi-1.5.0/capture.mjs19.3 KB
- 其他packages/adapters/test/fixtures/pi-1.5.0/force-transport.jsonl16.2 KB
- 其他packages/adapters/test/fixtures/pi-1.5.0/green-transport.jsonl17.3 KB
- 参考文档packages/adapters/test/fixtures/pi-1.5.0/native-network.json3.0 KB
- 参考文档packages/adapters/test/fixtures/pi-1.5.0/native-processes.json1.6 KB
- 参考文档packages/adapters/test/fixtures/pi-1.5.0/native-qualification.json7.1 KB
- 脚本packages/adapters/test/fixtures/pi-1.5.0/network-guard.mjs2.3 KB
- 参考文档packages/adapters/test/fixtures/pi-1.5.0/provenance.json5.9 KB
- 参考文档packages/adapters/test/fixtures/pi-1.5.0/README.md1.5 KB
- 其他packages/adapters/test/fixtures/pi-1.5.0/red-denied-tool.jsonl15.6 KB
- 其他packages/adapters/test/fixtures/pi-1.5.0/red-extension-filtered.jsonl15.5 KB
- 其他packages/adapters/test/fixtures/pi-1.5.0/red-extension-ok.jsonl15.8 KB
- 其他packages/adapters/test/fixtures/pi-1.5.0/red-length.jsonl11.6 KB
- 其他packages/adapters/test/fixtures/pi-1.5.0/red-mcp-denied.jsonl15.5 KB
- 其他packages/adapters/test/fixtures/pi-1.5.0/red-provider-error.jsonl10.9 KB
- 其他packages/adapters/test/fixtures/pi-1.5.0/red-schema-invalid.jsonl17.1 KB
- 其他packages/adapters/test/fixtures/pi-1.5.0/red-schema-valid.jsonl16.3 KB
- 其他packages/adapters/test/fixtures/pi-1.5.0/red-tool-error.jsonl16.1 KB
- 其他packages/adapters/test/fixtures/pi-1.5.0/red-transport.jsonl15.6 KB
- 参考文档packages/adapters/test/fixtures/pi-1.5.2/adapter-multi-turn.result.json3.8 KB
- 参考文档packages/adapters/test/fixtures/pi-1.5.2/adapter-network.json6.1 KB
- 参考文档packages/adapters/test/fixtures/pi-1.5.2/adapter-pipeline.json358 B
- 参考文档packages/adapters/test/fixtures/pi-1.5.2/adapter-processes.json380 B
- 参考文档packages/adapters/test/fixtures/pi-1.5.2/adapter-qualification.json3.1 KB
- 其他packages/adapters/test/fixtures/pi-1.5.2/force-transport.jsonl15.8 KB
- 其他packages/adapters/test/fixtures/pi-1.5.2/green-transport.jsonl16.8 KB
- 参考文档packages/adapters/test/fixtures/pi-1.5.2/native-network.json3.0 KB
- 参考文档packages/adapters/test/fixtures/pi-1.5.2/native-processes.json1.6 KB
- 参考文档packages/adapters/test/fixtures/pi-1.5.2/native-qualification.json7.2 KB
- 参考文档packages/adapters/test/fixtures/pi-1.5.2/provenance.json5.8 KB
- 参考文档packages/adapters/test/fixtures/pi-1.5.2/README.md1.7 KB
- 其他packages/adapters/test/fixtures/pi-1.5.2/red-denied-tool.jsonl15.2 KB
- 其他packages/adapters/test/fixtures/pi-1.5.2/red-extension-filtered.jsonl15.0 KB
- 其他packages/adapters/test/fixtures/pi-1.5.2/red-extension-ok.jsonl15.4 KB
- 其他packages/adapters/test/fixtures/pi-1.5.2/red-length.jsonl11.2 KB
- 其他packages/adapters/test/fixtures/pi-1.5.2/red-mcp-denied.jsonl15.0 KB
- 其他packages/adapters/test/fixtures/pi-1.5.2/red-provider-error.jsonl10.5 KB
- 其他packages/adapters/test/fixtures/pi-1.5.2/red-schema-invalid.jsonl16.7 KB
- 其他packages/adapters/test/fixtures/pi-1.5.2/red-schema-valid.jsonl15.9 KB
- 其他packages/adapters/test/fixtures/pi-1.5.2/red-tool-error.jsonl15.5 KB
- 其他packages/adapters/test/fixtures/pi-1.5.2/red-transport.jsonl15.1 KB
- 其他packages/adapters/test/fixtures/pi-json/multi-turn-turn1.jsonl3.6 KB
- 其他packages/adapters/test/fixtures/pi-json/multi-turn-turn2.jsonl3.6 KB
- 其他packages/adapters/test/fixtures/pi-json/parallel-out-of-order.jsonl12.0 KB
- 参考文档packages/adapters/test/fixtures/pi-json/parity-print-mode.txt5 B
- 参考文档packages/adapters/test/fixtures/pi-json/README.md3.5 KB
- 其他packages/adapters/test/fixtures/pi-json/sequential-tool-calls.jsonl59.1 KB
- 其他packages/adapters/test/fixtures/pi-json/single-turn.jsonl5.1 KB
- 其他packages/adapters/test/fixtures/pi-json/subagent-call.jsonl9.0 KB
- 其他packages/adapters/test/fixtures/pi-json/tool-call.jsonl8.4 KB
- 其他packages/adapters/test/fixtures/pi-json/tool-error.jsonl7.2 KB
- 脚本packages/adapters/test/model-pricing.test.ts5.5 KB
- 脚本packages/adapters/test/pi-1.0.4-contract.test.ts5.0 KB
- 脚本packages/adapters/test/pi-1.0.4-outcome.test.ts10.2 KB
- 脚本packages/adapters/test/pi-1.1.0-contract.test.ts2.3 KB
- 脚本packages/adapters/test/pi-1.4.0-contract.test.ts5.4 KB
- 脚本packages/adapters/test/pi-1.5-contract.test.ts6.4 KB
- 脚本packages/adapters/test/pi-daddy-contract.test.ts33.1 KB
- 脚本packages/adapters/test/pi-daddy-record-v1.test.ts15.5 KB
- 脚本packages/adapters/test/pi-daddy-v3-contract.test.ts11.3 KB
- 脚本packages/adapters/test/pi-decision.test.ts5.2 KB
- 脚本packages/adapters/test/pi-json-contract.test.ts11.2 KB
- 脚本packages/adapters/test/pi-json.test.ts11.5 KB
- 脚本packages/adapters/test/pi-profile.test.ts1.4 KB
- 脚本packages/adapters/test/pi-structured.test.ts8.2 KB
- 脚本packages/adapters/test/pi.test.ts12.6 KB
- 脚本packages/adapters/test/trajectory-adapter.test.ts34.7 KB
- 参考文档packages/adapters/tsconfig.json176 B
- 参考文档packages/cli/package.json994 B
- 参考文档packages/cli/README.md4.3 KB
- 脚本packages/cli/src/cli.ts47.2 KB
- 脚本packages/cli/src/decision.ts323 B
- 脚本packages/cli/src/diagnose-format.ts2.5 KB
- 脚本packages/cli/src/serve.ts15.5 KB
- 脚本packages/cli/test/authoring-cmd.test.ts6.9 KB
- 脚本packages/cli/test/coverage-cmd.test.ts4.2 KB
- 脚本packages/cli/test/grade-cmd.test.ts12.3 KB
- 脚本packages/cli/test/grade-selection.test.ts5.4 KB
- 脚本packages/cli/test/init-cmd.test.ts2.2 KB
- 脚本packages/cli/test/lint-cmd.test.ts5.3 KB
- 脚本packages/cli/test/regressions.test.ts695 B
- 脚本packages/cli/test/release-pack.test.ts35.6 KB
- 脚本packages/cli/test/run-tuning.test.ts3.7 KB
- 脚本packages/cli/test/serve.test.ts25.0 KB
- 脚本packages/cli/test/smoke-script-contract.test.ts2.6 KB
- 脚本packages/cli/test/specs-overlay.test.ts8.3 KB
- 脚本packages/cli/test/stability-cmd.test.ts5.9 KB
- 脚本packages/cli/test/suggest-cmd.test.ts5.8 KB
- 脚本packages/cli/test/test-skip-contract.test.ts2.9 KB
- 脚本packages/cli/test/version.test.ts7.0 KB
- 参考文档packages/cli/tsconfig.json203 B
- 参考文档packages/core/package.json947 B
- 参考文档packages/core/README.md1.1 KB
- 脚本packages/core/src/adapters/types.ts7.8 KB
- 脚本packages/core/src/arms.ts14.1 KB
- 脚本packages/core/src/authoring-contract-review.ts25.3 KB
- 脚本packages/core/src/authoring-preflight.ts8.1 KB
- 脚本packages/core/src/authoring.ts25.3 KB
- 脚本packages/core/src/calibration.ts28.2 KB
- 脚本packages/core/src/canary.ts8.7 KB
- 脚本packages/core/src/capture-trace-types.ts7.3 KB
- 脚本packages/core/src/command-cost.ts638 B
- 脚本packages/core/src/criterion-dimensions.ts12.6 KB
- 脚本packages/core/src/defaults.ts2.0 KB
- 脚本packages/core/src/diagnose.ts5.6 KB
- 脚本packages/core/src/discover.ts3.6 KB
- 脚本packages/core/src/downgrade.ts4.4 KB
- 脚本packages/core/src/execution-trace.ts25.1 KB
- 脚本packages/core/src/grade.ts17.3 KB
- 脚本packages/core/src/improvement.ts8.9 KB
- 脚本packages/core/src/index.ts1.8 KB
- 脚本packages/core/src/instruction-coverage.ts12.0 KB
- 脚本packages/core/src/journal.ts4.5 KB
- 脚本packages/core/src/judge-evidence.ts9.1 KB
- 脚本packages/core/src/judge-policy.ts3.8 KB
- 脚本packages/core/src/judge-references.ts1.8 KB
- 脚本packages/core/src/lift.ts16.2 KB
- 脚本packages/core/src/lint.ts21.4 KB
- 脚本packages/core/src/metrics.ts2.8 KB
- 脚本packages/core/src/prompt-normalization.ts677 B
- 脚本packages/core/src/provider-failure.ts5.7 KB
- 脚本packages/core/src/redaction.ts2.2 KB
- 脚本packages/core/src/regate.ts25.0 KB
- 脚本packages/core/src/regrade.ts24.7 KB
- 脚本packages/core/src/report.ts9.2 KB
- 脚本packages/core/src/reps.ts11.3 KB
- 脚本packages/core/src/rescore.ts7.5 KB
- 脚本packages/core/src/resource-monitor.ts2.7 KB
- 脚本packages/core/src/restamp.ts12.0 KB
- 脚本packages/core/src/results.ts57.3 KB
- 脚本packages/core/src/run-summary.ts8.8 KB
- 脚本packages/core/src/run.ts40.7 KB
- 脚本packages/core/src/sandbox.ts1.4 KB
- 脚本packages/core/src/saved-run.ts7.9 KB
- 脚本packages/core/src/scaffold.ts7.7 KB
- 脚本packages/core/src/scheduler.ts1.2 KB
- 脚本packages/core/src/score.ts4.1 KB
- 脚本packages/core/src/seeded.ts27.4 KB
- 脚本packages/core/src/sources.ts34.6 KB
- 脚本packages/core/src/spec-write.ts5.0 KB
- 脚本packages/core/src/spec.ts22.3 KB
- 脚本packages/core/src/stability.ts17.2 KB
- 脚本packages/core/src/trace-gates.ts26.4 KB
- 脚本packages/core/src/trajectory-events.ts7.6 KB
- 脚本packages/core/src/trends.ts7.8 KB
- 脚本packages/core/src/util/env.ts2.6 KB
- 脚本packages/core/src/util/exec.ts1.8 KB
- 脚本packages/core/src/util/fixture-inventory.ts3.2 KB
- 脚本packages/core/src/util/process.ts16.1 KB
- 脚本packages/core/src/version.ts1.2 KB
- 脚本packages/core/src/vote-panel.ts2.5 KB
- 脚本packages/core/src/workspace.ts14.2 KB
- 脚本packages/core/test/arm-lint-lineage.test.ts4.9 KB
- 脚本packages/core/test/arm-run-integration.test.ts14.0 KB
- 脚本packages/core/test/arms.test.ts18.1 KB
- 脚本packages/core/test/authoring-contract-review.test.ts19.3 KB
- 脚本packages/core/test/authoring-preflight.test.ts8.5 KB
- 脚本packages/core/test/authoring.test.ts14.2 KB
- 脚本packages/core/test/calibration.test.ts29.4 KB
- 脚本packages/core/test/canary.test.ts18.7 KB
- 脚本packages/core/test/ci-workspace-isolation.test.ts1.7 KB
- 脚本packages/core/test/command-cost.test.ts1.8 KB
- 脚本packages/core/test/criterion-dimensions.test.ts13.2 KB
- 脚本packages/core/test/delivery-gate.test.ts3.0 KB
- 脚本packages/core/test/diagnose.test.ts2.3 KB
- 脚本packages/core/test/discover.test.ts1.9 KB
- 脚本packages/core/test/doc-examples.test.ts3.4 KB
- 脚本packages/core/test/downgrade.test.ts4.2 KB
- 脚本packages/core/test/env.test.ts3.5 KB
- 脚本packages/core/test/execution-cost.test.ts4.6 KB
- 脚本packages/core/test/execution-trace.test.ts10.4 KB
- 脚本packages/core/test/field-roundtrip.test.ts17.3 KB
- 脚本packages/core/test/fixture-limits.test.ts3.5 KB
- 脚本packages/core/test/force-scoring.test.ts12.8 KB
- 脚本packages/core/test/golden-run.test.ts14.8 KB
- 脚本packages/core/test/grade-column-parity.test.ts5.4 KB
- 脚本packages/core/test/grade.test.ts15.3 KB
- 脚本packages/core/test/improvement.test.ts4.8 KB
- 脚本packages/core/test/instruction-coverage.test.ts10.3 KB
- 脚本packages/core/test/journal.test.ts2.3 KB
- 脚本packages/core/test/judge-context.test.ts8.7 KB
- 脚本packages/core/test/judge-evidence.test.ts8.8 KB
- 脚本packages/core/test/judge-policy.test.ts3.9 KB
- 脚本packages/core/test/judge-references.test.ts4.4 KB
- 脚本packages/core/test/lift-parity.test.ts5.9 KB
- 脚本packages/core/test/lift.test.ts21.2 KB
- 脚本packages/core/test/lint.test.ts17.4 KB
- 脚本packages/core/test/metrics.test.ts5.0 KB
- 脚本packages/core/test/model-profile.test.ts9.2 KB
- 脚本packages/core/test/observation.test.ts9.2 KB
- 脚本packages/core/test/orchestration.test.ts10.8 KB
- 脚本packages/core/test/output-gates.test.ts2.6 KB
- 脚本packages/core/test/p1-multipliers.test.ts25.9 KB
- 脚本packages/core/test/pi-settlement.test.ts5.9 KB
- 脚本packages/core/test/process.test.ts6.1 KB
- 脚本packages/core/test/provenance.test.ts5.3 KB
- 脚本packages/core/test/provider-failure-run.test.ts4.1 KB
- 脚本packages/core/test/provider-failure.test.ts3.7 KB
- 脚本packages/core/test/recorded-provenance.test.ts2.0 KB
- 脚本packages/core/test/redaction.test.ts1.6 KB
- 脚本packages/core/test/regate.test.ts19.1 KB
- 脚本packages/core/test/regrade.test.ts22.3 KB
- 脚本packages/core/test/regressions.test.ts2.1 KB
- 脚本packages/core/test/report.test.ts12.6 KB
- 脚本packages/core/test/reps.test.ts6.5 KB
- 脚本packages/core/test/rescore.test.ts5.2 KB
- 脚本packages/core/test/resource-monitor.test.ts4.3 KB
- 脚本packages/core/test/results.test.ts23.8 KB
- 脚本packages/core/test/review-regressions.test.ts15.4 KB
- 脚本packages/core/test/run-resilience.test.ts9.7 KB
- 脚本packages/core/test/sandbox.test.ts1.5 KB
- 脚本packages/core/test/saved-run.test.ts11.9 KB
- 脚本packages/core/test/scaffold.test.ts6.0 KB
- 脚本packages/core/test/scheduler.test.ts2.9 KB
- 脚本packages/core/test/schema-artifacts.test.ts4.9 KB
- 脚本packages/core/test/score.test.ts5.2 KB
- 脚本packages/core/test/seeded.test.ts33.8 KB
- 脚本packages/core/test/sources-frontmatter.test.ts18.8 KB
- 脚本packages/core/test/sources-split.test.ts9.1 KB
- 脚本packages/core/test/sources.test.ts10.1 KB
- 脚本packages/core/test/spec-write.test.ts4.7 KB
- 脚本packages/core/test/spec.test.ts15.5 KB
- 脚本packages/core/test/stability.test.ts18.9 KB
- 脚本packages/core/test/structured-flag.test.ts11.8 KB
- 脚本packages/core/test/thrown-adapter-regrade.test.ts5.6 KB
- 脚本packages/core/test/trace-gates.test.ts12.1 KB
- 脚本packages/core/test/trace-integration.test.ts11.9 KB
- 脚本packages/core/test/trends.test.ts10.3 KB
- 脚本packages/core/test/ungraded.test.ts13.2 KB
- 脚本packages/core/test/vote-panel.test.ts1.7 KB
- 脚本packages/core/test/workspace-exclude.test.ts6.4 KB
- 脚本packages/core/test/workspace.test.ts17.3 KB
- 参考文档packages/core/tsconfig.json133 B
- 脚本packages/pi-extension/build.mjs1.6 KB
- 参考文档packages/pi-extension/package.json515 B
- 脚本packages/pi-extension/src/authoring-reference.ts5.1 KB
- 脚本packages/pi-extension/src/authoring.ts36.2 KB
- 脚本packages/pi-extension/src/commands.ts12.9 KB
- 脚本packages/pi-extension/src/dashboard.ts5.9 KB
- 脚本packages/pi-extension/src/evidence-review-pattern.ts17.9 KB
- 脚本packages/pi-extension/src/improvement.ts6.4 KB
- 脚本packages/pi-extension/src/index.ts2.7 KB
- 脚本packages/pi-extension/src/jev-advice.ts9.3 KB
- 脚本packages/pi-extension/src/jev-candidate.ts1.8 KB
- 脚本packages/pi-extension/src/jev-control.ts3.8 KB
- 脚本packages/pi-extension/src/jev-evidence.ts2.6 KB
- 脚本packages/pi-extension/src/jev-packet.ts9.2 KB
- 脚本packages/pi-extension/src/jev-prompt-hints.ts10.2 KB
- 脚本packages/pi-extension/src/jev-session.ts37.6 KB
- 脚本packages/pi-extension/src/runner.ts5.7 KB
- 脚本packages/pi-extension/src/tool.ts4.2 KB
- 脚本packages/pi-extension/test/authoring-reference.test.ts7.4 KB
- 脚本packages/pi-extension/test/authoring.test.ts14.7 KB
- 脚本packages/pi-extension/test/bundle.test.ts2.3 KB
- 脚本packages/pi-extension/test/commands.test.ts12.5 KB
- 脚本packages/pi-extension/test/dashboard.test.ts5.1 KB
- 脚本packages/pi-extension/test/evidence-review-pattern.test.ts6.8 KB
- 脚本packages/pi-extension/test/improvement.test.ts3.8 KB
- 脚本packages/pi-extension/test/index.test.ts1.6 KB
- 脚本packages/pi-extension/test/installed-package.test.ts2.3 KB
- 脚本packages/pi-extension/test/jev-advice.test.ts66.5 KB
- 脚本packages/pi-extension/test/jev-control.test.ts5.8 KB
- 脚本packages/pi-extension/test/jev-prompt-hints.test.ts14.4 KB
- 脚本packages/pi-extension/test/jev-session.test.ts8.4 KB
- 脚本packages/pi-extension/test/runner.test.ts4.4 KB
- 脚本packages/pi-extension/test/tool.test.ts6.0 KB
- 脚本packages/pi-extension/test/version-lockstep.test.ts1.1 KB
- 参考文档packages/pi-extension/tsconfig.json297 B
- 参考文档PUBLISHING.md4.5 KB
- 参考文档README.md48.8 KB
- 参考文档schemas/results-v2.schema.json10.7 KB
- 参考文档schemas/results-v3.schema.json15.4 KB
- 参考文档schemas/specification-v1.schema.json5.5 KB
- 脚本scripts/build-decision.mjs261 B
- 脚本scripts/check-pi-daddy-contract.mjs8.2 KB
- 脚本scripts/check-pi-daddy-v3-contract.mjs3.5 KB
- 脚本scripts/check-recorded-provenance.mjs3.3 KB
- 脚本scripts/pi-qualification/capture.mjs19.6 KB
- 脚本scripts/release-pack.mjs41.6 KB
- 脚本scripts/smoke-packed-artifacts.sh827 B
- 脚本scripts/smoke-real-pi.sh5.4 KB
- 脚本scripts/smoke/.gitignore183 B
- 脚本scripts/vendor-pi-daddy-contract.mjs6.1 KB
- 脚本scripts/vendor-pi-daddy-ledger-v3-contract.mjs5.2 KB
- 脚本scripts/vendor-pi-daddy-record-v1-contract.mjs3.1 KB
- 脚本scripts/verify-pi-daddy-builders.mjs19.8 KB
- 脚本scripts/verify-pi-daddy-v3-builders.mjs11.4 KB
- 脚本scripts/vitest-temp-cleanup.mjs852 B
- 参考文档tsconfig.base.json398 B
- 参考文档tsconfig.json435 B
- 脚本vitest.config.ts1.6 KB
03 SKILL.md 正文
Develop skills in Pi
Use the current Pi session and selected model to turn a user's idea into a skill, its evaluations, and optionally a separate pi-daddy agent definition. The authoring workflow requires Pi and a model. Generated skills require Pi alone; do not make their instructions depend on skill-harness, pi-daddy or Principal.
From an idea to a tested skill
-
Capture the brief. Use
skill_authorto start or resume a named project. Record what the user already told you. Ask one necessary question at a time;questionasks the next missing answer in Pi. Establish purpose, observable outcomes, boundaries, tools, skill versus skill-and-agent delivery, and same-repo versus separate-repo evaluations. Do not ask for subject or judge models yet. -
Author the files. First call
skill_authorwith{ "action": "reference" }for the installed schema, generic example and file paths; no saved project is required. Do not guess the format or search other projects for it. Produce a conciseSKILL.md, useful supporting resources, and eval scenarios grounded in the agreed requirements. Create fixtures and references as needed; do not demand that the user supply examples or YAML. Stage throughskill_author draft, inspect its changes and issues, then apply the exact digest within the user's authorized scope. Import only an explicitly selected existing skill. Existing files and historical results are not disposable. For performance/reproduction or normal-path evidence tasks, the optionalreferencepatternevidence-reviewsupplies existing-format proposed controls; use it only when relevant. Correct arithmetic is not semantic approval. -
Check the evaluation itself. Run
lintandpreflightbefore model calls. Map requirements to observable scenarios both ways. Name source files when the subject cannot discover them with its tools. Useenv.toolsto state the tool profile andjudge_referencesfor explicit judge-only source files. Choosejudge_evidence: responsefor answer tasks; seeded mode is a workspace choice, not evidence that the task is code generation. -
Calibrate. After the valid draft is applied (the specification must exist), use
skill_author task-contractto read the saved brief, subject tasks and explicitly selected methods without preferred answers or labels. Record the actual obligations, distinguishing necessary implications from optional improvements. Prepare proposed controls withcontrols, then usecontract-reviewto justify each blocking criterion and select a minimally sufficient valid alternative. A source instruction or judge-only fact cannot silently add a user requirement. These are agent preparation steps: code checks identity and coverage, while semantic review checks authority and materiality.review-controlspresents one concise operator decision with full evidence available; reuse exact accepted reviews while their inputs remain current. Propose positive, negative and equivalent-valid controls grounded in the complete contract, with reasons. For new controls, proposeexpectedCriteria: one PASS/FAIL per checklist item in order, so an overall FAIL for the wrong reason does not count as a match. Keep expectations outside judge inputs. Have the author review them; your proposal is not human acceptance. Use the shared calibration tooling, which computes evidence identities and preserves the same transcript framing as real grading. Do not invent hashes or write bespoke judge scripts. Ask for the subject model when testing starts, then an independent judge if not already selected. Reuse authorization instead of asking repeatedly. -
Measure. Run a red baseline and a skill-enabled comparison through Pi with selected models. Green advertises the skill for on-demand loading; force supplies the instructions directly.
--canarychecks one explicitly requested load and quote; neither pass nor failure establishes autonomous selection. Preserve source identities, exact answers, trace gates and judge inputs. Critical/B-series failures block SHIP; counts of passing answers and fully passing scenarios are different measures. Small samples are not a stability guarantee. -
Diagnose before improving. Inspect
diagnoseand the retained evidence. Separate infrastructure errors, invalid evals, judge disagreements, model errors and skill weaknesses. Do not treat a score as its own explanation. For changed rubrics, regrade saved answers; for changed inputs or instructions, run fresh subjects. Followdiagnose's per-scenario reuse plan. Usegrade --onlywith a new--outdirectory to preserve the original; usecalibrate --plan --reuseto reuse exactly matching judgments before spending more calls. Reused records are historical evidence, not new stability samples. Use fresh transfer cases before claiming a general improvement.Keep task correctness separate from additional method completeness. Optional
criterion_dimensionsclassifies each checklist item astask-correctness,method-completeness, ormixedfor offline reporting. A combined item stays mixed: never invent separate votes from its reason text. The declared grouping changes no grade or acceptance rule; unmapped or unsupported evidence stays explicit. Report semantic audit observations separately from saved judge votes.
Keep evaluation launches sequential unless parallel work was deliberately selected.
After an interruption, inspect the bounded resources-*.json summaries before
restarting. They describe harness-owned processes and cleanup, not whole-PC health.
Do not rerun a complete comparison just because one control or rubric changed.
You may create and edit the skill when the user requests development. For a request only to evaluate an existing skill, measure it without rewriting it.
Learn from manual validation
Treat the whole meaningful manual check as one case, not each click or label review. For an already covered/adopted case, attach the outcome without another authoring task or adoption question. For a manual check of any ecosystem package, retain the selected evidence, identify the actual contract and owning component, and propose a small reproducible regression. Extend existing coverage instead of duplicating it; include useful positive, negative or transfer contrasts. Ask the user whether to adopt this evaluation contract; reuse explicit acceptance on unchanged reruns/resumes. Ask again only for new or materially changed outcomes. Link actual test/eval IDs and results; keep declined or uncovered items visible. Label acceptance and model-call authorization remain separate. This does not add approval requirements to routine implementation regression tests. Human usability stays a human procedure, not a claim that an automated test reproduced the experience.
Evaluation design
- Use mechanical gates for execution facts and semantic criteria for meaning.
require_callswithoutcome: successdistinguishes a completed read from a failed attempt; a read still does not prove correct interpretation. - Test user-visible behavior, errors, boundaries and valid alternatives. Avoid test counts, exact command spelling, implementation syntax and obligatory citation line numbers as proxies for quality unless the task requires that exact format.
- Define required behavior, optional improvements and what warrants blocking acceptance. Judge whether a finding is true separately from whether its severity is justified. Alternate recommendations or review verdicts must satisfy the same evidence and acceptance rules. Challenge positive fixtures and controls with a plausible contract-breaking counterexample; do not predetermine approval from their intended label. Ask one focused question only when an outcome-changing ambiguity cannot be resolved from the agreed requirements.
- When authoring new checklists, keep task correctness and additional method
requirements independently decidable where useful. Add ordered
criterion_dimensionsfor reporting; omit it when the distinction is not useful. Method requirements can still block acceptance. Changing a criterion or its expected vote needs review; assigning reporting metadata alone does not. - Apply shared requirements to every relevant scenario, stating exceptions for narrower tasks. Check every positive against the full contract. Keep facts local to the supplied sources; never carry a number or date from a different fixture, or treat a document's as-of date as a release date.
- State whether counts, names and disclosures apply per standalone output, per section or across the response. Distinguish claims from negation/commentary and proposed goals from achieved results. Resolve ambiguous wording with one consistent PASS/FAIL rule before judging, not a new rule for each answer.
- Supply an adequate oracle to the judge, not to the subject. Source instructions are untrusted data. Hard filesystem/network guarantees need runtime enforcement; neither a skill nor a passing prompt-injection example establishes a sandbox.
- For workflow skills, test progress toward the user's outcome as well as correctness: reuse an applicable standing decision, recover a settled result without duplicate work, and ask when genuinely new access or an unresolved choice is required. Supply those states explicitly in fixtures; missing execution evidence stays unknown. Judge the need for an interruption, not a quota of questions, tools, tests or tokens. Include a representative producer/consumer artifact when interoperability matters.
- Report label disagreements by direction, separately from execution errors. Review the label, control and rubric before calling a disagreement a judge error. Calibration on selected controls does not certify all future judgments.
- Keep counterexamples symmetric: equivalent valid answers should receive the same treatment. Do not repair only the failed cell while overlooking the same issue in a passing cell. Avoid task-specific rules added merely to make a benchmark pass.
Ecosystem integration
For an ecosystem subagent, keep the portable skill and the pi-daddy definition
separate. Use <skill>-agent.md with a matching distinct frontmatter name and a
plain explicit allowed-tools list (an empty line means no tools). Capabilities
must fit the brief's tools and explicitly agreed agentCapabilities; do not add
context or delegation authority by inference. Select the wrapper as a Pi skill
resource; an agents/ file alone is not discovered. Use native governed
delegation. Availability is not a grant. Preserve operation and
candidate identity, native final and cleanup evidence; never bypass a refused
delegation by launching another child through a shell. Principal workflows are
optional. A social-media role does not itself authorize posting or account changes.
Optional jev_advice can assess one consequential evidence choice after ordinary
checks: reuse matching evidence, retrieve selected project context through Bookie,
research a public gap with Scout, or run a targeted check. State one proposed action
and the uncertainty it could resolve; do not score every alternative or add a call
per tool or turn. Skip advice for cheap obvious steps, unavailable tools, permissions,
identity checks and duplicate detection. Off or unavailable advice does not stop the
ordinary authorized workflow. A probability grants no authority or acceptance.
Existing session paid-call and storage choices still apply. Bookie save/retrieval permission is not permission to send private memory to JEV; select only public or already-redacted evidence. Authoring and diagnostic data are not automatically training examples. Independent labels, source rights and separate export/training approval still apply. See decision advice.
Selected lessons and improvement proposals
When Bookie is loaded, skill_improvement can turn one explicitly selected lesson
into a proposal for an existing skill_author project. Supply its exact context or
producer reference, the proposed change, when it applies, when it does not apply,
and when it should be reconsidered. Bookie checks the current project and destination
before returning a bounded excerpt. Harness stores its digest and reference, not the
raw excerpt. No background memory search or automatic skill installation occurs.
Proposals stay under .skill-harness/authoring/<name>/improvements/. Identical input
is reused; inspection returns metadata and target-currentness, not protected lesson
text. Re-resolve the selected source before using a stored proposal. A source digest
is evidence identity, not acceptance. Continue the existing draft, calibration,
comparison and explicit adoption workflow. Reuse saved answers or judgments only
when their exact dependencies still match. Nothing is designated for training.
If Bookie is missing or refuses the source, this tool records nothing. Ordinary authoring from the user's supplied idea continues without Bookie, Daddy or Principal. Generated skills still need only Pi and their declared tools.
Commands and evidence
Start in Pi with /skill-harness create <idea>; resume with
/skill-harness author <name>. skill_author retains the brief and draft under
.skill-harness/authoring/<name>. Use --specs <eval-root> with a separate eval
repository; its layout is <eval-root>/<skill>/tests.
Free offline commands: init, lint, preflight, diagnose, list, rescore,
restamp, stability, and coverage. calibrate <manifest> validates without
model calls; calibrate --run invokes the selected judge. suggest is the existing
Pi drafting shortcut for a skill that already exists.
expectedCriteria is optional for older controls. When only those expectations
change, review the new manifest and compare saved raw criterion votes through
calibration reuse. Exact judge-prompt bindings must match; expectation-only changes
do not require fresh model calls. Authored calibration is not execution evidence.
Use the cheapest honest remedy: stimulus drift → run; rubric drift → grade;
policy drift → rescore; gate drift → regate (may judge fail-to-pass cases).
grade spends judge tokens only. run spends subject and judge tokens. Honor the
metered-judge opt-in and do not enable it by inference. Missing evidence is ERROR,
never zero or PASS. The author owns verdict overrides; preserve their notes.
See docs/EVAL-AUTHORING.md for file contracts and docs/USAGE.md for the command walkthrough.
04 同仓库更多 Skills
trace-smoke
Use when diagnosing a failure that should be delegated to a planning subagent.
delivery-smoke
Minimal release-pack lint fixture with a deterministic response contract.
skill-harness
Create, test and improve Pi skills and optional ecosystem subagents. Use when the user describes a role they need, asks to create a skill, add evaluations, calibrate a judge, diagnose a skill score or compare models. Start from an idea or an existing skill; examples and specifications are not prerequisites.
output-gates
Test fixture for final assistant output assertions.
golden-skill
Use when testing the skill-harness pipeline itself.
sales-adviser
Prepare accurate account briefs, sales proposals and outreach drafts while respecting pricing and sending authority.