release-lanorme
Use when cutting, shipping, releasing, or publishing a new LaNorme version (a "0.x.y" bump, a tag, a PyPI release). Runs every release gate (unit tests, the dogfood, generated docs in sync), records a eval audit (precision/recall/F1 per scored rule plus end-to-end performance, stamped with the version and hardware) to evals/results/, bumps the version, tags, and creates the GitHub Release that auto-publishes to PyPI and deploys the versioned docs.
- License 许可
- MIT
- 所属仓库
- lanorme/lanorme
- 最近更新
- 2026-11-14
- Stars
- ★ 3
- Forks
- 6
01 前言 Front Matter
- name
- release-lanorme
- license
- MIT
- metadata
- { "project": "lanorme" }
- description
- Use when cutting, shipping, releasing, or publishing a new LaNorme version (a "0.x.y" bump, a tag, a PyPI release). Runs every release gate (unit tests, the dogfood, generated docs in sync), records a eval audit (precision/recall/F1 per scored rule plus end-to-end performance, stamped with the version and hardware) to evals/results/, bumps the version, tags, and creates the GitHub Release that auto-publishes to PyPI and deploys the versioned docs.
- compatibility
- Requires Python 3.13+, uv, git, and gh, run from a clean main checkout.
03 SKILL.md 正文
Release LaNorme
This page explains how to cut a release and the discipline every release runs:
the gates, a recorded eval audit, regenerated docs, and the automated
publish. Creating a GitHub Release fires .github/workflows/release.yml (builds
and publishes to PyPI through Trusted Publishing, OIDC, no token handled) and
.github/workflows/docs.yml (deploys that version's docs with mike). uv publish is never run by hand.
The release discipline
Every release records the same evidence so a green tag is auditable later:
- Tests and dogfood pass (
pytest tests/unit,lanorme check .). - Generated docs are in sync with the tool (
scripts/gen_docs.py --check): the configuration reference, JSON schema, rule index, and llms files. - A eval audit is recorded to
evals/results/vX.Y.Z.json: the accuracy metrics (precision, recall, F1 per scored rule against the labelled corpora, for the dev split, the sealed holdout split and the gap between them) and the end-to-end performance numbers, each stamped with the LaNorme version, the git commit, the Python version, and the hardware (platform and processor). Accuracy is deterministic and is the audit's backbone; performance is informative and machine-dependent (the hardware stamp is what makes it interpretable). - No holdout regression. The audit runs with
--gate latest: it fails when any rule's holdout precision or recall falls more than 0.02 below the best value any comparable committedevals/results/v*.jsonrecorded (one that scored the same holdout files), not merely the latest, and lists the rules. It also fails when a holdout file the newest recorded audit digested was removed or changed, unlessevals/holdout_revisions.jsonaccepts that exact new digest with a reason. Dev numbers are informational and never block. - RULES.md reflects the measured F1 for every rule that has a corpus. If a rule's F1 moved, update its line before tagging.
scripts/release.sh enforces steps 1 to 4 (it refuses to tag if any gate fails
or the docs are stale); step 5 is a human check on the audit output.
Versioning
The public surface is the rule codes used in select / ignore /
per-file-ignores and the config keys under [tool.lanorme]. The deciding
question is whether a green codebase could go red on upgrade:
- patch (
0.y.z): no existing codebase's result changes (fixes, docs, opt-in checks, new config keys with safe defaults). - minor (
0.y.0): a green codebase can newly fail (a new default-on check, a default-on rule made stricter, a renamed or removed rule code, a changed default). Before 1.0, every breaking change is a minor. - major (
1.0.0): the stability commitment.
The README "Versioning" section is canonical; keep them in step.
Steps
-
Pick the new version
X.Y.Z. -
Add a
## [X.Y.Z]section toCHANGELOG.mddescribing the user-facing changes. Required: the release notes are taken verbatim from it. -
Regenerate the docs and review the audit numbers:
uv run python scripts/gen_docs.py uv run python evals/audit.py --version X.Y.Z --output /tmp/preview.json --gate latestRead the preview; if any F1 changed, update that rule's line in
docs/RULES.md. A holdout regression fails the preview and the release: fix the rule, never the holdout files (seeCONTRIBUTING.md). Commit the regenerated docs and any RULES.md change. Do not commit the audit file yourself:release.shrecords the committedevals/results/vX.Y.Z.jsonfor you, against the release commit (step 4), so its version and commit stamp match the released tree. -
From the repo root, run the helper:
scripts/release.sh X.Y.ZIt refuses unless you are on
main, the CHANGELOG section exists, the docs are in sync, and the gates pass (including an eval-audit precheck that the corpora are complete and not stale and that no holdout number regressed). Then it bumps the version inpyproject.tomlandsrc/lanorme/__init__.py, builds, runstwine check, commits the release, records the eval audit against that commit (a secondRecord X.Y.Z eval auditcommit), tagsvX.Y.Z, pushes, and creates the GitHub Release. -
Watch the publish and docs workflows, then verify the package is live:
gh run watch $(gh run list --workflow=release.yml --limit 1 --json databaseId --jq '.[0].databaseId') --exit-status uvx --refresh --from lanorme==X.Y.Z lanorme --versionThe docs workflow publishes
X.Y.Z(and moves thelatestalias) at https://lanorme.github.io/lanorme/.
Gotchas
scripts/release.shrefuses to do anything unless you are onmain, the working tree is clean, the## [X.Y.Z]CHANGELOG section exists, and the gates pass. It is safe to run; it tags nothing until every gate is green.- The version lives in two files (
pyproject.tomlandsrc/lanorme/__init__.py);release.shbumps both. The manual fallback must too, or the build andlanorme --versiondisagree. uv publishis never run by hand. PyPI publishing is OIDC Trusted Publishing, fired only by the GitHub Release.- The eval audit's accuracy step is strict: if a scorer sees a finding that
is not in its corpus
labels.json, orvalidate_corpora.pyfinds an unlabelled file or comment, a label whose line hash is missing or no longer matches its line, a label of the wrong polarity for its directory, or a file off its recorded split, it errors rather than scoring a wrong number. That means a fixture went stale, not that the release is blocked on performance; fix the labels. The split is recorded per file inlabels.json(the name hash only proposes one for a new file), anduv run python evals/validate_corpora.py --stampfills a missing split or line hash. - The holdout gate holds each rule to the best comparable release over the
whole history of
evals/results/v*.json, not the newest alone. A rule no comparable audit has holdout numbers for is skipped, not failed, and the gate prints a note when it gated nothing, so the first release after a corpus gains a holdout split records the baseline the next one is held to. - Performance numbers are machine-dependent. The audit stamps the hardware so they are interpretable, but do not compare them across machines.
If something fails
- A gate (tests, dogfood, stale docs) fails: nothing is committed or tagged. Fix and re-run.
- The eval audit's accuracy step fails (a scorer flags an unlabelled finding): the corpus is out of date. Fix the labels or the fixture, re-run.
- The holdout gate fails: a change since the last release made a rule worse on
data it was not tuned on, or a holdout file changed. Fix or revert the rule
change; do not edit the holdout files to pass. A holdout edit that is right
on its own merits (a label proved wrong) goes in its own reviewed change with
an
evals/holdout_revisions.jsonentry naming the new digest and the reason. - The publish workflow fails (for example a PyPI outage): the tag and release
already exist, so do not re-tag. Re-run with
gh run rerun <id>orgh workflow run release.yml.
Manual fallback (no script)
Edit CHANGELOG.md, run uv run python scripts/gen_docs.py, uv run --group dev pytest tests/unit, and uv run lanorme check .. Bump version in
pyproject.toml and __version__ in src/lanorme/__init__.py, then uv build, git commit -m "Release X.Y.Z". Now record the audit against that
commit: uv run python evals/audit.py --version X.Y.Z --gate latest and git commit -m "Record X.Y.Z eval audit" evals/results/. Finally git tag -a vX.Y.Z, git push origin main, git push origin vX.Y.Z, and gh release create vX.Y.Z dist/* --notes "...".
04 同仓库更多 Skills
valid-minimal
A minimal valid skill used as a parser fixture.
valid-links
A valid skill that links to a bundled reference file.
valid-folded
A valid skill whose description is a folded block scalar spanning multiple lines, to make sure the parser does not call it empty.
unterminated
暂无简介
missing-desc
暂无简介
invalid-no-frontmatter
暂无简介