lean-elaboration-test
Run a repeatable elaboration test of ONE Lean 4 repo and write a dated report — snapshot size (date, commit, lines excluding comment-only files), clean project-only rebuild with wall/CPU/parallelism/RSS, heavy-tail and per-namespace breakdown, warm own-file profiles of the worst files with likely causes, and improvement suggestions. Use when asked to run an elaboration test, profile a repo build, or track elaboration health over time.
- License 许可
- CC-BY-4.0
- 最近更新
- 2026-11-14
- Stars
- ★ 12
- Forks
- 5
01 前言 Front Matter
- name
- lean-elaboration-test
- description
- Run a repeatable elaboration test of ONE Lean 4 repo and write a dated report — snapshot size (date, commit, lines excluding comment-only files), clean project-only rebuild with wall/CPU/parallelism/RSS, heavy-tail and per-namespace breakdown, warm own-file profiles of the worst files with likely causes, and improvement suggestions. Use when asked to run an elaboration test, profile a repo build, or track elaboration health over time.
02 资源文件 8
- 参考文档agents/openai.yaml234 B
- 参考文档INDEX.md2.9 KB
- 许可证LICENSE18.2 KB
- 参考文档LICENSES/Apache-2.0.txt11.1 KB
- 参考文档PROVENANCE.md2.6 KB
- 脚本scripts/count_lean_lines.py7.0 KB
- 参考文档SOURCES.md1.5 KB
- 参考文档THIRD_PARTY_NOTICES.md2.1 KB
03 SKILL.md 正文
Lean Elaboration Test
A standardized, comparable snapshot of how one repository elaborates. The method incorporates repeated report-series corrections for partial default targets, contention artifacts, and build-log versus own-file timing.
Scope rule: elaborate JUST THIS REPO. Never mathlib, never upstream path-dependencies (their oleans are inputs, not measurands). A healthy timed build compiles only this repo's own modules; if the log shows upstream compilation, the run is invalid — kill it, fix the cache, restart.
The companion skill lean-elaboration consumes this test's findings.
Phase 0 — Preconditions (all mandatory)
- Sync if asked / note the commit. All size numbers come from git at the
measured commit, never
findon a working tree (mid-flight agent edits corrupted a baseline once):git rev-parse HEAD,git status --short. Requiregit diff --quiet HEAD -- '*.lean'before a commit-attributed run; staged, modified, or deleted tracked Lean files must not enter the baseline. Review untracked.leanfiles and build configuration changes too. If they affect the selected target, stop or measure a separate explicitly labeled working-tree snapshot with matching size counts. Recheck source state after the run; concurrent source edits invalidate a commit-attributed baseline. - Upstream olean guards. Count oleans for Mathlib and any dependencies
(e.g.
find .lake/packages/mathlib/.lake/build/lib -name '*.olean' | wc -l); compare against repository-documented thresholds. If no threshold exists, verify that required dependency artifacts are populated before a timed run. If artifacts are missing, stop and use only the repository-approved cache restoration or one-time dependency setup procedure. - Separate machine load from artifact ownership. You may record
pgrep -x lakeandps -eo user,etime,args | grep -E "[l]ake|[l]ean "(bracket trick so grep doesn't match itself) to disclose co-running load. Other checkouts' processes are load information, not a reason to refuse the test. Before artifact invalidation, require exclusive use of this checkout's build tree through a lock honored by every build wrapper, held through the timed run, or use an isolated checkout with its own project artifacts. Confirm that an isolated.lake/builddoes not resolve to a shared tree. A process listing alone cannot prevent a new same-checkout build from starting. If exclusive artifact ownership cannot be established, do not delete artifacts or run the clean baseline; report it as not performed. - Read the repo's agent instructions (olean preservation, guard hooks, default build target). Respect them; the test must be a no-op for everyone else on the machine.
- NEVER
lake clean. NEVERlake update.
Phase 1 — Size snapshot
Record: date (UTC), commit, .lean file count, total lines, non-comment code
lines, comment-only files excluded. Use
scripts/count_lean_lines.py (self-contained;
counts from git ls-tree at the measured commit; a file is comment-only if
it has no non-comment, non-whitespace character — such files are excluded
from the file/line counts and listed by name). It also counts and lists
non-module files (no module header; lakefile.lean exempt). Record the
count. It must be 0; the script reports it but does not fail.
Also record what the build root covers if it differs from the git tree (quarantined/off-facade modules make tree > built; report both).
Verify the default target covers what you think it covers. A prior report
series measured partial builds because the root facade silently omitted a
large subtree. Check the root .lean facade's imports, or
enumerate targets explicitly. Record the complete transitive set of project
modules selected by those targets, including the root facade. Map them to exact
git-tree source paths in built-files.json, a nonempty JSON array such as
["Project.lean", "Project/Basic.lean"]. Reconcile this manifest with all own-module
build records in Phase 2, including records without displayed timings; total
Lake job counts may include non-module jobs and are not a source-file count.
From the measured repository root, count the exact selected files with the
bundled counter (replace <skill-dir> with this installed skill's directory):
commit=$(git rev-parse HEAD)
python3 <skill-dir>/scripts/count_lean_lines.py "$commit" --files-from built-files.json
Use this count for built-module per-line metrics. Prefix filters alone do not identify a target's dependency closure. Keep the unrestricted tree-wide count as a separate size snapshot; do not include excluded or off-target modules in the built denominator.
Phase 2 — Clean project-only rebuild (the timed run)
Use the repository's guarded project-only rebuild command when it has one.
Establish exclusive artifact ownership as required by Phase 0 before calling
any wrapper that invalidates artifacts. If no wrapper exists, identify the exact
library root (for example Project) and prepare
an explicit project-artifact invalidation plan. Confirm every resolved path is
inside this repository's own build subtree and cannot match a dependency tree
before removing anything. A generic shape is:
rm -rf .lake/build/lib/lean/<Root>
rm -f .lake/build/lib/lean/<Root>.{olean,ilean,trace,olean.hash,ilean.hash}
rm -rf .lake/build/ir/<Root>
/usr/bin/time -v <repo-approved-build-command> > <unique-log-path> 2>&1
Treat the snippet as a template, not authorization to delete unresolved or broad paths. Run invalidation and the build within the same exclusive lease; an ad hoc lock that other builders do not honor offers no protection. Do not use the template when local instructions provide a wrapper.
Use GNU time -v (gives %CPU and peak RSS; bash time doesn't): typically
/usr/bin/time on Linux or gtime on macOS, whose /usr/bin/time is BSD time.
Note the tool in the report — cross-tool CPU splits aren't comparable.
Watch the first minutes: any .lake/packages/mathlib compile lines, or
bulk upstream-module compiles → kill immediately, diagnose, do not let it
run.
Record from /usr/bin/time -v + the log:
- exit status, error count,
sorrycount (vs authorized list), warning count - wall-clock (Elapsed), user CPU, sys CPU, total CPU (user + sys), %CPU, peak RSS
- total jobs (
[N/M]progress lines), and confirm only own-repo modules compiled
Phase 3 — Per-file extraction from the build log
rg --only-matching 'Built <Root>(\.[^ ]+)? \(([0-9.]+)s\)' <log>
The pattern includes the root module and its submodules; replace <Root> with
the regex-escaped root name. Check the pinned Lake version's log format before
using this seconds-only template. Lake's build timing source
uses IO.monoMsNow; inspect the corresponding file in the selected Lean tag
when checking a different toolchain.
- Cumulative logged elapsed time = Σ of displayed Built durations. Lake measures these with a monotonic clock, not a CPU clock. Sub-threshold timings may be omitted; report the timing-visible fraction of built modules.
- Logged overlap ratio = cumulative logged elapsed time ÷ wall-clock. Overlapping jobs and contention affect this ratio; it is not CPU parallelism.
- Average CPU parallelism = total CPU from Phase 2 ÷ wall-clock.
- Per-line CPU cost = total CPU ÷ built-module code lines (ms/line). If also reporting logged elapsed time per line, label it separately and disclose omitted timings. Use the exact Phase 1 module manifest as the denominator, not the whole git tree.
- Heavy-tail tiers: file counts ≥10s / ≥20s / ≥30s / ≥40s (the policy metric that should only ratchet down).
- Top-30 worst files, annotated
vs <prior report date>where a prior snapshot exists. - Per-namespace / per-directory aggregation of logged elapsed time with Δ vs prior — this localizes regressions to groups of files.
Phase 4 — Warm own-file profiling of the worst files
Build wall-clock ≠ own-file cost (includes upstream rebuilds, olean/IR writes, contention; ~25–40% inflated under parallel load, 2.5× observed). Confirm each candidate serially, after the build, one at a time:
profile_log=$(mktemp "${TMPDIR:-/tmp}/lean-profile.XXXXXX")
if <repo-approved-lean-command> --profile <Root>/<Path>/File.lean > "$profile_log" 2>&1; then
profile_status=0
else
profile_status=$?
fi
tail -n 25 "$profile_log"
printf 'Lean exit status: %s; full log: %s\n' "$profile_status" "$profile_log"
(exit "$profile_status")
The last subshell returns Lean's status after displaying the tail; the full
log remains available. Classify the file only when profile_status is zero.
For the top 5–10 files record wall + the cumulative profiling times block
(typeclass inference / simp / elaboration / interpretation:<tactic> /
import / tactic execution / type checking), events >100ms. Classify each
file's dominant phase and likely cause:
import-dominated → not a local problem; candidate is upstream or the file's import list.interpretation:nlinarith/linarith→ arithmetic-closer elimination.typeclass inference, few large events → head-class cache candidate (name the searched class! e.g. eight ~1sSMulsearches = one missing cache).typeclass inference, thousands of sub-100ms events → diffuse floor; don't promise a fix.- large unattributed
elaboration→ rerun with-D trace.profiler.output=out.jsonfor per-declaration attribution before claiming a cause.
Phase 5 — Trend comparison (when prior reports exist)
- Compare wall-clock and logged elapsed times only with the same timing tool, build scope, toolchain, and comparable parallelism and machine load. Report measured total CPU separately; neither logged elapsed time nor default profile phase time is immune to contention.
- Contention artifact check before believing any regression: (a) user CPU down while logged-sum up? (b) parallelism jumped? (c) same-file timings moving in opposite directions? Any two → report the run as contention-bound and skip structural conclusions.
- Predicted-vs-actual: Δbuilt code lines × prior CPU ms/line vs actual Δtotal CPU. Big ratios (>2×) warrant checking load and downstream effects before attributing the change to code growth or a regression.
- Flag any single file on a monotonic climb across ≥3 reports.
Phase 6 — The report
Write ELABORATION_REPORT_<YYYY-MM-DD>.md (or the repo's established
naming) at the repo root. Sections, in order:
- Setup/provenance — date-time window, commit, host, tool, exact commands, co-running load observed, which build target.
- Size snapshot — files, lines, non-comment lines, comment-only files excluded, built-vs-tree scope note.
- Headline table — wall, user/sys/total CPU, %CPU, average CPU parallelism, logged elapsed sum and overlap ratio, peak RSS, jobs, built-module per-line CPU ms; columns vs prior snapshots with comparability caveats.
- Build health — exit status, errors, sorries vs authorized, warning census (top kinds), rule compliance (no upstream compiles, file-size limits, heartbeat overrides).
- Heavy tail — tier table (≥10/20/30/40s) with Δ vs prior; top-30 files.
- Per-namespace logged elapsed time with Δ vs prior.
- Own-file profiles of the worst files — the phase table, dominant category per file.
- Findings ranked by actionability — each with the evidence, the named lever (per the lean-elaboration skill), and expected value; separate "measurement only, no fix applied" honestly.
- What is NOT established — unattributed time, unprofiled tails, contention limits. Every number a future reader might over-trust gets a caveat here.
- Methodology — commands verbatim, so the next run is identical.
- Pointers — prior reports with their roles (last clean baseline, peak regression, etc.).
Cost & etiquette notes
- The timed rebuild can be expensive. Estimate from the repository's prior reports when available; on a shared machine, warn before running. Other agents' builds may run concurrently; record the load and mark timing comparisons contention-bound when appropriate. Concurrent builds must use separate artifact trees; keep the Phase 0 exclusive-ownership guard.
- Profiling (Phase 4) is cheap (~10–60s/file) but must be serial.
- If the measured artifact tree changes concurrently, report clearly that the numbers are contaminated; do not infer contamination from a process listing.
- The report is measurement; applying fixes is the lean-elaboration skill's loop with per-change A/B — do not mix the two in one pass.
04 同仓库更多 Skills
zarr-python
Stores and queries chunked N-D scientific arrays with Zarr-Python 3, including codecs, sharding, S3/GCS storage, and NumPy/Dask/Xarray integration. Use for array layout, bounded I/O, format migration, or scientific metadata preservation.
yeet
Publish local changes to GitHub by confirming scope, committing intentionally, pushing the branch, and opening a draft PR through the GitHub app from this plugin, with `gh` used only as a fallback where connector coverage is insufficient.
xarray
N-dimensional labeled arrays and datasets in Python. Built on top of NumPy and Dask. It introduces labels in the form of dimensions, coordinates, and attributes on top of raw NumPy-like arrays, making data analysis in physical sciences more intuitive and less error-prone. Use for working with multi-dimensional scientific data, NetCDF/GRIB/Zarr files, climate/weather/oceanographic datasets, remote sensing, geospatial imaging, large out-of-memory datasets with Dask, and labeled array operations.
writing-mstest-tests
Use when asked to fix, rewrite, update, improve, modernize, show corrected code for, or explain existing MSTest tests or MSTest-specific configuration. Use for "review" when corrected code or edits are wanted, even for one pasted assertion or passing tests with bad failure output. Covers expected/actual labels; generic Boolean, collection, string, numeric, null, identity, exception, hard-cast, and object[] checks; TestContext/lifecycle; timeout/cancellation; OS/CI conditions, retry, cleanup, parallelization, MSTest.Sdk project setup, and MSTESTxxxx. Honor the installed MSTest version. DO NOT USE to design new test cases (code-testing), perform report-only audits, create project files rather than explain MSTest setup, run tests, migrate frameworks, or handle non-MSTest/non-.NET code.
wgs-prs
End-to-end WGS to polygenic risk score pipeline. Takes paired-end FASTQ files (or a pre-existing VCF) through nf-core/sarek for variant calling, applies VCF QC (normalisation, hard filtering, Ti/Tv and Het/Hom checks), then computes polygenic risk scores via the PGS Catalog. Fills the FASTQ to VCF gap upstream of the gwas-prs skill.
webapp-testing
Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.