perf-optimization
Profile-driven performance optimization of mecatl using the offline benchmark + scenario harness. Use when optimizing allocations or latency, reducing allocs/op or memory, profiling a Go benchmark, pinpointing a hotspot with pprof, proving a win with benchstat, adding a regression benchmark, wiring profile-guided optimization (PGO), or investigating "why is this slow / allocating" or a suspected perf regression. Covers task bench, task perf:scenarios, memprofile -> pprof, benchstat A/B, allocs-first gating, PGO setup, and the discipline (follow the profile not the hypothesis; keep pure-perf changes byte-identical; mutation-test cache guards; skip the wrong abstraction). NOT for the live perf MCP server (use perf-mcp-interpretation) or non-mecatl Go profiling.
- License 许可
- Apache-2.0
- 所属仓库
- stacklok/mecatl
- 最近更新
- 2026-11-15
- Stars
- ★ 309
- Forks
- 29
01 前言 Front Matter
- name
- perf-optimization
- metadata
- { "tags": [ "performance", "profiling", "benchmarks", "pprof", "benchstat", "allocations" ], "author": "mecatl" }
- description
- Profile-driven performance optimization of mecatl using the offline benchmark + scenario harness. Use when optimizing allocations or latency, reducing allocs/op or memory, profiling a Go benchmark, pinpointing a hotspot with pprof, proving a win with benchstat, adding a regression benchmark, wiring profile-guided optimization (PGO), or investigating "why is this slow / allocating" or a suspected perf regression. Covers task bench, task perf:scenarios, memprofile -> pprof, benchstat A/B, allocs-first gating, PGO setup, and the discipline (follow the profile not the hypothesis; keep pure-perf changes byte-identical; mutation-test cache guards; skip the wrong abstraction). NOT for the live perf MCP server (use perf-mcp-interpretation) or non-mecatl Go profiling.
02 资源文件 1
- 参考文档references/playbook.md5.2 KB
03 SKILL.md 正文
Perf optimization (mecatl offline harness)
The companion to the regression-tracking design in
docs/perf-tracking.md. This skill
is the offline benchmark/scenario workflow: measure → profile → optimize →
prove → guard. For diagnosing a running harness via the perf MCP server, use
the perf-mcp-interpretation skill instead — different tool, different signals.
The harness (where the numbers come from)
task bench— hot-path microbenchmarks (engine/prompt,engine/governance,engine/agent).BENCHCOUNTdefault 10. Offline (mockllm + memfs). Not part oftask test. The engine is its own Go module, sotask bench/task fuzzrun these ascd engine && go test … ./prompt/ ./governance/ ./agent/. An ad-hoc re-run on an engine package must do the same:cd engine && go test -bench=… ./agent/(an explicit./engine/agent/path also resolves via the committedgo.work, but the./...wildcard does not cross the module boundary).task perf:scenarios— five whole-loop scenarios. Four live inperf/scenarios/(single-session-long, team-fanout, background-subagents, compaction-cycle); the fifth (tui-scrollback) lives incmd/mecatui/ui/, and the task runs both packages — sogo test ./perf/scenarios/alone gives only four.BENCHCOUNTdefault 6.MECATL_PERF_JSON=.scratch/x.jsonwrites the KPI JSON.- Both are deterministic and offline — never reach for a live model/network.
- PGO/bench captures over
engine/packages depend on the activego.work(the committed workspace wires./enginein alongside the root) — aGOWORK=offinvocation resolves the engine module standalone and won't see the host repo.
What to gate on (allocs-first)
allocs/op, bytes/op, goroutine-delta, and cache-hit-rate are deterministic and
portable — these are the real signal. ns/op and rss_* are machine-specific —
treat as advisory shape, never the headline. A perf change is done only when
allocs/op moves in benchstat; a wall-clock-only "win" on a shared machine is noise.
Workflow
1. Measure — pick the benchmark that actually exercises the path
Run the relevant benchmark and confirm it reflects the workload you care about. A
benchmark whose per-iteration setup dwarfs the code under test, or that is
all-miss by construction, will hide a real win. If no benchmark covers the path,
add one first (match the existing bench_test.go style: for b.Loop(), results to a
package-level sink, offline).
2. Profile — pinpoint the site, do NOT guess
Capture an allocation profile on the benchmark and follow it to a file:line:
go test -run='^$' -bench=BenchmarkX -benchmem -memprofile=.scratch/x.mprof -count=3 ./pkg/
go tool pprof -alloc_space -top -nodecount=25 .scratch/x.mprof # bytes
go tool pprof -alloc_objects -top -nodecount=25 .scratch/x.mprof # object count
go tool pprof -list=FuncName .scratch/x.mprof # line-level
Save profiles under .scratch/ (repo rule — never /tmp). Follow the profile to
the real site. The hypothesis is often wrong (see the playbook: the TUI hotspot was
the string join, not SetContent as assumed). Let -list show you the exact lines.
3. Design — the smallest change at the proven hotspot
Optimize only what the profile proves is hot. Weigh the win against the real cost: a microsecond saved once per turn is invisible next to an LLM round-trip. The wrong abstraction is worse than the allocation — if the clean seam doesn't exist or the fix adds stateful invalidation surface for a marginal gain, it is a legitimate NO-GO. Say so and skip it rather than forcing it.
4. Prove — benchstat before/after, count >= 10
go test -run='^$' -bench=BenchmarkX -benchmem -count=10 ./pkg/ > .scratch/before.txt
# ... apply the change ...
go test -run='^$' -bench=BenchmarkX -benchmem -count=10 ./pkg/ > .scratch/after.txt
benchstat .scratch/before.txt .scratch/after.txt # go install golang.org/x/perf/cmd/benchstat@latest
allocs/op / B/op must drop with a statistically significant delta. Re-run
task perf:scenarios and confirm the scenario KPI moved in the expected direction.
Update the baseline snapshot in docs/perf-tracking.md.
5. Guard — keep behaviour identical, prove the guard isn't vacuous
- Pure-perf changes must be byte-identical. For the TUI,
task test:goldenmust stay green with zero golden diffs — you change HOW, never WHAT. Caveat: golden refresh runs-updatefirst, so a plaingo test ./...(no-update) against the committed goldens is what actually catches a regression; don't rely on the refresh step to catch a stale serve. - Any cache/oracle you add must be mutation-tested. Temporarily break it (force
the stale/wrong path), confirm a test goes red, then restore (back up with
cp, restore withcp— nevergit checkout, it wipes uncommitted work). A guard that stays green when the behaviour is broken is worse than none. - A perf change touching an EXPORTED
engine/symbol trips theapi-compatCI gate (task api:check) — a failure mode a perf optimizer wouldn't expect. Keep pure-perf changes byte-identical to the engine's public surface and it never fires; if the surface legitimately changed, runtask api:updateand add anengine/CHANGELOG.mdentry classified perengine/COMPATIBILITY.md.
Common pitfalls
- All-miss benchmark hides the win. A streaming bench that mutates state every iteration never hits the cache you added — it's the worst-case floor, flat by design. Add a steady-state bench for the cache-hit path to show the real win.
- Setup swamps the signal. Build fixtures outside
b.Loop(); confirm with a quick profile that setup isn't the dominant allocator. - Chasing
ns/opon a shared machine. Gate on allocs; ns is advisory. - Memoizing across the byte-stable prompt prefix. Any prompt-inventory cache must produce byte-identical output or it tanks the provider cache-hit rate — the exact thing perf-tracking exists to protect. Usually not worth it (see playbook).
Complementary: PGO (free compiler-level wins)
Beyond hand-optimizing a hot path, Profile-Guided Optimization lets the compiler optimize from a CPU profile (typically 2–14% CPU — but here sub-1% of wall-clock, since cost is network-dominated: a free set-and-forget win, not a latency feature). The mechanism is already wired:
task pgo:collectbuilds a PROVISIONAL offline profile under.scratch/pgo/.cmd/mecated/default.pgois the reserved slot —go build's-pgo=autoapplies it automatically the moment a profile is dropped there (nothing in the build chain passes-pgo=off).- Do NOT commit an offline-collected profile — it trains the compiler on the
mockllm path that ships in no production binary (it can't pessimize, but it wastes
the one slot). Commit only a real
/debug/pprof/profilecapture from a runningmecatedunder load.
Full rationale, the per-binary decision, and the production refresh + staleness
process live in perf-tracking.md "Profile-guided optimization" —
read it before touching PGO.
See Also
references/playbook.md— pprof flag cookbook + two worked examples (a real win and a real NO-GO) showing the discipline end to end.docs/perf-tracking.md— the KPI design, gating posture, and baselines.perf-mcp-interpretationskill — the live counterpart (running-harness diagnosis via the perf MCP server).
04 同仓库更多 Skills
repo-fact
Use when asked for this repository's registered fact or codeword. Loads the canonical answer format.
greet
Use when asked to greet someone with the ceremonial mecatl greeting. Loads the exact greeting format.
go-table-tests
Write idiomatic table-driven Go tests that stay offline and follow this repo's conventions.
commit-style
Write Conventional-Commits messages for this repo, with the required Co-Authored-By trailer.
tech-writer
Use when writing or substantively editing user-facing documentation: drafting a new page, rewriting or restructuring an existing one, adding a major section, or turning engineering material (PR descriptions, specs, release notes, rough notes) into docs. Writes clear, focused technical documentation following the Diataxis framework and Mecatl's canonical style guide. Use it even when the request doesn't mention writing quality; it governs how documentation gets written. Not for editorial review of finished work.
perf-mcp-interpretation
Interpret the mecatl perf MCP server's output to diagnose latency, goroutine leaks, GC pressure, allocation churn, and memory growth in a running mecatl harness. Use when connected to the mecatl perf MCP server (the perf:// resources or the query_metric / top_cpu_functions / capture_cpu_profile / top_allocations / list_slow_turns / capture_flight_recorder tools) and investigating why mecatl is slow, leaking, or growing. Covers tool routing/cost, reading pprof rankings and runtime metrics, and the leak/contention/GC signatures. NOT for generic Go profiling or non-mecatl MCP servers.