codespine-workload
Benchmark a program in a controlled environment and attribute the cost with the code graph. Use when asked to profile under a CPU/memory cap, measure server capacity ("how many requests per second before p99/latency breaks an SLO"), decide "when do I need more servers", find where runtime is spent under load, or run a workload in a constrained "one box" container. Two modes: cpu-profile (where the time goes) and loadtest (how much load it takes). Requires the codespine CLI; Docker is optional (falls back to running on the host).
- License 许可
- MIT
- 最近更新
- 2026-06-29
- Stars
- ★ 7
- Forks
- 0
01 前言 Front Matter
- name
- codespine-workload
- description
- Benchmark a program in a controlled environment and attribute the cost with the code graph. Use when asked to profile under a CPU/memory cap, measure server capacity ("how many requests per second before p99/latency breaks an SLO"), decide "when do I need more servers", find where runtime is spent under load, or run a workload in a constrained "one box" container. Two modes: cpu-profile (where the time goes) and loadtest (how much load it takes). Requires the codespine CLI; Docker is optional (falls back to running on the host).
02 资源文件 4
- 参考文档reference/two-track.md2.8 KB
- 脚本templates/cpu_profile_driver.template.ts2.0 KB
- 其他templates/Dockerfile1.0 KB
- 脚本templates/loadtest_driver.template.ts13.6 KB
03 SKILL.md 正文
codespine-workload
Exercise a program under a chosen environment, then attribute the cost back onto the code graph. This turns vague runtime questions ("why is it slow?", "how many requests can it take?", "when do I add a server?") into measured, graph-grounded answers.
The capability is one idea in three parts — workload kind × environment × attribution:
- kind —
cpu-profile(loop the hot path) orloadtest(ramp a running server). - environment —
--host(uncapped baseline) or a container under an enforced--cpus/--memorycap (a realistic "one server" box). - attribution —
cpu-profile→enrich→hotspots/cost(graph-grounded where the time goes);loadtest→ a latency-vs-throughput curve and the knee (how many requests/second before the SLO breaks).
When to use this skill
- "Where is the time spent / why is this slow under load?" → cpu-profile.
- "How many requests/second can my server handle before p99 crosses my SLO?" → loadtest.
- "When do I need another server / how do I size capacity?" → loadtest.
- "Will it hold up on a small box (0.5 CPU / 512 MB)?" → either mode, docker environment.
- The agent is mid-task in
/codespine-interviewor/codespine-optimizeand hits aneeds-workloadsituation (it must measure a runtime claim, not guess) → use this.
For deciding which symbol matters from an existing profile, use the
codespine-query skill (hotspots, cost). This skill is about producing the
measurement in a controlled environment in the first place.
Realism vs. determinism — the honesty rule
There are two tracks, and they answer different questions. Keep them separate:
- Determinism track (the
benchmarkcommand): low-variance before/after deltas for "did my edit make X faster?". Stable median, advisory. - Realism track (this skill, under a cap): "will it hold up on a constrained
box?". It intentionally introduces a cap and accepts scheduler noise. These
numbers are a capacity estimate, not a deterministic gate. Never diff two
realism runs as a pass/fail, and never use a realism number to approve an
optimization — that is what
benchmark/verifyare for.
State this caveat whenever you report realism-track numbers. See
reference/two-track.md for the full framing.
Prerequisites
- The codespine CLI (
codespine …, ornpx codespine …). - For cpu-profile attribution: a built graph at
./.codespine/graph.kuzu. If it is missing, build it once:codespine extract . --semanticthencodespine load. - For the docker environment: a container runtime (Docker Desktop / OrbStack /
podman). If absent, use
--host. - For loadtest: the server must be runnable locally and expose a readiness endpoint (something that returns 2xx when ready).
Where artifacts live, and what this skill ships
All per-project artifacts go in the user's repo under ./.codespine/workload/
(gitignorable) — never in the codespine package. This skill ships starting-point
templates next to this file under templates/:
templates/Dockerfile— a tinynode:24-slimrunner image (tsx+autocannon).templates/cpu_profile_driver.template.ts— skeleton for the cpu-profile loop.templates/loadtest_driver.template.ts— a working HTTP capacity driver; edit the marked block for your endpoints.
Run codespine workload scaffold --kind cpu-profile|loadtest|both to copy the
relevant template(s) into ./.codespine/workload/, then fill in the
// EDIT FOR YOUR PROJECT block. The rest is generic.
Mode A — cpu-profile: where does the time go?
- Build the graph if needed:
codespine extract . --semantic && codespine load. - Scaffold:
codespine workload scaffold --kind cpu-profilewrites the driver + Dockerfile to./.codespine/workload/. - Author the driver: in the EDIT block of
./.codespine/workload/cpu_profile_driver.ts, import the project's hot module(s) and exercise the real public API in a deterministic loop (no timing, just work). It must live outside the extracted source root so it never becomes a graph node; imports are module-relative. - Run it — one command profiles, enriches the graph, and prints the top self
time. On the host (simplest; uses the project's existing
node_modules):
…or under an enforced cap for the "constrained box" answer —codespine workload run --driver ./.codespine/workload/cpu_profile_driver.ts --root .--projectis the directory mounted at/work, and the driver must live inside it:codespine workload run --driver ./.codespine/workload/cpu_profile_driver.ts --root . \ --docker --cpus 0.5 --memory 512m --project .--rootmust be the directory you passed toextract(usually., or a subdirectory like./srcif you extracted one) — otherwise the profile's frame paths will not resolve onto the graph and most samples come back "unattributed". - Read the ranked views:
codespine hotspots --by self-time(the leaves where CPU burns) andcodespine cost(which symbols the time is spent under). Report the top frames, tie them to source, and — if asked — to optimization tasks.
Mode B — loadtest: how much load can it take?
- Learn the server: find the entrypoint (how it boots, what port), the readiness endpoint, and the routes + their request contracts (read the route handlers). For a write path, find the durability config (does it fsync?).
- Scaffold + author: copy
templates/loadtest_driver.template.tsto./.codespine/workload/loadtest_driver.tsand edit the marked block: how to start the server, the readiness path, andbuildRequests()(your endpoint mix). A filled example for a typical REST API is in the template's comments. - Run — host baseline first, then the cap:
# host (uncapped upper bound): SERVER_ENTRY=src/main.ts PORT=3000 DB_PATH=/tmp/app.loadtest.db \ LOADTEST_PROFILE=mixed LOADTEST_SLO_P99_MS=200 \ node --import tsx .codespine/workload/loadtest_driver.ts # docker (one constrained "server" box — realism): docker build -t codespine-workload-runner .codespine/workload docker run --rm --cpus 0.5 --memory 512m --memory-swap 512m \ -v "$PWD:/work:ro" -v "$PWD/.codespine/workload:/opt/runner/wl" \ -e SERVER_ENTRY=/work/src/main.ts -e SERVER_CWD=/opt/runner \ -e PORT=3000 -e DB_PATH=/tmp/app.loadtest.db \ -e LOADTEST_PROFILE=mixed -e LOADTEST_SLO_P99_MS=200 \ codespine-workload-runner \ node --import tsx /opt/runner/wl/loadtest_driver.ts - Read the ramp: the driver prints a table (offered rps → p50/p99) and the knee. Capacity = the highest sustained rps with p99 under the SLO. "Time to add a server" = steady-state load approaching that number.
- Critical: use a file-backed datastore for any write-inclusive profile
(set
DB_PATH/equivalent to a file). With an in-memory store, fsync is a no-op and write capacity is fictitious.
Choosing host vs. docker
- Host — uncapped, uses the project's own
node_modules, no container friction. The easy starting point and the upper-bound baseline. A single-threaded server effectively uses ~one core regardless of host core count. - Docker — an enforced cgroup cap is the only way to get a faithful "one
server" box (macOS has no native per-process hard cap). Use it for the realism
answer. Co-locate the load client in the container against
127.0.0.1(avoids the host↔VM network-latency floor; the trade-off is the client shares the CPU cap). If the project has runtime dependencies with native addons, the container needs a Linux build of them (a macOS-arm64node_moduleswill not run in the Linux VM) — provision them into a volume or install in the image. When that is painful, prefer host mode.
Run both and compare: the gap tells you how much of the slowness is the cap (cgroup throttling + a squeezed page cache) versus the program itself.
Interpreting and reporting
- Always attach the realism caveat to capped numbers (noise; estimate not gate).
- At low offered rates, p99 is sample-thin (few requests) — lengthen the step duration for a stable tail, and say so.
- A sharp knee (throughput plateaus while latency explodes) usually means a serial bottleneck (e.g. a synchronous DB on one event loop) — more cores will not help a single-threaded server; "more servers" means more processes/replicas.
- Tie findings to the graph:
cpu-profile→ name the hot symbols fromhotspots; for a slow endpoint, read its handler → service → datastore path.
Limitations / when to escalate
This MVP assumes the workload is runnable via node/tsx. Projects needing a
build step, a non-npm package manager, non-HTTP load, custom readiness, or auth
need the driver/run-command adapted by hand — note that to the user rather than
reporting a bogus number. For cpu-profile, codespine workload run (host and
--docker) and codespine workload scaffold already wrap this orchestration; the
loadtest kind is run from the scaffolded driver directly for now. The cpu-profile
--docker path auto-provisions a Linux build of the project's dependencies into a
cached volume (keyed by the lockfile); the loadtest kind under --docker is not
wired yet (use host).
04 同仓库更多 Skills
codespine-query
Query a TypeScript codebase as a knowledge graph to answer impact, caller, dependency, and dead-code questions. Use this instead of grep/glob for "who calls X", "what breaks if I change X" (blast radius), "is X dead code", "what references this type", and "what is X connected to". Requires the codespine CLI and a built graph database.
codespine-query
暂无简介