[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6D24P1X28FSHDG4AC0AYVSY":3,"skill-same:01M6D24P1X28FSHDG4AC0AYVSY":71},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":43,"linkOnly":44,"markdownContent":45,"markdownBody":46,"frontMatter":47,"refName":48,"commitSha":49,"skillDirectory":50,"forksCount":51,"repositoryHtmlUrl":52,"repositoryHomepageUrl":53,"ownerHtmlUrl":54,"ownerBlogUrl":53,"ownerDescription":55,"updatedAt":56,"install":57,"resources":70},"01M6D24P1X28FSHDG4AC0AYVSY","performance-profiling",null,"Finding where time and memory go: sampling versus instrumenting profilers, CPU, heap, allocation, lock and I\u002FO profiles, flame graphs and how to read them, the tools for Node, browsers, Python, Go, Rust, the JVM, .NET and native code, profiling in production safely, and turning a profile into a change. Read before optimising anything, and when something is slow for no obvious reason.","enowdev\u002Fenowxcli","enowdev","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F179420349?v=4","01M6D1MVX2KC0X3Z9NESE7P22K","Enow Dev.","01M6D1MW1FMZGBDKWAJY4GQX81",67,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fenowdev\u002Fenowxcli\u002Fblob\u002Fmain\u002Fcrates\u002Fenowx-core\u002Fskills\u002Fperformance-profiling\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000030","开发工程师","PROFESSION","适用职业",[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42],"performance","performance profiling","profiling","jvm","go","graphs","time","sampling","profile","read","node","how","Tools","Production","code","change","rust","memory","python","2026-11-10 10:12:30",false,"---\nname: performance-profiling\ndescription: \"Finding where time and memory go: sampling versus instrumenting profilers, CPU, heap, allocation, lock and I\u002FO profiles, flame graphs and how to read them, the tools for Node, browsers, Python, Go, Rust, the JVM, .NET and native code, profiling in production safely, and turning a profile into a change. Read before optimising anything, and when something is slow for no obvious reason.\"\n---\n\n# Profiling: where time and memory go\n\nWithout a profile, optimisation is a guess: timers sprinkled around the\nfunction that looks slow, a debug build profiled, a flame graph's colours\nread as heat and its left-to-right order as time, a library frame\n\"optimised\" when the real question is why the code calls it ten thousand\ntimes. This skill picks the profile for the symptom, gives the commands\nper runtime, and shows how to read the result and turn it into a change.\nMeasuring before and after is in `performance`.\n\n## 1. Pick the profile from the symptom\n\n| Symptom | Look with |\n|---|---|\n| A core at 100%, slow | CPU sampling profile (on-CPU) |\n| Slow, but the CPU mostly idle | wall-clock or off-CPU profile, tracing spans, lock and I\u002FO profiles |\n| Memory grows or is too high | heap snapshots over time, heap profile (`performance-memory`) |\n| High GC time, p99 spikes | allocation profile, GC logs |\n| Threads or tasks waiting on each other | lock, mutex or block profile; thread dumps |\n| A slow query | EXPLAIN and the database's statement statistics (`database-queries`) |\n| A janky page or a slow interaction | the browser's Performance panel (`frontend-performance`) |\n\nA minute with the operating system tells you the row: on Linux `top -H`,\n`vmstat 1` and `iostat -x 1`; anywhere, `time \u003Ccommand>`, where `real` far\nabove `user + sys` means the process waits (a CPU profile will look empty).\n\n## 2. Sampling, instrumenting and tracing\n\n- **Sampling** records stacks at a fixed rate: 99 Hz in production (an odd\n  rate avoids lockstep with periodic work), 1,000 Hz or more locally. The\n  overhead is low and even across functions, so it is safe in production;\n  it is statistical, so collect thousands of samples (30 to 60s under\n  load) and expect it to miss short, rare events.\n- **Instrumenting** wraps every call (cProfile, manual timers, spans):\n  exact counts, but overhead grows with the number of calls, so small hot\n  functions look worse than they are. Use it for counts, not proportions.\n- **Tracing** records the timeline of one run or request: order,\n  concurrency and gaps (the Performance panel, `go tool trace`,\n  OpenTelemetry spans, JFR events). A profile aggregates; a trace shows\n  sequence and waiting.\n\n## 3. On-CPU, off-CPU and wall-clock\n\nA CPU profile shows only time spent running: time blocked on a socket, a\ndisk, a lock, a pool or the database is invisible in it. Use instead:\n\n- Wall-clock profiles, which sample every thread, running or not: py-spy\n  with `--idle`, async-profiler `-e wall`, pyinstrument (wall-clock by\n  default), fgprof for Go.\n- Off-CPU analysis on Linux: bcc's `offcputime` records where threads\n  blocked and for how long, drawn as a flame graph (section 5).\n- The runtime's own waiting views: Go's block and mutex profiles, JFR\n  monitor and park events, async-profiler `-e lock`, tokio-console for\n  async tasks, Node's event loop delay; and spans, for the gaps and the\n  calls made in series that could run concurrently (`performance-backend`).\n\n## 4. Reading a flame graph\n\n- Each box is a function; the boxes above it are what it called. Width is\n  the share of samples in it and everything it called (total time); height\n  is only stack depth.\n- In a classic flame graph the x-axis is sorted alphabetically to merge\n  stacks, so left to right is not time; in a flame chart (the Performance\n  panel, speedscope's Time Order view) it is. Colour is random or by kind\n  of code, not heat, unless the legend says otherwise.\n- **Self time** is the top edge of a box not covered by its children: the\n  function itself did the work. A wide box with narrow children is hot\n  itself; a wide box whose width is all children is only on the path.\n- Find the wide plateaus at the top, then walk down the stack to the first\n  frame in your own code: that is where the decision to do the work was\n  made, and usually what you can change.\n- One function in many thin towers under different callers adds up\n  unseen: a bottom-up view (DevTools Bottom-Up, flamegraph.pl `--reverse`,\n  speedscope's Sandwich) ranks functions by self time across all callers.\n  Compare before and after with a diff (`difffolded.pl`, Pyroscope,\n  `pprof -diff_base`).\n- `[unknown]` frames and broken stacks: build with symbols and frame\n  pointers (`-fno-omit-frame-pointer`, Rust `-C force-frame-pointers=yes`)\n  or unwind with DWARF (`perf record --call-graph dwarf`); JIT runtimes\n  need a perf map (`node --perf-basic-prof`). Inlined functions vanish\n  into their callers; line views (pprof `list`, `perf annotate`) find them.\n\n## 5. The tools, by runtime\n\nProfile an optimised build with a realistic workload running: never a\ndebug build, a development server or an idle process.\n\n**Node.js**\n\n```bash\nnode --cpu-prof --cpu-prof-dir=.\u002Fprof app.js   # open in DevTools or with npx speedscope\nnode --heap-prof app.js                         # sampled allocations, .heapprofile\nnode --inspect app.js                           # chrome:\u002F\u002Finspect: Performance, Memory\n```\n\n- `--cpu-prof` writes on a normal exit: a server needs a `SIGINT` handler\n  that calls `process.exit()`. clinic.js (`clinic flame`) and 0x draw a\n  flame graph in one command; check they support the project's Node first.\n- A blocked event loop is CPU work on the main thread. Measure it with\n  `perf_hooks.monitorEventLoopDelay()`: a p99 in the tens of milliseconds\n  means every request queues behind that work; move it to `worker_threads`\n  or a job.\n\n**Browsers**: the Performance panel with CPU throttling (4x or 6x), then\nthe Main track (a flame chart), Bottom-Up for self time, and long tasks\n(over 50ms, flagged red). For a slow interaction, INP attribution splits\ninput delay, processing and presentation; the `web-vitals` attribution\nbuild and the Long Animation Frames API report it from real users. React\nDevTools' Profiler shows which components rendered and why.\n\n**Python**\n\n```bash\npy-spy top --pid \u003Cpid>                                  # live, no restart\npy-spy record -o prof.svg --pid \u003Cpid> --duration 30     # add --idle for wall-clock\npy-spy dump --pid \u003Cpid>                                 # every thread's stack, for a hang\npython -m cProfile -o out.prof app.py && snakeviz out.prof\npyinstrument app.py                                     # wall-clock call tree, async-aware\nscalene app.py                                          # Python, native and system time per line\n```\n\npy-spy attaches without code changes (sudo on macOS, `SYS_PTRACE` in a\ncontainer); `--native` adds C extension frames, `--subprocesses` follows\ngunicorn or multiprocessing workers. cProfile instruments: right for call\ncounts, inflated for small functions. line_profiler (`kernprof -l -v`)\ntimes each line of functions marked `@profile`; Python 3.12+ works with\nLinux perf via `python -X perf`.\n\n**Go**: register the handlers with `import _ \"net\u002Fhttp\u002Fpprof\"` and serve\nthem on a private port, `go http.ListenAndServe(\"localhost:6060\", nil)`. If\nthe public server uses `http.DefaultServeMux`, the import exposes them there.\n\n```bash\ngo tool pprof -http=:8081 'http:\u002F\u002Flocalhost:6060\u002Fdebug\u002Fpprof\u002Fprofile?seconds=30'\ngo tool pprof -http=:8081 http:\u002F\u002Flocalhost:6060\u002Fdebug\u002Fpprof\u002Fheap     # in use\ngo tool pprof -http=:8081 http:\u002F\u002Flocalhost:6060\u002Fdebug\u002Fpprof\u002Fallocs   # all allocations\ncurl -o trace.out 'http:\u002F\u002Flocalhost:6060\u002Fdebug\u002Fpprof\u002Ftrace?seconds=5' && go tool trace trace.out\ngo test -run='^$' -bench=Parse -cpuprofile cpu.out -memprofile mem.out .\u002Fparser\n```\n\n- Block and mutex profiles stay empty until enabled with\n  `runtime.SetBlockProfileRate(n)` and `runtime.SetMutexProfileFraction(n)`\n  (a smaller `n` records more and costs more).\n- In pprof: `top` (self), `top -cum` (total), `list Func` (per line), the\n  flame graph under View. `go tool trace` shows scheduler latency, GC\n  pauses, syscalls and blocked goroutines. A production CPU profile saved\n  as `default.pgo` in the main package drives PGO (Go 1.21+).\n\n**Rust**\n\n```bash\nCARGO_PROFILE_RELEASE_DEBUG=true cargo flamegraph --bin app -- \u003Cargs>   # perf or dtrace\nsamply record .\u002Ftarget\u002Frelease\u002Fapp \u003Cargs>                              # Firefox Profiler UI\nperf record -F 99 --call-graph dwarf .\u002Ftarget\u002Frelease\u002Fapp\nperf script | inferno-collapse-perf | inferno-flamegraph > flame.svg\n```\n\nRelease builds with symbols (`debug = \"line-tables-only\"` under\n`[profile.release]`, or the variable above). Heap: the `dhat` crate or\nheaptrack. Async: tokio-console (`console-subscriber`, built with\n`RUSTFLAGS=\"--cfg tokio_unstable\"`) shows tasks that poll too long.\n\n**JVM**\n\n```bash\nasprof -d 30 -e cpu -f cpu.html \u003Cpid>      # async-profiler 3+; also -e wall, alloc, lock\njcmd \u003Cpid> JFR.start duration=60s filename=rec.jfr settings=profile\njfr view hot-methods rec.jfr                # JDK 21+, or open rec.jfr in JDK Mission Control\njcmd \u003Cpid> Thread.print                     # thread dumps: three, 5s apart, for a hang\n```\n\nasync-profiler and JFR sample without safepoint bias (thread-dump\nsamplers blame the wrong lines). JFR costs about 1% (default) to 2%\n(`profile`), cheap enough for production. GC detail: `-Xlog:gc*`.\n\n**.NET**\n\n```bash\ndotnet-counters monitor -p \u003Cpid> --counters System.Runtime   # CPU, GC, allocation, thread pool\ndotnet-trace collect -p \u003Cpid> --duration 00:00:00:30 --format Speedscope\ndotnet-gcdump collect -p \u003Cpid>                                # heap by type\n```\n\nPerfView (Windows), Visual Studio and dotTrace go deeper. Thread pool\nstarvation (usually sync over async: `.Result`, `.Wait()`) shows as thread\ncount and queue length rising while CPU stays low.\n\n**Native code and the whole machine**\n\n```bash\nperf stat -d .\u002Fapp                                   # cycles, IPC, cache and branch misses\nperf record -F 99 -g -p \u003Cpid> -- sleep 30 && perf report\nperf script | stackcollapse-perf.pl | flamegraph.pl > cpu.svg\noffcputime-bpfcc -df -p \u003Cpid> 30 > off.stacks        # bcc; offcputime on some distributions\nflamegraph.pl --color=io --countname=us \u003C off.stacks > off.svg\nbpftrace -e 'profile:hz:99 \u002Fpid == 1234\u002F { @[ustack] = count(); }'\n```\n\n- In containers and VMs, perf needs `kernel.perf_event_paranoid` lowered\n  or `CAP_PERFMON`; hardware counters may be missing.\n- macOS: Instruments (Time Profiler, Allocations) or `sample \u003Cpid> 10`.\n  VTune and AMD uProf add hardware detail; Valgrind's callgrind counts\n  every instruction, tens of times slower.\n- IPC under about 1 with many cache misses means the loop waits on memory,\n  not arithmetic: the fix is data layout (`systems-memory`).\n\n**Databases** profile themselves (`database-queries`):\n\n```sql\n-- Postgres, with pg_stat_statements in shared_preload_libraries\nSELECT calls, round(total_exec_time) AS total_ms,\n       round(mean_exec_time::numeric, 1) AS mean_ms, rows, query\nFROM pg_stat_statements ORDER BY total_exec_time DESC LIMIT 20;\n```\n\nRank by total time, not mean: a 2ms query run 50,000 times an hour costs\nmore than a 3s report run twice. Then `EXPLAIN (ANALYZE, BUFFERS)` on the\ntop ones. MySQL: the slow query log and `performance_schema` digests.\n\n## 6. Profiling in production, safely\n\n- Sample, never instrument: 99 Hz for 30 to 60 seconds on one instance,\n  out of the load balancer when the tool is heavy. Continuous profilers\n  (Grafana Pyroscope, Parca with eBPF, Datadog and other APMs) keep a\n  low-rate profile of every instance, so a bad hour can be compared with a\n  good one afterwards.\n- Profiling endpoints (`\u002Fdebug\u002Fpprof`, the Node inspector, JMX) are never\n  public: localhost or an internal network behind auth. An open inspector\n  port is remote code execution.\n- Heap dumps hold whatever was in memory (tokens, passwords, personal\n  data): store them like secrets, never attach them to tickets or chats,\n  delete them after. They pause the process and can need as much memory\n  again as the heap: take them from an instance out of rotation.\n\n## 7. From a profile to a change\n\nTake the top five frames by self time and by total time; for each, find\nwhy it is called (walk down to your code) and how often, then ask in order:\n\n1. **Avoid it?** Results never used, log lines formatted at a disabled\n   level, validation repeated at every layer, data fetched just in case.\n2. **Do it once?** Hoisted out of the loop, per request instead of per\n   item, memoised, cached (`backend-caching`), precomputed on write.\n3. **Batch it?** One query or bulk call for N items (`performance-backend`).\n4. **Make it cheaper?** A better algorithm or data structure, fewer\n   allocations and copies (`performance-memory`), a measured faster library.\n5. **Move it?** To a job (`backend-jobs`), a worker thread, or later.\n6. **Parallelise it?** Only when the work is independent and cores sit idle.\n\nUsual findings: JSON parsing and serialisation of large payloads; regular\nexpressions compiled in a loop; strings built by repeated concatenation;\nORM hydration of thousands of objects; logging; reflection; exceptions as\ncontrol flow; the same data sorted repeatedly; lock contention; GC driven\nby a high allocation rate. Password hashing (argon2, bcrypt) is slow on\npurpose: protect it with rate limits, never lower its cost.\n\nThen profile again under the same workload: the plateau is narrower,\nnothing new has grown. Keep both profiles with the report.\n\n## Check it\n\n- The profile came from an optimised build under a realistic workload,\n  with thousands of samples and resolved symbols (no wall of `[unknown]`),\n  and its type matched the symptom (wall-clock or off-CPU when idle).\n- The change targets a frame that was wide, a second profile shows it\n  narrower, and the end-to-end measurement confirms it (`performance`).\n\n## Avoid\n\nOptimising without a profile; profiling a debug build, an idle process or\na toy input; reading a flame graph's x-axis as time or its colours as heat;\na CPU profile to explain a service that waits; instrumenting profilers for\nproportions; tuning a library frame instead of asking why your code calls\nit; pprof or inspector ports reachable from the internet; heap dumps shared\nor kept; declaring victory without a second profile.\n","# Profiling: where time and memory go\n\nWithout a profile, optimisation is a guess: timers sprinkled around the\nfunction that looks slow, a debug build profiled, a flame graph's colours\nread as heat and its left-to-right order as time, a library frame\n\"optimised\" when the real question is why the code calls it ten thousand\ntimes. This skill picks the profile for the symptom, gives the commands\nper runtime, and shows how to read the result and turn it into a change.\nMeasuring before and after is in `performance`.\n\n## 1. Pick the profile from the symptom\n\n| Symptom | Look with |\n|---|---|\n| A core at 100%, slow | CPU sampling profile (on-CPU) |\n| Slow, but the CPU mostly idle | wall-clock or off-CPU profile, tracing spans, lock and I\u002FO profiles |\n| Memory grows or is too high | heap snapshots over time, heap profile (`performance-memory`) |\n| High GC time, p99 spikes | allocation profile, GC logs |\n| Threads or tasks waiting on each other | lock, mutex or block profile; thread dumps |\n| A slow query | EXPLAIN and the database's statement statistics (`database-queries`) |\n| A janky page or a slow interaction | the browser's Performance panel (`frontend-performance`) |\n\nA minute with the operating system tells you the row: on Linux `top -H`,\n`vmstat 1` and `iostat -x 1`; anywhere, `time \u003Ccommand>`, where `real` far\nabove `user + sys` means the process waits (a CPU profile will look empty).\n\n## 2. Sampling, instrumenting and tracing\n\n- **Sampling** records stacks at a fixed rate: 99 Hz in production (an odd\n  rate avoids lockstep with periodic work), 1,000 Hz or more locally. The\n  overhead is low and even across functions, so it is safe in production;\n  it is statistical, so collect thousands of samples (30 to 60s under\n  load) and expect it to miss short, rare events.\n- **Instrumenting** wraps every call (cProfile, manual timers, spans):\n  exact counts, but overhead grows with the number of calls, so small hot\n  functions look worse than they are. Use it for counts, not proportions.\n- **Tracing** records the timeline of one run or request: order,\n  concurrency and gaps (the Performance panel, `go tool trace`,\n  OpenTelemetry spans, JFR events). A profile aggregates; a trace shows\n  sequence and waiting.\n\n## 3. On-CPU, off-CPU and wall-clock\n\nA CPU profile shows only time spent running: time blocked on a socket, a\ndisk, a lock, a pool or the database is invisible in it. Use instead:\n\n- Wall-clock profiles, which sample every thread, running or not: py-spy\n  with `--idle`, async-profiler `-e wall`, pyinstrument (wall-clock by\n  default), fgprof for Go.\n- Off-CPU analysis on Linux: bcc's `offcputime` records where threads\n  blocked and for how long, drawn as a flame graph (section 5).\n- The runtime's own waiting views: Go's block and mutex profiles, JFR\n  monitor and park events, async-profiler `-e lock`, tokio-console for\n  async tasks, Node's event loop delay; and spans, for the gaps and the\n  calls made in series that could run concurrently (`performance-backend`).\n\n## 4. Reading a flame graph\n\n- Each box is a function; the boxes above it are what it called. Width is\n  the share of samples in it and everything it called (total time); height\n  is only stack depth.\n- In a classic flame graph the x-axis is sorted alphabetically to merge\n  stacks, so left to right is not time; in a flame chart (the Performance\n  panel, speedscope's Time Order view) it is. Colour is random or by kind\n  of code, not heat, unless the legend says otherwise.\n- **Self time** is the top edge of a box not covered by its children: the\n  function itself did the work. A wide box with narrow children is hot\n  itself; a wide box whose width is all children is only on the path.\n- Find the wide plateaus at the top, then walk down the stack to the first\n  frame in your own code: that is where the decision to do the work was\n  made, and usually what you can change.\n- One function in many thin towers under different callers adds up\n  unseen: a bottom-up view (DevTools Bottom-Up, flamegraph.pl `--reverse`,\n  speedscope's Sandwich) ranks functions by self time across all callers.\n  Compare before and after with a diff (`difffolded.pl`, Pyroscope,\n  `pprof -diff_base`).\n- `[unknown]` frames and broken stacks: build with symbols and frame\n  pointers (`-fno-omit-frame-pointer`, Rust `-C force-frame-pointers=yes`)\n  or unwind with DWARF (`perf record --call-graph dwarf`); JIT runtimes\n  need a perf map (`node --perf-basic-prof`). Inlined functions vanish\n  into their callers; line views (pprof `list`, `perf annotate`) find them.\n\n## 5. The tools, by runtime\n\nProfile an optimised build with a realistic workload running: never a\ndebug build, a development server or an idle process.\n\n**Node.js**\n\n```bash\nnode --cpu-prof --cpu-prof-dir=.\u002Fprof app.js   # open in DevTools or with npx speedscope\nnode --heap-prof app.js                         # sampled allocations, .heapprofile\nnode --inspect app.js                           # chrome:\u002F\u002Finspect: Performance, Memory\n```\n\n- `--cpu-prof` writes on a normal exit: a server needs a `SIGINT` handler\n  that calls `process.exit()`. clinic.js (`clinic flame`) and 0x draw a\n  flame graph in one command; check they support the project's Node first.\n- A blocked event loop is CPU work on the main thread. Measure it with\n  `perf_hooks.monitorEventLoopDelay()`: a p99 in the tens of milliseconds\n  means every request queues behind that work; move it to `worker_threads`\n  or a job.\n\n**Browsers**: the Performance panel with CPU throttling (4x or 6x), then\nthe Main track (a flame chart), Bottom-Up for self time, and long tasks\n(over 50ms, flagged red). For a slow interaction, INP attribution splits\ninput delay, processing and presentation; the `web-vitals` attribution\nbuild and the Long Animation Frames API report it from real users. React\nDevTools' Profiler shows which components rendered and why.\n\n**Python**\n\n```bash\npy-spy top --pid \u003Cpid>                                  # live, no restart\npy-spy record -o prof.svg --pid \u003Cpid> --duration 30     # add --idle for wall-clock\npy-spy dump --pid \u003Cpid>                                 # every thread's stack, for a hang\npython -m cProfile -o out.prof app.py && snakeviz out.prof\npyinstrument app.py                                     # wall-clock call tree, async-aware\nscalene app.py                                          # Python, native and system time per line\n```\n\npy-spy attaches without code changes (sudo on macOS, `SYS_PTRACE` in a\ncontainer); `--native` adds C extension frames, `--subprocesses` follows\ngunicorn or multiprocessing workers. cProfile instruments: right for call\ncounts, inflated for small functions. line_profiler (`kernprof -l -v`)\ntimes each line of functions marked `@profile`; Python 3.12+ works with\nLinux perf via `python -X perf`.\n\n**Go**: register the handlers with `import _ \"net\u002Fhttp\u002Fpprof\"` and serve\nthem on a private port, `go http.ListenAndServe(\"localhost:6060\", nil)`. If\nthe public server uses `http.DefaultServeMux`, the import exposes them there.\n\n```bash\ngo tool pprof -http=:8081 'http:\u002F\u002Flocalhost:6060\u002Fdebug\u002Fpprof\u002Fprofile?seconds=30'\ngo tool pprof -http=:8081 http:\u002F\u002Flocalhost:6060\u002Fdebug\u002Fpprof\u002Fheap     # in use\ngo tool pprof -http=:8081 http:\u002F\u002Flocalhost:6060\u002Fdebug\u002Fpprof\u002Fallocs   # all allocations\ncurl -o trace.out 'http:\u002F\u002Flocalhost:6060\u002Fdebug\u002Fpprof\u002Ftrace?seconds=5' && go tool trace trace.out\ngo test -run='^$' -bench=Parse -cpuprofile cpu.out -memprofile mem.out .\u002Fparser\n```\n\n- Block and mutex profiles stay empty until enabled with\n  `runtime.SetBlockProfileRate(n)` and `runtime.SetMutexProfileFraction(n)`\n  (a smaller `n` records more and costs more).\n- In pprof: `top` (self), `top -cum` (total), `list Func` (per line), the\n  flame graph under View. `go tool trace` shows scheduler latency, GC\n  pauses, syscalls and blocked goroutines. A production CPU profile saved\n  as `default.pgo` in the main package drives PGO (Go 1.21+).\n\n**Rust**\n\n```bash\nCARGO_PROFILE_RELEASE_DEBUG=true cargo flamegraph --bin app -- \u003Cargs>   # perf or dtrace\nsamply record .\u002Ftarget\u002Frelease\u002Fapp \u003Cargs>                              # Firefox Profiler UI\nperf record -F 99 --call-graph dwarf .\u002Ftarget\u002Frelease\u002Fapp\nperf script | inferno-collapse-perf | inferno-flamegraph > flame.svg\n```\n\nRelease builds with symbols (`debug = \"line-tables-only\"` under\n`[profile.release]`, or the variable above). Heap: the `dhat` crate or\nheaptrack. Async: tokio-console (`console-subscriber`, built with\n`RUSTFLAGS=\"--cfg tokio_unstable\"`) shows tasks that poll too long.\n\n**JVM**\n\n```bash\nasprof -d 30 -e cpu -f cpu.html \u003Cpid>      # async-profiler 3+; also -e wall, alloc, lock\njcmd \u003Cpid> JFR.start duration=60s filename=rec.jfr settings=profile\njfr view hot-methods rec.jfr                # JDK 21+, or open rec.jfr in JDK Mission Control\njcmd \u003Cpid> Thread.print                     # thread dumps: three, 5s apart, for a hang\n```\n\nasync-profiler and JFR sample without safepoint bias (thread-dump\nsamplers blame the wrong lines). JFR costs about 1% (default) to 2%\n(`profile`), cheap enough for production. GC detail: `-Xlog:gc*`.\n\n**.NET**\n\n```bash\ndotnet-counters monitor -p \u003Cpid> --counters System.Runtime   # CPU, GC, allocation, thread pool\ndotnet-trace collect -p \u003Cpid> --duration 00:00:00:30 --format Speedscope\ndotnet-gcdump collect -p \u003Cpid>                                # heap by type\n```\n\nPerfView (Windows), Visual Studio and dotTrace go deeper. Thread pool\nstarvation (usually sync over async: `.Result`, `.Wait()`) shows as thread\ncount and queue length rising while CPU stays low.\n\n**Native code and the whole machine**\n\n```bash\nperf stat -d .\u002Fapp                                   # cycles, IPC, cache and branch misses\nperf record -F 99 -g -p \u003Cpid> -- sleep 30 && perf report\nperf script | stackcollapse-perf.pl | flamegraph.pl > cpu.svg\noffcputime-bpfcc -df -p \u003Cpid> 30 > off.stacks        # bcc; offcputime on some distributions\nflamegraph.pl --color=io --countname=us \u003C off.stacks > off.svg\nbpftrace -e 'profile:hz:99 \u002Fpid == 1234\u002F { @[ustack] = count(); }'\n```\n\n- In containers and VMs, perf needs `kernel.perf_event_paranoid` lowered\n  or `CAP_PERFMON`; hardware counters may be missing.\n- macOS: Instruments (Time Profiler, Allocations) or `sample \u003Cpid> 10`.\n  VTune and AMD uProf add hardware detail; Valgrind's callgrind counts\n  every instruction, tens of times slower.\n- IPC under about 1 with many cache misses means the loop waits on memory,\n  not arithmetic: the fix is data layout (`systems-memory`).\n\n**Databases** profile themselves (`database-queries`):\n\n```sql\n-- Postgres, with pg_stat_statements in shared_preload_libraries\nSELECT calls, round(total_exec_time) AS total_ms,\n       round(mean_exec_time::numeric, 1) AS mean_ms, rows, query\nFROM pg_stat_statements ORDER BY total_exec_time DESC LIMIT 20;\n```\n\nRank by total time, not mean: a 2ms query run 50,000 times an hour costs\nmore than a 3s report run twice. Then `EXPLAIN (ANALYZE, BUFFERS)` on the\ntop ones. MySQL: the slow query log and `performance_schema` digests.\n\n## 6. Profiling in production, safely\n\n- Sample, never instrument: 99 Hz for 30 to 60 seconds on one instance,\n  out of the load balancer when the tool is heavy. Continuous profilers\n  (Grafana Pyroscope, Parca with eBPF, Datadog and other APMs) keep a\n  low-rate profile of every instance, so a bad hour can be compared with a\n  good one afterwards.\n- Profiling endpoints (`\u002Fdebug\u002Fpprof`, the Node inspector, JMX) are never\n  public: localhost or an internal network behind auth. An open inspector\n  port is remote code execution.\n- Heap dumps hold whatever was in memory (tokens, passwords, personal\n  data): store them like secrets, never attach them to tickets or chats,\n  delete them after. They pause the process and can need as much memory\n  again as the heap: take them from an instance out of rotation.\n\n## 7. From a profile to a change\n\nTake the top five frames by self time and by total time; for each, find\nwhy it is called (walk down to your code) and how often, then ask in order:\n\n1. **Avoid it?** Results never used, log lines formatted at a disabled\n   level, validation repeated at every layer, data fetched just in case.\n2. **Do it once?** Hoisted out of the loop, per request instead of per\n   item, memoised, cached (`backend-caching`), precomputed on write.\n3. **Batch it?** One query or bulk call for N items (`performance-backend`).\n4. **Make it cheaper?** A better algorithm or data structure, fewer\n   allocations and copies (`performance-memory`), a measured faster library.\n5. **Move it?** To a job (`backend-jobs`), a worker thread, or later.\n6. **Parallelise it?** Only when the work is independent and cores sit idle.\n\nUsual findings: JSON parsing and serialisation of large payloads; regular\nexpressions compiled in a loop; strings built by repeated concatenation;\nORM hydration of thousands of objects; logging; reflection; exceptions as\ncontrol flow; the same data sorted repeatedly; lock contention; GC driven\nby a high allocation rate. Password hashing (argon2, bcrypt) is slow on\npurpose: protect it with rate limits, never lower its cost.\n\nThen profile again under the same workload: the plateau is narrower,\nnothing new has grown. Keep both profiles with the report.\n\n## Check it\n\n- The profile came from an optimised build under a realistic workload,\n  with thousands of samples and resolved symbols (no wall of `[unknown]`),\n  and its type matched the symptom (wall-clock or off-CPU when idle).\n- The change targets a frame that was wide, a second profile shows it\n  narrower, and the end-to-end measurement confirms it (`performance`).\n\n## Avoid\n\nOptimising without a profile; profiling a debug build, an idle process or\na toy input; reading a flame graph's x-axis as time or its colours as heat;\na CPU profile to explain a service that waits; instrumenting profilers for\nproportions; tuning a library frame instead of asking why your code calls\nit; pprof or inspector ports reachable from the internet; heap dumps shared\nor kept; declaring victory without a second profile.\n",{"name":5,"description":7},"main","38293fe1b096e6bf3646c7344c38786ed688f464","crates\u002Fenowx-core\u002Fskills\u002Fperformance-profiling",21,"https:\u002F\u002Fgithub.com\u002Fenowdev\u002Fenowxcli","https:\u002F\u002Fenowx.ai","https:\u002F\u002Fgithub.com\u002Fenowdev","Pretending to be a Developer","2026-10-06 02:05:35",{"source":8,"skillName":5,"commands":58,"prompt":68,"zipAvailable":69},[59,62,65],{"tool":60,"command":61},"npx","npx skills add enowdev\u002Fenowxcli --skill performance-profiling",{"tool":63,"command":64},"bunx","bunx skills add enowdev\u002Fenowxcli --skill performance-profiling",{"tool":66,"command":67},"pnpm","pnpm dlx skills add enowdev\u002Fenowxcli --skill performance-profiling","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fenowdev\u002Fenowxcli\u002Fblob\u002Fmain\u002Fcrates\u002Fenowx-core\u002Fskills\u002Fperformance-profiling\u002FSKILL.md",true,[],[72,88,111,127,151,172],{"skillNo":73,"name":74,"nameZh":6,"description":75,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":76,"categories":77,"keywords":83,"publishedAt":43},"01M6D2HQYSQD9N7MTNDT4A7CBM","demo-skill","Project-scope demo skill for tests.","https:\u002F\u002Fgithub.com\u002Fenowdev\u002Fenowxcli\u002Fblob\u002Fmain\u002Fworkspace\u002F.agents\u002Fskills\u002Fdemo-skill\u002FSKILL.md",[78],{"categoryNo":79,"categoryName":80,"taxonomyType":81,"taxonomyTypeDesc":82},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[84,85,86,87,33],"demo skill","demo","tests","project",{"skillNo":89,"name":90,"nameZh":6,"description":91,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":92,"categories":93,"keywords":100,"publishedAt":43},"01M6D2HM0GCPM0TM2HPG9HHR2R","writing","Writing words people read: the specific thing instead of claims, words to drop, sentences, interface copy (buttons, labels, errors, empty states, confirmations), page copy, notifications and emails, alt text and link text, numbers and dates, words that translate well, documentation and voice. Read before writing a page's copy, an interface's text or a document.","https:\u002F\u002Fgithub.com\u002Fenowdev\u002Fenowxcli\u002Fblob\u002Fmain\u002Fcrates\u002Fenowx-core\u002Fskills\u002Fwriting\u002FSKILL.md",[94,97],{"categoryNo":95,"categoryName":96,"taxonomyType":81,"taxonomyTypeDesc":82},"01JBSEEDCAT000000000000018","写作与文案",{"categoryNo":98,"categoryName":99,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000036","内容创作者",[90,101,102,103,104,105,33,106,107,108,109,110],"text","emails","empty","drop","document","page","voice","notifications","documentation","translate",{"skillNo":112,"name":113,"nameZh":6,"description":114,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":115,"categories":116,"keywords":120,"publishedAt":43},"01M6D2HG07YS49NM196Q0R3AZ8","ui","Designing or restyling an interface: direction, layout, type, colour, icons, components, states, responsive behaviour. Read before building a page, a screen or a set of components.","https:\u002F\u002Fgithub.com\u002Fenowdev\u002Fenowxcli\u002Fblob\u002Fmain\u002Fcrates\u002Fenowx-core\u002Fskills\u002Fui\u002FSKILL.md",[117],{"categoryNo":118,"categoryName":119,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000032","设计师",[113,121,122,123,124,106,125,33,126],"icons","components","SET","responsive","layout","before building",{"skillNo":128,"name":129,"nameZh":6,"description":130,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":131,"categories":132,"keywords":137,"publishedAt":43},"01M6D2HB59HZHWG1RYZDV4SJ91","ui-themes","Light and dark themes, and colour that holds in both: tokens with a value per theme, and what each kind of content needs to stay visible (icons, SVG drawings and line art, logos, photos, screenshots, charts, code, maps, borders, shadows, focus, form controls), switching without a flash, and checking both. Read before building a second theme or a theme toggle, and before any drawing, logo or chart on a page that has two themes.","https:\u002F\u002Fgithub.com\u002Fenowdev\u002Fenowxcli\u002Fblob\u002Fmain\u002Fcrates\u002Fenowx-core\u002Fskills\u002Fui-themes\u002FSKILL.md",[133,136],{"categoryNo":134,"categoryName":135,"taxonomyType":81,"taxonomyTypeDesc":82},"01JBSEEDCAT000000000000015","视觉设计",{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[113,138,139,140,141,142,143,144,145,33,126,146,106,38,147,148,121,149,150],"ui themes","switching","photos","tokens","drawing","focus","maps","Art","flash","Charts","content","screenshots","logo",{"skillNo":152,"name":153,"nameZh":6,"description":154,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":155,"categories":156,"keywords":160,"publishedAt":43},"01M6D2H5R5KRAE524C8QX0T7Q0","ui-structure","Structure before styling: the blueprint written before any interface code (the screen's job, the regions its kind of page needs, their order and why, the layout skeleton in numbers, the components and their states), the anatomy and order of a homepage, a dashboard, a list, a detail page, settings, a form, sign-in, pricing, docs, an article and a checkout, and how to turn the blueprint into code (tokens, layout primitives, components, sections, page). Read first, before building or rebuilding any page or screen.","https:\u002F\u002Fgithub.com\u002Fenowdev\u002Fenowxcli\u002Fblob\u002Fmain\u002Fcrates\u002Fenowx-core\u002Fskills\u002Fui-structure\u002FSKILL.md",[157,158,159],{"categoryNo":95,"categoryName":96,"taxonomyType":81,"taxonomyTypeDesc":82},{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":98,"categoryName":99,"taxonomyType":21,"taxonomyTypeDesc":22},[161,113,162,122,163,164,38,165,166,167,125,141,168,169,170,33,126,106,171,35],"structure","ui structure","pricing","dashboard","Blueprint","article","anatomy","styling","primitives","list","why",{"skillNo":173,"name":174,"nameZh":6,"description":175,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":176,"categories":177,"keywords":185,"publishedAt":43},"01M6D2H0ZZBJG77MBGPZHA6MWD","ui-stack-vue","Building interfaces in Vue 3 and Nuxt: project layout, script setup with typed defineProps, defineEmits and defineModel, attribute fallthrough, slots and scoped slots, template refs, ref, reactive and computed, watch versus watchEffect, composables and VueUse, provide and inject, Pinia, keyed v-for, transitions, Nuxt 4 (app directory, pages, layouts, useFetch and useAsyncData keys, $fetch in handlers, server routes, runtime config, useSeoMeta, NuxtImg, middleware), VeeValidate forms, Reka UI and Nuxt UI, testing, vue-tsc, and the traps. Read when the project uses Vue or Nuxt.","https:\u002F\u002Fgithub.com\u002Fenowdev\u002Fenowxcli\u002Fblob\u002Fmain\u002Fcrates\u002Fenowx-core\u002Fskills\u002Fui-stack-vue\u002FSKILL.md",[178,181,184],{"categoryNo":179,"categoryName":180,"taxonomyType":81,"taxonomyTypeDesc":82},"01JBSEEDCAT000000000000008","测试与质量",{"categoryNo":182,"categoryName":183,"taxonomyType":81,"taxonomyTypeDesc":82},"01JBSEEDCAT000000000000009","前端开发",{"categoryNo":118,"categoryName":119,"taxonomyType":21,"taxonomyTypeDesc":22},[113,186,187,188,189,190,191,192,125,87,193,194,33,195,196,197,198,199,200,201],"vue","stack","ui stack vue","forms","pages","watch","runtime","transitions","fetch","pinia","nuxt","template","server","config","directory","testing"]