AGENT SKILL

uarch-gpu-npu-dataflow-and-numeric-formats

Use for accelerators: GPU microarchitecture including warps, occupancy and divergence, GPU memory systems with coalescing and the bandwidth hierarchy, NPUs and dataflow architectures such as systolic arrays and why they suit tensor workloads, and numeric formats from FP32 down to FP8 and the block-scaled formats with their accuracy and throughput trade-offs.

License 许可
MIT
最近更新
2026-11-15
Stars
★ 5
Forks
3

01 前言 Front Matter

name
uarch-gpu-npu-dataflow-and-numeric-formats
description
Use for accelerators: GPU microarchitecture including warps, occupancy and divergence, GPU memory systems with coalescing and the bandwidth hierarchy, NPUs and dataflow architectures such as systolic arrays and why they suit tensor workloads, and numeric formats from FP32 down to FP8 and the block-scaled formats with their accuracy and throughput trade-offs.

03 SKILL.md 正文

Microarchitecture and Memory: GPU Microarchitecture, GPU Memory Systems, NPUs and Dataflow Architectures, and Numeric Formats

Part 3 of 6 of the CPU, GPU, NPU and Memory Microarchitecture reference (plugin microarchitecture-and-memory-systems), covering §11–§14. Sibling skills: uarch-pipelining-out-of-order-branch-prediction-and-simd (§0–§5), uarch-caches-coherence-consistency-and-virtual-memory (§6–§10), uarch-dram-memory-controllers-power-and-security (§15–§19), uarch-isa-simulation-measurement-roofline-and-specialization (§20–§24), uarch-reference (§25–§30). Section numbers are shared across the set; a reference written as §N → skill points into that sibling skill.

Currency: The principles are stable. Two areas moved. See §25 → uarch-reference for low-precision numeric formats, and next-generation memory standards.

⚠️ SCOPE, because this sits between two neighbours. ⚠️ A semiconductor reference covers device physics and fabrication — how a transistor works and how it's made. A computer-hardware reference covers system integration — components, builds, facilities. ⚠️ THIS file is the layer in between: how billions of transistors are ORGANIZED into something that executes instructions fast, and why memory systems are the hard part. ⚠️ It assumes the other two rather than repeating them.

⚠️ GOTCHA boxes mark where architectural intuition is wrong and where the abstraction leaks.

The three ideas that organize this document:

  1. ⚠️ THE PROCESSOR LIES TO YOU, DELIBERATELY (§2–§4 → uarch-pipelining-out-of-order-branch-prediction-and-simd). Instructions do not execute in program order, memory operations do not become visible in program order, and the architecture works hard to maintain the illusion that they do. Every performance surprise and several security disasters live in the gap.
  2. ⚠️ MOVING DATA COSTS FAR MORE THAN COMPUTING ON IT (§6 → uarch-caches-coherence-consistency-and-virtual-memory, §13, §23 → uarch-isa-simulation-measurement-roofline-and-specialization). Energy per arithmetic operation has fallen enormously; energy per byte moved has not. Modern architecture is organized around minimizing data movement, and that single fact explains caches, dataflow accelerators, quantization and HBM alike.
  3. ⚠️ PARALLELISM IS EXTRACTED AT EVERY LEVEL, AND EACH HAS A DIFFERENT LIMIT (§3 → uarch-pipelining-out-of-order-branch-prediction-and-simd, §11, §14). ILP is limited by dependencies and branches, DLP by divergence and memory, TLP by synchronization and coherence traffic. Knowing which one you're hitting is the entire diagnostic skill.

§11. GPU Microarchitecture

⚠️ THE ORGANIZING PRINCIPLE  ⚠️ hide latency with PARALLELISM
   rather than with caches and speculation. ⚠️ When a warp stalls
   on memory, the scheduler switches to another — so latency is
   tolerated rather than reduced
⚠️ THE HIERARCHY  ⚠️ SM / CU containing warp schedulers, register
   file, SIMD lanes, ⚠️ SHARED MEMORY / LDS (⚠️ software-managed
   scratchpad — the key GPU-specific resource), L1, tensor units
⚠️ SIMT  ⚠️ threads grouped into warps (32) or wavefronts (32/64)
   executing in lockstep
   ⚠️ WARP DIVERGENCE — ⚠️ if threads in a warp take different
   branch paths, the hardware SERIALIZES them. ⚠️ Worst case a
   32-way divergent branch costs 32× — the single biggest GPU
   performance trap
⚠️ ⚠️ OCCUPANCY  active warps per SM, limited by ⚠️ REGISTERS PER
   THREAD, ⚠️ SHARED MEMORY PER BLOCK, and block size.
   ⚠️ GOTCHA: higher occupancy is NOT automatically better —
   beyond the point where latency is hidden, more warps just
   thrash cache. ⚠️ Register-heavy kernels with low occupancy
   frequently outperform "optimized" high-occupancy versions
⚠️ MEMORY COALESCING  ⚠️ consecutive threads accessing consecutive
   addresses combine into one transaction; ⚠️ scattered access
   multiplies the transaction count and is the second big trap
⚠️ BANK CONFLICTS in shared memory ⚠️ serialize access
⚠️ THE REGISTER FILE IS HUGE  ⚠️ larger than the L1 cache — because
   thousands of resident threads each need private state

§12. GPU Memory Systems

⚠️ GDDR versus HBM: ⚠️ GDDR is cheaper with fewer, faster pins; ⚠️ HBM uses stacked dies and an extremely wide interface (§15 → uarch-dram-memory-controllers-power-and-security, and see a semiconductor reference §20) for far higher bandwidth per watt at much higher cost. ⚠️ Bandwidth is the design centre: ⚠️ GPUs are usually bandwidth-bound (§23 → uarch-isa-simulation-measurement-roofline-and-specialization), so arithmetic intensity determines whether you can use the compute at all. ⚠️ Caches on GPUs behave differently: ⚠️ smaller per thread, and used more for coalescing and reuse across warps than for latency reduction. ⚠️ Unified/managed memory simplifies programming and ⚠️ can hide expensive migration — convenience that costs performance silently. ⚠️ Multi-GPU: ⚠️ NVLink and Infinity Fabric provide far higher bandwidth than PCIe, and collective operations make the interconnect part of the compute path (see a computer-hardware reference §21).


§13. ⚠️ NPUs and Dataflow Architectures

⚠️ The least-covered area elsewhere, and the design logic is genuinely different from both CPU and GPU.

⚠️ THE PREMISE  ⚠️ neural network inference and training are
   dominated by MATRIX MULTIPLY, which has enormous, REGULAR,
   STATICALLY KNOWN parallelism. ⚠️ You do not need speculation,
   branch prediction or coherence — so remove them and spend
   the area on arithmetic and local memory
⚠️ ⚠️ THE ENERGY ARGUMENT IS THE REAL ONE  ⚠️ a multiply-accumulate
   costs far less energy than fetching its operands from DRAM.
   ⚠️ Therefore the architecture is designed around DATA REUSE,
   not around arithmetic throughput
⚠️ SYSTOLIC ARRAYS  ⚠️ a grid of MAC units where data flows
   rhythmically between neighbours — ⚠️ each value fetched once
   and reused across the array. Google's TPU is the canonical
   modern example
⚠️ DATAFLOW TAXONOMY  ⚠️ weight-stationary (keep weights in place,
   stream activations) · output-stationary (accumulate in place) ·
   row-stationary. ⚠️ The choice depends on layer shape, and
   flexible accelerators support several
⚠️ THE MEMORY HIERARCHY IS EXPLICIT AND SOFTWARE-MANAGED —
   ⚠️ no transparent caches. ⚠️ TILING and scheduling are the
   compiler's job, and the compiler is most of the product
⚠️ SPARSITY  ⚠️ structured (e.g. 2:4) is hardware-exploitable;
   unstructured sparsity is much harder to convert into speedup
⚠️ COMPUTE-IN-MEMORY  research direction — do the MAC where the
   data lives, attacking the movement cost directly
⚠️ EDGE NPUs  ⚠️ optimized for INT8 and now INT4/FP4 (§25.1),
   with fixed function blocks and very tight power budgets

§14. ⚠️ Numeric Formats

⚠️ Precision is an architectural parameter now, not a given.

⚠️ IEEE 754  FP64, FP32, ⚠️ FP16 (5 exponent bits — limited RANGE,
   which causes training overflow)
⚠️ BF16  ⚠️ FP32's 8 exponent bits with fewer mantissa bits —
   ⚠️ same RANGE as FP32, less precision. ⚠️ This is why BF16
   largely displaced FP16 for training: range mattered more
   than mantissa
⚠️ FP8  E4M3 (more precision) and E5M2 (more range) — typically
   used together for forward and backward passes
⚠️ INTEGER  INT8, INT4 — with quantization scale and zero-point
⚠️ ⚠️ BLOCK FLOATING POINT / MICROSCALING  ⚠️ the key modern idea:
   a group of low-precision values SHARES a higher-precision
   scale factor, recovering dynamic range at almost no bit cost
   (§25.1)
⚠️ THE TRADE  ⚠️ halving bit width roughly halves memory, memory
   BANDWIDTH and energy, and can more than double throughput —
   ⚠️ which is why quantization is the highest-leverage
   optimization available for inference
⚠️ WHAT BREAKS  ⚠️ ACTIVATION OUTLIERS — a few very large values
   force a scale that crushes everything else. ⚠️ Hence
   per-channel and per-block scaling, and Hadamard rotations
   that spread outlier energy across dimensions
⚠️ ⚠️ NOT EVERYTHING QUANTIZES  softmax, layer norm and
   accumulation are typically kept at higher precision

PART III — MEMORY SYSTEMS

04 同仓库更多 Skills

the-vibey-project
the-vibey-project/vibey

white-paper-writing

Use when writing white papers, technical reports, policy documents, thought leadership pieces, or any long-form authoritative document. Triggers on requests to write, structure, outline, or review white papers, technical reports, position papers, policy briefs, or evidence-driven arguments. Also use for B2B content marketing strategy, technical documentation requiring authority, academic-style reports, or any document that must educate while persuading. Use when someone asks "how should I structure this" for a long-form document.

★ 5
the-vibey-project
the-vibey-project/vibey

prose-technical-mechanics

Use when editing or writing prose at a technical/mechanical level: choosing between Anglo-Saxon and Latinate diction, placing information in topic vs stress positions, writing periodic vs cumulative sentences, handling sentence rhythm, punctuation as logic, transitions, paragraph structure, or phonesthetics. Use when someone wants to understand WHY prose works or doesn't, not just surface-level style advice. Triggers on word choice, sentence structure, emphasis, rhythm, diction, or any deep writing mechanics question.

★ 5
the-vibey-project
the-vibey-project/vibey

prose-mechanics

Use when writing, editing, or analyzing prose style; when trying to avoid generic AI-sounding writing; when applying specific prose styles such as journalistic, academic, narrative, KJV biblical, Hemingway plain, Latinate/Jamesian, Orwellian, Augustan, Victorian, Early Modern (Bacon, Browne, Milton), or free indirect discourse; or when coaching someone on writing voice and mechanics. Triggers on any request to improve, analyze, or write in a specific prose style, to fix "AI-sounding" writing, to make prose more vivid or concrete, to vary sentence rhythm, or to understand how a particular writer achieves their effects.

★ 5
the-vibey-project
the-vibey-project/vibey

narrative-structure

Use when structuring stories, pitches, presentations, white papers, or any persuasive narrative; analyzing the hero's journey in fiction or non-fiction; applying StoryBrand for business communication; designing narrative arcs for thought leadership; or understanding Vogler, Snyder, Propp, or Campbell frameworks. Triggers on any narrative structure, storytelling, pitch structure, StoryBrand, hero's journey, three-act structure, Freytag, Save the Cat, narrative arc, or "how should I structure this story/pitch/presentation" request.

★ 5
the-vibey-project
the-vibey-project/vibey

legal-brief-writing

Use when writing legal briefs, motions, or any court filing; when analyzing judicial cognition and persuasion; when applying elite brief-writing techniques from Roberts, Clement, Kagan, Garner; or when structuring legal arguments for maximum persuasion. Also use for autistic/neurodivergent advocates who need to understand how their natural communication style maps to elite legal advocacy. Triggers on any legal writing, brief drafting, motion preparation, or legal argument structuring request.

★ 5
the-vibey-project
the-vibey-project/vibey

journal-publishability-criteria

Use whenever a research paper, article or preprint is being written, revised, reviewed or published, and specifically before every publication of docs/paper.md. Covers what legitimate journals actually gate on, in order of gating — the editor's desk screen (scope, novelty, package problems), what reviewers judge (methodological soundness, claims the evidence can carry, statistical and qualitative reporting), EQUATOR reporting guidelines, the ICMJE authorship criteria, COPE's view of text recycling, the near-universal generative-AI policy (no AI author, disclosure, author responsibility), TOP 2025 transparency standards, how to verify a journal is legitimate, submitting without an institution, and a pre-submission checklist. Triggers on requests to evaluate, review, prepare or submit a paper, to judge publishability, to write a declarations or data-availability section, to disclose AI use, or to check a journal.

★ 5