# Bandwidth The **maximum rate** at which data can move across a channel, measured in bytes (or bits) per second. It is the *width of the pipe*, not the speed of any single drop of water through it. ==Bandwidth is a ceiling, not a measurement.== It is what the channel *could* deliver under ideal conditions. What you *actually* get is [[Throughput]]. Confusing the two is the single most common mistake in performance reasoning. ## Why You Should Care: Sizing the Wrong Pipe You are moving a 100 GB dataset from CPU memory to a GPU for training. The [[PCIe]] 4.0 x16 link is rated at ~64 GB/s. ==Naive math: 100 GB ÷ 64 GB/s ≈ 1.6 seconds. "The transfer is basically free."== Then you measure it. It takes **8 seconds**. Five times slower than the spec. The rated number is **bandwidth** (the theoretical ceiling). The 8-second reality is **[[Throughput]]** (what survived protocol overhead, small-transfer inefficiency, contention with other devices on the bus, and the CPU not feeding the link fast enough). The pipe is 64 GB/s wide; you were only pushing ~12 GB/s of water through it. ==Bandwidth tells you the best case you can never exceed. It tells you nothing about the case you will actually hit.== Knowing the difference is the whole game in systems performance: you size hardware by bandwidth, but you debug performance by throughput. ## Prerequisites | # | Prerequisite | What you need from it | |---|--------------|------------------------| | 1 | [[Bus (computing)]] | The physical channel bandwidth is a property of | | 2 | [[Throughput]] | The *achieved* rate, vs bandwidth's *maximum* rate | | 3 | [[Latency]] | Delay per operation, the orthogonal axis to rate | | 4 | [[Memory Bandwidth]] | The most important specialization (CPU/GPU to memory) | **Reading order if starting from scratch:** 1. [[Bus (computing)]] — the channel that has a width 2. **This note** — the width of that channel (the ceiling) 3. [[Throughput]] — what you actually get out of it 4. [[Latency]] — why a wide pipe can still feel slow ## Ontology — Where This Sits **Entity type**: Concept (performance metric) ### Is-A (Parents) ``` Entity └── Performance Metric └── Rate Metric (per unit time) └── Bandwidth (maximum data rate of a channel) ``` ### Lateral Peers (Siblings) - [[Throughput]] — achieved rate, not theoretical max (bandwidth ≥ throughput, always) - [[Latency]] — delay per operation, not rate (orthogonal axis) - [[QPS]] — throughput counted in requests, not bytes - [[FLOPS]] — rate of *compute*, not *data movement* ### Composition (Has-A) **Two knobs** (for a digital bus): bus **width** (parallel wires) and **clock** (toggle rate). See [[Memory Bandwidth]]. **Units**: bits/s or bytes/s (and originally Hz, see below). **Specializations**: [[Memory Bandwidth]], network bandwidth, disk/IO bandwidth. ### What Makes This Distinct Bandwidth is the *capacity* of a channel: a fixed, advertised property. Its peers measure what *happens* on the channel ([[Throughput]]), how *long* one trip takes ([[Latency]]), or the rate of a *different* resource ([[FLOPS]] for compute). A channel has exactly one bandwidth; it has a different throughput every second. ## The Two Meanings: Hz vs Bytes/sec The word is borrowed from radio, and the borrowing is why "bandwidth" sometimes means Hertz and sometimes means bytes per second. | Era | Meaning | Unit | Example | |---|---|---|---| | Original (signal processing) | **Width of a frequency band** | Hz | An FM station occupies 200 kHz of spectrum | | Modern (computing, colloquial) | **Data rate of a channel** | bits/s or bytes/s | PCIe 4.0 x16 = 64 GB/s | The link between them is the **Shannon-Hartley theorem**: channel capacity (bits/s) is proportional to the frequency bandwidth (Hz). A wider frequency band carries more bits per second. Over decades, engineers collapsed the cause (Hz) into the effect (bits/s), and now "bandwidth" almost always means data rate in computing contexts. ==So "bandwidth" literally means *width of a band of frequencies*. The bytes/sec meaning is a downstream consequence that ate the original.== The Korean `대역폭` (帯域幅) preserves this: 帯域 = "band/zone", 幅 = "width". Literally "band width". ## Formula For a digital bus, the bytes/sec bandwidth is: ``` bandwidth = bus_width_bits × clock_Hz × transfers_per_cycle / 8 ``` Two knobs: **width** (how many wires in parallel) and **clock** (how fast each wire toggles). You scale bandwidth far more cheaply by *widening* than by *clocking faster*, which is why [[HBM]] beats DDR by being 16x wider, not faster. (Full treatment in [[Memory Bandwidth]].) For a communication channel, the ceiling instead comes from Shannon-Hartley: `capacity = bandwidth_Hz × log2(1 + SNR)`. ## Bandwidth vs Throughput vs Latency These three get conflated constantly. They are independent axes. | Metric | Question it answers | Analogy (highway) | Unit | |---|---|---|---| | **Bandwidth** | How wide is the channel? (max possible) | Number of lanes | bytes/s (max) | | [[Throughput]] | How much actually got through? | Cars/hour actually observed | bytes/s (achieved) | | [[Latency]] | How long for one item to traverse? | Minutes for one car end-to-end | seconds | Key relationships: - **Bandwidth ≥ Throughput, always.** Throughput is bandwidth minus all real-world losses (protocol overhead, contention, small transfers, idle gaps). - **Bandwidth and latency are independent.** A satellite link can have huge bandwidth (a fat pipe) *and* terrible latency (600 ms round trip). ==A wide pipe does not make a short pipe.== - **The "fat slow pipe" problem**: a high-bandwidth, high-latency link needs a large in-flight window (bandwidth-delay product) to stay full, or it sits mostly idle waiting on round trips. ## Where Bandwidth Shows Up Every data path has a bandwidth, and the *narrowest* one gates the whole system (see [[Bandwidth Not FLOPS Is the Multi-GPU Bottleneck (개소리)]]). | Domain | Channel | Typical bandwidth | |---|---|---| | GPU internal memory | [[HBM]]2e (8-stack) | ~2 TB/s | | GPU to GPU | [[NVLink]] 3.0 | ~600 GB/s | | CPU main memory | DDR4 (4-channel) | ~200 GB/s | | CPU to GPU | [[PCIe]] 4.0 x16 | ~64 GB/s | | NVMe SSD read | [[NVMe]] (PCIe 4.0 x4) | ~7 GB/s | | Datacenter network | 100 GbE | ~12.5 GB/s | | Home internet | 1 Gbps fiber | ~0.125 GB/s | Note the seven orders of magnitude. ==Moving a byte the short distance (inside a chip) is thousands of times cheaper than moving it the long distance (across a network).== This hierarchy is the foundation for [[Model Parallelism]] and data-placement decisions. ## Why It Matters In any compute system, the question is rarely "is there enough compute?" ([[FLOPS]]). It is almost always "which channel is the narrowest pipe?" Compute units stall the instant data cannot arrive fast enough. Understanding bandwidth, and crucially distinguishing the *ceiling* (bandwidth) from the *reality* ([[Throughput]]), is the prerequisite for reasoning about where any system actually spends its time. ## Related Notes [[Memory Bandwidth]] | [[Throughput]] | [[Latency]] | [[Bus (computing)]] | [[PCIe]] | [[HBM]] | [[NVLink]] | [[NVMe]] | [[FLOPS]] | [[Model Parallelism]] | [[QPS]]