Computer Science &
How Computers Work

A curated set of free, technical resources for learning about computers from the transistor level upward.

📌 Tried hard to avoid linking to resources that force account creation to access material.

The core set — learn the whole stack without opening 150 tabs

The full page is an archive. If you want a coherent route instead, these are the strongest direct-open resources. Read them roughly in this order; skip exercises whenever you do not care about them.

DIRECT PDF1. CMOS transistor theory

MOS capacitor, inversion, VGS/VDS, cutoff/linear/saturation, NMOS/PMOS and parasitic capacitance.

FREE TEXTBOOK2. CMOS gates

See PMOS/NMOS networks turn transistor behavior into inverter, NAND and NOR gates; also compares CMOS with TTL.

OPEN NOTES3. MIT computation structures

No enrollment. The strongest complete bridge from MOS gates to sequential logic, processors, memory and systems.

PUBLIC PROJECT4. Nand2Tetris projects

Actually build NAND → gates → ALU → registers → memory → CPU → assembler. Use the direct self-study material; ignore Coursera.

PUBLIC PROJECT5. Ben Eater 8-bit computer

Makes the clock, registers, ALU, shared bus, RAM, program counter and control unit physically visible.

SCHEMATICS6. Ben Eater schematics

Trace every module electrically rather than relying on block diagrams.

PUBLIC NOTES7. CS61C datapath

A clean modern CPU datapath: fetch, decode, execute, memory, write-back and control.

INTERACTIVE8. Easy 6502

Connect machine instructions to registers, flags, addresses, stack and real memory contents.

DIRECT PDF9. MOS 6500 hardware manual

Original hardware documentation: CPU pins, clocks, address/data buses, timing, RAM/ROM and peripheral interfacing.

DIRECT PDF10. Apple-1 original manual

A complete real computer small enough to inspect: CPU, RAM, PIA, terminal/video logic, power and schematics.

INTERACTIVE11. Visual6502

Execute a 6502 while watching individual transistor-level nodes switch.

TECH ARTICLE12. 74181 ALU die analysis

The important TTL-era ALU taken from package → die → bipolar transistors → gates → ALU circuitry.

TECH ARTICLE13. 8086 microcode engine

Shows exactly how machine instructions select and execute lower-level microinstructions/control operations.

TECH ARTICLE14. SRAM transistor cells

Cross-coupled inverters, 6T SRAM, multi-port register cells, wordlines/bitlines and actual silicon layout.

TECH ARTICLE15. DRAM die analysis

One-transistor DRAM cells, capacitors, sense amplifiers, row/column decoding and refresh.

DIRECT PDF16. CMOS wires / interconnect

Resistance, capacitance, RC delay, crosstalk, repeaters and why physical wires limit real chips.

DIRECT PDF17. Clock distribution

Packaging, power, clock trees/distribution, skew and the physical side of clocks.

DIRECT PDF18. RP2040 hardware design

A real modern board: power rails, decoupling, core regulator, crystal/clock, QSPI flash, USB and routing.

KERNEL DOCS19. Linux DMA guide

How peripherals move data through bus addresses, physical RAM and IOMMUs instead of CPU copying each byte.

FREE BOOK20. OSTEP

Once the hardware makes sense, continue into processes, virtual memory, concurrency, storage and filesystems.

Four sensible routes through this page

You do not need to read this file top to bottom. Pick a route, then branch sideways whenever a term becomes interesting.

I want the complete mental model

CMOS inverter → gates → latches/flip-flops → ALU/registers/bus → memory read cycle → instruction set → complete 6502/Apple-1 or Z80 machine.

I want to build something

NandGame/Nand2Tetris → Ben Eater 8-bit → logic datasheets → address decoding → 6502 or Z80 system → Verilog/FPGA if you want to synthesize it.

I want the silicon underneath

MOS transistor theory → CMOS inverter → fabrication/layout → SRAM cell → Ken Shirriff die analysis → wires/RC delay → power/clock distribution → PLL/metastability.

I want historical computers

Intel 4004/MCS-4 → PDP-8/PDP-11 → Apple-1/6502 → Altair/8080 → Z80 → 68000 → IBM PC → CRAY-1. Compare buses, clocks, memory maps and control.

Mental model: what connects to what

A computer is easier to understand if you keep separate the energy/time infrastructure from the information paths. Power makes the electronics possible; clocks and reset establish orderly state changes; address/data/control paths move information among CPU, memory and devices.

POWER / TIMING

 wall/battery
     |
 regulators -> VDD/VCC rails --------------------------+
     |                                                   |
    GND/VSS reference -----------------------------------+----> every IC
                                                         |
 crystal/oscillator -> clock tree -> flip-flops/registers|
 power-on reset / reset supervisor -> RESET ------------+

INFORMATION PATHS

                 +---------------- CPU ----------------+
                 | PC | registers | ALU | control unit |
                 +-----------+--------------------------+
                             |
               address / data / control
                             |
            +----------------+----------------+
            |                                 |
          RAM / ROM                     device controllers
      instructions + data              GPIO / UART / USB /
            |                           PCIe / display / disk
            |                                 |
            +----------- system fabric ------+

Classic machines may expose literal shared buses. Modern systems often replace them
with point-to-point links and bus fabrics, but the same questions remain: who drives,
who listens, what address selects it, when is data valid, and what clock/reset domain is it in?

When reading any schematic, trace one signal end-to-end. Ask: what drives it, what voltage means 0/1, what enables it, when it is sampled, where the value is stored, and what happens on reset.

Component-by-component cheat sheet

When reading a motherboard or historical computer schematic, identify these blocks first. The implementation changes by era, but the roles and connections recur constantly.

Component / block	What it does	What it normally connects to
Power supply / regulator / VRM	Creates stable voltage rails; CPU VRMs convert a higher rail into low-voltage, high-current core/SoC power.	Power source, inductors/capacitors, VDD/VCC rails, GND/VSS, often control/telemetry signals.
Decoupling capacitor	Supplies local transient current and lowers high-frequency power-distribution impedance.	Placed physically close between an IC's supply pin and ground.
Crystal / oscillator	Provides a periodic frequency reference.	Clock generator/PLL/divider or a chip's oscillator pins.
PLL / clock generator	Multiplies, divides or phase-aligns clocks and creates clock domains.	Reference oscillator, supply/ground, clock outputs, configuration/control.
Reset / supervisor	Keeps logic in a defined startup state until power and clocks are sufficiently valid.	RESET pins, power-good/voltage monitor, watchdog, sometimes firmware-controlled reset lines.
CPU / processor	Fetches/decodes instructions and performs state transitions through registers, ALUs, control logic and memory/I/O transactions.	Power, clocks, reset, memory/interconnect, interrupts, debug.
RAM	Volatile working storage.	Memory controller or address/data/control bus; power; DRAM additionally needs refresh/timing.
ROM / flash	Nonvolatile firmware/program/data storage.	CPU/chipset through parallel or serial bus; power; chip-select/control.
Address decoder	Maps address ranges to particular memory/peripheral chip-select signals.	Address lines in; CS/enable outputs to RAM, ROM and I/O.
Bus transceiver / buffer	Drives, isolates or changes direction between bus segments.	Two buses plus enable/direction controls.
Multiplexer (MUX)	Selects one of several inputs to drive an output.	Candidate datapath sources, select/control bits, destination.
Register / latch	Stores a multi-bit value.	Data input/output, clock/load enable, reset/clear.
ALU	Performs arithmetic, Boolean logic, compares, shifts or related operations.	Operand buses/registers, control/function select, result path, flags.
Program counter	Stores the instruction-fetch address or architecture-defined next-PC state.	Incrementer/adder, address/fetch path, branch/jump selection.
Instruction register / decode	Captures instruction bits and derives operation/control information.	Instruction-fetch data, control unit, immediate/register selection.
Control unit	Sequences datapath actions and memory/I/O control.	Opcode, clock/state, condition flags, register enables, MUX selects, bus enables.
Memory controller	Turns CPU/cache requests into DRAM protocol commands while scheduling banks/rows and refresh.	CPU/cache interconnect, DDR command/address/data bus, DIMMs/DRAM.
Cache	Keeps copies of recently/nearby used memory blocks close to execution units.	CPU pipelines, higher/lower cache levels, memory/coherence interconnect.
MMU / TLB	Translates virtual addresses to physical addresses and enforces page permissions.	CPU load/store/fetch paths, page tables, caches/interconnect.
Interrupt controller	Collects, prioritizes and routes interrupt sources.	Device IRQs/messages, CPU interrupt inputs/CSRs, configuration bus.
DMA engine	Moves blocks between devices and memory without one CPU load/store per transferred word.	RAM/system interconnect, peripheral queues/FIFOs, MMIO control, interrupt completion.
UART	Parallel-byte ↔ asynchronous serial conversion.	CPU/MMIO bus, TX/RX pins, clock, interrupt.
SPI controller	Synchronous serial master/controller using clock, select and data lines.	CPU/MMIO bus, SCLK/MOSI/MISO/CS, peripheral.
I²C/SMBus controller	Addressed shared serial bus using open-drain clock/data.	CPU/MMIO bus, SDA/SCL, pull-ups, sensors/EEPROMs/controllers.
PCIe root complex	Host side of PCI Express hierarchy; routes packets between CPU/memory and endpoints.	CPU/interconnect, PCIe lanes/switches/endpoints, IOMMU.
PCH / chipset	Aggregates platform I/O such as USB, SATA, extra PCIe, SPI/eSPI, SMBus, RTC and GPIO.	Processor link such as DMI, peripherals, firmware flash, board-management devices.
GPU	Highly parallel processor with its own schedulers/execution units/caches; often has dedicated VRAM.	PCIe/coherent fabric, VRAM controllers, display engines, power/clocks.
Storage controller	Implements SATA/NVMe/UFS/other command protocols and moves blocks between storage media and RAM.	PCIe/SATA/etc., DMA/system memory, interrupts, flash/HDD media electronics.
PHY	Physical-layer transceiver converting digital protocol-side data into actual electrical/optical signaling.	MAC/controller side plus connector/cable/medium side.

Signal-name decoder for real schematics

A bar over a signal name, a leading /, a trailing #, or names such as nRESET usually mean active low: the function is asserted when the signal is logic 0.

VDD / VCC	Positive supply rail. VDD is common MOS/CMOS nomenclature; VCC comes from bipolar-transistor conventions.
VSS / GND	Lower/reference supply rail, commonly treated as 0 V.
CLK / CK / φ1 / φ2	Clock signals. φ1 and φ2 commonly indicate separate phases in older MOS designs.
RESET#, /RESET, nRESET	Active-low reset.
A0…An	Address bits; A0 is conventionally the least-significant address bit.
D0…Dn / DQ	Data lines. DQ is common in memory interfaces.
R/W	Read-versus-write direction/control; exact polarity and timing depend on the processor.
RD# / OE#	Active-low read or output-enable.
WR# / WE#	Active-low write or write-enable.
CS# / CE#	Active-low chip-select / chip-enable.
IRQ / INT	Interrupt request/input.
NMI	Non-maskable interrupt on architectures that define one.
READY / WAIT	Allows a target to extend or stall a bus transaction.
DTACK#	68000-style data-transfer acknowledge handshake.
REQ / GNT, BR / BG	Request/grant signals used for arbitration or shared-resource ownership.
ALE / AS	Address-latch-enable or address-strobe; marks valid address phase on certain buses.
MREQ / IORQ	Z80-style distinction between memory and I/O bus cycles.
RAS / CAS	Classic DRAM row/column address strobes; modern DDR signaling is more command-oriented but the historical names survive in timing terminology.
CK / CK#	Differential memory or high-speed reference clock pair in interfaces that use complementary clock signaling.
DQS / DQS#	DDR data-strobe pair used to time DQ data transfers.
SCLK / SCK	Serial clock, commonly SPI.
MOSI / COPI	SPI controller-out/peripheral-in data.
MISO / CIPO	SPI peripheral-out/controller-in data.
CS# / SS#	SPI chip/peripheral select, commonly active low.
SCL / SDA	I²C clock and data; normally open-drain with external pull-up resistors.
TX / RX	Serial transmit and receive.
MDC / MDIO	Ethernet PHY management clock/data interface.
PERST#	PCIe fundamental reset signal.
CLKREQ#	Clock-request signal used by PCIe/platform power-management mechanisms.
PWROK / PWRGOOD	Power-good indication that a supply/platform has reached acceptable conditions.

How to read a real logic datasheet: a worked SN74HC245 method

A datasheet is not a catalog entry. It is the electrical contract between your circuit and the component. For digital logic, learn to separate survival limits, guaranteed operating conditions, DC logic-level guarantees, and AC timing guarantees.

WORKED DEVICE: SN74HC245 OCTAL 3-STATE BUS TRANSCEIVER 1. FUNCTION / BLOCK DIAGRAM A0..A7 ⇄ B0..B7 DIR chooses direction OE# enables outputs; disabled = high impedance 2. PINOUT identify VCC, GND, DIR, OE#, A bus, B bus 3. ABSOLUTE MAXIMUM RATINGS 'do not exceed' stress limits — NOT normal operating targets 4. RECOMMENDED OPERATING CONDITIONS legal VCC, input/output conditions, temperature, edge-rate constraints 5. DC ELECTRICAL CHARACTERISTICS VIH = guaranteed input HIGH threshold condition VIL = guaranteed input LOW threshold condition VOH/VOL = guaranteed output levels at specified source/sink current II / IOZ / ICC = leakage, high-Z leakage and supply-current limits 6. AC / SWITCHING CHARACTERISTICS tPLH / tPHL = propagation delays output-enable / disable delays matter for shared-bus handoff every timing number is tied to VCC, temperature and load conditions 7. LOAD / TEST CIRCUIT understand the capacitance/resistance under which timing was measured 8. DESIGN CHECK sender VOH(min) > receiver VIH(min) with margin sender VOL(max) < receiver VIL(max) with margin source/sink current within guarantee worst-case delay fits timing budget only one tri-state bus driver enabled at a time

Datasheet section	Correct question to ask	Common mistake
Absolute Maximum Ratings	What stress might permanently damage the part?	Treating max VCC/current as a supported operating point.
Recommended Operating Conditions	Under what ranges does the vendor intend guaranteed operation?	Ignoring temperature, edge-rate or supply constraints.
VIH / VIL	What voltage will definitely be accepted as HIGH/LOW?	Using a typical threshold or VCC/2 as if guaranteed.
VOH / VOL at IOH/IOL	What voltage can this output guarantee while sourcing/sinking this current?	Assuming a logic output is an ideal 0 V/VCC voltage source.
Propagation delay	How late can output become valid after input/control changes?	Using only a typical number rather than max/worst-case.
3-state disable/enable time	Can one bus driver release before another begins driving?	Creating momentary bus contention.
Input/output capacitance	How much electrical load and edge slowing does each pin add?	Ignoring fanout and trace/load capacitance.
Thermal/package	Can the package dissipate the expected power and be assembled correctly?	Thinking identical logic in different packages is thermally/mechanically identical.

Best habit: when quoting any electrical/timing number, also state the conditions under which it is guaranteed—supply voltage, temperature, load, output current and transition direction. A number without its conditions is often not an engineering specification.

DIRECT PDFTI — Understanding and Interpreting Standard-Logic Data Sheets (direct PDF)

Outstanding free 59-page guide to logic datasheets: function tables, pinouts, absolute maximum ratings, recommended conditions, VIH/VIL, VOH/VOL, leakage, propagation delay, capacitance, power and packaging.

https://www.ti.com/lit/an/szza036c/szza036c.pdf

DIRECT DATASHEETTI SN74HC245 datasheet — direct PDF

Use this as the lab specimen. It is an eight-channel bidirectional transceiver with DIR and active-low OE, CMOS inputs/outputs and 3-state bus isolation.

https://www.ti.com/lit/ds/symlink/sn74hc245.pdf

OFFICIAL DOCSTI SN74HC245 product/document page

Public HTML summary and datasheet access. Useful for package variants, current status and finding related application notes.

https://www.ti.com/product/SN74HC245

HOW TO READ A COMPUTER SCHEMATIC:

Find the power rails and ground first. Note every supply voltage and decoupling capacitor.
Find the clock source and RESET. Determine which edge stores state and what reset actually clears/loads.
Find the CPU address pins, data pins, and read/write or enable control pins.
Follow address decoding until you can say which address ranges select RAM, ROM, and each peripheral.
On a shared data bus, identify every possible driver and the enable/tri-state signal that prevents contention.
For one instruction, trace fetch → decode → operands → ALU → memory/I/O → write-back across clock cycles.
Only after the logic makes sense, worry about faster modern variants such as caches, pipelines, DDR, PCIe and multiple clock domains.

Direct books, PDFs and manuals — click and read

These are books, manuals, schematics, or PDFs you can open directly.

DIRECT PDFComputer Science from the Bottom Up — Ian Wienand (PDF)

Creative-Commons book. Starts with binary and Boolean operations, then computer architecture, operating systems, virtual memory, executable files, linking and the low-level software stack.

https://www.bottomupcs.com/csbu.pdf

DIRECT PDFDigital Circuit Projects — Charles W. Kann (PDF)

Open textbook built around actual digital circuits: adders, decoders, multiplexers, D flip-flops and simple state machines. Designed so the circuits can be built outside a university laboratory.

https://cupola.gettysburg.edu/cgi/viewcontent.cgi?article=1000&context=oer

DIRECT PDFPC Assembly Language — Paul A. Carter (PDF)

Freely distributed full book on x86 assembly, registers, memory addressing, the stack, calling conventions, bit operations and interfacing assembly with C. Old 32-bit x86, but excellent for making machine state concrete.

https://www.plantation-productions.com/AssemblyLanguage/pcasm-book.pdf

DIRECT PDFMIT 6.012 Lecture 14 — CMOS inverter, delay and dynamic power (PDF)

Twenty-five pages directly on CMOS: NMOS and PMOS states, VDD, transfer curves, noise margins, propagation delay, capacitive loading and dynamic power. This is a technical PDF, not an enrollment page.

https://ocw.mit.edu/courses/6-012-microelectronic-devices-and-circuits-fall-2005/6bec6dd1b07b02a1a84098b78f068cc3_lec14.pdf

DIRECT PDFApple-1 Operation Manual — original 1976 manual and schematics (PDF)

Original Apple Computer Company manual. Includes specifications and the machine's schematics. Study the clock, 6502, RAM, PIA, terminal/video logic and power rails as one complete early personal computer.

https://bitsavers.org/pdf/apple/apple_I/AppleI_Manual.pdf

DIRECT PDFPDP-8 User's Handbook — 1966 (PDF)

Original DEC handbook for a historically important minicomputer. Registers, instruction format, memory addressing, I/O and programming are documented at a scale that is still possible to hold in your head.

https://bitsavers.trailing-edge.com/pdf/dec/pdp8/handbooks/1966_PDP8_UsersHandbook.pdf

DIRECT PDFMOS Technology MCS6500 Family Hardware Manual — 1976 (PDF)

Original 6502-family hardware manual. Especially relevant to how components connect: CPU pins, clocks, address/data buses, memory and peripheral interfacing, timing and system organization.

https://computermuseum.informatik.uni-stuttgart.de/ftp/mirrors/maben.homeip.net/static/S100/6502/MOS%20Technology%20MCS6500%20Family%20Hardware%20Manual.pdf

DIRECT PDFMOS Technology MCS6500 Programming Manual — 1976 (PDF)

Original 6502-family programming manual. Read beside the hardware manual to connect registers, buses, ALU behavior, flags, addressing modes and machine instructions.

https://computermuseum.informatik.uni-stuttgart.de/ftp/mirrors/maben.homeip.net/static/S100/6502/MOS%20Technology%20MCS6500%20Programming%20Manual.pdf

1. Start here: the strongest complete sets of open material

OPEN NOTESMIT 6.004 — Computation Structures (2009)

NO LOGIN / NO ENROLLMENT. MIT OpenCourseWare is simply MIT publishing its class material on the web. It starts with MOS transistors, then logic gates, combinational/sequential logic, finite-state machines, a processor, memory, and complete systems. Read the notes or watch the lectures; ignore labs/exams unless you want them.

https://ocw.mit.edu/courses/6-004-computation-structures-spring-2009/

OPEN NOTESMIT 6.004 — Computation Structures (2017)

NO LOGIN / NO ENROLLMENT. A newer public MIT OpenCourseWare presentation with annotated slides and topic videos. Especially good for timing, pipelining, synchronization, performance, and building one abstraction layer on top of another.

https://ocw.mit.edu/courses/6-004-computation-structures-spring-2017/

PUBLIC WEBNand2Tetris — From NAND to Tetris

DIRECT FREE SELF-STUDY SITE. No classroom is required. You create gates, an ALU, registers, memory, a CPU, an assembler, a VM, a compiler, and a tiny OS. The site's lectures, project files, and tools are free/open for self-study. Ignore the separate Coursera route.

https://www.nand2tetris.org/

PUBLIC WEBNand2Tetris — Projects

DIRECT FREE PROJECT FILES. No enrollment. This is the actual project sequence. For hardware, concentrate on Boolean Logic, Boolean Arithmetic, Memory, Machine Language, Computer Architecture, and Assembler.

https://www.nand2tetris.org/course

PUBLIC NOTESUC Berkeley CS61C — Great Ideas in Computer Architecture notes

PLAIN PUBLIC WEB NOTES. No login needed to read them. Covers number representation, RISC-V, synchronous digital logic, a complete single-cycle CPU datapath, pipelining, caches, performance, and virtual memory.

https://notes.cs61c.org/

FREE BOOKDive into Systems — free online textbook

A free systems textbook that follows a vertical slice from C and binary representation down toward assembly, architecture, memory, operating systems, and parallelism. Useful for tying hardware to the software that runs on it.

https://www.diveintosystems.org/

PUBLIC PROJECTBen Eater — 8-bit breadboard computer

PLAIN PUBLIC SITE + VIDEOS + SCHEMATICS. One of the clearest physical demonstrations of a CPU. A complete programmable 8-bit computer is built from simple 74-series logic: clock, registers, ALU, RAM, program counter, bus, output, and control logic. Buying a kit is optional.

https://eater.net/8bit

How a transistor becomes a chip: fabrication and layout

This is the missing layer between a MOSFET cross-section and a photographed CPU die: wafers, doping, oxide, polysilicon, photolithography, implantation/diffusion, etching, deposition, contacts and metal interconnect.

DIRECT PDFCMOS Cookbook — Don Lancaster, author-hosted full PDF

The author/publisher hosts this complete 521-page edition directly. Chapter 1 starts with the CMOS process and physically builds n-channel and p-channel MOS devices, then inverters, logic/transmission gates, supplies and breadboarding. Later chapters cover clocks, flip-flops, counters, registers and memories.

https://www.tinaja.com/ebooks/cmoscb.pdf

DIRECT PDFIntel Manufacturing 101 — The Transistor, Explained (PDF)

Short, visual bridge from vacuum tubes and discrete transistors to planar MOSFET fabrication and photolithography. Good orientation before the denser fabrication material.

https://download.intel.com/newsroom/2024/tech101/transistor/manufacturing-101-the-transistor-explained.pdf

FREE TEXTBOOKAll About Circuits — Semiconductor Manufacturing Techniques

Plain textbook chapter on silicon crystal growth, wafers, oxide, photoresist, masks, etching, doping/diffusion, contacts and metallization.

https://www.allaboutcircuits.com/textbook/semiconductors/chpt-2/semiconductor-manufacturing-techniques/

DIRECT PDFMIT 6.012 — Review of IC Technology (PDF)

Compact technical review connecting IC design to fabrication: lithography, etching, oxide/nitride, ion implantation, metal interconnect and the MOSFET cross-section.

https://live.ocw.mit.edu/courses/6-012-microelectronic-devices-and-circuits-spring-2009/9ac876bcefd795c0bb90bc35ba289922_MIT6_012S09_rec01.pdf

OPEN NOTESMIT 6.152J — Micro/Nano Processing lecture notes

Public notes indexed by fabrication process: oxidation, diffusion, implantation, CVD, sputtering, evaporation, lithography, wet/dry etching and CMOS. No signup.

https://ocw.mit.edu/courses/6-152j-micro-nano-processing-technology-fall-2005/pages/lecture-notes/

TECH ARTICLEASML — How microchips are made

Modern manufacturing overview emphasizing repeated patterned layers and alignment on a silicon wafer. Useful for understanding how the textbook process scales into a contemporary fab.

https://www.asml.com/en/technology/all-about-microchips/how-microchips-are-made

TECH ARTICLEASML — Lithography principles

Explains masks/reticles, projection optics, photoresist, DUV/EUV lithography and why feature size and overlay matter.

https://www.asml.com/en/technology/lithography-principles

TECH ARTICLEKen Shirriff — HP Nanoprocessor die and mask-level explanation

A real vintage MOS processor used to show how masks, doped regions, oxide and metal wiring appear on silicon. Excellent after reading a fabrication overview because you can connect process steps to an actual die.

https://www.righto.com/2020/09/inside-hp-nanoprocessor-high-speed.html

2. Electricity, semiconductors, MOSFETs, NMOS, PMOS, VDD and GND

OPEN NOTESMIT 6.002 — Circuits and Electronics

For the electrical foundations underneath digital logic: voltage, current, circuit models, switches, MOS transistors, capacitors, digital abstraction, state, memory, and circuit speed. Includes video lectures and labs.

https://ocw.mit.edu/courses/6-002-circuits-and-electronics-spring-2007/

OPEN NOTESMIT 6.012 — Microelectronic Devices and Circuits

Go here when 'a MOSFET is a switch' is no longer enough. Covers semiconductor physics, electrons and holes, doping, MOS electrostatics, MOSFET I–V behavior, device models, and digital circuits.

https://ocw.mit.edu/courses/6-012-microelectronic-devices-and-circuits-fall-2005/

OPEN NOTESMIT 6.012 — complete lecture notes

PDF lecture notes for the device course. The sequence around lectures 9–14 is particularly relevant: MOSFET characteristics, equivalent models, NMOS inverter, CMOS inverter, delay, scaling, and VLSI.

https://ocw.mit.edu/courses/6-012-microelectronic-devices-and-circuits-fall-2005/pages/lecture-notes/

FREE TEXTBOOKAll About Circuits — MOSFET structure and operation

A concise technical explanation of the four-terminal MOSFET (gate, drain, source, body), including NMOS versus PMOS and why gate voltage controls conduction.

https://www.allaboutcircuits.com/technical-articles/mosfet-structure-and-operation-for-analog-ic-design/

FREE TEXTBOOKAll About Circuits — CMOS gate circuitry

Very useful for NMOS + PMOS working together. Shows CMOS inverters and gates and explains the VDD convention. A good bridge from transistor behavior to Boolean gates.

https://www.allaboutcircuits.com/textbook/digital/chpt-3/cmos-gate-circuitry/

PUBLIC WEBMIT 6.004 (1998) — Logic Gates lecture

Old but unusually direct. Covers static discipline, voltage transfer curves, NMOS, PMOS, CMOS gates, rise/fall time, propagation delay, contamination delay, and composition.

https://people.csail.mit.edu/devadas/6.004/Lectures/lect2/

INTERACTIVEFalstad Circuit Simulator

Interactive analog/digital circuit simulator in the browser. Use it to experiment with MOSFETs, RC networks, oscillators, latches, and power rails instead of only reading equations.

https://www.falstad.com/circuit/

INTERACTIVEFalstad — example circuit index

Direct index of example circuits, including CMOS inverter examples with and without capacitance. Excellent for seeing why a real CMOS transition has finite speed and consumes dynamic power.

https://www.falstad.com/circuit/doc/circuits.html

The CMOS inverter, without hand-waving

This is the smallest circuit worth understanding extremely well. A CMOS inverter has a PMOS pull-up connected toward VDD and an NMOS pull-down connected toward GND/VSS. Their gates are tied together as the input; their drains meet at the output.

Input ≈ 0 V	PMOS ON, NMOS OFF	Output capacitance charges toward VDD → logic 1
Input ≈ VDD	PMOS OFF, NMOS ON	Output capacitance discharges toward GND → logic 0
Input in transition	Both may conduct partly	Short-circuit current can flow while the output capacitance is being charged/discharged

The output does not change instantly because transistors have finite drive current and every node has capacitance. That is the seed of propagation delay, and propagation delay is the seed of clock-speed limits.

DIRECT PDFCMOS transistor theory — David Harris (direct PDF)

Nineteen technical pages that explicitly leave the ideal-switch model behind: MOS capacitor, inversion, VGS/VDS, cutoff/linear/saturation regions, NMOS and PMOS I-V behavior, mobility and parasitic capacitance.

https://pages.hmc.edu/harris/class/e158/16/lect3.pdf

PLAIN WEBCMOS transmission-gate demonstration — DTU

Plain public explanation of why a lone NMOS passes a weak high and a lone PMOS passes a weak low, and why putting NMOS + PMOS in parallel creates a better bidirectional CMOS transmission gate.

https://www.imm.dtu.dk/courses/02206/java/transmission.html

Before transistors: relays, vacuum tubes and the same Boolean logic

The abstraction 0/1 → Boolean gate → latch/register → arithmetic/control predates the transistor. A bit can be represented by a relay armature position, a vacuum-tube voltage, a transistor node, a magnetic core direction, or charge in a semiconductor cell. The logic is the same; the physical device changes the speed, size, power, reliability and wiring.

electromechanical relay contact open/closed → logical state ↓ vacuum-tube triode / diode logic electronic voltage/current switching; no moving contacts ↓ discrete bipolar transistor + diode logic ↓ integrated TTL / ECL / MOS logic families ↓ NMOS / CMOS VLSI ↓ millions → billions of transistors on one die Across every era: logic state → gates → stored state → datapath/control → complete computer

Technology	What physically switches	What it was good/bad at
Relay	Electromagnet mechanically moves contacts.	Easy to see and reason about; slow, bulky, audible, contacts wear.
Vacuum tube / valve	Electric field controls electron flow through vacuum.	Electronic and much faster than relays; hot, large, power-hungry, finite tube life.
Discrete BJT	Base current controls collector-emitter current.	Smaller/cooler/more reliable than tubes; still needs many separate parts/wires.
TTL integrated circuit	Bipolar transistors/resistors integrated on one silicon die.	Standard reusable logic blocks, strong drive, historically important; significant static power.
NMOS IC	N-channel MOSFET networks plus loads.	High integration and simpler fabrication; used by early microprocessors such as 6502/8080 families.
CMOS	Complementary NMOS/PMOS networks.	Extremely low ideal static logic power and enormous density; dominant digital IC technology.

MUSEUM EXPLAINERComputer History Museum — How digital computers 'think'

Compact public explanation that bits and Boolean logic can be represented mechanically or electronically; includes the important historical example that a dual-triode vacuum tube could be wired as a one-bit flip-flop.

https://www.computerhistory.org/revolution/digital-logic/12/269

PUBLIC PROJECTHarry Porter's Relay Computer

A complete relay CPU where the state and control are mechanically visible. Read it beside transistor-level material to see how little the logical organization depends on transistor technology.

https://web.cecs.pdx.edu/~harry/Relay/

HISTORICAL PROJECTColossus rebuild — National Museum of Computing

Detailed account of rebuilding a vacuum-tube computer from surviving photographs, notes and circuit fragments. Particularly useful for appreciating real valve logic, counters, rings and high-speed paper-tape input.

https://www.tnmoc.org/rebuilding-colossus

PRIMARY PDFENIAC original operating manual — June 1946 (direct PDF)

Primary-source manual for a room-sized vacuum-tube computer. This predates the convenient stored-program model: programming and operation involve units, panels, switches, cables and explicit sequencing.

https://www.bitsavers.org/pdf/univOfPennsylvania/eniac/ENIAC_Operating_Manual_Jun46.pdf

HISTORICAL EXPLAINERIBM 603 — vacuum-tube arithmetic before general-purpose transistor computers

IBM historical page explaining how electronic vacuum-tube flip-flops/addition replaced slower electromechanical arithmetic and pointed toward fully electronic computers.

https://www.ibm.com/history/603

Logic families: CMOS is not the only way digital gates were built

Modern CPUs are overwhelmingly CMOS, but historically important machines were often built from bipolar TTL ICs. TTL uses NPN bipolar transistors and resistors, normally around a 5 V supply; CMOS uses complementary MOSFET networks. Their input currents, output stages, static power, thresholds and floating-input behavior differ, so 'logic 1' is an electrical specification, not a metaphysical state.

Property	Classic TTL	CMOS
Active devices	Mostly NPN bipolar junction transistors	NMOS + PMOS field-effect transistors
Typical historical rail	5 V VCC	VDD varies by family/process; older 4000-series tolerated broad ranges
Static input current	Nonzero, especially for LOW inputs	Ideally tiny DC gate current; capacitance still matters dynamically
Static power	Significant bias current	Low ideally, but leakage exists; switching power dominates much digital use
Floating input	Classic TTL tends to read HIGH, but relying on floating inputs is bad practice	Must not be left floating; undefined input can switch/noise and increase current
Historical use	Minicomputers, early computers, 7400-series glue logic	Modern CPUs, RAM, SoCs, modern logic families

FREE TEXTBOOKTTL NAND and AND gates — transistor-level textbook explanation

Plain public textbook material. Walks through the unusual multi-emitter input transistor and shows how a real TTL NAND gate works electrically.

https://www.allaboutcircuits.com/textbook/digital/chpt-3/ttl-nand-and-gates/

FREE TEXTBOOKLogic voltage levels — TTL and CMOS compatibility

Explains that output-high/output-low guarantees and input thresholds must overlap; includes the classic problem of feeding TTL outputs into CMOS inputs.

https://www.allaboutcircuits.com/textbook/digital/chpt-3/logic-signal-voltage-levels/

TECH ARTICLEInside the 74181 ALU chip — TTL transistors to a complete ALU

A rare full bridge from NPN transistor circuitry to TTL gates to a 4-bit arithmetic/logic unit used in real minicomputers.

https://www.righto.com/2017/01/die-photos-and-reverse-engineering.html

TECH ARTICLE74181 carry lookahead and arithmetic structure

Explains why ripple carry is slow, how generate/propagate carry-lookahead works, and why this famous ALU has its strange collection of functions.

https://www.righto.com/2017/03/inside-vintage-74181-alu-chip-how-it.html

Worked example: from bipolar transistors to a 4-bit ALU

The 74181 is a useful antidote to the vague phrase 'the ALU does arithmetic.' It is a real 1970s chip with about 170 transistors implementing a 4-bit arithmetic/logic slice. Multiple slices could be combined for wider processors.

NPN bipolar transistors + resistors ↓ TTL inverter / multi-emitter input structures ↓ AND-OR-INVERT / XOR logic ↓ generate + propagate terms ↓ carry-lookahead network ↓ 4-bit arithmetic / logic result ↓ several 74181s + carry generator → wider CPU ALU

3. Boolean logic, gates, adders, multiplexers and digital abstraction

FREE TEXTBOOKAll About Circuits — Logic Gates textbook section

Free textbook material on digital signals, voltage levels, NAND/NOR/NOT/XOR, CMOS gate circuitry, TTL, gate universality, and practical IC packaging.

https://www.allaboutcircuits.com/textbook/product/logic-gates/

INTERACTIVENandGame

An interactive puzzle sequence where each component you build becomes a building block for the next. Begins with NAND and progresses toward a working computer.

https://nandgame.com/

SOURCE / FILESLogisim-evolution

Free open-source desktop logic simulator. Very good for building a CPU without breadboards: gates, registers, memories, buses, TTL parts, timing diagrams, and hierarchical subcircuits.

https://github.com/logisim-evolution/logisim-evolution

OPTIONAL EXERCISESHDLBits — Verilog practice

NO ACCOUNT REQUIRED TO USE IT. This one is intentionally exercise-based: you write small Verilog circuits and the site checks them. An account is optional only if you want progress saved across browsers. Skip it entirely if you do not want exercises.

https://hdlbits.01xz.net/wiki/Main_Page

Arithmetic is circuitry: adders, carry, multiplication, division and flags

An ALU is not one mysterious arithmetic object. Addition begins with half-adders/full-adders; wider adders connect bit positions with a carry network. A naïve ripple-carry adder waits for carry to move through successive bits, while carry-lookahead/prefix structures compute carry information more aggressively. Subtraction is normally implemented by adding a two's-complement negation. Multiplication builds and combines shifted partial products. Division is usually iterative or otherwise substantially more expensive than a simple add.

1-bit FULL ADDER A ─┐ B ─┼──→ SUM Cin┘ CARRY OUT N-bit ripple adder: bit 0 carry → bit 1 carry → bit 2 carry → ... → bit N-1 faster adder families: generate/propagate signals → carry-lookahead / prefix tree → sums subtraction: A - B = A + (~B + 1) multiplication: partial products → shifted rows → adder tree / carry-save reduction → product division: compare/subtract/shift state machine, often many cycles for a simple implementation

Flag	What hardware is noticing
Z / Zero	Result bits are all zero.
N / Sign	Usually copies the most-significant result bit for two's-complement arithmetic.
C / Carry	Unsigned carry out of the most-significant position; also useful for multiword arithmetic.
V / Overflow	Signed result cannot be represented in the chosen two's-complement width; distinct from unsigned carry.
Borrow conventions	Subtraction flags vary by ISA; some architectures expose carry-as-not-borrow or define it differently.

DIRECT PDFMIT — Ripple carry, serial addition and carry-lookahead (direct PDF)

Five direct pages on ripple-carry addition, serial adders, faster addition and carry-lookahead. No enrollment; this link is the PDF itself.

https://ocw.mit.edu/courses/6-896-theory-of-parallel-hardware-sma-5511-spring-2004/8f34153fa569c5f6ca74cc35b967d76c_lect2.pdf

PUBLIC PROJECTProject F — Numbers in Verilog

Hands-on explanation of signed/unsigned hardware arithmetic, two's complement, subtraction, multiplication widths and what synthesis tools infer.

https://projectf.io/posts/numbers-in-verilog/

PUBLIC PROJECTProject F — Multiplication with FPGA DSP blocks

Shows a modern FPGA implementation perspective: dedicated multiplier/DSP hardware, pipelining and why multiplication has different timing/resource costs than addition.

https://projectf.io/posts/multiplication-fpga-dsps/

PUBLIC PROJECTProject F — Division in Verilog

Implements iterative binary division as actual stateful hardware. The simple design takes one cycle per input bit, making the cost of division concrete.

https://projectf.io/posts/division-in-verilog/

Floating point is hardware too: exponent alignment, significand arithmetic, normalization and rounding

IEEE-754 gives bit-level representations and arithmetic semantics; an FPU has to realize those rules with shifters, adders, multipliers, leading-zero/normalization logic, rounding logic and special-case handling. A floating-point adder is therefore considerably more involved than an integer ripple or carry-lookahead adder.

IEEE-754 binary32 layout 31 30........23 22.........................0 +---------+-----------+---------------------------+ | sign 1 | exponent 8| fraction / significand 23 | +---------+-----------+---------------------------+ NORMAL FLOATING-POINT ADD/SUBTRACT (conceptual) unpack sign/exponent/fraction ↓ compare exponents ↓ right-shift smaller significand until exponents align ↓ add/subtract significands according to signs ↓ normalize result (shift + adjust exponent) ↓ round using discarded guard/round/sticky information ↓ handle overflow/underflow/subnormal/zero/Inf/NaN cases ↓ pack sign + exponent + fraction MULTIPLY (conceptual) signs XOR; exponents add and rebias; significands multiply; normalize; round; special cases FMA = a*b + c with one final rounding rather than separately rounding multiply then add

IEEE-754 issue	Hardware consequence
NaN / infinity / signed zero	Input classifiers and special-case datapaths bypass ordinary arithmetic in many cases.
subnormal numbers	No implicit leading 1; normalization and gradual-underflow support become more complex.
rounding modes	Result must be adjusted according to round-to-nearest-even, toward zero, toward ±infinity, etc.
inexact	Hardware tracks whether discarded bits mean the exact mathematical result was not representable.
overflow / underflow	Exponent-range detection and standard-defined result/flag behavior.
FMA	Large multiply-add datapath keeps extra internal precision and rounds once at the end.
pipeline stages	High-frequency FPUs split alignment, arithmetic, normalization and rounding across registers to meet timing.

PUBLIC NOTESCS61C — IEEE-754 floating-point representation

Current public notes beginning with fixed-point motivation and proceeding into IEEE-754 binary floating point. No login required.

https://notes.cs61c.org/content/floating-point/

PUBLIC NOTESCS61C — normalized binary32/binary64 fields

Sign, biased exponent, significand, implicit leading one and binary32/binary64 layouts explained directly.

https://notes.cs61c.org/content/floating-point/fp-floating-point/

PUBLIC NOTESCS61C — floating-point addition and rounding discussion

Walks the conceptual arithmetic sequence: match exponents, add significands, normalize and round; also discusses why FP arithmetic is more expensive than integer addition.

https://notes.cs61c.org/content/floating-point/fp-discussion/

PUBLIC NOTESCS61C — zero, infinity, NaN and subnormals

Public explanation of IEEE-754 special encodings, gradual underflow and the cases an FPU must detect and handle.

https://notes.cs61c.org/content/floating-point/fp-special-numbers/

SOURCE / FILESOpenHW CVFPU / FPnew — real open SystemVerilog FPU

A real parameterized floating-point unit supporting add/subtract, multiply, fused multiply-add, division, square root, conversions, comparisons, multiple IEEE-style formats and standard exception flags.

https://github.com/openhwgroup/cvfpu

PUBLIC DOCSFPnew architecture documentation

Describes the actual top-level FPU interface, operation-group blocks, format slices, pipelining, output arbitration and configurable pipeline-register placement.

https://github.com/openhwgroup/cvfpu/blob/develop/docs/README.md

SOURCE / FILESBerkeley HardFloat

Open hardware floating-point units used in RISC-V research/implementations. Includes fused multiply-add, conversions, recoded internal representation and division/square-root source.

https://github.com/ucb-bar/berkeley-hardfloat

Power, ground, clocks and reset — the infrastructure logic needs

A schematic is not only gates and buses. Real digital circuits fail if the supply rails bounce, a clock edge arrives badly, reset releases too early, or a signal violates setup/hold time. These links make VDD/VSS, decoupling, oscillators and reset concrete.

DIRECT PDFTI — Clocking Design Guide / clock-system fundamentals (PDF)

Technical application report on why digital systems need clocks, resonators/crystals, oscillators, clock generators/buffers, multiple frequencies and control of phase/jitter.

https://www.ti.com/lit/pdf/snaa217

DATASHEET PDFTI SN74HC74 — D flip-flop datasheet (PDF)

Read a real timing table: setup time, hold time, clock pulse width, propagation delay and maximum clock frequency. This turns timing jargon into numbers attached to an actual flip-flop.

https://www.ti.com/lit/ds/symlink/sn54hc74.pdf

DIRECT PDFTI — Simultaneous-Switching Performance / ground bounce (PDF)

Shows that GND is not magically perfect. Fast output transitions and package inductance can move the apparent ground level enough to alter CMOS thresholds or cause false switching.

https://www.ti.com/lit/an/szza038b/szza038b.pdf

DATASHEET HTMLMicrochip — power-up, oscillator-start and brown-out reset timing

A real device timing/specification page showing reset pulse width, power-up delay, oscillator start-up and brown-out thresholds. Useful for understanding why reset is an electrical subsystem.

https://onlinedocs.microchip.com/oxy/GUID-D4E09E0B-194B-4BED-B95A-F7BA5CB174DF-en-US-14/GUID-E68D37C3-8D67-4ABC-91CE-36A3565F4E23.html

DIRECT PDFRP2040 Hardware Design with RP2040 (PDF)

Board-level reference showing core/I/O power rails, decoupling capacitors, regulator wiring, crystal/clock, QSPI flash, USB and reset-related design in one real system.

https://datasheets.raspberrypi.com/rp2040/hardware-design-with-rp2040.pdf

Power delivery is part of logic correctness: VRMs, decoupling, impedance, droop and on-die power grids

Millions of gates can switch within a tiny time interval. Their current demand changes much faster than a motherboard regulator can react through long traces, vias, package inductance and planes. A processor therefore relies on a hierarchical Power Distribution Network (PDN): regulator, bulk capacitance, board MLCCs, package capacitance and on-die decoupling/power grids all cover different time/frequency ranges.

12 V / input supply ↓ multiphase buck VRM MOSFETs → inductors → bulk/output capacitors ↓ motherboard power planes / pours ↓ nearby ceramic MLCC decoupling package balls / lands ↓ package planes + parasitic R/L/C die bumps / pads ↓ on-die power grid + local decoupling capacitance ↓ CMOS gates switch → short transient current pulses Approximate small-signal idea: ΔV(f) ≈ ΔI(f) × Z_PDN(f) Target-impedance design heuristic: Z_target ≈ allowed supply droop / worst load-current step A real capacitor is C plus ESR plus ESL: low f: capacitive impedance falls with frequency near self resonance: minimum impedance high f: package/mount inductance dominates and impedance rises Therefore 'just add one giant capacitor' does not solve high-frequency transient delivery.

PDN concept	Why it matters
IR drop	DC/low-frequency voltage loss from current flowing through finite resistance in planes, vias, package and on-die metal.
L·di/dt droop	Rapid current change across inductance causes voltage disturbance; this is why physical loop inductance matters.
decoupling capacitor	Local energy/charge reservoir reducing rail movement while slower supply paths respond.
bulk capacitor	Larger capacitance for lower-frequency load transients and regulator-loop support.
ESR	Equivalent Series Resistance; contributes damping, loss and voltage drop.
ESL	Equivalent Series Inductance; limits capacitor effectiveness at high frequency.
self-resonant frequency	Frequency at which capacitor's C and parasitic L resonate and impedance reaches a minimum.
anti-resonance	Parallel capacitor/plane combinations can create impedance peaks that are worse than either part alone.
target impedance	Maximum acceptable PDN impedance across relevant frequency range derived from allowed voltage ripple and load-current demand.
power grid	Wide/meshed on-chip metal network distributing VDD/GND while controlling IR drop/electromigration.
ground bounce	Local reference movement due to simultaneous switching current through shared inductive/resistive return paths.

A decoupling capacitor does not 'filter noise' by magic. Its usefulness comes from providing a sufficiently low-impedance local current path over a particular frequency range. Placement matters because trace/via/package inductance can isolate a physically distant capacitor from a nanosecond-scale load transient.

TECH ARTICLEAnalog Devices — ferrite beads and real power-supply impedance

Public app note with realistic bead/inductor/capacitor impedance, LC resonance and damping. Good antidote to ideal-component diagrams.

https://www.analog.com/en/resources/app-notes/an-1368.html

TECH ARTICLEAnalog Devices — decoupling placement example

Concrete board-layout guidance showing small high-frequency decoupling capacitors placed closest to the IC and very short ground/device connections.

https://www.analog.com/en/resources/app-notes/an-1562.html

DIRECT PDFTI — multiphase buck converters for processor-class rails (direct PDF)

Public application report on multiphase buck conversion, the regulator architecture commonly used when low-voltage digital loads need large transient currents.

https://www.ti.com/lit/pdf/slva882

DIRECT PDFKeysight — power integrity / PDN application note (direct PDF)

Public technical introduction to power-distribution-network impedance, decoupling and power-integrity measurement/simulation.

https://www.keysight.com/us/en/assets/3119-1097/application-notes/5992-4272.pdf

PUBLIC DOCSOpenROAD — PDN generation

Shows the chip-design side: straps, rings, grids and connections used to distribute supply through physical IC layout.

https://openroad.readthedocs.io/en/latest/main/src/pdn/README.html

A PCB trace is only half a circuit: return current, reference planes and controlled impedance

At high edge rates, the useful mental model is a transmission line, not an isolated copper wire. Signal current leaves the driver on the trace and returns through a nearby reference structure. The geometry of trace plus reference plane sets impedance and field distribution. Breaking the return path with a plane split, badly placed via transition or connector discontinuity increases loop inductance, reflections and EMI.

MICROSTRIP-LIKE CROSS SECTION signal trace ===============================→ load dielectric / electric field GND reference --------------------------------- return current ←=============================== At high frequency, return current concentrates near the trace because that nearby path minimizes loop inductance / impedance. BAD: SIGNAL CROSSES A GAP IN REFERENCE PLANE signal ====================>====================== GND ------------------- ------------------ GAP return current cannot cross directly beneath signal ↓ detours around gap / through distant connection ↓ larger current loop → more inductance + EMI + impedance discontinuity LAYER CHANGE signal via changes from layer referenced to GND plane A to layer referenced to GND plane B ↓ place GND stitching via close to signal via ↓ gives return current a nearby path between reference planes DIFFERENTIAL PAIR P and N traces couple to each other AND to reference plane. Even differential links still benefit from continuous reference/return geometry.

PCB term	Why it matters
reference plane	Nearby conductor, usually GND, that forms the return path and transmission-line geometry for high-speed traces.
controlled impedance	Trace width/spacing/dielectric/plane geometry chosen so characteristic impedance stays near target value.
microstrip	Trace over a reference plane near PCB surface.
stripline	Trace embedded between reference planes.
return-path discontinuity	Gap/plane change/connector geometry that forces return current away from the signal path.
stitching via	Ground/reference via placed near a signal transition so high-frequency return current can change layers locally.
stub	Unused branch of transmission line that can reflect energy; long vias/pads/branches become important at high edge rates.
termination	Resistive/network treatment intended to match source/load/line enough to control reflections.
rise time	Often more relevant than clock frequency for deciding whether interconnect behaves as a transmission line.
differential impedance	Impedance seen by differential-mode current flowing through a coupled P/N pair and its reference environment.
eye diagram	Overlay of many received bit intervals showing timing/voltage margin, jitter, noise and intersymbol interference.

Do not route a fast signal across a split/void in its reference plane. TI's current 2026 high-speed layout guidance explicitly says high-frequency return current normally follows the adjacent reference plane and recommends a continuous GND plane; where a signal changes between GND references, nearby stitching vias provide the return transition.

DIRECT PDFTI — High-Speed Layout Guidelines for Signal Conditioners and USB Hubs (2026 direct PDF)

Current January 2026 guide with an unusually clear return-current chapter: high-frequency return current follows the lowest-impedance nearby reference plane; avoid split planes and control the reference path.

https://www.ti.com/tw/lit/pdf/slla414

DIRECT PDFTI — High-Speed Interface Layout Guidelines (direct PDF)

Public guide covering stackup, impedance, differential routing, layer changes and GND stitching near signal vias.

https://www.ti.com/lit/an/spraar7j/spraar7j.pdf

OFFICIAL GUIDETI — current GND stitching example

Concrete no-login example requiring a continuous GND reference along a high-speed trace and nearby symmetric stitching vias when the reference changes.

https://www.ti.com/document-viewer/lit/html/SLLA653/GUID-A4F1CD83-9D39-45B1-B7A5-0E03429E4305

DIRECT PDFAnalog Devices — high-speed OSPI hardware/layout guide (direct PDF)

Real interface example: match DQ/DQS delay, keep signals on the same reference, avoid split planes, and add a nearby stitching capacitor if a reference-plane change is unavoidable.

https://www.analog.com/media/en/technical-documentation/application-notes/ee-437.pdf

Why voltage, frequency, power and temperature are coupled

A processor does not have one immutable clock/voltage point. Modern CPUs move among operating points as workload, current, package power and temperature change. At the CMOS circuit level, switching power is often approximated by:

Pdynamic ≈ α · C · V² · f

Here α is switching activity, C effective switched capacitance, V supply voltage, and f clock frequency. The square on voltage is why voltage reduction is powerful, but lowering voltage also reduces transistor drive and therefore limits achievable frequency. Modern chips additionally lose power to leakage and have complex local clock/power gating.

VRM	Motherboard/package voltage-regulator circuitry converts a higher supply such as 12 V into low-voltage, very-high-current processor rails.
P-state / performance point	A frequency/voltage operating point or range selected to balance performance and power.
C-state	Idle-state concept where progressively more processor resources can be clock-gated or powered down, usually with increasing wake latency.
clock gating	Stops unnecessary clock transitions in idle logic, reducing dynamic switching power.
power gating	Cuts supply to an inactive block, reducing leakage but requiring state/wakeup management.
thermal throttling	Hardware/firmware lowers voltage/frequency or modulates clocks to keep temperature/current/power inside safe limits.

DIRECT PDFCMOS VLSI — Power (direct PDF)

Direct technical PDF. Explicitly develops CMOS switching power and the αCV²f relationship, then discusses leakage and power reduction.

https://pages.hmc.edu/harris/class/e158/lect7-power.pdf

OFFICIAL DOCSIntel Adaptive Thermal Monitor — current public datasheet

Real processor documentation showing thermal control reducing frequency and voltage, including ordering of voltage/frequency transitions.

https://edc.intel.com/content/www/us/en/design/platforms/core-processor-series-3-datasheet-volume-1-of-2/001/adaptive-thermal-monitor/

DIRECT PDFAMD EPYC CPU power management (PDF)

Public white paper on processor telemetry, voltage/current/thermal limits, system-management hardware, P-states and idle states in a modern server CPU.

https://www.amd.com/content/dam/amd/en/documents/products/processors/server/epyc/epyc-8004-and-9004-series-cpu-power-management-white-paper.pdf

Changing performance means changing clocks, voltage or both: clock gating, dividers, PLLs and DVFS

Clock gating and DVFS attack different terms in dynamic power. Clock gating reduces the effective switching activity by stopping clock edges to idle state elements. Frequency scaling reduces the number of switching opportunities per second. Voltage scaling can reduce dynamic power quadratically, but only if the transistor timing margin still supports the chosen frequency.

REFERENCE OSCILLATOR e.g. crystal / external clock ↓ PLL multiplies/conditions frequency ↓ clock mux selects source ↓ divider selects domain rate ↓ integrated clock-gating cell ↓ CPU/peripheral/register clock tree CLOCK GATING domain idle clock-enable deasserted ↓ gating cell holds clock output inactive at a safe phase ↓ flip-flop clock pins stop toggling combinational state largely stops changing ↓ dynamic power falls without losing retained register state DVFS PERFORMANCE INCREASE (conceptual safe sequence) current: 0.8 V @ 1.5 GHz request: 1.0 V @ 3.0 GHz ↓ raise regulator voltage first / wait for rail readiness ↓ change PLL/divider/performance request to higher frequency DVFS PERFORMANCE DECREASE current: 1.0 V @ 3.0 GHz request: 0.8 V @ 1.5 GHz ↓ lower frequency first ↓ then reduce voltage once slower timing requirement is safe MODERN CPU P-STATE CONTROL scheduler utilization / OS policy ↓ CPUFreq governor or hardware-managed performance controller ↓ request/min/max performance hints ↓ hardware chooses actual clock/voltage within thermal, current, package-power and silicon limits Therefore 'requested 4.0 GHz' does not guarantee every cycle executes at 4.0 GHz.

Power/performance mechanism	What changes	Main tradeoff
clock gating	Stops selected clock edges	Strong dynamic-power saving; state retained; wake usually fast.
frequency scaling	Changes clock rate via divider/PLL/performance control	Lower throughput and dynamic power roughly proportional to frequency when voltage/activity unchanged.
voltage scaling	Changes supply rail/operating voltage	Large dynamic-power benefit but lower voltage reduces timing margin/maximum safe frequency.
DVFS	Coordinates voltage and frequency operating point	Transition latency and regulator/PLL sequencing must preserve correctness.
power gating	Cuts supply to a block/domain	Reduces leakage strongly but may lose state and require isolation/retention/wakeup sequence.
P-state	Processor performance operating point/request range	Balances performance against power/current/thermal constraints.
turbo/boost	Raises selected cores above nominal/base range when headroom permits	Depends on power, current, temperature, active-core count and firmware/hardware policy.
schedutil	Linux governor using scheduler utilization to request CPU performance	Tightly couples workload demand and frequency selection.
hardware-managed P-state	Processor firmware/hardware autonomously selects performance within OS hints/limits	Faster local decisions; software may control policy bounds rather than every frequency transition.

Never gate a clock with an arbitrary AND gate in a synchronous design. If the enable changes while the clock is high, a naive gate can create a runt pulse or extra edge. Real ASIC/FPGA clock-control resources latch or otherwise constrain the enable so downstream sequential logic sees legal clock pulses.

LINUX PERFORMANCE-SCALING LAB # CPUFreq policies and driver/governor for p in /sys/devices/system/cpu/cpufreq/policy*; do echo ===$p=== cat "$p"/scaling_driver "$p"/scaling_governor 2>/dev/null cat "$p"/scaling_min_freq "$p"/scaling_max_freq "$p"/scaling_cur_freq 2>/dev/null done # available governors cat /sys/devices/system/cpu/cpufreq/policy0/scaling_available_governors 2>/dev/null # x86 Intel systems may expose intel_pstate status/hints find /sys/devices/system/cpu/intel_pstate -maxdepth 1 -type f -print -exec cat {} \; 2>/dev/null # frequency under load is still limited by thermal/power/current conditions. # Read-only inspection is safest on an important machine.

KERNEL DOCSLinux CPU Performance Scaling

Current CPUFreq architecture: core, governors and scaling drivers. schedutil uses scheduler utilization and the hardware/driver may still select an actual frequency subject to limits.

https://docs.kernel.org/6.17/admin-guide/pm/cpufreq.html

KERNEL DOCSLinux intel_pstate

Current detailed x86 example with active/passive modes and Hardware-Managed P-states (HWP), where the processor can make its own P-state selections within software policy.

https://docs.kernel.org/admin-guide/pm/intel_pstate.html

KERNEL DOCSLinux Common Clock Framework

Shows software-visible clock trees with gate, mux and rate-changing operations and a concrete MMIO gate-bit implementation.

https://docs.kernel.org/driver-api/clk.html

KERNEL DOCSLinux clock framework API overview

Current documentation explicitly distinguishes hardware clock-tree control from OS timekeeping and explains clock gating as a power-management mechanism.

https://docs.kernel.org/next/core-api/kernel-api.html

CURRENT VENDOR DOCAMD PL Power Management 2026.1

Current June 2026 AMD guide quantifies clock gating, frequency scaling and logic gating as separate power-saving techniques.

https://docs.amd.com/r/en-US/ug1556-power-design-manager/PL-Power-Management

CURRENT VENDOR DOCAMD Zynq power management — clocks/PLL

Current 2026 SoC example: lowering PLL frequency and disabling unused clocks/PLLs reduces power.

https://docs.amd.com/r/en-US/ug585-zynq-7000-SoC-TRM/Power-Management

An idle CPU is not necessarily spinning: C-states trade wake latency for lower leakage and clock power

Performance scaling answers 'how fast should an active CPU run?' CPU idle management answers 'what should hardware turn off while there is nothing runnable here?' Deeper idle states can gate more clocks, power down larger structures and sometimes affect package-level resources, but they take longer to enter/exit and may only save energy if the idle interval is long enough.

SCHEDULER FINDS NO RUNNABLE TASK FOR CPU ↓ idle loop asks CPUIdle governor for state known next timer deadline = 400 µs away recent non-timer wakeup history predicts ~250 µs idle PM QoS / latency constraints require wake ≤ 20 µs available illustrative states: C1: target residency 2 µs, exit latency 1 µs C2: target residency 20 µs, exit latency 6 µs C6: target residency 150 µs, exit latency 40 µs C6 rejected because its 40 µs exit latency violates 20 µs latency constraint C2 fits predicted residency + latency ↓ CPUIdle driver executes architecture/platform idle instruction/request ↓ hardware gates/stops selected clocks and possibly powers down deeper domains ↓ CPU remains logically idle until wake event WAKE SOURCES timer/clockevent device interrupt / MSI IPI from another CPU platform wake event ↓ hardware exits idle state ↓ exit latency CPU resumes kernel idle-loop path scheduler sees newly runnable work and dispatches it TARGET RESIDENCY minimum expected idle duration where deeper state's extra entry/exit cost is predicted to pay back in energy saved EXIT LATENCY worst-case time budget from wake request until CPU can execute again P-STATE ≠ C-STATE P-state: performance while doing work C-state: how deeply to idle when not doing work

CPUIdle concept	Meaning
idle loop	Kernel path executed when a logical CPU has no runnable task selected.
CPUIdle governor	Policy code choosing an idle state based on predicted sleep duration and latency constraints.
CPUIdle driver	Platform/CPU-specific code asking hardware to enter the selected state.
target residency	Minimum predicted idle duration needed for a state to save enough energy to justify entry/exit cost.
exit latency	Worst-case time from wakeup request until CPU resumes instruction execution.
shallow idle state	Less hardware disabled; lower savings but fast wakeup.
deep idle state	More clocks/power/resources disabled; more savings but higher entry/exit cost.
package C-state	Idle state involving resources shared by multiple cores/logical CPUs, often requiring coordination.
timer wakeup	Clockevent deadline that ends idle when scheduled work becomes due.
IPI wakeup	Another processor sends an inter-processor interrupt to make this CPU respond/reschedule.
menu / TEO	Linux CPUIdle governors that predict/use timer and observed wake behavior differently.
PM QoS latency constraint	Policy limiting how much wakeup latency power management is allowed to introduce.

The deepest state is not always the most energy-efficient choice for a short idle gap. Entry/exit itself costs time and energy. Linux therefore models both target residency and exit latency, and the nearest timer deadline provides an upper bound on how long the CPU can remain asleep unless another interrupt wakes it first.

LINUX CPU-IDLE LAB # governor / driver cat /sys/devices/system/cpu/cpuidle/current_governor_ro 2>/dev/null cat /sys/devices/system/cpu/cpuidle/current_driver 2>/dev/null # per-CPU idle-state names, latency, target residency and usage for s in /sys/devices/system/cpu/cpu0/cpuidle/state*; do echo ===$s=== grep . "$s"/{name,desc,latency,residency,usage,time,disable} 2>/dev/null done # Compare an idle system with a workload that wakes every 100 us / 10 ms / 1 s. # Frequent timers can prevent deep-state residency even when CPU utilization is low. # Do not globally disable idle states on a laptop/server merely for curiosity; # it can increase power/temperature substantially.

KERNEL DOCSLinux CPU Idle Time Management

Current CPUIdle architecture: governor + driver, target residency, exit latency, nearest timer event, menu/TEO governors and sysfs state counters.

https://docs.kernel.org/admin-guide/pm/cpuidle.html

KERNEL DOCSLinux high-resolution timers and dynamic ticks

Connects CPU idle to timer infrastructure: dynamic ticks and programmable clockevents allow CPUs to avoid unnecessary periodic wakeups.

https://docs.kernel.org/next/timers/highres.html

Devices have their own power state machine: runtime PM, PCI D-states, wakeup and link ASPM

CPU P-states/C-states are only part of system power. I/O devices can be runtime-suspended independently while the machine remains fully awake. Linux's runtime-PM core coordinates driver callbacks, usage counts, autosuspend delays, parent/child dependencies and wakeup; PCI adds standardized D0–D3 power states and PCIe links can also enter lower-power ASPM states.

DEVICE IS ACTIVE runtime_status = RPM_ACTIVE usage_count > 0 while driver/client needs device last I/O completes ↓ driver calls pm_runtime_mark_last_busy() pm_runtime_put_autosuspend() ↓ usage_count reaches 0 autosuspend timer/delay runs ↓ runtime_idle() / policy says device is idle ↓ runtime_suspend() driver quiesces queues/engines stops DMA / saves device-private state as required ↓ PCI core saves standard config state programs wakeup capability if allowed chooses deepest wake-capable low-power state ↓ D0 → D1/D2/D3hot or platform/upstream power control may permit D3cold D3hot: configuration space still software-accessible normal MMIO/I/O disabled Vcc remains present D3cold: main device power removed wake requires auxiliary/platform support returning to D0 resembles power-on reset for device context NEW I/O / REMOTE WAKE ↓ pm_runtime_resume_and_get() OR PCIe PME / ACPI wake signal ↓ PCI subsystem restores full-power D0 + config state ↓ driver runtime_resume() restores operational state ↓ device can process DMA/interrupts again PCIE LINK POWER IS SEPARATE endpoint may remain logically D0 while link enters ASPM L0s/L1/L1-substates ↓ lower idle link power ↔ additional exit latency RUNTIME PM ≠ SYSTEM SUSPEND runtime PM opportunistically idles one device during normal operation; system suspend coordinates the entire device dependency tree.

Power concept	Meaning
runtime_suspend()	Driver/subsystem callback that quiesces a device for opportunistic runtime low-power state.
runtime_resume()	Restores normal device functionality after PM core/subsystem brings the device back.
usage_count	Runtime-PM reference count preventing suspend while active clients hold the device.
autosuspend_delay_ms	Idle delay before runtime suspend, reducing wasteful rapid suspend/resume oscillation.
D0	PCI full-power operating state.
D1/D2	Optional intermediate PCI low-power states.
D3hot	Software-accessible low-power PCI state with main power present but normal I/O/memory decoding disabled.
D3cold	PCI device main supply removed; resume generally loses ordinary device context and resembles reset.
PME	PCI/PCIe Power Management Event used to signal wakeup from supported low-power states.
remote wakeup	Device-originated event requesting runtime resume while the device/platform is suspended.
ASPM	PCIe Active State Power Management: link power-state mechanism independent of endpoint D-state.
L0	PCIe active link state.
L0s/L1	Lower-power PCIe link states trading power for exit latency.

Power state has latency consequences. A deeper device D-state or PCIe link state can save power but increases resume/access latency. Linux's real-time hardware guidance explicitly calls out PCIe ASPM as a source of added device-access latency.

READ-ONLY RUNTIME-PM LAB # choose a sysfs device directory, for example one under /sys/bus/pci/devices/ DEV=/sys/bus/pci/devices/0000:BB:DD.F grep . $DEV/power/{control,runtime_status,runtime_active_time,runtime_suspended_time,autosuspend_delay_ms} 2>/dev/null # PCI current power state/capability text lspci -vv -s BB:DD.F | grep -A12 -Ei 'Power Management|ASPM|LnkCtl|LnkSta' # ASPM policy, if exposed cat /sys/module/pcie_aspm/parameters/policy 2>/dev/null # Avoid writing power/control, autosuspend delays or forcing ASPM on hardware # you rely on; bad power-management settings can cause latency spikes or device lockups.

KERNEL DOCSLinux Runtime Power Management Framework

Current PM-core design: runtime_suspend/resume/idle callbacks, usage counters, autosuspend, parent-child relationships, wakeups and synchronization with system sleep.

https://docs.kernel.org/power/runtime_pm.html

KERNEL DOCSLinux PCI Power Management

Current PCI-specific power model: D0-D3, D3hot versus D3cold, PME/wakeup and the runtime-PM callbacks used by PCI devices.

https://docs.kernel.org/power/pci.html

KERNEL DOCSLinux PCI support library

Current driver APIs for PCI power-state transitions, state save/restore, wake capability and D3cold enablement.

https://docs.kernel.org/driver-api/pci/pci.html

KERNEL DOCSLinux real-time hardware considerations

Explains the latency tradeoff of bus power management, specifically PCIe ASPM suspending links and delaying device access.

https://docs.kernel.org/core-api/real-time/hardware.html

KERNEL DOCSLinux kernel parameters — PCIe ASPM

Current pcie_aspm boot-policy documentation and explicit warning that forcing ASPM on unsupported hardware may cause lockups.

https://docs.kernel.org/admin-guide/kernel-parameters.html

Runtime power management is not system suspend: s2idle, suspend-to-RAM and hibernation stop the whole machine differently

Runtime PM lets an individual device become idle while the rest of the system keeps running. System sleep is a different layer: userspace is frozen, devices are suspended in an ordered dependency-aware sequence, CPUs and platform logic enter a global low-power state, and only selected wakeup sources are allowed to bring the machine back. Linux exposes several variants because “sleep” can mean anything from keeping DRAM powered while CPUs idle deeply to writing a memory snapshot to storage and removing power from RAM.

NORMAL RUNNING SYSTEM userspace + kernel + devices active ↓ request system sleep freeze userspace / prepare kernel subsystems ↓ suspend devices in dependency order ↓ late / noirq phases quiesce interrupt-driven device activity ↓ choose global sleep mechanism S2IDLE (suspend-to-idle) CPUs enter deep idle states DRAM remains powered platform may stay relatively awake wakeup comes through interrupt-capable paths SUSPEND-TO-RAM / "deep" devices suspended nonboot CPUs offline / platform enters a deeper low-power state DRAM retains memory contents only configured wake sources remain armed ↓ wake event platform + CPUs resume ↓ resume devices in dependency-aware order ↓ thaw userspace HIBERNATION freeze activity ↓ create snapshot of RAM state ↓ write hibernation image to persistent storage ↓ power can be removed from DRAM ↓ later boot/resume kernel restores image ↓ continue from saved system state

Mechanism	What remains true while asleep	Typical tradeoff
runtime PM	The operating system is still running; selected idle devices may enter low-power states.	Fine-grained savings with essentially no global suspend/resume cycle.
s2idle	Memory remains powered; CPUs idle deeply and devices are suspended, but no firmware-defined deep platform state is required.	Fast and broadly available, but platform power can be higher than deep suspend.
suspend-to-RAM / deep	RAM retains state while much more platform logic can enter low-power states.	Lower sleep power, usually with more platform/firmware involvement and resume latency.
hibernation	RAM contents are represented by a stored snapshot, so DRAM itself need not stay powered.	Very low long-duration power use, but image write/read makes entry and resume much slower.
wakeup source	A device/IRQ path explicitly permitted to abort suspend or wake a suspended machine.	Convenience versus unwanted wakeups and residual sleep power.

Do not equate platform sleep states with device D-states. Linux exposes generic suspend mechanisms such as s2idle, shallow and deep; the firmware/platform implementation underneath is machine-specific. A PCI device's D-state is a separate per-device power concept.

KERNEL DOCSLinux system sleep states

Current kernel documentation distinguishing suspend-to-idle, standby, suspend-to-RAM and hibernation, including the /sys/power/state and /sys/power/mem_sleep interfaces.

https://docs.kernel.org/admin-guide/pm/sleep-states.html

KERNEL DOCSSystem suspend and device interrupts

Explains how interrupt handling changes during suspend, how wake IRQs are armed and why suspend-to-idle treats interrupt-driven wakeups differently from deeper platform sleep.

https://docs.kernel.org/power/suspend-and-interrupts.html

# Inspect supported Linux sleep modes without changing anything: cat /sys/power/state cat /sys/power/mem_sleep 2>/dev/null # Inspect devices that expose wakeup policy: find /sys/devices -path '*/power/wakeup' -readable -print -exec cat {} \; 2>/dev/null | less # Do not write to these interfaces casually on a remote machine: # entering suspend can make the system unreachable until a valid wake event occurs.

Heat has a datapath too: junction → package → TIM → heatsink → air → thermal control loop

Electrical power dissipated in transistors becomes heat. The important temperature is often the silicon junction temperature, not room temperature. Heat flows through package materials, heat spreader, thermal-interface material and cooling hardware to ambient air/liquid. Sensors and firmware/OS/hardware control loops then trade fan speed, voltage, frequency and performance against thermal limits.

POWER GENERATION ON DIE dynamic CMOS power ≈ α · C · V² · f leakage/static power adds temperature- and process-dependent loss ↓ silicon junction / hotspots ↓ thermal resistance + thermal capacitance die attach / package substrate ↓ IHS / package top (when present) ↓ TIM: thermal interface material ↓ heatsink / cold plate ↓ airflow or liquid loop ↓ ambient environment STEADY-STATE FIRST APPROXIMATION ΔT ≈ P × θ Tjunction ≈ Tambient + P × θJA # only when θJA assumptions are appropriate Example: P = 50 W, effective thermal path = 0.8 °C/W idealized steady rise ≈ 40 °C above the reference ambient TRANSIENT REALITY thermal mass stores heat → temperature does not jump instantly electrical analogy: temperature ↔ voltage heat flow/power ↔ current thermal resistance ↔ electrical resistance thermal capacitance ↔ electrical capacitance CONTROL LOOP on-die / board temperature sensor ↓ compare against trip/target fan / pump increase frequency/voltage/power limits reduced ↓ temperature falls critical limit exceeded despite control ↓ hardware/firmware/kernel can force deeper throttle or shutdown.

Thermal term	Meaning
Tjunction / Tj	Temperature of semiconductor junction/die region; can be much hotter than ambient.
TjMax	Processor/device-specific maximum junction or thermal-control reference limit.
θJA	JEDEC-style junction-to-ambient thermal resistance in °C/W; useful but strongly dependent on test/environment assumptions.
θJC	Junction-to-case thermal resistance under specified test conditions.
TIM	Thermal Interface Material filling microscopic gaps between package/IHS and cooler to lower contact thermal resistance.
thermal capacitance	Heat-storage property making temperature respond over time instead of instantaneously.
hotspot	Small die region dissipating more power/temperature than package-average measurement suggests.
trip point	Temperature threshold at which software/firmware/hardware changes cooling/performance policy.
passive cooling	Reduce generated heat, e.g. lower CPU frequency/voltage/power.
active cooling	Increase heat removal, e.g. fan or pump speed.
thermal throttling	Automatic performance/power reduction to keep temperature below a protection limit.
thermal shutdown	Last-resort protection stopping/rebooting/powering down hardware when temperature cannot be controlled safely.

θJA is not a universal property you can blindly multiply by CPU watts. JEDEC theta measurements depend strongly on package, board, airflow and fixture assumptions. Use device/system thermal models or measurements for serious cooler design; the simple ΔT=Pθ model is mainly a first-order teaching/comparison tool.

LINUX THERMAL OBSERVATION LAB # kernel thermal zones for z in /sys/class/thermal/thermal_zone*; do echo ===$z=== cat "$z/type" "$z/temp" 2>/dev/null done # hwmon sensors (labels/availability vary) for h in /sys/class/hwmon/hwmon*; do echo ===$h $(cat "$h/name" 2>/dev/null)=== grep . "$h"/temp*_input 2>/dev/null | head done # if lm-sensors is installed sensors # correlate load, frequency and temperature watch -n 1 'grep -m1 "cpu MHz" /proc/cpuinfo; sensors 2>/dev/null | head -30' # thermal-zone units in sysfs are commonly millidegrees C. # Do not disable thermal protection or force unsafe power limits for an experiment.

KERNEL DOCSLinux generic thermal sysfs

Current kernel architecture for thermal zones, trip points and cooling devices. It exposes temperature, passive/active/critical trips, processor/fan cooling states and emergency behavior.

https://docs.kernel.org/driver-api/thermal/sysfs-api.html

KERNEL DOCSLinux coretemp — Intel on-die thermal sensors

Concrete sensor interface: core/package temperature, Tcontrol and TjMax-derived critical values from processor thermal registers.

https://docs.kernel.org/hwmon/coretemp.html

KERNEL DOCSLinux PECI CPU temperature interface

Exposes die temperature, Tcontrol, Tthrottle and TjMax/Tprochot concepts for supported Intel platforms.

https://docs.kernel.org/hwmon/peci-cputemp.html

CURRENT VENDOR DOCIntel — What is throttling? (reviewed April 2026)

Current 2026 Intel explanation: processor throttling reduces clock speed when temperature exceeds the configured junction/case limit to protect the processor.

https://www.intel.com/content/www/us/en/support/articles/000088048/processors.html

CURRENT VENDOR DOCAMD Ryzen Master 3.1 — thermal throttle control (May 2026)

Current 2026 AMD documentation defines Max Temperature as the temperature at which thermal throttling of clock frequencies and voltages occurs.

https://docs.amd.com/r/en-US/68886-ryzen-master-user-guide/Additional-Controls

TECH ARTICLEAnalog Devices — thermal design basics

Clear thermal/electrical analogy: ΔT = P×θ, thermal resistance in °C/W and the junction-to-ambient heat-flow model.

https://www.analog.com/en/resources/analog-dialogue/articles/maximize-power-capability-in-thermal-design-part-1.html

Boolean minimization, finite-state machines and the glitches ideal logic diagrams hide

A truth table tells you what a combinational circuit should compute, but there may be many gate networks that implement it. Boolean algebra and Karnaugh maps reduce redundant logic. Once storage is added, the design becomes a state machine: current state + inputs determine next state and outputs. Real gates also have unequal propagation delays, so logically equivalent paths can temporarily disagree and create glitches/hazards.

COMBINATIONAL DESIGN requirements → truth table → Boolean expression → simplify → gates SEQUENTIAL / FSM DESIGN requirements → states + transitions ↓ state register (flip-flops) ── current state ─┐ ├→ next-state combinational logic → D inputs external inputs ──────────────────────────────┘ clock edge → state register captures next state REAL-TIME PROBLEM input changes ├→ short logic path settles first └→ long logic path settles later ↓ temporary wrong output = glitch / hazard

Term	Meaning
minterm	Product term corresponding to one input combination for which a function is 1.
maxterm	Sum term corresponding to one input combination for which a function is 0.
SOP	Sum of Products: OR of AND/product terms.
POS	Product of Sums: AND of OR/sum terms.
Karnaugh map	Gray-code-arranged truth table used to spot groups and reduce small Boolean functions by inspection.
Moore FSM	Outputs depend on stored state (and not directly on current external input).
Mealy FSM	Outputs can depend on both stored state and current inputs.
static hazard	Output should remain 0 or 1 but briefly pulses to the opposite value because path delays differ.
dynamic hazard	Output should change once but toggles multiple times before settling.
registered output	Combinational result captured in a flip-flop at a clock edge; often used so downstream synchronous logic sees only settled values.

FREE TEXTBOOKKarnaugh maps, truth tables and Boolean expressions

Free public textbook page showing that relay logic, gate diagrams, truth tables, Karnaugh maps and Boolean equations are alternative descriptions of the same logical function.

https://www.allaboutcircuits.com/textbook/digital/chpt-8/karnaugh-maps-truth-tables-boolean-expressions/

FREE TEXTBOOKComplete free Boolean algebra / Karnaugh-map chapter index

Direct index to Boolean identities, DeMorgan's laws, truth-table conversion, Karnaugh maps, minterms/maxterms and logic simplification. No account.

https://www.allaboutcircuits.com/textbook/product/boolean-algebra/

FREE TEXTBOOKFinite State Machines — from state diagram to flip-flop circuit

Walks a small sequential design from behavioral description through state diagram, state encoding, Karnaugh-map equations and a final D/JK flip-flop circuit.

https://www.allaboutcircuits.com/textbook/digital/chpt-11/finite-state-machines/

FREE TEXTBOOKROM + register as a finite-state machine

Especially useful mental model: a ROM contains next-state data, its outputs feed back through a clocked register, and the stored state selects what happens next.

https://www.allaboutcircuits.com/textbook/digital/chpt-16/finite-state-machine/

OPEN NOTESMIT 6.004 lecture-note index — sequential logic, FSMs, synchronization and pipelines

Public notes with separate PDFs on sequential logic, storage/FSMs, synchronization/metastability and pipelining. No signup or enrollment.

https://ocw.mit.edu/courses/6-004-computation-structures-spring-2009/pages/lecture-notes/

From truth table to hardware: ROM, PLA, PAL, CPLD and FPGA LUT

There are several ways to turn a Boolean truth table into physical hardware. Fixed gates hard-wire one expression. A ROM stores every output value indexed by inputs. PLAs/PALs implement programmable sum-of-products structures. Modern FPGAs usually implement ordinary combinational logic in small look-up tables (LUTs) whose configuration bits encode the truth table.

BOOLEAN FUNCTION F(A,B) = A XOR B truth table: A B | F 0 0 | 0 0 1 | 1 1 0 | 1 1 1 | 0 FIXED GATES (~A & B) | (A & ~B) → transistor/gate network fixed at manufacture ROM VIEW {A,B} are a 2-bit ADDRESS stored bits at addresses 0,1,2,3 = 0,1,1,0 memory[address] = F 2-INPUT FPGA LUT VIEW 4 configuration bits implement that same truth table INIT[0]=0 for 00 INIT[1]=1 for 01 INIT[2]=1 for 10 INIT[3]=0 for 11 INIT[3:0] = 4'b0110 = hex 6 A,B ─→ mux/address-selection tree ─→ selected configuration bit ─→ F 6-input LUT: 6 inputs address 2^6 = 64 stored truth-table bits therefore one LUT can implement ANY one-output Boolean function of ≤6 inputs SYNTHESIS Verilog equation / case statement ↓ Boolean optimization technology mapping ↓ FPGA LUT INIT bits + dedicated mux/carry/FF/routing resources

Technology	Core logic idea	Typical character
ROM	Input bits address stored output word.	Can represent arbitrary truth table but stores entries for the full input space.
PLA	Programmable AND plane feeds programmable OR plane.	Flexible sum-of-products implementation.
PAL	Programmable AND terms feed fixed OR terms.	Simpler/faster/cheaper historical programmable glue logic.
SPLD	Small PAL/PLA-like programmable logic with macrocells.	Replaces handfuls of TTL glue-logic packages.
CPLD	Multiple PLD-like macrocells/blocks connected by programmable routing.	Larger deterministic control/glue logic than an SPLD.
FPGA LUT	Small truth-table memory/mux function generator plus registers, routing and dedicated resources.	Fine-grained highly parallel reconfigurable datapaths/control.
configuration memory	Bits controlling LUT contents and routing/mux switches.	SRAM-, flash- or other technology depending on FPGA family.
carry chain	Dedicated fast inter-LUT arithmetic path.	Avoids slow general routing for adders/counters/comparators.

'FPGA = a bag of gates' is an approximation. Modern devices contain LUTs, flip-flops, carry chains, block RAM, DSP blocks, clock networks, I/O SERDES and programmable routing. Synthesis maps Boolean/RTL structure onto those resources rather than literally instantiating a sea of discrete NAND gates.

OFFICIAL DOCSAMD LUT6 primitive — current 2026.1

Official current primitive documentation: a six-input LUT implements any six-input Boolean function and its 64-bit INIT directly encodes the truth table.

https://docs.amd.com/r/en-US/ug953-vivado-7series-libraries/LUT6

OFFICIAL DOCSAMD LUT6_2 — LUT as logic or small ROM

Shows that the same physical LUT structure can be viewed as a dual asynchronous 32-bit ROM, two 5-input functions, or a six-input function.

https://docs.amd.com/r/en-US/ug974-vivado-ultrascale-libraries/LUT6_2

OFFICIAL DOCSAMD Versal CLB — Look-Up Table architecture

Current architectural view of LUTs as the combinational building blocks inside a configurable logic block, with static-memory-controlled muxes and carry-related outputs.

https://docs.amd.com/r/en-US/am005-versal-clb/Look-Up-Table

OFFICIAL EXPLAINERMicrochip — What is an FPGA? PAL → CPLD → FPGA evolution

Readable no-login history: PALs use programmable AND/fixed OR arrays; CPLDs combine PLD blocks through routing; FPGAs evolved with smaller logic blocks and abundant programmable interconnect.

https://developerhelp.microchip.com/xwiki/bin/view/products/fpga/hello-fpga/what-is/

OFFICIAL DOCSMicrochip — PAL architecture

Current public documentation of programmable AND terms feeding fixed OR terms, output macrocells, feedback, output enables and registers.

https://onlinedocs.microchip.com/oxy/GUID-C8F8D1B0-413D-4C1A-AC0A-696EE4E657FF-en-US-1/GUID-AAADBF72-FA5D-416B-BF53-023FB302F4F7.html

OFFICIAL OVERVIEWMicrochip SPLD/CPLD portfolio — modern descendants of PAL/PLA glue logic

Useful historical-to-current bridge: 16V8/22V10-style SPLDs still exist and descend from PLA/PAL-type programmable glue logic.

https://www.microchip.com/en-us/products/fpgas-and-plds/spld-cplds

PUBLIC DOCSYosys technology mapping

Connects RTL/Boolean logic to real target primitives: synthesis maps generic logic into technology-specific cells/LUTs instead of leaving it as abstract operators.

https://yosyshq.readthedocs.io/projects/yosys/en/latest/using_yosys/synthesis/techmap_synth.html

4. State, latches, flip-flops, clocks, clock speed and timing

PUBLIC PROJECTBen Eater — clock module

Builds a computer clock around 555 timers, with adjustable automatic clocking and manual single-step mode. Useful for understanding what the clock physically is rather than treating CLK as magic.

https://eater.net/8bit/clock

OPEN NOTESMIT 6.004 — digital timing / dynamic discipline

Explains contamination delay, propagation delay, setup time, hold time, and how these constraints determine the minimum legal clock period.

https://ocw.mit.edu/courses/6-004-computation-structures-spring-2017/pages/c5/c5s1/

OPEN NOTESMIT 6.004 — synchronization and metastability

Shows why asynchronous inputs can violate setup/hold timing, why metastability cannot be wished away, and how digital systems reason about synchronization.

https://ocw.mit.edu/courses/6-004-computation-structures-spring-2017/pages/c6/c6s1/

OPEN NOTESMIT 6.004 — performance and pipelining

Explains the connection between combinational delay, clock period, latency, throughput, pipeline stages, and the slowest stage/critical path.

https://ocw.mit.edu/courses/6-004-computation-structures-spring-2017/pages/c7/c7s1/

PUBLIC WEBNandland — Setup and Hold Time

A compact practical explanation of setup time, hold time, propagation delay, and why increasing clock frequency eventually breaks a synchronous design.

https://nandland.com/lesson-12-setup-and-hold-time/

PUBLIC WEBNandland — Metastability

Practical explanation of what happens when a flip-flop samples data too close to a clock edge, especially useful before learning clock-domain crossing.

https://nandland.com/lesson-13-metastability/

What clock speed actually means

A clock is not a magic 'speed setting.' In a synchronous path, one register launches data on a clock edge; the data propagates through combinational logic; another register must receive stable data before its next active edge. The slowest such path is the critical path.

Tclock(min) ≳ tCQ + tlogic(max) + tsetup + clock_skew + jitter_margin

fmax ≈ 1 / Tclock(min)

So a higher clock frequency requires the entire register-to-register path to settle sooner. Pipelining can shorten each combinational stage and raise throughput, but it inserts more state and usually more latency. Clock distribution itself also consumes power and creates skew/jitter that designers must budget.

DIRECT PDFPackaging, power and clock distribution — David Harris (direct PDF)

Public technical slides on chip packaging, power delivery and clocks. Useful for moving from an abstract CLK wire to the physical problem of distributing a low-skew timing reference across a chip.

https://pages.hmc.edu/harris/class/e158/16/lect21.pdf

DIRECT PDFPLLs and DLLs — David Harris (direct PDF)

Twenty-one pages introducing phase-locked loops and delay-locked loops used to synthesize, align and distribute clocks. Covers the feedback-loop blocks rather than merely saying 'the clock comes from a crystal.'

https://pages.hmc.edu/harris/class/e158/16/lect22.pdf

TECH ARTICLEPhase-Locked Loop fundamentals — Analog Devices

Free public technical article: phase-frequency detector, charge pump, loop filter, VCO, dividers, lock, phase noise and frequency synthesis. Good bridge between oscillators and CPU clock generation.

https://www.analog.com/en/resources/analog-dialogue/articles/phase-locked-loop-pll-fundamentals.html

DIRECT PDFUnderstanding metastability in FPGAs — Altera/Intel (direct PDF)

No-login white paper on what happens when a flip-flop samples an asynchronous signal near its clock edge, why metastability cannot be completely eliminated, and how synchronizer MTBF is reasoned about.

https://cdrdv2-public.intel.com/650346/wp-01082-quartus-ii-metastability.pdf

Static timing analysis: proving that every register-to-register path meets setup and hold

A synchronous circuit is only correct if data arrives inside the capture flip-flop's legal timing window. Static Timing Analysis (STA) checks paths mathematically using characterized cell delays, interconnect parasitics and clock constraints instead of requiring an exhaustive functional simulation of every possible input sequence.

REGISTER-TO-REGISTER PATH launch clock ─→ [FF A] ──Q── combinational logic / wires ──D→ [FF B] ←─ capture clock │ │ └ tCQ ├ tSETUP └ tHOLD SETUP CHECK = maximum-delay / 'too late' problem zero-skew simplification: tCQ(max) + tDATA(max) + tSETUP + uncertainty ≤ Tclock HOLD CHECK = minimum-delay / 'too early' problem zero-skew simplification: tCQ(min) + tDATA(min) ≥ tHOLD + hold uncertainty STA vocabulary: arrival time = when data can actually reach endpoint required time = latest/earliest time architecture+constraints permit slack = margin between required and arrival times negative slack = violation REAL FLOW RTL → synthesis → gates + Liberty cell timing models + SDC clocks/input/output/exceptions + placed/routed SPEF parasitic R/C ↓ timing graph ↓ report worst setup / hold paths ↓ resize / buffer / restructure / move / reroute / adjust CTS ↓ timing closure

Timing term	Meaning
setup time	Data must already be stable this long before the active capture edge.
hold time	Data must remain stable this long after the active capture edge.
clock-to-Q (tCQ)	Delay from launch flip-flop's active clock edge until its Q output changes/validates.
cell delay	Delay through gates/buffers as a function of input slew, output capacitance, transition direction and PVT.
net delay	Delay from interconnect resistance/capacitance and coupling after placement/routing.
clock skew	Difference in clock arrival time at launch and capture registers.
jitter / uncertainty	Allowance for clock-edge variation and modeling/variation margin.
setup slack	How much later the path could become before violating setup; negative means too slow.
hold slack	How much minimum-delay margin exists before a too-fast path violates hold.
critical path	Path with limiting/worst timing margin for a particular timing check/domain.
false path	Path intentionally excluded because it cannot/need not satisfy the ordinary timing relationship.
multicycle path	Path intentionally allowed more than one clock cycle for setup, with hold constraints adjusted correctly.
PVT corner	Process/voltage/temperature operating condition used to bound cell/interconnect timing behavior.

Setup and hold fail in opposite directions. A setup path is too slow; adding clock period often helps. A hold path is too fast relative to the capture edge; lowering frequency generally does not help because the check concerns the same capture edge. Hold repair commonly adds data-path delay/buffers or changes clock-tree skew while preserving setup.

OPEN-SOURCE TOOLOpenSTA — open-source static timing analyzer

Real open STA engine. Reads Verilog netlists, Liberty timing libraries, SDC constraints, SDF and SPEF parasitics; models clocks, uncertainty, min/max delay, exceptions and timing reports.

https://github.com/The-OpenROAD-Project/OpenSTA

PUBLIC DOCSOpenROAD OpenSTA documentation

No-login documentation listing supported clock properties, exception paths, delay calculation and setup-check/report capabilities.

https://openroad.readthedocs.io/en/latest/main/src/sta/README.html

PUBLIC DOCSOpenROAD — complete physical-design flow

Places STA in context: floorplan → placement → clock-tree synthesis → setup/hold optimization → routing → parasitic extraction → final static timing analysis.

https://openroad.readthedocs.io/en/latest/main/README.html

DIRECT PDFCMOS VLSI — sequential timing (direct PDF)

Direct slides covering setup, hold, clock-to-Q, time borrowing and sequential timing at circuit level.

https://pages.hmc.edu/harris/cmosvlsi/4e/lect/lect10.pdf

Read timing diagrams as waveforms, not decoration

Datasheet timing diagrams and HDL traces are the same kind of reasoning tool: signals have values over time. The important questions are not just 'is CS low?' but 'when did CS become low relative to address, clock and data, and how long did every constraint remain satisfied?'

clock __/‾‾\__/‾‾\__/‾‾\__ address ==== A0 ====== A1 ====== chip_sel# ‾‾‾\________/‾‾\________ read# ‾‾‾‾\______/‾‾‾‾\______ data ZZZZ====D0====ZZZZ====D1= ↑ sample Things to measure: address setup → select/read assertion device access time → data valid data setup/hold around CPU sample point bus release → high-Z before next driver

Simulate a register with clock, D, Q and reset. Export a VCD/FST trace and open it in GTKWave.
Add combinational logic between two registers. Make the path longer and observe that Q-to-Q response spans finite time in a delay-aware simulator/model.
Trace a tiny bus cycle: address, chip-select, read/write, data-enable and data. Mark which device is allowed to drive the data lines.
Trace an FSM with current-state bits, next-state logic and outputs. Verify that state changes only at the intended clock event.
Compare your trace with a real 6502/Z80/68000 timing diagram. Identify which abstract signals in your toy system correspond to real pins.

OPEN-SOURCE TOOLGTKWave project

Open waveform viewer for standard Verilog VCD/EVCD and efficient FST traces.

https://github.com/gtkwave/gtkwave

PUBLIC DOCSVerilator trace options

Official source documentation for --trace-vcd and --trace-fst, depth/width controls and waveform generation.

https://github.com/verilator/verilator/blob/master/docs/guide/exe_verilator.rst

Protocol debugging with a logic analyzer: decode second, inspect the raw waveform first

A protocol decoder is only as trustworthy as the captured waveform and the settings you gave it. The most useful debugging habit is to identify the electrical idle level, transitions, bit time and transaction boundaries manually before asking software to print decoded bytes.

SAFE / USEFUL WORKFLOW 0. READ THE ANALYZER INPUT RATINGS confirm logic voltage, threshold, maximum input voltage and whether inputs are isolated 1. POWER OFF WHILE ATTACHING PROBES WHEN PRACTICAL analyzer GND → DUT GND / reference channels → signals of interest 2. CAPTURE RAW DIGITAL WAVEFORMS FIRST choose sample rate comfortably above edge/bit activity 3. IDENTIFY PROTOCOL FROM WIRES UART: idle TX/RX, no clock SPI: SCK + CS# + MOSI/MISO I²C: SCL/SDA both normally pulled HIGH 4. MEASURE BEFORE DECODING UART: bit width → baud estimate SPI: idle clock polarity + which edge has stable data I²C: START/STOP shape + clock rate + rise time 5. ENABLE DECODER WITH EXPLICIT SETTINGS compare decoded boundaries against raw transitions 6. DEBUG A BAD DECODE IN THIS ORDER wrong channel assignment? wrong threshold / voltage compatibility? inadequate sample rate? wrong baud / CPOL / CPHA / inversion / address format? noise/glitch/rise-time problem? transaction itself malformed? 7. CORRELATE WITH FIRMWARE put a GPIO marker around register writes / ISR / transaction start capture marker + protocol signals together now software event and bus event share one timeline

Measurement	What it diagnoses
idle voltage/state	Inversion, missing pull-up, wrong reference, disconnected device.
bit/clock period	Wrong configured baud/SCK rate or clock-divider programming.
setup time before sample edge	SPI data launched too late, excessive propagation delay or wrong CPHA.
I²C LOW→HIGH rise time	Pull-up too weak, capacitance too high, probe/load problem.
CS# setup/hold	Peripheral selection timing does not satisfy datasheet requirements.
ACK/NACK bit	Wrong I²C address, target not powered/configured, direction/transaction issue.
UART stop-bit level	Baud mismatch, line noise or frame-format mismatch.
unexpected narrow pulse	Glitch, crosstalk, signal-integrity issue or firmware toggling line incorrectly.

Grounding and voltage compatibility are safety issues, not decoder settings. Many USB logic analyzers are not galvanically isolated from the PC and share a common ground. Never assume RS-232, automotive, mains-referenced or differential signals are safe to connect directly; check the exact instrument's ratings and use suitable transceivers/isolation when required.

FREE TUTORIALSaleae — How to Use a Logic Analyzer

Free practical guide covering ground reference, signal probes, sample rate and protocol decoding.

https://articles.saleae.com/logic-analyzers/how-to-use-a-logic-analyzer

FREE TUTORIALSaleae — probe setup and loading

Explains why probe capacitance, ground lead placement and probe location increasingly matter as digital edge rates rise.

https://articles.saleae.com/logic-analyzers/logic-analyzer-tutorial-probe-setup

SAFETY DOCSSaleae — grounding / electrical isolation

Explicitly notes that Saleae inputs share a common ground and are not isolated from the USB-connected PC; at least one ground connection is required.

https://www.saleae.com/support/specifications-hardware/electrical-characteristics/are-the-ground-pins-required-for-each-input-used

SAFETY DOCSSaleae — safety and input-voltage limits

Current safety page showing that voltage limits vary by analyzer generation/model. Always use the rating for the exact instrument in hand.

https://www.saleae.com/support/specifications-hardware/product-comparison-and-selection/safety-and-warranty

SAFETY DOCSSaleae — differential/high-voltage signal guidance

Explains why RS-232/RS-485/RS-422 and higher-voltage logic may require compatible hardware, level shifting or a transceiver; older analyzers cannot safely accept those voltages directly.

https://www.saleae.com/support/protocol-analyzers/analyzer-user-guides/decode-differential-and-high-voltage-data

OPEN-SOURCE TOOLsigrok protocol decoders

Open-source ecosystem with more than a hundred protocol decoders, including I²C/SPI/UART and many device-specific layers.

https://www.sigrok.org/wiki/Protocol_decoders

PUBLIC HOWTOsigrok Protocol Decoder HOWTO

Advanced exercise: protocol decoders are Python modules, so you can inspect or write the state machine that turns raw sampled edges into higher-level transactions.

https://www.sigrok.org/wiki/Protocol_decoder_HOWTO

Four small serial-bus projects that force the wires to make sense

UART loopback: transmit a repeating 0x55 then 0x00/0xFF pattern. Measure bit time manually, decode it, deliberately select the wrong baud rate, and identify the framing failure in the waveform.
SPI shift-register loopback: connect MOSI to MISO in a simulator or safe hardware setup. Send known bytes in all four CPOL/CPHA modes and confirm which edge launches versus samples.
I²C EEPROM/register read: capture START → address+W → ACK → register index → repeated START → address+R → ACK → data → NACK → STOP. Then remove/disconnect the target and observe the address NACK.
Write a tiny protocol decoder: feed a saved UART/I²C/SPI capture into sigrok or your own Python state machine and emit decoded events. This makes protocol decoding itself a finite-state-machine exercise.

WHAT TO WRITE DOWN FOR EACH PROJECT wire names + voltage domain idle state who is allowed to drive each wire who supplies timing/clock what creates a frame/transaction boundary bit order acknowledgment/error mechanism what software register/FIFO starts the transfer what status/interrupt says it finished what you actually measured on the wire

GHz, MHz, MT/s, GT/s, GB/s, latency and IPC are not the same thing

Hardware specifications mix several different quantities. Keeping them separate prevents a lot of bad intuition about what 'fast' means.

Quantity	Typical unit	What it actually measures
Clock frequency	MHz / GHz	Clock cycles per second. It does not by itself tell you how much useful work is completed per cycle.
Transfer rate	MT/s / GT/s	Millions/billions of transfers per second on an interface. DDR memory and PCIe are commonly specified this way.
Bandwidth / throughput	MB/s / GB/s	Amount of payload/data that can move per second, after accounting for width and sometimes encoding/protocol overhead.
Latency	ns / µs / cycles	How long one operation or dependency takes before its result is available.
IPC	instructions/cycle	Instructions retired per CPU cycle for a workload; depends heavily on microarchitecture and the code.
CPI	cycles/instruction	Inverse-style view of instruction throughput: average cycles required per retired instruction.
IOPS	operations/s	Storage or I/O operations completed per second; says nothing by itself about bytes per operation or latency distribution.

DDR example: Double Data Rate memory transfers data on both clock edges. So the advertised MT/s figure is a transfer rate, not simply the oscillator frequency in MHz. This distinction becomes even more important when memory-controller clocks can run at ratios different from the DRAM data rate.

PLAIN WEBMT/s vs MHz — Kingston

A direct explanation of why DDR transfer rate is measured in MT/s and why it is not identical to the underlying clock frequency.

https://www.kingston.com/en/blog/pc-performance/mts-vs-mhz

KERNEL DOCSCPU Performance Scaling — Linux kernel documentation

Public kernel documentation explaining CPU operating performance points/P-states and the frequency/voltage tradeoff in modern processors.

https://docs.kernel.org/admin-guide/pm/cpufreq.html

Where the clock comes from, and why reset exists

A quartz crystal is a resonator, not a logic clock by itself. An oscillator circuit sustains electrical/mechanical oscillation around the crystal's resonance. Modern chips may then divide or multiply that reference with PLLs and distribute it through clock networks. Reset solves a different problem: flip-flops and analog circuits cannot be assumed to power up in the state your architecture needs, and supply voltage/clock may take time to become valid.

DIRECT PDFOscillator and Crystal Basics — Texas Instruments (PDF)

Technical application note on crystal equivalent circuits, load capacitance, negative resistance, oscillator startup and practical crystal behavior.

https://www.ti.com/lit/an/swra495i/swra495i.pdf

TECH NOTECrystal basics and PCB layout — Microchip AN2648

Public application note explaining crystal basics, oscillator testing and PCB-layout considerations. No course or enrollment.

https://www.microchip.com/en-us/application-notes/an2648

TECH NOTESReset and supervisor documentation — STMicroelectronics

Public application notes explaining power-up reset, voltage monitoring and how reset circuits are connected to microprocessors.

https://www.st.com/en/reset-and-supervisor-ics/resets-and-voltage-detectors/documentation.html

TECH ARTICLEMicroprocessor supervisor basics — Analog Devices

Explains power-on reset, power-fail detection, watchdogs and why a processor may need external supervision as rails rise and fall.

https://www.analog.com/en/resources/design-notes/supervisor-ics-monitor-batterypowered-equipment.html

One pushbutton crosses analog, asynchronous and software boundaries before it becomes an interrupt

A mechanical button is a good antidote to the idea that digital inputs are born as perfect 0s and 1s. Contacts bounce; wires pick up noise; a pin may float without a bias resistor; and the transition arrives asynchronously relative to the CPU/peripheral clock. A reliable input path may therefore use several layers: pull-up/down, input buffer or Schmitt hysteresis, synchronizer, debounce/filter, edge detector, interrupt latch and software acknowledgment.

PHYSICAL BUTTON INPUT EXAMPLE VDD | Rpullup | +------ GPIO pin ------ button ------ GND released → pull-up produces HIGH pressed → contact pulls LOW REAL PRESS ideal: 111111111000000000 physical: 111110101001000000 ← mechanical contact bounce/noise ↑ asynchronous to GPIO module clock possible input chain: pad / ESD protection ↓ CMOS input buffer / optional Schmitt trigger ↓ hysteresis rejects slow/noisy threshold chatter clock-domain synchronizer ↓ reduces metastability propagation probability digital debounce/noise filter ↓ require stable state for N clock samples / time interval edge detector ↓ falling/rising edge event interrupt-state latch ↓ interrupt controller → CPU trap/ISR ↓ software reads DATA_IN / event state ↓ W1C interrupt-status bit acknowledges/clears latched event Not every MCU/SoC puts every block above in hardware. Sometimes debounce is software; some pads lack Schmitt mode; always read the actual pad/GPIO/controller documentation.

Problem	Typical solution
floating input	Pull-up or pull-down resistor establishes a defined idle level.
slow/noisy threshold crossing	Schmitt-trigger input hysteresis or suitable analog conditioning.
metastability risk	Synchronizer stages before synchronous logic uses an asynchronous input.
mechanical bounce	Hardware digital filter, RC/Schmitt network, timer-based software debounce or state-machine filter.
short glitch	Input filter requiring stable samples or minimum pulse width.
event notification	Rising/falling/level interrupt generation after filtering/synchronization.
lost edge before software runs	Latched interrupt-state/status bit retains event until acknowledged.
simultaneous software/hardware register update	W1C/masked access avoids unsafe read-modify-write races.
level mismatch	Proper level shifter/input tolerance; never assume a 5 V source is safe for a 1.8/3.3 V GPIO.

Schmitt hysteresis and debouncing are different. Hysteresis makes the analog threshold less sensitive to noise/slow edges. Debouncing handles repeated real electrical transitions caused by a mechanical contact or similar source. A design can need both.

LOGIC-ANALYZER PROJECT capture the raw button node while pressing/releasing 20 times ↓ measure bounce duration and number of transitions ↓ enable hardware/software debounce ↓ capture both RAW pin and a GPIO marker toggled by the ISR ↓ compare: raw contact transition time filter/debounce delay interrupt latency ISR software response time This turns 'button press' into a measurable cross-layer event.

CURRENT LABMicrochip — GPIO Input Handling and Debouncing (2026)

Current July 2026 lab explicitly shows contact bounce producing multiple transitions and uses input debouncing to ensure one stable event per actuation.

https://developerhelp.microchip.com/xwiki/bin/view/products/mcu-mpu/32bit-mcu/ap-exercises/lab6/step1/

CURRENT LABMicrochip — complete 2026 GPIO switch lab

No-login lab tying GPIO inputs, pull-up/down concepts, external interrupts, source files and hardware debouncing together.

https://developerhelp.microchip.com/xwiki/bin/view/products/mcu-mpu/32bit-mcu/ap-exercises/lab6/

OFFICIAL DOCSMicrochip — Schmitt Trigger input (2026.1)

Current device documentation explicitly states that Schmitt-trigger hysteresis helps filter receiver noise and avoid double-glitching from noisy edges.

https://onlinedocs.microchip.com/oxy/GUID-AFCB5DCC-964F-4BE7-AA46-C756FA87ED7B-en-US-21/GUID-F215C692-D2D1-4486-8A0C-912B223F39DD.html

PUBLIC DOCSOpenTitan GPIO — real input filter and interrupt hardware

Open SoC GPIO implementation: configurable rising/falling/level interrupts and an optional input filter requiring the input to remain stable for 16 module-clock cycles before the register/interrupt path sees the change.

https://opentitan.org/book/hw/ip/gpio/

PUBLIC DOCSOpenTitan GPIO programmer's guide

Concrete register programming for enabling filtered GPIO edge/level interrupts and clearing latched interrupt state.

https://opentitan.org/book/hw/ip/gpio/doc/programmers_guide.html

PUBLIC REGISTERSOpenTitan GPIO register map

Real DATA_IN, edge/level enable, filter-enable and RW1C INTR_STATE fields—perfect bridge from raw input pin to the MMIO register-semantics section.

https://opentitan.org/book/hw/ip/gpio/data/gpio.html

APPLICATION NOTEMicrochip AC161 — Schmitt triggers for slow input edges

Older but still useful application note focused on why ordinary CMOS inputs dislike slow/noisy transitions and how Schmitt-trigger hysteresis helps.

https://www.microchip.com/en-us/application-notes/ac161

Clock-domain crossing: why two perfectly good clocks can still break a design

If two clock domains have no guaranteed phase relationship, a signal launched in one domain can change inside the receiving flip-flop's setup/hold aperture. That receiving flip-flop can enter metastability: not a legal 0 or 1 for some unpredictable settling time. You cannot eliminate the possibility; you design the crossing so the probability of unresolved metastability reaching useful logic becomes acceptably tiny.

SINGLE-BIT STATUS / LEVEL source domain signal ↓ [FF1] ← destination clock ← FF1 may go metastable ↓ allow one full clock period to settle [FF2] ← destination clock ↓ destination logic uses FF2 MULTI-BIT STREAM source data + write clock ↓ dual-clock / asynchronous FIFO ├── write pointer in source clock domain ├── read pointer in destination clock domain ├── Gray-coded pointer crossings └── synchronized FULL/EMPTY state ↓ destination data + read clock Do NOT simply put an independent 2-FF synchronizer on every bit of a changing bus: different bits can be captured from different source words.

Crossing	Typical technique
slow single-bit level	Two- or three-flip-flop synchronizer in destination domain.
single-cycle pulse	Pulse stretching, toggle synchronizer or request/acknowledge handshake so destination cannot miss it.
multi-bit control word	Handshake: hold data stable while a synchronized valid/request crosses, then acknowledge.
continuous data stream	Dual-clock/asynchronous FIFO.
counter/pointer crossing	Often Gray code so only one encoded bit changes between adjacent values, then synchronize.
asynchronous reset release	Common practice is asynchronous assertion with controlled/synchronous deassertion per clock domain.

OFFICIAL DOCSAMD UltraScale — asynchronous clock-domain crossing

Current public vendor documentation: asynchronous crossings require synchronization; single/Gray-coded values can use register synchronizers, while multi-bit buses are generally handled with dual-clock FIFOs.

https://docs.amd.com/r/en-US/ug574-ultrascale-clb/Asynchronous-Clock-Domain-Crossing

OFFICIAL DOCSAMD 2026.1 design methodology — Clock Domain Crossing

Current 2026 guidance on CDC reliability, synchronizer recognition, ASYNC_REG placement and report_cdc methodology.

https://docs.amd.com/r/en-US/ug949-vivado-design-methodology/Clock-Domain-Crossing

OFFICIAL DOCSAMD dual-clock FIFO documentation

Explains a FIFO whose read/write clocks may have unrelated frequency and phase, with each interface synchronous only to its own domain.

https://docs.amd.com/r/en-US/ug573-ultrascale-memory-resources/Independent-Clock/Dual-Clock-FIFO

TECH ARTICLEZipCPU — asynchronous FIFO, Gray pointers and formal verification

Deep practical walkthrough of why independent synchronizers on a binary multi-bit pointer fail, why Gray-coded pointers help, and how async FIFO full/empty state is crossed safely.

https://zipcpu.com/blog/2018/07/06/afifo.html

Reset is a distributed protocol: POR, reset trees, watchdogs and safe deassertion

Reset is not just a button wired to every flip-flop. Real systems have several reset causes, multiple voltage/clock domains, different cold/warm-reset scopes and reset trees that must release logic only when its clock/power assumptions are valid. An asynchronous reset can assert immediately, but its deassertion is itself a clock-domain-crossing problem.

POWER RAMP VDD rises ↓ POR / power-good circuitry filters + qualifies supply root POR asserted while supply/clock state is unsafe ↓ oscillator / PLL / clock muxes stabilize ↓ reset manager distributes reset trees per clock/power domain COMMON ASYNC-ASSERT / SYNC-DEASSERT PATTERN async_reset_n ─────────┐ ↓ async clear dest_clk ─────────→ [FF1] → [FF2] → reset_n_for_domain ↑ ↑ clocked release stages assertion: external/root reset can force FFs into reset immediately deassertion: reset removal propagates only on destination-clock edges → avoids arbitrary asynchronous release relative to local sequential logic OTHER RESET SOURCES watchdog timeout ─┐ software reset ──┼→ reset manager / cause logic → selected reset trees debug reset ──┤ brownout/error ───┤ security fault ───┘ RESET_INFO / cause register lets firmware distinguish cold POR, watchdog, software, low-power exit and other reasons.

Reset term	Meaning
POR / power-on reset	Root reset generated while supply/clock conditions are not yet safe after power application.
power-good	Analog/digital indication that a rail has reached acceptable conditions; often filtered/delayed before reset release.
cold reset	Broad reset approximating power-on initialization; usually clears more state/domains.
warm reset	Narrower reset preserving selected always-on/platform state while restarting CPU/system portions.
watchdog reset	Hardware timer reset caused when software fails to prove forward progress before timeout.
reset tree	Distribution network producing correctly scoped/timed reset signals for many consumers/domains.
reset synchronizer	Sequential logic ensuring reset release occurs in a safe relation to a destination clock.
reset-domain crossing	Verification/design problem created where reset generation/release interacts with one or more clock domains.
reset cause	Latched reason for the latest reset, used by boot software for diagnosis/recovery policy.
minimum reset pulse width	How long reset must remain asserted so all targeted sequential elements reliably recognize it.

Asynchronous assertion / synchronous deassertion is a common pattern, not a universal commandment. Some blocks use fully synchronous reset; others use dedicated POR circuitry. What matters is that reset release obeys the destination logic's clocking/timing assumptions and that the verification methodology knows the intended reset architecture.

OFFICIAL DOCSAMD XPM_CDC_ASYNC_RST — current 2026.1 reset synchronizer

Current official primitive: reset output asserts asynchronously from the input but always deasserts synchronously to the destination clock, with configurable synchronizer depth/minimum pulse behavior.

https://docs.amd.com/r/en-US/ug953-vivado-7series-libraries/XPM_CDC_ASYNC_RST

OFFICIAL DOCSAMD XPM_CDC_SYNC_RST — fully synchronous reset synchronizer

Useful comparison: this macro makes both assertion and deassertion synchronous to the destination clock.

https://docs.amd.com/r/en-US/ug953-vivado-7series-libraries/XPM_CDC_SYNC_RST

PUBLIC DOCSOpenTitan Reset Manager — theory of operation

Excellent real-SoC reset architecture: POR tree, filtered/stretched power indication, per-clock reset synchronization, reset causes, low-power/warm resets, watchdog/software/debug/security reset requests.

https://opentitan.org/book/hw/top_earlgrey/ip_autogen/rstmgr/doc/theory_of_operation.html

PUBLIC DOCSOpenTitan Reset Manager — top-level documentation

Shows reset interfaces and generated reset trees in a security-oriented open SoC rather than an isolated textbook circuit.

https://opentitan.org/book/hw/top_earlgrey/ip_autogen/rstmgr/

A hardware watchdog is a deadline for proving the machine is still making progress: heartbeat → timeout → recovery/reset

A watchdog timer is deliberately simple hardware: software must periodically prove that the system is alive before a countdown expires. Linux exposes hardware watchdogs through the watchdog core and devices such as /dev/watchdog0. A userspace supervisor can open the device, configure a timeout, and send keepalives. If the system stops refreshing the watchdog because the daemon, kernel, scheduler, or broader platform has failed, the hardware can assert reset or another platform-specific recovery action.

BOOT / NORMAL OPERATION firmware may leave hardware watchdog running ↓ Linux watchdog driver registers watchdog_device ↓ watchdog core exposes /dev/watchdog0 (and legacy /dev/watchdog for id 0) ↓ userspace watchdog/supervisor opens device ↓ WDIOC_SETTIMEOUT / driver timeout policy ↓ periodic write() or WDIOC_KEEPALIVE ↓ kernel watchdog core / driver pets hardware timer ↓ hardware countdown reloads FAILURE heartbeat stops long enough ↓ optional PRETIMEOUT event / interrupt where supported ↓ final watchdog expiry ↓ platform action: usually reset/reboot, sometimes NMI/other recovery mechanism ↓ next boot can inspect reset-cause state and choose recovery/fallback policy NOWAYOUT if watchdog is configured as non-stoppable once armed, closing the fd cannot silently remove the recovery deadline.

Mechanism	What it means
hardware watchdog timer	Independent countdown circuit/peripheral that triggers a recovery action unless refreshed before expiry.
`/dev/watchdogN`	Linux userspace interface to a registered watchdog device.
heartbeat / keepalive	Software action that reloads or otherwise proves liveness to the watchdog before the timeout.
timeout	Maximum allowed interval between valid keepalives before final watchdog action.
pretimeout	Optional earlier notification before final expiry, useful for diagnostics or crash capture when hardware supports it.
`nowayout`	Policy preventing an armed watchdog from being disabled, so failure of the watchdog daemon cannot accidentally remove the safety net.
boot-enabled watchdog handoff	Kernel watchdog core can keep some already-running watchdogs refreshed while boot proceeds until userspace assumes responsibility.
reset-cause register	Platform state indicating whether the previous restart came from watchdog, power-on, software reset, brownout, etc.

A heartbeat must represent useful progress. A trivial high-priority thread that keeps petting the watchdog even while the application, storage path, or control loop is irrecoverably stuck defeats the design. Production watchdog daemons often gate the keepalive on multiple health checks. Also do not confuse a hardware reset watchdog with Linux's soft-lockup/hard-lockup detector infrastructure; they can complement each other, but they are different mechanisms.

OBSERVATION LAB (do not arm a production watchdog casually) ls -l /dev/watchdog* 2>/dev/null cat /sys/class/watchdog/watchdog0/{identity,timeout,timeleft,status,nowayout} 2>/dev/null # Driver/module-specific state may also be visible under: find /sys/class/watchdog/watchdog0 -maxdepth 1 -type f -print 2>/dev/null # Reset-cause reporting is platform-specific; inspect firmware/platform docs # before experimenting with watchdog expiry.

KERNEL DOCSLinux watchdog userspace API

Canonical description of /dev/watchdog, keepalive writes/ioctls, timeouts, Magic Close and nowayout-style behavior.

https://docs.kernel.org/watchdog/watchdog-api.html

KERNEL DOCSLinux watchdog core kernel API

Shows the watchdog_device abstraction, driver operations, timeout/pretimeout fields, registration and the framework connecting hardware drivers to userspace.

https://docs.kernel.org/watchdog/watchdog-kernel-api.html

KERNEL DOCSLinux watchdog parameters and boot handoff

Current watchdog-core and driver parameters, including handling of boot-enabled watchdogs, userspace open timeout and nowayout controls.

https://docs.kernel.org/watchdog/watchdog-parameters.html

From dead board to first instruction: what power-on actually does

A computer cannot begin with 'the operating system starts.' Before any useful software runs, the electrical machine has to reach a state in which supply rails are valid, clocks are oscillating, reset is released, the CPU knows where to fetch from, and at least some executable storage is readable.

power source ↓ regulators / VRMs establish rails ↓ power-good / reset logic holds chips reset ↓ reference oscillator starts ↓ PLL / dividers / clock tree become usable ↓ reset is released ↓ CPU architectural state is forced to defined reset values ↓ PC / reset-vector mechanism chooses first fetch address ↓ boot ROM / firmware instruction is fetched ↓ firmware initializes DRAM + devices ↓ later boot stage / bootloader is loaded ↓ OS kernel is loaded and entered

The power circuitry raises supply rails. Real systems may have several rails that must appear in a specified order.
Reset logic keeps the CPU and peripherals from executing while voltage and clocks are unstable.
An oscillator starts from a crystal, resonator, RC network, MEMS oscillator, or other reference. PLLs may synthesize faster clocks from it.
When reset is released, the CPU does not contain your operating system. It has architecturally defined reset state and a defined way to choose its first instruction address.
That first instruction normally resides in ROM/flash or another boot-visible device. Early firmware has to initialize hardware that was not usable at reset.
On many modern systems, external DRAM is not usable until firmware configures the memory controller and trains the DRAM interface.
A bootloader or later firmware stage can then inspect storage, parse an image/filesystem, copy/decompress the kernel into RAM, construct boot parameters, and branch to the kernel entry point.
The kernel then installs its own page tables, interrupt/trap handlers, drivers, scheduler, and other state before ordinary programs run.

PUBLIC DOCSU-Boot: booting from tiny early stages (TPL/VPL/SPL)

Plain public documentation. Shows a very real reason multi-stage boot exists: the main bootloader may be too large to load before DRAM is initialized. SPL commonly sets up SDRAM, then loads larger firmware or the kernel.

https://docs.u-boot.org/en/v2026.01/usage/spl_boot.html

PUBLIC DOCSU-Boot board-initialization flow

Shows execution beginning in architecture-specific assembly and proceeding through low-level initialization. Good antidote to the idea that firmware starts in a fully initialized C environment.

https://docs.u-boot.org/en/stable/develop/init.html

KERNEL DOCSLinux x86 boot protocol

Official Linux documentation for the handoff from bootloader to kernel, including the conventional memory layout and boot parameters. Advanced, but concrete.

https://docs.kernel.org/arch/x86/boot.html

SOURCE / BOOKxv6 RISC-V source and book

MIT's public page points directly to the xv6 RISC-V source and explanatory text. No account is required. Useful once you want to see what a tiny kernel does after firmware hands it control.

https://pdos.csail.mit.edu/6.1810/2025/xv6.html

Where firmware actually lives: SPI/QSPI NOR flash, erase-before-program and execute-in-place

Many PCs, routers, microcontrollers and SoCs store boot firmware in external serial NOR flash. It behaves very differently from RAM: reads are random and comparatively easy; programming changes erased 1-bits toward 0; returning cells to the erased state requires erasing a much larger sector/block first. Firmware update software must work around that asymmetry.

SERIAL NOR DEVICE host / SPI controller CS# SCK IO0..IO3 (SPI / Dual / Quad depending mode) ↓ serial NOR flash array READ command + address [+ dummy cycles] → flash shifts out bytes address can auto-increment for sequential reads PROGRAM WREN # set Write Enable Latch PAGE PROGRAM addr data... CS# rises → internal high-voltage/self-timed program begins poll STATUS.BUSY until complete typical page boundary e.g. 256 bytes on many parts ERASE WREN SECTOR ERASE address poll BUSY whole erase unit returns to 0xFF / all 1s Important asymmetry: read byte: cheap / random program bits: usually 1 → 0 within already-erased page erase: large sector/block, resets bits to 1 XIP / MEMORY-MAPPED READ CPU fetch address in flash window ↓ memory controller/SPI-XIP unit turns fetch into serial flash reads ↓ instruction bytes returned as if flash occupied normal address space On a PC, firmware may be copied/decompressed into faster RAM/cache later; the serial flash remains the nonvolatile backing store.

NOR concept	Meaning / consequence
JEDEC ID	Standard command-readable manufacturer/device identification used to select capabilities/parameters.
SFDP	Serial Flash Discoverable Parameters: standard tables describing erase types, timings, read modes and capabilities.
page program	Programs a bounded page-sized span; crossing a page boundary may wrap or require a new command depending on device.
sector erase	Returns a larger erase unit to all-1 state before future programming.
WREN / WEL	Write-enable command/latch preventing accidental program/erase until explicitly armed.
BUSY bit	Status bit software polls while self-timed erase/program operation is active.
block protection	Protection bits/regions preventing program/erase of selected flash areas.
XIP	Execute In Place: CPU instruction/data accesses are translated into flash reads without pre-copying entire image to RAM.
Quad I/O	Uses multiple serial data pins per clock to raise read/program bandwidth versus 1-bit SPI.
firmware descriptor/partition	Logical organization dividing one flash chip into bootblock, firmware volumes, settings, recovery images or vendor-specific regions.

Updating firmware is not like overwriting RAM. A robust updater must respect erase units, power-failure windows, protection bits and image authenticity. That is why many systems use recovery partitions, A/B slots or an immutable first-stage boot block rather than rewriting the only bootable image in one unsafe step.

DIRECT DATASHEETMicrochip SST26VF064B — Serial Quad I/O flash datasheet (direct PDF)

Concrete serial NOR datasheet: up-to-256-byte page programming, write enable, BUSY polling, sector/block/chip erase, protection and SPI/quad commands.

https://ww1.microchip.com/downloads/en/DeviceDoc/SST26VF064B-SST26VF064BA-Serial-Quad-IO-SQI-Flash-Memory-Data-Sheet-DS20005119H.pdf

OFFICIAL DOCSMicrochip — external serial flash organization

Readable public example explaining 256-byte pages, 4-KiB sectors, page program, erase-before-program and sequential reads.

https://onlinedocs.microchip.com/oxy/GUID-324A966D-1464-4B35-A7D1-DCAE052AC22C-en-US-5/GUID-BD447F5D-7DA4-4462-8CF4-35D6ECDC77FD.html

OFFICIAL DOCSMicrochip — SFDP driver

Current public implementation docs for discovering JEDEC/SFDP flash parameters and performing page writes, sector/bulk/chip erases and Quad-I/O operation.

https://onlinedocs.microchip.com/oxy/GUID-EDFB1AB8-CD6B-446F-8E25-F2167287A1AF-en-US-5/GUID-89E68D44-2219-4AF3-9593-A4A25F50CE08.html

OFFICIAL DOCSMicrochip — SQI execute-in-place support

Concrete controller documentation showing serial-flash commands plus execute-in-place (XIP) setup.

https://onlinedocs.microchip.com/oxy/GUID-450989FA-38E4-4D68-AB61-15ADB29AD718-en-US-6/GUID-E654FE2A-B9EE-464E-9EBE-BD84A46B723C.html

PUBLIC DOCScoreboot CBFS — firmware files in flash

Shows how real PC firmware stores bootblock/stages/files in a ROM image and how the x86 reset-vector region maps to the top of firmware flash.

https://doc.coreboot.org/lib/cbfs.html

A real PC boot detail: x86 reset vector, SPI flash, coreboot/UEFI and pre-x86 firmware

The generic power-on section explains reset abstractly. A conventional x86 core has a very specific architectural start point: after reset it begins at the reset-vector location conventionally visible at physical address 0xFFFFFFF0. Firmware images therefore arrange their first-stage boot code at the top of the addressable flash mapping. But this is not necessarily the first code anywhere in the platform: modern Intel/AMD systems can run security/authentication processors or authenticated modules before the normal x86 reset-vector code.

ATX power / rails / power-good ↓ chipset / SoC reset sequencing ↓ [platform-dependent early security firmware may execute here] ↓ x86 BSP released to architectural reset state ↓ physical reset-vector mapping ≈ 0xFFFFFFF0 ↓ top of SPI firmware image / bootblock ↓ very early assembly; DRAM usually not ready yet ↓ temporary execution/storage techniques + silicon/chipset init ↓ DRAM training / memory-controller init ↓ larger firmware stages in RAM ↓ UEFI environment or another payload / bootloader ↓ OS loader → kernel

Modern caveat: 'the CPU's first instruction is at 0xFFFFFFF0' describes the traditional x86 architectural reset vector. For example, coreboot documents AMD Family 17h systems where the PSP boot ROM verifies/initializes firmware and DRAM before releasing the x86 processor. Intel TXT platforms can likewise execute an authenticated code module before the traditional reset vector.

PUBLIC DOCScoreboot architecture — bootblock, romstage and ramstage

Explains real early firmware staging: bootblock runs immediately after CPU reset and must establish enough environment for later C code and DRAM-dependent stages.

https://doc.coreboot.org/getting_started/architecture.html

PUBLIC DOCSAMD Family 17h early boot flow in coreboot

Public technical documentation showing the PSP boot ROM and off-chip bootloader running before the x86 core, DRAM initialization before release, and the eventual x86 reset-vector behavior.

https://doc.coreboot.org/soc/amd/family17h.html

PUBLIC DOCSIntel TXT boot flow — code before the traditional x86 reset vector

Documents a case where an Intel authenticated code module executes before the normal x86 reset vector, illustrating why platform boot is more complex than the architectural CPU reset state.

https://doc.coreboot.org/security/intel/txt.html

OFFICIAL SPECSUEFI Forum specification downloads

Official direct-open specification page. No account is required merely to read/download the specs. Current page lists UEFI 2.11 and Platform Initialization 1.10 among the latest versions.

https://uefi.org/specifications

OFFICIAL SPECUEFI Specification 2.11 — direct download page

Official page exposing the UEFI 2.11 PDF directly. Use after understanding the simpler boot chain; this is the normative interface between platform firmware and OS loaders/pre-boot applications.

https://uefi.org/node/5130

OFFICIAL SPECUEFI Platform Initialization Specification 1.10 — direct download page

Current 2026 PI specification download page. This is deeper than the UEFI boot-services interface and describes phases/services used by platform firmware initialization.

https://uefi.org/node/5233

CPU behavior can be patched after fabrication: x86 microcode updates at boot

The microcode-control section explains microcode as one possible internal implementation technique for turning architectural instructions into lower-level control actions. Modern x86 processors add another layer: vendors can publish microcode update blobs that patch selected internal behavior after the silicon has shipped. These patches are commonly used for processor errata and security mitigations. They do not replace the CPU's ISA or turn an arbitrary processor into a different design; they update implementation-defined internal behavior supported by the processor's update mechanism.

POWER-ON / RESET ↓ CPU starts with microcode revision built into silicon / platform state ↓ firmware may apply a vendor microcode patch ↓ bootloader loads kernel + initrd ↓ Linux EARLY MICROCODE LOADER scans initrd ↓ matching vendor/family/model patch found? ├── no → continue with current revision └── yes → apply patch to bootstrap processor (BSP) ↓ bring up application processors (APs) ↓ apply matching cached patch to each CPU/core as required ↓ normal kernel initialization continues SUSPEND / RESUME cached matching patch can be reapplied when CPUs return from a state where the update would otherwise be lost. LATE LOAD /sys/devices/system/cpu/microcode/reload exists on supported setups, but modern Linux does not enable late loading by default because a live update must safely synchronize CPUs and preserve software-visible behavior.

Layer	What it contributes
CPU silicon	Contains the hardware update mechanism and a baseline microcode revision.
CPU vendor	Produces model-specific update data intended for compatible processors.
firmware	May apply microcode before the OS starts, depending on platform design and firmware version.
initrd / firmware files	Linux distributions commonly carry Intel or AMD microcode blobs so the kernel can apply a newer matching revision very early.
early kernel loader	Applies a matching patch before normal kernel initialization has exposed much CPU behavior to software.
late loader	Attempts a live update on running CPUs; it is harder to make universally safe because all relevant logical CPUs and software-visible state must remain coherent.

Microcode is below the ISA contract, but a patch can still matter to software. It may change how an instruction, MSR or speculation behavior is implemented, and mitigation status can depend on the revision present during boot. That is why distributions normally prefer an early update rather than treating microcode like an ordinary hot-swappable driver.

KERNEL DOCSLinux x86 microcode loader

Current kernel documentation for early loading from the initrd, application to BSP/AP CPUs, cached patches across resume and the hazards of late loading.

https://docs.kernel.org/arch/x86/microcode.html

KERNEL DOCSLinux hardware-vulnerability note — old microcode

Shows the practical security connection: the kernel can report systems whose boot-time microcode predates revisions needed for known CPU issues and points users toward distribution/vendor microcode updates.

https://docs.kernel.org/admin-guide/hw-vuln/old_microcode.html

# Inspect the revision Linux reports for each logical CPU: grep -E '^(processor|microcode)' /proc/cpuinfo # Kernel boot messages often show loader/update status: dmesg | grep -i microcode # Distribution-specific package names vary; do not copy a random blob # from another CPU model. Use the platform/distribution update path.

System Management Mode is firmware execution below the OS: SMI → SMRAM handler → RSM

x86 has an execution environment that is deliberately separate from normal operating-system privilege rings. A System Management Interrupt (SMI) causes the processor to enter System Management Mode (SMM), save architectural state in a protected management-memory region, execute firmware code, and later return with RSM. The interrupted kernel or application normally does not participate in this transition and resumes afterward as though it merely experienced an unexplained pause.

NORMAL EXECUTION application / kernel / hypervisor code ↓ platform condition or software-defined SMI source processor recognizes SMI ↓ enter SMM save interrupted CPU state in SMM save-state area switch to SMM execution environment / SMBASE ordinary maskable interrupts initially inhibited ↓ execute firmware handler from protected SMRAM / MMRAM inspect platform state perform platform-management work ↓ RSM (resume from system management mode) ↓ restore saved architectural state ↓ resume the instruction stream that was interrupted From the OS perspective: normal execution ────────[time disappears into SMM]──────── normal execution

Mechanism	What it means
SMI	Special system-management interrupt that transfers execution into SMM. It is not an ordinary maskable IRQ that the OS can simply disable with normal interrupt masking.
SMM	CPU operating mode intended for platform firmware/system-management work that is normally transparent to the OS.
SMRAM / MMRAM	Memory reserved for management-mode code/data and normally hidden or protected from ordinary OS execution after firmware initialization.
SMBASE	x86 base associated with the SMM entry environment; firmware commonly relocates management-mode state during initialization.
save-state area	Processor state captured so the SMM handler can later restore the interrupted context.
RSM	x86 instruction used to leave SMM and resume the saved execution context.
latency consequence	Time spent in SMM can appear to the OS as unexplained scheduling/interrupt latency because ordinary software is not executing while the CPU is in the handler.

SMM is not “ring −2” in the architectural privilege-ring model. That nickname is sometimes used informally, but the useful mental model is a separate firmware execution mode with protected memory and its own entry/return mechanism. It is also different from VMX root/non-root virtualization modes.

DIRECT MANUAL PDFIntel SDM Volume 3C — System Management Mode

Official architecture manual. Chapter 34 documents SMI entry, the SMM state-save map, SMBASE/SMRAM behavior, interrupt handling while in SMM and RSM.

https://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-vol-3c-part-3-manual.pdf

OFFICIAL SPECUEFI PI 1.9 — Management Mode Core Interface overview

Current platform-firmware specification for Management Mode initialization, protected MMRAM, entry/save-state concepts and returning to interrupted execution; x86 SMM is one concrete implementation of this generic MM model.

https://uefi.org/specs/PI/1.9/V4_Overview.html

UEFI does not boot “a disk”; GPT names partitions and the EFI System Partition holds firmware-readable boot files

Modern firmware normally sees a block device as an array of logical blocks, then interprets a partition table before it can find a boot filesystem. With GPT, metadata is duplicated at the beginning and end of the disk: a primary header and partition-entry array describe partitions by type GUID, unique GUID and starting/ending LBA, while a backup copy gives recovery tools another authoritative structure if one end is damaged.

UEFI BOOT STORAGE PATH physical SSD / NVMe namespace / SATA disk ↓ logical block addresses (LBAs) LBA 0: protective MBR ↓ LBA 1: primary GPT header ↓ primary partition-entry array ↓ choose entry with EFI System Partition type GUID EFI System Partition (ESP) ↓ firmware FAT filesystem driver \EFI\...\*.efi PE/COFF image ↓ UEFI Boot#### NVRAM option or removable-media fallback path firmware LoadImage()/StartImage() ↓ bootloader OR Linux EFI-stub kernel ↓ ExitBootServices() OS takes control Near end of disk: backup partition-entry array → backup GPT header

Object	What it means	What it does not mean
protective MBR	Legacy-compatible record at LBA 0 advertising a GPT-managed disk so old MBR-only software is less likely to overwrite it.	It is not the authoritative modern partition map.
GPT header	Describes disk GUID, usable LBA range, partition-entry-array location/size and CRCs.	It does not contain a filesystem.
GPT partition entry	Names one partition by type GUID, unique partition GUID and LBA range.	A partition type is not itself proof that the contents are valid.
EFI System Partition	A firmware-readable system partition, conventionally FAT-formatted, containing UEFI applications/loaders and related files.	It is not the same thing as the OS root filesystem.
Boot#### variable	UEFI NVRAM boot option that can identify a device path and executable.	The partition table itself does not define boot priority.
PARTUUID	OS-facing representation of a GPT partition's unique GUID, useful for stable partition identity.	It is distinct from a filesystem UUID stored inside the partition.

Three layers must stay separate: GPT answers “which LBA ranges are partitions?”, a filesystem answers “which named files live inside this partition?”, and UEFI boot policy answers “which executable should firmware start?” A machine can have perfectly valid GPT metadata and still be unbootable because the ESP or boot option is wrong.

CURRENT SPECUEFI Specification 2.11 — current HTML specification

Current UEFI specification. Chapter 5 defines GPT layout, protective MBR, headers and partition entries; the media/file-system chapters define System Partitions and firmware-readable files.

https://uefi.org/specs/UEFI/2.11/

KERNEL DOCSLinux x86-64 UEFI support — GPT/ESP setup

Concrete Linux-side bridge from GPT creation to an EFI System partition, VFAT formatting and placing an EFI executable where firmware can load it.

https://docs.kernel.org/arch/x86/x86_64/uefi.html

UEFI variables are firmware-backed key/value state: Runtime Services → efivarfs → NVRAM

UEFI variables are not files stored on the EFI System Partition. They are named firmware variables, separated into namespaces by vendor GUID, with attributes controlling persistence and when firmware/OS code may access them. Typical examples include boot-order entries, Secure Boot databases and vendor settings. Linux exposes this firmware state through efivarfs, normally mounted at /sys/firmware/efi/efivars.

UEFI FIRMWARE VARIABLE STORE nonvolatile platform storage / firmware implementation ↑ ↓ GetVariable / SetVariable / GetNextVariableName ↑ UEFI Runtime Services ↓ Linux EFI runtime-service layer ↓ efivarfs mounted at /sys/firmware/efi/efivars ↓ Name-GUID file first 4 bytes = UEFI variable attributes remaining bytes = variable payload ↓ read/write/unlink may invoke firmware variable services Boot#### / BootOrder are variables. The .efi executable itself normally lives on the EFI System Partition. Those are different storage objects.

Concept	What it means
Variable name + vendor GUID	Together identify a variable without requiring every vendor to coordinate one global name namespace.
NON_VOLATILE attribute	Requests persistence across reset/power cycles; the UEFI specification notes that nonvolatile variable storage can be limited.
BOOTSERVICE_ACCESS	Variable can be accessed while UEFI Boot Services are active.
RUNTIME_ACCESS	Variable remains visible through Runtime Services after `ExitBootServices()`.
efivarfs	Linux filesystem interface that translates file operations into access to EFI variables; it is not a normal disk-backed filesystem.
immutable safeguard	Linux marks many non-standard efivarfs files immutable by default because buggy firmware has historically failed to boot after some vendor variables were deleted.

Do not confuse the ESP with the variable store. The ESP is an ordinary FAT-formatted partition containing boot files. UEFI variables are firmware-managed persistent state accessed through Runtime Services; their physical backing is platform-specific.

KERNEL DOCSLinux efivarfs — UEFI variable filesystem

Current kernel documentation for mounting efivarfs, the Name-GUID file representation, the four-byte attribute prefix and the immutable-file safety behavior.

https://docs.kernel.org/filesystems/efivarfs.html

OFFICIAL SPECUEFI 2.11 — Runtime Services and Variable Services

Normative definition of GetVariable, SetVariable, variable GUID/name identity, access attributes and persistence expectations.

https://uefi.org/specs/UEFI/2.11/08_Services_Runtime_Services.html

Updating platform firmware is a staged handoff: fwupd → ESRT/FMP metadata → UEFI capsule → reboot → firmware flash

An operating system can help deliver a motherboard or device firmware update without being the code that actually rewrites platform flash. On UEFI systems, update software such as fwupd can identify firmware-updatable components, verify vendor metadata/payload packaging, stage a UEFI capsule, arrange the required boot/reset path, and then hand control to platform firmware. The firmware—not the normal Linux driver—performs the privileged update operation according to the platform's capsule/Firmware Management Protocol implementation.

USER / UPDATE SERVICE fwupdmgr / desktop updater ↓ fwupd daemon identifies updatable component ├─ hardware/device GUIDs ├─ current firmware version ├─ ESRT / Firmware Management Protocol metadata where supported └─ vendor/LVFS metadata and update policy ↓ download + validate packaged update ↓ extract UEFI capsule payload ↓ STAGE FOR FIRMWARE ├─ Capsule-on-Disk: write capsule under EFI System Partition │ (for example EFI/UpdateCapsule when supported) └─ runtime/helper path: arrange capsule handoff / BootNext helper as required ↓ reboot / firmware-controlled update phase ↓ UEFI firmware consumes capsule ↓ platform flash / device firmware is rewritten ↓ reboot into OS ↓ ESRT/update status + new version can be reported back to userspace fwupd stages/orchestrates; firmware owns the final flash operation.

Object	Role
fwupd	Userspace firmware-update daemon that discovers supported devices, consumes update metadata, stages payloads and reports results.
capsule	UEFI-defined container delivered from an OS-present environment to firmware for deferred or immediate processing.
ESRT	EFI System Resource Table describing firmware resources that can be updated, including resource GUID, version and prior update status.
Firmware Management Protocol	UEFI interface used by firmware implementations to expose firmware-image descriptors and update behavior for components.
EFI System Partition	Firmware-readable FAT partition that can also carry Capsule-on-Disk files when the platform supports that delivery method.
BootNext/helper path	Some update paths arrange a one-shot boot into a signed EFI helper which passes/stages the update before the normal OS boots.
Secure Boot	Authenticates EFI executables in the boot chain; it is related to—but not itself—the mechanism that verifies every vendor firmware payload.

A firmware updater is not a universal flash writer. The exact authorization, anti-rollback rules, power/reset requirements and flash-write mechanism are platform/vendor specific. UEFI defines capsule transport and update interfaces, while the platform firmware decides what payloads it accepts and how an update is committed.

CURRENT FWUPD DOCSfwupd UEFI Capsule plugin

Current fwupd documentation for capsule staging, ESRT discovery, Capsule-on-Disk, EFI helper/BootNext behavior and Secure Boot considerations.

https://fwupd.github.io/libfwupdplugin/uefi-capsule-README.html

CURRENT FWUPD DOCSfwupd UEFI ESRT plugin

Documents fwupd's handling of UEFI firmware-update capability surfaced through firmware attributes and the ESRT-related update path.

https://fwupd.github.io/libfwupdplugin/uefi-esrt-README.html

Secure Boot and Measured Boot solve different problems

Verified/Secure Boot decides whether a stage is authorized to execute, usually by checking a cryptographic signature against a trusted key/policy. Measured Boot records cryptographic measurements of what was loaded/executed into TPM Platform Configuration Registers (PCRs). Measurement by itself does not block execution; it creates evidence that later software or a remote verifier can inspect.

CHAIN OF TRUST FOR EXECUTION (verified boot concept) immutable hardware/ROM trust anchor ↓ verifies signature/hash policy mutable firmware stage A ↓ verifies next stage firmware stage B / boot manager ↓ verifies allowed OS loader/kernel image operating system If verification fails → stop, recovery path, alternate slot, policy failure, etc. MEASURED BOOT / TPM PCR component bytes → HASH(component) = h ↓ PCR_new = HASH(PCR_old || h) ↓ event log stores 'what was measured' + digest metadata Next component extends same PCR: PCR2 = HASH(PCR1 || h2) Because extend is cumulative/order-dependent, the final PCR represents boot history, not simply 'the hash of the last file'. ATTESTATION / QUOTE verifier sends nonce/challenge ↓ TPM signs selected PCR values + nonce with attestation key ↓ verifier checks signature + event log / expected policy Secure Boot asks: 'may this image run?' Measured Boot asks: 'what actually ran / was measured?' TPM quote asks: 'can this TPM cryptographically attest those PCR values now?'

Mechanism	Primary purpose	Typical failure result
immutable ROM key/root	Bootstrap trust from code/data attacker cannot ordinarily update.	Device may refuse mutable stage or enter recovery.
signature verification	Authenticate image origin/integrity before execution.	Stage rejected if signature/policy invalid.
rollback protection	Reject cryptographically valid but too-old/vulnerable firmware versions.	Older image not accepted despite valid signer.
UEFI Secure Boot db	Allow images signed by trusted cert/hash entries.	Unknown/untrusted image can be rejected.
UEFI dbx	Explicitly revoke forbidden image hashes/certificates/signers.	Image rejected even if otherwise chains to an allowed signer.
TPM PCR extend	Accumulate ordered measurements into tamper-resistant state.	PCR value differs from expected boot history.
TPM event log	Records which measured components/events explain PCR evolution.	Verifier can detect mismatch between replayed log and quoted PCR.
TPM quote	Sign current selected PCR values plus anti-replay nonce.	Remote/local verifier rejects unexpected PCR state or invalid signature.

A signature is not enough to prevent rollback. If an old vulnerable image is still correctly signed, a verified-boot system needs a monotonic version/security policy to reject it. OpenTitan is useful here because its public secure-boot design explicitly separates immutable ROM, mutable signed ROM_EXT/owner stages and policy/usage constraints.

LINUX / TPM OBSERVATION LAB (where supported) ls /sys/class/tpm ls -l /dev/tpm* /dev/tpmrm* 2>/dev/null # If tpm2-tools is installed: tpm2_pcrread # EFI Secure Boot variables on many UEFI Linux systems: ls /sys/firmware/efi/efivars | grep -E 'SecureBoot|PK-|KEK-|db-|dbx-' # systemd-based systems may expose measurement/event-log tooling separately. # Do not alter Secure Boot key databases or TPM ownership/state merely as an experiment.

OFFICIAL SPECUEFI 2.11 — Secure Boot and Driver Signing

Official no-login current Secure Boot chapter. Defines PK/KEK, allowed db, forbidden dbx and UEFI image authorization/validation.

https://uefi.org/specs/UEFI/2.11/32_Secure_Boot_and_Driver_Signing.html

DIRECT SPEC PDFUEFI 2.11 — full specification (direct PDF)

Direct official specification PDF; Chapter 32 covers Secure Boot and image validation.

https://uefi.org/sites/default/files/resources/UEFI_Spec_Final_2.11.pdf

PUBLIC DOCSOpenTitan — Secure Boot

Excellent open hardware/software chain-of-trust example: immutable ROM contains trust keys, hashes/authenticates ROM_EXT and only then unlocks execution/transfers control.

https://opentitan.org/book/doc/security/specs/secure_boot/

PUBLIC DOCSOpenTitan — ROM_EXT boot stage

Shows a mutable but signed intermediate boot stage that executes only after ROM signature verification, then authenticates the first owner stage.

https://opentitan.org/book/sw/device/silicon_creator/rom_ext/index.html

PUBLIC DOCSOpenTitan — boot image manifest

Concrete fields used by signed flash-resident boot stages, including code boundaries, entry point and image metadata.

https://opentitan.org/book/sw/device/silicon_creator/rom_ext/doc/manifest.html

PUBLIC TOOL DOCStpm2_pcrextend — PCR extend tool

No-login manual showing direct PCR extension with selected hash banks. Useful for learning the extend operation without a platform course.

https://tpm2-tools.readthedocs.io/en/latest/man/tpm2_pcrextend.1/

PUBLIC TOOL DOCStpm2_checkquote — verify TPM quote

Companion utility for checking quote signature, PCR values and qualifying data.

https://tpm2-tools.readthedocs.io/en/latest/man/tpm2_checkquote.1/

KERNEL DOCSLinux EFI boot stub

Connects UEFI image loading to Linux: the kernel can itself present as a PE/COFF EFI executable and be entered directly by EFI firmware.

https://docs.kernel.org/admin-guide/efi-stub.html

ARM TrustZone can run a second trusted execution environment beside Linux: client → /dev/tee → SMC → OP-TEE trusted application

ARM TrustZone is a hardware security architecture that distinguishes security states rather than merely adding another Unix privilege level. A TrustZone-based Trusted Execution Environment (TEE) can run trusted software in the secure world while Linux runs in the normal world. OP-TEE is a widely used open-source trusted OS for that secure environment. Linux communicates with it through the generic TEE subsystem, an OP-TEE driver, shared memory, and Secure Monitor Calls (SMCs).

NORMAL WORLD (Linux) SECURE WORLD application / library ↓ open /dev/tee0 TEE_IOC_OPEN_SESSION / TEE_IOC_INVOKE ↓ Linux generic TEE subsystem ↓ OP-TEE driver ↓ prepare message + shared-memory buffers ↓ SMCCC / SMC instruction ───────────────→ secure monitor / firmware transition ↓ OP-TEE trusted OS ↓ Trusted Application (TA) ↓ cryptographic key / secure service / protected peripheral where platform policy allows ↓ return value + shared-memory output ←──── SMC return ↓ client continues in Linux SECURE-WORLD RPC OP-TEE may request selected normal-world services ↓ kernel OP-TEE driver or tee-supplicant userspace helper ↓ result returned to secure world.

Piece	Role
normal world	Non-secure TrustZone state where the ordinary OS such as Linux commonly runs.
secure world	Secure TrustZone state used by trusted firmware/TEE software and secure-only resources according to platform design.
TEE	Trusted Execution Environment: isolated trusted software environment with a defined client interface.
OP-TEE	Open-source trusted OS implementing a TrustZone-based TEE on supported ARM systems.
Trusted Application (TA)	Service/application executing under the TEE rather than as a normal Linux process.
SMC / SMCCC	Architecture calling mechanism/convention used to request secure monitor or trusted-firmware services across security states.
shared memory	Explicit buffer region accessible to both sides for messages/data; secure-only memory is not simply mapped into Linux.
`tee-supplicant`	Normal-world userspace helper that services selected OP-TEE RPC requests requiring Linux-side resources.

TrustZone is orthogonal to ordinary Linux privilege. Linux root is still normal-world software and does not automatically gain access to secure-world memory, keys or devices. Conversely, putting code in a TEE does not magically make its entire design trustworthy: boot-chain integrity, secure-monitor code, TA validation, shared-memory validation, side channels and platform-specific resource partitioning still matter.

NO-ACCOUNT OBSERVATION (on systems with a Linux TEE driver) ls -l /dev/tee* 2>/dev/null ls /sys/class/tee 2>/dev/null dmesg | grep -Ei 'op-tee|optee|trusted execution|tee' # Presence and naming are platform/kernel-configuration dependent. # Do not infer that a machine has TrustZone/OP-TEE merely because the CPU family can support it.

KERNEL DOCSLinux OP-TEE driver documentation

Current kernel documentation for the TrustZone-based OP-TEE path, including SMCCC/SMC calls, OP-TEE message protocol, shared memory and RPC handling.

https://docs.kernel.org/tee/op-tee.html

KERNEL DOCSLinux TEE userspace API

Defines /dev/tee*, shared-memory allocation, session open/invoke/close ioctls and the distinction between normal clients and supplicants.

https://docs.kernel.org/userspace-api/tee.html

TPM 2.0 can gate secret release on platform state and can sign evidence about that state: sealing and quoting are different operations

Measured boot fills Platform Configuration Registers (PCRs) with cumulative measurements, but PCR values become much more useful when they feed a policy. A sealed object can hold or protect secret material such that the TPM releases it only after an authorization policy is satisfied—for example, when selected PCRs match an expected state. A quote does not release a secret: an attestation key signs selected PCR state plus caller-supplied freshness data so another party can verify evidence about the platform.

SEALING / LOCAL POLICY-GATED SECRET RELEASE boot components extend PCRs ↓ PCR state represents measured platform history ↓ create policy digest, e.g. PolicyPCR(selected PCRs / expected values) ↓ create sealed TPM object bound to that authorization policy ↓ reboot / later use application starts TPM policy session ↓ TPM evaluates current PCRs + other policy clauses ├── policy satisfied → TPM unseals / releases protected data └── policy fails → secret remains unavailable QUOTING / ATTESTATION EVIDENCE remote verifier generates fresh nonce ↓ platform asks TPM attestation key (AK) to quote selected PCRs + nonce ↓ TPM returns signed attestation structure ↓ verifier checks: AK/public-key trust signature nonce freshness quoted PCR digest event log / expected measurements / policy ↓ verifier decides whether platform state is acceptable PCRs themselves are not a password vault. They are measured state that policies and attestation statements can reference.

TPM concept	What it does
PCR	Small TPM-maintained register whose value is normally extended with measurements; multiple hash-algorithm banks may exist.
event log	External log describing measured events so software can replay/explain how quoted PCR values were reached.
policy session	TPM authorization mechanism in which clauses such as PCR state are evaluated before an object/operation may be used.
sealed object	TPM object containing/protecting data whose release is gated by the object’s authorization policy.
Attestation Key (AK)	Signing key used for TPM attestation statements such as quotes; verifiers still need a reason to trust the key’s provenance.
quote	Signed attestation over selected PCR state plus qualification data, commonly a verifier nonce for freshness.
nonce	Fresh challenge included in the quote to prevent replay of an old, otherwise valid attestation.

A good quote is evidence, not a verdict. Signature verification only proves that the quoted data was signed by the corresponding attestation key. A verifier must also decide whether that key is trusted, whether the nonce is fresh, whether the PCR/event-log state matches policy, and whether the measurement chain actually covers the components it cares about.

KERNEL DOCSLinux TPM documentation

Current kernel documentation index covering TPM event logs, PCR integrity, secrets guarding, device interfaces and virtual/firmware TPM implementations.

https://docs.kernel.org/next/security/tpm/index.html

KERNEL DOCSLinux trusted/encrypted keys — TPM PCR sealing

Explains kernel Trusted Keys and optional binding of key unsealing to TPM PCR state, including TPM 2.0 usage.

https://docs.kernel.org/security/keys/trusted-encrypted.html

PUBLIC TOOL DOCStpm2_quote — signed PCR attestation

Shows quote creation over selected PCRs, including qualification data/nonce and signature output.

https://tpm2-tools.readthedocs.io/en/latest/man/tpm2_quote.1/

PUBLIC TOOL DOCStpm2_unseal — release data from a TPM sealing object

Concrete TPM 2.0 workflow for creating PCR policy state and unsealing protected data only when the policy session can be satisfied.

https://tpm2-tools.readthedocs.io/en/latest/man/tpm2_unseal.1/

Runtime integrity is a separate layer from boot trust: IMA measures/appraises objects while EVM protects security metadata

Measured Boot tells you what was extended into TPM PCRs during the boot chain, but a long-running system keeps opening, executing and mapping files after boot. Linux's Integrity Measurement Architecture (IMA) can apply policy at those runtime access points: hash selected objects, append structured records to an in-kernel measurement log, optionally extend a TPM PCR, and—under appraisal policy—require an expected hash or signature before access succeeds. The Extended Verification Module (EVM) protects integrity-sensitive metadata such as security extended attributes so that an attacker cannot simply replace the label or IMA metadata independently of the file.

FILE / KERNEL OBJECT ACCESS exec / mmap-for-exec / open / critical-data event selected by policy ↓ IMA policy match? ├─ no → normal path continues └─ yes ↓ compute / obtain object digest ↓ MEASUREMENT path create template record (digest + name + optional metadata/signature) ↓ append runtime measurement list ↓ optionally extend configured TPM PCR APPRAISAL path (when policy requires) read security.ima hash/signature or supported fs-verity digest/signature ↓ verify against current content + trusted key/policy ├─ valid → allow access └─ invalid/missing under enforce policy → deny EVM complements this by protecting security metadata/xattrs (e.g. security.ima, security.selinux, security.capability) against offline tampering.

Mechanism	What it answers
Secure Boot	Should this EFI/boot-chain executable be allowed to run according to platform trust policy?
Measured Boot	What boot components/configuration were measured into PCR history?
IMA measurement	Which runtime-selected files/data were observed, and what digests were recorded/extended?
IMA appraisal	Does this object have acceptable integrity metadata/signature for the current policy before use?
EVM	Has integrity/security metadata itself been altered independently of the protected object?
fs-verity	Does each read from an immutable file match its Merkle-tree-authenticated contents? IMA can consume fs-verity digests, but the mechanisms are distinct.

A PCR is not a file whitelist. Extending a digest into a TPM PCR records state in an append-only hash chain; enforcement requires a policy/appraisal decision somewhere else. Likewise, IMA measurement can be enabled without IMA appraisal, so “measured” does not automatically mean “blocked if modified.”

KERNEL DOCSLinux IMA template management

Current kernel documentation for IMA measurement-record templates, including digest, filename, signature, xattr and metadata fields used in runtime measurement logs.

https://docs.kernel.org/security/IMA-templates.html

KERNEL DOCSLinux IMA measurement-list export/staging

Explains the in-kernel runtime measurement list, securityfs interfaces, TPM-protected integrity assumptions and exporting measurements for attestation workflows.

https://docs.kernel.org/security/IMA-export-delete.html

PUBLIC MANUALevmctl — IMA/EVM signing and verification utility

Documents IMA signatures/hashes and EVM signatures/HMACs, including the distinction between protecting file content and protecting security metadata/xattrs.

https://linux-ima.sourceforge.net/evmctl.1.html

Secure random bytes come from a seeded kernel generator, not from “random-looking” timing alone

Cryptographic software needs bytes that an attacker cannot predict. Linux therefore collects unpredictable input from device drivers and other environmental sources, mixes that material into its random subsystem, and uses it to initialize a cryptographically secure pseudorandom number generator (CSPRNG). After initialization, applications normally ask the kernel generator for output rather than trying to harvest hardware timing noise themselves.

BOOT kernel random subsystem starts ↓ collect/mix environmental and platform inputs examples can include interrupt/device timing and trusted hardware RNG inputs ↓ track whether enough seed material has arrived for secure initialization ↓ CSPRNG READY ↓ getrandom(buf, n, 0) ├── before initialization → blocks (unless GRND_NONBLOCK requested) └── after initialization → returns CSPRNG output /dev/urandom → another interface to the kernel generator /dev/hwrng → raw output from a hardware RNG driver when available → not the same thing as the kernel CSPRNG and not automatically a trust guarantee APPLICATION random bytes → nonces / salts / session keys / ASLR-related consumers / etc.

Term/interface	What it means
entropy input	Unpredictable seed material mixed into the kernel random subsystem. The exact sources are platform- and driver-dependent.
entropy pool / random state	Kernel-maintained state that accumulates/mixes inputs and supports initialization/reseeding of the generator.
CSPRNG	Deterministic generator whose internal seeded state expands a comparatively small amount of unpredictable seed material into many cryptographically suitable output bytes.
`getrandom(..., 0)`	Preferred Linux system-call interface for ordinary random-byte requests; waits for secure initialization during early boot.
`GRND_NONBLOCK`	Requests an immediate `EAGAIN` instead of waiting when the generator is not initialized.
`/dev/hwrng`	Character device exposing a hardware RNG driver. Kernel documentation explicitly warns that its raw data is not automatically fitness-tested merely because it came from hardware.

Entropy and generated output are not the same quantity. Once a secure generator is properly seeded, applications do not need one fresh physical-noise bit for every random output bit. The CSPRNG expands secret internal state; new environmental inputs are mixed in to initialize and reseed that state.

CURRENT PUBLIC MANUALrandom(7) — Linux random-number interfaces

Current overview of the kernel CSPRNG, entropy-pool initialization and the intended relationship among getrandom(), /dev/urandom and /dev/random.

https://man7.org/linux/man-pages/man7/random.7.html

CURRENT PUBLIC MANUALgetrandom(2)

Precise blocking, nonblocking and return-value behavior for obtaining random bytes without opening a special file.

https://man7.org/linux/man-pages/man2/getrandom.2.html

KERNEL DOCSLinux hardware RNG framework

Explains /dev/hwrng, hardware-specific RNG drivers and why raw hardware RNG output should not be confused with the kernel's complete random subsystem.

https://docs.kernel.org/admin-guide/hw_random.html

# See whether the kernel currently exposes entropy-accounting information: cat /proc/sys/kernel/random/entropy_avail cat /proc/sys/kernel/random/poolsize # Ask the kernel for bytes using a normal userspace tool backed by its RNG: python3 -c 'import os; print(os.getrandom(16).hex())' # Hardware RNG device exists only when a suitable driver/device is present: ls -l /dev/hwrng 2>/dev/null

From bootloader handoff to PID 1: decompression, start_kernel(), initcalls, rootfs and exec

Firmware and a bootloader do not initialize Linux device drivers one by one. Their job is to place the kernel image, command line and optional initrd/initramfs where the Linux boot protocol expects them and transfer control. Linux then establishes architecture/MMU state, initializes core subsystems, runs built-in driver initcalls, starts kernel threads, mounts or constructs a root filesystem and finally executes the first userspace program as PID 1.

UEFI/BOOTLOADER SIDE (one common modern x86 path) firmware loads EFI boot manager / Linux EFI-stub image ↓ bootloader/stub obtains memory map, command line, initrd ↓ Linux kernel image + boot parameters are placed in RAM ↓ jump to architecture-defined kernel entry point EARLY x86 KERNEL compressed image / decompressor path as applicable ↓ startup_64 / architecture entry ↓ page tables / CPU mode / relocations / early console / architecture setup ↓ start_kernel() start_kernel() BROADLY INITIALIZES architecture/platform discovery memory management scheduler/timekeeping interrupts/IRQs RCU/workqueues VFS and other core subsystems security/framework infrastructure ↓ rest_init() ↓ create PID 1 kernel-init thread FIRST create kthreadd (PID 2 in the usual boot) boot CPU enters idle/scheduling world kernel_init() ↓ kernel_init_freeable() ↓ do_basic_setup() ↓ do_initcalls() early/core/postcore/arch/subsys/fs/device/late initcall levels built-in drivers/subsystems register/probe ↓ prepare_namespace() / rootfs/initramfs path as configured INITRAMFS CASE built-in/external cpio archive unpacked into rootfs ↓ if /init exists → kernel executes /init as PID 1 ↓ /init discovers/assembles real root (LVM, RAID, encryption, network, etc.) ↓ switch_root/pivot-like transition ↓ exec real init NO /init IN INITRAMFS / LEGACY ROOT PATH kernel mounts configured root filesystem ↓ tries configured init= or standard init paths ↓ exec userspace init as PID 1 PID 1 (systemd on many distributions) ↓ mounts/services/devices/login/network/session startup At this point the system is not 'finished booting'; userspace has merely taken over orchestration.

Boot object/stage	What it is for
bzImage	Common compressed x86 Linux boot image containing setup/entry/decompressor and kernel payload.
boot_params / zero page	x86 bootloader→kernel structure carrying command line, memory/loader information and boot-protocol data.
initrd	Initial RAM-disk image supplied separately by bootloader; modern setups usually contain an initramfs cpio archive.
rootfs	Kernel's initial root filesystem, backed by ramfs/tmpfs semantics; initramfs is unpacked into it.
initramfs	cpio archive containing early userspace such as /init plus tools/modules needed before the real root filesystem is available.
start_kernel()	Architecture-independent central kernel initialization function after early architecture entry/setup.
initcall	Function registered into an ordered kernel initialization level, commonly used by built-in subsystems/drivers.
rest_init()	Creates the init thread first so it gets PID 1, then creates kthreadd and transitions boot CPU toward idle/scheduling.
kthreadd	Kernel thread that becomes the ancestor/manager used for creation of many later kernel threads.
PID 1	First userspace process; special init responsibilities include bringing up userspace and reaping orphaned child processes.
init=	Kernel command-line override choosing an alternative userspace init program.
rdinit=	Kernel command-line override for the initial ramdisk/initramfs init program.

Kernel boot contains two very different worlds called 'init'. Kernel __init/initcalls are one-time kernel initialization code that can often be freed after boot. Userspace /init or /sbin/init is an executable process—PID 1—not a kernel initcall.

LINUX BOOT OBSERVATION LAB # kernel command line handed over at boot cat /proc/cmdline # current PID 1 ps -p 1 -o pid,ppid,comm,args # kernel boot log with timestamps dmesg -T | less # monotonic kernel timestamps are better for boot timing than human wall-clock formatting dmesg | less # systemd systems: userspace boot timing/dependency chain systemd-analyze 2>/dev/null systemd-analyze critical-chain 2>/dev/null | less # inspect initramfs when distro tool exists; examples: lsinitramfs /boot/initrd.img-$(uname -r) 2>/dev/null | less lsinitrd 2>/dev/null | less # kernel source trail: # arch/x86/boot/ → arch/x86/boot/compressed/ → arch/x86/kernel/head_64.S # → init/main.c:start_kernel() → rest_init() → kernel_init() # Add initcall_debug to a disposable/test boot configuration if you want # per-initcall timing; don't alter a critical boot setup without a recovery route.

SOURCE / FILELinux init/main.c — start_kernel, rest_init and kernel_init

The real current source. rest_init() explicitly creates the init thread first so it obtains PID 1, then creates kthreadd; start_kernel() and kernel_init() show the initialization progression.

https://github.com/torvalds/linux/blob/master/init/main.c

SOURCE / FILELinux init.h — __init and initcall infrastructure

Real definitions/comments for one-time __init code and initcall registration levels used by built-in kernel initialization.

https://github.com/torvalds/linux/blob/master/include/linux/init.h

KERNEL DOCSLinux — ramfs, rootfs and initramfs

Explains rootfs/initramfs cpio unpacking and the crucial transition: if rootfs contains /init, the kernel executes it as PID 1; otherwise it falls back to mounting a root partition and executing a standard init path.

https://docs.kernel.org/filesystems/ramfs-rootfs-initramfs.html

KERNEL DOCSLinux — Explaining 'No working init found'

Useful source-code-adjacent troubleshooting document for the kernel's final kernel_execve() transition to userspace init, including ELF interpreter/library failures.

https://github.com/torvalds/linux/blob/master/Documentation/admin-guide/init.rst

PUBLIC MANUALsystemd(1) — current PID 1 userspace manager

Current manual: when systemd runs as the first process on boot (PID 1), it acts as the init system that brings up and maintains userspace services.

https://man7.org/linux/man-pages/man1/systemd.1.html

Early userspace is temporary: initramfs finds the real root, then hands PID 1 to it

The kernel can unpack an initramfs cpio archive into its initial rootfs and execute /init before the final root filesystem is available. This early userspace exists because finding the real root may itself require userspace policy and tools: load storage firmware/modules, wait for devices, assemble RAID/LVM, unlock encryption, configure networking, or discover a network root. Once the final root is mounted, early userspace must change the mount tree so that the real filesystem becomes /, preserve or move API filesystems such as /proc, /sys, /dev and /run, and finally exec the real init/service manager.

FIRMWARE / BOOTLOADER ↓ kernel + optional external initramfs kernel decompresses and initializes enough hardware ↓ unpack cpio archive into initial rootfs ↓ exec /init as early userspace ↓ load modules / firmware; discover storage assemble RAID / LVM / dm-crypt or configure network root ↓ mount final root filesystem somewhere (for example /sysroot) ↓ move/preserve /proc /sys /dev /run as needed ↓ switch_root helper OR a suitable pivot_root/mount sequence ↓ old initramfs contents no longer define / ↓ exec real /sbin/init or service manager as PID 1 ↓ normal userspace boot continues

Mechanism	What it actually means
rootfs	The kernel's initial root mount, backed by ramfs/tmpfs-style infrastructure; it exists before a disk root is mounted.
initramfs	A cpio archive unpacked into the initial rootfs. Its `/init` is ordinary userspace running very early.
initrd	The older initial-RAM-disk model: a filesystem image presented as a RAM block device. Modern distributions normally use initramfs semantics instead.
`switch_root`	A userspace helper intended for the common initramfs handoff. It makes an already-mounted new root become `/`, moves key virtual filesystems and executes the new init.
`pivot_root()`	A Linux system call that swaps the root mount with another mount while placing the old root underneath the new one. Its mount-namespace restrictions matter.
`chroot()`	Changes pathname resolution for a process; it is not by itself a mount-tree root handoff and is not a security boundary.

Do not confuse the initramfs with the final root filesystem. The initramfs is an early-userspace staging environment. Its purpose may be only a few hundred milliseconds long, but it can contain sophisticated policy because the kernel deliberately leaves root-device discovery, decryption and assembly to userspace.

SEE EARLIERKernel boot → rootfs → PID 1

The earlier boot section already links the kernel rootfs/initramfs documentation and shows where early userspace fits after start_kernel() and before the final PID 1 environment.

PUBLIC MANUALswitch_root(8) — util-linux

Current util-linux manual for the normal initramfs handoff helper: it moves selected API mounts to the new root, makes that mount the root filesystem and executes the requested init program.

https://man7.org/linux/man-pages/man8/switch_root.8.html

PUBLIC MANUALpivot_root(2) — change the root mount

Current Linux manual for the underlying root-mount operation, including mount-point, propagation and capability constraints and the special caveat around the initial rootfs.

https://man7.org/linux/man-pages/man2/pivot_root.2.html

After PID 1 starts, boot becomes dependency and lifetime management: systemd units, cgroups, restart policy and socket activation

Reaching PID 1 is only the kernel-to-userspace handoff. On a systemd-based Linux system, the system manager then builds a dependency/ordering graph of units, starts services and mounts/devices/targets as dependencies become ready, tracks service processes using Linux cgroups, records lifecycle state, and applies configured restart/stop/resource policy. Socket activation separates “own the listening endpoint” from “the daemon process is already running”: PID 1 can create/listen on the socket first and pass the open file descriptor to the service when activation occurs.

KERNEL execs PID 1 ↓ systemd system manager ↓ load unit configuration + generator output ↓ construct requirement / ordering graph Requires=/Wants= describe pull-in relationships Before=/After= describe ordering constraints ↓ activate target units and dependencies ↓ .service unit starts process(es) ↓ processes tracked in unit cgroup ↓ READY/RUNNING/FAILED/EXITED state + restart/stop policy SOCKET ACTIVATION .socket unit asks PID 1 to create/bind/listen first ↓ client arrives / socket becomes active ↓ associated .service is started if needed ↓ open listening/connection fd is inherited/passed to daemon ↓ daemon serves work without having raced another process for bind() A service's kernel scheduler still schedules threads normally; systemd is supervising lifetime/dependencies and configuring cgroup policy.

systemd concept	Role
unit	Named object with state/dependencies: service, socket, target, mount, device, timer, path and other unit types.
.service	Defines how a daemon/process is started, considered ready, stopped/reloaded and optionally restarted.
.socket	Lets the service manager own an IPC/network listening endpoint and activate the matching service on demand.
target	Synchronization/grouping unit used to pull together boot or operational milestones rather than a process itself.
cgroup	Kernel process hierarchy systemd uses to track a unit's process tree and apply CPU/memory/I/O/PID resource controls.
ordering dependency	`Before=`/`After=` controls activation order; it does not by itself pull another unit into the transaction.
requirement dependency	`Requires=`/`Wants=` changes what else is pulled in; ordering is a separate relationship.
restart policy	Configured reaction to exits/failures, such as restarting a crashed service under specified conditions and rate limits.

“Started after” and “depends on” are not synonyms. systemd deliberately separates ordering edges from requirement edges. This is why a large boot can run many independent services in parallel while still enforcing the few ordering constraints that matter.

CURRENT MANUALsystemd.service(5)

Current upstream manual for supervised service units, service types, dependencies, restart behavior, execution and resource-control integration.

https://www.man7.org/linux/man-pages/man5/systemd.service.5.html

CURRENT MANUALsystemd.socket(5)

Defines socket units used for socket-based activation of IPC/network services and their relationship to matching service units.

https://man7.org/linux/man-pages/man5/systemd.socket.5.html

CURRENT MANUALsystemd.resource-control(5)

Current documentation for systemd's use of Linux cgroups and unit-level CPU, memory, I/O, PID and delegation controls.

https://man7.org/linux/man-pages/man5/systemd.resource-control.5.html

Powering off is a coordinated teardown, not merely cutting power: stop services → flush/unmount storage → kernel shutdown → firmware/platform reset or power-off

An orderly Linux shutdown is the reverse of boot across several layers. A service manager first stops userspace work and tears down dependencies; remaining filesystems are unmounted or remounted read-only, swap/storage are detached, and processes are terminated. Only after userspace cleanup does the kernel receive a final reboot/power-off request. The kernel then runs device shutdown callbacks and architecture/platform code that ultimately asks firmware or hardware to reset, halt or remove power.

USER REQUEST systemctl poweroff / reboot / shutdown ↓ PID 1 queues poweroff.target / reboot.target ↓ stop services in dependency order close sockets, terminate daemons, stop mounts where possible ↓ systemd-shutdown takes over final userspace teardown kill remaining processes unmount filesystems or remount read-only disable swap detach remaining storage where possible ↓ final reboot()/power-off operation ↓ KERNEL sync/shutdown paths already expected to be complete run device/system shutdown callbacks stop secondary CPUs / interrupts as architecture requires ↓ platform-specific final action ACPI/firmware power-off reset controller / firmware reset halt loop or kexec into another kernel ↓ hardware powers off, resets, halts or starts replacement kernel FORCED PATHS skip parts of this sequence; that is why "reboot -f" or an abrupt power cut can lose data.

Stage	What it accomplishes
service stop	Lets applications finish transactions, close files/sockets and save state while their dependencies still exist.
filesystem teardown	Flushes pending state and removes writable mounts so on-disk metadata is left in a recoverable state.
swap/storage detach	Stops new I/O against devices that are about to disappear or lose power.
`reboot(2)`	Privileged Linux system call selecting restart, halt, power-off, kexec or related final operations.
device shutdown callbacks	Give drivers a last chance to quiesce hardware before reset/power loss.
platform power/reset mechanism	Architecture/firmware-specific mechanism that actually changes machine power/reset state.

sync() and “power off” are not synonyms. Flushing dirty filesystem data is only one part of a clean shutdown. The machine also needs coordinated process termination, mount/storage teardown and a final kernel/platform transition. Conversely, a forced reboot can intentionally skip much of that coordination.

CURRENT MANUALsystemd-shutdown — final userspace shutdown logic

Current upstream manual for the final systemd shutdown binary: unmount/remount, swap disable, storage detach, remaining-process termination and the handoff to poweroff/reboot/kexec.

https://www.man7.org/linux/man-pages/man8/systemd-shutdown.8.html

CURRENT MANUALreboot(2) — Linux final restart/power-off system call

Defines Linux reboot operations including restart, power-off and kexec, plus the warning that invoking final reboot operations without prior synchronization can lose data.

https://www.man7.org/linux/man-pages/man2/reboot.2.html

ACPI: how firmware describes platform topology, power and devices to the operating system

UEFI gets much of the attention during boot, but once the OS is running it also needs structured information about CPUs, interrupt controllers, NUMA domains, power states, batteries, thermal zones and platform devices. ACPI provides tables plus an interpreted namespace/method language so firmware can describe that hardware and expose platform-control methods to the operating system.

firmware builds ACPI tables + namespace ↓ OS discovers RSDP → XSDT/RSDT ↓ fixed tables + AML namespace ├── MADT → CPUs + Local APICs / I/O APICs / interrupt topology ├── SRAT → CPU/memory proximity domains (NUMA affinity) ├── SLIT → relative NUMA distance matrix ├── FADT → fixed ACPI/platform controls ├── MCFG → PCIe ECAM configuration-space windows ├── HPET / timer-related tables where applicable └── DSDT/SSDT AML → devices, methods, power/thermal namespace OS power-management policy ↓ C-states / low-power idle: stop doing work, trade wake latency for power P/performance controls / CPPC: choose requested performance level device D-states: device power states system sleep/low-power states: whole-platform transitions

ACPI object/table	Why a computer learner should care
MADT	Explains which processors/interrupt controllers exist and how the OS learns APIC topology.
SRAT	Associates processors, memory and initiators with proximity domains/NUMA nodes.
SLIT	Supplies a relative-distance matrix between NUMA localities.
FADT	Contains fixed platform information/control including power-management-related fields.
DSDT/SSDT	Contain AML bytecode describing devices, configuration and control methods in the ACPI namespace.
_CST / _LPI	Describe processor idle/low-power states and their latency/power characteristics.
_PSS / CPPC	Expose processor performance capabilities/controls; modern platforms increasingly use CPPC-style abstract performance levels.
thermal zones	Describe temperatures, trip points, cooling relationships and thermal-policy interfaces.
GPE	General Purpose Event mechanism for platform events not represented by ordinary device interrupts.

ACPI is not simply 'the BIOS controlling power.' On an ACPI system the OS is normally the policy manager (OSPM). Firmware supplies standardized descriptions/methods; the OS decides when to idle CPUs, change performance, suspend devices or enter system sleep states within the available hardware/firmware contract.

OFFICIAL SPECACPI Specification 6.6 — current official HTML

Current ACPI specification (version 6.6, released May 2025). Public HTML with searchable chapters on tables, processor power, devices, thermal management, NUMA and platform errors.

https://uefi.org/specs/ACPI/6.6/

DIRECT SPEC PDFACPI 6.6 — direct PDF

Direct official PDF of the complete ACPI 6.6 specification. No account required.

https://uefi.org/sites/default/files/resources/ACPI_Spec_6.6.pdf

OFFICIAL SPECACPI 6.6 — NUMA SRAT/SLIT

Defines SRAT affinity structures and SLIT's relative-distance matrix between system localities/NUMA nodes.

https://uefi.org/specs/ACPI/6.6/05_ACPI_Software_Programming_Model.html

OFFICIAL SPECACPI 6.6 — processor C-states and performance-state concepts

Readable conceptual section explaining processor C-states as idle states with different power/latency tradeoffs and performance states within the active C0 state.

https://uefi.org/specs/ACPI/6.6/03_ACPI_Concepts.html

KERNEL DOCSLinux intel_idle

Concrete OS implementation showing how Linux exposes processor idle states and may obtain platform/state information from ACPI.

https://docs.kernel.org/admin-guide/pm/intel_idle.html

ACPI is partly executable platform description: ASL → AML → namespace → GPE/EC event handling

ACPI tables are not only static lists. The DSDT and SSDTs contain AML bytecode compiled from ACPI Source Language (ASL). The operating system loads those definition blocks into one hierarchical ACPI namespace and evaluates control methods through an AML interpreter. Those methods can return configuration data, select power states, expose resources and—in tightly defined operation regions—cause I/O or memory accesses needed to control platform hardware.

FIRMWARE / OEM BUILD TIME ASL source Device(...) / Name(...) / Method(...) / OperationRegion(...) ↓ iASL compiler AML bytecode inside DSDT / SSDT definition blocks OS BOOT RSDP → XSDT/RSDT → DSDT + SSDTs ↓ ACPI subsystem / ACPICA AML interpreter ↓ loads objects into one ACPI namespace \_SB.PCI0... \_TZ... \_GPE... ↓ OS/driver evaluates methods such as _STA, _CRS, _PS0, _PS3, _DSM ... ↓ method may read/write declared operation regions (memory / I/O / PCI config / embedded-controller space, subject to ACPI rules) PLATFORM EVENT PATH hardware condition (lid, thermal event, EC request, device event) ↓ SCI / GPE status bit ↓ OS ACPI event code identifies GPE ├── native ACPI-aware driver handles it └── AML _Lxx / _Exx method is queued/evaluated EMBEDDED CONTROLLER EXAMPLE EC raises its ACPI event GPE ↓ Linux/OS EC driver services command/status interface ↓ query command EC returns query value NN ↓ optional AML _QNN control method / registered native handler ↓ platform-specific battery, thermal, hotkey or other action

Piece	Role
ASL	Human-readable source language normally written by firmware/platform authors.
AML	Compact bytecode stored in ACPI definition blocks and interpreted by the OS ACPI subsystem.
ACPI namespace	Hierarchical object tree formed by loading the DSDT and SSDTs; devices, names and methods can reference one another there.
control method	Evaluated AML object that can compute values and perform ACPI-defined platform operations.
OperationRegion	Declared address space through which AML can access defined platform registers/data.
SCI	System Control Interrupt used for ACPI runtime events on conventional ACPI hardware.
GPE	General-Purpose Event bit/source dispatched to an AML method or an ACPI-aware native driver.
`_Lxx` / `_Exx`	Conventional level-/edge-triggered GPE control methods associated with event number `xx`.
embedded controller (EC)	Platform microcontroller with a standardized ACPI host interface and query mechanism; often manages OEM-specific board functions.
`_Qxx`	Optional EC query method convention used to handle a query value returned by the embedded controller.

AML is executed by an OS interpreter, not by “calling back into the BIOS” for every method. Firmware supplies the bytecode and hardware interface contract; the OS ACPI implementation evaluates that bytecode and decides policy. Some events are deliberately dispatched to native drivers instead of AML, and a bad or overly slow firmware method can still affect OS behavior because the interpreter is executing vendor-supplied platform logic.

READ-ONLY ACPI INSPECTION LAB # List firmware ACPI tables exported by Linux ls -l /sys/firmware/acpi/tables 2>/dev/null # acpidump/acpixtract/iasl are ACPICA tools when installed # acpidump > acpi.out # acpixtract -a acpi.out # iasl -d dsdt.dat # Kernel ACPI namespace / event debugging interfaces are configuration-dependent. # Do not write arbitrary EC/ACPI registers on a production machine.

OFFICIAL DOCSACPICA documentation and tools

Official ACPICA documentation hub for the AML interpreter, iASL compiler/disassembler and ACPI Component Architecture used by operating systems including Linux.

https://www.intel.com/content/www/us/en/developer/topic-technology/open/acpica/documentation.html

OFFICIAL SPECACPI 6.6 — runtime architecture and AML interpreter model

Current ACPI specification introduction explaining definition blocks, AML interpretation, ACPI registers/platform firmware and the OS-directed power-management model.

https://uefi.org/specs/ACPI/6.6/01_Introduction.html

SMBIOS/DMI is firmware inventory, not hardware discovery: entry point → typed records → sysfs/dmidecode

ACPI and Device Tree help the operating system describe/configure hardware; PCI/USB probing discovers real devices on buses. SMBIOS serves a different purpose: platform firmware publishes a typed inventory describing the system, baseboard, firmware, processors, memory devices, slots and other management-oriented facts. On Linux this historical data model is often called DMI, and the kernel exposes both raw tables and decoded identifiers.

firmware builds SMBIOS entry point + structure table ↓ entry point says SMBIOS version / table location or size ↓ sequence of typed structures ├── Type 0 BIOS / system firmware information ├── Type 1 system product / vendor / UUID ├── Type 2 baseboard ├── Type 4 processor information ├── Type 9 system slots ├── Type 16 physical memory array └── Type 17 memory device (DIMM/socket-oriented metadata) ↓ Linux scans DMI/SMBIOS early ├── may use selected identifiers for platform quirks └── exposes raw data under /sys/firmware/dmi/ ↓ userspace tools such as dmidecode interpret records THIS IS INVENTORY METADATA not proof that a DIMM/device is electrically present, not PCI/USB enumeration, and not the same DMI as Intel Direct Media Interface.

SMBIOS concept	What it means
entry point	Small firmware-provided structure identifying the SMBIOS version and where/how the table can be accessed.
structure type	Numeric record class whose formatted bytes and following strings have a specification-defined interpretation.
handle	Firmware-provided 16-bit identifier that other SMBIOS records can reference; Linux documentation cautions that firmware data can be imperfect.
Type 17	Memory Device metadata such as locator, size, type/speed fields and manufacturer/serial information where firmware supplies them.
`/sys/firmware/dmi/tables/`	Raw SMBIOS entry point and DMI table exported by Linux as an alternative to reading physical memory directly.
`/sys/firmware/dmi/entries/`	Per-entry Linux sysfs view exposing type, length, instance, handle and raw structure bytes.
DMI match / quirk	Kernel workaround selected by firmware-reported machine identity. It is a compatibility mechanism, not generic hardware detection.

Do not over-trust inventory strings. SMBIOS is authored by platform firmware. Linux explicitly notes that it cannot guarantee exported DMI data is correct, so serial numbers, slot labels, memory details and OEM fields should be treated as firmware claims. Bus enumeration, SPD reading, EDAC data or device-specific telemetry may answer different questions.

DIRECT SPEC PDFDMTF SMBIOS 3.9.0 specification — direct PDF

Current published SMBIOS standard. Defines the entry points and typed management structures used for system, processor, memory, slot and other platform inventory records.

https://www.dmtf.org/sites/default/files/standards/documents/DSP0134_3.9.0.pdf

STANDARD INDEXDMTF — SMBIOS standard page

Official version index. As of this revision, DMTF lists SMBIOS 3.9.0, published 19 August 2025, as the current standard.

https://www.dmtf.org/standards/smbios

KERNEL ABI DOCLinux — raw SMBIOS/DMI tables in sysfs

Short kernel ABI document explaining /sys/firmware/dmi/tables/smbios_entry_point and /sys/firmware/dmi/tables/DMI, and why Linux exposes them instead of requiring /dev/mem access.

https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-firmware-dmi-tables

Device Tree turns a board schematic into boot-time data: DTB → nodes/properties → driver population

Many embedded ARM, RISC-V, PowerPC and other systems contain hardware that cannot discover itself the way PCI or USB devices can. The kernel still needs to know which UART sits at which MMIO address, which interrupt line it uses, which clocks and regulators feed it, how GPIOs are wired, where RAM begins, and which driver understands the block. A Devicetree supplies that board/SoC description as data rather than hard-coding every board into the kernel.

BOARD / SoC DESCRIPTION human-readable DTS + included .dtsi files ↓ device-tree compiler (dtc) Flattened Devicetree binary: .dtb / FDT ↓ firmware / bootloader selects or modifies DTB ├── board revision / RAM size ├── bootargs + initrd addresses in /chosen └── overlays or fixups where platform supports them ↓ boot kernel with pointer to FDT blob ↓ early kernel scan ├── /memory → usable physical RAM ├── /cpus → CPU topology/data ├── /chosen → command line / initrd metadata └── interrupt / timer / platform essentials ↓ unflatten into kernel OF/devicetree objects ↓ walk nodes that represent devices ↓ compatible = "vendor,specific-device", "fallback-device" ↓ match against driver's OF match table platform / I2C / SPI / other device object ↓ driver probe() ↓ reg → MMIO resources interrupts → IRQ resources clocks / resets / regulators / GPIO phandles → supplier resources ↓ configured kernel device

Devicetree concept	What it means
DTS / DTSI	Human-readable Devicetree source and include fragments. They describe hierarchy and properties; they are not executable driver code.
DTB / FDT	Flattened binary representation passed from a boot program to the kernel in memory.
node	One object in the hardware tree, often a bus, controller, CPU, memory region or attached device.
`compatible`	Ordered strings describing the device programming model; drivers use these strings for matching, usually from most specific to more general fallbacks.
`reg`	Address/size tuples in the parent bus address space, commonly describing MMIO register windows.
`interrupts`	Interrupt specifier interpreted using the referenced/parent interrupt-controller binding.
phandle	Cross-reference from one node to another, used for relationships such as clocks, resets, regulators, GPIO controllers or DMA engines.
`status`	Common property controlling whether a described device is available/enabled.
binding	Schema/contract defining which properties a device class or specific compatible string accepts and what those properties mean.
overlay	A mechanism that can add/modify parts of a base tree for optional hardware or runtime board configuration; support and safe use are platform-specific.

Devicetree is description, not automatic hardware detection. A DT node can say that a UART exists at an address even when no hardware is actually there; conversely, a missing node can make non-discoverable hardware invisible to the OS. Self-enumerating buses such as PCI and USB can discover many child devices dynamically, while DT often describes the host controller and fixed board wiring around them.

ACPI and Devicetree solve overlapping platform-description problems with different models. ACPI combines standardized tables with an AML namespace/method language and is dominant on PCs/servers; Devicetree is primarily declarative data and is especially common on embedded systems. A Linux kernel can support both mechanisms.

OFFICIAL SPECDevicetree Specification — current online specification

Defines the logical tree, standard properties such as compatible/reg/ranges, required nodes, bindings and the flattened DTB encoding used between boot programs and client software.

https://devicetree-specification.readthedocs.io/en/latest/

KERNEL DOCSLinux and the Devicetree — usage model

Explains Linux's boot-time use of FDT data for platform identification, runtime configuration and device population, including compatible matching and platform devices.

https://docs.kernel.org/devicetree/usage-model.html

KERNEL DOCSLinux — writing Devicetree bindings in json-schema

Shows how kernel DT bindings formally constrain compatible strings and properties so a DT is a validated hardware interface rather than an arbitrary bag of names.

https://docs.kernel.org/devicetree/bindings/writing-schema.html

How a computer knows time: RTC, clocksource, clockevent, TSC and timer interrupts

There is no single magical 'computer clock.' The CPU clock drives synchronous logic; an RTC preserves calendar time while power is off; a clocksource is a counter used to measure a timeline; a clockevent device generates an interrupt at a programmed future point; and the OS combines those hardware mechanisms with wall-clock offsets and synchronization such as NTP.

POWER OFF battery-backed RTC / platform timekeeper keeps approximate calendar time BOOT firmware / kernel reads RTC → initializes wall-clock estimate ↓ kernel selects a CLOCKSOURCE e.g. invariant x86 TSC, HPET, architectural timer, SoC counter ↓ repeated counter reads raw counter ticks × scale + accumulated offset ↓ kernel timekeeping base ├→ CLOCK_MONOTONIC ├→ CLOCK_BOOTTIME ├→ CLOCK_REALTIME = timeline + wall-time offset/corrections └→ other clock APIs FUTURE WAKEUP / TIMER hrtimer deadline ↓ kernel programs CLOCKEVENT DEVICE e.g. Local APIC timer / architectural timer / HPET comparator ↓ hardware counts until deadline timer interrupt ↓ kernel runs expired timer callbacks / wakes tasks / scheduler work Key split: CLOCKSOURCE answers 'what time is it on the monotonic timeline?' CLOCKEVENT answers 'interrupt me when we reach this future point.'

Clock/timer	Behavior
CPU core clock	High-frequency clock used to sequence core logic; may vary with DVFS/turbo and is not itself necessarily OS wall-clock time.
RTC	Low-power/battery-backed calendar/seconds clock surviving normal system power-off.
TSC	x86 Time Stamp Counter readable by RDTSC/RDTSCP; invariant TSC on modern Intel CPUs runs at a constant reference rate independent of P/C/T-state changes.
HPET	Memory-mapped fixed-rate counter plus comparators capable of generating timer interrupts.
clocksource	Kernel-selected monotonically advancing counter used as base timeline.
clockevent	Programmable hardware event source used to request an interrupt at/after a deadline.
CLOCK_REALTIME	Wall-clock time; settable/adjustable and may jump when corrected.
CLOCK_MONOTONIC	Nonsettable monotonic Linux timeline not subject to discontinuous wall-clock changes; does not include suspended time.
CLOCK_BOOTTIME	Like monotonic but includes system suspend duration.
CLOCK_MONOTONIC_RAW	Raw hardware-based monotonic time before normal frequency adjustments.

CPU frequency is not the same thing as TSC frequency. On Intel processors with invariant TSC, the TSC advances at a constant rate across ACPI performance/idle state changes, while the actual core can execute at changing frequencies. Intel recommends performance counters rather than TSC ticks when you need actual core-cycle behavior.

LINUX OBSERVATION LAB cat /sys/devices/system/clocksource/clocksource0/current_clocksource cat /sys/devices/system/clocksource/clocksource0/available_clocksource cat /proc/driver/rtc ls -l /dev/rtc* cat /proc/interrupts | grep -Ei 'timer|hpet|rtc|lapic' clock_gettime() APIs: compare CLOCK_REALTIME, CLOCK_MONOTONIC, CLOCK_MONOTONIC_RAW and CLOCK_BOOTTIME On x86: grep -E 'constant_tsc|nonstop_tsc' /proc/cpuinfo | head Exact names/availability depend on CPU, kernel, hypervisor and platform.

KERNEL DOCSLinux — clock sources, clock events and sched_clock()

Excellent kernel architecture explanation: clocksources provide the timeline; clockevent devices schedule interrupts at points on that timeline.

https://docs.kernel.org/timers/timekeeping.html

KERNEL DOCSLinux — high-resolution timers and dynamic ticks

Explains the generic time-of-day, clocksource, clockevent and hrtimer layers and why per-CPU clockevent devices are useful on SMP systems.

https://docs.kernel.org/timers/highres.html

KERNEL DOCSLinux — HPET driver

Concrete x86 timer hardware: one fixed-rate counter plus multiple comparators that can raise one-shot/periodic interrupts.

https://docs.kernel.org/timers/hpet.html

KERNEL DOCSLinux — RTC drivers

Explains battery-backed wall-clock hardware, /dev/rtcN, alarms and the distinction between RTC hardware and the kernel's running system clock.

https://docs.kernel.org/admin-guide/rtc.html

PUBLIC MANUALclock_gettime(2) — Linux clocks

Current user-space definitions for CLOCK_REALTIME, CLOCK_MONOTONIC, CLOCK_MONOTONIC_RAW, CLOCK_BOOTTIME and CPU-time clocks.

https://man7.org/linux/man-pages/man2/clock_gettime.2.html

DIRECT PDFIntel SDM Volume 3B — Time Stamp Counter (direct PDF)

Official system-programming manual explaining RDTSC/RDTSCP and invariant TSC: a constant-rate timestamp source even when core power/performance states change.

https://cdrdv2-public.intel.com/858456/253669-088-sdm-vol-3b.pdf

Clock synchronization is a feedback-control problem: NTP, PTP, PHCs and frequency correction

A local clocksource can be monotonic and extremely precise yet still disagree with UTC or another machine because its oscillator has an offset and a frequency error. Network time synchronization therefore does more than copy a timestamp. Software measures offset and path delay, rejects bad samples, estimates oscillator drift, and then steers the system or hardware clock gradually enough to preserve useful time behavior.

LOCAL FREE-RUNNING CLOCK quartz/reference oscillator ↓ clocksource / TSC / PHC counter ↓ local estimate of time NTP OVER AN IP NETWORK client timestamp t1 ───────────────► server receives t2 client receives t4 ◄─────────────── server sends t3 ↓ use four timestamps to estimate: path delay ≈ (t4 - t1) - (t3 - t2) clock offset ≈ ((t2 - t1) + (t3 - t4)) / 2 ↓ filter / select peers / reject outliers ↓ clock discipline loop ├─ small error: slew frequency/phase gradually └─ large/bootstrap error: implementation may step under policy ↓ adjtimex()/clock_adjtime() tunes kernel clock discipline PTP WITH HARDWARE TIMESTAMPS Ethernet MAC/NIC timestamps packet near the wire ↓ PTP Hardware Clock (PHC), e.g. /dev/ptp0 ↓ ptp4l synchronizes PHC to grandmaster PHC ↓ phc2sys system CLOCK_REALTIME Key split: NTP/PTP exchange timing information. The kernel/daemon servo decides how to steer a local clock. A PHC is a clock in hardware, not merely a packet timestamp field.

Term	Meaning
offset	Difference between two clocks at a chosen instant.
frequency error / drift	Difference in clock rate; even after offset is corrected, an oscillator can slowly diverge again.
slew	Correct time by temporarily changing the effective clock rate rather than jumping the displayed time.
step	Discontinuous change to the clock value. Useful during bootstrap but potentially disruptive to software using wall time.
NTP	Internet clock-synchronization protocol and associated clock-filter/selection/discipline model.
PTP	Precision Time Protocol; designed for much tighter synchronization, especially when network hardware supplies transmit/receive timestamps.
PHC	PTP Hardware Clock exported by Linux as a clock-capable character device such as `/dev/ptp0`.
hardware timestamp	Timestamp captured close to the MAC/PHY packet boundary, reducing variable software/queueing latency.
clock servo	Feedback controller estimating phase/frequency error and applying corrections to keep clocks aligned.
PPS	Pulse-per-second hardware timing signal often used as a precise phase reference.

Synchronizing clocks is not the same as choosing a clocksource. The clocksource supplies a local monotonic measurement basis. NTP/PTP compare that local notion of time with an external reference and discipline its phase/frequency. CLOCK_MONOTONIC_RAW intentionally exposes a timeline without NTP frequency corrections, while wall-clock time can be disciplined.

LINUX TIME-SYNC INSPECTION # Kernel clock-discipline state and frequency correction adjtimex --print 2>/dev/null || true # PTP hardware clocks exposed by NIC/platform drivers ls -l /dev/ptp* 2>/dev/null for d in /sys/class/ptp/ptp*; do [ -e "$d" ] && echo "$d: $(cat "$d/clock_name" 2>/dev/null)"; done # NIC timestamping capabilities, if ethtool supports them ethtool -T eth0 2>/dev/null # linuxptp examples (do not run blindly on production clocks) # ptp4l -i eth0 -m # phc2sys -s /dev/ptp0 -c CLOCK_REALTIME -m # Observe clocks with different correction semantics clock_gettime() APIs: CLOCK_REALTIME / CLOCK_MONOTONIC / CLOCK_MONOTONIC_RAW

OFFICIAL RFCRFC 5905 — Network Time Protocol Version 4

The core NTPv4 protocol and algorithms: four-timestamp exchanges, clock offset/delay estimation, peer selection and clock-discipline concepts. The RFC has later updates, so use the RFC Editor status page to follow amendments.

https://www.rfc-editor.org/info/rfc5905/

KERNEL DOCSLinux PTP hardware clock infrastructure

Current kernel interface for PHCs: get/set/adjust time and frequency, timestamp external events, generate periodic outputs and expose each registered hardware clock to userspace.

https://docs.kernel.org/driver-api/ptp.html

PUBLIC MANUALadjtimex(2) / clock_adjtime(2)

Shows the Linux clock-discipline API used to adjust phase, frequency, status, PPS behavior and synchronization state rather than merely setting a timestamp.

https://man7.org/linux/man-pages/man2/adjtimex.2.html

PROJECT DOCSlinuxptp ptp4l

The reference Linux PTP daemon for ordinary, boundary and transparent clock roles; documents Ethernet/UDP transports and delay mechanisms.

https://www.linuxptp.org/documentation/ptp4l/

PROJECT DOCSlinuxptp phc2sys

Explains the second synchronization step common on Linux: steer the system clock from a PHC that ptp4l has synchronized to the PTP domain.

https://www.linuxptp.org/documentation/phc2sys/

High-resolution timers: clocksource measures time, clockevent causes the wakeup, hrtimer orders deadlines

The existing timekeeping section separates clocksource from clockevent. The next layer is how Linux turns an application's absolute or relative deadline into a hardware wakeup. High-resolution timers are maintained in time order using per-CPU hrtimer structures; the earliest deadline drives the next programmable clockevent when high-resolution mode is available.

clock_gettime(CLOCK_MONOTONIC) ↓ read CLOCKSOURCE counter counter ticks × mult/shift + accumulated timekeeping base ↓ monotonic nanoseconds returned timerfd_settime(fd, ABSOLUTE deadline D) ↓ kernel converts clock-relative request to hrtimer expiry ↓ insert hrtimer into time-ordered per-CPU red-black tree ↓ is this now the earliest relevant deadline? ├── no → leave existing programmed event └── yes ↓ CLOCKEVENT driver programs hardware comparator for D CPU may become idle ↓ hardware counter reaches programmed event ↓ timer interrupt ↓ clockevent handler / hrtimer expiry processing ↓ timerfd expiration count increments wake task sleeping in poll/epoll/read on timerfd ↓ scheduler makes task runnable ↓ read(timerfd) returns number of expirations PERIODIC TIMER expiration interval = P if process wakes late by several periods, timerfd read returns the number of expirations that accumulated CLOCK_REALTIME ABSOLUTE TIMERS wall-clock changes matter CLOCK_MONOTONIC timers are insulated from discontinuous wall-clock setting NO_HZ / TICKLESS IDLE idle CPU need not wake at a fixed scheduler-tick rate if no work is due; kernel programs the next actual event instead.

Timing object	Purpose
clocksource	Free-running counter used to tell where the system is on a time line.
clockevent device	Programmable hardware timer capable of interrupting at a requested future time.
hrtimer	Nanosecond-resolution kernel timer object ordered by expiry time.
ktime_t	Kernel time representation used by hrtimer APIs, conceptually 64-bit nanoseconds.
timer wheel	Lower-overhead structure optimized for many coarse timeout-style timers rather than precision deadlines.
timerfd	File-descriptor interface exposing timer expirations through read()/poll()/epoll().
absolute timer	Deadline expressed as a clock value rather than delay-from-now; avoids cumulative drift in repeated scheduling.
periodic timer	Timer automatically rearmed at an interval; expiration count can accumulate when consumer is late.
NO_HZ	Dynamic-tick machinery reducing unnecessary periodic scheduler-tick interrupts when a CPU does not need them.
sched_clock()	Fast per-CPU-ish timestamping source used for scheduler/tracing accounting, distinct from user wall-clock APIs.

Clocksource and clockevent may be different hardware. One stable counter can define the timeline while another per-CPU timer/comparator supplies interrupts. This separation lets Linux choose the best counter for timekeeping and the best programmable event device for deadlines.

KERNEL DOCSLinux clocksource / clockevent timekeeping

Current explanation: clocksources provide the timeline, while clockevent devices program future interrupts and are ideally per-CPU on SMP systems.

https://docs.kernel.org/next/timers/timekeeping.html

KERNEL DOCSLinux hrtimer subsystem

Deep implementation notes: hrtimers use time-ordered red-black trees and nanosecond time rather than the coarse timer-wheel/jiffies model.

https://docs.kernel.org/next/timers/hrtimers.html

PUBLIC MANUALtimerfd_create(2)

User-space timer as a file descriptor: set expirations, block/read counts and integrate timers directly with poll/epoll.

https://man7.org/linux/man-pages/man2/timerfd_create.2.html

KERNEL DOCS/proc fdinfo timerfd fields

Current /proc documentation shows timerfd clock ID, accumulated ticks, remaining value and interval exposed through fdinfo.

https://docs.kernel.org/next/filesystems/proc.html

Some “system calls” do not enter the kernel every time: vDSO timekeeping fast paths

A normal system call crosses from user mode into kernel mode, runs privileged code and returns. That transition has nonzero cost. For read-mostly information that the kernel can safely publish, Linux can instead map a tiny kernel-built ELF image—the vDSO—into every process. libc can call an exported vDSO function using the ordinary ABI, so operations such as clock_gettime() can often compute a result in userspace without executing a syscall instruction.

application calls clock_gettime(CLOCK_MONOTONIC, &ts) ↓ libc wrapper resolves __vdso_clock_gettime (when available) ↓ normal userspace CALL, not SYSCALL vDSO code reads kernel-published timekeeping data + architecture clocksource/counter (for example TSC where suitable) ↓ retry-safe consistency check around concurrently updated data ↓ compute current time = base time + scaled counter delta ↓ return directly to application FAST PATH user → libc → vDSO → user (no privilege transition) FALLBACK / UNSUPPORTED CASE user → libc → syscall instruction → kernel timekeeping code → return WHY CONSISTENCY MATTERS kernel updates base/multiplier/clocksource-related data concurrently ↓ vDSO reader must never combine half-old + half-new fields ↓ read-side sequence/retry scheme provides a coherent snapshot.

Object	Role
vDSO	Kernel-supplied ELF shared object automatically mapped into a process address space.
AT_SYSINFO_EHDR	Auxiliary-vector entry that points to the vDSO ELF header on supported systems.
__vdso_clock_gettime	Typical architecture-specific exported symbol used by libc for fast clock reads.
clocksource	Kernel-selected hardware counter used to measure time passage.
syscall fallback	Kernel entry used when a requested operation/clock cannot be satisfied by the vDSO implementation.

Observable consequence: strace reports system-call entries. If libc satisfies a clock read entirely through the vDSO, there is no kernel syscall entry for strace to display. That is a useful reminder that a C library function name is not synonymous with “one syscall happened.”

CURRENT PUBLIC MANUALvdso(7) — current Linux man-pages

Explains how the kernel maps the virtual dynamic shared object, how programs locate it through the auxiliary vector, symbol/version conventions and architecture-specific exported helpers.

https://man7.org/linux/man-pages/man7/vdso.7.html

PUBLIC MANUALclock_gettime(3) — Linux manual

Use with the vDSO page to connect userspace clock APIs to monotonic/realtime clock semantics and the kernel timekeeping subsystem described earlier.

https://man7.org/linux/man-pages/man3/clock_gettime.3.html

# See the mapping: grep -E '\[vdso\]|\[vvar\]' /proc/self/maps # On many systems this prints clock calls without a matching syscall trace, # because libc uses the vDSO fast path: strace -e clock_gettime date +%s.%N

Wires are electrical: logic levels, pull-ups, loading and signal integrity

Digital schematics draw a wire as a line, but real wires and IC pins have resistance, capacitance and inductance. A receiver does not see an abstract 0 or 1; it sees a voltage that must cross specified thresholds soon enough, with acceptable noise and edge shape. This is where VIH, VIL, VOH, VOL, fan-out, pull-ups, open-drain, tri-state, rise time, reflections and crosstalk become real engineering rather than jargon.

DIRECT PDFTI — Implications of Slow or Floating CMOS Inputs (PDF)

Excellent application note on why CMOS inputs must not be left floating or held near the switching threshold. Shows simultaneous NMOS/PMOS conduction, excess current, oscillation and the role of pull-up/pull-down or bus-hold circuits.

https://www.ti.com/lit/an/scba004e/scba004e.pdf

DIRECT PDFTI — High-Speed Layout Guidelines (PDF)

Explains when PCB traces must be treated as transmission lines, why fast edge rate matters even when clock frequency seems modest, and how reflections, delay and crosstalk arise.

https://www.ti.com/lit/an/scaa082/scaa082.pdf

Real glue logic: the chips that connect blocks together

Old computers are full of small ICs whose jobs are now buried inside SoCs and chipsets. Reading a few of these datasheets makes bus direction, tri-state outputs, address decoding, counters and serial/parallel conversion much less abstract.

DATASHEETTI SN74HC138 — 3-to-8 decoder

Classic address-decoding/chip-select logic: three input bits select one of eight outputs. This is the kind of glue logic used to decide which RAM, ROM or peripheral responds to an address.

https://www.ti.com/product/SN74HC138

DATASHEETTI SN74HC161 — synchronous binary counter

Shows clocked counting, enable, load, reset and carry cascading. Good physical model for pieces of program counters, timing generators and address counters.

https://www.ti.com/product/SN74HC161

DATASHEET PDFTI SN74HC595 — shift register with output register (PDF)

Serial-in/parallel-out shift register with a second storage register and tri-state outputs. Excellent for seeing how two clock domains/signals can shift data internally and then update external outputs together.

https://www.ti.com/lit/ds/symlink/sn74hc595.pdf

5. Registers, ALU, buses, control signals and a complete CPU datapath

PUBLIC PROJECTBen Eater — complete 8-bit schematics

A high-value page: complete schematics for the clock, A/B/instruction registers, ALU, memory address register, RAM, program counter, output register, and control logic.

https://eater.net/8bit/schematics

PUBLIC PROJECTBen Eater — connecting the shared bus

Shows how independent CPU modules are connected to a common multi-bit bus and how control signals decide which module may drive or receive data.

https://eater.net/8bit/bus

PUBLIC PROJECTBen Eater — CPU control logic

Shows the control unit as the part that turns opcode + timing state into control signals. Includes microcode, flags, conditional jumps, reset behavior, and schematics.

https://eater.net/8bit/control

PUBLIC NOTESBerkeley CS61C — CPU datapath introduction

One of the best free explanations of datapath versus control. Builds a RISC-V processor through instruction fetch, decode, execute, memory access, and write-back.

https://notes.cs61c.org/content/datapath/

PUBLIC NOTESBerkeley CS61C — datapath state elements

Focuses on the program counter, register file, instruction memory, and data memory, including when state changes relative to the clock.

https://notes.cs61c.org/content/datapath/elements/

PUBLIC NOTESBerkeley CS61C — control logic design

Shows how instruction bits become signals that steer muxes, the ALU, registers, memory reads/writes, branches, and write-back.

https://notes.cs61c.org/content/datapath/control/

PUBLIC NOTESBerkeley CS61C — single-cycle datapath summary

A compact end-to-end view of the hardware blocks needed for IF, ID, EX, MEM, and WB.

https://notes.cs61c.org/content/datapath/summary/

PUBLIC NOTESBerkeley CS61C — five-stage pipeline

After understanding a single-cycle CPU, use this to see why pipeline registers are inserted and how multiple instructions occupy different stages simultaneously.

https://notes.cs61c.org/content/pipeline/five-stage-pipeline/

Hardwired control versus microcode: how an instruction becomes control signals

An opcode such as ADD does not directly 'tell the ALU to add.' The processor must generate a timed sequence: select source registers, route values through multiplexers/buses, choose an ALU function, latch the result, update flags, possibly access memory, and advance or replace the program counter. Hardwired control derives those signals with logic/FSM circuitry; microcoded control stores much of the sequencing as words in a control ROM.

TECH ARTICLEHow the 8086 microcode engine works — Ken Shirriff

Detailed die-level explanation of the 8086's 512×21-bit microcode, micro-address sequencing, shared routines, jumps/calls, translation ROM and group-decode ROM.

https://www.righto.com/2022/11/how-8086-processors-microcode-engine.html

INTERACTIVE8086 Microcode Explorer

No-login interactive viewer of all 512 decoded 8086 microinstructions. Hover fields to see register moves, ALU actions, branches and micro-address behavior.

https://nand2mario.github.io/8086_microcode.html

Bits do not inherently mean numbers, text or instructions

A wire or storage cell only carries a physical state that digital logic abstracts as 0 or 1. A bit pattern gets meaning from the circuit/software interpreting it. The same 32 physical bits may be treated as an unsigned integer, signed two's-complement integer, IEEE-754 floating-point number, instruction encoding, address, character data, flags, or four raw bytes.

Bits / bytes	One possible interpretation
0100 0001	Unsigned integer 65; in ASCII the byte value 65 denotes 'A'.
1111 1111	Unsigned 255, or signed 8-bit two's-complement −1.
0x3F800000	If interpreted as IEEE-754 binary32, this bit pattern represents +1.0.
instruction word	The decoder treats selected bit fields as opcode/register/immediate/control information according to the ISA.
address-sized bit pattern	The MMU/cache/interconnect may interpret it as a virtual/physical location rather than arithmetic data.

Endianness is another interpretation rule: when a multi-byte value is stored in byte-addressed memory, the architecture defines which byte goes at the lowest address. This changes byte order in memory, not the mathematical bit significance inside each byte.

PUBLIC NOTESCS61C — integer representations and overflow

Public notes on unsigned, sign-magnitude, one's complement, two's complement, bias encoding, overflow and binary arithmetic. Explicitly emphasizes that the same bits can mean different things depending on interpretation.

https://notes.cs61c.org/content/number-rep/integer-representations/

PUBLIC NOTESCS61C — two's complement

Focused public explanation of two's-complement representation and why the same binary addition hardware works for signed and unsigned addition.

https://notes.cs61c.org/content/number-rep/twos-complement/

PLAIN WEBCornell/Duke architecture notes — Two's Complement and Floating Point

Plain HTML notes with separate compact explanations of two's-complement integers and IEEE-754 floating point. No login.

https://www.cs.cornell.edu/~tomf/notes/cps104/

PUBLIC NOTESCornell numerical analysis — binary floating point

Current public technical notes on normalized binary floating point, sign/exponent/significand and why finite precision produces non-obvious numerical behavior.

https://www.cs.cornell.edu/courses/cs4220/2026sp/lec/2026-02-02.html

Worked machine-code decode: 32 raw bits become register selects and ALU control

A machine instruction is just a bit pattern until the CPU's decoder assigns meaning to fields. RISC-V is particularly clean for learning because the base 32-bit formats keep rd, rs1 and rs2 in fixed positions across formats, reducing decode wiring complexity.

EXAMPLE: add x5, x6, x7 machine word = 0x007302B3 binary = 0000000 00111 00110 000 00101 0110011 funct7 rs2 rs1 funct3 rd opcode bits 31:25 24:20 19:15 14:12 11:7 6:0 decoder wiring concept: instruction[6:0] = 0110011 → OP / register-register major opcode instruction[11:7] = 00101 → rd = x5 instruction[14:12] = 000 → arithmetic ADD/SUB class instruction[19:15] = 00110 → rs1 = x6 instruction[24:20] = 00111 → rs2 = x7 instruction[31:25] = 0000000 → selects ADD rather than SUB for this funct3/opcode register file: read port 1 address = 6 → operand A = x6 value read port 2 address = 7 → operand B = x7 value write port address = 5 control decode: regfile_read_a = 1 regfile_read_b = 1 alu_op = ADD regfile_write = 1 memory_read = 0 memory_write = 0 branch = 0 ALU computes x6 + x7 ↓ result routed to register-file write data ↓ clock/retirement rules make x5 architecturally contain the result The exact internal control-signal names vary by CPU implementation; the ISA fields and architectural result do not.

SECOND EXAMPLE: addi x5, x6, -4 machine word = 0xFFC30293 binary = 111111111100 00110 000 00101 0010011 imm[11:0] rs1 funct3 rd opcode imm = 0xFFC as a 12-bit signed two's-complement number = -4 decoder sign-extends bit 31 across XLEN ALU input A = x6 ALU input B = sign_extended(-4) ALU op = ADD write rd=x5 Same adder hardware can therefore perform address calculations, stack-pointer adjustments and ordinary integer ADDI operations.

Field	Hardware use
opcode	Selects broad instruction class/datapath behavior.
rd	Addresses destination register/write-back target.
rs1 / rs2	Address register-file source read ports.
funct3 / funct7	Further decode within an opcode family, e.g. ADD versus SUB and shift/logical variants.
immediate	Constant/address displacement assembled/sign-extended by immediate-generation logic.
decoder outputs	Enable register reads/writes, choose ALU op, select mux inputs, request load/store, branch/jump, CSR, exception or other actions.
illegal encoding detector	Raises an illegal-instruction trap when bit combination is not a supported/legal instruction for the configured ISA.

The assembler mnemonic is not stored in memory. Memory contains instruction bits. The string add x5,x6,x7 is a human-readable assembler representation of the encoding; hardware sees fields and control conditions.

OFFICIAL SPECRISC-V unprivileged ISA — current January 2026 RV32I instruction formats

Current ratified text. Defines fixed rd/rs1/rs2 positions, immediate layouts and ADD/ADDI/register-register semantics.

https://docs.riscv.org/reference/isa/v20260120/unpriv/rv32.html

OFFICIAL SPECRISC-V RV32/64 instruction encoding tables

Compact official table of R/I/S/B/U/J formats and major instruction encodings—excellent for decoding raw words by hand.

https://docs.riscv.org/reference/isa/unpriv/rv-32-64g.html

SOURCE / FILEIbex real SystemVerilog decoder

Actual production-quality open CPU decoder. The module is combinational and turns instruction fields into register read/write enables, ALU operations, branch/load/store/control signals and illegal-instruction detection.

https://github.com/lowRISC/ibex/blob/master/rtl/ibex_decoder.sv

PUBLIC LABIbex demo lab — modifying the real decoder

Hands-on public lab that adds a custom opcode, edits ibex_decoder.sv and adds ALU operations. No course account is required to read or clone the material.

https://github.com/lowRISC/ibex-demo-system-labs/blob/main/lab4.md

OFFICIAL SPECRISC-V compressed-instruction extension

After fixed 32-bit encoding makes sense, this shows how 16-bit compressed instructions encode common operations and expand to ordinary base instructions.

https://docs.riscv.org/reference/isa/unpriv/c-st-ext.html

From C source to bytes the CPU can fetch: compiler, assembler, linker and loader

A CPU never executes C, Rust or Python source text directly. For a conventional ahead-of-time compiled C program, the useful simplified path is preprocess → compile → assemble → link → load → execute. Each stage changes the representation and resolves a different class of names/addresses.

hello.c │ preprocessor: #include, #define, conditional compilation ↓ preprocessed C │ compiler: parse + optimize + instruction selection ↓ assembly text (.s) │ assembler: encode instructions + emit symbols/relocations ↓ relocatable object (.o) │ linker: combine objects/libraries, resolve symbols, relocate, choose layout/entry ↓ ELF executable / shared objects │ OS loader: map loadable segments, arrange process image, dynamic linking as needed ↓ entry point → startup code → main() ↓ machine instructions fetched by CPU

Thing	Why it exists
symbol	Human/toolchain name such as function or global variable; later associated with an address or definition.
relocation	Record saying 'this encoded location depends on an address that is not known yet; patch it when layout is known.'
.text	Conventional ELF section containing executable machine code.
.rodata	Read-only constants.
.data	Initialized writable data.
.bss	Zero-initialized/uninitialized writable storage represented compactly in the file.
section	Link-time/object-file organization used by assemblers/linkers.
segment / program header	Loader-oriented mapping description telling the OS which file ranges become memory mappings with which permissions.
entry point	Address where control begins after the loader has prepared the process.

OFFICIAL DOCSGCC — the four compilation stages

Official GNU documentation explicitly describing preprocessing, compilation proper, assembly and linking, plus -E, -S and -c so you can stop and inspect each intermediate representation.

https://gcc.gnu.org/onlinedocs/gcc/Overall-Options.html

OFFICIAL DOCSGNU assembler — what an object file is

Official documentation: the assembler translates assembly language into numeric instruction/data encodings plus information the linker needs to integrate that object into a runnable file.

https://www.sourceware.org/binutils/docs/as/Object.html

OFFICIAL DOCSGNU linker ld — overview

Official public manual explaining how object/archive files are combined, references are resolved, relocation happens and an executable output is produced.

https://sourceware.org/binutils/docs/ld.html

OFFICIAL DOCSGNU linker scripts — memory layout is not magic

Shows that linkers explicitly map input sections into an output memory layout. Essential for embedded computers, ROM/RAM placement and understanding why addresses end up where they do.

https://sourceware.org/binutils/docs/ld/Scripts.html

PUBLIC MANUALELF format — Linux man page

Plain public reference to ELF headers, program headers, section headers, executable files, relocatable objects and shared objects.

https://man7.org/linux/man-pages/man5/elf.5.html

INTERACTIVECompiler Explorer

No account required for ordinary use. Type a small C/C++ function and immediately see the assembly produced by different compilers/optimization levels; useful for connecting source constructs to ISA instructions.

https://godbolt.org/

Writing code bytes and executing them are separate memory streams: JITs need instruction-fetch synchronization

A JIT compiler, dynamic loader, debugger breakpoint patcher or kernel live patch can modify memory that will later be fetched as instructions. Data stores reaching the coherent memory system do not universally guarantee that an instruction cache or fetch pipeline immediately sees the new bytes. The required synchronization is architecture-specific.

JIT / CODE PATCHER allocate writable memory ↓ CPU executes DATA STORES containing generated instruction bytes ↓ D-cache / store buffer / coherent memory system now has new bytes QUESTION: will a later INSTRUCTION FETCH see them? x86-style strongly coherent instruction/data hierarchy: cache-coherence machinery keeps memory contents coherent, but self-/cross-modifying code must still obey architectural serialization/synchronization rules so prefetched/decoded old instructions are not executed. ARM64-style explicit maintenance path: clean modified D-cache lines toward point of unification invalidate corresponding I-cache lines barriers / context synchronization Linux flush_icache_range() encapsulates required operations RISC-V: stores to instruction memory ↓ FENCE.I on executing hart synchronizes prior visible stores with later instruction fetches on THAT hart other harts may still have stale instruction state ↓ data-ordering step + remote fence.i / OS-mediated shootdown as required LINUX RISC-V USERSPACE ordinary userspace cannot safely solve migration-to-another-hart by one local FENCE.I alone ↓ riscv_flush_icache syscall / runtime helper asks kernel to coordinate MM-wide coherence PORTABLE COMPILER INTERFACE __builtin___clear_cache(begin, end) ↓ on targets requiring I-cache maintenance: compiler emits/calls appropriate mechanism on targets not requiring an explicit cache flush: it may be a no-op W^X / WX POLICY common safe pattern: map/write as RW finish code generation instruction-cache synchronization change mapping to RX execute Some runtimes use dual mappings/other OS facilities instead of toggling one mapping.

Instruction-coherence term	Meaning
self-modifying code	One execution context writes bytes that it may later fetch as instructions.
cross-modifying code	One CPU/thread changes instruction bytes another CPU/thread may execute; requires inter-core synchronization.
instruction-cache coherence	Guarantee/mechanism that instruction fetch eventually observes stores that changed executable memory.
point of unification	ARM cache concept at which instruction/data memory streams are guaranteed to meet for cache-maintenance purposes.
FENCE.I	RISC-V instruction ordering prior visible stores before subsequent instruction fetches on the same hart.
remote FENCE.I	Mechanism making other harts refresh instruction-fetch state after code changes.
flush_icache_range()	Kernel architecture abstraction synchronizing newly written/modified instruction memory before execution.
__builtin___clear_cache()	GCC portable builtin requesting instruction-cache synchronization for a generated/modified code range.
W^X	Security policy/design principle avoiding writable-and-executable memory at the same time.
JIT write-protect transition	Runtime changes memory from writable generation state to executable state after publication/synchronization.
stale fetch/decode	Old instruction bytes may remain in I-cache, predecode/µop/fetch structures even after data-side stores changed memory.

A data-memory fence and an instruction-fetch fence are not automatically the same thing. RISC-V makes this especially explicit: ordinary FENCE orders explicit data accesses, while FENCE.I exists specifically to synchronize the instruction stream.

SAFE JIT / CACHE-SYNC LAB IDEA 1. mmap one page RW (not executable). 2. Write a tiny architecture-specific function into it in a disposable test program. 3. Call __builtin___clear_cache(page, page + code_len). 4. mprotect(page, page_size, PROT_READ|PROT_EXEC). 5. Call through a correctly typed function pointer. Compare generated assembly for __builtin___clear_cache on different targets using Compiler Explorer: x86-64 may need no cache-flush sequence for the builtin itself ARM/RISC-V targets may require architecture/runtime support For Linux/RISC-V, read arch/riscv/mm/cacheflush.c and sys_riscv.c to see why the kernel tracks stale instruction caches across hart migration. Keep W and X permissions separated where your OS/runtime supports that design.

OFFICIAL SPECRISC-V Zifencei / FENCE.I specification

Official current specification: FENCE.I synchronizes prior visible stores with later instruction fetches on the same hart and is the standard RISC-V instruction-fetch coherence mechanism.

https://docs.riscv.org/reference/isa/unpriv/zifencei.html

SOURCE / FILELinux RISC-V instruction-cache flush implementation

Current kernel code coordinates local and remote instruction-cache flushing, including IPIs/SBI remote fences and deferred flushes for CPUs not currently executing the mm.

https://github.com/torvalds/linux/blob/master/arch/riscv/mm/cacheflush.c

SOURCE / FILELinux RISC-V flush-icache syscall

Current source explains why one userspace FENCE.I is insufficient under task migration and implements the RISC-V-specific OS-mediated icache synchronization syscall.

https://github.com/torvalds/linux/blob/master/arch/riscv/kernel/sys_riscv.c

SOURCE / FILELinux arm64 cacheflush implementation

Concrete contrasting architecture: flush_icache_range() performs cache clean/invalidate work and forces CPU context synchronization so new instructions are refetched.

https://github.com/torvalds/linux/blob/master/arch/arm64/include/asm/cacheflush.h

COMPILER DOCSGCC builtin_clear_cache

Portable compiler interface for code-generation runtimes: GCC emits a target-specific instruction-cache flush/call where required and does nothing when the target requires none.

https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html

Randomization changes addresses, not permissions: userspace ASLR and kernel KASLR

Address Space Layout Randomization (ASLR) makes useful addresses harder to predict across executions. It does not make memory unreadable or non-executable by itself; page permissions, credentials and control-flow defenses still do the actual enforcement. Linux also randomizes selected kernel regions at boot through KASLR and related layout randomization.

SAME PIE EXECUTABLE, TWO RUNS RUN 1 main executable base = 0x55a1c9... libc base = 0x7f48b2... mmap region = 0x7f48... stack = 0x7ffd91... VDSO = 0x7ffd93... heap/brk = 0x55a1f2... RUN 2 main executable base = 0x55d8a6... libc base = 0x7f0b31... mmap region = different stack = different VDSO = different heap/brk = different when full randomization enabled /proc/sys/kernel/randomize_va_space 0 = disable process address randomization 1 = randomize mmap base, stack, VDSO; PIE code base also randomized 2 = plus heap/brk randomization (typical default) PIE MATTERS ET_EXEC fixed-address main executable: shared libraries/stack/mmap may randomize main text traditionally loads at fixed virtual address PIE (ET_DYN main executable): loader can relocate main image to randomized base KASLR boot-time entropy ↓ relocate kernel physical/virtual text base when CONFIG_RANDOMIZE_BASE enabled ↓ module-load region/base also receives randomization other kernel layout randomizations may affect stacks/dynamic regions/build-time structure layout WHY RANDOMIZATION HELPS attacker needs code/data/gadget target address ↓ address differs from prior run/other machine ↓ exploit often needs an INFORMATION LEAK or brute force first WHY RANDOMIZATION IS NOT ENOUGH memory disclosure can reveal actual bases forked children may share some inherited layout limited entropy reduces search space logic bugs still operate with valid addresses ASLR is probabilistic defense; NX/SMEP/CET/etc. are separate mechanisms.

Randomized component	Purpose / condition
mmap base	Changes placement of shared libraries and many anonymous/file mappings.
stack base	Moves initial userspace stack and associated stack-resident objects.
VDSO	Moves kernel-provided userspace helper mapping.
PIE executable base	Allows main executable text/data to load at a randomized base.
brk/heap	Additional process-heap randomization under randomize_va_space=2.
kernel text base	KASLR boot-time relocation under CONFIG_RANDOMIZE_BASE.
module base	Randomized module region/load offset reduces common kernel-module addresses.
kernel stack/dynamic regions	Additional kernel self-protection can randomize layout or offsets beyond the main text base.
structure layout randomization	Build-time randomization can vary sensitive kernel structure field layouts across builds.

ASLR is probabilistic. Linux's own self-protection documentation classifies KASLR as a statistical defense: it raises exploit difficulty, but an information disclosure can reveal the randomized layout.

ASLR LAB cat /proc/sys/kernel/randomize_va_space # Compare addresses across repeated runs for i in 1 2 3; do /bin/sh -c 'printf "pid=%s\n" $$; head -10 /proc/$$/maps' done # Is an executable PIE? readelf -h /bin/ls | grep 'Type:' # kernel command line / KASLR config clues cat /proc/cmdline grep '^CONFIG_RANDOMIZE_BASE=' /boot/config-$(uname -r) 2>/dev/null # Do not disable ASLR globally on a normal system for experimentation. # If you need a deterministic debugger lab, use a disposable process/debugger setting.

KERNEL DOCSLinux randomize_va_space sysctl

Current kernel documentation defines values 0/1/2 and which userspace regions are randomized, including PIE code and heap/brk under full randomization.

https://docs.kernel.org/admin-guide/sysctl/kernel.html

KERNEL DOCSLinux Kernel Self-Protection

Current self-protection guide explicitly classifies KASLR as probabilistic and describes kernel text/module/dynamic-layout randomization.

https://docs.kernel.org/security/self-protection.html

x86 CET protects control flow: shadow stacks check returns and IBT constrains indirect-branch targets

Stack canaries and NX make some code-reuse attacks harder but do not directly verify where returns and indirect branches go. x86 Control-flow Enforcement Technology (CET) adds hardware support for two complementary mechanisms: Shadow Stack protects return addresses, and Indirect Branch Tracking (IBT) validates indirect call/jump landing sites.

NORMAL CALL WITH USER SHADOW STACK ENABLED CALL foo normal RSP stack: push return RIP = R shadow stack: hardware also records R in protected shadow-stack memory ↓ foo executes attacker corrupts NORMAL stack return address to gadget G ↓ RET pop normal return = G obtain shadow return = R compare G != R ↓ hardware raises control-protection exception (#CP) instead of silently returning to gadget G INDIRECT BRANCH TRACKING compiler/linker marks valid indirect-entry targets with ENDBR64 indirect CALL/JMP target = function entry with ENDBR64 ↓ allowed indirect CALL/JMP target = middle of ordinary instruction sequence ↓ CET tracking state expects ENDBR ↓ #CP if target is not a valid marked landing pad DIRECT branches are not checked by IBT in the same way. LINUX STATUS userspace shadow stack: hardware + kernel CONFIG_X86_USER_SHADOW_STACK + supporting loader/libc executable advertises SHSTK ELF GNU property loader enables feature through arch_prctl interface kernel IBT: kernel built/instrumented with IBT support indirect kernel branch targets use ENDBR-compatible instrumentation CET is complementary to: ASLR/KASLR NX/W^X stack canaries CFI/compiler hardening SMEP/SMAP It does not replace memory-safety bug prevention.

CET item	Role
Shadow Stack	Protected secondary return-address stack maintained by hardware alongside the normal call stack.
#CP	x86 control-protection exception raised on CET control-flow violations.
ENDBR64	Instruction marking a valid indirect-branch landing site for 64-bit IBT.
IBT	Indirect Branch Tracking: indirect call/jump target validation using ENDBR landing pads.
GNU_PROPERTY_X86_FEATURE_1_SHSTK	ELF GNU property indicating userspace shadow-stack capability.
ARCH_SHSTK_ENABLE	Linux arch_prctl operation used by runtime/loader to enable a userspace shadow-stack feature.
SSP	Shadow Stack Pointer tracking the protected return-address stack.
WRSS	CET instruction family allowing controlled writes to shadow-stack memory under defined permissions.
kernel IBT	Linux kernel control-flow hardening using ENDBR landing pads for indirect branch targets.
userspace SHSTK	Linux support for hardware shadow stacks in compatible userspace applications/runtimes.

Linux support is asymmetric today. Current x86 kernel documentation states that 64-bit Linux supports userspace shadow stack and kernel IBT; hardware may implement both features, but kernel/userspace enablement is a separate software question.

CET OBSERVATION # CPU/kernel feature strings when supported grep -m1 '^flags' /proc/cpuinfo | tr ' ' '\n' | grep -E 'shstk|ibt|user_shstk' # Executable GNU properties readelf -n /bin/ls | grep -A4 -Ei 'x86 feature|SHSTK|IBT' # Kernel configuration clues grep -E '^CONFIG_X86_(USER_SHADOW_STACK|KERNEL_IBT)=' /boot/config-$(uname -r) 2>/dev/null # Compiler-produced ENDBR landing pads (if binary was built with IBT) objdump -d -M intel /bin/ls 2>/dev/null | grep -m10 -B1 -A1 'endbr64' # Absence of an ELF SHSTK note does not mean CPU lacks CET; # hardware support, kernel support and application enablement are separate.

KERNEL DOCSLinux x86 CET Shadow Stack

Current kernel documentation explains CET shadow stacks and IBT, return-address comparison, ENDBR landing pads, userspace enabling and Linux's current userspace-shadow-stack/kernel-IBT support split.

https://docs.kernel.org/arch/x86/shstk.html

OFFICIAL DOCSIntel Software Developer Manuals

Primary architecture reference for CET, shadow-stack paging state, control-protection faults, ENDBR and IBT semantics.

https://www.intel.com/content/www/us/en/developer/articles/technical/intel-sdm.html

What happens during an ordinary function call: ABI, stack, return address and saved registers

The ISA defines instructions such as CALL/RET or JAL/JALR, but it does not by itself define how C functions agree on argument registers, return values, preserved registers, stack alignment or object layout. Those rules come from an Application Binary Interface (ABI).

C source: long f(long a, long b) { return a + b; } x = f(10, 20); x86-64 System V ABI (integer/simple case) caller places a=10 in RDI caller places b=20 in RSI CALL pushes/records return address and transfers RIP callee may create stack frame / save callee-saved registers callee computes result result returned in RAX RET restores next instruction address from stack RISC-V standard integer ABI caller places a=10 in a0 (x10) caller places b=20 in a1 (x11) JAL/JALR records return address in ra (x1) callee preserves s0..s11 if it modifies them temporaries t0..t6 need not survive the call result returns in a0 (and a1 for second return register when required) JALR via ra returns The stack is ordinary memory. SP is just a register whose ABI-defined convention makes that memory region usable for frames, spills, locals and calls.

ABI idea	Why it is necessary
argument registers	Caller and callee must agree where incoming values are found.
return-value registers	Both sides need a fixed place for returned scalars/aggregates or rules for indirect return.
caller-saved registers	Caller must preserve them itself if it needs their values after a call.
callee-saved registers	Callee must restore them before returning if it used/changed them.
stack pointer/alignment	Keeps stack frames and vector/aggregate accesses aligned according to ABI requirements.
return address	Identifies instruction at which caller continues after callee returns.
frame pointer	Optional stable reference into a stack frame; compilers often omit it when unnecessary.
red zone	x86-64 SysV permits limited stack-adjacent temporary space below RSP for leaf functions; this is ABI-specific, not universal.
unwinding metadata	Allows debuggers/exceptions/profilers to reconstruct call stacks even when frame pointers are absent.

ABI ≠ ISA. Linux x86-64, Windows x64 and firmware environments all execute the same basic x86-64 instruction set but use different calling conventions in important details. Likewise, RISC-V's ISA and its psABI are separate specifications.

OFFICIAL ABIx86-64 System V psABI — official source repository

Maintained public psABI source with a link to the latest generated PDF. The function-calling section defines register use, parameter passing, stack frames/alignment and the red zone.

https://gitlab.com/x86-psABIs/x86-64-ABI/-/tree/master

OFFICIAL ABIx86-64 psABI — function calling sequence source

Direct public source for the calling-convention chapter, including register preservation, stack-frame layout and argument classification.

https://gitlab.com/x86-psABIs/x86-64-ABI/blob/master/x86-64-ABI/low-level-sys-info.tex

OFFICIAL ABIRISC-V psABI — current public specification

Current pre-release version 1.1 dated August 13, 2026. Defines ra/sp/gp/tp, temporary/callee-saved registers, a0-a7 argument registers, procedure calling, ELF, relocations and more.

https://riscv-non-isa.github.io/riscv-elf-psabi-doc/

execve() replaces the process image: ELF segments, stack, auxiliary vector and dynamic linker

execve() does not create a new process ID. It replaces the current process image with a new executable image. For an ELF program, the kernel reads program headers, creates the required virtual mappings, builds a new user stack containing arguments/environment plus an auxiliary vector, and—when PT_INTERP requests it—starts the dynamic linker so shared libraries and relocations can be resolved before ordinary program startup.

shell / parent process ↓ fork() or equivalent process-creation path child calls execve('/bin/ls', argv, envp) ↓ kernel opens executable and recognizes ELF ↓ read ELF program headers ↓ discard/replace old userspace mappings ↓ map PT_LOAD segments with R/W/X protections ↓ create user stack ├── argc ├── argv[] strings/pointers ├── envp[] strings/pointers └── ELF auxiliary vector (AT_* entries) ↓ if PT_INTERP exists: map/start dynamic linker (ld-linux.so) ↓ load needed shared objects perform required relocations / symbol binding initialize libraries jump to program entry/startup code otherwise: jump to executable entry point/startup directly ↓ _start → libc/runtime initialization → main(argc, argv, envp) PID can remain the same even though code/data/stack mappings are now different.

Typical mapping	What lives there
executable PT_LOAD R-X	Machine instructions and read-only executable contents.
executable/library R--	Read-only constants, ELF metadata or relocation-related mapped data.
executable/library RW-	Writable globals, relocation targets and related data.
[heap]	Traditional brk()-managed process heap area; malloc can also use anonymous mmaps elsewhere.
anonymous mmap regions	Allocator arenas, large allocations, thread stacks, JIT/data areas and other anonymous mappings.
shared-library mappings	libc, libm, loader and other DSOs selected by dynamic linking.
[stack]	Initial/main thread stack including call stack plus startup argv/env/auxiliary-vector region.
[vdso]	Kernel-provided user-mapped helper code for selected operations without a normal syscall transition.

NO-ACCOUNT INSPECTION LAB readelf -l /bin/ls → find PT_LOAD and PT_INTERP cat /proc/$$/maps → inspect the shell's actual VMAs and R/W/X permissions pmap -x $$ → alternate view of mappings/resident usage LD_SHOW_AUXV=1 /bin/true → display ELF auxiliary-vector entries passed by the kernel LD_DEBUG=libs /bin/true → watch the dynamic linker search/load shared objects strace -f -e execve,mmap,mprotect,brk /bin/ls >/dev/null → watch process-image setup related system calls/mappings

PUBLIC MANUALexecve(2) — current Linux manual

Current manual explains process-image replacement, ELF PT_INTERP behavior, dynamic linker invocation and which process attributes survive or reset across exec.

https://man7.org/linux/man-pages/man2/execve.2.html

KERNEL DOCSLinux ELF-specific behavior

Current kernel documentation for Linux ELF details including PT_INTERP and PT_GNU_STACK handling.

https://docs.kernel.org/next/ELF/ELF.html

PUBLIC MANUALld.so(8) — dynamic linker/loader

Current public manual describing how ld-linux loads shared objects, resolves search paths and transfers control to the program; includes LD_DEBUG and diagnostic modes.

https://man7.org/linux/man-pages/man8/ld.so.8.html

PUBLIC MANUAL/proc/PID/maps — actual process virtual memory map

Current manual for mapped ranges, permissions, offsets, file backing, [heap], [stack] and [vdso]. It explicitly suggests correlating ELF mappings with readelf -l.

https://man7.org/linux/man-pages/man5/proc_pid_maps.5.html

PUBLIC MANUAL/proc/PID/smaps — mapping-by-mapping resident memory

Adds RSS/PSS/private/shared/anonymous/dirty and other memory accounting to every VMA so virtual size can be compared with physical residency.

https://man7.org/linux/man-pages/man5/proc_pid_smaps.5.html

PUBLIC MANUALgetauxval(3) — ELF auxiliary vector

Explains the AT_* information the kernel ELF loader places near argv/environment for the dynamic linker and program, and shows LD_SHOW_AUXV=1.

https://man7.org/linux/man-pages/man3/getauxval.3.html

PUBLIC MANUALpmap(1) — report a process memory map

Simple no-login process memory-map tool backed by /proc data; useful when raw maps/smaps are too dense initially.

https://man7.org/linux/man-pages/man1/pmap.1.html

A call into libc can be patched at runtime: ELF relocations, GOT, PLT and the dynamic linker

When an executable calls a function defined in a shared object, the final virtual address may not be known at static-link time. ELF therefore records dynamic relocations and symbol/dependency metadata for the runtime loader. On x86-64, one common/canonical model routes external function calls through a Procedure Linkage Table (PLT) stub and a Global Offset Table (GOT/GOTPLT) slot.

BUILD TIME main.c: printf("x=%d\n", x); ↓ compiler main.o: unresolved reference to symbol 'printf' ↓ static linker (ld, normally driven by compiler driver) ELF executable: PT_INTERP → /lib64/ld-linux-x86-64.so.2 (example path) DT_NEEDED → libc.so.6 .dynsym/.dynstr → dynamic symbols/strings relocation entries → locations loader must fix PLT/GOT-related entries for external calls EXEC TIME kernel maps executable + interpreter ↓ ld-linux maps DT_NEEDED shared objects ↓ applies required RELA/REL/RELR relocations resolves symbols according to ELF lookup/interposition rules applies RELRO protections when relocation phase permits ↓ transfers control through startup code toward main() CANONICAL LAZY FUNCTION BINDING (x86-64 style concept) first call printf@PLT ↓ PLT stub jumps through printf's GOTPLT slot ↓ initial slot routes into resolver path dynamic linker receives relocation index/symbol ↓ search loaded ELF objects for 'printf' ↓ write resolved libc printf address into GOTPLT slot ↓ tail/jump to resolved function second call printf@PLT ↓ GOTPLT slot already contains final address ↓ jump directly to libc printf EAGER BINDING LD_BIND_NOW=1 or link with -Wl,-z,now ↓ resolve applicable function relocations during startup ↓ no first-call lazy resolver step

ELF/runtime object	Role
PT_INTERP	Names the userspace dynamic linker/interpreter for a dynamically linked executable.
DT_NEEDED	Records shared-library dependencies that the dynamic linker must locate/load.
.dynsym / .dynstr	Dynamic symbol table and names used for runtime resolution.
relocation	Instruction/data describing how a location must be adjusted once symbol/base addresses are known.
GOT	Writable/relocatable table of addresses/data used by position-independent code and dynamic linking.
GOTPLT	GOT region/slots associated with PLT-mediated function calls in common ELF implementations.
PLT	Code stubs that route external calls through resolved or resolver-mediated addresses.
R_X86_64_JUMP_SLOT	x86-64 dynamic relocation traditionally associated with PLT/GOT function binding.
R_X86_64_GLOB_DAT	x86-64 relocation used to place a resolved symbol address into a data/GOT location.
RELRO	Marks selected relocated data read-only after relocation to reduce runtime overwrite attack surface.
BIND_NOW / -z now	Requests eager runtime symbol resolution rather than lazy first-call binding where applicable.
symbol interposition	ELF lookup behavior allowing a symbol in one loaded object to override/interpose on references from others under applicable rules.
IFUNC	GNU ELF indirect function whose final address is chosen by a resolver, often based on CPU/runtime properties.

The PLT/GOT sequence is implementation- and optimization-dependent. Modern toolchains may use eager binding, direct GOT-indirect calls (-fno-plt), linker relaxation, IFUNC resolution or architecture-specific PLT forms. What remains invariant is the need to bind references whose final address depends on the process's runtime load map.

NO-ACCOUNT ELF LINKING LAB cat > hello.c <<'EOF' #include <stdio.h> int main(void) { puts("hello"); return 0; } EOF gcc -O0 -fno-builtin -o hello hello.c readelf -l hello | grep -A2 INTERP readelf -d hello | grep -E 'NEEDED|BIND_NOW|FLAGS' readelf -rW hello readelf -sW hello | grep -E 'puts|UND' objdump -d -M intel hello | grep -A8 -E '<puts@plt>|<main>' # watch loader activity LD_DEBUG=libs,reloc,bindings ./hello 2>&1 | less # compare eager binding LD_BIND_NOW=1 LD_DEBUG=bindings ./hello 2>&1 | less # compare compiler/linker choices gcc -O2 -fno-plt -o hello-noplt hello.c gcc -O2 -Wl,-z,now,-z,relro -o hello-now hello.c readelf -rW hello-noplt objdump -d -M intel hello-noplt | less # Exact relocation names/stubs vary by architecture and toolchain.

CURRENT GLIBC DOCSGNU C Library — Dynamic Linker Hardening

Current glibc guidance recommending eager binding for hardened programs, avoiding text relocations and making DT_NEEDED dependencies explicit.

https://www.sourceware.org/glibc/manual/latest/html_node/Dynamic-Linker-Hardening.html

ABI SOURCEx86-64 psABI — GOT/PLT linker optimization source

Official ABI source showing GOTPLT slots, R_X86_64_JUMP_SLOT/GLOB_DAT relocations and alternative optimized PLT/GOT arrangements.

https://gitlab.com/x86-psABIs/x86-64-ABI/-/blob/master/x86-64-ABI/linker-optimization.tex

PUBLIC MANUALdladdr(3) — PLT/GOT example behavior

Current manual gives a concrete x86/x86-64 example of a function pointer referring to a PLT entry and position-independent code loading final symbol addresses through the GOT.

https://man7.org/linux/man-pages/man3/dladdr.3.html

main() is not the ELF entry point: _start, libc initialization, constructors, main and exit

For a normal glibc-linked Linux C program, the kernel does not call main(). After execve(), control reaches the ELF entry address. In a conventional executable that entry is startup code from crt1.o, commonly named _start. That code interprets the initial stack/register state and calls __libc_start_main(), which performs libc/program initialization, runs constructors and finally invokes main().

execve("./hello", argv, envp) ↓ kernel ELF loader maps PT_LOAD segments maps PT_INTERP dynamic linker if present builds initial user stack: argc argv[0..argc-1], NULL envp[...], NULL ELF auxiliary vector (AT_PHDR, AT_PAGESZ, AT_RANDOM, ...) ↓ dynamic linker maps dependencies / relocates / establishes runtime state ↓ jump to executable ELF e_entry glibc crt1.o: _start clear outer frame pointer as ABI convention extract argc / argv from initial stack preserve rtld_fini callback supplied by dynamic linker align stack for ABI pass pointer to main ↓ __libc_start_main(main, argc, argv, ..., rtld_fini, stack_end) ↓ establish libc process state environment / auxiliary vector already available TLS / thread-control state established by loader/runtime as required stack/pointer guards and libc internals initialized ↓ RUN INITIALIZERS DT_PREINIT_ARRAY handled by dynamic-loader init path for dynamic executables legacy DT_INIT if present DT_INIT_ARRAY / .init_array constructors in order ↓ call main(argc, argv, environ) main returns int status ↓ __libc_start_call_main / exit path ↓ atexit()/destructor handlers and stdio/libc cleanup as specified by exit() dynamic-linker finalization callbacks / .fini_array handling as applicable ↓ _exit / exit_group syscall ↓ kernel destroys thread/process resources and reports exit status to parent If user code calls _exit() directly, normal libc atexit/stdio cleanup is bypassed.

Startup object/symbol	Role
ELF e_entry	Virtual entry address where loader/runtime transfers control; not intrinsically the address of main().
crt1.o	C runtime startup object linked into ordinary executables and normally providing _start.
_start	Architecture-specific assembly startup entry that turns ABI process-entry state into a call to libc startup.
__libc_start_main	glibc startup routine coordinating process/libc initialization, constructors, main() and termination.
argc/argv/envp	Argument/environment vectors established on the initial process stack by the kernel/loader ABI.
auxv	ELF auxiliary vector supplying page size, program headers, randomness, hardware/platform information and loader-related values.
.preinit_array	Functions intended to run before normal dynamic-object initialization for the main executable.
.init_array	Ordered array of constructor function pointers run before main().
.fini_array	Destructor function pointers used during normal process termination/unloading.
constructor attribute	Compiler mechanism commonly placing a function reference into initialization sections such as .init_array.
atexit()	Registers callbacks for normal exit() processing.
exit()	libc process termination path running normal cleanup/atexit handlers before entering the kernel.
_exit()/exit_group	Kernel-facing termination without normal stdio/atexit processing; exit_group terminates the entire thread group.

Startup is ABI- and libc-specific. The exact register protocol, crt objects and initialization sequence differ across architectures, static versus dynamic linking, musl versus glibc, PIE versus non-PIE, and custom freestanding programs. The worked path here is the conventional glibc/Linux model, not a universal law of C.

ELF STARTUP LAB cat > start.c <<'EOF' #include <stdio.h> __attribute__((constructor)) static void ctor(void) { puts("constructor"); } __attribute__((destructor)) static void dtor(void) { puts("destructor"); } int main(int argc, char **argv) { puts("main"); return 7; } EOF gcc -O0 -o start start.c # ELF entry point readelf -h start | grep 'Entry point' # startup / constructor sections readelf -SW start | grep -E 'init|fini|interp|dynamic' readelf -x .init_array start 2>/dev/null objdump -d -M intel start | grep -A25 '<_start>' # dynamic-loader initialization trace LD_DEBUG=libs,reloc ./start 2>&1 | less # syscall end of life strace -e execve,exit_group ./start echo $? # Compare readelf's entry address with nm/objdump addresses of _start and main.

SOURCE / FILEglibc x86-64 _start source

Current generated glibc source shows x86-64 _start receiving argc/argv on the initial stack, preserving rtld_fini and preparing arguments for __libc_start_main.

https://codebrowser.dev/glibc/glibc/sysdeps/x86_64/start.S.html

SOURCE / FILEglibc libc-start.c

Current glibc startup implementation. It locates/runs DT_INIT and DT_INIT_ARRAY constructors for dynamic executables, initializes runtime state and ultimately calls main.

https://codebrowser.dev/glibc/glibc/csu/libc-start.c.html

PUBLIC MANUALexit(3)

Normal libc termination behavior including atexit/on_exit handlers and stdio flushing before kernel termination.

https://man7.org/linux/man-pages/man3/exit.3.html

PUBLIC MANUAL_exit(2)

Kernel-level termination path contrasting with exit(): it does not run atexit handlers or flush stdio streams.

https://man7.org/linux/man-pages/man2/_exit.2.html

A process can be dead but still have a PID: exit → zombie → wait → final reap

Process termination has two distinct phases. The exiting task first tears down nearly all of the expensive execution state that made it runnable—its address space, file references, timers and other resources as appropriate—then the kernel keeps a small amount of parent-visible termination state. That retained record is the zombie. It exists so the parent can later collect the child's exit status and accounting information with wait(), waitpid() or waitid().

CHILD PROCESS RUNNING ↓ return from main(), exit(), _exit()/exit_group(), fatal signal, ... termination path ↓ stop being runnable release address-space / thread-group resources as appropriate close/release file references and other process-owned kernel objects record termination status + accounting information ↓ notify parent (normally SIGCHLD) ↓ EXIT_ZOMBIE no userspace instructions execute ordinary address space is gone minimal task/PID + exit information retained ↓ parent calls waitpid()/waitid()/wait() ↓ kernel copies status/accounting result to parent ↓ REAP remaining task/PID bookkeeping can be released IF PARENT EXITS FIRST living children are reparented ↓ nearest configured child subreaper, otherwise the namespace's reaper/init role ↓ that process becomes responsible for later child-exit collection

State / mechanism	What it means
exit status	Small termination result retained for the parent; normal exits carry a status value while signal termination is reported through wait-status semantics.
`SIGCHLD`	Normal parent notification that a child changed relevant state. Signal delivery and collecting the child with wait are related but separate mechanisms.
zombie	Terminated child that has not yet been reaped. It is not consuming CPU and is not a sleeping executable process; it is retained kernel bookkeeping.
`waitpid()`	Selects a child/process group according to the PID argument and can block until an eligible child changes state.
`waitid()`	More explicit interface selecting which child state changes to report and returning information through `siginfo_t`.
reaping	Parent consumes the child's retained termination state, permitting final process-table/PID-related cleanup.
orphan	A still-running child whose parent terminated. Linux reparents it to an appropriate reaper rather than leaving it parentless.
subreaper	Process marked with `PR_SET_CHILD_SUBREAPER` that can adopt orphaned descendants before they reach the normal init/reaper process.

Zombie does not mean “stuck running process.” A zombie has already terminated. The problem is that its parent has not yet consumed the retained termination record. This is why killing a zombie itself does not solve the underlying issue; the parent/reaper relationship is the relevant part.

PROCESS-LIFECYCLE LAB # Show process state; Z denotes zombie in /proc/PID/stat and ps-style views ps -eo pid,ppid,state,stat,comm | head -30 # Inspect one process's parent/state fields cat /proc/<PID>/stat cat /proc/<PID>/status | grep -E '^(Name|State|Pid|PPid):' # Trace creation and collection in a small program strace -f -e clone,fork,vfork,execve,exit_group,wait4,waitid ./program # In C, compare waitpid(child, &status, 0) with WNOHANG and waitid().

SOURCE / FILELinux kernel exit.c — do_exit(), exit_notify() and zombie reaping

Current kernel source showing the real teardown/notification/reparenting path, including transition through EXIT_ZOMBIE and final task release.

https://github.com/torvalds/linux/blob/master/kernel/exit.c

PUBLIC MANUALPR_SET_CHILD_SUBREAPER(2const)

Current Linux manual for descendant reparenting to a designated subreaper, the mechanism used by service/session managers that need to collect descendants.

https://man7.org/linux/man-pages/man2/PR_SET_CHILD_SUBREAPER.2const.html

PUBLIC MANUAL/proc/PID/stat — observable process state

Documents the process-state field used by tools such as ps, including Z for zombie.

https://man7.org/linux/man-pages/man5/proc_pid_stat.5.html

A PID is a reusable number; a pidfd is a stable reference to one specific task

Traditional Unix process APIs identify a target with an integer PID. That works, but PIDs are eventually recycled. If software observes PID 4242, waits too long, and then calls kill(4242, ...), the original process may already be gone and an unrelated process may now own the same number. Linux PID file descriptors solve that identity race by turning a process reference into an ordinary file descriptor whose lifetime is tied to the referenced task rather than to a reusable numeric name.

TRADITIONAL PID-ONLY CONTROL observe child/service PID = 4242 ↓ time passes original 4242 exits and is reaped ↓ kernel may later reuse PID 4242 ↓ kill(4242, SIGTERM) risks targeting a different process PIDFD MODEL pidfd = pidfd_open(4242, 0) ↓ file descriptor refers to that particular task ├── poll/epoll: becomes readable when target exits/becomes zombie ├── pidfd_send_signal(pidfd, SIGTERM, ...) ├── waitid(P_PIDFD, pidfd, ...) when relationship/permissions allow └── pidfd_getfd(pidfd, targetfd, ...) under ptrace-style permission checks after target is gone, the pidfd does NOT silently retarget a later process that happens to receive the same numeric PID.

Mechanism	Why it matters
`pidfd_open()`	Creates a close-on-exec file descriptor referring to an already existing task/process.
`CLONE_PIDFD`	Allows process creation and stable process-handle acquisition to happen together, avoiding a creation-to-open race.
`poll()/epoll()`	A pidfd can participate in the same readiness loop as sockets, pipes, eventfds and other file-descriptor event sources.
`pidfd_send_signal()`	Sends a signal through the stable reference, avoiding the classic PID-reuse race of a later `kill(pid,...)`.
`pidfd_getfd()`	With sufficient permission, duplicates one of the target process's file descriptors into the caller; the result refers to the same open file description.
numeric PID	Still useful as a namespace-visible name and for human/admin interfaces, but it is not a permanently unique object identity.

This is the same design idea seen elsewhere in Unix: resolve a reusable name once, then operate through a stable kernel object reference. A pathname becomes an open file descriptor; a PID can become a pidfd. The descriptor survives renaming/recycling of the external name without changing which object it denotes.

CURRENT PUBLIC MANUALpidfd_open(2) — obtain a file descriptor for a task

Current Linux man-page description of pidfd creation, pollability, thread-vs-process behavior and use with waitid().

https://man7.org/linux/man-pages/man2/pidfd_open.2.html

CURRENT PUBLIC MANUALpidfd_send_signal(2) — signal a process through its pidfd

Explains the process-identity advantage directly: a stable pidfd avoids accidentally signaling a new process after PID reuse.

https://www.man7.org/linux/man-pages/man2/pidfd_send_signal.2.html

CURRENT PUBLIC MANUALpidfd_getfd(2) — duplicate a target process's descriptor

Shows how a pidfd can name another process for a permission-checked operation that duplicates one of its open descriptors into the caller.

https://man7.org/linux/man-pages/man2/pidfd_getfd.2.html

A debugger temporarily becomes part of another thread's execution control loop: ptrace stop → inspect/change state → resume

ptrace() is the Linux process-control primitive behind breakpoint debuggers and classic syscall tracers. The key mental model is not “read another process whenever you want”; it is tracer/tracee synchronization around explicit stops. A tracer attaches to one thread, waits for a ptrace stop, examines or changes registers/memory/state, and then explicitly resumes that thread. In a multithreaded program, tracing is thread-specific even though several tracees may belong to one thread group.

DEBUGGER / TRACER TRACEE THREAD attach with PTRACE_SEIZE/PTRACE_ATTACH (or child uses PTRACE_TRACEME) ↓ tracee enters a ptrace-stop ← signal / breakpoint / syscall-stop / event ↓ waitpid()/waitid() reports stop to tracer ↓ PTRACE_GETREGSET / PEEK* / other inspection ↓ optional state change PTRACE_SETREGSET / POKE* / signal choice ↓ PTRACE_CONT → run normally until next stop PTRACE_SINGLESTEP → execute roughly one instruction then stop PTRACE_SYSCALL → stop around next system-call entry/exit ↓ repeat until detach or tracee exits strace-style view: userspace → syscall-entry stop → tracer observes nr/args → kernel syscall executes → syscall-exit stop → tracer observes return value → userspace continues

Mechanism	What it means
ptrace stop	The tracee is stopped in a state where the tracer can perform the documented ptrace operations; this is distinct from simply having a PID or pidfd.
`PTRACE_SEIZE` / attach	Establishes the tracer relationship. Exact stop behavior differs by request and options.
`PTRACE_GETREGSET`	Reads architecture register sets so a debugger can inspect PC/SP/general registers and other supported state.
`PTRACE_SINGLESTEP`	Uses architecture/kernel support to resume execution until a single-step trap/stop is produced.
`PTRACE_SYSCALL`	Requests stops around system calls, which is the basis of classic `strace`-style tracing.
`process_vm_readv/writev`	Separate bulk cross-process memory-transfer syscalls; they still use ptrace-style permission checks but are not themselves the execution-control protocol.
Yama / ptrace access checks	Credentials, dumpable state, capabilities, user namespaces and LSM policy can restrict who may inspect or attach to whom.

pidfd and ptrace solve different problems. A pidfd gives a race-resistant reference to a process and can support polling/signaling; ptrace gives a debugger controlled stops and access to execution state. A stable identity handle does not by itself grant debugging authority.

CURRENT PUBLIC MANUALptrace(2) — Linux process tracing

Current Linux manual for attach/seize, ptrace stops, register/memory operations, syscall tracing, signal injection, events and ptrace access-mode checks.

https://man7.org/linux/man-pages/man2/ptrace.2.html

KERNEL DOCSYama LSM

Current documentation for extra DAC hardening such as ptrace_scope to limit same-UID process inspection/credential-stealing paths.

https://docs.kernel.org/admin-guide/LSM/Yama.html

CURRENT PUBLIC MANUALprocess_vm_readv(2) / process_vm_writev(2)

Direct cross-process memory transfer using local and remote iovec arrays. Useful for separating bulk memory access from ptrace's stop/control machinery; permission is still governed by ptrace-style access checks.

https://man7.org/linux/man-pages/man2/process_vm_readv.2.html

A Linux process can be frozen, serialized and reconstructed later: CRIU checkpoint/restore

A runnable process is more than its anonymous RAM. To restart it faithfully, checkpoint/restore must preserve a graph of state: process and thread relationships, registers, VMAs and page contents, file descriptors, pipes, sockets, namespaces, credentials, timers, signals and references to external files or devices. CRIU (Checkpoint/Restore In Userspace) builds most of that image from ordinary Linux kernel interfaces such as /proc, ptrace, socket diagnostics and memory-management APIs, then recreates equivalent kernel objects before letting restored threads run again.

CHECKPOINT running process tree ↓ walk tasks / threads and freeze them ↓ collect kernel-visible state ├─ registers / thread state ├─ VMA layout + mapped files ├─ anonymous/private memory pages ├─ fd table, pipes, eventfds, terminals ├─ sockets + namespace relationships └─ credentials, timers, signals, misc. metadata ↓ write checkpoint image files ↓ optionally copy images/filesystem state elsewhere RESTORE read checkpoint metadata ↓ recreate namespaces + shared resources ↓ fork/clone process tree and threads ↓ recreate exact mappings + refill memory ↓ reopen/reconnect file descriptors and supported sockets ↓ restore credentials, timers, registers and signal state ↓ resume execution from the saved point

State	Why restore is difficult
virtual memory	Mappings must return at compatible virtual addresses with the right sharing/COW relationships and page contents.
file descriptors	An fd is a reference to a kernel object, not just a pathname; pipes, epoll sets, eventfds and deleted files need object-specific handling.
TCP / Unix sockets	Connection state can extend beyond one process or host, so checkpointing may need kernel socket diagnostics, peer coordination or explicit external-resource handling.
namespaces / cgroups	The restored tree may depend on a particular view of mounts, PIDs, users, networking and resource-control hierarchy.
external resources	Hardware devices, remote peers and mutable filesystems can change while the process is frozen; CRIU cannot manufacture consistency outside the state it controls.
incremental/live migration	Soft-dirty/page-tracking techniques can reduce downtime, but a usable migration also needs storage/network/environment coordination.

Checkpoint/restore is not a universal “save program” button. A checkpoint can encode only resources the kernel and CRIU know how to reconstruct or reconnect. Device state, remote services, filesystem mutations and application-visible time can impose constraints even when all process memory was captured perfectly.

PROJECT DOCSCRIU — project overview and current release documentation

Primary project entry point for Linux userspace checkpoint/restore, supported use cases, releases and documentation.

https://criu.org/Main_Page

PROJECT DOCSCRIU — checkpoint and restore design

Walks the process-tree freeze, /proc and ptrace state collection, checkpoint images, resource recreation, process-tree reconstruction and final restorer context.

https://criu.org/Checkpoint/Restore

PROJECT DOCSCRIU — memory dumping and restoring

Explains VM-area discovery, pagemap/soft-dirty information, page extraction and restoration of COW/shared-memory relationships.

https://www.criu.org/Memory_dumping_and_restoring

6. Machine code, ISA, assembly and the hardware/software boundary

PUBLIC NOTESBerkeley CS61C — RISC-V ISA introduction

Explains what an instruction-set architecture actually specifies: registers, instructions, machine-code bit encodings, memory access, and architectural behavior.

https://notes.cs61c.org/content/rv-intro/rv-isa/

PUBLIC NOTESBerkeley CS61C — machine-language instruction formats

Makes the jump from assembly mnemonics to actual instruction bits and fields.

https://notes.cs61c.org/content/rv-isa-formats-i/

PUBLIC NOTESBerkeley CS61C — loads and stores

Shows the processor-centric meaning of load/store and how addresses connect the register file to memory.

https://notes.cs61c.org/content/rv-data-transfer/

SPECRISC-V Unprivileged ISA specification

The actual specification. Dense by design. Read RV32I once the simplified explanations make sense; it shows what a real modern ISA contract looks like.

https://docs.riscv.org/reference/isa/unpriv/unpriv-index.html

INTERACTIVEEasy 6502 — interactive ebook + emulator

A friendly but technical introduction to 6502 assembly with an assembler, debugger, memory view, registers, flags, branches, addressing modes, stack, and a tiny graphics environment.

https://skilldrick.github.io/easy6502/

PUBLIC PROJECTBen Eater — build a 6502 computer

A practical bridge between CPU-chip architecture and an actual computer: CPU, ROM, RAM, address decoding, bus timing, I/O, LCD, assembly, interrupts, and debugging.

https://eater.net/6502

Computers meet continuous voltages through ADCs and DACs

The physical world is not naturally a sequence of integers. Temperature, microphone voltage, light level and sensor output vary continuously. An ADC samples and quantizes an analog input into finite digital codes; a DAC performs the reverse mapping from digital code to an analog voltage/current waveform.

ADC CONCEPT continuous analog voltage x(t) ↓ anti-alias low-pass filter sample/hold at discrete instants n·Ts ↓ quantize amplitude to one of 2^N codes ↓ N-bit digital sample stream x[n] for an ideal N-bit converter across full-scale range VFS: 1 LSB ≈ VFS / 2^N sampling theorem: Fs > 2 × highest input frequency is the classic minimum condition for a bandlimited signal, with real anti-alias-filter guard band required SAR ADC CORE IDEA sample/hold → comparator ↑ internal DAC ← successive-approximation register ↑ binary search each bit from MSB to LSB DAC CONCEPT digital samples/codes ↓ DAC / current-steering / R-2R / sigma-delta-style converter ↓ stepped/high-frequency-rich analog signal ↓ reconstruction low-pass filter / output amplifier continuous electrical output

Converter term	Meaning
sample rate	Number of analog measurements/output sample updates per second.
resolution	Number of digital bits/codes used to represent amplitude.
quantization error	Difference between continuous input value and nearest representable digital code.
aliasing	High-frequency content folds into lower apparent frequencies when sampled without sufficient rate/filtering.
anti-alias filter	Analog filter before ADC attenuating content that would alias into the sampled band.
reconstruction filter	Analog output filter after DAC attenuating sampling images/steps outside desired signal band.
reference voltage	Precision analog level defining conversion scale in many ADC/DAC architectures.
INL / DNL	Integral/differential nonlinearity: deviations of real converter transfer steps from ideal positions/widths.
ENOB	Effective Number Of Bits inferred from noise/distortion performance; typically lower than nominal bit width.
sample-and-hold	Circuit captures input voltage so converter decision logic sees a sufficiently stable value during conversion.

FREE LABAnalog Devices — Analog-to-Digital Conversion lab

Free lab-style explanation of sampling, quantization, Nyquist/aliasing and a SAR ADC's sample/hold, comparator, internal DAC and successive-approximation register.

https://www.analog.com/en/resources/analog-dialogue/studentzone/studentzone-february-2022.html

FREE LABAnalog Devices — SAR ADC hands-on activity

Build-oriented public explanation of a SAR converter as a binary search using a DAC and comparator.

https://wiki.analog.com/university/courses/alm1k/alm-signals-labs/alm-sar-adc-1

TECH ARTICLEAnalog Devices — sigma-delta ADC tutorial

Deep public tutorial on oversampling, quantization noise, noise shaping and digital decimation filtering.

https://www.analog.com/en/resources/technical-articles/2022/07/16/08/06/sigmadelta-adcs-tutorial.html

REFERENCE DESIGNAnalog Devices — real 16-bit R/2R DAC signal chain

Concrete DAC reference design with voltage reference, segmented R/2R CMOS DAC and output buffer; includes actual LSB-size reasoning.

https://www.analog.com/en/resources/reference-designs/circuits-from-the-lab/cn0348.html

TECH ARTICLEDigiKey — ADC/DAC architecture tutorial

Readable no-login survey covering SAR, sigma-delta and pipelined ADCs plus binary-weighted, string and R-2R DACs.

https://www.digikey.com/en/articles/adc-dac-tutorial

One audio sample end to end: microphone → ADC → DMA/RAM → program → DAC → speaker

Audio is an excellent complete-system example because it crosses almost every abstraction layer: analog physics, converter clocks, serial digital audio, DMA, RAM buffers, kernel drivers, application processing and then the entire path in reverse.

CAPTURE air-pressure wave ↓ microphone transducer → tiny analog voltage/current ↓ bias / preamplifier / anti-alias filter ↓ audio-codec ADC ↓ PCM sample words, e.g. 48 kHz × 24-bit × 2 channels I²S / TDM digital audio interface ├── BCLK / bit clock ├── LRCLK / word-select / frame clock └── serial DATA ↓ SoC audio peripheral / DAI ↓ DMA cyclic/ring PCM buffer in RAM ↓ ALSA / kernel sound driver ↓ application / mixer / DSP PLAYBACK application PCM samples ↓ ALSA buffer in RAM DMA → SoC DAI → I²S/TDM ↓ codec DAC ↓ reconstruction/output filtering headphone / line / class-D amplifier ↓ speaker coil moves cone ↓ air-pressure wave

Audio term	What it controls
PCM	Sequence of numeric amplitude samples; uncompressed digital audio representation.
sample rate	Samples/channel per second, e.g. 48 kHz.
sample format/bit depth	Integer/floating representation and nominal amplitude resolution, e.g. signed 16/24/32-bit PCM.
channel count	Independent streams such as left/right.
BCLK	Serial bit clock shifting digital audio bits.
LRCLK / WS	Frame/word-select timing identifying channel/sample boundaries in I²S-style links.
MCLK	Optional higher-frequency master/reference clock used by some codecs/converters/PLLs.
DAI	Digital Audio Interface between SoC and codec/DSP; can use I²S/TDM/PCM-style framing.
period	ALSA buffer subdivision at which hardware/software commonly gets a progress interrupt/callback.
XRUN	Overrun/underrun: capture buffer not consumed or playback buffer not refilled in time.
latency	Time from analog/input event to software/output effect, accumulated across converter, buffers, DMA, scheduler and processing.

I²S does not carry 'sound.' It carries framed binary sample words plus timing. The codec is the mixed-signal component that converts between those numbers and analog voltages; DMA merely moves the sample words between the serial interface and RAM without CPU copying every sample.

KERNEL DOCSLinux Sound Subsystem documentation

Current no-login kernel documentation index for ALSA, PCM, HD Audio, SoC audio, codecs, DAIs, DMA, clocking and power management.

https://docs.kernel.org/sound/index.html

KERNEL DOCSLinux ASoC layer

Shows the architecture of embedded audio: codec drivers, digital audio interface drivers, platform/DMA drivers, machine drivers, audio clocking and power graph.

https://docs.kernel.org/sound/soc/index.html

KERNEL DOCSLinux ASoC platform driver — audio DMA

Concrete kernel documentation that separates audio DMA, SoC DAI and DSP responsibilities, making the RAM↔peripheral sample flow explicit.

https://docs.kernel.org/sound/soc/platform.html

KERNEL DOCSLinux ASoC DAI documentation

Public implementation documentation for codec/DAI behavior, formats, clocks and stream configuration.

https://docs.kernel.org/sound/soc/codec.html

ALSA PCM is a producer/consumer ring: application pointer, hardware pointer, periods, DMA and XRUNs

The end-to-end audio section shows samples crossing ADC/DAC and DMA, but the kernel/userspace boundary has its own precise state machine. An ALSA PCM stream is usually a cyclic ring buffer shared logically between software and a DMA-capable audio device. The application advances an application pointer; the device/driver advances a hardware pointer. Their distance is the amount of playable or captured audio currently buffered.

PLAYBACK PCM application generates frames ↓ write()/snd_pcm_writei() OR mmap() ring area ALSA runtime ring buffer ├── appl_ptr: software has produced through here └── hw_ptr: device has consumed through here ↓ cyclic DMA descriptors / hardware DMA engine I²S/TDM/HD-audio interface ↓ codec / DAC speaker As DMA crosses a PERIOD boundary ↓ IRQ / position update / wakeup ↓ poll/select/async callback or blocked writer becomes runnable ↓ application refills more frames before hw_ptr catches appl_ptr If hw_ptr catches the producer on playback: UNDERRUN → XRUN → usually -EPIPE → prepare/recover stream CAPTURE reverses the ownership: ADC/device produces frames and advances hw_ptr application consumes frames and advances appl_ptr if software falls behind and unread data is overwritten: OVERRUN → XRUN

PCM mechanism	Role
ring/buffer size	Total number of PCM frames the runtime can hold. Larger buffers tolerate scheduling jitter but increase worst-case latency.
period size	Chunk of progress commonly associated with a hardware interrupt or application wakeup boundary.
`appl_ptr`	Logical position through which userspace has produced playback data or consumed capture data.
`hw_ptr`	Logical device progress position, derived from DMA/controller position and maintained by the driver/runtime.
available frames	Playback space available to fill, or capture frames available to consume, computed from the two positions and stream state.
start threshold	Amount of queued playback data that can trigger automatic stream start.
wakeup threshold	Software policy controlling how much availability should exist before userspace is woken.
mmap PCM	Userspace obtains direct access to ring-buffer areas and commits progress, avoiding an extra library/kernel copy when the hardware/runtime supports it.
XRUN	Playback underrun or capture overrun caused by producer/consumer progress failing to keep the ring in a valid range.
latency	Not just “buffer size”: also includes queued frames, hardware FIFO, converter/filter delay, scheduling, mixer/DSP stages and output path.

A period is not necessarily a second buffer. The hardware usually cycles through one ring while periods mark progress/wakeup granularity. Likewise, “zero-copy mmap” means the application can access the PCM area directly; it does not eliminate DMA or the physical transfer from RAM to the audio peripheral.

ALSA PCM INSPECTION LAB # Enumerate cards and PCM endpoints cat /proc/asound/cards 2>/dev/null cat /proc/asound/pcm 2>/dev/null # Runtime stream state while audio is active find /proc/asound -path '*/pcm*/sub*/status' -o -path '*/pcm*/sub*/hw_params' 2>/dev/null # Common command-line endpoints if alsa-utils is installed aplay -l 2>/dev/null arecord -l 2>/dev/null # A useful experiment: # choose two buffer/period configurations and compare wakeup rate, # scheduling tolerance and underrun risk under CPU load.

ALSA DOCSALSA-lib PCM digital-audio interface

The userspace PCM model in one place: ring buffers, application/hardware pointers, periods, read/write versus mmap transfer, states and XRUN recovery.

https://www.alsa-project.org/alsa-doc/alsa-lib/pcm.html

KERNEL DOCSWriting an ALSA driver — PCM runtime and DMA buffers

Kernel-side implementation guide covering DMA buffer fields, PCM callbacks, position reporting and how the driver connects ALSA runtime state to actual hardware.

https://docs.kernel.org/sound/kernel-api/writing-an-alsa-driver.html

KERNEL DOCSALSA PCM /proc files and XRUN debugging

Documents per-card PCM runtime/debug files, including stream information and optional XRUN diagnostics useful for observing a live system.

https://docs.kernel.org/sound/designs/procfile.html

Peripheral connections: UART, SPI and I²C

After shared CPU buses make sense, these three interfaces are useful examples of different ways chips exchange bits. UART has no shared clock; SPI has an explicit clock and chip-select; I²C shares two open-drain lines and therefore depends on pull-up resistors and wired-AND behavior.

PUBLIC DOCSMicrochip — UART, baud rate and framing

Plain technical explanation of asynchronous serial: TX/RX wires, baud rate, start bit, data bits, optional parity and stop bits. Useful for seeing how two devices communicate without sharing a clock line.

https://developerhelp.microchip.com/xwiki/bin/view/products/mcu-mpu/32bit-mcu/ap-exercises/lab7/step1/

TECH ARTICLEAnalog Devices — Introduction to SPI Interface

Clear explanation and diagrams for SCLK, chip-select, MOSI and MISO, including clock edges and simultaneous shifting/sampling.

https://www.analog.com/en/resources/analog-dialogue/articles/introduction-to-spi-interface.html

SPEC PDFNXP — I²C-bus specification and user manual (official download)

The actual I²C specification. Read the early electrical sections for SDA/SCL open-drain behavior, pull-up resistors, START/STOP conditions, acknowledgement, arbitration and timing.

https://www.nxp.com/webapp/Download?colCode=UM10204&location=null

One UART byte on a wire: start bit, LSB-first data, parity and stop

UART is asynchronous: there is no shared clock wire. Both endpoints agree on a nominal bit rate, and the receiver uses the falling edge of the start bit to align its internal sampling clock with the incoming frame.

EXAMPLE: 8N1, 115200 bit/s, byte 0x55 = 01010101b bit time = 1 / 115200 ≈ 8.68 µs UART TX idles HIGH time → idle START D0 D1 D2 D3 D4 D5 D6 D7 STOP idle logic 1 0 1 0 1 0 1 0 1 0 1 1 |<------ 1 bit each ≈ 8.68 µs ------->| 0x55 is sent LSB first: D0=1 D1=0 D2=1 D3=0 D4=1 D5=0 D6=1 D7=0 Receiver behavior (typical asynchronous UART): idle HIGH ↓ detect sustained/valid LOW = START internal oversampling clock aligns near middle of bit cells ↓ sample D0..D7 near their centers ↓ optional parity sample sample STOP = expected HIGH ↓ move completed byte from receive shift register into RX FIFO/register ↓ set RX-ready flag / optional interrupt FRAMING ERROR: expected stop bit is not HIGH at sample time. PARITY ERROR: parity bit does not match selected odd/even rule.

UART term	Meaning
baud / bit rate	Nominal serial symbol/bit timing; simple binary UART commonly uses one bit per symbol.
8N1	8 data bits, No parity, 1 stop bit.
start bit	Transition/LOW interval that tells an idle receiver a new frame is beginning.
stop bit	Expected idle/HIGH interval ending a frame and providing resynchronization margin.
oversampling	Receiver samples line several times per bit, commonly 8× or 16×, to locate bit centers and reject edge uncertainty.
framing error	Stop-bit timing/level was not valid for the configured frame.
parity	One extra error-detection bit making total number of 1s odd or even; detects some errors but does not correct them.
TX/RX crossing	Device A TX connects to device B RX; device B TX connects to device A RX, with compatible electrical levels/reference.
TTL/CMOS UART	Logic-level signaling around a device's I/O rail; not electrically the same as RS-232.
RS-232	Separate physical-layer standard using different/inverted and often bipolar voltage levels; requires a proper transceiver unless measurement hardware supports those levels.

A UART decoder can produce plausible garbage when the baud rate is wrong. First inspect the raw waveform: idle level, start-bit edge and the width of the narrowest stable bit cells. Then set data bits, parity, stop bits and inversion correctly.

OFFICIAL DOCSMicrochip USART — asynchronous receiver and oversampling

Real UART/USART hardware documentation: the asynchronous receiver oversamples RX at 8× or 16× baud, detects a start bit, then samples data, parity and stop bits.

https://onlinedocs.microchip.com/oxy/GUID-D4CAD149-CB72-498B-B8E7-7E8255593463-en-US-7/GUID-CA7913BD-9B00-4D71-9C17-DAB3F8536FCC.html

OFFICIAL DOCSMicrochip USART — data reception

Shows the concrete receive datapath: detect start bit, shift data into a receive shift register, recognize stop bit, then move complete frame to RX buffer and raise a receive-complete flag/interrupt.

https://onlinedocs.microchip.com/oxy/GUID-9175774B-9B45-45D7-A62D-28F07E411073-en-US-4/GUID-3EE2AF6F-9462-40D3-9B1C-2E98E71B123D.html

FREE TUTORIALSaleae — Learn Asynchronous Serial

No-login visual tutorial on bit rate, data length, stop bits, parity, bit order and inversion.

https://www.saleae.com/support/protocol-analyzers/learn-digital-protocols/learn-asynchronous-serial

FREE TOOL DOCSSaleae Async Serial Analyzer guide

Practical decoder guide for UART-style start/stop framing and measuring an unknown baud rate from the captured bit width.

https://www.saleae.com/support/protocol-analyzers/analyzer-user-guides/using-async-serial

SPI is two shift registers connected by wires: CS#, SCK, MOSI and MISO

SPI is synchronous and usually full-duplex. The controller selects one peripheral with chip-select, supplies the serial clock, shifts outgoing bits on MOSI and simultaneously samples incoming bits on MISO. There is no universal packet format above those wires; the peripheral's datasheet defines commands, address fields and transaction boundaries.

BASIC 4-WIRE SPI controller peripheral CS# --------------------------→ chip select SCK --------------------------→ serial clock MOSI --------------------------→ serial data into peripheral MISO ←-------------------------- serial data from peripheral CONTROLLER TX SHIFT REG PERIPHERAL RX SHIFT REG [7 6 5 4 3 2 1 0] --MOSI----→ [7 6 5 4 3 2 1 0] CONTROLLER RX SHIFT REG PERIPHERAL TX SHIFT REG [7 6 5 4 3 2 1 0] ←---MISO--- [7 6 5 4 3 2 1 0] Each clock edge pair shifts both directions at once. SPI MODE = CPOL + CPHA CPOL=0: clock idles LOW CPOL=1: clock idles HIGH CPHA=0: receiver samples on FIRST edge after selection; transmitter changes data on SECOND/trailing edge CPHA=1: transmitter changes on FIRST edge; receiver samples on SECOND edge Modes: mode 0 = CPOL0 CPHA0 → sample rising, change falling mode 1 = CPOL0 CPHA1 → change rising, sample falling mode 2 = CPOL1 CPHA0 → sample falling, change rising mode 3 = CPOL1 CPHA1 → change falling, sample rising Example register read used by many SPI peripherals (DEVICE-SPECIFIC): CS# LOW MOSI: READ_OPCODE → ADDRESS → dummy byte(s) MISO: don't-care → don't-care → returned data CS# HIGH ends transaction Inactive peripherals must release a shared MISO line to high-Z.

SPI property	Consequence
synchronous clock	No baud-recovery/start bit; SCK tells receiver exactly when bit cells occur.
full duplex	A bit is shifted in each direction during the same clocks, even when one direction is dummy/ignored.
chip select	Defines which peripheral is active and commonly delimits a command transaction.
CPOL	Defines idle SCK polarity.
CPHA	Defines whether data is sampled on first or second clock transition after selection.
MISO tri-state	Unselected devices must release a shared return line to avoid bus contention.
no universal addressing	Unlike I²C, standard SPI itself does not encode a shared-bus device address; selection is commonly one CS# per peripheral.
no mandatory ACK	Basic SPI has no protocol-level ACK bit; command/status semantics are device-specific.
dummy clocks/data	Controller may have to send meaningless bits merely to generate clocks that let peripheral shift return data.

OFFICIAL DOCSMicrochip SPI — Clock Formats / CPOL and CPHA

Official peripheral documentation defining all four clock formats: CPOL sets idle level; CPHA chooses first-edge versus second-edge sampling.

https://onlinedocs.microchip.com/oxy/GUID-A52628F4-6F6F-4C77-80CB-113A0C62DB75-en-US-10/GUID-1D652BF8-5B01-4AA9-87D2-0CB61426D53E.html

FREE TUTORIALSaleae — Learn SPI

No-login visual explanation of MOSI/MISO, tri-state MISO, CPOL/CPHA, bit order and word length.

https://www.saleae.com/support/protocol-analyzers/learn-digital-protocols/learn-spi

FREE TOOL DOCSSaleae SPI Analyzer guide

Practical decoder guide; especially useful for diagnosing wrong CPOL/CPHA settings from raw captured clock/data.

https://www.saleae.com/support/protocol-analyzers/analyzer-user-guides/using-spi

I²C is a shared open-drain bus: START, address, ACK/NACK, data and clock stretching

I²C uses two shared open-drain lines: SDA for data and SCL for clock. Devices actively pull a line LOW but release it for HIGH, so pull-up resistors create the HIGH state. This electrical rule is what makes shared ACK bits, clock stretching and multi-controller arbitration possible.

ELECTRICAL MODEL VDD | | Rp Rp | | SDA SCL | | +-- open-drain transistors from every attached device to GND nobody pulls LOW → resistor makes line HIGH any device pulls LOW → line is LOW START CONDITION SDA: HIGH → LOW while SCL remains HIGH 7-BIT ADDRESS + R/W controller shifts A6 A5 A4 A3 A2 A1 A0 R/W, MSB first data normally changes while SCL LOW receiver samples while SCL HIGH / on SCL rising edge in simple view 9th CLOCK = ACK/NACK transmitter RELEASES SDA receiver: pulls SDA LOW → ACK leaves SDA HIGH → NACK WRITE EXAMPLE TO 7-bit ADDRESS 0x50 START 0x50 + W(0) → ACK register/address byte 0x20 → ACK data byte 0xA5 → ACK STOP COMMON RANDOM-READ STYLE SEQUENCE (DEVICE-SPECIFIC ABOVE I²C) START 0x50 + W → ACK register index 0x20 → ACK REPEATED START 0x50 + R → ACK target sends data byte controller sends NACK to say 'done reading' STOP CLOCK STRETCH controller releases SCL HIGH target is not ready → target keeps SCL physically LOW controller must wait until actual line returns HIGH STOP CONDITION SDA: LOW → HIGH while SCL remains HIGH

I²C concept	Meaning
START	SDA falling while SCL high; marks beginning of bus transaction/control transfer.
repeated START	New START without prior STOP; keeps bus ownership while changing direction/address phase.
STOP	SDA rising while SCL high; releases current transaction/bus sequence.
7-bit address	Device address sent MSB-first before the R/W direction bit.
R/W bit	0 requests controller→target write direction; 1 requests target→controller read direction.
ACK	Receiver pulls SDA low during ninth clock to acknowledge a byte.
NACK	Receiver leaves SDA high during acknowledge clock; meaning depends on transaction stage.
clock stretching	Target/controller holds SCL low to delay next clock high period until ready.
open-drain	Devices only actively pull low and otherwise release line, allowing wired sharing.
pull-up resistor	Returns released SDA/SCL toward VDD; value interacts with bus capacitance and rise-time limits.
bus capacitance	Wiring, pins and probes slow the passive LOW→HIGH rise through pull-up resistance.
multi-controller arbitration	Controllers observe actual SDA while transmitting; a controller intending HIGH but reading LOW loses to another device pulling LOW.

I²C's rising edge is analog. HIGH is produced through pull-up resistance charging the bus capacitance, so the waveform is not an ideal vertical edge. A decoder failure can therefore be an electrical problem—weak pull-up, too much capacitance, noise, or probe loading—not merely a software/address mistake.

DIRECT SPEC PDFNXP UM10204 — I²C-bus specification and user manual Rev. 7.0 (direct PDF)

The defining public I²C reference: START/STOP, ACK/NACK, clock synchronization/stretching, arbitration, addressing and Standard/Fast/Fast-mode Plus/High-speed timing.

https://cache.nxp.com/docs/en/user-guide/UM10204.pdf

FREE TUTORIALSaleae — Learn I²C

No-login visual walkthrough of START, 7-bit address + direction, ninth-bit ACK/NACK, bytes, repeated START, STOP and clock stretching.

https://www.saleae.com/support/protocol-analyzers/learn-digital-protocols/learn-i2c

FREE TOOL DOCSSaleae I²C Analyzer guide

Practical decoder documentation for SDA/SCL, address display and diagnosing noise/glitches around SCL edges.

https://www.saleae.com/support/protocol-analyzers/analyzer-user-guides/using-i2c

OPEN-SOURCE DOCSsigrok I²C protocol decoder

Open-source decoder documentation with exact START/STOP rules, 7-bit addressing, ACK/NACK and example sigrok-cli decoding.

https://www.sigrok.org/wiki/Protocol_decoder%3AI2c

One concrete bus transaction: CPU reads a byte from memory

Exact signals differ between CPUs, but the following sequence is the core idea behind a conventional parallel memory bus.

The CPU places the target address on its address lines.
Address-decoder logic examines high address bits and asserts the chip-select / enable input for the RAM, ROM or peripheral that owns that address range.
The CPU asserts a READ control condition (the exact signal names may be RD, R/W, OE, AS, etc.).
The selected memory/device decodes the remaining address bits internally and drives the requested value onto the data bus.
After the required access time, the CPU samples the data bus into an internal register or pipeline latch.
The read/select signals are deasserted. The memory stops driving the data bus; tri-state or point-to-point interface rules prevent two devices from driving incompatible values at once.
The CPU can now use the value: feed it into the ALU, write it into another register, interpret it as an instruction, or pass it onward.

This is why address lines, data lines, chip select, output enable, write enable, clock/timing and electrical drive rules all matter together. A schematic becomes understandable when you trace those roles one signal at a time.

DIRECT PDFM68000 User's Manual — Motorola/NXP (direct PDF)

A particularly good real CPU manual for bus thinking. It documents address bus, data bus, asynchronous bus control, bus arbitration, interrupts, system control, clock, power, read/write cycles, reset and detailed timing diagrams.

https://www.nxp.com/docs/en/reference-manual/MC68000UM.pdf

DIRECT PDFZ80 Family CPU User Manual — Zilog (direct PDF)

Official 306-page manual. Architecture, signal pins, machine cycles, memory and I/O transactions, interrupts, refresh and timing are all exposed. Compare it with the 6502 and 68000 to see different bus philosophies.

https://www.zilog.com/docs/z80/z80cpu_um.pdf

Why old computers expose parallel buses while modern systems use serial links

The Apple-1, Z80 systems and ISA/PCI-era machines expose many separate address/data/control wires. Modern external interconnects increasingly serialize transactions onto high-speed differential lanes. This is not because parallel logic disappeared inside chips; it is because wide PCB buses become difficult to time, route and signal-integrity-manage at high edge rates.

Old shared parallel bus	Modern point-to-point serial link
Many address/data/control conductors in parallel	Few differential lane pairs running at very high symbol rates
Often electrically shared by several devices	Usually one transmitter/receiver pair per link; switches build a fabric
Bus ownership/arbitration is explicit	Packets/TLPs/frames carry transaction information
Timing skew across many wires becomes difficult	Clock recovery/encoding/equalization moves complexity into PHY circuitry
Easy to probe with many-channel logic analyzer at low speed	Usually requires specialized high-speed protocol/PHY tools
Examples: 6502 bus, Z80 bus, ISA, conventional PCI	Examples: PCIe, USB 3.x/USB4, SATA, Ethernet serial PHYs

Differential signalling, serializers/deserializers and clock-data recovery

At low speed, a single-ended CMOS wire can represent a bit relative to ground. High-speed links such as PCIe, high-rate Ethernet and many display links use differential pairs: the receiver responds mainly to the voltage difference between two complementary conductors. A serializer converts many internal parallel bits into a fast serial stream; a deserializer reconstructs parallel data. Clock-data-recovery circuitry extracts sampling timing from transitions in the received serial stream.

parallel internal data [b0 b1 b2 ... bN] ↓ encoding / scrambling / framing ↓ SERIALIZER + transmit PLL ↓ differential TX+ / TX− pair ↓ controlled-impedance PCB trace / connector / cable channel loss + reflections + crosstalk + jitter ↓ differential receiver / equalization ↓ CDR (clock and data recovery) ↓ DESERIALIZER ↓ decoded parallel internal data

Concept	Why it exists
differential pair	Receiver measures V+ − V−, giving good common-mode noise rejection and low emissions when the pair is tightly coupled.
termination	Matches the link's characteristic impedance so travelling waves are absorbed rather than strongly reflected.
serializer	Reduces many parallel wires to one/few high-speed lanes.
deserializer	Recovers groups of bits/words from the incoming serial stream.
CDR	Recovers a sampling clock/phase from received data transitions.
scrambler	Randomizes data patterns to improve spectral/transition properties without normally changing information content.
line code	Maps data into transmitted symbols with properties useful for synchronization, DC balance, control symbols or error detection.
equalization	Compensates frequency-dependent channel loss so the receiver eye opens sufficiently at high data rates.
eye diagram	Overlay of many received bit intervals showing timing/amplitude margin and signal integrity.

TECH ARTICLELVDS and M-LVDS implementation guide — Analog Devices

Free public technical article explaining differential voltage, common-mode voltage, current-mode drivers, 100-ohm termination, noise immunity and why paired traces reduce emissions.

https://www.analog.com/en/resources/app-notes/an-1177.html

TECH ARTICLEClock and Data Recovery fundamentals — Analog Devices

Excellent explanation of parallel-to-serial conversion, line encoding, receiver deserialization and PLL-based clock-data recovery.

https://www.analog.com/en/resources/technical-articles/hfta070-precision-reference-clock-usage-in-clock-and-data-recovery-circuits.html

DIRECT PDFFPD-Link II SerDes overview — TI (direct PDF)

Concrete serializer/deserializer example: a wide parallel video bus plus clock becomes a high-speed serial differential signal; receiver recovers clock/data and recreates the parallel interface.

https://www.ti.com/lit/an/snla102b/snla102b.pdf

TECH ARTICLEAnalog Devices — LVDS basics and signal distribution

Plain-web overview of LVDS as low-swing differential signalling for clocks/data across cables, boards and backplanes.

https://www.analog.com/en/resources/technical-articles/highspeed-signal-distribution-using-lowvoltage-differential-signaling-lvds.html

How several chips can share wires: push-pull, tri-state and open-drain outputs

A logic output is not just 'a voltage.' The output stage determines whether the pin can actively drive high, actively drive low, or disconnect itself. This matters enormously when several chips share a bus.

Output type	Can drive HIGH?	Can drive LOW?	High-Z?	Typical use
push-pull / totem-pole	Yes	Yes	Usually no	Ordinary point-to-point CMOS/TTL outputs; fast strong edges.
tri-state	Yes	Yes	Yes	Classic shared parallel data buses: only the selected driver is enabled.
open-drain / open-collector	No — external pull-up raises line	Yes	Effectively yes when released	I²C, wired-AND/OR signaling, shared interrupt/status lines, level shifting.

TRI-STATE SHARED BUS CPU output driver ─┐ RAM output driver ─┼── D0..D7 shared wires ─→ receivers ROM output driver ─┘ Rule: exactly one active driver at a time; all others = high impedance Z. OPEN-DRAIN SHARED LINE VDD │ [pull-up resistor] │ ├──────────── SDA/SCL / IRQ-style shared wire │ │ │ FET to GND FET to GND FET to GND device A device B device C Any device can pull LOW; nobody actively forces HIGH.

DIRECT PDFUnderstanding the I²C bus — open-drain explained electrically (direct PDF)

Excellent direct PDF. It explicitly shows an I²C output as a pull-down FET plus input buffer: devices can pull the line low or release it, while a resistor pulls it high.

https://www.ti.com/lit/an/slva704/slva704.pdf

DIRECT PDFOpen-drain pull-up/pull-down design — TI application report

Discusses why open-drain outputs need external bias resistors and how resistor/current/voltage constraints are chosen.

https://www.ti.com/lit/pdf/sbva045

Beyond the CPU: how memory and devices connect to the machine

A tiny historical computer may literally share address, data and control buses among CPU, RAM and peripherals. Modern machines are more complicated: memory controllers, point-to-point links, bridges, PCIe, DMA engines, interrupt controllers and IOMMUs move transactions around. The underlying questions are still the same: who owns an address, who moves the data, who is allowed to write, and how does the CPU learn that a device needs attention?

PLAIN NOTESMemory-mapped I/O — University of Maryland

Very direct explanation of the classic CPU/RAM/peripheral connection. It walks through address bus, data bus, read/write control, chip enable and address decoding, then shows how an I/O device can occupy addresses just like memory.

https://www.cs.umd.edu/class/sum2003/cmsc311/Notes/IO/mapped.html

PUBLIC NOTESInput/Output notes — Cornell CS3410

Public notes on programmed I/O and memory-mapped I/O. Useful for connecting device registers in hardware to the load/store operations software performs.

https://www.cs.cornell.edu/courses/cs3410/2025fa/notes/io.html

KERNEL DOCSLinux Dynamic DMA Mapping Guide

Advanced but unusually concrete description of how a real device accesses system RAM without the CPU copying every byte. The opening diagrams distinguish CPU virtual addresses, CPU physical addresses, bus addresses, MMIO, host bridges and IOMMU translation.

https://docs.kernel.org/core-api/dma-api-howto.html

KERNEL DOCSLinux PCI driver guide

Real documentation for how PCI/PCIe devices appear to an operating system: BAR address regions, MMIO resources, DMA, interrupts, device discovery and driver setup. Read this after basic buses and MMIO.

https://docs.kernel.org/PCI/pci.html

KERNEL DOCSLinux MSI/MSI-X interrupt guide

Shows a modern form of device interrupt: instead of only toggling a dedicated interrupt wire, a PCI device can perform a special write transaction that causes the CPU to receive an interrupt vector.

https://docs.kernel.org/PCI/msi-howto.html

DIRECT PDFRaspberry Pi RP2040 datasheet

A large but excellent real chip manual. Its block diagrams and chapters expose CPUs, SRAM banks, bus fabric, DMA, clocks, resets, interrupt controller, GPIO, UART, SPI, I2C, USB and programmable I/O in one complete system-on-chip.

https://datasheets.raspberrypi.com/rp2040/rp2040-datasheet.pdf

SPECRISC-V system architecture specifications

For the layer around a modern CPU core: interrupt architecture, IOMMU, platform-level interrupt controller and SoC integration. Dense specification material, but useful once simple memory-mapped I/O makes sense.

https://docs.riscv.org/reference/hardware-overview/index.html

SPECRISC-V Platform-Level Interrupt Controller (PLIC)

A concrete specification for how many external interrupt sources are prioritized, routed and presented to processor contexts. Good example of a component that sits outside the CPU execution datapath but is essential to a complete system.

https://docs.riscv.org/reference/plic/index.html

Reading a register map correctly: RW, W1C, read-to-clear, reserved bits and side effects

A memory-mapped register is not ordinary RAM just because software accesses it with an address. Reads and writes can have side effects, individual bits can have different access policies, hardware can modify them concurrently, and reserved bits may need specific write values. Driver code must follow the register specification exactly.

EXAMPLE DEVICE REGISTER BLOCK BASE + 0x00 CTRL bit 0 ENABLE RW bit 1 RESET WO / self-clearing trigger bits 31:2 RESERVED BASE + 0x04 STATUS bit 0 READY RO (hardware updates) bit 1 ERROR RW1C / W1C bit 2 IRQ_PEND RW1C / W1C BASE + 0x08 RXDATA bits 7:0 DATA RC / read pops FIFO entry BASE + 0x0C IRQ_ENABLE bits 2:0 RW CLEAR ERROR WITHOUT DAMAGING OTHER STATUS BITS writel(BIT(1), STATUS); # writing 1 clears ERROR DANGEROUS GENERIC READ-MODIFY-WRITE ON W1C REGISTER tmp = readl(STATUS); # suppose READY=1, ERROR=1, IRQ_PEND=1 tmp |= BIT(1); writel(tmp, STATUS); # writes 1 back to every currently-set W1C bit # may accidentally clear IRQ_PEND too READ-TO-CLEAR / FIFO REGISTER x = readl(RXDATA); # the READ itself changes device state debugger/extra probe may consume data unintentionally RESERVED BITS never assume writing arbitrary 1s is harmless use documented masks / reset values / prescribed access widths

Register access type	Meaning
RO	Software reads value; writes are ignored/illegal/undefined according to device spec.
RW	Normal read/write storage or control field.
WO	Write-only command/data register; read value may be zero, undefined or otherwise device-specific.
W1C / RW1C	Writing a 1 clears selected bit; writing 0 leaves it unchanged. Common for latched interrupt/error status.
W1S / RW1S	Writing a 1 sets selected bit; useful for atomic set operations.
RW0C	Writing 0 clears; writing 1 preserves.
RC	Read returns state and clears/consumes it as a side effect.
self-clearing bit	Software writes a command bit; hardware automatically returns it to zero after accepting/completing operation.
reserved	Meaning not allocated to software; required read/write treatment is specification-defined and may matter for future compatibility.
REGWEN/write-lock	One-way or controlled bit gates future writes to protected configuration registers until reset/unlock policy allows changes.
hardware-set/software-clear	Hardware asynchronously/independently latches an event; software clears acknowledgment without losing simultaneous new events.

Read-modify-write is only safe when the register semantics permit it. W1C status, read-clear FIFOs, hardware-updated bits and write-only command fields can all make the ordinary RAM pattern tmp = *reg; tmp |= mask; *reg = tmp; incorrect.

PUBLIC DOCSOpenTitan reggen — register access semantics

Excellent no-login formal vocabulary for real hardware register fields: RO, RC, RW, R0W1C, RW1S, RW1C, RW0C, WO, hardware access and REGWEN write protection.

https://opentitan.org/book/util/reggen/index.html

PUBLIC REGISTERSOpenTitan PWM registers — concrete WO and write-lock fields

Real generated register map with reserved bits, write-only alert trigger and an rw0c REGWEN that permanently locks control writes until reset.

https://opentitan.org/book/hw/ip/pwm/doc/registers.html

OFFICIAL SPECRISC-V IOMMU — memory-mapped register interface

Current official example specifying alignment/access-width rules and noting that an 8-byte MMIO access may internally become two 4-byte transactions.

https://docs.riscv.org/reference/iommu/v20260222/iommu_registers.html

KERNEL DOCSLinux — bus-independent device I/O accessors

Use readl()/writel()/ioread*()/iowrite*() rather than treating MMIO as ordinary C memory; the API encodes architecture/bus ordering requirements.

https://docs.kernel.org/driver-api/device-io.html

OFFICIAL SPECRISC-V — memory/I/O FENCE ordering

Official architecture explanation of FENCE ordering among memory reads/writes and device input/output operations.

https://docs.riscv.org/reference/isa/unpriv/rv32.html

A writel() may retire before the device sees it: posted MMIO writes, barriers and read-back flushing

Memory-mapped I/O looks syntactically like memory access, but PCIe and many interconnects allow posted writes: the CPU/host bridge can accept the write and continue before the transaction has physically reached the device. Drivers therefore need to reason separately about CPU memory ordering, MMIO accessor ordering and whether a posted write must be forced to complete before a later operation.

EXAMPLE: BUILD DMA DESCRIPTOR THEN RING DEVICE DOORBELL descriptor->addr = dma_addr; descriptor->len = 4096; descriptor->flags = OWNED_BY_DEVICE; ↓ dma_wmb(); # publish descriptor memory before notification ↓ writel(new_tail, DOORBELL); # MMIO write What must be true: device must not observe doorbell and fetch stale descriptor contents POSTED WRITE PATH CPU executes writel(CMD_START, CONTROL) ↓ host bridge / PCIe root accepts request into posted-write buffer ↓ CPU may continue transaction still travelling through fabric ↓ device eventually receives register write CASE: DRIVER MUST KNOW WRITE REACHED DEVICE BEFORE CONTINUING writel(STOP, CONTROL); readl(STATUS); # safe register in same device ↓ PCI read is non-posted: completion cannot come back until prior ordering constraints force the relevant posted write through the path ↓ after read returns, STOP write is known to have reached device under documented pattern LOCKING DOES NOT AUTOMATICALLY FLUSH POSTED MMIO spin_lock() writel(...) spin_unlock() ↓ CPU mutual exclusion is satisfied, but posted device write may still be in bridge/fabric readl()/writel() vs *_relaxed() normal accessors include stronger architecture-specific ordering against memory/DMA relaxed accessors omit some expensive serialization and require driver proof/barriers ioremap_wc() DEVICE MEMORY writes may be combined/reordered much more aggressively ↓ excellent for framebuffers/streaming apertures, usually wrong for side-effecting control registers unless specifically designed/documented.

I/O ordering tool	What it solves
readl()/writel()	Portable ordered MMIO accessors with architecture-defined serialization and PCI-style endian behavior.
readl_relaxed()/writel_relaxed()	Cheaper MMIO accessors omitting some ordering against normal memory/DMA; caller supplies needed barriers.
dma_wmb()/dma_rmb()	Orders CPU accesses to DMA-coherent/shared descriptor/data memory relative to device ownership/protocol transitions.
wmb()/rmb()/mb()	CPU memory barriers; exact use with MMIO depends on accessor and architecture contract.
read-back flush	Non-posted read from a safe register on the same device can force prior posted writes to complete.
mmiowb()	Special ordering primitive for cases where MMIO writes under a lock must be ordered against another CPU's MMIO sequence on affected architectures.
ioremap_wc()	Write-combining mapping suitable for device memory apertures where merged/reordered streaming writes are allowed.
__raw_* I/O	Low-level access without normal barriers/byte-order guarantees; generally inappropriate for portable control-register access.
spinlock	Serializes CPUs accessing shared software state, but does not by itself guarantee a posted PCI/MMIO write has reached the device.

Read-back flushing must use a register that is safe to read. Do not use a read-to-clear FIFO/status register merely because it belongs to the same device. Linux's device-I/O documentation recommends a safe same-device read specifically to drain posted writes when completion must be known.

KERNEL DOCSLinux memory barriers

Deep reference for CPU/device/DMA ordering and the distinction between normal memory barriers, DMA barriers and I/O accessors.

https://docs.kernel.org/core-api/wrappers/memory-barriers.html

KERNEL DOCSLinux DMA API

Use beside MMIO ordering to distinguish descriptor-buffer ownership/cache visibility from the separate doorbell/register transaction ordering problem.

https://docs.kernel.org/core-api/dma-api.html

x86 memory types: why RAM, framebuffers and MMIO cannot all be cached the same way

Page tables do more than translate addresses and enforce read/write/execute permissions. On x86, the mapping also participates in selecting a memory type: ordinary RAM is normally write-back cached, while device register windows often need uncached-style semantics and large framebuffer-style apertures may benefit from write combining. Using the wrong type can be much worse than merely slow: speculative reads, merged writes or stale cache lines can violate a device's register protocol.

CPU executes load/store to virtual address ↓ page-table translation physical address + page cache-mode bits ↓ PAT (Page Attribute Table) selects candidate memory type ↓ MTRR / architectural combination rules may further constrain effective type ↓ EFFECTIVE MEMORY TYPE WB — write-back normal cacheable RAM loads/stores use coherent CPU caches writes may remain in cache before later writeback WT — write-through reads can cache; writes propagate toward memory as well WC — write-combining commonly useful for write-mostly apertures such as framebuffers adjacent writes may be gathered into larger bursts reads are not normal WB-cache hits; ordering is weaker UC / UC- — uncached-style used for side-effecting device registers / MMIO where ordinary caching is unsafe PCI BAR example BAR0 control registers → ioremap()/ioremap_uc() → UC/UC-like mapping BAR2 prefetchable framebuffer → ioremap_wc() → WC mapping normal DRAM → direct map / userspace page → WB SAME PHYSICAL RANGE, CONFLICTING VIRTUAL ALIASES ↓ WB alias + UC/WC alias can create incoherent/undefined behavior ↓ Linux PAT APIs track reserved memory types to prevent unsafe aliasing.

Type/API	What it means in practice
WB	Normal write-back cacheability for ordinary system RAM; best general-purpose CPU performance.
WC	Write-combining: permits merging/bursting of writes. Useful for write-heavy device apertures, not a replacement for normal RAM caching.
UC / UC-	Uncached or strongly restricted caching behavior for regions such as side-effecting MMIO registers.
PAT	Per-page x86 mechanism selecting memory attributes; more flexible than physical-range MTRRs.
MTRR	Older physical-range memory-type mechanism. Modern Linux drivers normally use PAT-aware mapping APIs instead of directly programming MTRRs.
ioremap()	Kernel maps device physical address space with architecture-appropriate I/O attributes.
ioremap_wc()	Requests a write-combining mapping for suitable device memory.

Do not collapse memory type into ordering. Cacheability (WB/WC/UC) and ordering barriers are related but distinct. A WC or UC mapping does not eliminate the need to obey the device's documented ordering rules, and normal RAM synchronization still needs the correct CPU/compiler memory-order primitives.

KERNEL DOCSLinux x86 PAT — Page Attribute Table

The most direct public reference for WB/WT/WC/UC memory types, ioremap variants, alias tracking and the interaction between PAT and MTRRs.

https://docs.kernel.org/arch/x86/pat.html

KERNEL DOCSLinux x86 MTRR documentation

Explains the older physical-range mechanism and, importantly, why modern drivers should prefer PAT-aware interfaces.

https://docs.kernel.org/arch/x86/mtrr.html

Concrete inspection: on a kernel built with PAT debug support, /sys/kernel/debug/x86/pat_memtype_list shows tracked physical ranges and their memory types. Treat debugfs as an inspection interface, not a configuration playground on a production machine.

Polling, interrupts and DMA solve different parts of device I/O

These mechanisms are frequently presented as alternatives even though real drivers often combine all three. Polling answers how software discovers device progress by repeatedly checking. Interrupts let hardware notify software asynchronously. DMA moves bulk data between device and memory without making the CPU execute one load/store per byte.

PROGRAMMED I/O / POLLING loop: status = readl(STATUS_REG) if (!(status & READY)) continue data = readl(DATA_REG) simple, low setup cost but CPU burns cycles/energy while waiting INTERRUPT-DRIVEN PIO CPU configures device then does other work ↓ device raises IRQ/MSI-X when event occurs ↓ CPU enters handler ↓ handler reads/writes device registers/data less idle spinning, but high event rates can create interrupt overhead DMA + INTERRUPT driver prepares descriptor ring + DMA buffers in RAM ↓ device DMA engine transfers many bytes independently ↓ device updates completion descriptor ↓ MSI-X driver/kernel processes completed batch DMA + POLLING same descriptor/data movement by hardware but CPU polls completion ring instead of waiting for IRQ ↓ can reduce latency/interrupt overhead under sustained load at cost of consuming CPU cycles continuously LINUX NAPI HYBRID interrupt says 'work arrived' ↓ mask/reduce further IRQs poll a bounded batch ↓ queue empty → re-enable IRQs Real systems choose hybrids based on latency, throughput, power and event rate.

Technique	CPU waiting cost	Notification latency	Bulk-transfer efficiency	Typical use
busy polling	High	Very low/predictable when CPU polls fast	Independent of transfer method	Low-latency queues, tiny embedded peripherals, short waits.
interrupts	Low while idle	Interrupt-entry/scheduling overhead	Independent of transfer method	Sparse/asynchronous events.
PIO	CPU performs data register accesses	Depends on poll/IRQ	Poor for large high-rate transfers	Control/status, tiny transfers.
DMA	CPU sets up descriptors rather than moving each byte	Completion can use IRQ or poll	High	NICs, storage, audio, cameras, GPUs.
IRQ coalescing	Lower interrupt rate	Adds some batching delay	Good for high throughput	NIC/NVMe high event rates.
NAPI-style hybrid	Interrupt when idle, polling during load	Balanced/tunable	Excellent for batched packet I/O	Linux networking.

DMA does not tell the CPU that DMA finished. Completion notification is a separate design choice: interrupt, polled completion queue, doorbell/status register, shared flag, event queue or another mechanism.

KERNEL DOCSLinux NAPI — interrupt/poll hybrid

Current documentation explicitly describes basic IRQ→NAPI polling, busy polling and IRQ mitigation; busy polling trades CPU cycles for lower latency.

https://docs.kernel.org/networking/napi.html

KERNEL DOCSLinux DMAengine provider API

Concrete framework for memory/device DMA transactions, cyclic transfers, descriptors and completion callbacks.

https://docs.kernel.org/driver-api/dmaengine/provider.html

KERNEL DOCSLinux USB Host API — URBs and interrupt endpoint polling

Useful counterexample to terminology confusion: USB interrupt URBs specify a polling interval because the host controller schedules the endpoint.

https://docs.kernel.org/6.6/driver-api/usb/usb.html

Before PCIe carries TLPs: the link has to train itself

PCIe configuration space and TLPs only work after the physical link comes up. Each port runs a Link Training and Status State Machine (LTSSM) that detects a partner, exchanges training ordered sets, agrees lane width/numbering and speed, performs equalization where required, and eventually reaches the normal active L0 state.

Fundamental reset / no usable link ↓ Detect discover electrical receiver/link partner ↓ Polling exchange training ordered sets; achieve bit/symbol/block lock ↓ Configuration negotiate link width, lane numbers and related parameters ↓ Recovery / Equalization as needed speed change, receiver lock, Tx/Rx equalization and re-training ↓ L0 normal Transaction/Data Link Layer traffic can flow Later transitions can enter low-power states, Recovery, Hot Reset, Disabled or Loopback.

PHY/link concept	Purpose
lane	One bidirectional PCIe lane consists of one TX differential pair and one RX differential pair.
x1 / x4 / x8 / x16	Number of lanes bonded into a logical link; width negotiation can fall back to fewer usable lanes.
TS1 / TS2	Training ordered sets carrying information used during link initialization/recovery.
lane reversal	Allows physical lane ordering to be reversed in supported topologies without changing logical function.
polarity inversion	Receiver can compensate if differential pair polarity is swapped where architecture permits.
equalization	Transmitter/receiver settings are tuned to overcome frequency-dependent channel loss at higher data rates.
Recovery	LTSSM state family used to retrain, change speed or recover a degraded link.
L0	Normal active operating state after successful link training.

OFFICIAL DOCSAMD Versal PCIe physical layer — current 2026 documentation

Current August 2026 public docs. Explicitly describes the physical layer, LTSSM, scrambling, electrical sub-block, serial transceivers, lane reversal and polarity inversion.

https://docs.amd.com/r/en-US/pg346-cpm-pcie/Physical-Layer

OFFICIAL DOCSAMD Vivado PCIe LTSSM link debug — 2026.1

Shows that real hardware designers can capture and inspect LTSSM transitions when a PCIe link fails to train.

https://docs.amd.com/r/en-US/ug908-vivado-programming-debugging/PCI-Express-Link-Debug

OFFICIAL DOCSAMD Vivado PCIe Link Debug GUI — LTSSM state transitions

Current GUI documentation showing the state-transition trace/diagram used to debug Detect, Polling, Configuration, Recovery and L0 behavior.

https://docs.amd.com/r/en-US/ug908-vivado-programming-debugging/PCI-Express-Link-Debug-GUI-Usage

OFFICIAL FAQPCI-SIG FAQ — LTSSM details

Public PCI-SIG answers to detailed LTSSM questions, including Recovery behavior and training ordered sets.

https://pcisig.com/faq?keys=LTSSM

OFFICIAL MATERIALPCI-SIG — PCIe 6.0 LTSSM troubleshooting session

Public description of why link bring-up is a real state-machine/timing problem rather than automatic magic.

https://pcisig.com/events/pci-sig-developers-conference-2024-agenda/pci-sig

PCIe enumeration: how firmware/OS discovers devices and gives BARs real addresses

A PCIe endpoint does not arrive with a universally fixed MMIO address. The host discovers functions through PCI configuration space, identifies bridges/endpoints, determines resource requirements, assigns bus numbers and address windows, programs Base Address Registers (BARs), and only then can ordinary MMIO accesses reach the device's register/memory apertures.

ROOT COMPLEX / HOST BRIDGE ↓ configuration-space accesses by Bus:Device.Function read Vendor ID / Device ID / Class / Header Type ↓ endpoint? ───────────────┐ bridge? │ ↓ assign secondary/subordinate bus numbers ↓ scan child bus(es) │ └──────────────────────┘ ↓ inspect BARs and resource requirements ↓ allocate non-overlapping host address windows ↓ program endpoint BAR values + bridge windows ↓ enable Memory Space / I/O Space as appropriate ↓ driver requests/maps BAR resource ↓ ioremap / pci_iomap ↓ CPU load/store to mapped MMIO VA ↓ page table / host bridge PCIe Memory Read/Write transaction ↓ endpoint register / device memory Separately: set Bus Master Enable before the endpoint is allowed to originate DMA.

PCI concept	Role
Domain:Bus:Device.Function	Hierarchical software address used to identify a PCI function, e.g. 0000:03:00.0.
configuration space	Standard per-function registers for IDs, command/status, BARs, capabilities and device-specific configuration.
BAR	Base Address Register advertising/holding an MMIO or I/O-port aperture assigned by firmware/OS.
bridge window	Address/bus range forwarded by a bridge toward devices below it.
class code	Standard category such as network controller, display controller, storage controller or bridge.
capability list	Extensible configuration blocks for MSI/MSI-X, PCIe capabilities, power management, AER, SR-IOV, etc.
Memory Space Enable	Command-register bit allowing function to respond to its memory BAR accesses.
Bus Master Enable	Command-register bit allowing a device to originate PCI memory/DMA transactions.
ECAM	Enhanced Configuration Access Mechanism mapping PCIe extended configuration space into a memory-mapped host region.
Resizable BAR	PCIe capability allowing supported BAR apertures to be reconfigured to larger/smaller supported sizes.

LINUX INSPECTION LAB lspci -nn → domain:bus:device.function + vendor/device IDs lspci -t → bridge/tree topology lspci -s 03:00.0 -vv → BAR regions, link width/speed, capabilities, MSI/MSI-X, driver cat /sys/bus/pci/devices/0000:03:00.0/resource → kernel-assigned host address ranges for BAR resources ls -l /sys/bus/pci/devices/0000:03:00.0/resource* Do not write PCI configuration registers merely to experiment.

KERNEL DOCSLinux — ACPI considerations for PCI host bridges

Especially useful for enumeration: ACPI describes host bridges/windows, while the OS discovers downstream PCI devices through standard configuration accesses and sizes/assigns their BARs.

https://www.kernel.org/doc/html/latest/PCI/acpi-info.html

KERNEL DOCSLinux — PCI host controller drivers and ECAM

Shows how a root-complex driver exposes configuration-space reads/writes by Bus/Device/Function and how ECAM can provide the underlying access mechanism.

https://docs.kernel.org/PCI/controller/pci-controller-drivers.html

PUBLIC MANUALlspci(8)

No-login manual for inspecting PCI devices, configuration-space details and topology.

https://man7.org/linux/man-pages/man8/lspci.8.html

How hardware gets a driver: device discovery, bus matching, probe(), resources and deferred probing

Linux separates discovering a device from binding a driver. PCI, USB, ACPI, Device Tree/platform code and other buses create kernel device objects. Drivers register match tables/callbacks with a bus. The driver core compares unbound devices with registered drivers; on a match, it calls the driver's probe() method in task context.

PCI EXAMPLE firmware/OS enumerates BDF 0000:03:00.0 vendor = 0x8086 device = 0x1234 class = network controller BAR0 = MMIO resource range ↓ PCI core creates struct pci_dev embedded struct device joins Linux device hierarchy ↓ /sys/bus/pci/devices/0000:03:00.0 appears modalias describes hardware identity DRIVER SIDE module/built-in driver registers struct pci_driver .id_table = supported vendor/device/class IDs .probe = my_probe .remove = my_remove ↓ driver core / PCI bus match(dev, drv) ↓ match temporarily associate driver with device ↓ my_probe(struct pci_dev *pdev, ...) TYPICAL probe() WORK enable PCI function request/reserve BAR resources map BAR with pci_iomap()/devm_ioremap_resource() set DMA mask allocate DMA rings/buffers configure MSI/MSI-X vectors request IRQ handlers reset/read hardware identity/version load firmware if required create/register netdev/block/input/DRM/etc. child/class object ↓ probe returns 0 ↓ device is BOUND; sysfs driver symlink reflects relationship DEPENDENCY NOT READY probe needs regulator/clock/IOMMU/PHY/controller supplied by another driver ↓ return -EPROBE_DEFER ↓ driver core places device on deferred-probe path ↓ later supplier appears probe is attempted again UNPLUG / UNBIND stop new I/O quiesce hardware / cancel work unregister child/class interfaces free IRQ/DMA/resources driver remove()/managed cleanup ↓ device becomes unbound or disappears

Driver-model object	Role
struct device	Generic kernel device object embedded by bus-specific types such as pci_dev/platform_device.
struct device_driver	Generic registered driver object containing probe/remove and driver-core state.
bus_type	Bus-specific match/probe/remove/uevent policy connecting devices and drivers.
match()	Bus callback deciding whether a driver can handle a particular device.
probe()	Driver initialization callback after a match; returns 0 only when binding successfully completed.
remove()	Driver teardown callback when binding is removed/device disappears.
driver data	Per-device driver state stored through helpers such as dev_set_drvdata()/pci_set_drvdata().
devres / devm_*	Managed-resource mechanism automatically releasing selected resources when a device unbinds.
modalias	String encoding device identity used by userspace/kernel module tools to locate a matching module.
-EPROBE_DEFER	Special probe result saying the driver probably matches but a supplier/resource is not ready yet.
device link	Explicit supplier→consumer relationship used for probe ordering, runtime PM and removal/shutdown ordering.
class	Functional userspace-facing grouping such as block, net, tty, input or DRM, distinct from physical bus topology.

Enumeration does not mean the driver is ready. A PCI function can exist in sysfs with BARs and IDs while no functional driver is bound. Conversely, a driver can be loaded while no matching hardware is present.

READ-ONLY DRIVER-BINDING LAB lspci -nnk # pick a PCI BDF BDF=0000:03:00.0 readlink /sys/bus/pci/devices/$BDF/driver 2>/dev/null cat /sys/bus/pci/devices/$BDF/modalias 2>/dev/null cat /sys/bus/pci/devices/$BDF/uevent 2>/dev/null cat /sys/bus/pci/devices/$BDF/resource 2>/dev/null # find drivers on the PCI bus ls /sys/bus/pci/drivers | less # deferred-probe diagnostics when debugfs is mounted cat /sys/kernel/debug/devices_deferred 2>/dev/null # Do not write to bind/unbind/remove/rescan on a machine whose # storage, network or display matters; those controls are operational, not merely descriptive.

KERNEL DOCSLinux Driver Binding

Core model: a bus match callback compares device/driver, then the driver's probe() callback is invoked and per-device state is initialized.

https://docs.kernel.org/driver-api/driver-model/binding.html

KERNEL DOCSLinux Device Drivers — probe() and deferred probe

Current probe semantics, including successful binding, cleanup on failure and -EPROBE_DEFER when suppliers/resources are not ready.

https://docs.kernel.org/driver-api/driver-model/driver.html

KERNEL DOCSLinux Device Driver infrastructure

Current bus_type/driver-core APIs including match, uevent, probe, remove and deferred-probe diagnostics.

https://docs.kernel.org/driver-api/infrastructure.html

KERNEL DOCSLinux platform devices and drivers

Concrete non-PCI example: discovery is separate from the driver, and driver registration checks unbound platform devices for matches.

https://docs.kernel.org/driver-api/driver-model/platform.html

KERNEL DOCSLinux device links

Supplier/consumer dependency tracking used for probe ordering, power-management ordering and safe teardown.

https://docs.kernel.org/driver-api/device_link.html

A bound device driver may still need another program: request_firmware() loads code/data into the device

Driver binding does not imply that every device is ready to operate. Wi-Fi adapters, GPUs, DSPs, storage controllers and other peripherals often contain an embedded microcontroller that needs a vendor firmware image. The Linux driver can call the firmware-loader API, obtain a blob by name, validate or interpret it as appropriate for that driver, then transfer it into device RAM or flash through MMIO, DMA or a device-specific protocol.

DEVICE DISCOVERY / DRIVER PROBE PCI / USB / platform device appears ↓ bus match selects driver ↓ driver probe() ↓ request_firmware("vendor/device.bin", device) ↓ Linux firmware loader searches configured locations ├─ built-in firmware if compiled into kernel ├─ initramfs / filesystem firmware paths └─ optional fallback path where enabled ↓ firmware blob returned to driver ↓ driver validates/parses device-specific format ↓ MMIO / DMA / mailbox / bus transfer into device ↓ device microcontroller starts → driver finishes initialization Kernel driver code ≠ device firmware image ≠ UEFI/BIOS firmware.

Layer	Role
kernel driver	Host CPU code that controls the device and asks Linux for a firmware blob when the hardware requires one.
firmware-loader API	Kernel mechanism behind synchronous and asynchronous firmware requests, search paths, caching and optional fallbacks.
firmware file	Opaque-to-the-core blob whose format and destination are normally understood by the device-specific driver/firmware pair.
device processor	Embedded execution engine on the peripheral that runs the uploaded image or consumes configuration/calibration data.
initramfs concern	If a device is required before the real root filesystem is available, its firmware may need to be present in the initramfs or built into the kernel.

“Firmware” is overloaded. CPU microcode, motherboard UEFI, SSD controller firmware and a Wi-Fi firmware blob all execute below ordinary applications, but they have different loading paths, storage locations and trust/update mechanisms.

KERNEL DOCSLinux Firmware API

Current index for request_firmware(), asynchronous requests, firmware search paths, built-in firmware, caching, fallback mechanisms and firmware upload interfaces.

https://docs.kernel.org/driver-api/firmware/index.html

KERNEL DOCSLinux firmware-loader introduction

Explains why drivers request firmware, including device microcontroller code and calibration/information data, and distinguishes synchronous from asynchronous requests.

https://docs.kernel.org/driver-api/firmware/introduction.html

`ioctl()` is the escape hatch when read/write are not enough: fd + request code + structured control data

Unix makes many objects look like file descriptors, but not every device operation is naturally “read bytes” or “write bytes.” A camera must negotiate pixel formats, a terminal must change line settings, a network interface may expose device-specific configuration, and a GPU needs rich command/control APIs. Linux commonly expresses these operations through ioctl(): an open file descriptor selects the kernel object, while a request number selects a device/subsystem-specific operation.

userspace open("/dev/example", O_RDWR) ↓ fd → open file description → struct file ↓ ioctl(fd, REQUEST_CODE, &userspace_struct) ↓ syscall entry VFS resolves fd to struct file ↓ file_operations.unlocked_ioctl(file, cmd, arg) ↓ driver/subsystem decodes cmd ├── validate request + version/flags ├── copy_from_user() structured input when required ├── check permissions / object state ├── change driver or hardware state ├── wait / queue asynchronous work where API defines it └── copy_to_user() output/result when required ↓ return value / errno to application REQUEST NUMBER CONVENTION _IO(type, nr) no typed payload _IOR(type, nr, T) kernel → userspace payload _IOW(type, nr, T) userspace → kernel payload _IOWR(type, nr, T) bidirectional payload The encoded size/direction bits are conventions for the ABI; the driver still has to implement and validate the contract correctly.

Interface	Best mental model
`read()` / `write()`	Move a stream or sequence of bytes/messages through an already-configured object.
`mmap()`	Map pages or device/shared buffers into a process address space so access happens through ordinary loads/stores.
`ioctl()`	Issue typed, subsystem-specific control/query operations on an fd when the operation does not fit generic stream I/O.
`fcntl()`	Generic file-descriptor/open-file-description controls such as descriptor flags, status flags and locking families.
sysfs	Text/binary attributes representing relatively simple device/kernel objects and configuration; not a replacement for every transactional ABI.
Netlink	Structured message-oriented control/event transport used heavily by networking and other kernel subsystems, often better for extensible object-oriented APIs.

An ioctl ABI is part of the userspace/kernel contract. Request-number layout, structure size/alignment, pointer handling, 32-bit compatibility, reserved fields and information-leak avoidance matter because applications may keep using an ioctl for decades. Kernel documentation explicitly warns that ioctl interfaces are flexible but easy to design badly and hard to repair after deployment.

PUBLIC MANUALioctl(2) — Linux system-call manual

Defines the fd/request/argument interface, traditional direction/size encoding, return conventions and the fact that semantics depend on the underlying device or subsystem.

https://man7.org/linux/man-pages/man2/ioctl.2.html

KERNEL DOCSLinux — ioctl based interfaces

Driver-author guidance for stable ioctl UAPIs: command-number macros, versioning, return values, timestamps, compat handling, structure layout, information leaks and alternatives.

https://docs.kernel.org/driver-api/ioctl.html

A .ko file is relocatable kernel code: module aliases, ELF sections, symbol resolution, relocation and init

A loadable kernel module is not a userspace shared library. A .ko is an ELF module built for a particular kernel ABI/configuration. Userspace tools such as modprobe find the correct file and its dependencies; the kernel parses the ELF image, validates compatibility/signature policy, allocates executable/data memory, resolves exported kernel/module symbols, applies relocations, and invokes the module's initialization function.

HARDWARE MODALIAS pci:v00008086d00001234sv...bc02sc00i00 ↓ userspace modalias database / modprobe alias resolution ↓ matching module: example_driver.ko[.xz/.zst/...] ↓ modprobe loads dependency modules first using modules.dep.bin ↓ finit_module(fd, parameters, flags) KERNEL MODULE LOADER verify privilege / modules_disabled policy decompress in-kernel if supported/requested parse ELF ET_REL image validate architecture / vermagic / symbol versions verify module signature if configured/enforced allocate module text/rodata/data/bss/percpu regions copy/lay out allocatable sections resolve undefined references against exported kernel + loaded-module symbols apply architecture-specific ELF relocations finalize permissions: executable text, read-only data, writable data register module kallsyms/sysfs/trace state ↓ call module init function ↓ driver registers itself with PCI/USB/platform/etc. bus ↓ bus match → probe matching devices MODULE_INIT SECTION module_init(my_init) module_exit(my_exit) MODULE_DEVICE_TABLE(pci, ids) MODULE_LICENSE(...) MODULE UNLOAD modprobe -r / rmmod ↓ delete_module syscall kernel checks references/dependencies and unload policy ↓ call module exit function wait/flush required work as implementation requires remove symbols/sysfs state free module memory If references are still held, normal unload fails rather than freeing executing code.

Module concept	Meaning
.ko	Kernel object/module file; ELF relocatable image plus module-specific metadata.
ET_REL	ELF relocatable-file type: section addresses are not final until loader/linker placement and relocation.
vermagic	Kernel/version/config/compiler-related compatibility string checked during module loading unless forced policy permits otherwise.
CONFIG_MODVERSIONS	Optional symbol-version CRC mechanism strengthening module/kernel ABI compatibility checks.
EXPORT_SYMBOL	Makes a kernel/module symbol eligible for resolution by other modules.
undefined module symbol	Reference that module loader must resolve against exported symbols before module can execute.
module relocation	Architecture-specific patch to code/data after final kernel virtual addresses are assigned.
module_init()	Marks/registers the function invoked when module finishes loading.
module_exit()	Marks cleanup function invoked during supported unload.
MODULE_DEVICE_TABLE	Emits bus device-ID metadata used to generate module aliases for automatic hardware matching.
module reference count	Tracks active users/dependencies so ordinary unload cannot free a module still in use.
module signature	Cryptographic signature appended to module and verified by kernel according to configured trust/enforcement policy.
kernel taint	Diagnostic flag recording conditions such as unsigned/out-of-tree/forced modules that can affect support/debug interpretation.

modprobe does not perform the ELF relocation itself. Current kmod documentation explicitly says modern modprobe leaves symbol resolution and parameter understanding to the kernel. This is the opposite of the ordinary userspace dynamic linker model where ld-linux performs relocations in userspace.

MODULE INSPECTION LAB — NO LOADING REQUIRED MOD=$(modinfo -n e1000e 2>/dev/null || true) echo "$MOD" modinfo e1000e 2>/dev/null | less modprobe --show-depends e1000e 2>/dev/null modprobe --show-modversions "$MOD" 2>/dev/null | head # compressed distro modules may need decompression before generic ELF tools # for an uncompressed .ko: readelf -h module.ko readelf -SW module.ko | less readelf -rW module.ko | less nm -u module.ko | head -50 # currently loaded modules cat /proc/modules | less ls /sys/module | less # Do not insmod/rmmod arbitrary modules on a useful system; loading code into # kernel privilege can crash or compromise the entire machine.

CURRENT PUBLIC MANUALinit_module(2) / finit_module(2)

Current 2026 manual explicitly says the kernel loads an ELF image, performs symbol relocations, initializes parameters and runs the module init function.

https://man7.org/linux/man-pages/man2/init_module.2.html

CURRENT PUBLIC MANUALmodprobe(8)

Current kmod manual for dependency/alias resolution and module insertion; it explicitly notes symbol resolution is performed inside the kernel.

https://man7.org/linux/man-pages/man8/modprobe.8.html

SOURCE / FILELinux kernel module loader source

Real current loader: ELF/module validation, symbol search, relocation preparation, memory layout, reference counting, init/unload and module sysfs state.

https://github.com/torvalds/linux/blob/master/kernel/module/main.c

SOURCE / FILEx86 module relocation source

Architecture-specific current x86 module relocation and executable-memory handling.

https://github.com/torvalds/linux/blob/master/arch/x86/kernel/module.c

KERNEL DOCSLinux module signing facility

Current signature verification and enforcement model, including kernel-side checking and supported public-key signature families.

https://docs.kernel.org/admin-guide/module-signing.html

CURRENT PUBLIC MANUALdelete_module(2)

Current unload syscall semantics and failure cases when a module remains in use or unloading is disabled.

https://man7.org/linux/man-pages/man2/delete_module.2.html

Kernel livepatching redirects selected functions at runtime, but only after tasks reach a consistent state

Replacing a running kernel is harder than loading a new module. A CPU may already be executing an old function, a sleeping task may resume into its old stack frame, and related functions may need to switch together to preserve locking or data-structure invariants. Linux livepatch therefore combines replacement function code with a consistency model: the patch is loaded, function redirection is prepared, and tasks transition to the patched state only at points where switching is considered safe.

build livepatch module against the target kernel ↓ insmod / finit_module() ↓ kernel validates module + livepatch metadata ↓ resolve OLD target functions and NEW replacement functions ↓ register livepatch redirection through ftrace machinery ↓ /sys/kernel/livepatch/<patch>/enabled = 1 ↓ PATCH TRANSITION STARTS some tasks may still be in old code other tasks can already be switched to new code ↓ per-task consistency checks syscall/return boundaries and reliable stack traces help determine safety sleeping/running tasks converge when they reach a safe state ↓ all relevant tasks report patched state ↓ transition = 0 replacement functions are now consistently active CUMULATIVE / ATOMIC REPLACE new livepatch can replace older livepatch function stacks ↓ transition completes ↓ obsolete redirections can be removed

Mechanism	What it means
replacement function	New implementation compiled into the livepatch module and associated with an existing kernel symbol.
ftrace redirection	Kernel machinery redirects calls for patched functions toward the currently selected implementation.
transition	Temporary state in which different tasks may still be converging from old to new code (or back again).
per-task consistency	A task changes patch state only when doing so will not resume through an unsafe mixture of old and new function semantics.
callbacks / shadow variables	Mechanisms for patches that also need controlled state preparation, migration or auxiliary per-object state rather than only function replacement.
atomic replace	A cumulative patch can supersede older livepatches so the final active patch set is easier to reason about.

Livepatch is not a universal substitute for rebooting. Some fixes change data layouts, initialization assumptions, architecture state or too much code to transition safely. The livepatch framework solves a constrained runtime code-replacement problem; it does not make every kernel update hot-swappable.

KERNEL DOCSLinux kernel livepatching documentation

Current documentation index for the livepatch consistency model, lifecycle, callbacks, cumulative replacement, ELF format, shadow variables and APIs.

https://docs.kernel.org/livepatch/index.html

KERNEL DOCSLinux livepatching APIs

Implementation-facing reference for enabling patches, replacement-function metadata, transition state and livepatch shadow/state interfaces.

https://docs.kernel.org/livepatch/api.html

KERNEL DOCSLinux livepatch callbacks

Explains pre/post patch and unpatch callbacks used when a live update needs controlled state changes in addition to redirecting code.

https://docs.kernel.org/livepatch/callbacks.html

How a kernel device becomes visible to userspace: sysfs, uevents, devtmpfs, udev rules and /dev

The Linux device model exports topology and attributes through sysfs, while device creation/removal can generate a kobject uevent for userspace. On systemd-based distributions, systemd-udevd consumes those events, applies rules, records properties, manages device-node permissions and creates stable symlinks. With devtmpfs enabled, the kernel can provide the basic device-node namespace that userspace policy then decorates/manages.

USB STORAGE DEVICE IS PLUGGED IN USB host controller reports port/device event ↓ USB core enumerates descriptors ↓ creates struct device / USB device + interface objects ↓ sysfs hierarchy appears under /sys/devices/.../usb... ↓ kernel sends kobject UEVENT environment/properties include concepts such as: ACTION=add DEVPATH=/devices/... SUBSYSTEM=usb PRODUCT=... MODALIAS=usb:v.... ↓ systemd-udevd receives event ↓ udev rules inspect kernel name, subsystem, attributes, parent attributes, properties, tags, modalias and builtin helpers ↓ possible outcomes: request/load matching kernel module through modalias handling set OWNER/GROUP/MODE permissions add stable /dev symlinks such as /dev/disk/by-id/... add device properties/tags to udev database rename network interfaces through supported policy DRIVER BINDS mass-storage stack eventually exposes block device major:minor ↓ devtmpfs/kernel device machinery provides basic node, e.g. /dev/sdb ↓ udev applies policy and creates meaningful symlinks /SYS vs /DEV /sys/... = object topology, attributes, driver/bus/class relationships /dev/sdb = special file whose inode stores block/character major+minor ↓ open('/dev/sdb') VFS recognizes device inode type ↓ major/minor selects registered block/char device implementation ↓ normal file descriptor now reaches device-driver operations Not every device has a /dev node: network interfaces are normally controlled through sockets/netlink/sysfs rather than open('/dev/eth0').

Userspace/device concept	Role
sysfs	Virtual filesystem exposing kernel object/device/bus/driver/class hierarchy and attributes.
kobject	Reference-counted kernel object embedded in many device-model objects and represented in sysfs.
uevent	Kernel→userspace notification for add/remove/change/move and related object actions.
DEVPATH	Sysfs-relative path identifying the object associated with a uevent.
SUBSYSTEM	Device-model subsystem/bus/class identifier carried with event/property processing.
MODALIAS	Hardware identity alias userspace can resolve against kernel module aliases.
systemd-udevd	Userspace daemon consuming kernel uevents and applying udev rules.
udev database	Userspace property/state database accumulated for devices after rule processing.
devtmpfs	Kernel-maintained device-node filesystem supplying basic char/block device nodes when configured.
major/minor	Numeric character/block device identifier connecting a special /dev inode to a registered kernel device number.
udev symlink	Stable/human-useful alternate pathname such as /dev/disk/by-id/... pointing to a kernel-named node.
coldplug	Boot-time replay/triggering of device events for hardware already present before udev began normal hotplug processing.

udev is policy, not the hardware driver. The kernel driver handles MMIO/DMA/IRQs/protocol state. udev reacts to kernel device events and assigns userspace-facing names, permissions, properties and helper actions. Deleting a udev rule does not make the PCIe/USB hardware disappear.

LIVE DEVICE-EVENT LAB # terminal 1: observe kernel and processed udev events udevadm monitor --kernel --udev --property # terminal 2: plug a USB device, then inspect one resulting path udevadm info --query=all --name=/dev/sdX 2>/dev/null | less udevadm info --attribute-walk --name=/dev/sdX 2>/dev/null | less # map /dev node to sysfs udevadm info --query=path --name=/dev/sdX 2>/dev/null ls -l /dev/disk/by-id /dev/disk/by-path 2>/dev/null | less # char/block major:minor stat -c '%F %t:%T %n' /dev/null /dev/sdX 2>/dev/null # Test a rule against a sysfs device without physically replugging it udevadm test /sys/path/to/device 2>&1 | less # udevadm trigger is operational: it replays events. Prefer monitor/info/test # first on a system whose device setup matters.

CURRENT PUBLIC MANUALudev(7) — current systemd 262~devel documentation

Current September 2026 rendering: udev manages device events, device-node permissions, additional /dev symlinks and network-interface naming policy.

https://man7.org/linux/man-pages/man7/udev.7.html

CURRENT PUBLIC MANUALsystemd-udevd.service(8)

Current device-event daemon documentation: systemd-udevd listens for kernel uevents and executes matching udev-rule instructions.

https://man7.org/linux/man-pages/man8/systemd-udevd.service.8.html

CURRENT PUBLIC MANUALudevadm(8)

Current 2026 debugging/inspection tool for info, monitor, trigger, test, verify, wait and the udev database.

https://man7.org/linux/man-pages/man8/udevadm.8.html

KERNEL DOCSLinux kobject uevent API

Current kernel API explicitly defines kobject_uevent() as notifying userspace and kobject_uevent_env() as attaching event environment data.

https://docs.kernel.org/next/driver-api/basics.html

SOURCE / FILELinux devtmpfs source

Real current kernel implementation for the device-node filesystem integrated with driver-core device add/remove events.

https://github.com/torvalds/linux/blob/master/drivers/base/devtmpfs.c

KERNEL DOCSLinux Kernel Device Model

Connects the sysfs device hierarchy to bus/device/driver discovery and userspace-visible attributes.

https://docs.kernel.org/driver-api/driver-model/overview.html

Linux exposes kernel state through several filesystem-shaped ABIs, but /proc, /sys, debugfs and configfs have different contracts

Many Linux “files” are not persistent disk bytes at all. The VFS gives kernel subsystems a familiar pathname/read/write interface for exporting state or accepting configuration, but the semantics depend heavily on the mounted pseudo-filesystem. Treating every file below /proc or /sys like an ordinary data file leads to bad mental models and sometimes bad software.

procfs (/proc) kernel/process runtime state /proc/<pid>/status, maps, fd/... /proc/meminfo /proc/sys/... sysctl-style knobs ↓ namespace-sensitive views for process-related state sysfs (/sys) kobject / device-model hierarchy /sys/devices /sys/bus /sys/class /sys/module ↓ attributes call kernel show()/store()-style methods ↓ documented sysfs interfaces are userspace ABI debugfs (/sys/kernel/debug) developer/debug instrumentation ↓ intentionally weak stability guarantees ↓ do NOT build production ABI assumptions around arbitrary entries configfs (commonly /sys/kernel/config) userspace drives kernel-object lifecycle ↓ mkdir creates a config item/object ↓ write attributes configures it ↓ rmdir destroys it when references permit SAME SHELL OPERATIONS, DIFFERENT SEMANTICS cat / write / mkdir / readdir ↓ VFS dispatches into that pseudo-filesystem's kernel callbacks ↓ no ordinary on-disk inode/data-block persistence is implied.

Pseudo-filesystem	Primary mental model	ABI expectation
`procfs`	Processes and selected global/runtime kernel state; includes namespace-sensitive process views and `/proc/sys` controls.	Many interfaces are long-lived userspace ABI, but exact files/permissions depend on kernel/configuration/namespaces.
`sysfs`	Structured view of kobjects, devices, buses, classes, drivers and simple attributes.	Documented interfaces are intended to be stable; one-value-per-file/simple-attribute conventions are deliberate.
`debugfs`	Ad-hoc kernel debugging/developer instrumentation.	Explicitly not intended as a stable userspace ABI.
`configfs`	Userspace-created/configured kernel objects: directory creation controls object lifecycle.	Subsystem-specific configuration ABI; lifecycle is driven from userspace rather than merely observed.
ordinary disk filesystem	Persistent namespace mapping names to inodes/data/extents on storage.	File contents represent stored data, subject to filesystem durability semantics.

“Everything is a file” does not mean “every file is storage.” These interfaces reuse file descriptors, pathname permissions and VFS operations as a control/observation language. Reads can execute code and synthesize current state; writes can invoke configuration actions; directory creation can instantiate kernel objects.

KERNEL DOCSLinux — the /proc filesystem

Detailed procfs documentation covering per-process state, system information, /proc/sys configuration and namespace-aware mount behavior.

https://docs.kernel.org/filesystems/proc.html

KERNEL DOCSLinux — sysfs and kernel objects

Explains sysfs as the kobject/device hierarchy exported to userspace, its attribute callbacks and ABI conventions.

https://docs.kernel.org/filesystems/sysfs.html

KERNEL DOCSLinux — debugfs

Explains debugfs as a developer-facing interface with intentionally weak stability guarantees rather than a normal production ABI.

https://docs.kernel.org/filesystems/debugfs.html

KERNEL DOCSLinux — configfs

Shows the inverse-of-sysfs lifecycle: userspace mkdir()/rmdir() operations create and destroy kernel configuration objects.

https://docs.kernel.org/filesystems/configfs.html

PCI Express: how a modern add-in device actually talks to memory

PCIe is not an old parallel bus with dozens of shared address/data wires. It is a packetized, point-to-point serial interconnect. A CPU/root complex, switches and endpoints exchange Transaction Layer Packets (TLPs). Software still sees familiar concepts such as configuration registers, memory-mapped device registers, DMA and interrupts.

CPU / memory system │ ↓ PCIe Root Complex │ point-to-point serial link ↓ optional PCIe switch │ ↓ endpoint device (NVMe SSD / NIC / GPU / etc.) CPU writes BAR-mapped register ─────→ device command/control device sends Memory Read/Write TLPs ─→ DMA to/from system RAM device sends MSI/MSI-X message ──────→ interrupt delivery

During enumeration, system software discovers the endpoint and reads its PCI configuration space.
The device advertises Base Address Registers (BARs), which describe memory or I/O regions it needs. The OS assigns address ranges.
The driver maps those BAR-backed regions and writes device registers to configure queues, buffers, modes and command state.
For DMA, the driver arranges RAM buffers and provides device-visible addresses. An IOMMU may translate/restrict those addresses.
The device can then send PCIe memory-read/write transactions to access RAM without asking the CPU to execute one load/store per byte.
When work completes, the device can signal the processor using MSI/MSI-X: logically, an interrupt is generated by a special write/message rather than a dedicated legacy IRQ wire.

TECH ARTICLEDown to the TLP — PCIe primer, part I

Excellent plain-web explanation of TLPs, address spaces, BARs, DMA/bus mastering and MSI interrupts. No signup and much less opaque than jumping straight into the full standard.

https://xillybus.com/tutorials/pci-express-tlp-pcie-primer-tutorial-guide-1

PCIe on the wire: TLPs, completions, tags and credit-based flow control

After link training and enumeration, PCIe carries transactions as Transaction Layer Packets (TLPs). A CPU MMIO write, a CPU MMIO read, a device DMA write and a DMA read are not all the same traffic class. PCIe distinguishes Posted requests, Non-Posted requests and Completions, and uses receiver-advertised credits so a transmitter does not overflow the next hop's buffers.

MMIO WRITE TO DEVICE BAR CPU store → root complex ↓ Memory Write Request TLP header: requester/address/attributes/length payload: written bytes ↓ POSTED request: no Transaction-Layer Completion TLP is required MMIO READ FROM DEVICE BAR CPU load → root complex ↓ Memory Read Request TLP requester ID + TAG identify this outstanding request ↓ NON-POSTED request ↓ device obtains requested register/memory data Completion with Data TLP completer ID + requester/tag + status + payload ↓ root complex matches completion to waiting read ↓ CPU receives result DMA WRITE endpoint becomes requester → Memory Write TLPs → host RAM DMA READ endpoint sends Memory Read Request TLP root/memory side returns Completion-with-Data TLP(s) CREDIT-BASED FLOW CONTROL receiver advertises buffer capacity to upstream transmitter 6 logical credit pools: Posted Header (PH) Posted Data (PD) Non-Posted Header (NPH) Non-Posted Data (NPD) Completion Header (CplH) Completion Data (CplD) before sending a TLP, transmitter must have sufficient relevant credits ↓ send packet → consume credits receiver drains packet → later advertises returned/updated credits This is hop-by-hop flow control, not end-to-end TCP-style congestion control.

PCIe traffic	Posted?	Response at Transaction Layer?	Typical example
Memory Write	Yes	No completion required	CPU writes MMIO register; NIC/GPU DMA writes host RAM.
Memory Read	No	Completion with Data	CPU reads device register; endpoint DMA reads host RAM.
Configuration Read/Write	Non-Posted class	Completion required	Enumeration/config-space access.
Completion	Response traffic	It is the response	Returns read data/status to original requester.
Message	Often Posted	Depends on message class/semantics	Interrupt/error/power-management-style protocol messages.

Why a read has a tag: PCIe permits multiple Non-Posted requests to be outstanding at once. Responses may return later, and a read can be split into multiple Completions. Requester identity plus tag/order metadata lets the requester associate returned data with the correct original operation.

OFFICIAL DOCSAMD Versal PCIe — current 2026 flow-control credit interface

Current August 2026 documentation exposes Posted, Non-Posted and Completion header/data credit pools exactly as maintained by a real PCIe controller.

https://docs.amd.com/r/en-US/pg346-cpm-pcie/Configuration-Flow-Control-Interface

OFFICIAL DOCSAMD PCIe — Flow Control Credit Information

Concrete explanation of all six PCIe flow-control pools and the meaning of header/data credits and receive-buffer capacity.

https://docs.amd.com/r/en-US/pg054-7series-pcie/Flow-Control-Credit-Information

OFFICIAL DOCSAMD PCIe — Transmit Credit Flow Control

Shows that transmit credits represent the link partner's receive-buffer limits/consumption and how zero credits can block packet transmission.

https://docs.amd.com/r/en-US/pg054-7series-pcie/Transmit-Credit-Flow-Control-Information

PCIe errors are protocol events too: AER logging, severity and driver-coordinated recovery

PCIe links detect malformed/corrupt/replayed/timeout conditions below ordinary device-driver semantics. Advanced Error Reporting (AER) adds standardized extended capability registers that capture error status, source/requester information and header logs. Linux can collect those reports at Root Ports/RCECs and coordinate recovery with the drivers below the affected hierarchy.

PCIe LINK / TRANSACTION ERROR endpoint / switch / root-port logic detects problem ↓ classify CORRECTABLE hardware/protocol recovers without data/function loss examples include retry/replay-style link recovery classes ↓ log/counters may be updated Linux AER logs and clears status; no device reset required UNCORRECTABLE NON-FATAL one transaction/function may be unreliable link itself can remain usable ↓ Error Message → upstream Root Port Root Port AER registers record severity/source/header information ↓ interrupt to CPU Linux AER service finds affected hierarchy ↓ driver error_detected(...) drivers can report CAN_RECOVER / NEED_RESET / DISCONNECT ↓ possibly mmio_enabled() / slot_reset() depending recovery outcome ↓ resume() if recovery succeeds UNCORRECTABLE FATAL link/hierarchy considered unreliable ↓ drivers notified with frozen I/O state upstream reset is required in normal recovery path reinitialize device/config/firmware as necessary ↓ resume or permanent failure PERMANENT FAILURE cancel outstanding I/O refuse new I/O / report errors upward device is effectively removed/dead until stronger recovery/reboot/replacement

AER/error object	Role
Correctable Error Status	Records protocol/link errors hardware recovered from without functional loss.
Uncorrectable Error Status	Records errors that can invalidate a transaction or link/function.
Uncorrectable Error Severity	Classifies selected uncorrectable conditions as non-fatal or fatal.
Header Log	Captures TLP header information for some reported uncorrectable errors to aid diagnosis.
Error Source Identification	Root-level information identifying requester/reporter associated with the error.
ACPI _OSC	Firmware/OS ownership negotiation; Linux handles native AER only when firmware grants appropriate control.
error_detected()	Driver callback announcing normal/frozen/permanent channel state after PCI error.
mmio_enabled()	Recovery callback after MMIO access becomes safe again where applicable.
slot_reset()	Driver callback to restore/reinitialize hardware after PCI slot/bus/function reset.
resume()	Final callback telling driver normal I/O can restart after successful recovery.
FLR	Function Level Reset: PCIe reset mechanism targeting one function when supported/appropriate.
Secondary Bus Reset	Bridge-level reset affecting devices below a PCIe bridge/port.

AER is not the same as a device's own error register. AER concerns PCIe hierarchy/link/transaction integrity. An NVMe media error, GPU firmware fault or NIC packet CRC problem can be reported through completely different device-specific mechanisms even though the device itself sits on PCIe.

LINUX AER OBSERVATION # find AER capability on a PCIe function/root port lspci -vv -s <BDF> | grep -A20 -i 'Advanced Error Reporting' # recent kernel PCIe/AER logs dmesg | grep -Ei 'AER|PCIe Bus Error|Corrected|Uncorrected|DPC' | less # PCIe error counters may be available through sysfs/debugfs/rasdaemon # depending on platform, kernel and distribution. # DO NOT use AER error-injection modules/tools on a useful machine; # the recovery paths intentionally reset devices/buses and can destroy in-flight I/O.

KERNEL DOCSLinux PCIe AER HOWTO

Current kernel guide: AER gathers comprehensive error information, reports it and performs recovery. It distinguishes correctable, uncorrectable non-fatal and fatal errors and documents Root Port logging/interrupt behavior.

https://docs.kernel.org/PCI/pcieaer-howto.html

KERNEL DOCSLinux PCI Error Recovery

Detailed driver recovery state machine: disconnect/freeze I/O, error_detected(), optional MMIO recovery, reset, slot_reset(), resume() or permanent failure.

https://docs.kernel.org/PCI/pci-error-recovery.html

A PCIe device can appear or disappear while the OS is running: hotplug is a coordinated lifecycle, surprise removal is not

PCIe discovery is not only a boot-time event. A hotplug-capable slot can report card-presence and link-state changes while Linux is running. On insertion, the PCI core rescans the hierarchy, discovers configuration-space functions, assigns/validates resources, creates device objects and lets matching drivers probe. A planned removal reverses that software ownership before power/link removal; a surprise removal can make hardware vanish first, so every layer must be prepared for failed MMIO, aborted commands and a device that can no longer complete DMA or interrupts normally.

PLANNED HOT-ADD slot/card-presence event ↓ PCIe link trains ↓ PCI core scans configuration space ↓ new pci_dev + BAR/resource setup ↓ sysfs / uevent ↓ matching driver probe() ↓ register netdev / block device / GPU / etc. PLANNED HOT-REMOVE userspace / slot controller requests removal ↓ stop new upper-layer I/O ↓ driver remove(): quiesce device disable interrupts stop DMA / cancel work unregister subsystem objects release mappings/resources ↓ PCI device objects removed ↓ link/power may be turned off SURPRISE REMOVAL link disappears before software teardown ↓ config/MMIO transactions may fail outstanding requests must be failed or timed out ↓ driver/core mark device inaccessible/dead ↓ teardown software state without assuming hardware still responds

Event/object	What it means
presence detect / link change	Physical slot or link state changed; this is only the start of software discovery or removal.
rescan	Walk configuration space below a bus again and instantiate newly discovered functions.
`probe()`	A matching driver takes ownership and allocates device-specific software/DMA/IRQ state.
`remove()`	Driver teardown callback: stop traffic and unregister everything created by probe before the device object disappears.
sysfs `remove`	Software-removes a PCI function from Linux; it is not the same thing as physically powering off a hotplug slot.
surprise removal	The hardware becomes unreachable before orderly driver quiescence. Error handling must not depend on further successful device accesses.

Hotplug is more than “run enumeration again.” Discovery can create a new function, but safe removal is primarily a lifetime problem: prevent new users, drain or fail old users, stop DMA/interrupts, release references, and only then forget the device. Surprise removal compresses those steps into recovery after the link is already gone.

KERNEL DOCSLinux PCI sysfs interface — remove and device resources

Current kernel documentation for PCI sysfs objects, including the per-device remove file and the important distinction that software removal does not itself perform physical hotplug power control.

https://docs.kernel.org/PCI/sysfs-pci.html

KERNEL DOCSLinux PCI tracepoints — hotplug and link events

Current trace documentation for pci_hp_event and PCIe link events, with card-present/card-not-present and link-up/link-down states that can be observed through tracefs.

https://docs.kernel.org/trace/events-pci.html

IOMMU: the MMU for devices, and why unrestricted DMA would be dangerous

A DMA-capable device can originate memory transactions without the CPU executing matching load/store instructions. That is powerful—and dangerous. An IOMMU sits on the device-to-memory path and translates/restricts device-visible I/O virtual addresses (IOVAs), much as a CPU MMU translates/restricts virtual addresses for CPU memory accesses.

WITHOUT IOMMU PCIe device DMA address ─────────────→ system physical memory malicious/buggy device can potentially overwrite arbitrary RAM if platform permits it WITH IOMMU device_id / requester ID + device-generated IOVA ↓ IOMMU device context ↓ I/O page-table walk / IOTLB ↓ permission check + translation allowed IOVA → system physical address ↓ RAM unmapped / forbidden DMA → IOMMU fault instead of arbitrary memory corruption

CPU MMU concept	IOMMU analogue
virtual address	IOVA / device-visible DMA address
process/page-table context	device/process context selected by requester/device ID and optionally PASID
TLB	IOTLB / cached I/O translations
page-table walk	I/O page-table walk
page fault	DMA/IOMMU translation or permission fault
process isolation	device isolation / VM device assignment protection
two-stage guest translation	guest IOVA/GPA through nested IOMMU translation to host/system physical address

OFFICIAL SPECRISC-V IOMMU specification — current ratified library

Current February 2026 RISC-V IOMMU publication. The base architecture is version 1.0 ratified, with current clarifications/extensions collected in the v20260222 release.

https://docs.riscv.org/reference/iommu/iommu_preface.html

OFFICIAL SPECRISC-V IOMMU introduction — device contexts and two-stage translation

Excellent system-level explanation: DMA-capable devices are identified, assigned contexts, translated through I/O page tables and restricted from arbitrary system memory.

https://docs.riscv.org/reference/iommu/v20260222/iommu_intro.html

DIRECT SPEC PDFRISC-V IOMMU — direct specification PDF

Direct public PDF covering IOMMU translation, device/process contexts, queues, MSI translation, ATS/PRI integration and faults.

https://docs.riscv.org/reference/hardware/iommu/_attachments/riscv-iommu.pdf

KERNEL DOCSLinux VFIO — safe direct device access through IOMMU isolation

Public kernel documentation explaining why unrestricted DMA is the critical risk and how IOMMU isolation enables direct assignment of physical devices to userspace/virtual machines.

https://docs.kernel.org/driver-api/vfio.html

KERNEL DOCSLinux IOMMU userspace API

Current public documentation for guest IOVA/SVA, PASID binding, cache invalidation and virtualization-oriented IOMMU interaction.

https://docs.kernel.org/next/userspace-api/iommu.html

DMA coherency: IOMMU translation, cache coherence and memory ordering are three different problems

A DMA-capable device and a CPU can both access the same RAM, but that does not automatically mean they see the same bytes at the same instant. An IOMMU primarily translates/protects device addresses; DMA cache coherency determines whether CPU caches and device traffic automatically observe each other's changes; and memory ordering determines the order in which descriptor/data/register updates become visible.

THREE ORTHOGONAL QUESTIONS 1) ADDRESS / PROTECTION — IOMMU device IOVA → translation / permission → physical RAM address 2) CACHE COHERENCY CPU cache may contain newer/stale copy of a RAM cache line device DMA reads/writes RAM or coherent fabric question: are cache lines automatically snooped/kept coherent? 3) ORDERING CPU writes descriptor fields and then rings a device doorbell question: is the descriptor guaranteed visible before the doorbell? NON-COHERENT TX EXAMPLE CPU edits packet buffer in cache ↓ dma_sync_*_for_device / DMA mapping operation ↓ clean/write back required cache lines as platform needs ↓ device DMA reads correct bytes from memory NON-COHERENT RX EXAMPLE device DMA writes packet bytes to RAM ↓ driver waits for completion/ownership return ↓ dma_sync_*_for_cpu / unmap ↓ invalidate/update stale CPU cache lines as platform needs ↓ CPU reads fresh packet bytes COHERENT DESCRIPTOR EXAMPLE desc->addr = buffer_dma; memory barrier / dma_wmb(); desc->valid = 1; writel(doorbell, mmio); Even cache-coherent DMA can still require barriers.

Mechanism	Question it answers	Does not automatically solve
IOMMU	Which system physical pages may this device/requester access?	CPU-cache visibility or descriptor/register ordering.
coherent DMA	Will CPU/device memory accesses stay coherent without explicit cache flush/invalidate?	Ordering of independent stores or posted MMIO writes.
streaming DMA mapping	How should this buffer be handed to/from a device efficiently and portably?	Permanent simultaneous CPU/device ownership.
dma_sync_*	Transfer/restore buffer ownership and perform needed cache maintenance on noncoherent platforms.	Synchronization with a still-running device operation.
memory barrier	Which RAM/DMA-visible operations must become visible before later operations?	Flushing posted MMIO writes on every bus/platform.
MMIO accessor/readback	Perform device register access with architecture-specific ordering; a safe read may flush posted writes.	Ordinary RAM cache-coherency management.
dma-fence	Has an asynchronous device/GPU operation completed before another user starts?	CPU cache maintenance by itself.

Ownership is a useful mental model. With a streaming DMA buffer on a non-coherent system, the driver should treat the buffer as owned by the device after synchronizing it for the device, and should not casually touch it again until ownership is returned/synchronized for the CPU. Cache maintenance and asynchronous-operation synchronization are related but not identical.

KERNEL DOCSLinux — ordering memory-mapped I/O writes

Concrete example showing that posted MMIO writes can arrive later/out of order on some systems and that reading a safe register can flush pending writes.

https://docs.kernel.org/driver-api/io_ordering.html

KERNEL DOCSLinux dma-buf synchronization

Useful GPU/media extension: dma-buf shares buffers between devices/processes while dma-fence/dma-resv coordinate asynchronous work; cache coherency and operation completion are separate concerns.

https://docs.kernel.org/driver-api/dma-buf.html

Keeping a page resident is not the same as lending it to a device: mlock(), GUP and FOLL_PIN solve different problems

User memory can participate in low-latency CPU code, direct I/O, RDMA and device DMA, but those uses impose different contracts on the VM. mlock() is a userspace residency request: keep pages from being swapped out. A driver that needs stable pages for DMA instead uses the kernel's pin_user_pages*() family so the VM can track that page data is externally accessible. Long-lived DMA registrations are more restrictive again and use the long-term pin model.

USERSPACE BUFFER mmap()/malloc() virtual range ↓ ordinary state: page may fault in, be reclaimed/swapped when allowed, be COWed, migrated or unmapped according to VM rules CASE A — mlock(addr, len) ↓ mark mapping/pages locked against swapping/reclaim-to-swap subject to RLIMIT_MEMLOCK / CAP_IPC_LOCK rules ↓ CPU latency/residency guarantee ↓ NOT a device-DMA registration NOT a substitute for pin_user_pages() CASE B — short-lived direct I/O / DMA buffer driver receives userspace address ↓ pin_user_pages*() ↓ internally uses FOLL_PIN accounting obtain struct page / folio references ↓ dma_map_*() or subsystem mapping ↓ IOMMU/direct-DMA mapping gives device-visible address ↓ device DMA reads/writes page data ↓ completion DMA unmap + unpin_user_page(s) CASE C — long-lived RDMA-style registration pin_user_pages*() + FOLL_LONGTERM ↓ stricter eligibility / stronger VM impact ↓ register device translation/key state ↓ RNIC may DMA for an extended interval ↓ unregister → stop device access → unmap → unpin ALTERNATIVE ON CAPABLE ACCELERATORS MMU notifier + replayable device page faults / SVA ↓ device can track invalidations or fault pages on demand ↓ avoids pinning every page for the full lifetime in some designs

Mechanism	Contract
mlock()/mlock2()	Userspace API keeping selected virtual-memory pages resident in RAM; mainly about avoiding swap/page-in latency, not giving a device stable DMA ownership.
get_user_pages*() / FOLL_GET	Kernel references to userspace-backed pages for cases that do not require the DMA-pin tracking semantics.
pin_user_pages*() / FOLL_PIN	Kernel API for pages whose data will be accessed under DMA/direct-I/O-style pinning semantics; tracked distinctly so VM/filesystem code can notice possible pins.
FOLL_LONGTERM	Additional restriction for long-duration pins such as conventional RDMA registrations; it implies the FOLL_PIN model.
DMA mapping	Separate step converting pinned/owned memory into addresses and mappings valid for a particular device/IOMMU domain.
MMU notifier	Lets device/driver mappings react when CPU page tables or VM mappings are invalidated; important for designs that avoid permanent pins.
unpin	Ends the VM pin after device access has stopped, allowing normal memory-management actions to proceed again.

A virtual address is never by itself a DMA address. Pinning establishes a lifetime relationship with the underlying pages; the DMA/IOMMU API establishes the device-visible mapping. Conflating those two steps is a common source of incorrect “zero-copy” mental models.

PINNING / RESIDENCY LAB # mlock accounting for a process cat /proc/$$/status | grep -E 'VmLck|VmPin' # system-wide FOLL_PIN accounting where exposed grep -E 'nr_foll_pin_(acquired|released)' /proc/vmstat 2>/dev/null # Observe RDMA devices / registered-memory-capable hardware ls /sys/class/infiniband 2>/dev/null rdma link 2>/dev/null # Compare with ordinary mapping residency cat /proc/$$/smaps | less

KERNEL DOCSLinux pin_user_pages() / FOLL_PIN documentation

Current kernel explanation of FOLL_GET versus FOLL_PIN/FOLL_LONGTERM, DMA-pinned page accounting, direct-I/O and RDMA cases, MMU-notifier alternatives and why long-term pins have stronger restrictions.

https://docs.kernel.org/core-api/pin_user_pages.html

PUBLIC MANUALmlock(2) / mlock2(2) — memory residency

Current Linux manual for locking virtual-memory pages into RAM, MLOCK_ONFAULT, resource limits and the real-time/latency motivation. Read it specifically to contrast CPU residency with DMA pinning.

https://man7.org/linux/man-pages/man2/mlock.2.html

See also the existing DMA coherency/IOMMU section, DMA mapping section and RDMA section for the device side of the same lifetime.

The driver gives a device DMA addresses, not CPU pointers: scatter/gather, IOMMU merging and bounce buffers

A device normally cannot consume an arbitrary kernel virtual pointer. The Linux DMA API converts CPU-owned buffers/pages into DMA addresses valid for that particular device. The mapping layer may create IOMMU translations, merge adjacent scatter/gather entries, perform cache maintenance on noncoherent machines, or transparently allocate a SWIOTLB bounce buffer when the device cannot address the original memory.

FILESYSTEM / NETWORK / DRIVER HAS A LOGICAL 24 KiB BUFFER CPU-visible pages: page A at physical 0x81234000 4 KiB page B at physical 0x1f7a9000 4 KiB page C at physical 0x81235000 4 KiB page D ... scatterlist describes pages/offsets/lengths ↓ count = dma_map_sg(dev, sg, nents, DMA_TO_DEVICE) DMA LAYER OPTIONS 1. DIRECT DMA device can address physical region directly DMA addr may closely correspond to bus/physical address 2. IOMMU allocate IOVA range for this device/domain install IOVA → physical-page translations physically discontiguous pages can appear as fewer contiguous DMA segments 3. SWIOTLB BOUNCE original page is outside device DMA mask / must be isolated / encrypted-memory case ↓ allocate accessible temporary bounce buffer DMA_TO_DEVICE: CPU copies original → bounce before device owns it program device with bounce DMA address DMA_FROM_DEVICE: after completion, CPU copies bounce → original dma_map_sg() RETURNS MAPPED SEGMENT COUNT input nents = 6 physical/scatter entries returned count = 3 DMA segments ↓ driver programs hardware descriptors using ONLY: sg_dma_address(sg) sg_dma_len(sg) for the mapped result ON COMPLETION streaming mapping reused? dma_sync_*_for_cpu() before CPU accesses device-written data dma_sync_*_for_device() before returning ownership to device or final: dma_unmap_sg(dev, sg, ORIGINAL nents, direction) Important API asymmetry: use returned 'count' to PROGRAM hardware, but use original 'nents' to UNMAP the scatterlist.

DMA term	Meaning
dma_addr_t	Device-visible DMA address token; CPU must not treat it as an ordinary pointer.
DMA mask	Maximum address bits/range the device can generate for DMA.
scatterlist	Kernel list describing one logical buffer across one or more pages/segments.
dma_map_sg()	Maps a scatterlist into DMA segments for one device and direction; result count may be smaller than input count.
IOMMU merge	Multiple adjacent/contiguous logical segments can be represented as fewer device-visible IOVA segments.
streaming mapping	Temporary DMA ownership/mapping around a specific transfer; direction matters.
coherent mapping	DMA allocation/mapping whose CPU/device visibility obeys coherent-buffer semantics, but ordering barriers can still be necessary.
DMA_TO_DEVICE	CPU prepares data, then device reads it.
DMA_FROM_DEVICE	Device writes data, then CPU consumes it after required sync/unmap.
SWIOTLB	Linux software I/O translation/bounce layer allocating device-accessible temporary buffers.
bounce buffer	Temporary DMA-accessible buffer copied to/from an original buffer the device cannot safely access directly.
dma_sync_*()	Transfers/establishes buffer ownership/visibility between CPU and device for reusable streaming mappings.

DMA mapping can fail. IOVA space, bounce-buffer capacity, address masks and platform constraints are finite. Linux's DMA documentation explicitly requires drivers to test mapping failures rather than blindly programming an invalid descriptor.

READ-ONLY DMA/IOMMU OBSERVATION # DMA masks and device topology often appear in driver/debug output, not one universal sysfs field. # IOMMU groups: find /sys/kernel/iommu_groups -maxdepth 2 -type l 2>/dev/null | head -100 # boot-time IOMMU/SWIOTLB state dmesg | grep -Ei 'IOMMU|DMAR|AMD-Vi|swiotlb|bounce' | less # block-layer maximum segment/transfer constraints for f in /sys/block/*/queue/{max_segments,max_segment_size,max_sectors_kb}; do [ -r "$f" ] && echo "$f: $(cat "$f")" done # Driver source trail: # look for dma_map_sg(), dma_mapping_error(), sg_dma_address(), dma_unmap_sg(). # Do not disable the IOMMU or force swiotlb settings on a valuable system merely for observation.

KERNEL DOCSLinux Dynamic DMA Mapping Guide

Step-by-step current guide for streaming DMA, directions, scatterlists, map/unmap, returned segment counts and noncoherent cache-ownership rules.

https://docs.kernel.org/next/core-api/dma-api-howto.html

KERNEL DOCSLinux DMA and SWIOTLB

Current deep explanation of bounce buffering when a device cannot directly access original memory, including 32-bit addressing constraints and encrypted/confidential-computing guests.

https://docs.kernel.org/next/core-api/swiotlb.html

KERNEL DOCSLinux DMA Engine client API

Shows a real peripheral-DMA client flow where a driver first DMA-maps the scatterlist, then passes the mapped entries to a DMA engine until completion.

https://docs.kernel.org/driver-api/dmaengine/client.html

KERNEL DOCSLinux PCI peer-to-peer DMA

Advanced extension showing the same generic dma_map_sg() interface can also route eligible memory directly between PCI devices without first staging through normal host RAM.

https://docs.kernel.org/driver-api/pci/p2pdma.html

Hardware virtualization: guest privilege, VM exits and two-dimensional page translation

A virtual machine is not just an emulator copying every instruction in software. Modern CPUs contain virtualization extensions so most ordinary guest instructions can run directly on the real core, while privileged or configured-sensitive events transfer control to a hypervisor. Memory translation gains another stage so the guest OS can believe it owns 'physical' memory without being allowed to address arbitrary host RAM.

Guest application virtual address (GVA) ↓ guest page tables Guest physical address (GPA) ↓ second-stage translation: Intel EPT / AMD NPT / RISC-V G-stage Host / system physical address (HPA/SPA) ↓ real caches + DRAM ordinary guest instruction ↓ usually executes natively configured sensitive event / privileged operation / fault ↓ VM EXIT / trap to hypervisor ↓ hypervisor handles/emulates/configures state ↓ VM ENTRY / return to guest

Concept	Hardware role
guest mode / non-root execution	Runs guest OS/app instructions with controlled privilege.
VMCS / VMCB / virtualization state	Hardware-defined control/state structure describing what causes exits and what guest/host state to load.
VM exit	Hardware transition from guest execution to hypervisor because of configured event or exception.
VM entry	Hardware transition back into a guest context.
EPT / NPT / G-stage	Second translation stage mapping guest physical addresses into host/system physical addresses.
virtual interrupt	Hypervisor/hardware injects interrupt state into guest rather than exposing raw host interrupt wiring directly.
device emulation	Hypervisor intercepts MMIO/PIO/config accesses and provides software model of a virtual device.
device passthrough	Physical device is assigned more directly to a VM, normally requiring IOMMU protection and interrupt remapping.

OFFICIAL MANUALIntel SDM Volume 3C — VMX / Intel VT

Official volume specifically covering system-management mode, VMX instructions and Intel Virtualization Technology.

https://www.intel.com/content/www/us/en/content-details/868148/intel-64-and-ia-32-architectures-software-developer-s-manual-volume-3c-system-programming-guide-part-3.html

OFFICIAL MANUALAMD64 Architecture Programmer's Manual Volume 2 — System Programming

Current March 2026 AMD64 system-programming manual. Covers page translation/protection and later sections for nested paging/virtualization-related architecture.

https://docs.amd.com/api/khub/documents/sD1_QL~h4Afq2_tvzxqqSQ/content

OFFICIAL SPECRISC-V H extension — Hypervisor Support v1.0

Ratified public architecture describing HS/VS modes and the extra translation stage from guest physical to supervisor/system physical address.

https://docs.riscv.org/reference/isa/priv/hypervisor.html

KERNEL DOCSLinux KVM x86 documentation

Public implementation documentation for real hardware virtualization: nested VMX, shadow MMU, timekeeping virtualization, SEV/TDX and x86-specific behavior.

https://www.kernel.org/doc/html/latest/virt/kvm/x86/index.html

KERNEL DOCSKVM shadow MMU / EPT / NPT translation

Detailed address-path documentation explicitly showing guest virtual → guest physical → host physical translation and two-dimensional paging through Intel EPT or AMD NPT.

https://docs.kernel.org/7.0/virt/kvm/x86/mmu.html

Confidential VMs change the trust boundary: guest-private memory is protected even from the host VMM

Traditional hardware virtualization isolates one guest from another, but the host hypervisor still normally has enough privilege to inspect or modify guest RAM and CPU state. Confidential-computing VMs change that threat model. Technologies such as AMD SEV-SNP and Intel TDX use processor-managed keys, protected metadata and measured launch state so selected guest memory and execution state remain confidential—and, depending on the technology, integrity-protected—even if the host-side VMM is compromised.

NORMAL VM Guest private-looking RAM ↓ EPT/NPT managed by host host VMM can normally inspect/map guest pages CONFIDENTIAL VM measured guest launch ↓ hardware security manager / protected CPU mode ↓ guest PRIVATE pages ├─ encrypted with VM-specific key ├─ ownership / translation metadata checked by hardware └─ host cannot simply map plaintext BUT DEVICES / HOST COMMUNICATION STILL NEED SHARING private buffer ↓ guest explicitly converts/copies selected pages SHARED memory ↓ virtio / MMIO / DMA / hypercall interface ↓ untrusted host or device backend REMOTE ATTESTATION measurement + security-version state + verifier nonce ↓ signed / MAC-protected report or quote remote verifier ↓ release secret only if policy accepts measured VM state

Mechanism	What it changes
guest-private memory	CPU/memory-controller protection prevents the ordinary host mapping path from reading plaintext guest pages.
shared memory	Explicit communication region intentionally exposed to the host or device backend; confidential guests must treat its contents as untrusted.
measured launch	Initial guest/firmware state contributes to a measurement that can later be attested.
attestation report / quote	Evidence binding measurements and security/version state to a verifier-supplied challenge so a remote service can decide whether to trust the VM.
AMD SEV-SNP	Adds Secure Nested Paging, page ownership/integrity metadata, protected guest state and SNP attestation on AMD platforms.
Intel TDX	Runs Trust Domains behind the TDX module/SEAM boundary with private-vs-shared memory semantics and TD attestation.
denial of service	Generally remains under host control: the host can still pause the VM, withhold CPU, memory or I/O, or terminate it.

Confidential computing is not disk encryption and it is not ordinary measured boot. dm-crypt protects data at rest; TLS/IPsec protect data in transit; TPM measured boot records platform state. A confidential VM primarily protects data in use and execution state from a more powerful host-side adversary, then uses attestation so an external party can decide whether to provision secrets.

KERNEL DOCSLinux confidential-computing threat model for SEV-SNP/TDX

Current kernel security document defining the stronger VM threat model: private memory/register confidentiality and integrity against a potentially malicious host, while host-controlled resource denial remains out of scope.

https://docs.kernel.org/security/snp-tdx-threat-model.html

KERNEL DOCSLinux Intel TDX architecture support

Concrete guest/host explanation of TDX private versus shared memory, #VE handling, DMA sharing and the TDREPORT→Quote attestation path.

https://docs.kernel.org/arch/x86/tdx.html

OFFICIAL SPECAMD SEV-SNP Firmware ABI Specification

Current AMD publication for SNP launch, guest requests, attestation reports and firmware ABI behavior; revision 1.59 was released in August 2026.

https://docs.amd.com/v/u/en-US/56860_PUB_SEV_SNP

OFFICIAL DOCSIntel Trust Domain Extensions documentation hub

Current Intel TDX architecture, module, ABI, migration and attestation specifications collected in one official index.

https://www.intel.com/content/www/us/en/developer/tools/trust-domain-extensions/documentation.html

Virtio avoids emulating every device register: guest and host exchange buffers through virtqueues

Hardware virtualization gives a guest CPU and a protected memory view, but a virtual machine still needs disks, network interfaces, consoles and other devices. Fully emulating a particular physical NIC or storage controller can force the hypervisor to reproduce legacy register behavior and trap many accesses. virtio instead standardizes a paravirtual device interface designed for virtualization: the guest loads an ordinary virtio driver and exchanges descriptor chains with the device/backend through shared-memory queues called virtqueues.

GUEST LINUX DRIVER allocate request buffers in guest memory ↓ build virtqueue descriptor chain descriptor: address + length + flags + optional next ↓ publish descriptor head to DRIVER/AVAILABLE area ↓ required memory ordering notify / "kick" device only if needed ↓ VIRTIO TRANSPORT PCI / MMIO / CCW exposes configuration + queue notification ↓ HYPERVISOR / VHOST / HARDWARE VIRTIO DEVICE reads available descriptor chain ↓ performs device operation virtio-net: consume/produce packet buffers virtio-blk: read/write guest data buffers ↓ write completion to DEVICE/USED area ↓ interrupt / notification, often suppressible or coalesced ↓ GUEST DRIVER callback reclaims descriptors and completes I/O

Virtio piece	What it means
transport	How the virtio device is discovered and configured. Common transports include PCI and MMIO; the device model is intentionally separate from the transport.
feature bits	Driver and device negotiate optional behavior before normal operation. A feature may change queue format, offloads or device-specific capabilities.
virtqueue	Shared-memory queue used to pass buffers and completion state between driver and device.
descriptor	Describes one memory buffer and whether the device may read it or write it; descriptors can be chained for scatter/gather I/O.
split ring	Classic layout with descriptor table, driver/available ring and device/used ring in separate regions.
packed ring	Alternative compact layout where driver/device state is packed into one descriptor ring and negotiated with `VIRTIO_F_RING_PACKED`.
kick / notification	Tells the other side that queue state changed. Notification suppression reduces expensive VM exits or interrupts when polling/batching is sufficient.
`VIRTIO_F_ACCESS_PLATFORM`	Indicates that device-visible addresses must obey the platform DMA mapping/IOMMU rules rather than assuming raw guest physical addresses.

Virtio is not “copyless by definition.” Its main win is a standardized shared-memory queue protocol with batching and fewer emulated-register traps. Whether payload bytes are copied depends on the backend, IOMMU/mapping arrangement, networking/storage path and host implementation.

KERNEL DOCSLinux — Virtio on Linux

Current kernel documentation showing virtio transports, virtqueues and the guest-driver data structures used to register buffers for device consumption.

https://docs.kernel.org/driver-api/virtio/virtio.html

OFFICIAL SPECOASIS Virtio 1.4 specification

Current 2026 committee specification. Chapter 2 defines feature negotiation, notifications and both split and packed virtqueue formats; later chapters define virtio-net, virtio-blk and other device types.

https://docs.oasis-open.org/virtio/virtio/v1.4/virtio-v1.4.html

vhost accelerates virtio by moving the backend data path closer to the host kernel or a dedicated backend process

A guest using virtio always sees the same basic contract: negotiate features, publish buffers in virtqueues, notify the device/backend, then consume completions. The expensive question is where the host-side virtqueue work happens. QEMU can service queues itself in userspace, but Linux vhost can move much of that data path into the host kernel. vhost-user keeps QEMU as the virtual-device frontend/control coordinator while a separate userspace backend maps shared guest memory and consumes the virtqueues directly.

GUEST virtio-net / virtio-blk driver ↓ descriptors in guest RAM virtqueue available ring ↓ kick / eventfd-style notification BACKEND CHOICE A — QEMU USERSPACE QEMU vCPU exits / event loop notices queue ↓ QEMU reads descriptors + services backend ↓ completion → used ring → guest interrupt BACKEND CHOICE B — KERNEL VHOST QEMU configures vhost and supplies guest-memory / vring metadata ↓ host kernel vhost worker consumes virtqueue ↓ TAP/socket/block-style host path without QEMU handling every packet/request ↓ used-ring completion + notification to guest BACKEND CHOICE C — VHOST-USER QEMU frontend ⇄ Unix-domain control socket ⇄ backend daemon ↓ pass shared-memory fds / vring state / eventfds backend daemon maps guest-shared memory ↓ backend consumes virtqueues directly ↓ completion returned through the same virtio queue contract GUEST VIEW still a virtio device; the guest normally does not care which host backend implementation serviced the queue.

Layer	Role
virtio frontend	Guest-visible standardized device/queue ABI: features, configuration, virtqueues and notifications.
QEMU virtio device model	Can implement the backend itself or configure an accelerated backend while still exposing the guest-facing virtual device.
vhost	Linux host-kernel framework that can service virtqueues for selected virtio-style devices with fewer trips through QEMU’s userspace data path.
vhost-user	Protocol that lets QEMU share virtqueue/memory state with a separate userspace backend over a Unix-domain control channel.
eventfd / notification fd	Common host mechanism for signaling queue kicks/completions without polling every transition.
shared guest memory	Backend-visible mappings containing descriptors and payload buffers; access still has to obey the VM/IOMMU/memory-sharing model.

vhost is an implementation placement choice, not a different guest protocol. The performance benefit comes from reducing host scheduling/VM-exit/userspace traversal around virtqueue servicing. Whether payload bytes are copied, mapped, offloaded or handed to another host subsystem depends on the backend and transport.

QEMU DOCSQEMU — vhost-user protocol

Defines the frontend/backend control plane used to share virtqueue state, guest-memory mappings and notification file descriptors with an external backend process.

https://www.qemu.org/docs/master/interop/vhost-user.html

QEMU DOCSQEMU — vhost-user backends

Current device documentation showing how vhost-user services virtio device requests outside QEMU itself.

https://www.qemu.org/docs/master/system/devices/virtio/vhost-user.html

KERNEL DOCSLinux net_failover — virtio-net accelerated datapath context

Concrete Linux documentation using a vhost-backed virtio-net interface as a persistent paravirtual datapath alongside a faster passthrough VF.

https://docs.kernel.org/networking/net_failover.html

A memory balloon lets the hypervisor reclaim guest RAM by convincing the guest to give pages up voluntarily

A VM may be configured with more guest-physical memory than the host wants it to consume at every moment. Simply stealing arbitrary pages behind the guest's back would break the guest's page tables and allocator. A virtio balloon solves this cooperatively: the hypervisor asks the guest balloon driver to inflate, the guest allocates pages it can relinquish and reports them to the device, and the host can stop backing those guest-physical pages with ordinary host RAM. Deflation returns pages to the guest when memory is needed again.

HOST / HYPERVISOR wants guest to surrender memory ↓ virtio-balloon target num_pages increases ↓ configuration notification GUEST balloon driver ↓ allocate/select guest pages ↓ publish page-frame addresses on inflate virtqueue ↓ device acknowledges ↓ guest treats those pages as balloon-owned / unavailable ↓ HOST can reclaim or avoid backing corresponding guest memory LATER: guest should regain memory ↓ target balloon size decreases ↓ guest submits ballooned pages on deflate queue ↓ after required acknowledgement rules pages return to guest allocator RELATED BUT DISTINCT: FREE-PAGE REPORTING buddy allocator already has unused pages ↓ guest reports those free pages to virtualization backend ↓ host learns backing can be reclaimed ↓ reported pages return to guest free lists; they were not permanently ballooned

Mechanism	Meaning
inflate	Guest supplies pages to the balloon and stops using them as normal RAM, reducing memory effectively available inside the guest.
deflate	Previously ballooned pages are returned to the guest allocator.
balloon target	Desired balloon size communicated by the virtual device; the driver converges its actual balloon toward that target.
balloon compaction	Makes balloon-owned pages movable so normal memory compaction/migration is not unnecessarily blocked by their physical placement.
free-page reporting	Reports already-free guest pages to the host as reclaimable backing; unlike inflation, those pages remain logically free guest memory.
memory hotplug	Adds/removes guest-visible memory ranges at a coarser topology level; related to dynamic VM memory but not the same mechanism as a balloon.

Ballooning shifts pressure into the guest. If the host inflates a balloon aggressively, the guest may reclaim page cache, compact memory, swap, or invoke its OOM policy just as a smaller physical machine would. It changes how much RAM the guest can actively use; it does not make memory pressure disappear.

OPEN STANDARD PDFVirtio 1.4 — memory balloon device

Current OASIS virtio specification. The balloon device chapter defines inflate/deflate queues, target/actual page counts, statistics and free-page reporting features.

https://docs.oasis-open.org/virtio/virtio/v1.4/virtio-v1.4.pdf

KERNEL DOCSLinux free-page reporting

Explains the VM API commonly used by balloon-style virtualization drivers to report currently unused guest pages to a hypervisor.

https://docs.kernel.org/mm/free_page_reporting.html

KERNEL DOCSLinux page migration — movable balloon pages

Current page-migration documentation explicitly describes balloon pages as a movable non-LRU page type and connects balloon compaction with the VM migration machinery.

https://docs.kernel.org/mm/page_migration.html

RAM can appear and disappear at runtime: memory hotplug is add → online → migrate/offline → remove

Linux distinguishes physical memory being present from that memory being online and allocatable. Firmware, a hypervisor or a hot-pluggable DIMM can expose a new physical range; the kernel creates memory-block objects for it, then the range is onlined into an allocator zone before normal page allocation can use it. Removing memory reverses that process, but only after movable contents have been migrated elsewhere and no unmovable or pinned page prevents offlining.

ADD / ONLINE firmware or hypervisor exposes physical range ↓ kernel adds System RAM + memory-block devices ↓ block exists but may still be OFFLINE ↓ online / online_kernel / online_movable ↓ initialize page metadata + attach to allocator zone ↓ pages become allocatable OFFLINE / REMOVE select memory block ↓ stop new allocations from target range ↓ isolate pages ↓ migrate movable user/page-cache/huge pages elsewhere ↓ any unmovable or long-term pinned page left? ├─ yes → offlining fails / retries └─ no → block becomes OFFLINE ↓ platform may remove physical range

Term	Meaning
memory block	Sysfs hotplug-management unit such as `/sys/devices/system/memory/memoryXXX`; it represents a physical-address range, not one CPU page.
present vs online	Present memory exists in the physical map; online memory has been initialized for normal page allocation.
`online_kernel`	Online the range into an ordinary kernel-capable zone such as ZONE_NORMAL where unmovable kernel allocations may live.
`online_movable`	Online into ZONE_MOVABLE so only migration-compatible allocations are served there, improving future hot-remove reliability.
unmovable page	Page that cannot simply be relocated during offlining, such as some kernel allocations/page tables or architecture-specific objects.
long-term pin	DMA/RDMA/VFIO-style page pinning can prevent migration and therefore conflict directly with memory offlining.

“Hotpluggable” does not mean “guaranteed removable.” Linux can add memory very easily, but successful removal depends on evacuating the target physical range. ZONE_MOVABLE improves the odds by keeping most unmovable kernel allocations elsewhere; it does not eliminate every corner case.

KERNEL DOCSLinux — Memory Hot(Un)Plug administrator guide

Current user-facing documentation for adding/onlining/offlining memory blocks, sysfs state, automatic online policy, ZONE_MOVABLE and reasons offlining can fail.

https://docs.kernel.org/admin-guide/mm/memory-hotplug.html

KERNEL DOCSLinux — memory hotplug core API and notifier states

Kernel-side lifecycle documentation for MEM_GOING_ONLINE, MEM_ONLINE, MEM_GOING_OFFLINE and cancellation notifications used by subsystems that must react to changing physical memory.

https://docs.kernel.org/core-api/memory-hotplug.html

A running VM can move hosts by copying state faster than the guest can dirty it: live migration

Moving a powered-off VM is conceptually simple: copy its disks and saved machine state, then restart it elsewhere. Live migration tries to move the same state while guest CPUs keep executing. The difficult part is RAM: pages copied early may be modified again before switchover. A hypervisor therefore tracks dirty guest pages, repeatedly retransmits changed state, and eventually pauses the source briefly to transfer the final CPU/device state and any remaining dirty memory.

PRE-COPY LIVE MIGRATION source VM keeps running ↓ copy most guest RAM to destination ↓ meanwhile CPU/device writes dirty pages hardware/KVM/QEMU dirty tracking marks changed GPAs ↓ copy dirty pages again ↓ repeat while dirty rate is low enough to converge PAUSE SOURCE VM ↓ stop-and-copy final dirty RAM + vCPU state + virtual-device state ↓ destination loads final state ↓ START DESTINATION VM ↓ source is retired only after successful switchover IF DIRTY RATE ≈ OR EXCEEDS COPY RATE pre-copy may fail to converge ├─ throttle dirtying vCPUs / increase bandwidth / compress / multifd └─ optionally switch to POST-COPY POST-COPY send minimum execution/device state → start destination CPUs ↓ missing RAM access faults at destination ↓ request page from source ↓ install page + resume faulting vCPU ↓ background transfer remaining pages

State that must move	Why it matters
guest RAM	Contains application/kernel data and must represent one coherent point in execution despite concurrent writes during pre-copy.
vCPU architectural state	Registers, control state, interrupt state and virtualization metadata define exactly where execution resumes.
virtual device state	Queue indices, timers, interrupt state, emulated registers and in-flight protocol state must match the destination model.
dirty-page tracking	Records pages modified after they were copied so they can be sent again before switchover.
shared/persistent storage	May already be reachable from both hosts or may require separate block migration/replication; RAM migration alone does not move an arbitrary disk backend.
passthrough devices	Require explicit device migration support and DMA dirty tracking; an assigned physical device cannot be assumed migratable.

Pre-copy and post-copy trade failure behavior. Pre-copy keeps a complete runnable source until late in the process but can struggle when the guest dirties memory too quickly. Post-copy bounds repeated transfer by running the destination before all pages arrive, but the VM state is then split across hosts; losing the source or migration path can be much more serious.

QEMU DOCSQEMU migration documentation

Current entry point for QEMU migration: RAM and device-state transfer, transports, multifd, dirty limiting, post-copy, VFIO device migration and compatibility.

https://www.qemu.org/docs/master/devel/migration/

QEMU DOCSQEMU migration framework — save/load device state and transports

Explains the common save/restore machinery behind live migration and the byte-stream transports used to move VM state between QEMU instances.

https://www.qemu.org/docs/master/devel/migration/main.html

QEMU DOCSQEMU post-copy migration internals

Detailed state machine showing how destination CPUs can start before all RAM arrives, with missing pages faulted in from the source and the corresponding recovery/failure tradeoffs.

https://www.qemu.org/docs/master/devel/migration/postcopy.html

AF_VSOCK is a socket API for host↔guest communication that does not depend on the VM's IP network

A virtual machine often needs a control channel to its host even before DHCP, routing, firewall rules or a virtual NIC are configured. Linux VSOCK provides that channel with the ordinary socket programming model. Instead of an IP address and TCP/UDP port, a VSOCK endpoint is identified by a Context ID (CID) plus a port. Hypervisor transports such as virtio-vsock carry the bytes between guest and host.

GUEST PROCESS socket(AF_VSOCK, SOCK_STREAM, 0) ↓ connect({ cid = VMADDR_CID_HOST, port = 5000 }) ↓ Linux VSOCK core ↓ transport selected for this endpoint virtio-vsock / vhost-vsock / hypervisor-specific path ↓ shared-memory / virtqueue-style transport + notifications ↓ HOST VSOCK endpoint bind({ cid = local/ANY, port = 5000 }) listen() → accept() ↓ host service reads/writes ordinary stream bytes ADDRESSING CID = virtual-machine/host communication domain identifier port = service identifier inside that CID NO IP REQUIRED no ARP/NDP no IP route lookup no Ethernet MAC no TCP handshake The socket API looks familiar, but the transport boundary is the hypervisor/VM channel rather than an IP network.

Concept	Meaning
AF_VSOCK	Linux socket address family for communication between virtual machines and their host or supported peer domains.
CID	Context ID selecting a host/guest communication endpoint domain; it plays a role analogous to the host portion of a network address but is not an IP address.
port	Service endpoint within a CID, used with bind/connect much like a transport port.
SOCK_STREAM	Connection-oriented ordered byte stream exposed by VSOCK when supported by the underlying transport.
virtio-vsock	Virtio device/transport commonly used by a guest to exchange VSOCK traffic with the host.
vhost-vsock	Host-side acceleration/transport used by KVM-style virtualization stacks for guest↔host VSOCK.
network independence	The channel does not require configuring a guest NIC, DHCP, IP routes or an Ethernet topology.

VSOCK is not “TCP without Ethernet.” Applications get a familiar socket interface, but addressing and transport are virtualization-specific. If you need routable communication across ordinary networks, use IP sockets; VSOCK is for the local virtualization boundary.

OBSERVATION LAB # kernel support / protocol listing varies by system cat /proc/net/protocols | grep -i vsock 2>/dev/null # headers define address structures and special CIDs # /usr/include/linux/vm_sockets.h grep -E 'VMADDR_CID|sockaddr_vm' /usr/include/linux/vm_sockets.h 2>/dev/null # application pattern remains normal sockets: # socket(AF_VSOCK, SOCK_STREAM, 0) # bind/listen/accept OR connect # read/write/recv/send # Whether a CID is reachable depends on the loaded hypervisor/VSOCK transport.

PUBLIC MANUALvsock(7) — Linux VSOCK address family

Current Linux manual for AF_VSOCK, CID/port addressing, stream/datagram support and the host/guest communication model.

https://man7.org/linux/man-pages/man7/vsock.7.html

Giving a real PCIe device to a VM: SR-IOV, VFIO, IOMMU groups and interrupt remapping

A virtual machine can use an emulated device, a paravirtual device such as virtio, or—in selected setups—a real PCIe function directly. VFIO is Linux's framework for exposing direct device access to userspace/VMMs inside an IOMMU-protected boundary. SR-IOV helps by making one Physical Function (PF) expose multiple lighter-weight Virtual Functions (VFs), each with its own PCI identity/resources.

PHYSICAL NIC / GPU / ACCELERATOR PCIe Physical Function (PF) ↓ SR-IOV capability enabled PF remains management/control function ├→ VF0 appears as its own PCI B:D.F ├→ VF1 appears as its own PCI B:D.F └→ VF2 ... PASS VF1 TO GUEST host unbinds VF1 from ordinary host driver ↓ bind VF1 to vfio-pci / VFIO device interface ↓ VFIO checks isolation boundary / IOMMU group ↓ VMM maps guest memory into an IOMMU domain guest IOVA/GPA-visible DMA addresses ↓ IOMMU translation/protection host physical pages assigned/pinned/mapped for guest/device use GUEST DRIVER sees a normal-looking PCI VF ↓ guest programs VF BAR registers / queues ↓ VF DMA transactions are restricted by host IOMMU mappings ↓ MSI/MSI-X interrupts routed/remapped to guest virtual interrupt path WHY IOMMU GROUPS EXIST if two PCI functions cannot be independently isolated upstream (e.g. topology lacks ACS separation or multifunction backdoor concerns) ↓ they may belong to the same isolation group ↓ passing only one while host trusts the other may not be safe SR-IOV is not magic process isolation: PF firmware/driver often configures VF resources, queues, MAC/VLAN/rate policy, and device implementation quality still matters.

Virtualization object	Meaning
PF	SR-IOV Physical Function: full PCI function owning management/configuration of device virtualization resources.
VF	Virtual Function: lighter PCI function exposed by the PF for isolated datapath use.
VFIO	Linux userspace device-access framework designed to expose direct access through IOMMU protection.
IOMMU group	Smallest topology/security unit Linux can safely consider isolated for assignment under available hardware constraints.
vfio-pci	Generic VFIO PCI driver binding a host PCI function for userspace/VMM control.
guest IOVA	Address used by guest-visible device DMA before host IOMMU translation to host physical pages.
interrupt remapping	IOMMU/APIC feature restricting and translating device interrupt messages so assigned devices cannot arbitrarily target host CPUs/vectors.
ACS	PCIe Access Control Services that can help enforce upstream/routing separation between functions/ports.
sriov_numvfs	Linux sysfs control exposing how many VFs a supporting PF should instantiate.
sriov_totalvfs	Maximum number of VFs the PF/kernel reports as supported.
VF MSI-X allocation	Some devices partition a global interrupt-vector pool across VFs; current Linux exposes controls on supported devices.

Passthrough is safe only if DMA and interrupts are isolated too. A guest-controlled device that could DMA to arbitrary host RAM would bypass ordinary VM page-table protection completely. The IOMMU is therefore part of the security boundary, not just a performance feature.

READ-ONLY HOST INSPECTION # find SR-IOV-capable PFs find /sys/bus/pci/devices -name sriov_totalvfs -print -exec cat {} \; 2>/dev/null # inspect enabled/possible VFs for one PF cat /sys/bus/pci/devices/0000:BB:DD.F/sriov_totalvfs cat /sys/bus/pci/devices/0000:BB:DD.F/sriov_numvfs ls -l /sys/bus/pci/devices/0000:BB:DD.F/virtfn* 2>/dev/null # IOMMU group membership readlink /sys/bus/pci/devices/0000:BB:DD.F/iommu_group find /sys/kernel/iommu_groups -maxdepth 2 -type l 2>/dev/null | sort | less # current driver readlink /sys/bus/pci/devices/0000:BB:DD.F/driver 2>/dev/null # Enabling VFs, rebinding drivers or assigning devices can interrupt networking/storage/display. # Keep this lab read-only unless you have a disposable system and a recovery path.

KERNEL DOCSLinux PCI SR-IOV Howto

Stable public explanation of PF/VF enumeration and Linux sriov_numvfs control; VFs appear as ordinary hot-plugged PCI functions with their own BDF/MMIO resources.

https://docs.kernel.org/5.17/PCI/pci-iov-howto.html

CURRENT KERNEL ABILinux PCI sysfs ABI — current SR-IOV attributes

Current kernel ABI documentation for sriov_numvfs, sriov_totalvfs, sriov_drivers_autoprobe and per-VF MSI-X allocation controls.

https://docs.kernel.org/next/admin-guide/abi-testing.html

KERNEL DOCSLinux NIC SR-IOV APIs

Current network-driver view of configuring SR-IOV Virtual Functions and the shift toward switchdev-based management for modern NICs.

https://docs.kernel.org/networking/sriov.html

KERNEL DOCSLinux VFIO mediated devices

Shows the related case where hardware lacks native SR-IOV: VFIO can expose mediated device instances under an IOMMU-protected direct-access model.

https://docs.kernel.org/next/driver-api/vfio-mediated-device.html

USB enumeration: every device begins at endpoint 0 and tells the host what it is

USB is host-controlled and descriptor-driven. A newly attached device is not immediately 'a keyboard' or 'a flash drive' to the host. The host detects/reset-enables it, communicates through the mandatory default control endpoint 0, assigns a USB address, reads descriptors, selects a configuration, and only then binds interface drivers to usable endpoints.

COMMON USB 2.0-STYLE ENUMERATION SEQUENCE (simplified) device attaches / hub reports port connection ↓ host powers/resets port and establishes default control pipe ↓ device initially responds through default address / endpoint 0 ↓ GET_DESCRIPTOR(Device) → learn USB version, EP0 max packet, VID/PID, config count ↓ SET_ADDRESS → host assigns bus address ↓ more GET_DESCRIPTOR requests ├── full Device descriptor ├── Configuration descriptor tree │ ├── Interface descriptor(s) │ └── Endpoint descriptor(s) ├── String descriptors (optional/requested) └── class/BOS/etc. descriptors as applicable ↓ host chooses configuration / interface drivers ↓ SET_CONFIGURATION ↓ non-control endpoints become usable for class/device traffic CONTROL TRANSFER SETUP stage: 8-byte setup packet bmRequestType | bRequest | wValue | wIndex | wLength ↓ optional DATA stage, direction determined by request ↓ STATUS stage in opposite/no-data direction confirms completion All ordinary USB transfers are addressed to an ENDPOINT, not just vaguely 'to the device'.

USB transfer type	Typical purpose	Scheduling/reliability character
Control	Enumeration, standard/class/vendor requests, device configuration.	Structured SETUP/DATA/STATUS; endpoint 0 mandatory.
Bulk	Storage, printers, adapters, large reliable data.	Uses leftover bus time; retries/errors handled for reliable transfer.
Interrupt	Small latency-sensitive periodic state such as HID reports.	Host polls endpoint on scheduled interval; not an electrical device→host IRQ line.
Isochronous	Continuous time-sensitive streams such as audio/video.	Reserves/schedules bandwidth; tolerates packet loss rather than retrying late data indefinitely.

USB 'interrupt transfers' are named from the software/device-use perspective. On the bus, the USB host controller schedules/polls the endpoint at an interval; the peripheral does not asynchronously seize the USB wires like a legacy hardware interrupt line.

LINUX OBSERVATION LAB lsusb → bus/device address + VID:PID lsusb -t → hub/controller topology + class/driver/speed lsusb -v -d <vid>:<pid> → device/config/interface/endpoint descriptors lsusb.py -i -e → interface + endpoint-oriented view on newer usbutils installs cat /sys/kernel/debug/usb/devices → kernel text view of topology/descriptors (debugfs permissions required) usbmon can capture URB submissions/completions, including SETUP fields. Be careful: raw USB capture can expose keyboard/input or other sensitive traffic.

OFFICIAL SPECUSB-IF — USB 2.0 Specification

Official public USB 2.0 specification page. Chapter 9 is the core device-framework reference for descriptors, standard requests, states and endpoint 0 behavior.

https://www.usb.org/document-library/usb-20-specification

KERNEL DOCSLinux USB Host-Side API — current documentation

Public kernel reference for devices, configurations, interfaces, endpoints, URBs and synchronous control/bulk requests.

https://cdn.kernel.org/doc/html/latest/driver-api/usb/usb.html

KERNEL DOCSLinux USB Gadget API — enumeration from the device side

Excellent reverse viewpoint: gadget drivers must answer GET_DESCRIPTOR, support SET_ADDRESS and handle SET_CONFIGURATION before functional endpoints become active.

https://www.kernel.org/doc/html/latest/driver-api/usb/gadget.html

KERNEL DOCSLinux usbmon

USB equivalent of a low-level bus trace facility. It records URB submission/completion and decodes control SETUP fields such as bmRequestType, bRequest, wValue, wIndex and wLength.

https://docs.kernel.org/next/usb/usbmon.html

PUBLIC MANUALlsusb(8)

No-login USB inspection tool; verbose mode displays configuration/class descriptors and tree mode shows physical topology.

https://man7.org/linux/man-pages/man8/lsusb.8.html

PUBLIC MANUALlsusb.py(1) — interfaces/endpoints

Current upstream usbutils helper can display interface drivers and endpoint details directly.

https://man7.org/linux/man-pages/man1/lsusb.py.1.html

USB: host controller, enumeration, endpoints and packetized transfers

USB is also not 'just four wires.' It is a host-controlled protocol. The host detects attachment, resets and enumerates the device, reads descriptors, assigns an address, selects a configuration, and then schedules transactions to logical endpoints. Different endpoints can carry control, bulk, interrupt or isochronous traffic.

application / driver ↓ OS USB core ↓ host-controller driver ↓ xHCI/EHCI/OHCI/UHCI host controller hardware ↓ USB packets on differential physical link ↓ device controller ↓ endpoint 0 (control) + other IN/OUT endpoints ↓ device function (keyboard / storage / audio / camera / etc.)

KERNEL DOCSLinux USB Request Blocks (URBs)

Shows how software queues asynchronous USB work to endpoint queues and receives completion callbacks. Useful for connecting packet protocol to actual driver behavior.

https://docs.kernel.org/6.15/driver-api/usb/URB.html

USB Type-C is a connector state machine plus optional Power Delivery: orientation, roles, VBUS negotiation and alternate-mode routing

A USB-C receptacle is not just a reversible version of USB-A. The Configuration Channel (CC) pins let each end discover attachment/orientation and advertise or detect source/sink roles. The physical connector can carry USB 2.0, SuperSpeed/USB4-class signaling, power, and—when both ends support it—alternate-mode traffic. USB Power Delivery (PD) is a separate negotiation protocol that can select power contracts and exchange structured messages; the connector shape alone does not imply a particular data speed or wattage.

CABLE INSERTED ↓ CC termination detection ├── which plug orientation? ├── who is power Source / Sink? └── who is data Host / Device? (roles can be independent on capable ports) ↓ initial VBUS/current capability established ↓ if both sides support USB Power Delivery PD message exchange over the Type-C connection ↓ Source advertises capabilities Sink requests one supported operating point Source accepts / transitions supply ↓ POWER CONTRACT ↓ optional identity / SVID / mode discovery cable + partner capabilities learned ↓ optional Alternate Mode / USB4-style routing Type-C mux/switch routes high-speed lanes appropriately

Concept	What it controls	Common misconception
CC1 / CC2	Attachment, plug orientation, role/current signaling and the communications path used by USB-PD-capable ports.	They are not ordinary USB data pairs.
VBUS	Main power rail delivered through the connector under Type-C/PD rules.	A high-capability charger must not simply force an arbitrary high voltage onto every attached device.
power role	Which port is Source versus Sink for power.	Power role is not identical to USB host/device data role on dual-role-capable systems.
data role	Which side acts as host-facing DFP versus device-facing UFP in relevant USB modes.	Connector orientation does not determine data role.
e-marked cable	An electronically identified cable can report cable capabilities needed for some higher-power/high-speed operating modes.	All USB-C cables are not electrically equivalent.
Alternate Mode	Repurposes supported high-speed lanes for another protocol after capability/mode negotiation.	A USB-C-shaped port does not automatically support every alternate mode.

Keep connector, protocol and power capability separate. “USB-C” names the connector ecosystem; USB 2.0/USB 3.x/USB4 describe data transports; USB Power Delivery describes negotiated power/control behavior. A cable or port can support one subset without supporting every other feature.

CURRENT SPECUSB Type-C Cable and Connector Specification Release 2.5

USB-IF's April 2026 Type-C connector specification. Use it for the normative CC, cable/orientation, role and connector behavior rather than inferring capabilities from the receptacle shape.

https://www.usb.org/document-library/usb-type-cr-cable-and-connector-specification-release-25

CURRENT SPECUSB Power Delivery — current USB-IF specification package

USB-IF's current Power Delivery specification page; as of this revision it publishes USB PD Revision 3.2 Version 1.2. Follow the Source/Sink capability and contract state machines here when you need the actual protocol.

https://www.usb.org/document-library/usb-power-delivery

KERNEL DOCSLinux USB Type-C connector class

Concrete software view of Type-C ports, partners, cables, orientation switches, power operation mode and Alternate Modes as exposed through Linux drivers and /sys/class/typec.

https://docs.kernel.org/driver-api/usb/typec.html

xHCI: USB transfers are scheduled through DMA rings of TRBs

Modern PCs normally expose USB through an xHCI host controller. The USB driver stack describes work in host memory using Transfer Request Blocks (TRBs). Software produces Command and Transfer rings, rings MMIO doorbells, and the controller DMA-fetches those structures; the controller later writes Event TRBs to Event Rings so software can learn what completed.

ONE USB BULK/INTERRUPT TRANSFER THROUGH xHCI (conceptual) userspace read/write / class driver ↓ USB core builds/submits URB ↓ xHCI driver maps data buffer for DMA ↓ endpoint context identifies endpoint's TRANSFER RING ↓ software writes one or more Transfer TRBs ├── Normal TRB / Data Stage / Status Stage etc. ├── DMA buffer pointer ├── transfer length ├── Chain / IOC / direction-related fields └── Cycle bit indicates producer ownership generation ↓ memory barrier as required ↓ MMIO write to endpoint DOORBELL ↓ xHC DMA-fetches TRBs from host RAM ↓ xHC schedules packets on USB according to endpoint type/speed/timing ↓ USB device exchanges data ↓ xHC DMA writes/reads host data buffer ↓ completion/error → xHC writes Transfer Event TRB into EVENT RING ↓ MSI/MSI-X / interrupter notification ↓ xHCI driver consumes Event TRB ↓ complete URB → wake/callback upper driver/userspace Separate COMMAND RING: Enable Slot / Address Device / Configure Endpoint / Reset Endpoint / ... ↓ doorbell 0 xHC executes command ↓ Command Completion Event on Event Ring

xHCI structure	Producer	Consumer	Purpose
Command Ring	host software	xHC	Host-controller management commands such as Enable Slot, Address Device and Configure Endpoint.
Transfer Ring	host software	xHC	Describes USB transfer work and DMA buffers for an endpoint or stream.
Event Ring	xHC	host software	Returns command completion, transfer completion and other controller/device events.
TRB	depends on ring	depends on ring	Fixed-size descriptor carrying command/transfer/event parameters.
Transfer Descriptor (TD)	host software	xHC	One logical USB transfer represented by one or more chained Transfer TRBs.
Doorbell	host software	xHC MMIO register	Tells controller new command/endpoint transfer work is available.
Cycle bit	ring producer/consumer protocol	both sides	Distinguishes valid/current-generation entries as circular rings wrap.
Endpoint Context	host software/configuration commands	xHC	Stores endpoint type, dequeue pointer, max packet/burst and scheduling-related state.
Interrupter	xHC	CPU/software	Associates an Event Ring with interrupt moderation/notification state.

The xHCI ring is not the USB wire protocol. TRBs are a host-controller/software interface in system memory. The xHC translates scheduled Transfer Ring work into the appropriate USB transactions/packets on the physical bus and reports results back through Event TRBs.

OFFICIAL SPECIntel — current xHCI specification landing page

Official public source for current xHCI revisions. Intel currently lists xHCI Rev. 1.2c and 2.0 downloads.

https://www.intel.com/content/www/us/en/products/docs/io/universal-serial-bus/universal-serial-bus-specifications.html

DIRECT SPEC PDFIntel xHCI Rev. 1.2b specification (direct PDF)

Detailed host-controller architecture. Section 3.2.6 defines three ring classes: one Command Ring for the controller, Event Rings for interrupters, and Transfer Rings for endpoints/streams.

https://cdrdv2-public.intel.com/625472/625472_xHCI_Rev1_2b.pdf

OFFICIAL SPECIntel — xHCI Rev. 1.2c record

Current public Intel record for the newer 1.2c register-level host-controller specification.

https://www.intel.com/content/www/us/en/content-details/868295/extensible-host-controller-interface-for-universal-serial-bus-xhci-requirements-specification-r1-2c.html

SOURCE / FILESLinux xHCI driver source

Real source directory containing xhci-ring.c, xhci.c, xhci-mem.c and related code that creates TRBs, rings doorbells and consumes events.

https://github.com/torvalds/linux/tree/master/drivers/usb/host

One keypress end to end: switch → USB report → kernel event → program

A modern keyboard is itself a small computer. Pressing a key changes a switch matrix; keyboard firmware scans/debounces it and updates a HID report. The host controller receives USB traffic into memory, kernel HID/input code interprets the report, and user-space eventually receives a key event.

physical key switch closes ↓ keyboard MCU scans row/column matrix + debounces ↓ firmware updates HID keyboard report ↓ USB device controller transmits report when host polls/schedules transfer ↓ USB host controller DMA-writes received data into system RAM ↓ interrupt/completion kernel USB/HID driver parses report descriptor + report bytes ↓ Linux input subsystem emits EV_KEY / KEY_* events ↓ display server / terminal / application handles the key The printed character is NOT necessarily identical to the physical key code; keyboard layout, modifiers, compose/IME rules and application behavior can transform it.

DIRECT SPEC PDFUSB HID class specification (direct PDF)

The actual Human Interface Device class definition. Includes report protocol and the legacy/boot keyboard model. No account.

https://www.usb.org/sites/default/files/hid1_12.pdf

DIRECT SPEC PDFUSB HID Usage Tables — keyboard/keypad usages (direct PDF)

Official USB-IF usage definitions. The keyboard page assigns standardized usage identifiers to physical key positions/functions.

https://usb.org/sites/default/files/hut1_7.pdf

KERNEL DOCSLinux — introduction to HID report descriptors

Very useful public explanation of how HID descriptors describe the meaning and bit layout of reports and how the kernel turns report bytes into input events.

https://docs.kernel.org/6.16/hid/hidintro.html

KERNEL DOCSLinux — input event codes

Defines EV_KEY, KEY_A and the rest of the input-event protocol presented to user-space after lower-level device processing.

https://docs.kernel.org/input/event-codes.html

Terminal input is a kernel protocol stack: TTY, line discipline, PTY and the shell

A terminal is not merely “a stream of keyboard bytes.” Linux TTY objects sit between a character-producing endpoint and a process. The default N_TTY line discipline can implement canonical line buffering, echo, erase/kill editing and generation of job-control signals. A pseudoterminal (PTY) uses the same TTY machinery without a physical serial wire: a terminal emulator or SSH server owns the master side while the shell sees the slave side as its controlling terminal.

LOCAL TERMINAL EMULATOR keyboard/window events ↓ terminal emulator process PTY master fd (/dev/ptmx-created master) ⇅ kernel PTY pair PTY slave (/dev/pts/N) ↓ TTY core + N_TTY line discipline ├── ICANON: collect/edit a line before read() completes ├── ECHO: send typed characters back toward terminal display ├── ISIG: VINTR/VSUSP/etc. can generate signals for foreground pgrp └── termios: VMIN/VTIME, input/output transforms, flow control, etc. ↓ shell / interactive program reads stdin from slave TTY OUTPUT PATH shell write(1, "prompt", ...) ↓ slave TTY → line-discipline/output processing ↓ PTY master becomes readable terminal emulator reads master ↓ interprets terminal escape sequences ↓ draws glyphs in window SSH CASE remote terminal emulator ↔ encrypted network ↔ sshd ↓ PTY master ⇅ PTY slave ↓ shell The shell still sees an ordinary terminal abstraction.

TTY concept	Meaning
TTY	Kernel terminal object supporting terminal semantics, not necessarily a physical teletype or UART.
line discipline	Processing layer between low-level driver/PTY transport and userspace reads/writes; N_TTY is the normal default.
canonical mode	Input is line-oriented; editing characters are processed before the application receives the completed line.
raw/noncanonical mode	Applications can receive input without line assembly; exact behavior is controlled by termios flags plus VMIN/VTIME.
PTY master	Endpoint used by a terminal emulator, sshd, tmux, expect or similar controller.
PTY slave	Endpoint that behaves like a terminal device to the shell/application, commonly `/dev/pts/N`.
controlling terminal	TTY associated with a session; foreground process-group rules connect terminal-generated events to job-control signals.

Why Ctrl-C is not “ASCII 0x03 delivered to your program” in the usual terminal mode: with ISIG enabled, the line discipline recognizes the configured interrupt character and generates SIGINT for the foreground process group. Programs that switch the terminal to raw-like modes can instead request the byte-level behavior they need.

KERNEL DOCSLinux N_TTY line discipline

Kernel documentation for the default terminal line discipline, including receive/read/write/poll paths and internal buffering.

https://docs.kernel.org/driver-api/tty/n_tty.html

CURRENT PUBLIC MANUALtermios(3) — canonical, noncanonical, echo and signal controls

The userspace configuration contract for terminal behavior: ICANON, ECHO, ISIG, VMIN/VTIME and related input/output flags.

https://man7.org/linux/man-pages/man3/termios.3.html

CURRENT PUBLIC MANUALpty(7) — pseudoterminals

Explains master/slave PTYs and why terminal emulators, SSH, screen/tmux and similar software use them.

https://man7.org/linux/man-pages/man7/pty.7.html

# Inspect which terminal the shell uses: tty # Show termios settings: stty -a # Compare canonical input with a program that temporarily enables raw mode. # Run experiments in a disposable shell because bad terminal settings can make # the current terminal appear unresponsive until restored with: stty sane

A shell job is a process group inside a session: foreground control belongs to the terminal, not to one PID

Unix job control adds a hierarchy above individual PIDs. A shell normally places every process in one pipeline into the same process group, and all process groups controlled by that shell live in one session. The controlling terminal records one foreground process-group ID. That is why Ctrl-C can interrupt an entire pipeline and why a background job that tries to read from the terminal is stopped as a group.

INTERACTIVE SHELL SESSION session leader / shell SID = shell PID controlling terminal = /dev/pts/N ↓ foreground PGID stored in TTY │ ├── job 1: pipeline A | B | C │ A PID 4101 ┐ │ B PID 4102 ├─ PGID 4101 ← foreground │ C PID 4103 ┘ │ └── job 2: long_task & PID 4201 ─ PGID 4201 ← background User types Ctrl-C TTY line discipline generates SIGINT ↓ signal targets foreground process group 4101 ↓ A, B and C each receive SIGINT according to their dispositions Background job calls read(tty) ↓ kernel sends SIGTTIN to background process group ↓ job stops until shell moves it foreground / continues it

Mechanism	Role
process group (PGID)	Groups related processes for signaling and terminal job control; a pipeline is typically one process group.
session (SID)	Groups process groups and can own one controlling terminal.
session leader	Process that created the session with `setsid()`; its PID becomes the SID.
controlling terminal	TTY associated with the session; remembers which process group is foreground.
`setpgid()`	Creates/joins process groups, subject to session/job-control rules.
`tcsetpgrp()`	Changes which process group is foreground for a controlling terminal.
SIGTTIN / SIGTTOU	Terminal-generated stop behavior for disallowed background terminal reads/writes.
SIGTSTP / SIGCONT	Stop/continue signals used by interactive shells for suspend/resume.

kill(-pgid, sig) is conceptually different from killing one PID. Job control is intentionally group-oriented so a pipeline behaves like one interactive job even though it contains several processes.

CURRENT MANUALsetpgid(2) / getpgrp(2) — process groups and job-control rules

Current Linux man-pages description of process-group creation, sessions, foreground/background terminal behavior, SIGTTIN and shell job control.

https://man7.org/linux/man-pages/man2/setpgid.2.html

CURRENT MANUALsetsid(2) — create a new session

Defines session creation, the new process group created with it, and the initial absence of a controlling terminal.

https://man7.org/linux/man-pages/man2/setsid.2.html

CURRENT MANUALtcgetpgrp(3) / tcsetpgrp(3) — terminal foreground PGID

The interface a shell uses to hand the controlling terminal to a foreground job and later take it back.

https://www.man7.org/linux/man-pages/man3/tcsetpgrp.3.html

Ethernet hardware path: CPU → DMA → MAC → PHY → magnetics → cable

A network interface contains several conceptually different layers. The MAC deals with Ethernet frames and digital link-layer behavior. The PHY translates between digital MAC-side data and the continuous-time electrical signaling on the cable. A real NIC commonly uses DMA rings/descriptors so packet bytes move between RAM and the interface without the CPU copying each byte.

CPU / driver ↓ configures descriptors, queues, registers system RAM ↕ DMA Ethernet MAC ↕ MII / RMII / GMII / RGMII / SGMII Ethernet PHY ↕ analog transmit/receive circuitry magnetics / transformer interface ↕ twisted-pair cable / connector PHY management often uses separate MDC/MDIO register interface

DIRECT PDFTI — Ethernet PHY basics (direct PDF)

Very good concise hardware explanation. Shows the PHY's digital MAC-side interface (MII/RMII/GMII/RGMII/SGMII) and its analog media-side interface, with pin-count/speed tradeoffs.

https://www.ti.com/lit/ta/sszt321/sszt321.pdf

DIRECT PDFTI — CPU/MAC/PHY/DMA Ethernet architecture (direct PDF)

The opening diagram explicitly separates CPU, DMA, MAC and PHY. Good for understanding which block owns frames versus analog cable signaling.

https://www.ti.com/lit/pdf/sprad07

PUBLIC DOCSMicrochip — MAC/PHY and DMA abstraction model

Public documentation showing TX/RX DMA engines, buffer management, MAC filtering and the separate MII/RMII data path to an external PHY.

https://onlinedocs.microchip.com/oxy/GUID-B1AF6B2B-BC62-4D92-B329-A50140DF3437-en-US-4/GUID-F7839D3A-8587-4692-9177-B06FA803FF4D.html

PUBLIC DOCSMicrochip — PHY management through MDIO/MIIM

Explains that xMII carries packet data while a separate two-wire MDC/MDIO management interface accesses PHY registers.

https://onlinedocs.microchip.com/oxy/GUID-96763FAC-79CB-438A-AE34-CD3D4F0EAF71-en-US-1/GUID-123B0674-66D9-4BE7-AF33-7A1BFCD50B33.html

An Ethernet frame still has to become symbols and voltages: MAC, PCS, PMA/PMD and auto-negotiation

The Ethernet MAC handles frame-level logic—addresses, lengths, CRC/FCS, pause/control behavior and DMA-facing queues—but it does not directly drive twisted-pair magnetics in most systems. Between MAC and cable are physical-layer blocks: a MAC↔PHY chip interface such as MII/RMII/RGMII/SGMII, a Physical Coding Sublayer (PCS), a Physical Medium Attachment (PMA), and a medium-dependent analog front end.

TRANSMIT PATH — CONCEPTUAL GIGABIT ETHERNET NIC driver / DMA TX descriptor ↓ MAC builds/transmits Ethernet frame preamble/SFD + destination/source + EtherType/length + payload + FCS ↓ MAC↔PHY interface examples: GMII / RGMII / SGMII / USXGMII ↓ PCS — Physical Coding Sublayer line coding / block coding / alignment / in-band negotiation functions exact encoding depends on Ethernet PHY type/speed ↓ PMA — Physical Medium Attachment serialization/deserialization, clock recovery / lane functions as applicable ↓ PMD / analog PHY front end electrical/optical signaling specific to copper/fiber/backplane medium ↓ magnetics/connector/cable for BASE-T copper where applicable RECEIVE reverses the stack: analog waveform → PMD/PMA → recovered/aligned symbols → PCS decode → MAC frame/FCS handling → RX DMA descriptor → RAM AUTO-NEGOTIATION link partner A advertises supported speeds/duplex/pause/etc. link partner B advertises its supported set ↓ PHY/PCS negotiation chooses highest mutually supported valid mode ↓ link training/configuration where required by PHY type ↓ MAC is configured for resolved speed/duplex/pause parameters ↓ carrier/link becomes usable On Linux, phylink coordinates MAC, optional PCS and PHY/SFP link state.

Ethernet block/interface	What it does
MAC	Frame-level transmit/receive logic, FCS, flow control, DMA-facing queues and link-state configuration.
MII/RMII/GMII/RGMII	Parallel-ish on-board digital interfaces between MAC and an external PHY at various Ethernet speeds.
SGMII/USXGMII	Serial MAC↔PHY/PCS interfaces often carrying in-band link-status/speed information.
PCS	Physical Coding Sublayer: code groups/blocks, alignment, lane/symbol functions and sometimes in-band auto-negotiation.
PMA	Physical Medium Attachment: serializer/deserializer and timing/lane attachment functions between PCS and medium-specific circuitry.
PMD	Physical Medium Dependent portion implementing actual copper/fiber/backplane transmit/receive signaling.
MDIO/MDC	Low-speed management interface used by software/MAC side to read/write PHY registers.
auto-negotiation	Protocol for link partners to advertise capabilities and resolve a mutually supported operating mode.
link training	PHY-specific adaptation/calibration process used by some high-speed copper/backplane links after/beside negotiation.
pause resolution	Negotiated IEEE 802.3 flow-control capability that the resolved link state may pass into MAC configuration.
magnetics	Isolation/impedance coupling transformers commonly used on BASE-T copper Ethernet between PHY analog pins and connector.
carrier/link state	Kernel/network indication that the PHY/PCS/MAC path has established a usable physical link.

There is no single universal Ethernet encoding. 10BASE-T, 100BASE-TX, 1000BASE-T, 1000BASE-X, 10GBASE-R/KR and newer PHYs use different PCS/PMA/PMD mechanisms. The stable mental model is the layering: frame logic at the MAC, digital physical coding/attachment below it, then the medium-specific electrical/optical transceiver.

LINUX ETHERNET PHY LAB # negotiated link mode, partner advertisement, speed and duplex ethtool eth0 # extended link state when driver supports it ethtool --show-eee eth0 2>/dev/null ethtool --show-fec eth0 2>/dev/null # PHY statistics, cable test or module EEPROM where supported ethtool --phy-statistics eth0 2>/dev/null ethtool --cable-test eth0 2>/dev/null ethtool -m eth0 2>/dev/null | less # driver/MAC identity ethtool -i eth0 # network carrier as exposed by kernel cat /sys/class/net/eth0/carrier 2>/dev/null cat /sys/class/net/eth0/speed 2>/dev/null cat /sys/class/net/eth0/duplex 2>/dev/null # Don't force speed/duplex on a remote/important link unless you can recover it.

KERNEL DOCSLinux phylink — MAC/PCS/PHY coordination

Kernel design documentation explicitly describes optional PCS blocks handling encoding/decoding, link establishment and auto-negotiation, with phylink coordinating resolved speed/duplex/pause into the MAC.

https://docs.kernel.org/6.9/networking/sfp-phylink.html

KERNEL DOCSLinux PHY abstraction layer

Current PHY-layer interface catalog spanning MII/RGMII/SGMII/USXGMII/1000BASE-X/KX and many other MAC↔PHY/PCS modes.

https://docs.kernel.org/networking/phy.html

KERNEL DOCSLinux twisted-pair Ethernet Layer-1 diagnostics

Current diagnostic guide for cable/PHY problems, auto-negotiation mismatches, partner advertisements, speed/duplex and ethtool-based inspection.

https://docs.kernel.org/networking/diagnostic/twisted_pair_layer1_diagnostics.html

KERNEL DOCSLinux ethtool netlink interface

Current kernel API exposes supported/advertised/partner link modes, auto-negotiation status, resolved speed/duplex, master/slave state and lane information.

https://docs.kernel.org/networking/ethtool-netlink.html

OFFICIAL DOCSMicrochip — Gigabit Ethernet PHY example

Concrete current platform example: LAN8840 triple-speed 10/100/1000 PHY connected to a host MAC over RGMII with auto-negotiation enabled by default.

https://onlinedocs.microchip.com/oxy/GUID-F48EA780-526B-4D58-B933-0E800F9DCA1A-en-US-4/GUID-F6B44988-C28B-4F78-B480-C5DBC8E2DDD2.html

OFFICIAL DOCSMicrochip — Ethernet auto-negotiation testing

Practical public PHY-side view of configuring advertised modes, restarting auto-negotiation and checking negotiated link status.

https://onlinedocs.microchip.com/oxy/GUID-E4098E15-180F-4086-BC4A-070E637A8B56-en-US-1/GUID-B7445640-86EE-4040-90AD-EAA07D5D8F90.html

Wi-Fi is a shared radio link, not Ethernet with the cable removed: scan → authenticate → associate → contend → ACK/retry

An 802.11 station must discover and join a Basic Service Set before ordinary IP traffic can flow. It can learn candidate networks from beacons or active probe exchanges, perform 802.11 authentication, associate with an access point, and—on protected networks—complete the relevant security/key establishment. Only after link-layer setup does normal network configuration such as DHCP or IPv6 Router Advertisement/SLAAC begin.

USERSPACE NETWORK MANAGER / wpa_supplicant / iw ↓ Generic Netlink: nl80211 cfg80211 ↓ driver / mac80211 or firmware-managed FullMAC path WIRELESS NIC / RADIO DISCOVERY select channels / scan ├── passive: listen for BEACONS └── active: send PROBE REQUEST → receive PROBE RESPONSE ↓ choose BSS (SSID + BSSID + channel + capabilities) ↓ 802.11 AUTHENTICATION ↓ ASSOCIATION REQUEST / RESPONSE ↓ protected WLAN: establish/install link keys as required ↓ LINK IS UP — but there may still be no IP address ↓ DHCPv4 and/or IPv6 RA/SLAAC/DHCPv6 ↓ IP packets can flow ONE DATA TRANSMISSION ON A BUSY CHANNEL network-stack skb ↓ 802.3 ↔ 802.11 framing/encapsulation as appropriate 802.11 data frame queued ↓ carrier sense + contention/backoff (CSMA/CA family) ↓ radio transmits PHY frame ↓ receiver decodes + checks frame ↓ link-layer ACK when required ├── ACK arrives → success / rate-control feedback └── no ACK → retry subject to retry/rate policy

802.11 concept	Role
SSID	Human-facing network/service-set name; not itself the unique radio-interface address.
BSSID	Identifier for a particular Basic Service Set, commonly tied to an AP radio interface in infrastructure mode.
beacon	Periodic management frame advertising the BSS, timing and capabilities.
probe request/response	Active scanning exchange used to discover nearby BSSes/capabilities.
authentication	802.11 management step preceding association; do not confuse this term with the complete WPA/WPA2/WPA3 security process.
association	Establishes station membership/state with an AP so data service can begin.
CSMA/CA / backoff	Shared-medium access logic: stations sense the medium and randomize access rather than assuming a dedicated full-duplex wire.
link ACK / retry	Per-frame wireless reliability mechanism for unicast traffic; independent of TCP's end-to-end acknowledgments/retransmissions.
rate control	Selects modulation/coding/rate based on link conditions and observed delivery behavior.
802.11 ↔ 802.3 conversion	Wireless frames have different headers/addressing from Ethernet; driver/mac80211/firmware may translate before packets enter the ordinary Ethernet-like Linux network stack.

# Wireless interfaces and capabilities iw dev iw phy # Current association / BSSID / signal / bitrate iw dev wlan0 link # Scan results (usually requires suitable privileges) sudo iw dev wlan0 scan | less # Watch nl80211 wireless events iw event # Link-layer/IP state are separate ip link show wlan0 ip addr show wlan0

Association is not Internet connectivity. A station can be successfully associated at layer 2 yet have no usable IP configuration, DNS, route, or upstream connectivity. Conversely, TCP retransmission does not replace 802.11's local link ACK/retry mechanism; both can operate at different layers for different failure scopes.

KERNEL DOCSLinux cfg80211 — scan/authenticate/associate control plane

Current kernel documentation for the 802.11 configuration layer, including scanning/BSS tracking, authentication, association, regulatory handling and data-frame conversion helpers.

https://docs.kernel.org/driver-api/80211/cfg80211.html

PUBLIC DOCSLinux Wireless — mac80211 architecture

Developer documentation for the software-MAC layer used by many Linux Wi-Fi drivers, with links to authentication/association sequence diagrams, receive/transmit processing, rate control and key handling.

https://wireless.docs.kernel.org/en/latest/en/developers/documentation/mac80211.html

PUBLIC DOCSLinux Wireless — iw / nl80211 inspection

No-account command reference for inspecting wireless PHYs/interfaces, scanning, connecting and watching nl80211-backed wireless state.

https://wireless.docs.kernel.org/en/latest/en/users/documentation/iw.html

OFFICIAL STANDARD INDEXIEEE 802.11 Working Group — freely downloadable published standards

Official working-group index. IEEE makes published 802.11 standards available for free download after the program's release delay; use this when you want the normative MAC/PHY definitions behind the Linux implementation.

https://www.ieee802.org/11/

Bluetooth LE is a layered radio protocol stack: advertising → connection → ATT/GATT → application data

Bluetooth Low Energy does not begin with an IP address or an Ethernet-like link. A controller transmits and receives link-layer advertising and connection events on the 2.4 GHz radio. The host controls the controller through HCI, while higher layers such as ATT and GATT give applications a structured way to discover services and exchange characteristic values. Linux normally exposes controller management through BlueZ and kernel Bluetooth sockets, not by pretending the device is an ordinary IP NIC.

APPLICATION / DESKTOP SERVICE ↓ BlueZ D-Bus API BlueZ bluetoothd ↓ management / HCI sockets LINUX BLUETOOTH SUBSYSTEM ↓ HCI transport (USB / UART / PCIe-integrated path, device-dependent) BLUETOOTH CONTROLLER ↓ 2.4 GHz RADIO DISCOVERY / CONNECTION (LE) peripheral advertises on advertising PHY/channels ↓ central scans and receives advertising data ↓ optional CONNECT request / controller procedure LE connection established ↓ link-layer connection events + adaptive channel use ↓ L2CAP logical channels ↓ ATT request / response / command / notification / indication ↓ GATT service → characteristic → descriptor hierarchy ↓ application interprets characteristic bytes PAIRING / BONDING WHEN REQUIRED SMP negotiates authentication/key establishment ↓ link encryption can protect later traffic ↓ keys may be retained as a bond for future reconnection

Layer / object	What it does
advertising	Broadcast-style link-layer transmissions that let scanners discover devices/services and, depending on advertising mode, initiate a connection.
HCI	Host Controller Interface between host software and the Bluetooth controller; commands/events/data may travel over USB, UART or another controller transport.
LE Link Layer	Controls advertising, scanning, connections, channel selection, acknowledgments/retransmission and radio timing below host protocols.
L2CAP	Multiplexes logical channels above the controller/link layer and carries protocols including ATT.
ATT	Attribute Protocol: transports operations on a peer's typed attribute table, including reads, writes, notifications and indications.
GATT	Generic Attribute Profile: organizes ATT attributes into services, characteristics and descriptors with discovery conventions.
SMP / bonding	Security Manager procedures establish/authenticate keys; bonding means retaining suitable keys/state for later relationships.
BlueZ	Linux userspace Bluetooth stack/daemon and APIs that translate application operations into kernel/controller management and profile behavior.

# Linux controller state (BlueZ tools, if installed) bluetoothctl list bluetoothctl show # Interactive discovery / connection bluetoothctl # then: scan on # devices # info <device> # connect <device> # Low-level Bluetooth monitor when available sudo btmon # Do not expect an IP address merely because a BLE connection exists: ip addr

Bluetooth LE connection ≠ IP connectivity. ATT/GATT applications commonly exchange data directly over the Bluetooth protocol stack with no IP layer at all. IP-over-Bluetooth profiles exist, but they are separate profile choices rather than the default meaning of “connected.”

OFFICIAL SPECBluetooth Core Specification 6.3 — current adopted core

Official current adopted Bluetooth Core specification. Use the HTML/PDF volumes for LE Link Layer, HCI, L2CAP, ATT, SMP and the normative GATT architecture.

https://www.bluetooth.com/specifications/specs/core-specification-6-3/

PUBLIC DOCSBlueZ Bluetooth Management API

Documents Linux's management socket protocol for discovering controllers, powering/configuring them, scanning, pairing-related state and connection events.

https://bluez.readthedocs.io/en/latest/mgmt-api/

PUBLIC DOCSBlueZ GATT D-Bus API

Shows how local and remote GATT services, characteristics and descriptors are represented to Linux applications and how ATT operations map into D-Bus methods/properties.

https://bluez.readthedocs.io/en/latest/gatt-api/

PUBLIC DOCSBlueZ Device1 API

Application-facing device connection, pairing, trust and service-resolution state for Bluetooth peers.

https://bluez.readthedocs.io/en/latest/device-api/

One TCP send: bytes become TCP segments, IP packets and Ethernet frames

A socket does not put 'TCP directly on the wire.' Each layer adds its own addressing/control information for a different scope. TCP identifies the connection and reliable byte-stream state; IP chooses an internetwork destination/route; Ethernet delivers a frame across one local link to the next-hop MAC address.

APPLICATION send(sock, 'hello', 5) ↓ TCP stream bytes assigned sequence numbers TCP header adds src/dst ports, seq/ack, flags, window, checksum ↓ IP adds source/destination IP addresses + hop/routing fields route lookup chooses output interface + NEXT HOP ↓ NEIGHBOR RESOLUTION IPv4 local/next-hop IP → ARP cache cache miss → broadcast ARP request reply teaches next-hop 48-bit Ethernet MAC IPv6 uses Neighbor Discovery instead of ARP ↓ ETHERNET dst MAC = next-hop MAC src MAC = local interface MAC EtherType identifies payload (e.g. IPv4/IPv6) FCS/CRC protects frame over this link ↓ NIC TX ring / DMA / MAC / PHY → cable/fiber/radio-link bridge At a ROUTER: old Ethernet frame terminates IP packet is forwarded according to routing new link-layer frame is created for the next hop TCP connection is end-to-end between hosts. Ethernet MAC destination is only the next hop on this link.

Identifier/state	Scope / purpose
TCP source/destination port	Identifies transport endpoints/process-facing service within the IP hosts.
TCP sequence number	Positions bytes in the reliable ordered byte stream and drives acknowledgment/retransmission.
IP address	End-system/network-layer address used by routing across multiple networks.
route / next hop	Chooses which neighbor/interface should receive the packet next.
Ethernet MAC address	Local-link hardware address used to deliver the current frame to the next Ethernet hop.
EtherType	Says what protocol is encapsulated in an Ethernet frame payload, e.g. IPv4 or IPv6.
ARP cache / neighbor table	Caches mapping from IPv4 next-hop protocol address to local-link MAC address.
Ethernet FCS	Detects corruption over a particular Ethernet link; normally stripped/validated by MAC/NIC before the normal IP stack sees payload.

Reliability belongs to the appropriate layer. Ethernet FCS detects damaged local frames but does not create an end-to-end reliable stream. IP itself is best-effort. TCP adds end-to-end sequencing, acknowledgments, retransmission and flow/congestion-control behavior above IP.

INTERNET STANDARDRFC 9293 — current TCP Internet Standard

Modern consolidated TCP specification replacing RFC 793 for core TCP requirements; covers reliable byte-stream sequence/acknowledgment behavior.

https://www.rfc-editor.org/info/rfc9293/

PUBLIC RFCRFC 826 — Ethernet Address Resolution Protocol

Original ARP specification showing the concrete problem: an IP/protocol address must be translated to a 48-bit Ethernet destination address for local transmission.

https://www.rfc-editor.org/rfc/rfc826.html

PUBLIC RFCRFC 791 — IPv4

Classic IPv4 specification explaining datagrams, addressing and routing through an interconnected set of networks.

https://www.rfc-editor.org/rfc/rfc791.html

INTERNET STANDARDRFC 8200 — IPv6 Internet Standard

Current base IPv6 specification; useful after the IPv4/ARP path to understand 128-bit addressing and the modern network-layer successor.

https://www.rfc-editor.org/info/rfc8200/

KERNEL DOCSLinux networking API

Kernel-level bridge from sockets/sk_buffs/network devices to the packet-processing and transmit machinery used by Linux.

https://docs.kernel.org/networking/kapi.html

A machine cannot route ordinary traffic until somebody gives it network configuration: DHCP bootstraps the IPv4 host

When an IPv4 interface first comes up, the host may not yet know its usable address, subnet mask/prefix, default router or DNS server addresses. DHCP solves that bootstrap problem with a client/server exchange carried over UDP. The initial client message can be sent before the client has a normal unicast IPv4 identity, which is why DHCP uses well-known client/server ports and broadcast-capable behavior.

LINK COMES UP ↓ client has link-layer identity but no leased IPv4 address yet ↓ DHCPDISCOVER client UDP 68 → server UDP 67, commonly broadcast on the local link ↓ DHCPOFFER server proposes an address + lease + configuration options ↓ DHCPREQUEST client identifies the offer/address it wants ↓ DHCPACK server commits/acknowledges the lease and parameters ↓ client configures interface address/prefix ├── installs connected/on-link route ├── installs default/classless routes if supplied ├── records DNS resolver addresses if supplied └── starts lease timers T1: normally begin renewal with the original server T2: if renewal failed, broaden to rebinding lease expiry: stop using the address unless renewal/rebinding succeeded A DHCP relay can forward messages between a client LAN and a server on another network, so the server itself need not be on every broadcast domain.

DHCP item	What it accomplishes
UDP 67 / 68	Well-known DHCP/BOOTP server and client ports used during configuration exchange.
lease	Time-bounded permission for the client to use an assigned IPv4 address; renewal extends that lifetime.
subnet mask / prefix	Tells the host which IPv4 destinations are considered directly reachable on the local link.
router option	Can supply one or more on-link routers that become candidate default gateways.
DNS server option	Can supply recursive resolver addresses later used by the hostname-resolution path.
classless static routes	Modern DHCP can supply destination-prefix/next-hop routes rather than only one default router.
relay agent	Forwards DHCP between client segments and centralized servers while preserving information about the originating subnet.

DHCP configures IP; it is not ARP and it is not DNS. DHCP may tell the host what address, routes and DNS resolvers to use. Once configured, ordinary packets still use routing/FIB and neighbor resolution, while names are resolved separately through DNS.

IETF RFCRFC 2131 — Dynamic Host Configuration Protocol

The core DHCPv4 protocol: client/server states, address leases, DISCOVER/OFFER/REQUEST/ACK exchanges, renewal/rebinding timers and relay behavior.

https://www.rfc-editor.org/rfc/rfc2131.html

IETF RFCRFC 2132 — DHCP options

Defines the tagged option format and classic parameters such as subnet mask, router and domain-name-server options.

https://www.rfc-editor.org/rfc/rfc2132.html

IETF RFCRFC 3442 — Classless Static Route Option for DHCPv4

Shows how DHCP can supply arbitrary IPv4 destination-prefix/next-hop routes, including a classless default route.

https://www.rfc-editor.org/rfc/rfc3442.html

IPv6 often configures itself from the link outward: link-local address → DAD → Router Advertisement → SLAAC → Neighbor Discovery

IPv6 does not simply replace DHCPv4 with a differently numbered DHCP server. An interface can first construct a link-local address, verify that the candidate is not already in use with Duplicate Address Detection (DAD), discover routers through ICMPv6 Router Solicitation/Advertisement, and form one or more global addresses from advertised prefixes. The same Neighbor Discovery protocol also replaces IPv4 ARP-style neighbor resolution and tracks reachability of first-hop neighbors.

INTERFACE BECOMES USABLE ↓ form tentative link-local address (fe80::/10 scope) ↓ DUPLICATE ADDRESS DETECTION Neighbor Solicitation for the tentative address ├── conflict observed → address is not assigned └── no conflict → address becomes usable ↓ optional ROUTER SOLICITATION ↓ ROUTER ADVERTISEMENT (RA) ├── router lifetime → candidate default router ├── Prefix Information Option │ ├── L flag: prefix considered on-link under ND rules │ └── A flag: prefix can participate in SLAAC ├── M/O flags: indicate whether DHCPv6 configuration is relevant └── other options may describe MTU or additional configuration ↓ SLAAC forms global address(es) from advertised prefix + interface identifier ↓ DAD for each newly formed unicast address ↓ route/neighbor state now permits ordinary IPv6 traffic ↓ next-hop IPv6 address still needs link-layer resolution Neighbor Solicitation ⇄ Neighbor Advertisement ↓ neighbor cache + reachability state → Ethernet delivery

Mechanism	What it contributes
link-local address	Interface-local IPv6 identity used for same-link communication and router discovery even before global addressing is complete.
DAD	Tests whether a tentative unicast address appears to be duplicated on the link before normal use.
Router Advertisement	Announces routers and prefix/configuration information; a nonzero router lifetime can establish a default-router candidate.
SLAAC	Constructs an IPv6 address from an advertised autonomous prefix without requiring a stateful address lease.
Neighbor Solicitation / Advertisement	Resolve IPv6 next hops to link-layer addresses and confirm neighbor reachability; conceptually covers duties that IPv4 splits among ARP and other mechanisms.
DHCPv6	Can provide stateful addresses or other configuration when requested by network policy, but the IPv6 default-router mechanism is Router Advertisement rather than the ordinary DHCPv6 address lease itself.

Route selection and neighbor resolution are still different steps. An RA can tell a host which router/prefixes to use; Neighbor Discovery then resolves the selected on-link next hop to link-layer delivery state. This is the IPv6 counterpart to the earlier route-versus-neighbor distinction.

OFFICIAL RFCRFC 4861 — IPv6 Neighbor Discovery

Defines Router Solicitation/Advertisement, Neighbor Solicitation/Advertisement, Redirect, neighbor caches, address resolution and Neighbor Unreachability Detection. The RFC Editor page also records later updates to the base specification.

https://www.rfc-editor.org/info/rfc4861/

OFFICIAL RFCRFC 4862 — IPv6 Stateless Address Autoconfiguration

Normative SLAAC sequence: link-local generation, autonomous prefix processing and Duplicate Address Detection.

https://www.rfc-editor.org/info/rfc4862/

PUBLIC MANUALip-address(8) — inspect IPv6 address state and DAD flags

Linux-facing view of IPv6 addresses, including tentative/DAD-related behavior, prefix-route handling and address-generation controls.

https://man7.org/linux/man-pages/man8/ip-address.8.html

Sending to an IP address takes two lookups: choose a route, then resolve the next hop on the local link

An IP destination does not by itself tell the NIC what Ethernet destination to transmit to. The network layer first performs a route lookup to choose an output interface and next hop. Only then can the link layer resolve that next-hop IP address to a local-link identifier such as an Ethernet MAC address. Keeping those two decisions separate explains why a packet for a remote Internet server is usually placed in an Ethernet frame addressed to your router, not to the remote server's MAC.

APPLICATION HAS DESTINATION IP 203.0.113.20 ↓ IP ROUTE / FIB LOOKUP consider local table, policy rules/tables, destination prefix, metrics, etc. ↓ selected result = output interface + source choice + next hop CASE A: destination is directly on-link next hop = 203.0.113.20 itself CASE B: destination is remote next hop = configured gateway, e.g. 192.0.2.1 ↓ NEIGHBOR TABLE LOOKUP FOR NEXT HOP IPv4: ARP-based mapping IPv6: Neighbor Discovery / NUD machinery ↓ cache hit? ── yes → use link-layer address │ no ↓ probe / solicit neighbor, queue or fail traffic according to state ↓ ETHERNET FRAME dst MAC = next hop's MAC, not necessarily final IP destination's MAC ↓ router receives frame, removes L2 header, repeats an IP forwarding decision, and builds a new link-layer frame for its own next hop.

Object	Question it answers
route/FIB entry	For this destination prefix, where should an IP packet go next: local delivery, an interface, a gateway, a blackhole/unreachable route, or another policy table?
longest-prefix match	More-specific destination prefixes normally win over less-specific ones, subject to policy-routing/table rules.
default route	Fallback route used when no more-specific destination prefix matches.
next hop	The directly reachable neighbor that should receive this packet on the selected link; it can be the final host or a router.
neighbor table	Maps a next-hop network-layer address to link-layer reachability/address state.
NUD state	Tracks whether a neighbor is reachable, stale, being probed, unresolved or failed rather than treating an ARP/ND mapping as permanent truth.

Routing and ARP/Neighbor Discovery solve different problems. Routing chooses which IP next hop and interface to use. Neighbor resolution determines how to address that next hop on the local link. An ARP cache is therefore not a substitute for the routing table.

# Inspect the two layers separately on Linux ip route ip route get 1.1.1.1 ip neigh ip neigh get <next-hop-IP> dev <interface> # Compare: # destination IP chosen by the application # route-selected gateway/interface # neighbor-table MAC for that gateway/interface

CURRENT PUBLIC MANUALip-route(8) — Linux routing-table management and route lookup

Current upstream iproute2 manual for route types, tables, prefixes, next hops, policy-routing interactions and ip route get.

https://man7.org/linux/man-pages/man8/ip-route.8.html

CURRENT PUBLIC MANUALip-neighbour(8) — ARP/neighbor table and NUD states

Current upstream iproute2 manual for protocol-address → link-layer-address neighbor entries and states such as reachable, stale, incomplete, probe and failed.

https://man7.org/linux/man-pages/man8/ip-neighbour.8.html

Multicast is neither broadcast nor many unicasts: receivers join a group and routers forward only where listeners exist

With unicast, one destination address identifies one endpoint path. With IP multicast, a sender transmits to a group address and does not enumerate every receiver. Hosts explicitly join groups on interfaces; the local IP stack programs membership state and normally asks the link layer/NIC to accept the corresponding multicast traffic. IPv4 uses IGMP and IPv6 uses MLD so hosts can report local listener interest to neighboring multicast routers. Routing protocols can then build distribution trees rather than sending a separate copy from the source to every receiver.

RECEIVER PROCESS setsockopt(... JOIN GROUP ...) ↓ kernel records membership on interface ↓ link/NIC acceptance state updated as needed ↓ IGMPv3 (IPv4) or MLDv2 (IPv6) membership report ↓ local multicast router learns: group G has listener(s) on this link optionally source include/exclude filters ↓ multicast routing state forwards matching traffic only onto downstream links that need it SENDER sendto(multicast-group G) ↓ ONE IP multicast datagram stream ↓ network may replicate packets at branch points ↓ receiver kernels deliver copies only to sockets whose group/interface/source filters match

Mechanism	Role
multicast group	A logical destination shared by zero or more receivers; membership can change while the sender keeps using the same group address.
IGMPv3	IPv4 host↔local-router protocol for reporting group membership and optional source filtering.
MLDv2	IPv6 counterpart to IGMPv3, carried through ICMPv6 semantics.
ASM	Any-Source Multicast: receiver asks for group G and may accept traffic from many sources.
SSM	Source-Specific Multicast: receiver interest is effectively in a source/group pair, reducing ambiguity about who may send.
L2 multicast filtering/snooping	Local Ethernet/Wi-Fi delivery optimization; separate from the IP-layer membership and multicast-routing model.

Joining a multicast group is not the same operation as adding a unicast route. The normal route lookup still chooses an egress interface for a sender, but receiver membership says which multicast traffic should be admitted/delivered and gives multicast routers information for replication. DNS, DHCP, service discovery and streaming protocols can use multicast, but multicast itself provides no reliability or ordering.

INTERNET STANDARDRFC 9776 / STD 100 — IGMPv3

The current Internet Standard for IPv4 multicast listener reporting and source filtering; it obsoletes RFC 3376.

https://www.rfc-editor.org/info/rfc9776/

INTERNET STANDARDRFC 9777 / STD 101 — MLDv2

The current Internet Standard for IPv6 multicast listener discovery and source filtering; it obsoletes RFC 3810.

https://www.rfc-editor.org/info/rfc9777/

PUBLIC MANUALip(7) — IPv4 socket options and multicast membership

Linux userspace interface for IPv4 multicast operations such as joining/leaving groups, selecting multicast interfaces and controlling loop/TTL behavior.

https://man7.org/linux/man-pages/man7/ip.7.html

PUBLIC MANUALipv6(7) — IPv6 socket behavior and multicast groups

Linux IPv6 socket API reference, including IPv6 multicast group membership and interface-scoped behavior.

https://man7.org/linux/man-pages/man7/ipv6.7.html

A packet can fit the first Ethernet link and still be too large later: Path MTU, fragmentation and Packet Too Big

Every link has a maximum packet size it can carry in one piece. The Path MTU (PMTU) is the smallest link MTU along the route between sender and receiver. If an IP packet is larger than that bottleneck, something must change: the sender must send smaller packets, or—where the protocol permits—fragmentation must split the packet into pieces. Modern transports generally try to learn the PMTU and avoid fragmentation because losing one fragment can make the entire original packet unusable.

APPLICATION / TRANSPORT has data ↓ transport chooses packetization size ↓ IP packet leaves host, e.g. 1500 bytes ↓ router reaches a next-hop link with MTU 1280 ↓ CAN THIS PACKET BE FORWARDED AS-IS? IPv4 with DF set: router drops packet → ICMP Destination Unreachable / Fragmentation Needed → sender lowers cached PMTU IPv4 where fragmentation is allowed: router may fragment the datagram → receiver reassembles fragments IPv6: routers do NOT fragment transit packets → router sends ICMPv6 Packet Too Big with MTU → source sends smaller packets or source-fragments if needed PLPMTUD: transport/application sends controlled probes → infers which packet sizes succeed even when ICMP feedback is unreliable

Term	What it means
link MTU	Largest IP packet a particular link can carry without link-specific fragmentation or other special handling.
Path MTU	Minimum link MTU along the current source→destination path.
IPv4 DF	Don't Fragment flag. With DF set, a router that cannot forward the packet intact drops it and reports the constraint instead of fragmenting it.
IPv4 fragmentation	Splits one datagram into IP fragments that are reassembled at the destination; fragments share identification metadata and carry offsets.
IPv6 Packet Too Big	ICMPv6 feedback carrying the constraining MTU. IPv6 routers do not fragment transit packets.
PMTU cache	Host/transport state remembering a usable packet size for a destination/path; it can become stale when routing changes.
PLPMTUD	Packetization-layer probing that learns a usable size from successful/lost probes rather than depending only on ICMP messages.

MTU is not bandwidth. A 1500-byte MTU says how large one packet may be on a link, not how many bytes per second the link can carry. And TCP's MSS is smaller than the IP MTU because TCP/IP headers also occupy bytes.

IETF STANDARDRFC 1191 — IPv4 Path MTU Discovery

The classic IPv4 PMTUD mechanism: set Don't Fragment, react to ICMP “fragmentation needed,” and reduce the assumed path MTU until packets traverse the path intact.

https://www.rfc-editor.org/info/rfc1191

IETF STANDARDRFC 8201 — IPv6 Path MTU Discovery

Defines IPv6 PMTUD and the source's reaction to ICMPv6 Packet Too Big messages. It also explains the distinction between a link MTU and the minimum MTU across a path.

https://www.rfc-editor.org/info/rfc8201

IETF STANDARDRFC 8899 — Packetization Layer Path MTU Discovery

Robust probe-based PMTU discovery for datagram transports and protocols layered over them, including QUIC-style transports where relying only on ICMP feedback is undesirable.

https://www.rfc-editor.org/info/rfc8899

ICMP is IP's feedback channel: errors, echo diagnostics and the mechanism behind traceroute

IP deliberately does not promise delivery, ordering or retransmission. ICMP is the companion control protocol that lets routers and hosts report selected network-layer conditions back toward a sender. It carries messages such as Destination Unreachable, Time Exceeded and—on IPv6—Packet Too Big. Diagnostic tools such as ping and traceroute deliberately trigger or exchange ICMP messages to reveal reachability and the sequence of forwarding hops.

ORDINARY FORWARDING FAILURE host A sends IP packet ↓ router / destination cannot process or forward it ↓ original packet may be discarded ↓ ICMP error is generated when protocol rules allow it ↓ ICMP payload includes enough of the invoking packet for the sender/transport to identify what failed ↓ originating host may update state or report an error PING Echo Request ───────────────────────────────→ destination Echo Reply ←─────────────────────────────── destination measure RTT / loss observations TRACEROUTE probe with TTL/Hop Limit = 1 ↓ first router decrements to zero ICMP Time Exceeded ← router 1 probe with TTL/Hop Limit = 2 ICMP Time Exceeded ← router 2 ↓ repeat until destination is reached or probing stops Important: a missing ICMP reply does NOT prove the destination is down; filters, rate limiting, asymmetric routing or policy can hide responses.

ICMP concept	What it means
Echo Request / Echo Reply	Informational request/reply used by `ping` for reachability and RTT observation. It is not a TCP-style reliability acknowledgment.
Destination Unreachable	Reports that delivery failed for a defined reason such as no route, administratively prohibited traffic or an unreachable transport endpoint.
Time Exceeded	Reports that IPv4 TTL or IPv6 Hop Limit expired in transit; traceroute exploits this to expose successive forwarding hops.
Packet Too Big	ICMPv6 message carrying the next-hop MTU for Path MTU Discovery. IPv4 PMTU uses related ICMP Destination Unreachable signaling with fragmentation-needed semantics.
quoted invoking packet	ICMP errors carry part of the packet that triggered them so the receiver can associate the error with a flow/socket where possible.
rate limiting/filtering	Routers and hosts may limit or filter diagnostics. Control traffic is useful but should not be assumed to arrive reliably.

ICMP is not “just ping.” It is network-layer feedback used by core mechanisms such as error reporting and Path MTU discovery. Blocking every ICMP message can therefore break useful behavior even when application data itself uses TCP or UDP.

INTERNET STANDARDRFC 792 / STD 5 — Internet Control Message Protocol (IPv4)

The base ICMPv4 specification. It defines the control-message model and explains that ICMP provides feedback about datagram-processing problems without making IP reliable.

https://www.rfc-editor.org/info/rfc792/

INTERNET STANDARDRFC 4443 / STD 89 — ICMPv6

Current Internet Standard for ICMPv6, including Destination Unreachable, Packet Too Big, Time Exceeded and Echo Request/Reply behavior.

https://www.rfc-editor.org/info/rfc4443/

PUBLIC MANUALping(8) — ICMP Echo diagnostics

Current Linux/iputils manual showing IPv4/IPv6 Echo operation, RTT reporting, packet sizing and Path MTU probing controls.

https://man7.org/linux/man-pages/man8/ping.8.html

PUBLIC MANUALtraceroute(8) — TTL/Hop-Limit path probing

Practical reference for UDP-, ICMP- and TCP-based probes and the hop-by-hop replies used to infer a forwarding path.

https://man7.org/linux/man-pages/man8/traceroute.8.html

A container can have a real Linux network stack without a real NIC: veth pairs, bridges and VLAN filtering connect Layer-2 domains

A Linux veth pair behaves like a virtual patch cable: a frame transmitted into one end is received from the other. Put one end in a container/network namespace and the other on the host, then attach the host end to a Linux bridge, and the container can participate in an Ethernet broadcast domain just like a machine plugged into a switch. The bridge learns source MAC addresses into a forwarding database (FDB) and uses destination MAC addresses—optionally together with VLAN IDs—to decide which port should receive each frame.

CONTAINER / NETWORK NAMESPACE eth0 │ │ veth endpoint A ║ virtual pair: frames cross directly │ veth endpoint B (host namespace) ↓ LINUX BRIDGE br0 ├── veth-container-A ├── veth-container-B └── physical NIC or another virtual port FRAME ARRIVES ON ONE BRIDGE PORT ↓ learn SOURCE MAC → ingress port in FDB ↓ look up DESTINATION MAC (+ VLAN when filtering enabled) ├── known unicast → forward to learned port ├── local bridge MAC → deliver locally └── unknown/broadcast → flood eligible bridge ports WITH VLAN FILTERING untagged/tagged ingress → assign/check VLAN ID ↓ FDB + VLAN membership decide eligible egress ports ↓ add/remove/retain 802.1Q tag according to port configuration

Object	Role
network namespace	Own networking world: interfaces, routes, sockets, firewall state and related network objects.
veth pair	Two interconnected virtual Ethernet interfaces, often split across namespaces.
Linux bridge	Layer-2 software switch forwarding Ethernet frames among bridge ports.
FDB	Forwarding database mapping learned destination MAC addresses to bridge ports.
VLAN ID	802.1Q Layer-2 segmentation identifier. With bridge VLAN filtering, forwarding eligibility depends on both port/VLAN membership and MAC destination.
bridge port	An interface enslaved to the bridge; it can be a physical NIC, veth endpoint, tap interface or another supported netdevice.
router vs bridge	A bridge forwards frames using Layer-2 information; a router forwards packets between IP networks using Layer-3 routes. One Linux host can do both in different stages.

A veth pair is not itself a bridge or a router. It only connects two virtual Ethernet endpoints. Bridging, routing, NAT and firewalling are separate kernel mechanisms layered around those interfaces.

CURRENT PUBLIC MANUALveth(4) — virtual Ethernet device pairs

Current Linux man-pages description of veth pairs, including the common pattern of placing the two endpoints in different network namespaces.

https://man7.org/linux/man-pages/man4/veth.4.html

KERNEL DOCSLinux Ethernet bridging

Current kernel documentation for bridge behavior, FDB ageing, spanning-tree controls, multicast handling and 802.1Q/802.1ad VLAN filtering.

https://docs.kernel.org/networking/bridge.html

CURRENT PUBLIC MANUALbridge(8) — inspect FDBs, VLANs and bridge ports

Hands-on iproute2 interface for viewing and manipulating bridge links, forwarding-database entries, multicast groups, VLAN membership and related state.

https://man7.org/linux/man-pages/man8/bridge.8.html

Two Ethernet links can act as one logical interface: Linux bonding, LAGs and LACP negotiate membership but usually keep each flow on one member

Link aggregation combines multiple physical Ethernet interfaces behind one logical link-layer interface. Linux exposes this through the bonding driver. Some modes are purely local policies such as active-backup; 802.3ad / IEEE 802.1AX mode cooperates with the switch using the Link Aggregation Control Protocol (LACP) so both ends agree which member links belong to the active aggregation.

applications / IP stack ↓ bond0 (one logical net_device, one IP configuration) ↓ transmit-hash / active-member policy ┌────┴────┐ eth0 eth1 │ │ └── LACP ──┴── switch ports ↓ logical LAG on switch 802.3ad control plane: actor periodically exchanges LACPDUs with partner ↓ links that agree on aggregation parameters reach collecting/distributing state ↓ data frames are assigned to eligible member links by a stable selection/hash policy

Mode/mechanism	Behavior
active-backup	One member carries traffic; another can take over after link/failure detection. No switch-side LACP negotiation is required.
802.3ad / LACP	Host and switch negotiate an aggregate of compatible links and exchange LACPDUs to maintain membership/state.
transmit hash	Selects a member using packet/flow fields according to policy, normally keeping packets of one conversation on a stable path to avoid reordering.
miimon / link monitoring	Lets the bond detect member failure and remove unusable links from forwarding.
LAG capacity	Multiple independent flows can use multiple links concurrently, increasing aggregate capacity.
single flow	Usually remains on one physical member and therefore does not automatically gain N× one-link throughput.

A bond is not a bridge. A bridge forwards frames among different Layer-2 segments based on destination MAC/FDB state. A bond presents multiple physical links as one logical interface for redundancy/aggregate capacity. A machine can place a bond underneath a bridge, but they solve different problems.

KERNEL DOCSLinux Ethernet Bonding Driver HOWTO

Authoritative Linux description of bonding modes, failure monitoring, 802.3ad requirements, LACP timing and transmit-hash behavior.

https://docs.kernel.org/networking/bonding.html

IEEE STANDARD PAGEIEEE 802.1AX-2020 — Link Aggregation

IEEE's public standards-project page for Link Aggregation. It defines a Link Aggregation Group as multiple point-to-point links presented to a MAC client as one logical link and covers the standardized aggregation control behavior.

https://1.ieee802.org/tsn/802-1ax-rev/

PUBLIC MANUALip-link(8) — create/configure bond devices

Current iproute2 manual showing bond as a Linux link type and the bond/bond-slave controls used by modern Netlink-based configuration tooling.

https://man7.org/linux/man-pages/man8/ip-link.8.html

VXLAN makes one Layer-2 segment span an IP network: inner Ethernet → VNI → UDP/IP → remote VTEP

A VLAN tags Ethernet frames inside one Layer-2 switching domain. VXLAN does something different: it encapsulates an entire Ethernet frame inside UDP/IP so that a virtual Layer-2 network can cross a routed Layer-3 underlay. A 24-bit VXLAN Network Identifier (VNI) selects the overlay segment, allowing far more logical networks than the 12-bit VLAN identifier space.

CONTAINER / VM A inner Ethernet frame: dst MAC = VM B src MAC = VM A optional inner VLAN/IP/TCP/etc. ↓ Linux bridge/FDB chooses VXLAN device ↓ VXLAN VTEP needs remote tunnel endpoint for dst MAC FDB entry conceptually: VM-B-MAC → remote VTEP IP 192.0.2.20 ↓ ENCAPSULATE outer Ethernet outer IP: local VTEP → remote VTEP outer UDP: destination usually 4789 VXLAN header: VNI = 5000 inner Ethernet frame unchanged inside payload ↓ routed physical/underlay IP network ↓ REMOTE VTEP decapsulates UDP/VXLAN ↓ VNI 5000 → remote bridge/FDB ↓ VM / container B Unknown/broadcast/multicast inner traffic may be flooded to configured VTEPs, multicast groups, or handled by a separate control plane depending on design.

Term	Meaning
underlay	The ordinary IP network that routes outer packets between tunnel endpoints.
overlay	The virtual Ethernet/L2 topology carried inside the tunnel.
VTEP	VXLAN Tunnel Endpoint: encapsulates/decapsulates VXLAN traffic, often represented by a Linux VXLAN netdevice.
VNI	24-bit overlay network identifier carried in the VXLAN header; logically analogous to a much larger segmentation identifier than a VLAN ID.
FDB	Forwarding database mapping inner destination MACs toward local ports or remote VTEP IPs.
outer UDP/IP	Transport/routing envelope used by the underlay; the inner tenant Ethernet frame is payload from the underlay's perspective.
UDP 4789	IANA-assigned VXLAN destination port commonly used for the tunnel.
MTU overhead	Outer Ethernet/IP/UDP/VXLAN headers consume bytes, so the overlay MTU must account for encapsulation or the underlay must support a larger MTU.
offload	NICs may understand UDP tunnels so checksum, segmentation and receive-steering hardware can operate on encapsulated traffic efficiently.

VXLAN is not “a bigger VLAN tag.” VLAN is a Layer-2 tag carried directly in an Ethernet domain; VXLAN is an encapsulation that transports an inner Ethernet frame across an IP-routed network. The VNI labels the overlay, while the outer IP addresses identify tunnel endpoints in the underlay.

LINUX LAB (example only) # create a VXLAN interface using VNI 42 over eth0 ip link add vxlan42 type vxlan id 42 dev eth0 dstport 4789 ip link set vxlan42 up # inspect tunnel configuration ip -d link show vxlan42 # attach it to an existing bridge ip link set vxlan42 master br0 # inspect learned/static remote forwarding entries bridge fdb show dev vxlan42 # a static remote mapping can associate an inner MAC with a VTEP IP # bridge fdb add 02:00:00:00:00:42 dev vxlan42 dst 192.0.2.20 # exact learning/flood/control-plane design varies by deployment.

KERNEL DOCSLinux VXLAN networking documentation

Current kernel documentation for the Linux VXLAN netdevice, VNI creation, FDB entries and UDP-tunnel offload visibility.

https://kernel.org/doc/html/latest/networking/vxlan.html

OPEN STANDARDRFC 7348 — Virtual eXtensible Local Area Network (VXLAN)

The protocol specification describing VXLAN's 24-bit VNI and Layer-2 overlay carried over Layer-3 networks using UDP encapsulation.

https://www.rfc-editor.org/rfc/rfc7348.html

TUN/TAP turns a userspace file descriptor into one side of a virtual network interface

Linux TUN/TAP is a deliberate boundary between the normal kernel networking stack and a userspace packet-processing program. A program opens /dev/net/tun, uses the TUN/TAP ioctl interface to create or attach a virtual network device, and then exchanges packets with that device through ordinary read()/write()-style file-descriptor I/O. TUN carries Layer-3 IP packets; TAP carries Layer-2 Ethernet frames.

TUN — L3 packet interface application socket ↓ kernel TCP/UDP/IP + route lookup tun0 selected as egress device ↓ TUN kernel driver → bytes become readable on /dev/net/tun fd ↓ userspace VPN/tunnel program ↓ encrypt/encapsulate however that program chooses physical socket/NIC → network reverse direction: network → userspace tunnel program → write IP packet to TUN fd → kernel receives it as packet input on tun0 → routing/socket delivery TAP — L2 frame interface VM/emulator/userspace switch ⇄ TAP fd ⇄ tap0 Ethernet netdev ↓ Linux bridge / VLAN / host stack

Interface	Userspace reads/writes	Natural kernel integration
TUN	IP packets without an Ethernet header.	Layer-3 routing, addresses, firewall hooks and tunnel/VPN applications.
TAP	Ethernet frames including MAC headers.	Linux bridges, VLAN-aware Layer-2 domains, virtual machines and Ethernet emulation.
multiqueue TUN/TAP	Multiple file descriptors/queues for one virtual interface.	Lets a multithreaded userspace dataplane process packets in parallel.
`IFF_NO_PI`	Suppresses the optional small packet-information prefix used by the TUN/TAP fd format.	Makes userspace exchange just the protocol packet/frame bytes expected by many applications.

TUN/TAP is not itself a VPN protocol or encapsulation format. It only supplies a virtual network-device boundary. Encryption, authentication, compression, retransmission or outer UDP/TCP encapsulation—if any—belong to the userspace program or some other protocol layer.

KERNEL DOCSLinux — Universal TUN/TAP device driver

Current kernel documentation for /dev/net/tun, TUNSETIFF, TUN versus TAP frame formats, multiqueue operation and current qdisc/backpressure behavior.

https://docs.kernel.org/networking/tuntap.html

CURRENT PUBLIC MANUALip(8) — network-device, netns and tuntap control surface

Current iproute2 manual showing the tuntap object alongside links, addresses, routes, neighbors and network namespaces used to configure virtual interfaces around the kernel datapath.

https://man7.org/linux/man-pages/man8/ip.8.html

Many Linux “configuration commands” are structured kernel IPC: Netlink carries routes, links, neighbors and subsystem events

Commands such as ip link, ip route and many newer ethtool operations do not manipulate kernel tables by writing a magic configuration file. They open an AF_NETLINK socket and exchange typed messages with a kernel subsystem. Netlink is therefore a control/event plane between userspace and the kernel; it is distinct from the Ethernet/IP packets that those configuration changes may later cause the machine to send.

USERSPACE TOOL / DAEMON ip / network manager / ethtool / nft-like controller ↓ socket(AF_NETLINK, ... family ...) construct netlink message nlmsghdr: length, type, flags, sequence, port ID family-specific fixed header and/or TLV attributes ↓ sendmsg() KERNEL NETLINK FAMILY ↓ validate message + permissions + attributes subsystem operation link / address / route / neighbor / qdisc / other family state ↓ ACK / ERROR / REPLY / MULTIPART DUMP ↓ recvmsg() userspace matches sequence numbers and parses attributes ASYNCHRONOUS EVENTS kernel state change ↓ family multicast group subscribed userspace sockets receive notification ↓ daemon updates its cached view This is CONTROL-PLANE IPC inside one Linux system. It is not an IP packet being transmitted to another host.

Netlink concept	Role
`AF_NETLINK`	Socket address family used for datagram-like messages between userspace and kernel Netlink families.
family	Selects a protocol namespace such as `NETLINK_ROUTE`, `NETLINK_NETFILTER`, or `NETLINK_GENERIC`.
`nlmsghdr`	Common message envelope carrying length, type, flags, sequence number and sender/port identity metadata.
attributes / TLVs	Typed length-value fields that make many Netlink protocols extensible without one monolithic fixed C structure.
`NLM_F_ACK` / extended ACK	Requests explicit success/error reporting; modern families can return detailed attribute-level diagnostics.
multipart dump	One request can produce many reply messages terminated by `NLMSG_DONE`, as when dumping routes or interfaces.
multicast group	Lets listeners receive asynchronous notifications such as link/address changes rather than polling continuously.
sequence number	Userspace bookkeeping used to associate replies/ACKs with requests; important because notifications can share the same socket.
`ENOBUFS` / resynchronization	A listener can lose notifications if its receive buffer overflows; robust stateful consumers must detect loss and rebuild state from a fresh dump.

Netlink is not “the networking stack.” Its best-known users configure networking, but the transport itself is local kernel↔userspace IPC. The packet data path still uses NIC queues, sk_buffs/XDP buffers, IP, TCP/UDP and sockets described elsewhere on this page.

KERNEL DOCSLinux Netlink Handbook

Current kernel userspace-API documentation covering Netlink sockets, Generic Netlink, protocol specifications, attributes, operations and multicast groups.

https://docs.kernel.org/userspace-api/netlink/index.html

CURRENT PUBLIC MANUALnetlink(7) — AF_NETLINK message transport

Current Linux manual for Netlink sockets, message headers, sequence/port IDs, multipart messages, ACK/error handling and multicast groups.

https://man7.org/linux/man-pages/man7/netlink.7.html

CURRENT PUBLIC MANUALrtnetlink(7) — links, addresses, routes and neighbors

Concrete NETLINK_ROUTE protocol reference showing the message families behind interface, address, route, neighbor and traffic-control configuration.

https://man7.org/linux/man-pages/man7/rtnetlink.7.html

A Linux packet can cross policy hooks before or after routing: Netfilter, conntrack and NAT are separate mechanisms that cooperate

Linux networking is not only protocol parsing and route lookup. Netfilter exposes hook points along packet traversal where nftables or other kernel components can inspect, accept, drop, modify, queue or account for traffic. Conntrack maintains flow state that rules can match, while NAT installs address/port translation state for a flow. They are related, but none of the three is simply another name for the others.

PACKET ARRIVES FROM NIC ↓ PREROUTING hook ↓ routing decision ┌────┴─────────────┐ │ │ local destination forwarded packet ↓ ↓ INPUT hook FORWARD hook ↓ ↓ local socket/process next-hop output path ↓ POSTROUTING hook ↓ NIC LOCAL PROCESS SENDS ↓ route selection ↓ OUTPUT hook ↓ POSTROUTING hook ↓ NIC CONNTRACK first packets create/lookup flow state keyed from protocol + endpoint tuple ↓ later packets can match states such as ESTABLISHED / RELATED / INVALID STATEFUL NAT first packet matches a NAT rule and creates a translation binding ↓ subsequent packets in that flow use the stored binding in both directions

Mechanism	Role
Netfilter hook	Kernel interception point such as prerouting, input, forward, output or postrouting.
nftables rule	Packet/metadata classifier plus actions such as accept, drop, counter, mark, reject or NAT.
conntrack entry	Kernel state associating packets with a flow/connection and tracking protocol-dependent state.
NEW / ESTABLISHED	Common conntrack states used by stateful firewall policy; they are not the same thing as TCP's user-visible socket states.
DNAT	Changes destination address and/or port, commonly before the final routing decision for incoming traffic.
SNAT / masquerade	Changes source address and/or port, commonly on traffic leaving through an external interface.
packet mark	Kernel metadata that can connect filtering/classification with policy routing or traffic-control decisions without changing packet payload.

NAT is usually per-flow state, not a fresh rewrite decision on every packet. In nftables stateful NAT, the first packet that selects a NAT rule establishes the binding; follow-up packets are translated from that conntrack/NAT state. This is why NAT and connection tracking are tightly coupled even though they remain distinct subsystems.

PROJECT DOCSnftables — connection tracking system

Explains conntrack as the state engine used for stateful filtering and frequently by NAT, while distinguishing those components from nftables itself.

https://wiki.nftables.org/wiki-nftables/index.php/Connection_Tracking_System

PROJECT DOCSnftables — stateful NAT

Concrete SNAT/DNAT/masquerade documentation, including first-packet rule lookup followed by per-flow NAT binding behavior.

https://wiki.nftables.org/wiki-nftables/index.php/Performing_Network_Address_Translation_(NAT)

KERNEL DOCSLinux — Netfilter conntrack runtime parameters

Current kernel documentation for conntrack table sizing, accounting, checksum validation, timeout behavior and related runtime state.

https://docs.kernel.org/networking/nf_conntrack-sysctl.html

nftables is the rule engine on Netfilter: hook → base chain → expressions/sets/maps → verdict or stateful action

The earlier Netfilter section explains where packets can be intercepted and how conntrack/NAT cooperate. nftables is the modern kernel/userspace rule framework that programs policy at those hooks. The nft command sends structured rule-set transactions over Netlink; the kernel evaluates packet metadata and state through chains of expressions, efficient sets/maps and verdicts.

ADMIN / SERVICE nft add rule ... ↓ Netlink batch/transaction kernel nftables ruleset ├── table (family: inet/ip/ip6/bridge/netdev/...) ├── base chain attached to a Netfilter hook + priority ├── regular chains ├── sets / interval sets / concatenations ├── maps / verdict maps └── stateful objects: counters, quotas, limits, etc. PACKET ARRIVES AT A NETFILTER HOOK ↓ hook priority determines ordering relative to conntrack/NAT/etc. base chain ↓ rule expressions evaluated left→right match protocol / addresses / ports / interfaces / marks / ct state / metadata ↓ set lookup or map lookup can replace long linear rule lists ↓ statement/action ├── accept / drop ├── jump / goto / return ├── log / counter / mark ├── queue to userspace └── DNAT / SNAT / masquerade in NAT chains NAT detail: first packet establishes a conntrack/NAT binding; subsequent packets in that flow normally use the stored translation state.

nftables concept	What it controls
family	Address/protocol context such as `inet`, IPv4, IPv6, bridge or netdev ingress/egress.
base chain	Chain attached to a specific Netfilter hook with a chain type and priority. nftables does not create classic INPUT/FORWARD/OUTPUT chains automatically.
regular chain	Reusable rule sequence reached with jump/goto rather than directly attached to a hook.
set	Typed collection used for efficient membership tests, timeouts and interval matching instead of many repeated rules.
map / verdict map	Key→value or key→verdict lookup that can encode dispatch policy compactly.
`ct state`	Match against conntrack state such as established/related/new; nftables itself is not the conntrack engine.
priority	Ordering at a hook relative to other nftables chains and internal Netfilter operations such as defragmentation, conntrack and NAT.
atomic ruleset update	Netlink batch transactions can replace/change related rules as one coherent update rather than exposing partially edited policy.

nftables is not a second packet path. It programs classification/actions at the same Netfilter hook infrastructure used by conntrack, NAT, logging and userspace queueing. Traffic control (tc) is a separate subsystem and remains the place for queueing/shaping even when nftables performs filtering.

PROJECT DOCSnftables — Netfilter hooks and packet flow

Direct diagram-oriented documentation for prerouting/input/forward/output/postrouting placement, hook priorities and how nftables fits into the Netfilter hook framework.

https://wiki.nftables.org/wiki-nftables/index.php/Netfilter_hooks

PROJECT MANUALnft(8) — official nftables manual

Current project manual for tables, chains, rules, expressions, sets, maps, stateful objects, NAT statements and transactional ruleset management.

https://netfilter.org/projects/nftables/manpage.html

IPsec protects IP packets through policy and Security Associations: Linux XFRM selects → ESP transforms → peer reverses

IPsec is not a socket-level encryption wrapper like TLS. Its architecture applies security processing to IP traffic according to a Security Policy Database and one or more unidirectional Security Associations. On Linux, the XFRM framework represents those policies and states. Outbound traffic can match a policy, select an SA, gain an ESP header/trailer plus encryption/integrity protection, and then continue through ordinary IP routing; inbound traffic performs the inverse checks before the protected inner packet is delivered upward.

OUTBOUND application ↓ TCP / UDP / other upper layer inner IPv4/IPv6 packet ↓ XFRM policy lookup (SPD-like policy) ├── bypass / no policy → ordinary IP path └── protect → select XFRM state / Security Association • peer / mode • SPI • algorithms + keys • sequence / anti-replay state ↓ ESP processing • encapsulate payload according to transport/tunnel mode • encrypt and integrity/authentication-protect as configured • add SPI + sequence number ↓ outer IP packet / routing / neighbor resolution / NIC ↓ network INBOUND NIC → IP ↓ ESP identified (IP protocol 50, or encapsulated form where configured) find inbound SA using destination/SPI/etc. ↓ anti-replay + cryptographic verification/decryption ↓ recover protected payload / inner packet ↓ inbound policy validation ↓ TCP/UDP/socket or forwarding path

IPsec/XFRM object	Meaning
Security Policy Database (SPD)	Rules deciding whether matching traffic must be protected, bypassed or discarded and what kind of protection is required.
Security Association (SA)	Unidirectional security state: peer/mode, SPI, cryptographic algorithms/keys, sequence and lifetime/replay information.
SPI	Security Parameters Index carried in ESP/AH packets and used with other packet fields to identify an inbound SA.
ESP	Encapsulating Security Payload; provides confidentiality and/or integrity/authentication services according to the SA and chosen algorithms.
transport mode	Protects the upper-layer payload while retaining the original IP header as the packet's outer header.
tunnel mode	Protects an entire inner IP packet and places it inside a new outer IP packet, common for gateway/VPN designs.
anti-replay window	Tracks received sequence numbers so replayed protected packets can be rejected.
Linux XFRM policy/state	Kernel representation of policy and transformation/SA state configured through Netlink tools such as `ip xfrm` or key-management daemons.

# Inspect Linux XFRM/IPsec state (read-only) ip xfrm policy ip xfrm state # Monitor XFRM changes/events ip xfrm monitor # Relevant kernel statistics cat /proc/net/xfrm_stat # Compare with ordinary route selection ip route get 1.1.1.1

IPsec policy is not the same thing as routing or firewall policy. XFRM determines security transformations/requirements; routing determines where packets go; Netfilter can filter/NAT packets at its hooks. The subsystems interact in the packet path, but they solve different problems.

STANDARDS TRACK RFCRFC 4301 — Security Architecture for IP

Normative IPsec architecture defining Security Associations, policy processing and transport/tunnel concepts. RFC Editor currently classifies it as Proposed Standard / Standards Track.

https://www.rfc-editor.org/rfc/rfc4301.html

STANDARDS TRACK RFCRFC 4303 — Encapsulating Security Payload (ESP)

Normative ESP packet format and processing: SPI, sequence numbers, payload protection, anti-replay-related state and transport/tunnel use.

https://www.rfc-editor.org/rfc/rfc4303.html

CURRENT PUBLIC MANUALip-xfrm(8) — Linux XFRM policy/state configuration

Current iproute2 manual for ip xfrm state, policy and monitoring, including ESP/AH protocol, transport/tunnel modes, selectors, algorithms and offload fields.

https://man7.org/linux/man-pages/man8/ip-xfrm.8.html

KERNEL DOCSLinux XFRM framework documentation

Kernel documentation index for XFRM internals, statistics, migration/synchronization and hardware offload interfaces.

https://docs.kernel.org/networking/xfrm/index.html

WireGuard is a virtual Layer-3 interface: route an IP packet to wg0 → choose a peer by AllowedIPs → encrypt → UDP

WireGuard presents an ordinary IP network interface to the host, but packets sent through it are encrypted and encapsulated in UDP for transport to a peer's current endpoint. Each peer is identified by a static public key and has an AllowedIPs set. On transmit, those prefixes select the peer whose session should protect the inner packet; on receive, they also constrain which inner source addresses that authenticated peer may inject.

OUTBOUND application → TCP/UDP/ICMP → inner IP packet ↓ ordinary routing chooses wg0 WireGuard looks up destination in peer AllowedIPs ↓ Noise_IK-derived session keys / authenticated encryption ↓ WireGuard transport packet ↓ outer UDP + outer IP → physical NIC → Internet INBOUND NIC → outer IP/UDP → WireGuard socket ↓ authenticate + decrypt; identify peer by cryptographic state ↓ verify inner SOURCE address is allowed for that peer ↓ inner IPv4/IPv6 packet injected through wg0 ↓ ordinary host routing / local delivery Endpoint IP:port may roam after authenticated traffic arrives. AllowedIPs is not the same object as the host's main routing table.

Object	Role
wg interface	Virtual Layer-3 interface carrying ordinary IPv4/IPv6 packets before encryption and after decryption.
peer public key	Long-term cryptographic identity used by the authenticated key exchange.
AllowedIPs	Cryptokey-routing prefixes: outbound peer selection plus inbound source-address authorization for that peer.
endpoint	Outer IP address and UDP port used to reach a peer; it is transport location, not peer identity.
handshake	Noise_IK-based exchange that establishes rotating symmetric session keys; data transport remains connectionless at the UDP layer.
Generic Netlink	Linux control plane used to get/set WireGuard interface and peer configuration; it does not carry the encrypted data packets themselves.

WireGuard and IPsec sit at a similar conceptual layer but expose different control models. IPsec commonly separates Security Policy and Security Association databases through XFRM; WireGuard couples peer public keys to permitted inner prefixes through cryptokey routing.

OFFICIAL PROTOCOLWireGuard protocol and cryptography

Official protocol overview covering Noise_IK, Curve25519, ChaCha20-Poly1305, key rotation, UDP transport and the handshake/data-packet model.

https://www.wireguard.com/protocol/

KERNEL DOCSLinux WireGuard Generic Netlink specification

Current kernel specification for the control-plane messages and attributes used to inspect/configure WireGuard devices and peers.

https://docs.kernel.org/netlink/specs/wireguard.html

TECHNICAL PAPERWireGuard: Next Generation Kernel Network Tunnel (PDF)

Design paper explaining cryptokey routing, endpoints/roaming, handshake design and how inner IP addresses are bound to authenticated peers.

https://www.wireguard.com/papers/wireguard.pdf

UDP is message transport, not a tiny TCP: each send creates one datagram with no built-in delivery recovery

UDP exposes a datagram service. The application supplies one message, UDP adds source/destination ports, length and checksum information, and IP carries the resulting datagram toward the destination. Unlike TCP, UDP does not create a reliable byte stream, does not retransmit missing data, does not reorder messages for the application, and has no connection handshake that proves a peer is listening before data is sent.

sender sock = socket(AF_INET, SOCK_DGRAM, 0) sendto(sock, MESSAGE_A, dest) sendto(sock, MESSAGE_B, dest) ↓ UDP header per datagram [src port | dst port | length | checksum] ↓ IP route + PMTU decision ↓ packet(s) traverse network ↓ receiver UDP demultiplexes by addresses/ports ↓ receive queue contains DATAGRAM A and DATAGRAM B as separate messages ↓ recvfrom() returns one datagram at a time Network may drop, duplicate or reorder datagrams. UDP itself does not repair that. If the application needs retries, ordering, pacing or congestion control, its protocol must supply them (or choose a transport that does).

UDP property	Consequence
message boundaries preserved	Two sends are not merged into one continuous stream abstraction. Receive calls consume individual datagrams; an undersized receive buffer can truncate a datagram.
no handshake	`connect()` on a UDP socket mainly establishes a default peer and useful kernel filtering/error semantics; it does not perform TCP-style SYN/SYN-ACK negotiation.
no retransmission/order repair	Applications can observe loss or reordering and must define their own response if those events matter.
checksum	Detects corruption using the UDP header/data plus an IP-derived pseudo-header; checksum rules differ in detail between IPv4 and IPv6.
Path MTU interaction	Linux normally performs PMTU discovery; an oversized write can fail with `EMSGSIZE`, telling the application to reduce datagram size rather than relying on fragmentation.
no inherent congestion control	Internet applications using UDP still need responsible rate/congestion behavior; “UDP does not do it for you” is not permission to ignore congestion.

A connected UDP socket is still UDP. Calling connect() does not create a reliable session. It gives the socket a default destination (and affects receive/error handling), allowing send()/write() syntax, but the payload remains independent datagrams.

CURRENT PUBLIC MANUALudp(7) — Linux UDP behavior

Current Linux man page covering datagram semantics, bind/connect behavior, one-packet receive operations, PMTU discovery, error handling and UDP offload options.

https://man7.org/linux/man-pages/man7/udp.7.html

INTERNET STANDARDRFC 768 — User Datagram Protocol

The compact base UDP specification: ports, length, checksum and the minimal message-oriented service above IP.

https://www.rfc-editor.org/info/rfc768/

BEST CURRENT PRACTICEBCP 145 / RFC 8085 — UDP Usage Guidelines

Best-current-practice guidance explaining that protocols built on UDP must still address congestion, message sizing, reliability needs, checksums and related operational concerns.

https://www.rfc-editor.org/info/bcp145/

A hostname is not an IP address: getaddrinfo() → NSS → resolver → DNS cache/servers → address candidates

Network applications usually begin with a human-oriented name, but connect() needs a socket address. On a typical glibc Linux system, getaddrinfo() first participates in the Name Service Switch (NSS) policy for the hosts database. That policy may consult local files or other naming services; DNS is one possible source, not a synonym for every hostname lookup.

APPLICATION getaddrinfo("www.example", "443", hints, &result) ↓ GLIBC / NSS HOST LOOKUP POLICY /etc/nsswitch.conf hosts: ... ├── files → /etc/hosts may answer immediately ├── dns → resolver path below └── other configured NSS modules may participate DNS RESOLVER PATH stub resolver reads resolver configuration nameserver / search / options policy ↓ query recursive resolver for resource records QNAME = requested name QTYPE = A and/or AAAA as appropriate ↓ UDP or TCP transport as required recursive resolver cache HIT? ├── yes → cached RRset subject to TTL └── no → resolver follows DNS hierarchy/referrals root → TLD → authoritative zone server ↓ answer may contain A / AAAA and possibly CNAME chain ↓ getaddrinfo() returns one or more struct addrinfo candidates family + socket type + protocol + sockaddr ↓ application tries candidate address(es) ↓ socket()/connect() → routing/neighbor/TCP path described elsewhere

DNS / resolver term	Role
NSS	glibc policy layer that decides which configured sources supply host, user, group and other name-service databases.
stub resolver	Client-side resolver code that sends requests to one or more configured recursive DNS resolvers rather than walking the entire hierarchy itself.
recursive resolver	Server that obtains the answer on the client's behalf, follows referrals as needed and normally maintains a cache.
authoritative server	Name server serving authoritative data for a DNS zone.
A / AAAA	Resource-record types carrying IPv4 and IPv6 addresses respectively.
CNAME	Alias record pointing one owner name toward another canonical name; resolution may therefore require following additional records.
TTL	Cache lifetime supplied with DNS resource records. It limits how long cached data can normally be reused without refreshing it.
EDNS(0)	Extension mechanism that expands DNS protocol capability, including advertising larger UDP payload capacity than the original 512-byte DNS limit.

DNS does not create the TCP connection. It produces naming information—often one or more candidate IP addresses. Route lookup, ARP/neighbor discovery, TCP handshaking and later TLS are separate layers that happen afterward.

DNS / RESOLVER LAB getent ahosts example.com cat /etc/nsswitch.conf | grep '^hosts:' cat /etc/resolv.conf # If available, compare recursive/default lookup with explicit record queries dig example.com A dig example.com AAAA dig +trace example.com # See actual resolver traffic (interface may differ) sudo tcpdump -ni any port 53

PUBLIC MANUALgetaddrinfo(3) — name/service to socket-address candidates

Current Linux manual for the modern protocol-independent API that applications commonly use before creating/connecting sockets.

https://man7.org/linux/man-pages/man3/getaddrinfo.3.html

PUBLIC MANUALnsswitch.conf(5) — which naming source is consulted

Shows how glibc selects and orders sources such as files and dns for the hosts database.

https://man7.org/linux/man-pages/man5/nsswitch.conf.5.html

PUBLIC MANUALresolv.conf(5) — stub-resolver configuration

Current manual for configured resolver servers, search domains and resolver options used by the traditional glibc DNS resolver path.

https://man7.org/linux/man-pages/man5/resolv.conf.5.html

PUBLIC RFCRFC 1034 — DNS concepts and facilities

The foundational architecture: hierarchical namespace, zones, resolvers, authoritative servers, recursive queries, referrals and caching.

https://www.rfc-editor.org/rfc/rfc1034.html

PUBLIC RFCRFC 1035 — DNS message and resource-record format

Foundational wire-format specification for queries/responses, labels, record types and DNS transport over port 53.

https://www.rfc-editor.org/rfc/rfc1035.html

INTERNET STANDARDRFC 6891 / STD 75 — EDNS(0)

Modern extension mechanism that removes important limits of the original DNS wire protocol, including its small unextended UDP message size.

https://www.rfc-editor.org/info/rfc6891/

DNSSEC adds a signed chain of trust to DNS: trust anchor → DS/DNSKEY → RRSIG → validated RRset

Ordinary DNS tells a resolver what data a name server returned; by itself it does not cryptographically prove that the data is authentic. DNSSEC signs DNS resource-record sets and links child-zone keys to parent zones with DS records. A validating resolver starts from a configured trust anchor—commonly the DNS root—and verifies a chain down to the requested signed data.

VALIDATING RESOLVER HAS A TRUST ANCHOR root DNSKEY (trusted starting point) ↓ verify root RRSIGs root zone delegates example TLD ↓ parent publishes DS = digest/reference to child DNSKEY ↓ fetch child DNSKEY RRset verify DS matches an acceptable child DNSKEY verify DNSKEY RRset signatures ↓ repeat at each signed delegation ↓ requested zone DNSKEY validates its signed RRset A / AAAA / MX / etc + RRSIG ↓ cryptographic verification succeeds? ├── yes → SECURE answer └── no, where security was expected → BOGUS UNSIGNED CHILD UNDER A SIGNED PARENT signed proof that no DS exists ↓ branch can be classified INSECURE rather than forged-secure AUTHENTICATED DENIAL NSEC / NSEC3 + signatures can prove that a name/type does not exist.

DNSSEC object/state	Role
DNSKEY	Public keys published by a zone for verifying signatures over that zone's DNS data.
RRSIG	Signature covering an RRset; validators verify it using an appropriate DNSKEY.
DS	Parent-zone record that authenticates a digest/reference to a child-zone DNSKEY and therefore links the trust chain across a delegation.
NSEC / NSEC3	Signed denial-of-existence mechanisms used to authenticate negative answers rather than merely asserting “not found.”
secure	The validator can build a trust chain and successfully verify the relevant signatures.
insecure	The resolver can prove that a branch is deliberately unsigned; this is different from a broken signature.
bogus	DNSSEC validation was expected but cryptographic/protocol checks failed.

DNSSEC does not encrypt DNS. Its core purpose is origin authentication and integrity of DNS data. DNS-over-TLS/HTTPS/QUIC protect the transport channel; they solve a different problem and can be used with or without DNSSEC.

DNSSEC LAB # Ask for DNSSEC records/validation-related flags if dig is available: dig +dnssec example.com A dig . DNSKEY +dnssec dig com DS +dnssec # Trace delegation while requesting DNSSEC data: dig +trace +dnssec example.com # Look for DNSKEY, DS, RRSIG and NSEC/NSEC3 records. # A local validating resolver may expose an AD (Authenticated Data) bit, # but a stub must understand what it is trusting when relying on that bit.

BEST CURRENT PRACTICERFC 9364 / BCP 237 — DNS Security Extensions (DNSSEC)

Current best-current-practice map of the DNSSEC standards family. It identifies RFCs 4033/4034/4035 as the core and explains DNSSEC as origin authentication for DNS data.

https://www.rfc-editor.org/rfc/rfc9364.html

STANDARDS TRACK RFCRFC 4034 — DNSKEY, RRSIG, DS and DNSSEC resource records

Normative record formats and wire-level objects used to publish keys, signatures and delegation-signer information.

https://www.rfc-editor.org/rfc/rfc4034.html

STANDARDS TRACK RFCRFC 4035 — DNSSEC signing, serving and validation behavior

Core protocol rules for signed zones, security-aware resolvers, authentication chains and validation outcomes.

https://www.rfc-editor.org/rfc/rfc4035.html

DNSSEC authenticates DNS data; DoT, DoH and DoQ encrypt the transport to a resolver

Classic DNS commonly sends queries in cleartext over UDP or TCP, so an on-path observer can see or modify the traffic even when the answer is later validated with DNSSEC. Encrypted DNS transports solve a different problem: they create a confidential/authenticated channel between two DNS endpoints. DNS over TLS (DoT) carries DNS messages inside TLS; DNS over HTTPS (DoH) maps each DNS exchange into HTTPS; DNS over QUIC (DoQ) maps DNS onto dedicated QUIC connections.

APPLICATION ↓ hostname lookup stub resolver ↓ choose resolver + transport ├─ classic DNS → UDP/TCP, normally plaintext ├─ DoT → DNS framing over TLS/TCP ├─ DoH → DNS request/response carried by HTTPS └─ DoQ → DNS messages over dedicated QUIC connection ↓ authenticated/encrypted channel to chosen recursive resolver ↓ resolver cache / recursive lookup ↓ authoritative DNS hierarchy ↓ answer returns through encrypted client↔resolver channel DNSSEC, if used, is orthogonal: RRset + RRSIG + DNSKEY/DS chain → validate origin/integrity of DNS data

Mechanism	Protects	Does not automatically provide
DoT	DNS transport confidentiality/integrity using TLS, conventionally on a dedicated service endpoint.	DNSSEC validation or privacy beyond the TLS peer/resolver.
DoH	DNS exchanges inside HTTPS, sharing HTTP/TLS machinery and deployment patterns.	Proof that the DNS RRset itself is DNSSEC-authentic.
DoQ	Encrypted DNS over QUIC with QUIC loss recovery and independent transport streams/messages as defined by the protocol.	End-to-end secrecy from the recursive resolver to every authoritative server.
DNSSEC	Cryptographic authenticity/integrity of signed DNS data through a chain of trust.	Confidentiality; signed names and answers can still be visible on a plaintext transport.

Encrypted DNS moves the observation boundary; it does not erase it. Your local network may no longer see the query contents, but the selected recursive resolver still receives them, and subsequent recursive-to-authoritative traffic follows whatever transports those servers use. Resolver choice and policy therefore remain part of the privacy model.

STANDARDS TRACKRFC 7858 — DNS over TLS (DoT)

Defines DNS carried over TLS to provide transport privacy and integrity, with usage/performance considerations for encrypted resolver traffic.

https://www.rfc-editor.org/info/rfc7858/

STANDARDS TRACKRFC 8484 — DNS Queries over HTTPS (DoH)

Defines how DNS query/response pairs are mapped into HTTPS exchanges and identifies the DNS-specific media type and HTTP behavior.

https://www.rfc-editor.org/info/rfc8484/

STANDARDS TRACKRFC 9250 — DNS over Dedicated QUIC Connections (DoQ)

Defines DNS over QUIC with transport confidentiality, QUIC loss recovery and mappings for ordinary queries as well as broader DNS uses.

https://www.rfc-editor.org/info/rfc9250/

TCP gives you reliable bytes; TLS turns that byte stream into an authenticated encrypted channel

After DNS and the TCP three-way handshake, an HTTPS-style client still does not yet have a secure application channel. TLS 1.3 runs above a reliable ordered transport. Its handshake negotiates protocol parameters, establishes fresh shared key material, authenticates the server in the common certificate-based case, binds authentication to the handshake transcript, and derives traffic keys. The TLS record layer then protects application data with authenticated encryption.

TCP CONNECTION ESTABLISHED client server │ │ ├── ClientHello --------------------------------------→│ │ supported versions / algorithms │ │ key_share / signature algorithms / extensions │ │ │ │←-------------------------------------- ServerHello ──┤ │ selected parameters + key share │ │ │ both sides can derive handshake traffic secrets │ │ │ │←---------------------------- {EncryptedExtensions} ──┤ │←----------------------------------- {Certificate*} ──┤ │←----------------------------- {CertificateVerify*} ──┤ │←-------------------------------------- {Finished} ───┤ │ │ │ verify certificate chain + service identity policy │ │ verify transcript signature / Finished │ │ │ ├── {Finished} ---------------------------------------→│ │ │ │====== TLS application traffic keys established ======│ │←============= encrypted/authenticated records ======→│ │ HTTP or another application protocol │ {} = protected with handshake traffic keys [] = application traffic protected with application keys

TLS piece	What it proves/does
ClientHello	Starts negotiation and carries supported versions, cryptographic choices, key shares and extensions such as the server name/application protocol where applicable.
ServerHello	Selects key-establishment parameters. Together with the client's contribution it lets the peers derive handshake secrets.
Certificate	Common server-authentication credential chain. Possessing a certificate alone is not sufficient; the client must validate the chain and verify the intended service identity.
CertificateVerify	Signature over the handshake transcript proving possession of the private key corresponding to the authentication credential.
Finished	Keyed authenticator over the handshake transcript providing key confirmation and detecting handshake tampering.
HKDF/key schedule	Derives separated handshake/application traffic secrets and keys from the established shared secret and transcript context.
record layer	Frames and protects post-handshake data using authenticated encryption; TCP reliability remains underneath it.
0-RTT	Optional resumed-session mode allowing some early application data before the full handshake completes, with weaker replay properties than ordinary 1-RTT application data.

Encryption is not the same as knowing who is on the other end. A usable HTTPS-style security decision combines TLS key establishment with certificate/path validation and service-identity verification for the name the client intended to reach. DNS resolution, TCP delivery and TLS authentication solve different problems.

TLS LAB # Inspect the negotiated connection and certificate chain openssl s_client -connect example.com:443 -servername example.com -showcerts # Show protocol/cipher summary openssl s_client -connect example.com:443 -servername example.com -tls1_3 </dev/null 2>/dev/null | grep -E 'Protocol|Cipher|Peer certificate|Verification' # Packet captures can show record boundaries and cleartext handshake metadata, # but application data is encrypted after keys are established. sudo tcpdump -ni any host example.com and tcp port 443

STANDARDS TRACK RFCRFC 9846 — current TLS 1.3 specification

Current 2026 TLS 1.3 specification. It obsoletes RFC 8446 while retaining TLS version 1.3, and describes the handshake, key schedule, authentication messages and record protection.

https://www.rfc-editor.org/info/rfc9846/

PUBLIC RFCRFC 9525 — Service Identity in TLS

Explains the separate identity-verification step used when a TLS client authenticates an application service through PKIX/X.509 certificates.

https://www.rfc-editor.org/info/rfc9525/

A TLS certificate is useful only after the client builds and validates a path to a trust anchor and checks the intended service name

The TLS handshake section shows the server sending authentication material and proving possession of its private key. The client still has a separate PKIX problem: construct a plausible certificate path, validate every relevant signature/constraint in that path, anchor it in locally trusted configuration, and then verify that the leaf certificate identifies the service the application actually intended to reach. These checks authenticate a name-to-key binding; they do not make DNS or TCP trustworthy by themselves.

SERVER MAY SEND leaf certificate: example.com public key ↓ issued by intermediate CA certificate ↓ issued by another intermediate (optional) CLIENT ALSO HAS LOCAL TRUST ANCHORS (root CA keys/certificates or equivalent trusted configuration) PATH BUILDING leaf → intermediate(s) → candidate trust anchor ↓ PATH VALIDATION for relevant certificates/links: verify issuer signatures check validity interval reject unsupported critical extensions enforce Basic Constraints / CA status enforce key usage / extended key usage as applicable enforce path-length / name constraints / policy rules as applicable ↓ path terminates at an acceptable local trust anchor? ├─ no → authentication fails └─ yes ↓ SERVICE IDENTITY CHECK intended DNS name = www.example.com ↓ compare against leaf subjectAltName dNSName rules (wildcard rules are constrained; do not fall back to legacy Common Name) ↓ match? ── no → authentication fails │ yes ↓ certificate/path + service identity accepted ↓ TLS CertificateVerify/Finished checks bind the authenticated key and transcript to this connection REVOCATION / STATUS CRL or OCSP information may be consulted under application/ecosystem policy; revocation processing is a separate operational policy question from basic signature/path construction.

PKI concept	Meaning
leaf / end-entity certificate	Certificate for the server/service key being authenticated.
intermediate CA	CA certificate between the leaf and a configured trust anchor; servers commonly send needed intermediates.
trust anchor	Public key/name information trusted by local configuration. It is an input to path validation, not something made trustworthy merely because the peer sent it.
certification path	Ordered chain from the target certificate through issuer certificates to an acceptable trust anchor.
Basic Constraints	X.509 extension stating whether a certificate may act as a CA and optionally constraining path length.
Key Usage / EKU	Extensions constraining the cryptographic purposes for which a key/certificate is acceptable.
subjectAltName (SAN)	Extension carrying identities such as DNS names. Current TLS service-identity guidance checks DNS identities here rather than in the legacy Common Name.
name constraints	CA-imposed restrictions on namespaces that subordinate certificates may validly identify.
CRL / OCSP	Mechanisms for conveying revocation/status information; how strictly they are required is application/ecosystem policy.

The root certificate usually does not need to arrive from the server. A trust anchor is trusted because the client configured it (directly or through an OS/application trust store). A peer cannot make a new root trustworthy simply by appending a self-signed certificate to the handshake.

CERTIFICATE-VALIDATION LAB # Show exactly what the server sends openssl s_client -connect example.com:443 -servername example.com -showcerts # Fail the command if certificate verification fails openssl s_client -connect example.com:443 -servername example.com \ -verify_return_error </dev/null # Inspect one saved certificate in human-readable form openssl x509 -in cert.pem -noout -subject -issuer -dates -ext subjectAltName # Verify a leaf against an explicit CA path/file (paths vary by OS) openssl verify -CAfile trust-anchors.pem -untrusted intermediates.pem leaf.pem

STANDARDS TRACK RFCRFC 5280 — X.509 PKIX certificate and CRL profile

The core PKIX specification, including certificate syntax and the path-validation procedure that checks issuer relationships, signatures, validity, constraints, critical extensions and trust-anchor termination.

https://www.rfc-editor.org/info/rfc5280/

STANDARDS TRACK RFCRFC 6960 — Online Certificate Status Protocol (OCSP)

Defines certificate-status queries/responses as an alternative to relying only on downloaded CRLs. RFC 6960 is Standards Track and has subsequent updates.

https://www.rfc-editor.org/info/rfc6960/

For the current DNS-name matching rules, see the RFC 9525 resource in the preceding TLS 1.3 section; it is intentionally not duplicated here.

kTLS moves the TLS record data path into the kernel after the handshake, so encryption can integrate with sendfile, TCP and NIC offload

A normal userspace TLS library performs both the handshake and record encryption/decryption before calling ordinary socket I/O. Linux kernel TLS (kTLS) splits those jobs. After a TCP connection is established and userspace has completed the TLS handshake, the application can install the resulting symmetric record-layer state on the socket. The kernel TLS ULP can then frame and protect application records in software or, on supported NICs, hand cryptographic work to hardware while preserving the normal TCP stack.

USERSPACE TLS LIBRARY TCP connect ↓ TLS handshake: negotiate version/cipher, authenticate peer, derive traffic keys ↓ install TLS ULP + TX/RX crypto state on socket ↓ KERNEL DATA PATH (kTLS) application write / sendfile ↓ TLS record framing + authentication/encryption (TLS_SW) OR mark work for compatible NIC crypto offload (TLS_HW) ↓ normal TCP segmentation/retransmission/IP routing ↓ NIC RX reverses the record path and returns authenticated plaintext to the reader.

Layer	Responsibility
TLS handshake	Negotiates algorithms, authenticates peers and derives traffic secrets. For ordinary userspace TLS, this remains in the TLS library rather than being replaced by kTLS.
kTLS record layer	Consumes installed symmetric state and performs/coordinates TLS record protection on socket data.
TCP	Still supplies reliable ordered byte delivery, congestion control and retransmission beneath TLS.
TLS hardware offload	Capable NIC encrypts/decrypts selected record traffic while the kernel/TCP stack still owns connection semantics and fallbacks.

kTLS is not “TLS without userspace.” The ordinary data-path optimization starts after the cryptographic session has been established and its record keys/configuration are supplied to the kernel. Moving record processing down a layer can reduce copies/context transitions and allows integration with kernel send paths and NIC crypto features.

KERNEL DOCSLinux Kernel TLS userspace interface

Explains enabling the TCP TLS ULP, installing TX/RX cryptographic state after the handshake, the software record layer and optional zero-copy-related optimizations.

https://docs.kernel.org/networking/tls.html

KERNEL DOCSLinux kTLS NIC offload

Detailed TX/RX path for software versus hardware TLS, including how drivers advertise TLS offload, how packets remain integrated with TCP, and how software fallback handles unsupported/error cases.

https://docs.kernel.org/networking/tls-offload.html

SSH is three protocol layers over one secure connection: transport → user authentication → multiplexed channels

SSH is not simply “TLS for a shell.” The protocol family separates concerns. The transport layer negotiates algorithms, performs key exchange, authenticates the server host with a host key and establishes encryption/integrity. The user-authentication layer then proves which account the client may use. Finally, the connection layer multiplexes independent logical channels for a shell, a command, a PTY, port forwarding and related services over the same protected connection.

CLIENT SERVER TCP connection (normally port 22) │──────────────────────────────────────────────────→│ │ protocol identification strings │ │←─────────────────────────────────────────────────→│ │ │ SSH TRANSPORT LAYER algorithm negotiation + key exchange server proves possession of HOST private key client checks expected host identity / known_hosts policy │ └── encrypted + integrity-protected transport established SSH USER AUTHENTICATION client requests account name public-key / keyboard-interactive / password / other configured method server accepts or rejects USER identity │ └── authenticated user session SSH CONNECTION PROTOCOL one protected transport, many logical channels: channel 0 → interactive shell + optional PTY channel 1 → remote command channel 2 → TCP forwarding ... flow-control windows and channel open/close messages ... HOST AUTHENTICATION ≠ USER AUTHENTICATION host key answers: “which server did I reach?” user authentication answers: “which account may this client use?”

Layer/object	Role
host key	Longer-lived server identity key used during key exchange to authenticate the server side of the transport.
known_hosts	Client-side remembered host-key database/policy input. A changed host key is security-significant because encryption alone does not tell you which machine is on the other end.
session identifier	Value derived from the first key exchange and reused by higher SSH layers, including binding user-authentication signatures to this SSH session.
user authentication	Protocol above the protected transport for methods such as public-key, password or keyboard-interactive authentication.
channel	Multiplexed logical byte stream inside one SSH connection. Interactive shell, command execution and forwarded connections use channels rather than separate encrypted transports.
PTY request	For an interactive login the client can request a server-side pseudoterminal; this connects directly to the TTY/PTY machinery described elsewhere on this page.

Encryption without host verification is not enough. SSH transport cryptography can protect bytes from passive observers while a client still talks securely to the wrong machine. Host-key verification is what binds the encrypted transport to an expected server identity.

STANDARDS TRACK RFCRFC 4253 — SSH Transport Layer Protocol

Defines the transport-layer architecture: version exchange, algorithm negotiation, key exchange, server host authentication, encryption and integrity protection.

https://www.rfc-editor.org/info/rfc4253/

STANDARDS TRACK RFCRFC 4252 — SSH Authentication Protocol

Defines the user-authentication layer above SSH transport, including public-key, password and host-based methods.

https://www.rfc-editor.org/info/rfc4252/

STANDARDS TRACK RFCRFC 4254 — SSH Connection Protocol

Defines multiplexed channels for interactive sessions, command execution, PTYs and forwarding on top of the authenticated SSH transport.

https://www.rfc-editor.org/info/rfc4254/

CURRENT MANUALOpenSSH ssh(1) manual — host keys, authentication and PTYs

Current OpenSSH client manual showing known_hosts behavior, available authentication methods, interactive PTY allocation and remote command/session behavior.

https://man.openbsd.org/ssh

QUIC is not “TCP inside UDP”: it combines secure connection setup, reliable streams and loss recovery above UDP

QUIC uses UDP as the IP-facing datagram substrate, but implements its own connection semantics above it. A QUIC endpoint encrypts almost all transport metadata, carries one or more independent ordered byte streams inside QUIC frames, acknowledges packet-number ranges, detects losses, controls congestion and derives packet-protection keys through TLS 1.3. HTTP/3 is one application protocol built on QUIC; QUIC itself is the transport.

APPLICATION (for example HTTP/3) ↓ QUIC STREAMS stream 0 bytes ─┐ stream 4 bytes ─┼─→ STREAM frames + control frames stream 8 bytes ─┘ ↓ QUIC packetization ├── packet number ├── connection ID(s) ├── encrypted/authenticated payload └── ACK-eliciting / control semantics ↓ UDP datagram ↓ IP → link → network CONNECTION SETUP client Initial packet + TLS ClientHello ↓ QUIC crypto frames carry TLS 1.3 handshake messages ↓ TLS derives handshake/application secrets ↓ QUIC derives packet-protection keys for its encryption levels ↓ application streams become available according to handshake state LOSS OF ONE QUIC PACKET packet number 1201 lost ↓ ACK ranges / timers reveal loss ↓ transport information is retransmitted in NEW packet(s) with NEW packet numbers; packet 1201 itself is not replayed as “the same packet” ↓ congestion controller reduces/adjusts sending as required MULTIPLEXING loss affecting stream A need not stop delivery of already-received data on stream B. This avoids TCP's single-byte-stream head-of-line coupling between independent streams.

Mechanism	What it is doing
UDP	Provides the userspace-accessible datagram substrate and source/destination ports; UDP itself does not supply QUIC reliability.
connection ID	Identifies a QUIC connection independently of only the IP-address/port 4-tuple, supporting connection continuity across some path changes.
packet number	Monotonically increasing number within a QUIC packet-number space used for acknowledgment, nonce construction and loss detection; retransmitted information goes in a newly numbered packet.
stream	Independent ordered byte stream with its own offset/flow-control state; one connection can multiplex many streams.
ACK ranges	Report received packet-number ranges, letting a sender distinguish delivered, reordered and plausibly lost packets.
TLS 1.3 in QUIC	Authenticates endpoints according to TLS policy and establishes cryptographic secrets; QUIC maps those secrets into packet protection rather than running the normal TLS record layer over a TCP byte stream.
PTO / loss detection	Timers and acknowledgment evidence keep progress moving when packets or acknowledgments disappear.
congestion control	Limits bytes in flight and sending rate according to path feedback, analogous in purpose to TCP congestion control but implemented by the QUIC endpoint.

Two common misconceptions: QUIC does not make UDP reliable; QUIC itself implements reliability where required. And encryption does not remove congestion control: a QUIC sender is still responsible for sharing the path safely and reacting to loss/ECN/RTT signals.

OFFICIAL RFCRFC 9000 — QUIC: A UDP-Based Multiplexed and Secure Transport

The base transport specification: connections, streams, packet numbers, connection IDs, frames, flow control, migration and transport behavior.

https://www.rfc-editor.org/rfc/rfc9000.html

OFFICIAL RFCRFC 9001 — Using TLS to Secure QUIC

Defines how TLS 1.3 handshake messages and secrets are integrated with QUIC encryption levels and packet protection.

https://www.rfc-editor.org/rfc/rfc9001.html

OFFICIAL RFCRFC 9002 — QUIC Loss Detection and Congestion Control

Normative loss-detection, RTT/PTO and baseline congestion-control machinery for QUIC.

https://www.rfc-editor.org/rfc/rfc9002.html

HTTP semantics stay recognizable while the wire format changes: HTTP/1.1 text → HTTP/2 frames → HTTP/3 QUIC streams

HTTP is the application protocol above the transport/security layers already traced on this page. The important separation is semantics versus framing: methods, status codes, fields, resources and representations are shared concepts, while HTTP/1.1, HTTP/2 and HTTP/3 encode and multiplex those messages differently.

SAME HIGH-LEVEL REQUEST method: GET target: /manual/chapter fields: Host / authority, Accept, User-Agent, ... content: often none for GET ↓ server returns status + fields + optional representation bytes HTTP/1.1 TLS over TCP (for HTTPS) ↓ text request/status line + textual header fields ↓ message body framing from rules such as Content-Length, Transfer-Encoding: chunked, status/method semantics, or connection close ↓ persistent TCP connection can carry later requests (no native multiplexing of independent responses on one connection) HTTP/2 TLS/TCP in the common HTTPS deployment ↓ one connection contains many numbered STREAMS ↓ HTTP messages split into binary frames HEADERS / DATA / SETTINGS / WINDOW_UPDATE / ... ↓ HPACK compresses field sections ↓ multiple requests/responses can be in flight concurrently (TCP loss can still delay bytes for every stream because TCP is one ordered byte stream) HTTP/3 QUIC over UDP ↓ HTTP control + request/response data use QUIC streams ↓ QPACK compresses field sections with rules designed for QUIC's independently delivered streams ↓ loss on one QUIC stream need not impose TCP-style transport head-of-line blocking on unrelated streams

Layer/question	HTTP/1.1	HTTP/2	HTTP/3
HTTP semantics	Methods/status/fields/content	Same core semantics	Same core semantics
Transport	Usually TCP; HTTPS adds TLS	Commonly TLS over one TCP connection	QUIC, which already includes secure transport establishment
Message representation	Text start-line + fields + framed content	Binary frames on streams	HTTP/3 frames on QUIC streams
Concurrent exchanges	No native multiplexed response streams on one connection	Many HTTP streams multiplexed over one TCP byte stream	Many HTTP exchanges mapped onto QUIC streams
Field compression	No protocol field-compression layer	HPACK	QPACK
Transport head-of-line effect	TCP delivers one ordered stream	A lost TCP segment can hold delivery for all H2 streams sharing that connection	QUIC recovery is stream-aware, reducing cross-stream transport HOL blocking

HTTP/2 and HTTP/3 did not replace HTTP semantics. They replace how those semantics are represented and transported. A server can implement the same GET/POST/status/cache semantics across all three versions while using very different framing and loss-recovery machinery underneath.

NO-ACCOUNT OBSERVATION LAB # show protocol negotiation and response headers curl -v https://example.com/ -o /dev/null # where supported by your curl build/server curl --http1.1 -v https://example.com/ -o /dev/null curl --http2 -v https://example.com/ -o /dev/null curl --http3 -v https://example.com/ -o /dev/null # HTTPS protocol selection commonly uses TLS ALPN: # http/1.1 # h2 # h3 is discovered/negotiated through the HTTP/3/QUIC deployment path # packet capture can show TCP vs UDP/QUIC transport, # while encrypted HTTPS payload prevents simply reading application fields on the wire.

OPEN STANDARDRFC 9110 — HTTP Semantics

The shared semantic model: methods, status codes, fields, representations, intermediaries and the distinction between HTTP meaning and version-specific wire syntax.

https://www.rfc-editor.org/rfc/rfc9110.html

OPEN STANDARDRFC 9112 — HTTP/1.1 messaging and connection management

Defines HTTP/1.1 text message syntax, framing, parsing and persistent-connection behavior.

https://www.rfc-editor.org/rfc/rfc9112.html

OPEN STANDARDRFC 9113 — HTTP/2

Current HTTP/2 specification describing binary framing, concurrent streams, settings, flow control and HPACK-based field compression.

https://www.rfc-editor.org/info/rfc9113/

OPEN STANDARDRFC 9114 — HTTP/3

Defines how HTTP semantics are carried over QUIC streams and how HTTP/3 control/request streams differ from the HTTP/2-over-TCP model.

https://www.rfc-editor.org/rfc/rfc9114.html

OPEN STANDARDRFC 9204 — QPACK field compression for HTTP/3

Explains why HTTP/3 uses QPACK rather than directly reusing HPACK and how field compression is adapted to independently delivered QUIC streams.

https://www.rfc-editor.org/rfc/rfc9204.html

A TCP server has two different queues before your program gets a connected socket

listen() does not make the application execute once per arriving SYN. The kernel owns connection establishment. For TCP, an incoming request first exists as incomplete handshake state; after the three-way handshake succeeds, the established connection can wait in the listener's completed/accept queue until the server calls accept(). Linux therefore separates the limit for incomplete SYN state from the listen(backlog) limit for completely established connections awaiting acceptance.

SERVER SETUP socket(AF_INET/AF_INET6, SOCK_STREAM, 0) ↓ bind(local address, port) ↓ listen(fd, backlog) ↓ listener socket is passive CLIENT SERVER KERNEL connect() ↓ SYN -----------------------------------> create/track incomplete request state ↓ <------------------------------- SYN + ACK ↓ ACK -----------------------------------> handshake complete ↓ established connection waits in completed accept queue ↓ application thread blocked in accept()/epoll wakes ↓ accept4(listener, ..., SOCK_NONBLOCK|SOCK_CLOEXEC) ↓ returns NEW connected socket fd listener fd remains open for more connections TWO LIMITS incomplete handshake requests → tcp_max_syn_backlog (subject to SYN-cookie behavior) completed connections waiting for accept → listen() backlog, capped by somaxconn

Object/state	What it represents
listening socket	Passive endpoint bound to a local address/port and able to receive connection requests.
SYN/incomplete request state	Handshake work that has not yet become a fully established connection.
completed accept queue	Established connections waiting for the application to accept them.
`listen(backlog)`	Linux queue limit for completely established sockets waiting to be accepted, subject to the system `somaxconn` cap.
`accept()`	Dequeues a pending connection and returns a new connected socket file descriptor.
`accept4()`	Can atomically request flags such as nonblocking and close-on-exec on the returned descriptor.

The listener is not the connected socket. The listening file descriptor stays in the LISTEN state; every successful accept() returns another file descriptor referring to a distinct connected socket with its own transport state and peer.

CURRENT PUBLIC MANUALlisten(2) — backlog semantics

Current Linux man-page definition of passive sockets, the established-connection backlog and the separate incomplete-request limit.

https://man7.org/linux/man-pages/man2/listen.2.html

CURRENT PUBLIC MANUALaccept(2) / accept4(2)

Explains how one queued connection becomes a new connected file descriptor, blocking versus nonblocking behavior and atomic SOCK_NONBLOCK/SOCK_CLOEXEC.

https://man7.org/linux/man-pages/man2/accept.2.html

CURRENT PUBLIC MANUALtcp(7) — Linux TCP behavior and tuning

The protocol-level Linux reference tying socket API state to TCP connection management, buffering, timers and listener-related options.

https://man7.org/linux/man-pages/man7/tcp.7.html

# Observe listeners and established TCP sockets: ss -ltn ss -tn # Inspect queue-related limits without changing them: cat /proc/sys/net/core/somaxconn cat /proc/sys/net/ipv4/tcp_max_syn_backlog # A listener can become readable to epoll because accept() can make progress; # readiness is not a promise that a later blocking accept can never race.

TCP reliability and congestion control are feedback loops over sequence numbers, ACKs and timers

The previous TCP sections explain encapsulation and connection establishment. Once a connection carries data, TCP must solve two different control problems at once: reliability (which bytes arrived, which are missing, and when to retransmit) and rate control (how much unacknowledged data the sender should inject without overrunning the receiver or contributing to network congestion).

SENDER BYTE STREAM bytes 0 ........ 9999 ↓ segment into TCP sequence space send segment SEQ=1000 LEN=1000 send segment SEQ=2000 LEN=1000 ← suppose this one is lost send segment SEQ=3000 LEN=1000 ↓ RECEIVER gets 1000..1999, then 3000..3999 ↓ cumulative ACK still says next wanted byte = 2000 optional SACK blocks can report that 3000..3999 arrived anyway ↓ SENDER loss detection ├── ACK/SACK pattern indicates a hole → fast loss recovery/retransmit └── retransmission timer expires → retransmit + RTO backoff ↓ retransmit missing 2000..2999 ↓ receiver can advance cumulative ACK once contiguous byte range exists TWO WINDOWS LIMIT FLIGHT SIZE receiver flow-control window (rwnd): “I have this much receive-buffer space” congestion window (cwnd): “network feedback allows about this much in flight” usable sending window ≈ min(rwnd, cwnd) subject to protocol/implementation details CONGESTION FEEDBACK connection starts / recovers ↓ slow start grows cwnd rapidly as ACKs arrive ↓ congestion avoidance grows more cautiously ↓ loss/ECN signal → congestion algorithm reduces/adjusts sending rate ↓ continue probing available path capacity

Mechanism	What it solves
sequence number	Names positions in TCP's byte stream so reordered, duplicated and missing data can be detected.
cumulative ACK	Reports the next sequence position wanted, implicitly acknowledging the contiguous bytes before it.
SACK	Optionally reports noncontiguous received ranges so a sender can identify holes without blindly retransmitting everything after the first missing byte.
RTT estimate / RTO	Provides a retransmission deadline when ACK-based loss detection is insufficient; successive timeout recovery uses exponential backoff.
rwnd	Receiver-advertised flow-control limit protecting receive-buffer capacity.
cwnd	Sender-side congestion-control limit reflecting inferred path capacity/congestion, independent of the receiver's buffer advertisement.
slow start / congestion avoidance	Algorithms for increasing the congestion window while probing how much traffic the path can sustain.
ECN	Allows compatible network devices/endpoints to signal congestion without requiring packet loss as the only signal.

Flow control is not congestion control. A receiver can advertise a huge window while the network path is congested, or a path can be empty while the receiver has almost no buffer space. TCP therefore respects both limits. Modern systems can implement multiple congestion-control algorithms, but the reliable byte-stream and standards-level congestion requirements remain the common foundation.

INTERNET STANDARDRFC 9293 — Transmission Control Protocol

The consolidated current TCP Internet Standard: sequence space, acknowledgments, retransmission requirements, connection state and the requirement to implement baseline congestion-control behavior.

https://www.rfc-editor.org/rfc/rfc9293.html

STANDARDS TRACKRFC 5681 — TCP Congestion Control

Defines the classic slow-start, congestion-avoidance, fast-retransmit and fast-recovery framework used as the standards baseline.

https://www.rfc-editor.org/rfc/rfc5681.html

STANDARDS TRACKRFC 6298 — Computing TCP's Retransmission Timer

Defines RTT sampling, retransmission-timeout calculation and exponential backoff of the RTO.

https://www.rfc-editor.org/rfc/rfc6298.html

# Linux exposes per-connection transport estimates through ss: ss -tin # Typical fields can include RTT/RTO, cwnd, retransmission and pacing data. # Inspect which congestion-control algorithms this kernel currently exposes: sysctl net.ipv4.tcp_congestion_control sysctl net.ipv4.tcp_available_congestion_control

One reliable byte stream can use several ordinary TCP paths: MPTCP separates connection-level data from subflow transport

Multipath TCP (MPTCP) keeps the application-facing abstraction of a reliable ordered byte stream while allowing that one connection to use multiple underlying TCP subflows. The first subflow negotiates MPTCP capability during the TCP handshake. Additional subflows can later be created over other local/remote address pairs; connection-level sequence information maps application data onto the sequence spaces of those individual TCP flows. If one path fails, unacknowledged connection data can be reinjected on another surviving subflow without forcing the application to create a new logical connection.

APPLICATION read()/write() one reliable byte stream ↓ MPTCP CONNECTION connection-level data sequence space ↓ path manager chooses/creates paths ├── TCP subflow A: Wi-Fi addr → server addr │ own TCP 4-tuple + seq/ACK state ├── TCP subflow B: cellular addr → server addr │ own TCP 4-tuple + seq/ACK state └── optional more subflows ↓ packet scheduler chooses where to send data ↓ receiver maps subflow bytes back into ONE ordered MPTCP data stream initial handshake: SYN + MP_CAPABLE → negotiate MPTCP if peer supports it later path: new TCP handshake + MP_JOIN → attach another subflow path fails: connection can retransmit/reinject missing DATA on a surviving path instead of exposing a broken socket immediately.

Layer/object	What it means
MPTCP connection	The application-visible reliable byte stream and connection-level ordering/lifetime.
subflow	An ordinary TCP flow carrying some MPTCP data, with its own IP addresses, ports, TCP sequence numbers, RTT and congestion state.
`MP_CAPABLE`	TCP option used on the initial handshake to negotiate MPTCP support and establish connection-level keys/state.
`MP_JOIN`	Mechanism for authenticating and attaching an additional TCP subflow to an existing MPTCP connection.
DSS mapping	Data Sequence Signal information that relates connection-level data sequence numbers to bytes carried in a subflow sequence space.
path manager	Chooses/announces usable addresses and creates/removes subflows according to policy.
packet scheduler	Chooses which available subflow should carry the next connection data.

MPTCP is not link aggregation. Ethernet bonding or Wi-Fi radio scheduling combines links below IP; MPTCP operates at the transport layer and can span paths with different IP addresses and different networks. It also is not QUIC: MPTCP retains TCP subflows and TCP-compatible application semantics, whereas QUIC implements its transport above UDP.

IETF STANDARDRFC 8684 — Multipath TCP v1

Normative MPTCP protocol: MP_CAPABLE, MP_JOIN, additional subflows, connection-level data sequencing, address signaling and fallback behavior.

https://www.rfc-editor.org/rfc/rfc8684.html

KERNEL DOCSLinux Multipath TCP — concepts and socket API

Current Linux documentation explaining subflows, path managers, packet schedulers, fallback to plain TCP and the IPPROTO_MPTCP socket interface.

https://docs.kernel.org/networking/mptcp.html

KERNEL DOCSLinux MPTCP sysctls and policy controls

Current per-network-namespace controls for enabling MPTCP, selecting path-manager/scheduler behavior and related connection policy.

https://docs.kernel.org/networking/mptcp-sysctl.html

Closing TCP is a protocol exchange, not a local `close()`: FIN, half-close, LAST-ACK and TIME-WAIT

TCP is full duplex, so each direction has to be shut down independently. A FIN consumes sequence space and means “I will send no more bytes in this direction”; it does not erase already queued data and it does not prevent the peer from continuing to send in the opposite direction. This is why a graceful close can pass through several states instead of disappearing the instant one process calls close().

PEER A actively closes PEER B ESTABLISHED ESTABLISHED │ close()/shutdown(SHUT_WR) ├── FIN ----------------------------------→ receives EOF after prior bytes FIN-WAIT-1 CLOSE-WAIT ←---------------------------------- ACK ───┤ FIN-WAIT-2 CLOSE-WAIT │ application eventually closes ←---------------------------------- FIN ───┤ TIME-WAIT ├── ACK ----------------------------------→ LAST-ACK → CLOSED │ │ retain 4-tuple/sequence safety state for 2×MSL │ retransmit final ACK if peer repeats FIN ↓ CLOSED RST path: abort connection instead of graceful FIN exchange. Simultaneous close: both sides can pass through CLOSING and TIME-WAIT.

State	Meaning
FIN-WAIT-1	Local FIN sent; waiting for its ACK and/or the peer's FIN.
FIN-WAIT-2	Local FIN acknowledged; waiting for the peer to finish its sending direction.
CLOSE-WAIT	Peer FIN received and acknowledged, but the local application has not closed its sending direction yet. Large numbers here often point to application lifecycle bugs.
LAST-ACK	Passive closer has now sent its own FIN and waits for the final ACK.
TIME-WAIT	Active closer keeps protocol state after the final ACK so delayed duplicates from the old connection cannot contaminate a new incarnation using the same endpoint tuple and so a repeated FIN can be acknowledged.
RST	Abort/reset: discards normal graceful-close semantics and tells the peer the connection no longer exists.

TIME-WAIT is a correctness mechanism, not a socket leak. The endpoint that performs the active close normally retains TIME-WAIT state for twice the Maximum Segment Lifetime. Reusing connection tuples too aggressively can make delayed packets from an earlier incarnation ambiguous; RFC 1337 documents hazards caused by prematurely destroying TIME-WAIT state.

IETF RFCRFC 1337 — TIME-WAIT Assassination Hazards in TCP

Classic technical explanation of why TIME-WAIT exists, how old duplicate segments can endanger later connections and why simply killing the state early can be unsafe.

https://www.rfc-editor.org/rfc/rfc1337.html

PUBLIC MANUALshutdown(2) — close one or both socket directions

Linux/POSIX interface that makes TCP's independent read/write directions visible to applications through SHUT_RD, SHUT_WR and SHUT_RDWR.

https://man7.org/linux/man-pages/man2/shutdown.2.html

# Observe TCP lifecycle states on Linux: ss -tan # Filter common close states: ss -tan state time-wait ss -tan state close-wait # A high CLOSE-WAIT count usually means the peer closed, # but the local process has not closed its side yet.

One transmitted packet: send() → TCP/IP → qdisc → ndo_start_xmit() → TX DMA ring → NIC → wire

The receive section follows a packet from cable to socket. The transmit direction is not simply that diagram backwards: TCP may buffer/coalesce data, GSO can represent many future packets in one skb, qdiscs may queue/reorder/police traffic, and the NIC driver must manage finite TX descriptors plus DMA ownership before the MAC can emit frames.

APPLICATION send(sock, user_buf, 64 KiB, ...) ↓ SYSCALL + copy/pin/zero-copy path as applicable socket send-buffer accounting ↓ TCP copy/reference application bytes into skb payload/frags assign sequence numbers build TCP header/checksum metadata cork/coalesce/segment policy ↓ IP route lookup build IP header next-hop/neighbor resolution as needed ↓ Ethernet/neighbour layer prepend destination/source MAC + EtherType ↓ dev_queue_xmit(skb) QDISC LAYER classify / shape / schedule / queue according to active qdisc examples: noqueue, fq_codel, fq, mq + child qdiscs ↓ when packet selected pick netdev TX queue ↓ driver ndo_start_xmit(skb, netdev) DRIVER TX PATH check TX descriptor/ring space ↓ DMA-map skb linear data + page frags ↓ fill hardware TX descriptors: DMA addresses / lengths checksum/TSO/GSO offload metadata packet boundary / ownership bits ↓ publish descriptors with required memory ordering ↓ advance producer/tail MMIO/memory doorbell to NIC ↓ netdev_tx_sent_queue() / BQL accounts bytes queued to hardware NIC DMA reads packet bytes from RAM if TSO: segments large skb into MTU-sized TCP/IP frames computes checksum/FCS where offloaded MAC transmits frame → PCS/PMA/PHY → cable TX COMPLETION NIC updates completion descriptor/head ↓ IRQ/NAPI/poll driver discovers completed descriptors DMA-unmaps buffers frees/recycles skb netdev_tx_completed_queue() updates BQL ↓ if driver stopped TX queue because ring was full and room now exists: netif_tx_wake_queue() ↓ qdisc/network stack can feed more packets SOCKET BACKPRESSURE too much unacknowledged/queued data fills send buffer ↓ blocking send() sleeps OR nonblocking send returns EAGAIN ↓ ACKs/TX freeing/protocol progress eventually releases space and wakes writer

TX concept	Role
socket send buffer	Kernel memory/accounting bounding outstanding application transmit data.
qdisc	Queueing discipline between protocol stack and device queue; can schedule, shape, classify or drop packets.
dev_queue_xmit()	Core networking entry handing an skb toward qdisc/device transmit processing.
netdev TX queue	Software/hardware-facing transmit queue associated with a network device.
ndo_start_xmit()	Driver callback that maps/fills hardware descriptors and takes responsibility for eventual skb completion/freeing.
TX descriptor ring	Device-consumed ring describing packet DMA addresses, lengths and offload controls.
BQL	Byte Queue Limits: dynamic accounting limiting bytes queued in hardware to reduce excessive device-queue latency.
netdev_tx_sent_queue()	Driver reports bytes queued to hardware for BQL/DQL accounting.
netdev_tx_completed_queue()	Driver reports completed bytes/packets to release BQL accounting.
netif_tx_stop_queue()	Stops stack from feeding a hardware queue when descriptors/resources are exhausted.
netif_tx_wake_queue()	Restarts a stopped queue after completion frees enough descriptors.
TSO	NIC converts one large TCP skb into multiple wire-sized TCP/IP packets.
TX completion	Hardware/driver event proving DMA use of descriptors/buffers is finished so mappings/skbs can be reclaimed.

NETDEV_TX_OK does not mean the frame is already on the wire. It means the driver accepted responsibility for the skb. Current Linux driver documentation requires the driver to complete/free that skb in finite time; otherwise sockets can deadlock waiting for transmit-buffer space.

NETWORK TRANSMIT LAB # qdisc hierarchy/stats tc -s qdisc show dev eth0 # interface/NIC counters ip -s link show dev eth0 ethtool -S eth0 2>/dev/null | grep -Ei 'tx|queue|drop|busy|timeout' | less # offloads ethtool -k eth0 | grep -Ei 'tx-checksum|segmentation|tso|gso' # queue count / channel layout where supported ethtool -l eth0 2>/dev/null # network tracepoints vary, but useful source symbols include: # dev_queue_xmit, __dev_queue_xmit, sch_direct_xmit, # ndo_start_xmit implementations in your NIC driver. # Compare sending with a slow receiver: socket backpressure can block send() # long before the application has any direct knowledge of the NIC TX ring.

KERNEL DOCSLinux softnet driver transmit guidelines

Current TX-driver contract for ndo_start_xmit(), ring-full queue stopping/waking, scatter/gather ownership and the requirement to complete/free accepted skbs in finite time.

https://docs.kernel.org/networking/driver.html

KERNEL DOCSLinux netdevice ndo_start_xmit context

Current net_device operation documentation including ndo_start_xmit locking/execution context.

https://docs.kernel.org/networking/netdevices.html

SOURCE / FILELinux net/core/dev.c

Real core transmit path containing dev_queue_xmit/__dev_queue_xmit and handoff toward qdiscs and device driver start_xmit callbacks.

https://github.com/torvalds/linux/blob/master/net/core/dev.c

KERNEL DOCSLinux struct sk_buff

Read beside the TX path for skb fragments, GSO/checksum metadata, cloning and headroom used while protocol layers build a packet.

https://docs.kernel.org/networking/skbuff.html

KERNEL DOCSLinux segmentation offloads

Current GSO/TSO details explaining how one large software skb can become many wire packets at the driver/NIC boundary.

https://docs.kernel.org/networking/segmentation-offloads.html

MSG_ZEROCOPY avoids an application→kernel payload copy by temporarily sharing user pages with the transmit stack

Ordinary send() commonly copies application bytes into kernel-owned networking memory before returning, which lets the application immediately overwrite its buffer. Linux MSG_ZEROCOPY changes that ownership contract for large sends: the stack can reference/pin user pages instead of copying payload bytes, and later reports when those pages are safe for the application to reuse.

NORMAL send() application buffer ↓ copy_from_user / skb construction kernel-owned packet payload ↓ send() may return application may immediately overwrite original buffer ↓ protocol/qdisc/driver/DMA/NIC MSG_ZEROCOPY SETUP setsockopt(fd, SOL_SOCKET, SO_ZEROCOPY, 1) ↓ large send(..., MSG_ZEROCOPY) ↓ network stack references/pins user-backed pages where feasible ↓ headers / metadata still built by kernel ↓ qdisc → driver scatter/gather DMA mapping → NIC ↓ send() returns BEFORE page ownership is released ↓ APPLICATION MUST NOT MODIFY RELEVANT BUFFER YET ↓ stack releases all references to send-call buffer ↓ completion record queued on socket ERROR QUEUE SO_EE_ORIGIN_ZEROCOPY ↓ recvmsg(..., MSG_ERRQUEUE) consumes completion range ↓ application may reuse those completed buffers IMPORTANT zerocopy completion = kernel no longer holds the user buffer ≠ packet ACKed by TCP peer ≠ frame transmitted on wire ≠ NIC TX completion necessarily FALLBACK kernel may decide it must copy after all ↓ still sends a completion notification SO_EE_CODE_ZEROCOPY_COPIED tells application copy avoidance did not happen

Piece	Role
SO_ZEROCOPY	Socket option explicitly enabling the API so legacy callers that accidentally pass an unknown flag do not silently change semantics.
MSG_ZEROCOPY	Per-send flag requesting copy avoidance for that call; calls with and without it may be mixed.
page pin/reference	Keeps user-backed payload memory valid while the networking stack/device may still consume it.
error queue	Asynchronous channel carrying zerocopy completion notifications; applications read it with recvmsg(MSG_ERRQUEUE).
completion range	Identifies one or more successful MSG_ZEROCOPY send-call sequence numbers whose buffers are releasable.
SO_EE_CODE_ZEROCOPY_COPIED	Completion flag indicating the kernel had to fall back to a copy even though the zerocopy API contract was used.
TCP/UDP/VSOCK support	Current kernel documentation describes support for TCP, UDP and VSOCK using the virtio transport, subject to path/device constraints.

Copy avoidance has overhead. Page accounting/pinning and asynchronous completion processing cost work of their own, so it is mainly attractive for sufficiently large buffers. It complements rather than replaces sendfile()/splice(), which are especially useful when data already lives in kernel/file-backed pages.

MSG_ZEROCOPY OBSERVATION SKETCH socket(...) setsockopt(SO_ZEROCOPY) ↓ for each large buffer: send(..., MSG_ZEROCOPY) remember buffer until completion ↓ poll()/epoll() observes POLLERR when error-queue data exists ↓ recvmsg(..., MSG_ERRQUEUE) ↓ inspect sock_extended_err: ee_origin == SO_EE_ORIGIN_ZEROCOPY ee_info..ee_data = completed send-call range ee_code may report ZEROCOPY_COPIED ↓ recycle only completed application buffers

KERNEL DOCSLinux MSG_ZEROCOPY documentation

Authoritative interface and implementation guide covering SO_ZEROCOPY setup, send flags, page-sharing caveats, error-queue completion ranges, copied fallback and current TCP/UDP/VSOCK support.

https://docs.kernel.org/networking/msg_zerocopy.html

PUBLIC MANUALsendmsg(2) / send(2) flag interface

Current Linux manual for message-oriented socket sends and per-call flags; useful context for where MSG_ZEROCOPY sits relative to the ordinary sendmsg API.

https://man7.org/linux/man-pages/man2/sendmsg.2.html

Linux traffic control decides which packet leaves next: qdiscs can queue, classify, shape, pace, police and deliberately drop

The transmit path already passes through a qdisc before the driver. That hook is a programmable scheduling layer, not just a FIFO. Linux traffic control can smooth bursts, cap a class to a configured rate, give latency-sensitive flows fair service, mark/drop packets before queues become enormous, or classify packets into different scheduling classes.

NETWORK STACK PRODUCES skb ↓ root qdisc on egress interface ↓ optional CLASSIFICATION packet fields / skb priority / tc filter / BPF action ↓ optional CLASSFUL hierarchy class A: interactive class B: bulk class C: capped backup ↓ leaf qdisc queues packet ↓ SCHEDULER / SHAPER decides when packet is eligible ↓ DEQUEUE ↓ driver ndo_start_xmit() ↓ NIC TX queue / hardware scheduler EXAMPLES OF DIFFERENT JOBS fq_codel hash packets into per-flow queues fair scheduling between flows CoDel-style active queue management controls persistent queue delay HTB token/rate accounting in a hierarchy classes can receive guaranteed rates / configured ceilings each class can have its own child/leaf qdisc INGRESS packets already arrived, so ordinary shaping cannot delay transmission that already happened tc can classify/police/drop/redirect; IFB is often used when delayed ingress shaping is required.

Mechanism	What it controls
qdisc	Queueing discipline attached to a traffic-control point; decides enqueue/dequeue behavior.
class	Node in a classful qdisc hierarchy that can own bandwidth policy and another leaf qdisc.
filter/classifier	Chooses which class/action should handle a packet using packet metadata, fields or programmable classifiers.
shaping	Delays egress packets so a stream conforms to a configured rate/burst model.
policing	Checks traffic against a rate/profile and commonly drops or re-marks excess traffic instead of waiting for it.
AQM	Active Queue Management deliberately marks/drops before a queue grows without bound; CoDel is one example.
FQ	Fair queueing separates flows so one bulk flow is less able to monopolize queue service.
hardware TX queues	Exist below the software qdisc layer; mq/mqprio/offload configurations can map software policy toward NIC queues.

A qdisc does not make a slow link faster. It decides how finite link capacity and queueing delay are distributed. Good queue management can dramatically improve latency under load while preserving throughput; it cannot exceed the physical service rate of the bottleneck.

OBSERVATION LAB # current qdisc state and statistics tc -s qdisc show dev eth0 # current classes / filters tc -s class show dev eth0 tc filter show dev eth0 # example: replace root qdisc with fq_codel # tc qdisc replace dev eth0 root fq_codel # example HTB designs require a root qdisc, classes and usually leaf qdiscs/filters. # Do not change a remote machine's network qdisc unless you have a recovery path. # inspect queue-related interface statistics too ip -s link show dev eth0

PUBLIC MANUALtc(8) — Linux traffic control model

Current iproute2 manual explaining qdiscs, classes, filters, shaping, scheduling, policing and drop behavior.

https://man7.org/linux/man-pages/man8/tc.8.html

PUBLIC MANUALtc-fq_codel(8) — fair queueing plus CoDel AQM

Concrete classless qdisc combining per-flow fair queueing with controlled-delay active queue management.

https://man7.org/linux/man-pages/man8/tc-fq_codel.8.html

PUBLIC MANUALtc-htb(8) — Hierarchy Token Bucket

Classful shaping model for hierarchical rate guarantees, ceilings and link sharing.

https://www.man7.org/linux/man-pages/man8/tc-htb.8.html

One received network packet: cable → NIC DMA → NAPI → TCP/IP → socket → process

The Ethernet section explains MAC/PHY hardware. The missing upper half is what happens after the NIC has received a valid frame. A modern Linux NIC usually places packet data into host memory through DMA, records completion metadata in a receive ring/queue, and signals work. Linux then processes packets in batches through NAPI and moves protocol state upward toward a socket.

wire / PHY ↓ decoded Ethernet symbols NIC MAC validates frame / FCS, classifies queue ↓ NIC DMA engine writes packet bytes into pre-posted host RAM buffer ↓ NIC updates RX descriptor / completion ring ↓ MSI-X interrupt (or polling) driver schedules NAPI and usually masks/reduces interrupts ↓ NAPI poll drains many received packets up to a budget ↓ driver constructs/attaches Linux packet metadata (struct sk_buff or XDP path) ↓ Ethernet layer → IP routing/demux → TCP/UDP ↓ flow/socket lookup ↓ socket receive queue / TCP receive state ↓ wake waiting task / epoll readiness recv()/read() ↓ copy or otherwise expose payload to user buffer application

Object/mechanism	Role
RX descriptor/ring	Shared queue telling NIC where receive buffers live and telling driver which buffers now contain packets.
DMA	Moves received packet bytes from NIC to RAM without CPU copying every byte from a device register.
RSS	NIC hashes flows and spreads packets among hardware receive queues/CPUs.
MSI-X	PCIe message-signaled interrupt commonly associated with an RX/TX queue/vector.
NAPI	Linux mechanism that shifts high-rate packet handling from one interrupt per event toward bounded/batched polling.
sk_buff	Linux's principal packet metadata/data-buffer representation in the conventional networking stack.
protocol demultiplexing	EtherType/IP protocol/ports and connection state decide which higher layer/socket receives the packet.
socket receive queue	Kernel queue from which recv()/read()-style operations eventually consume data.

This path has many fast-path variations: XDP can run before normal sk_buff construction, GRO can merge packets, checksum/segmentation work can be offloaded, busy polling can replace the initial interrupt, and io_uring/zero-copy mechanisms can alter the userspace boundary. The diagram is the conventional conceptual path, not a promise about every Linux packet.

KERNEL DOCSLinux ENA Ethernet driver architecture

Concrete PCIe NIC driver example with RX/TX datapath, MMIO management interface, receive-side scaling, queue structures and interrupt handling.

https://docs.kernel.org/networking/device_drivers/ethernet/amazon/ena.html

Modern packet throughput depends on steering flows across NIC queues, interrupt vectors and CPUs without destroying cache locality

A fast NIC rarely has one receive queue and one interrupt. Multi-queue hardware exposes many RX/TX descriptor rings, often with separate MSI-X vectors. Linux then has several complementary steering layers. RSS chooses a hardware receive queue before the host CPU sees the packet; RPS can redirect later protocol processing in software; RFS tries to keep a flow near the CPU running its consuming application; and XPS chooses transmit queues/CPUs for outgoing traffic.

RECEIVE Ethernet frame arrives at NIC ↓ NIC computes RSS flow hash (often IPs + L4 ports) ↓ indirection table RX queue 0 / 1 / 2 / ... ↓ each queue commonly tied to MSI-X vector / IRQ affinity CPU selected for interrupt/NAPI poll ↓ XDP (if attached) → driver/NAPI receive path ↓ RPS enabled? ── yes → software may enqueue packet to another CPU backlog ↓ RFS enabled? ── may bias flow toward CPU where consuming task runs ↓ IP/TCP/UDP → socket receive queue → application TRANSMIT application CPU → socket/TCP/IP/qdisc ↓ XPS / queue selection ↓ TX ring chosen for locality/contention balance ↓ NIC DMA fetches descriptors/data → wire

Mechanism	Where decision happens	Main objective
RSS	NIC hardware before DMA/interrupt handling completes.	Hash flows across hardware RX queues so multiple CPUs can receive in parallel while packets in one flow normally stay ordered.
IRQ affinity / MSI-X	Interrupt-controller/kernel mapping of queue vectors to CPUs.	Determines which CPUs initially service queue interrupts/NAPI work.
RPS	Kernel receive path after the hardware queue has already been chosen.	Software-distribute protocol processing, including on NICs with limited hardware queue steering.
RFS	RPS-related software flow steering.	Improve data-cache locality by processing a flow near the CPU running the consuming application.
XPS	Transmit path before selecting hardware TX queue.	Reduce queue contention and preserve CPU/cache locality for outgoing work.

More CPUs is not automatically faster. Moving a packet to another CPU can cost an IPI, cache-line movement and NUMA traffic. A good queue/IRQ/RSS layout balances parallelism against locality; if hardware RSS already maps one active queue cleanly per target CPU, extra software RPS may be redundant.

KERNEL DOCSLinux receive-side scaling

Explains RSS/RPS/RFS and how receive queues are distributed across CPUs so packet processing can scale on multicore systems.

https://docs.kernel.org/networking/scaling.html

CURRENT PUBLIC MANUALethtool(8) — channels, RSS hash and indirection-table control

Current userspace control/reference for NIC queue/channel counts, receive-flow classification and RSS hash/indirection-table inspection and configuration.

https://man7.org/linux/man-pages/man8/ethtool.8.html

XDP can decide a packet's fate before the normal socket stack: driver RX buffer → eBPF action → stack, redirect or AF_XDP

The ordinary Linux receive path eventually builds an sk_buff and traverses protocol layers before a socket can read the data. XDP provides an earlier programmable hook, commonly in the network driver's receive path while the frame is still represented by a lightweight receive buffer. A verified eBPF program can drop, pass, transmit or redirect that frame before the full networking stack does its usual work. AF_XDP extends this model by redirecting selected frames into userspace-visible rings backed by a registered UMEM region.

NIC receives Ethernet frame ↓ DMA into driver/page-pool RX buffer RX queue / NAPI processing ↓ XDP PROGRAM sees packet bytes + metadata ├── XDP_DROP → discard here ├── XDP_PASS → continue into normal Linux stack │ → build/use skb → IP → TCP/UDP → socket ├── XDP_TX → send frame back out ingress netdev └── XDP_REDIRECT → devmap / cpumap / XSKMAP / other target ↓ AF_XDP XSKMAP target? ↓ yes userspace AF_XDP socket RX descriptor ring ↓ descriptors point into UMEM frames ↓ application processes packet TX ring can queue frames for transmit AF_XDP memory bookkeeping also uses FILL and COMPLETION rings so kernel and userspace can exchange ownership of UMEM buffers.

Piece	What it does
XDP program	`BPF_PROG_TYPE_XDP` program attached at a network-device ingress hook; executes under verifier-enforced constraints on packet buffer access.
`XDP_PASS`	Hands the frame onward to the ordinary Linux receive stack. XDP can therefore inspect/filter without bypassing TCP/IP.
`XDP_DROP`	Discards the frame at the early hook, avoiding much of the later per-packet stack work.
`XDP_TX`	Queues the frame for transmission back through the ingress device, useful for simple responders or forwarding patterns.
`XDP_REDIRECT`	Redirects through a BPF map/helper to another netdev, CPU, AF_XDP socket or supported target.
XSKMAP	BPF map associating RX queue keys with AF_XDP sockets so an XDP program can redirect matching frames to userspace.
UMEM	Registered userspace memory divided into packet frames; AF_XDP ring descriptors refer to offsets within this area rather than copying each packet into an arbitrary application buffer.
RX/TX rings	Producer/consumer descriptor rings for frames received by or queued from the application.
FILL/COMPLETION rings	Return empty UMEM frames to the kernel for RX and return ownership of completed TX frames to userspace.
copy vs zero-copy mode	Depending on driver/device support and setup, AF_XDP may copy packet data or let NIC/kernel/userspace operate on the same registered packet buffers.

Bypassing work also bypasses services. A frame redirected to AF_XDP does not automatically receive TCP reassembly, routing, firewalling, socket buffering or application protocol semantics from the normal stack. High-performance userspace networking gains control by taking responsibility for whatever layers it bypasses.

KERNEL DOCSLinux AF_XDP documentation

Current description of XSK sockets, RX/TX/FILL/COMPLETION rings, UMEM registration, XSKMAP redirection and copy versus zero-copy operation.

https://docs.kernel.org/networking/af_xdp.html

KERNEL DOCSBPF DEVMAP — XDP redirect to network devices

Shows how bpf_redirect_map() and DEVMAP/DEVMAP_HASH steer XDP frames to other network devices, including optional secondary XDP programs.

https://docs.kernel.org/bpf/map_devmap.html

KERNEL DOCSLinux netdev specification — XDP and AF_XDP capabilities

Current netdev control-plane schema exposes supported XDP actions, AF_XDP zero-copy capability, RX metadata features, queue/NAPI relationships and XSK information.

https://docs.kernel.org/next/netlink/specs/netdev.html

AF_PACKET exposes Ethernet frames to userspace before ordinary IP/TCP/UDP socket demultiplexing

Most applications open an IP transport socket and let the kernel build or parse Ethernet headers. Linux AF_PACKET sockets instead attach at the device/Layer-2 boundary. They can receive frames addressed to a network interface and can inject frames for transmission. SOCK_RAW exposes the link-layer header; SOCK_DGRAM uses a cooked link-layer interface. This is the family underneath tools such as packet capture programs and some userspace protocol experiments.

NIC RX descriptor completes ↓ driver/NAPI constructs normal receive representation ↓ AF_PACKET tap(s) may receive a copy/view of the L2 packet ├── ordinary recvmsg() packet socket └── PACKET_RX_RING / TPACKET mmap ring ↓ kernel owns free slots kernel fills packet + metadata ↓ marks slot for userspace userspace reads mapped slot without one recv syscall per frame ↓ returns slot ownership to kernel normal kernel networking continues separately: Ethernet → IP → TCP/UDP → ordinary socket

Interface	Key distinction
`AF_PACKET/SOCK_RAW`	User supplies/receives the link-layer header, typically an Ethernet header.
`AF_PACKET/SOCK_DGRAM`	"Cooked" packet interface where the kernel supplies/removes parts of the physical-layer header representation.
`PACKET_RX_RING`	Memory-mapped circular receive ring that amortizes syscall overhead and lets ownership of slots move between kernel and process.
`PACKET_TX_RING`	Mapped transmit ring for batching frames submitted by userspace.
AF_XDP	Different high-performance path integrated with XDP/UMEM and often earlier in RX processing; it is not simply a faster spelling of AF_PACKET.

Privilege boundary: creating packet sockets requires the appropriate raw-network capability in the user namespace governing the network namespace. Raw L2 access can observe or generate traffic that ordinary transport sockets cannot.

PUBLIC MANUALpacket(7) — Linux AF_PACKET sockets

Current interface reference for sockaddr_ll, raw versus cooked packet sockets, protocol/interface binding, multicast membership, packet rings and capability requirements.

https://man7.org/linux/man-pages/man7/packet.7.html

KERNEL DOCSLinux packet mmap / AF_PACKET

Useful alternate path showing packet-ring memory mapped directly into userspace for high-rate packet capture, avoiding ordinary per-packet recv() overhead.

https://docs.kernel.org/networking/packet_mmap.html

RDMA moves registered memory through NIC queues: MR → work request → QP → completion queue

Conventional sockets deliberately hide much of the NIC and DMA machinery behind the kernel networking stack. Remote Direct Memory Access exposes a lower-overhead model. An application registers memory, creates queue-based transport objects, posts work requests, and then the RNIC/HCA performs DMA and protocol processing while completion records tell software which operations finished.

SETUP / SLOW PATH userspace process ↓ libibverbs / librdmacm /dev/infiniband/uverbsN ↓ kernel RDMA core + device driver create Protection Domain (PD) register Memory Region (MR) ├─ pin/map pages for device access ├─ establish local key (lkey) └─ optional remote key (rkey) authorizes remote access create Completion Queue (CQ) create Queue Pair (QP) ├─ Send Queue (SQ) └─ Receive Queue (RQ) connect / transition QP state as transport requires FAST PATH application builds Work Request (WR) SGEs = address + length + lkey ↓ post WR to SQ/RQ ↓ userspace/provider updates queue / doorbell RNIC fetches queue entry ↓ RNIC DMA reads/writes registered RAM ↓ wire protocol (InfiniBand / RoCE / iWARP, depending stack) ↓ remote RNIC validates keys / QP context ↓ SEND/RECV: remote posted receive buffer is filled RDMA WRITE: remote registered memory is written directly RDMA READ: remote registered memory is read directly ↓ completion generated on CQ ↓ ibv_poll_cq() / completion event ↓ application can safely reuse/release operation resources

RDMA object	What it means
HCA / RNIC	RDMA-capable network adapter that executes queue entries, transport processing and DMA.
Protection Domain (PD)	Groups RDMA objects into a protection boundary; memory registrations and QPs must be associated compatibly.
Memory Region (MR)	Registered userspace memory plus access permissions and keys that authorize local/remote DMA operations.
lkey / rkey	Keys placed in work requests/protocol metadata to prove access to a registered region under the requested operation.
Queue Pair (QP)	Send Queue + Receive Queue transport endpoint used by verbs.
Work Request (WR)	Software description of a SEND, RECV, RDMA READ/WRITE, atomic or related operation.
Work Queue Element (WQE)	Device/provider queue representation generated from a work request.
Completion Queue (CQ)	Ring/queue of completed work records; applications poll or receive events rather than taking a syscall per packet.
scatter/gather element	Address/length/key tuple allowing one operation to reference one or more registered memory ranges.
RDMA CM	Connection-management API that gives socket-like setup semantics around QP-based RDMA transports.

“Zero copy” is not “no copies anywhere.” RDMA primarily avoids staging payload through kernel socket buffers and avoids CPU memcpy on the host data path. The NIC still performs DMA between RAM and the adapter/link, and registration/pinning/IOMMU mappings have real setup and memory-management costs.

RDMA MACHINE INSPECTION # Devices and character interfaces ls -l /dev/infiniband 2>/dev/null ls /sys/class/infiniband 2>/dev/null # rdma-core tooling when installed rdma link 2>/dev/null ibv_devices 2>/dev/null ibv_devinfo 2>/dev/null | less # Observe which PCIe device/driver owns an RDMA adapter lspci -nnk | grep -A4 -Ei 'Ethernet|InfiniBand|Network' # Conceptual microbenchmark: # compare a normal TCP copy path with an RDMA SEND/RECV or READ/WRITE path; # account separately for connection/registration setup and steady-state transfer.

KERNEL DOCSLinux userspace verbs access

Explains the uverbs architecture: resource-management operations cross /dev/infiniband/uverbsN while provider fast paths can mmap hardware queues/registers for syscall-free posting and polling.

https://docs.kernel.org/infiniband/user_verbs.html

RDMA-CORE MANUALrdma_cm(7) — RDMA connection manager

High-level setup model for reliable, connected and datagram transfers; shows address resolution, QP creation, connection establishment and verbs-based data transfer.

https://man7.org/linux/man-pages/man7/rdma_cm.7.html

RDMA-CORE MANUALibv_create_qp(3) — create a Queue Pair

Defines the core QP abstraction and requested send/receive queue capacities and scatter/gather limits.

https://man7.org/linux/man-pages/man3/ibv_create_qp.3.html

RDMA-CORE MANUALibv_post_send(3) — post work requests

Concrete fast-path API for placing SEND/RDMA/atomic work onto a QP send queue.

https://man7.org/linux/man-pages/man3/ibv_post_send.3.html

RDMA-CORE MANUALibv_poll_cq(3) — consume completions

Shows how userspace polls completion records after the adapter has executed posted work requests.

https://man7.org/linux/man-pages/man3/ibv_poll_cq.3.html

Linux packets are metadata plus fragments: struct sk_buff, headroom, clones, GSO and checksum state

struct sk_buff is Linux networking's central packet object, but the structure itself is mostly metadata. Packet bytes live in an associated linear head buffer and optional page fragments. This layout lets layers prepend/remove headers cheaply, clone packet metadata without copying payload, and describe packets larger than the NIC's eventual wire MTU for segmentation offload.

SKB METADATA OBJECT struct sk_buff dev / sk / protocol len / data_len network_header / transport_header offsets ip_summed / csum gso_size / gso_type / gso_segs queue/list/refcount fields │ └────→ skb->head allocation head buffer geometry: ┌──────────┬───────────────────────────────┬──────────┬────────────────────┐ │ headroom │ linear packet data │ tailroom │ skb_shared_info │ └──────────┴───────────────────────────────┴──────────┴────────────────────┘ ^skb->data ^skb->tail skb_shared_info may reference PAGE FRAGS: frag[0] → page + offset + length frag[1] → page + offset + length ... frag_list → optional chained skbs WHY HEADROOM EXISTS TCP starts with payload ↓ reserve room before skb->data TCP prepends TCP header IP prepends IP header Ethernet prepends L2 header ↓ avoid reallocating/copying full packet at every layer CLONING skb_clone(original) ↓ new sk_buff metadata object both metadata objects point to same packet data shared-info refcount increases ↓ copy-on-write/unshare only if a layer must modify shared bytes GSO TRANSMIT TCP hands networking one large ~64 KiB skb gso_size says e.g. 1448-byte TCP payload segments ↓ qdisc / driver ↓ NIC supports TSO? ├── yes → DMA large buffer + segmentation metadata; NIC emits many MTU-sized frames └── no → software GSO segments into smaller skbs before driver GRO RECEIVE multiple adjacent packets from one flow ↓ network stack coalesces them into larger skb representation ↓ upper TCP/IP processing handles fewer packet objects CHECKSUM OFFLOAD skb->ip_summed/checksum metadata says whether checksum is complete, partially prepared for NIC completion, or still requires software.

skb concept	Meaning
headroom	Unused bytes before skb->data reserved so lower layers can prepend headers without reallocating the packet.
tailroom	Unused linear-buffer capacity after current packet data.
linear data	Packet bytes stored contiguously in the skb head buffer.
page frag	Nonlinear packet payload held in separately referenced memory pages/fragments.
skb_shared_info	Tail metadata after the head buffer containing fragment arrays and shared-data state.
skb_clone()	Duplicates skb metadata while sharing the actual data buffers through refcounts.
GSO	Generic Segmentation Offload: stack can carry a large packet representation and segment later.
TSO	TCP Segmentation Offload: NIC segments a large TCP packet into wire-sized packets.
GRO	Generic Receive Offload: software merges compatible received packets before upper-stack processing.
checksum offload	NIC/stack split checksum computation/validation using skb checksum metadata.
truesize	Approximate memory accounting charged for an skb and its data, not simply the wire packet length.
skb_orphan()	Drops socket ownership/accounting association when packet lifecycle no longer needs it.

One skb does not necessarily equal one Ethernet frame. Before GSO/TSO segmentation it may represent many future wire packets; after GRO it may represent several received packets that were coalesced. This is why packet captures taken at different points can show surprising lengths/checksum states.

NETWORK OFFLOAD OBSERVATION # NIC offload capabilities ethtool -k eth0 | grep -Ei 'segmentation|scatter|checksum|gro|gso|tso' # interface counters ip -s link show dev eth0 ethtool -S eth0 2>/dev/null | less # packet capture caveat: # tcpdump on the host may observe packets before TX segmentation or after RX coalescing. # Source trail: # include/linux/skbuff.h # net/core/skbuff.c # net/core/dev.c # Disabling offloads alters performance/packet shape; do so only in a controlled lab.

KERNEL DOCSLinux checksum offloads

Current checksum metadata contract between networking stack and drivers, including CHECKSUM_PARTIAL and related skb state.

https://docs.kernel.org/networking/checksum-offloads.html

KERNEL DOCSLinux Page Pool API

Current RX-memory allocator optimized for recycling pages/page fragments used by skbs and XDP frames, bridging NIC DMA buffers to skb fragments.

https://docs.kernel.org/networking/page_pool.html

SOURCE / FILELinux skbuff source

Real implementation for skb allocation, cloning, copying, freeing, fragment manipulation and linearization.

https://github.com/torvalds/linux/blob/master/net/core/skbuff.c

How an x86 machine routes interrupts and wakes other CPUs: IO-APIC, Local APIC and IPIs

In a multiprocessor PC, an interrupt is not merely a wire going directly into 'the CPU.' External interrupt controllers and per-logical-CPU Local APIC state cooperate to route vectors to selected CPUs. The Local APIC also lets CPUs send inter-processor interrupts (IPIs) to one another.

LEGACY / LINE-BASED DEVICE INTERRUPT device IRQ ↓ I/O APIC ↓ route / vector / destination optional interrupt-remapping unit ↓ target CPU's Local APIC ↓ CPU accepts interrupt vector ↓ IDT handler / kernel interrupt code PCIe MSI/MSI-X device issues special interrupt message/write ↓ interrupt-remapping / APIC delivery path ↓ target Local APIC / CPU vector INTER-PROCESSOR INTERRUPT (IPI) CPU A writes/sends APIC interrupt command ↓ CPU B Local APIC receives vector ↓ CPU B enters kernel handler Uses: rescheduling, TLB shootdowns, function calls, stopping CPUs, debug/panic coordination and startup of secondary processors.

Term	Role
Local APIC	Per logical processor/core-local interrupt controller state handling local timers, vectors and IPIs.
I/O APIC	Routes external line-based interrupt inputs toward one or more processors/vectors.
vector	Numeric interrupt/trap identifier used by the processor to select an IDT entry/handler.
IPI	Interrupt intentionally sent from one processor to another.
MSI/MSI-X	Device-originated message-signaled interrupt used heavily by PCIe devices.
interrupt remapping	IOMMU/platform unit validates/translates interrupt messages and constrains which vector/CPU a device can target.
BSP	Bootstrap Processor: the logical processor that firmware/OS initially uses to bootstrap multiprocessor startup.
AP	Application Processor: additional processor brought online by bootstrap/startup mechanisms.
INIT/SIPI	Classic x86 startup sequence using INIT and Startup IPIs to begin secondary-processor execution at startup code.

KERNEL DOCSLinux — x86 IO-APIC

Public kernel documentation explaining that the IO-APIC routes hardware interrupts to multiple CPUs/CPU groups on SMP systems.

https://cdn.kernel.org/doc/html/latest/arch/x86/i386/IO-APIC.html

KERNEL DOCSLinux irq_domain hierarchy — x86 interrupt path

Current kernel docs explicitly show a typical path: Device → IOAPIC → Interrupt Remapping Controller → Local APIC → CPU.

https://docs.kernel.org/7.1/core-api/irq/irq-domain.html

KERNEL DOCSLinux x86 timekeeping — Local APIC timer and IPIs

Explains that each CPU has an APIC providing local timer interrupts and IPIs, with memory-mapped/xAPIC-era details.

https://docs.kernel.org/virt/kvm/x86/timekeeping.html

One MSI-X interrupt end to end: device vector table → PCIe write → interrupt remapper → Local APIC → IDT

MSI-X replaces a shared interrupt pin with a message. Software programs a per-vector MSI-X table entry containing a message address, message data and a mask bit. When the device wants to signal that vector, it emits a PCIe memory-write-style interrupt message. On x86 systems with interrupt remapping, the IOMMU/VT-d interrupt-remapping hardware can translate and validate that message before it reaches a Local APIC vector.

DRIVER SETUP pci_alloc_irq_vectors(dev, ..., PCI_IRQ_MSIX | PCI_IRQ_AFFINITY) ↓ Linux allocates Linux IRQ(s) + architecture vector(s) ↓ program MSI-X TABLE entry N in device BAR space Message Address[63:0] Message Data[31:0] Vector Control.Mask ↓ request_irq()/request_threaded_irq() installs handler DEVICE COMPLETION e.g. NVMe queue 7 writes CQE(s) into host RAM by DMA ↓ device selects MSI-X vector N associated with queue 7 IF VECTOR MASKED set corresponding Pending Bit Array (PBA) bit do not issue interrupt message yet IF UNMASKED device emits MSI-X interrupt message using programmed Address + Data ↓ PCIe fabric/root complex ↓ x86 INTERRUPT REMAPPING enabled? ├── yes → message encodes an Interrupt Remapping Table index/handle │ VT-d checks source identity + IRTE │ IRTE supplies allowed destination/vector/delivery policy └── no → platform/APIC-format message routes directly according to programmed fields ↓ target Local APIC accepts/prioritizes vector V ↓ CPU interrupt-recognition boundary ↓ IDT[V] → low-level x86 interrupt entry ↓ Linux generic IRQ layer maps vector/irq_desc → device handler ↓ NVMe/NIC/etc. handler drains completion queue(s) ↓ EOI / return path Why many vectors? queue0 → vector0 → CPU0 queue1 → vector1 → CPU1 queue2 → vector2 → CPU2 ... reduces one shared interrupt/lock bottleneck and improves locality.

MSI-X object	Role
MSI-X capability	PCI configuration-space capability locating table/PBA and enabling/masking MSI-X.
MSI-X table	BAR-mapped array of per-vector Message Address, Message Data and Vector Control fields.
PBA	Pending Bit Array: one pending bit per MSI-X vector for events that occur while masked.
vector mask bit	Suppresses transmission for one MSI-X vector while allowing others to remain enabled.
message address/data	Values device uses to generate the interrupt message; platform code programs them to target interrupt infrastructure.
Linux IRQ number	Kernel software identifier used by driver APIs; not necessarily the raw hardware vector number.
x86 APIC vector	8-bit interrupt vector ultimately selecting an IDT entry at the target logical processor.
IRTE	Interrupt Remapping Table Entry validating/translating a remappable device interrupt message.
source-ID checking	Interrupt-remapping protection tying an interrupt request to the expected PCI requester identity.
IRQ affinity	Policy choosing CPU(s) for an interrupt; MSI-X permits different vectors to target different CPUs.
interrupt moderation/coalescing	Device delays/groups completions before signaling to reduce IRQ rate at the cost of latency.

The interrupt message is not ordinary device payload DMA. Linux notes that MSI/MSI-X are writes to a special interrupt address, and on x86 the platform routes them into APIC/interrupt-remapping logic instead of storing normal data into RAM.

LINUX MSI-X LAB # find MSI/MSI-X capability and whether enabled lspci -vv -s <BDF> | grep -A12 -Ei 'MSI-X|MSI:' # allocated MSI IRQs for one PCI function ls /sys/bus/pci/devices/0000:BB:DD.F/msi_irqs 2>/dev/null # interrupt counts / queue names / target CPUs cat /proc/interrupts | grep -Ei '<driver|device|nvme|eth>' # IRQ affinity cat /proc/irq/<IRQ>/smp_affinity_list 2>/dev/null # IOMMU/interrupt-remapping boot state dmesg | grep -Ei 'DMAR|IOMMU|interrupt remapping|x2apic' | less # Do not rewrite MSI-X table entries manually on a live device; # a bad address/vector can misroute interrupts or wedge the device.

OFFICIAL DOCSAMD PCIe — MSI-X Vector Table and PBA

Concrete public table layout: 64-bit message address, 32-bit data, per-vector mask control and one Pending Bit Array bit per vector.

https://docs.amd.com/r/en-US/pg194-axi-bridge-pcie-gen3/MSI-X-Vector-Table-and-PBA-0x8

DIRECT SPEC PDFIntel VT-d — interrupt remapping architecture (direct PDF)

Primary x86 interrupt-remapping reference showing how MSI/MSI-X messages can encode an Interrupt Remapping Table index and be translated/validated before APIC delivery.

https://cdrdv2-public.intel.com/671081/vt-directed-io-spec.pdf

KERNEL DOCSLinux x86 IOMMU documentation

Useful system-level bridge from ACPI DMAR/IOMMU discovery to DMA and interrupt-remapping infrastructure.

https://docs.kernel.org/arch/x86/iommu.html

CURRENT VENDOR DOCIntel Core Ultra Series 3 — Interrupt Remapping Table register (2026)

Current public 2026 register documentation for the VT-d Interrupt Remapping Table base and x2APIC extended-interrupt mode.

https://edc.intel.com/content/www/us/en/design/publications/core-ultra-series-3-processors-cfg-and-mem-registers/001/interrupt-remapping-table-address-register-irta-reg-0-0-0-vtdbar-offset-b8/

Interrupts, traps and system calls: how execution gets diverted

The CPU normally follows control flow chosen by the current instruction stream. An interrupt, exception, or system call creates an architecturally defined diversion: save enough state, change privilege/context as required, choose a handler address, execute the handler, then eventually restore state and resume or terminate the interrupted work.

device event (timer / UART / disk / network / etc.) ↓ device sets status and raises IRQ / sends interrupt message ↓ interrupt controller prioritizes/routes event ↓ CPU recognizes interrupt at an architecturally legal point ↓ hardware saves/records return state + cause ↓ PC redirected to trap/interrupt vector ↓ kernel handler reads device/cause state ↓ handler acknowledges/clears source, moves data or schedules work ↓ return-from-interrupt restores previous execution

FREE WEB BOOKxv6 book — traps, system calls, interrupts and device drivers

A free public book tied to a tiny real kernel. Chapters 4–6 are unusually good for seeing trap entry, system calls, timer/device interrupts, UART and drivers as executable source rather than abstract boxes.

https://mit-pdos.github.io/xv6-riscv-book/

SOURCE / FILESMIT xv6 RISC-V source

Read trap.c, trampoline.S, kernelvec.S, uart.c, plic.c and virtio_disk.c alongside the book. The code is small enough to trace.

https://github.com/mit-pdos/xv6-riscv

SPECRISC-V privileged architecture specification

The actual contract for privilege levels, traps, interrupt state, control/status registers and address translation. This is specification-level material, not tutorial simplification.

https://docs.riscv.org/reference/isa/priv/priv-index.html

A system call is a controlled privilege transition; a context switch is a different operation

These concepts are often conflated. A system call transfers execution from an application into privileged kernel code while usually continuing on behalf of the same thread. A context switch changes which schedulable task/thread is running on a CPU. A syscall can return to the same task without any task switch, or it can block and cause the scheduler to choose another task.

USER PROCESS application calls libc write()/read()/mmap()/etc. ↓ wrapper places syscall number + arguments in ABI-defined registers ↓ x86-64 SYSCALL / RISC-V ECALL / architecture-specific trap instruction ↓ hardware changes privilege/control-flow according to architecture ↓ kernel entry assembly saves/normalizes machine state ↓ kernel dispatches syscall number → implementation ↓ fast case completes immediately ↓ return-to-user path restores user-visible state ↓ user program resumes OR, syscall blocks (disk/network/wait/lock/sleep/etc.) ↓ scheduler chooses another runnable task ↓ CONTEXT SWITCH save enough old task CPU state switch kernel stack / scheduling identity switch address-space context if needed restore new task CPU state ↓ new task continues where it previously stopped

Event	Must enter kernel?	Must switch task?	Typical trigger
system call	Yes	No	Application explicitly requests kernel service.
hardware interrupt	Yes/privileged handler	No	Device/timer/external event.
page fault	Yes	No	Address translation/permission issue requiring OS handling.
scheduler preemption	Already/enters scheduler	Yes if another task selected	Timeslice, wakeup or priority decision.
blocking syscall	Yes	Often	Current task cannot make progress until an event completes.
signal delivery	Kernel mediation	No	Kernel arranges user-space signal frame/handler for a process/thread.

Context switching is more than copying general registers. Depending on architecture and task state, the kernel may switch page-table/ASID/PCID context, lazy/eager extension state, debug registers, protection state and accounting. Hardware TLB tagging reduces the need to flush every translation on every switch.

PUBLIC MANUALLinux syscall(2) — architecture-specific syscall instructions/registers

Current public man page. It tabulates each architecture's kernel-entry instruction, syscall-number register, return registers and argument-register conventions; x86-64 uses SYSCALL.

https://man7.org/linux/man-pages/man2/syscall.2.html

KERNEL DOCSLinux x86-64 kernel entry documentation

Public kernel documentation for the actual assembly entry paths: 64-bit syscall entry, compat entries, interrupt vectors, APIC interrupts and architecture exceptions.

https://www.kernel.org/doc/html/latest/arch/x86/entry_64.html

OFFICIAL SPECRISC-V privileged architecture — privilege levels and traps

Current ratified spec explains U/S/M privilege, ECALL-style supervisor calls, interrupts/exceptions and returning from trap handlers.

https://docs.riscv.org/reference/isa/priv/priv-intro.html

OFFICIAL SPECRISC-V supervisor trap state — stvec, sstatus, SRET

Normative description of trap-vector base, interrupt enable/previous-enable state, SPP, stval and SRET return semantics.

https://docs.riscv.org/reference/isa/priv/supervisor.html

KERNEL DOCSLinux scheduler architecture notes — context switch

Kernel documentation discussing the architecture-specific switch_to stage and scheduler/context-switch mechanics.

https://docs.kernel.org/next/scheduler/sched-arch.html

OPEN-SOURCE TOOLstrace — official Linux syscall tracer

Free/open, no account. Lets you watch syscalls, signals and process-state transitions made by ordinary programs without recompiling them.

https://strace.io/

Linux privilege is a credential object plus capabilities, not just 'root or not root'

Every task carries security credentials used by permission checks: real/effective/saved user and group IDs, supplementary groups, filesystem IDs, capability sets, securebits, user-namespace membership and LSM-related security state. Traditional Unix treated effective UID 0 as broadly privileged; modern Linux splits many of those powers into individually testable capabilities.

PROCESS / THREAD CREDENTIALS real UID/GID → who originally invoked process effective UID/GID → principal used by many permission checks saved UID/GID → lets privileged transitions be dropped/reacquired under rules filesystem UID/GID → used by selected VFS permission paths supplementary groups user namespace capability sets ↓ kernel permission check EXAMPLE: open('/etc/shadow', O_RDONLY) VFS pathname/inode lookup ↓ DAC mode/ACL check against fs/effective credentials ↓ ordinary permission grants access? ├── yes → continue └── no ↓ does caller have relevant capability in governing user namespace? e.g. CAP_DAC_READ_SEARCH / CAP_DAC_OVERRIDE under exact rule ↓ still subject to LSM policy such as SELinux/AppArmor where configured CAPABILITY SETS PER THREAD Permitted = capabilities thread may make Effective/Inheritable under rules Effective = capabilities actually consulted by normal privileged checks Inheritable = candidates that can survive/participate across execve Bounding = ceiling restricting capabilities obtainable across execve Ambient = selected capabilities preserved across non-privileged execve EXECVE TRANSITION old thread credentials + file owner/mode setuid/setgid bits + file security.capability xattr + no_new_privs + bounding/inheritable/ambient sets + user namespace mappings ↓ kernel computes new permitted/effective/ambient IDs/capabilities FILE CAPABILITY EXAMPLE binary has security.capability = cap_net_bind_service=ep ↓ execve under allowed conditions process can bind privileged network ports without receiving every traditional root privilege THREAD DETAIL Linux capabilities are per-thread attributes; NPTL/glibc wrappers coordinate process-looking credential changes across sibling threads where POSIX semantics require that view.

Credential/capability term	Meaning
real UID	Identity associated with process origin/login and selected signal/accounting semantics.
effective UID	UID consulted by many normal privilege/permission decisions.
saved set-user-ID	Stored credential supporting controlled privilege drop/regain patterns.
supplementary groups	Additional group IDs considered by discretionary file permission checks.
effective capability set	Capabilities currently active for kernel privilege checks.
permitted capability set	Upper set from which effective/inheritable capabilities can be derived under the rules.
inheritable capability set	Capabilities designated for possible preservation across execve in combination with file settings.
bounding set	Per-thread ceiling limiting capabilities that can be acquired through execve.
ambient set	Capabilities designed to survive ordinary non-privileged execve without setuid/file-capability elevation.
file capability	security.capability extended attribute granting selected capability sets during execve.
securebits	Flags altering legacy root/setuid capability semantics and optionally locking those choices.
CAP_SYS_ADMIN	Very broad capability covering many unrelated administrative operations; kernel docs/man pages explicitly note it is overloaded.
user namespace	Scope in which UIDs/GIDs and capability authority are interpreted.
LSM	Linux Security Module framework providing additional mandatory/security-hook decisions beyond ordinary DAC/capability checks.

UID 0 inside a user namespace is not automatically global root. A process can be UID 0 and hold capabilities inside its namespace while mapping to an ordinary unprivileged UID outside it. Capability checks are evaluated relative to the user namespace governing the object/operation.

LINUX CREDENTIAL LAB id grep -E '^(Uid|Gid|Groups|CapInh|CapPrm|CapEff|CapBnd|CapAmb|NoNewPrivs):' /proc/$$/status # decode current shell's capability sets when capsh is installed capsh --print 2>/dev/null # file capabilities getcap -r /usr/bin /usr/sbin 2>/dev/null | head -80 # inspect one process getpcaps $$ 2>/dev/null # Safe experiment: use setpriv/capsh inside a disposable shell to DROP a capability. # Avoid granting new file capabilities to system binaries merely for experimentation.

CURRENT PUBLIC MANUALcapabilities(7) — current Linux man-pages

Current comprehensive reference: Linux splits traditional superuser power into independently enabled per-thread capabilities and defines effective/permitted/inheritable/bounding/ambient sets plus file capabilities.

https://man7.org/linux/man-pages/man7/capabilities.7.html

CURRENT PUBLIC MANUALcredentials(7) — current Linux man-pages

Current process-identity reference covering PID/session/group identifiers and UID/GID credential concepts used by Linux permission checks.

https://man7.org/linux/man-pages/man7/credentials.7.html

KERNEL DOCSLinux kernel credentials

Kernel-internal view of task credentials, immutable credential objects, subjective/objective contexts and how credentials are replaced safely.

https://docs.kernel.org/security/credentials.html

CURRENT PUBLIC MANUALgetcap(8)

Current no-login tool for inspecting security.capability extended attributes on executable files.

https://www.man7.org/linux/man-pages/man8/getcap.8.html

A login program does not need to own every authentication method: PAM turns auth/account/session policy into a configurable module stack

Linux-PAM is a library/API boundary between a privilege-granting application—such as a login program, display manager, sudo or an SSH server configured to use PAM—and the site's authentication policy. The application names a PAM service, starts one PAM transaction and calls phases such as authentication, account management, credential establishment and session open/close. Configuration selects a stack of modules for each phase, and control flags define how their success/failure results are combined.

APPLICATION login / sudo / display manager / PAM-enabled sshd ↓ pam_start(service_name, user, conversation, ...) ↓ load service policy from pam.d configuration ↓ AUTH STACK account STACK session STACK pam_authenticate pam_acct_mgmt pam_open_session ↓ modules ↓ modules ↓ modules password / token expiry / policy session setup, accounting, smartcard / MFA time/access rules environment, limits, keyrings, etc. ↓ module return codes combined according to control syntax (required / requisite / sufficient / optional / bracket rules) ↓ application decides whether to grant requested privilege/session ↓ pam_setcred as appropriate ↓ ... user session ... ↓ pam_close_session → pam_end PAM chooses/authenticates policy; the application still owns the privileged action.

PAM piece	Purpose
service name	Selects the policy stack for this calling application, normally from `/etc/pam.d/<service>` or vendor configuration paths.
auth	Establishes whether the supplied identity can authenticate using the configured modules/tokens.
account	Checks whether an already-authenticated account is currently allowed to use the service: expiry, access restrictions and similar policy.
password	Changes authentication tokens through the configured password-management modules.
session	Runs setup/teardown work around an accepted login/session; it is not the same operation as authenticating the password.
conversation function	Application-supplied callback through which PAM modules can request input or display prompts without hard-coding a terminal UI.
control flag	Defines how one module's result affects continuation and the final result of the stack.

PAM is not the password database. A PAM module may consult local hashes, a directory service, hardware token, biometric system or something else. PAM standardizes the transaction and module stacking; it does not require one credential type or one backing identity store.

CURRENT MANUALpam(3) — Linux-PAM application interface

Current Linux-PAM overview covering transaction start/end plus authentication, account, password, credential and session-management phases.

https://man7.org/linux/man-pages/man3/pam.3.html

CURRENT MANUALpam.d(5) — PAM service configuration

Explains per-service PAM configuration and the module/control-stack mechanism used to define authentication policy without recompiling the calling application.

https://man7.org/linux/man-pages/man5/pam.d.5.html

CURRENT MANUALpam_open_session(3)

Current reference for the distinct PAM session phase that runs after successful authentication/account checks and is paired with session close.

https://www.man7.org/linux/man-pages/man3/pam_open_session.3.html

Opening a pathname is not enough: the VFS still evaluates ownership, mode bits, ACLs, capabilities and security policy

Pathname resolution answers which inode?; authorization answers may this caller perform this operation on it? Linux discretionary access control starts from the caller's filesystem credentials and the inode's owner/group/mode state. POSIX ACLs can refine the classic owner/group/other model, while extended attributes provide a general name:value storage mechanism used by ACLs and many other subsystems. Capability and LSM checks then add separate privilege/security layers.

EXAMPLE: openat(dirfd, "reports/q3.txt", O_RDONLY) ↓ resolve pathname components ↓ for directory traversal: search/execute permission on each directory ↓ identify final inode ↓ DISCRETIONARY ACCESS CHECK caller fsuid/fsgid + supplementary groups versus inode owner / group / mode bits + optional access ACL CONCEPTUAL ACL DECISION caller is file owner? ── yes → owner::rwx entry │ no matching named user entry? ── yes → entry permissions ∩ ACL mask │ no matching owning/named group entries? ── yes → union ∩ ACL mask │ no ↓ other:: permissions then, where relevant: capability-based DAC override rules ↓ LSM hooks / security policy (SELinux, AppArmor, Landlock, ...) ↓ filesystem + mount + operation-specific constraints ↓ allow or -EACCES/-EPERM DIRECTORY CREATION WITH DEFAULT ACL parent directory default ACL + requested mode / umask interaction ↓ new child's access ACL + resulting mode-class permissions EXTENDED ATTRIBUTES (xattrs) inode ├─ user.* application metadata under filesystem rules ├─ security.* security labels/capability metadata ├─ trusted.* privileged metadata └─ system.* filesystem/system-defined metadata; ACLs may live here

Mechanism	What it controls or stores
mode bits	Classic owner/group/other read, write and execute/search permissions plus special mode bits.
access ACL	Fine-grained discretionary permissions for the file/directory itself, including named users/groups and an ACL mask.
default ACL	Directory template used when creating child objects; it influences the child's initial access ACL and mode-class permissions.
ACL mask	Upper bound on effective permissions of named-user, owning-group and named-group ACL entries; it is not the same thing as the process umask.
xattr	Persistent name:value metadata attached to an inode. ACLs, security labels, file capabilities and arbitrary user metadata may use xattrs.
fsuid/fsgid	Linux credentials used by selected VFS access checks; normally track the effective IDs unless explicitly changed.
capability	Can authorize narrowly defined privileged operations or bypass selected DAC checks under precise rules; it is not a blanket allow.
LSM	Additional security-hook decision layer; passing DAC does not imply that an LSM policy must allow the operation.
access()/faccessat()	Permission-probing interfaces with credential semantics that differ from simply attempting the eventual open; they should not be treated as a race-free authorization substitute.

Permission checks belong to the operation, not to a pathname string. A prior access() result can become stale before open(), and symlink/mount/name races can change what a pathname resolves to. For security-sensitive opens, combine the actual open operation with dirfd/openat2-style resolution constraints rather than doing a separate “check then use” authorization sequence.

PERMISSION / ACL / XATTR LAB # Classic ownership and mode bits stat -c 'mode=%A (%a) owner=%U:%G inode=%i' FILE namei -l /path/to/FILE # POSIX ACLs (when filesystem/tools support them) getfacl FILE setfacl -m u:someuser:r-- FILE # Extended attributes getfattr -d -m- FILE 2>/dev/null setfattr -n user.example -v hello FILE getfattr -n user.example FILE # Compare the process identity/capability side id grep -E '^(Uid|Gid|Groups|CapEff|CapPrm):' /proc/$$/status

PUBLIC MANUALacl(5) — Linux POSIX ACL semantics

Describes access/default ACLs, the named-user/group and mask entries, the discretionary access-check algorithm, object-creation inheritance and the relationship between ACL entries and file permission bits.

https://man7.org/linux/man-pages/man5/acl.5.html

PUBLIC MANUALxattr(7) — Linux extended attributes

Current manual for inode-associated name:value metadata and the user.*, trusted.*, security.* and system.* namespaces, including the relationship between xattrs and security features such as ACLs.

https://man7.org/linux/man-pages/man7/xattr.7.html

PUBLIC MANUALaccess(2) / faccessat(2) — permission-check semantics

Useful for understanding why an explicit permission probe is not equivalent to performing the eventual operation and why access() historically asks a different credential question than a normal open.

https://man7.org/linux/man-pages/man2/access.2.html

Linux can retain credentials and cryptographic material as kernel objects: keys live in searchable keyrings with their own permissions and lifetimes

Linux credentials such as UID, GID and capabilities answer who is this task? The kernel key-retention service answers a different question: what security material or authentication token can this task or kernel subsystem find and use? A kernel key has a type, description, serial number, payload, owner/permissions, state and optional expiry. A keyring is itself a special key whose payload is a set of links to other keys/keyrings, creating a searchable graph anchored to thread, process, session, user or persistent lifetimes.

TASK CREDENTIALS UID/GID/capabilities/security context ↓ also references keyrings thread keyring ─┐ process keyring ├─ search order / possession session keyring ┘ ↓ links [keyring] ├── user key: arbitrary readable payload ├── logon key: kernel-usable payload not readable back by userspace └── nested keyring → more keys request_key(type, description, ...) ↓ search caller-accessible keyrings ├── found → return key serial/handle └── absent ↓ optional request-key userspace upcall ↓ instantiate or negatively cache result ↓ kernel service / application consumes key under permission checks

Object/lifetime	Important behavior
key	Typed kernel object with a description, payload, serial number, permissions, state and optional expiration.
keyring	A key whose payload is a collection of links to keys/keyrings; linking keeps referenced keys alive and makes them discoverable through searches.
thread keyring	Private to one thread and tied closely to that thread's credential lifetime.
process keyring	Shared by threads in a process; distinct from the longer-lived session keyring.
session keyring	Designed to follow a login/session-style process tree and persist across `execve()`.
`user` key	General-purpose userspace-managed payload that can be read when permissions allow.
`logon` key	Secret payload created from userspace but intentionally not readable back through userspace key APIs; useful for kernel consumers.
`request_key()`	Searches relevant keyrings and can trigger a controlled userspace helper to instantiate missing material.

A key serial number is not authority by itself. Kernel key access is permission-checked and depends on possession/search rules plus key type semantics. Keyrings are also not a replacement for a userspace secret manager: they are a kernel retention/cache mechanism used by filesystems, authentication flows and other subsystems, with quotas and garbage collection.

KERNEL DOCSLinux Kernel Key Retention Service

Authoritative overview of key objects, key types, keyrings, process subscriptions, permissions, quotas, request-key upcalls and garbage collection.

https://docs.kernel.org/security/keys/core.html

PUBLIC MANUALkeyrings(7) — in-kernel key management and retention

Userspace-facing model of keys, possession, thread/process/session/user keyrings, permissions and key lifetimes.

https://man7.org/linux/man-pages/man7/keyrings.7.html

PUBLIC MANUALrequest_key(2) — search and instantiate a key

Current syscall documentation for searching keyrings, negative caching and the optional userspace instantiation callback path.

https://www.man7.org/linux/man-pages/man2/request_key.2.html

A container is still host processes: namespaces change what global resources those processes can see

Linux namespaces virtualize selected global kernel resources for groups of processes. They do not emulate a CPU or boot an independent kernel. A containerized process still executes ordinary host instructions and ordinary host system calls; the kernel interprets many of those calls through the process's namespace memberships.

HOST PROCESS PID(host view)=48321 UID(host view)=1000 network stack = host netns mount tree = host mount namespace unshare()/clone3() with namespace flags CLONE_NEWPID CLONE_NEWNS CLONE_NEWNET CLONE_NEWUTS CLONE_NEWIPC CLONE_NEWUSER ... ↓ same Linux kernel, new namespace objects INSIDE CONTAINER-LIKE VIEW PID namespace: process appears as PID 1 to descendants inside namespace same task may still be PID 48321 from host/ancestor namespace mount namespace: process gets its own mount-tree view bind mounts / overlay root / proc mount can differ from host network namespace: separate interfaces, routing tables, firewall/netfilter context, sockets/port-number space often connected to host by veth pair + bridge/routing UTS namespace: separate hostname/domain name IPC namespace: separate SysV IPC / POSIX message-queue namespace user namespace: inside UID 0 → mapped outside UID 100000 (example) capabilities are powerful inside namespace scope, not equivalent to capabilities in initial user namespace cgroup namespace: virtualizes process view of cgroup path/root, but DOES NOT itself impose resource limits time namespace: selected monotonic/boottime clock offsets differ setns(fd) joins an existing namespace subject to capability/rule checks NAMESPACE FILES /proc/<pid>/ns/mnt /proc/<pid>/ns/pid /proc/<pid>/ns/net /proc/<pid>/ns/user ... These are first-class namespace handles that can be opened and passed to setns().

Namespace	clone/unshare flag	Resource virtualized
Mount	CLONE_NEWNS	Mount points / filesystem mount-tree view.
PID	CLONE_NEWPID	Process-ID number space and namespace-local PID 1 semantics.
Network	CLONE_NEWNET	Network devices, routes, sockets, ports and associated networking state.
UTS	CLONE_NEWUTS	Hostname and NIS domain name.
IPC	CLONE_NEWIPC	System V IPC and POSIX message-queue namespace.
User	CLONE_NEWUSER	UID/GID mappings and namespace-scoped capabilities.
Cgroup	CLONE_NEWCGROUP	View of cgroup hierarchy/path, not controller resource limits themselves.
Time	CLONE_NEWTIME	Offsets for selected clocks such as monotonic/boottime.

Namespaces isolate names/views, not resource consumption. A process in a private PID or mount namespace can still allocate all host memory or consume all CPU unless separate mechanisms—typically cgroup controllers and scheduler/memory policy—constrain it.

NAMESPACE LAB — READ-ONLY FIRST lsns ls -l /proc/$$/ns readlink /proc/$$/ns/{mnt,pid,net,user,uts,ipc,cgroup,time} 2>/dev/null # compare namespace IDs of two processes for p in $$ 1; do echo PID=$p ls -l /proc/$p/ns 2>/dev/null done # unprivileged user-namespace experiment if distro policy permits: unshare --user --map-root-user --mount-proc sh -c ' echo inside_uid=$(id -u); cat /proc/self/uid_map; ls -l /proc/self/ns/user ' # Namespace support/policy varies by distro. Creating a user namespace can expose # more kernel interfaces, so production administrators often combine it with cgroup/security policy.

CURRENT PUBLIC MANUALnamespaces(7) — current Linux man-pages

Current 6.19 overview: namespaces wrap global resources so processes see isolated instances; the page enumerates Mount, PID, Network, IPC, User, UTS, Cgroup and Time namespaces.

https://man7.org/linux/man-pages/man7/namespaces.7.html

KERNEL DOCSLinux namespace administration guide

Kernel documentation index for namespace behavior and user-namespace resource-control considerations.

https://docs.kernel.org/admin-guide/namespaces/index.html

KERNEL DOCSLinux shared-subtree/mount propagation

Deep mount-namespace material explaining private/shared/slave mount propagation—critical for understanding container mount trees beyond a simple chroot model.

https://docs.kernel.org/filesystems/sharedsubtree.html

User namespaces remap identity and privilege: UID/GID maps make “root inside” different from root outside

A Linux user namespace creates a new view of user/group identifiers and a new capability scope. A task can be UID 0 inside a child user namespace while mapping to an ordinary unprivileged UID in the parent. This is the mechanism that makes many rootless-container operations possible, but it does not turn the task into unrestricted host root: capability checks are evaluated relative to the user namespace that owns the target resource, and IDs must map across namespace boundaries.

HOST / PARENT USER NAMESPACE real host user: UID 1000 ↓ clone(CLONE_NEWUSER) / unshare --user NEW USER NAMESPACE initially: UID/GID mappings must be established ↓ /proc/<pid>/uid_map inside-start outside-start count 0 100000 65536 example translation: inside UID 0 ↔ parent UID 100000 inside UID 1 ↔ parent UID 100001 ... inside UID 65535 ↔ parent UID 165535 Inside the namespace: getuid() may report 0 CAP_SYS_ADMIN may be effective IN THIS USER NAMESPACE ↓ operation on resource owned by child namespace? ├── capability in owning user namespace may authorize it └── host/global resource owned by initial user namespace? child-namespace capability is not equivalent to host CAP_SYS_ADMIN FILESYSTEM OWNERSHIP inode UID stored/mapped through namespace-aware ID mapping rules ↓ stat() presents an ID meaningful in caller's namespace when mapped unmapped IDs cannot simply become arbitrary host identities ROOTLESS CONTAINER PATTERN ordinary host user → user namespace + subordinate UID/GID ranges → mount/network/etc namespaces owned by that user namespace → scoped capabilities inside container-like environment

Mechanism	Why it matters
`uid_map` / `gid_map`	Defines ranges translating IDs between a user namespace and its parent; mappings are constrained and established under specific permission rules.
`/etc/subuid` / `/etc/subgid`	Delegates ranges of subordinate IDs that an account may map through helpers such as `newuidmap`/`newgidmap`.
namespace-scoped capabilities	UID 0 in a user namespace starts with capabilities there, but those capabilities only authorize operations whose governing resource/check is owned by that namespace or descendants as defined by kernel rules.
namespace ownership	Non-user namespaces are owned by a user namespace; many capability checks ask for a capability in that owning user namespace.
`setgroups` restriction	Unprivileged GID-map setup has extra safeguards; common rootless setup denies `setgroups` before writing a GID mapping.
unmapped ID	An identity with no mapping cannot be treated as an arbitrary valid parent/host UID or GID; mapping boundaries are part of the isolation model.

# Current process's user-namespace handle and maps readlink /proc/self/ns/user cat /proc/self/uid_map cat /proc/self/gid_map # Rootless experiment when distro policy allows it unshare --user --map-root-user sh -c ' echo "inside:"; id echo uid_map; cat /proc/self/uid_map echo gid_map; cat /proc/self/gid_map echo userns; readlink /proc/self/ns/user ' # Delegated subordinate ID ranges, when configured cat /etc/subuid 2>/dev/null cat /etc/subgid 2>/dev/null

“UID 0” is namespace-relative. A process showing uid=0 inside a rootless container can still map to an unprivileged host identity. Conversely, user namespaces enlarge the set of kernel code reachable by unprivileged users, which is why some deployments restrict or disable unprivileged user-namespace creation.

CURRENT PUBLIC MANUALuser_namespaces(7) — UID/GID maps and scoped capabilities

Current Linux man-pages guide to ID mappings, capability semantics, namespace ownership and the rules governing uid_map/gid_map.

https://man7.org/linux/man-pages/man7/user_namespaces.7.html

CURRENT PUBLIC MANUALnewuidmap(1) — delegated UID-map helper

Current shadow-utils manual for safely writing a child namespace's UID mapping using ranges delegated through subordinate-ID configuration.

https://man7.org/linux/man-pages/man1/newuidmap.1.html

CURRENT PUBLIC MANUALsubuid(5) — subordinate user-ID delegation

Current documentation for /etc/subuid / subordinate-ID delegation used by rootless namespace tooling.

https://man7.org/linux/man-pages/man5/subuid.5.html

A mount namespace is a private view of mount attachments; bind mounts and propagation decide how that view is assembled

A pathname does not identify a filesystem by itself. The VFS walks dentries through a process's mount namespace, a tree of mount attachments that says which filesystem/subtree appears at each mount point. Creating a new mount namespace copies the current mount list as a starting view; later mount/unmount changes can diverge. Containers then use bind mounts, OverlayFS, pivot_root()/chroot()-style root setup and propagation rules to construct a different filesystem view without copying the underlying files.

HOST MOUNT NAMESPACE / ├── /usr → root filesystem subtree ├── /proc → procfs mount ├── /sys → sysfs mount └── /srv/data → filesystem X unshare(CLONE_NEWNS) / clone(CLONE_NEWNS) ↓ NEW NAMESPACE starts with a copy of mount attachments ↓ make propagation private/slave as policy requires ↓ BIND MOUNT mount --bind /srv/data /container/data This creates another attachment/view of the same underlying subtree. It does NOT copy file contents. ↓ mount proc -t proc /container/proc mount overlay ... /container/root ↓ process enters/uses constructed root view ↓ /path lookup crosses mount points according to THIS namespace PROPAGATION EXAMPLE shared parent mount mount something below peer A ↓ corresponding mount event can propagate to peer B private mount neither sends nor receives propagation slave mount receives from master but does not propagate changes back upstream

Mount concept	Meaning
mount namespace	The set/tree of filesystem mounts visible to a process. Processes in the same mount namespace see the same attachment changes.
bind mount	Attaches an existing file or directory subtree at another location without creating another copy of its data.
recursive bind (`MS_BIND\|MS_REC`)	Replicates a subtree plus eligible submounts rather than only the top attachment.
shared	Mount belongs to a peer group; mount/unmount events under one peer can propagate to the others.
slave	Receives propagation from its master peer group but does not propagate events back to the master.
private	Neither sends nor receives mount propagation.
unbindable	Private and additionally cannot be bind-mounted as a source.
`/proc/<pid>/mountinfo`	Kernel view of a process's mount namespace including mount IDs, parents, roots, mount points and propagation metadata.

Namespace isolation and mount propagation are separate knobs. A process can have its own mount namespace while selected shared/slave mount relationships still deliberately carry later mount events across namespace boundaries. Container runtimes commonly make most of a container's tree private or slave precisely to prevent accidental mount leakage while still allowing selected host-originated mounts when desired.

READ-ONLY / THROWAWAY LAB (root or suitable user namespace required) unshare --user --map-root-user --mount --propagation private sh mount --bind /tmp /mnt 2>/dev/null findmnt -o TARGET,SOURCE,FSTYPE,PROPAGATION cat /proc/self/mountinfo | less # Compare namespace handles from another shell: readlink /proc/<PID>/ns/mnt # A bind mount is another attachment, not a copy: # editing an ordinary file through either path reaches the same inode/data.

PUBLIC MANUALmount_namespaces(7) — Linux mount namespaces and shared subtrees

The detailed userspace-facing explanation of namespace creation, copied mount lists, peer groups and shared/slave/private/unbindable propagation semantics.

https://man7.org/linux/man-pages/man7/mount_namespaces.7.html

PUBLIC MANUALmount(2) — bind, recursive bind, move and propagation flags

System-call documentation for attaching filesystems/subtrees, including MS_BIND, MS_REC, and the mount propagation flags.

https://man7.org/linux/man-pages/man2/mount.2.html

CURRENT MANUALmount(8) — bind mounts and shared-subtree operations

Current util-linux manual for the user-facing mount command, including bind/rbind, move operations and --make-shared, --make-slave, --make-private and recursive propagation controls.

https://man7.org/linux/man-pages/man8/mount.8.html

Idmapped mounts change the ownership view without rewriting every inode: one filesystem tree can appear under different UID/GID mappings

Unix ownership is normally stored as numeric user/group identities in filesystem metadata. Containers and portable filesystems complicate that model because the same numeric UID can mean different users in different user namespaces. An idmapped mount attaches an ID mapping to a particular mount, so VFS ownership translation happens while crossing that mount rather than by recursively rewriting on-disk ownership with chown().

ON DISK file owner represented in the filesystem's ID mapping ↓ VFS lookup through an idmapped mount mount ID mapping translates ownership view ↓ caller sees a UID/GID meaningful in its user namespace CREATE / CHOWN-LIKE OPERATIONS THROUGH THAT MOUNT caller UID/GID ↓ caller user-namespace mapping ↓ mount idmapping ↓ filesystem idmapping ownership stored in the filesystem's representation Other mounts of the same filesystem can expose a different ownership view. No recursive metadata rewrite is required.

Mechanism	What it changes	What it does not change
`chown()`	Persistently changes inode ownership metadata.	It does not create a mount-local alternative ownership view.
user namespace	Defines how a process's userspace UIDs/GIDs map into kernel IDs and scopes capabilities.	By itself it does not make an arbitrary host filesystem tree conveniently owned by the container's IDs.
idmapped mount	Applies a user-namespace-derived ID mapping at one mount, translating VFS ownership and related ACL/capability IDs.	It does not rewrite ownership for other mounts or permanently renumber the filesystem.
`mount_setattr(..., MOUNT_ATTR_IDMAP, ...)`	Attaches the selected user namespace's ID mapping to a suitable detached mount.	It does not make unsupported filesystems magically implement idmapped-mount semantics.

Container root and file ownership are separate questions. A process may be UID 0 inside a user namespace yet still encounter files whose kernel IDs do not map usefully into that namespace. Idmapped mounts solve the filesystem-view translation problem without granting host-root identity or rewriting the underlying tree.

KERNEL DOCSLinux filesystem idmappings and idmapped mounts

Detailed kernel explanation of caller, filesystem and mount idmappings, ownership crossmapping, VFS UID/GID types and the container/portable-home use cases that idmapped mounts solve.

https://docs.kernel.org/filesystems/idmappings.html

PUBLIC MANUALmount_setattr(2) — MOUNT_ATTR_IDMAP

Current Linux userspace API for changing mount attributes, including how MOUNT_ATTR_IDMAP takes its mapping from a user-namespace file descriptor and the constraints on creating an idmapped mount.

https://man7.org/linux/man-pages/man2/mount_setattr.2.html

Namespaces hide resources; cgroup v2 accounts, prioritizes and limits them

cgroup v2 organizes processes into a single hierarchical resource tree. Controllers attach accounting and policy to that hierarchy: CPU shares/bandwidth, memory protection/pressure/limits, I/O weights or maximum rates, PID counts, cpuset placement and more. This is the resource-control half of most container implementations.

UNIFIED CGROUP v2 TREE /sys/fs/cgroup/ ├── system.slice/ └── workloads/ ├── build-A/ │ ├── cgroup.procs │ ├── cpu.max │ ├── cpu.weight │ ├── memory.current │ ├── memory.high │ ├── memory.max │ ├── pids.current │ └── pids.max └── build-B/ MOVE A PROCESS echo PID > workloads/build-A/cgroup.procs ↓ process becomes member of that cgroup domain children created later normally inherit membership CPU BANDWIDTH cpu.max = '20000 100000' ↓ group may use up to ~20 ms CPU time each 100 ms period ↓ once quota spent scheduler throttles eligible cgroup tasks until period replenishes CPU WEIGHT cpu.weight = 100 vs sibling weight 900 ↓ under contention, scheduler distributes fair-class CPU time proportionally rather than imposing a hard absolute ceiling MEMORY memory.current = charged usage memory.high = throttling/reclaim pressure boundary memory.max = hard containment limit ↓ charge pushes group beyond memory.high task enters direct reclaim / is throttled ↓ cannot reclaim below memory.max hard pressure cgroup-scoped OOM handling may kill within the group PIDS pids.max = 512 ↓ fork/clone that would exceed controller limit fails with EAGAIN I/O io.weight adjusts relative priority io.max can cap BPS/IOPS for selected block device PSI / PRESSURE cpu.pressure / memory.pressure / io.pressure ↓ measures time tasks are stalled waiting for scarce resources A cgroup namespace may hide/relocate the visible path, but actual controller accounting/limits come from cgroup membership.

cgroup v2 file/controller	Meaning
cgroup.procs	Lists/moves process IDs in the cgroup.
cgroup.subtree_control	Enables selected controllers for child cgroups subject to hierarchy rules.
cpu.weight	Relative fair-class CPU share among siblings under contention.
cpu.max	Maximum fair-class CPU bandwidth in quota/period form.
memory.current	Current charged memory usage.
memory.high	Memory throttle/reclaim-pressure boundary; designed to degrade/throttle rather than directly OOM-kill.
memory.max	Hard memory-usage limit; unresolved pressure can invoke cgroup-local OOM handling.
memory.low/min	Best-effort/hard memory protection models under reclaim according to controller rules.
io.weight	Relative I/O weight for applicable schedulers/devices.
io.max	Absolute per-device BPS/IOPS limits.
pids.max	Maximum number of tasks/processes allowed by the pids controller.
cpuset.cpus	Restricts CPU placement for cgroup tasks under cpuset controller rules.
*.pressure	Pressure Stall Information measuring resource-stall time for tasks in the cgroup.
cgroup.kill	v2 control for killing all processes in a cgroup where supported.
cgroup.freeze	Freezes/thaws cgroup execution without individually signaling every process.

A resource limit is not an isolation namespace. Setting memory.max does not give the process a private view of RAM, and entering a PID namespace does not limit memory. Containers compose namespaces, cgroups, credentials/capabilities, filesystem policy and usually seccomp/LSMs.

CGROUP v2 OBSERVATION — READ-ONLY mount | grep 'type cgroup2' cat /proc/self/cgroup cat /sys/fs/cgroup/cgroup.controllers 2>/dev/null cat /sys/fs/cgroup/cgroup.subtree_control 2>/dev/null # current group stats when path is known CG=/sys/fs/cgroup grep . $CG/{cpu.stat,memory.current,memory.stat,pids.current} 2>/dev/null | head -120 # pressure cat $CG/cpu.pressure 2>/dev/null cat $CG/memory.pressure 2>/dev/null cat $CG/io.pressure 2>/dev/null # systemd systems expose units as cgroups; systemd-cgls is a convenient viewer. systemd-cgls 2>/dev/null | less # Avoid writing cpu.max/memory.max/cgroup.procs on a useful machine until you # understand which service/process tree you are modifying.

KERNEL DOCSLinux Control Group v2

Authoritative current cgroup-v2 design/interface document: hierarchical process organization plus CPU, memory, I/O, PID, cpuset and other controller semantics.

https://docs.kernel.org/admin-guide/cgroup-v2.html

KERNEL DOCSUser namespaces and resource control

Kernel guidance explicitly recommends resource control alongside user namespaces because namespace-local privilege can otherwise consume host-global resources.

https://docs.kernel.org/admin-guide/namespaces/resource-control.html

The memory cgroup is more than a byte counter: charges follow pages, reclaim is scoped, and limits change allocation behavior

The generic cgroup hierarchy says which workload a task belongs to; the memory controller decides how much memory that workload may consume and how strongly it is protected from reclaim. Linux charges major classes of memory—including anonymous pages, page cache, kernel objects and TCP buffers—to memory cgroups. Those charges remain associated with memory independently of which CPU happens to touch it, so a container can be pressured or OOM-killed even while the host still has memory available for other cgroups.

TASK IN CGROUP /workload-A ↓ page fault / cache fill / kernel allocation memory charge attempted against A ↓ memory.current increases UNDER memory.low / memory.min ↓ reclaim protection applies ABOVE memory.high ↓ allocation can still succeed ↓ task is throttled / routed through reclaim pressure ↓ memory.events: high increments APPROACH memory.max ↓ scoped reclaim tries to reduce A ↓ reclaim succeeds ───────────→ continue │ no ↓ MEMCG OOM ↓ choose victim inside constrained cgroup domain ↓ memory.events: oom / oom_kill SWAP IS ACCOUNTED SEPARATELY TOO memory.swap.current / memory.swap.max ↓ limits can constrain anonymous-memory escape into swap

Interface	Operational meaning
`memory.current`	Current charged memory for the cgroup and descendants.
`memory.low`	Best-effort protection from reclaim while usage remains under the effective boundary.
`memory.min`	Harder reclaim protection; overcommitting protected memory can push the system toward OOM.
`memory.high`	Pressure/throttling boundary. Crossing it drives reclaim but is deliberately not a direct hard-OOM limit.
`memory.max`	Hard containment boundary; if scoped reclaim cannot satisfy the charge, memcg OOM handling may kill tasks in the cgroup.
`memory.swap.max`	Limits swap consumption attributable to the cgroup; it is not the same as limiting resident memory.
`memory.reclaim`	Administrative trigger asking the kernel to reclaim a requested amount from the target cgroup.
`memory.events`	Counters for low/high/max pressure, OOM and kills; useful for observing whether a configured boundary is actively affecting the workload.

memory.high and memory.max are intentionally different controls. Treating them both as “RAM limits” hides the design: memory.high is the pressure/backpressure mechanism that makes a workload reclaim and slow down, while memory.max is the final containment wall. PSI complements these counters by measuring how much useful work is actually stalled under the resulting pressure.

KERNEL DOCSLinux cgroup v2 — memory controller

Current authoritative definitions for memory.current/min/low/high/max, swap controls, memory.reclaim, memory.events and cgroup-local OOM behavior.

https://docs.kernel.org/admin-guide/cgroup-v2.html#memory

KERNEL DOCSLinux Out Of Memory handling documentation

Kernel memory-management documentation for OOM handling, complementing the controller interface by showing what happens after reclaim can no longer satisfy allocation pressure.

https://docs.kernel.org/mm/oom.html

Utilization can look fine while work is stuck: PSI measures time lost to CPU, memory and I/O contention

Traditional utilization counters answer questions such as “how busy was the CPU?” or “how much memory is allocated?” They do not directly answer “how much wall-clock time did runnable work lose because a resource was unavailable?” Linux Pressure Stall Information (PSI) accounts that lost time for CPU, memory and I/O and exposes both recent averages and cumulative stall time.

WORKLOAD UNDER RESOURCE PRESSURE runnable / faulting / I/O-waiting tasks ↓ kernel identifies resource-stall states ├── CPU pressure: runnable work waiting for CPU time ├── memory pressure: reclaim / refault / allocation stalls └── I/O pressure: tasks blocked on I/O completion ↓ aggregate stall time ├── some = at least one task stalled └── full = all non-idle work stalled together ↓ /proc/pressure/{cpu,memory,io} ├── avg10 / avg60 / avg300 └── total microseconds ↓ optional threshold written to open PSI fd ↓ poll()/epoll() wakeup when pressure crosses threshold With cgroup v2: /cpu.pressure, memory.pressure, io.pressure measure one workload instead of the whole machine

Metric	Interpretation
`some`	At least part of the workload is stalled on the resource; useful for latency degradation before total collapse.
`full`	All non-idle work is simultaneously stalled for that resource; prolonged full memory/I/O pressure indicates severe loss of useful progress.
`avg10/60/300`	Recent stall percentages over rolling 10-, 60- and 300-second windows.
`total`	Cumulative microseconds stalled, useful for deltas and short spikes that averages can hide.
PSI trigger	Kernel threshold monitor on a PSI file descriptor; integrates resource-pressure alarms with `poll()`/`epoll()`.
cgroup PSI	Same idea scoped to one cgroup hierarchy node, allowing workload-local pressure measurement.

PSI is not another utilization percentage. A memory-heavy workload can have high RAM occupancy but little pressure if allocations proceed smoothly; conversely, a machine with seemingly moderate utilization can have damaging pressure if tasks repeatedly stall on reclaim or I/O.

KERNEL DOCSLinux PSI — Pressure Stall Information

Explains CPU/memory/I/O pressure metrics used globally and through cgroup controller pressure files to quantify time workloads spend stalled on scarce resources.

https://docs.kernel.org/accounting/psi.html

KERNEL DOCSLinux delay accounting and delaytop

Related task-delay accounting documentation, including current delaytop output that combines per-task delay information with system-wide PSI to diagnose where latency is being lost.

https://docs.kernel.org/accounting/delay-accounting.html

Seccomp reduces the syscall attack surface: no_new_privs, cBPF filters and per-syscall actions

Namespaces and cgroups change views and resources; they do not prevent a process from invoking arbitrary system calls that are otherwise permitted. seccomp adds a syscall-entry filter. In filter mode, the kernel runs a classic-BPF program over a small immutable record containing the syscall number, architecture and raw arguments, then applies the filter's action before normal syscall execution.

PROCESS WANTS TO SELF-RESTRICT prctl(PR_SET_NO_NEW_PRIVS, 1) ↓ irreversible for task + inherited by children/exec process promises execve will not grant new privilege through setuid/file caps build classic-BPF seccomp program ↓ seccomp(SECCOMP_SET_MODE_FILTER, flags, &prog) ↓ kernel validates filter program attaches filter to calling thread (or TSYNCs thread group when requested) LATER SYSCALL RAX = __NR_openat2 args in architecture syscall registers ↓ low-level syscall entry ↓ before normal syscall body seccomp builds struct seccomp_data: nr = syscall number arch = AUDIT_ARCH_X86_64 instruction_pointer args[0..5] = raw 64-bit argument values ↓ run cBPF filter(s) ↓ ACTION SECCOMP_RET_ALLOW continue normal syscall SECCOMP_RET_ERRNO | EPERM do NOT execute syscall return -1 / errno=EPERM to userspace SECCOMP_RET_KILL_PROCESS / KILL_THREAD terminate process/thread SECCOMP_RET_TRAP deliver SIGSYS SECCOMP_RET_LOG log then allow SECCOMP_RET_USER_NOTIF stop syscall and notify a supervisor fd ↓ supervisor may inspect/respond under strict API rules FILTER LIMITATION filter sees raw numeric arguments, not safely dereferenced user memory ↓ e.g. it can compare openat flags/fd values, but cannot securely parse the pathname string merely from its pointer FILTERS STACK new filters can further restrict an already-filtered thread; they cannot loosen earlier restrictions.

Seccomp concept	Meaning
no_new_privs	Sticky task attribute preventing execve from granting new privilege through setuid/setgid bits or file capabilities.
SECCOMP_MODE_STRICT	Very small legacy fixed syscall mode; much less expressive than filter mode.
SECCOMP_MODE_FILTER	Programmable syscall filtering using classic BPF over struct seccomp_data.
struct seccomp_data	Read-only filter input containing syscall number, architecture, instruction pointer and six raw arguments.
RET_ALLOW	Permit syscall to continue.
RET_ERRNO	Skip syscall and synthesize selected errno result.
RET_KILL_PROCESS	Terminate the entire process when a forbidden syscall is attempted.
RET_TRAP	Raise SIGSYS for the filtered syscall.
RET_LOG	Allow syscall but request logging under kernel audit/seccomp policy.
RET_USER_NOTIF	Send syscall request to a userspace supervisor via seccomp notification fd.
TSYNC	Filter-install flag attempting to synchronize the new filter across all threads in the thread group.
filter stacking	Additional filters may be added; effective result is constrained by all installed filters.
classic BPF	Small verified accumulator-based BPF language used by seccomp filters; distinct from modern eBPF program types.

Seccomp is not a complete sandbox by itself. Kernel documentation describes it as syscall-surface reduction. A useful sandbox usually combines seccomp with ordinary permissions/capabilities, namespaces, filesystem restrictions, cgroups and often an LSM. A syscall that remains allowed must still be treated as potentially attacker-controlled input by the kernel.

SECCOMP OBSERVATION / SAFE LAB # Current shell no_new_privs / seccomp state grep -E '^(NoNewPrivs|Seccomp|Seccomp_filters):' /proc/$$/status # Compare a containerized/service process grep -E '^(NoNewPrivs|Seccomp|Seccomp_filters):' /proc/<PID>/status # strace can reveal which syscalls an application actually uses before # designing a filter: strace -f -c ./program # systemd services expose syscall-filter policy declaratively; # inspect an existing unit before changing anything: systemctl cat <unit> 2>/dev/null | grep -E 'SystemCall|NoNewPrivileges' -n # Start filters permissively in a disposable test process. A too-tight filter can # make normal libc behavior fail in surprising places such as memory allocation, # threading, DNS, locale loading or signal setup.

KERNEL DOCSLinux seccomp BPF documentation

Current kernel guide: seccomp filters reduce exposed kernel syscall surface using BPF over syscall number/arguments; it explicitly warns seccomp is not a sandbox by itself.

https://docs.kernel.org/userspace-api/seccomp_filter.html

CURRENT PUBLIC MANUALseccomp(2) — current Linux man-pages

Current 6.19 syscall reference for filter installation, no_new_privs/CAP_SYS_ADMIN requirement, inherited filters, actions and thread synchronization.

https://man7.org/linux/man-pages/man2/seccomp.2.html

CURRENT PUBLIC MANUALPR_SET_NO_NEW_PRIVS

Current reference for the irreversible no_new_privs bit required for unprivileged seccomp filter installation.

https://man7.org/linux/man-pages/man2/PR_SET_NO_NEW_PRIVS.2const.html

KERNEL DOCSLinux no_new_privs documentation

Kernel rationale: once set, execve promises not to grant privilege that would not have been available without exec, enabling safer unprivileged restriction mechanisms.

https://docs.kernel.org/userspace-api/no_new_privs.html

Seccomp user notification can stop a syscall and ask a userspace broker what to do

A normal seccomp filter makes its decision entirely inside the kernel: allow, fail with an errno, trap, log, trace or kill. SECCOMP_RET_USER_NOTIF adds a different path. A filter installed with SECCOMP_FILTER_FLAG_NEW_LISTENER returns a listener file descriptor; when a matching syscall occurs, the calling task blocks while a supervisor process receives a notification, optionally performs work on the task's behalf, and sends a response.

sandbox / container manager installs seccomp filter with NEW_LISTENER ↓ listener fd returned to supervisor SANDBOXED TASK openat2(...) ↓ syscall entry seccomp filter returns SECCOMP_RET_USER_NOTIF ↓ kernel creates notification ID + seccomp_data sandboxed task blocks ↓ SUPERVISOR poll(listener_fd) SECCOMP_IOCTL_NOTIF_RECV ↓ inspect syscall number + raw register arguments ↓ if pointer arguments matter: copy/validate referenced memory carefully beware task can race mutable userspace memory ↓ possible responses return chosen value / errno CONTINUE original syscall (use with caution) ADDFD: atomically install an fd into target task ↓ SECCOMP_IOCTL_NOTIF_SEND ↓ blocked task resumes with supplied result / continued syscall

Piece	Role
NEW_LISTENER	Requests a notification listener fd when the seccomp filter is installed.
USER_NOTIF	Filter action that delegates this syscall instance to the listener rather than deciding locally.
notification ID	Identifies one outstanding request; supervisors should validate that the request is still live before acting on stale state.
NOTIF_RECV / SEND	Ioctls used by the broker to receive a blocked syscall request and return a result.
NOTIF_ADDFD	Lets the supervisor install a file descriptor into the target task, useful when the broker performs an open-like operation on its behalf.
CONTINUE	Allows the original syscall to proceed, but pointer arguments can create time-of-check/time-of-use hazards if a broker treated earlier memory contents as authoritative.

This mechanism is explicitly not a general security-policy engine. The Linux interface documentation warns about TOCTOU hazards and privileged-supervisor assumptions. Use seccomp filters themselves for the hard syscall policy boundary; user notification is primarily a controlled syscall-brokering/emulation mechanism.

PUBLIC MANUALseccomp_unotify(2) — userspace syscall notification

Detailed current manual for NEW_LISTENER, USER_NOTIF, receive/send ioctls, request IDs, ADDFD and the security/TOCTOU caveats.

https://man7.org/linux/man-pages/man2/seccomp_unotify.2.html

KERNEL UAPI SOURCELinux seccomp userspace ABI definitions

Authoritative UAPI header for notification structures, return actions, ioctl numbers and the warning attached to USER_NOTIF continuation semantics.

https://github.com/torvalds/linux/blob/master/include/uapi/linux/seccomp.h

eBPF is verified kernel-resident bytecode: load → prove safety → JIT → run at a hook → exchange data through maps

eBPF lets userspace load small programs that execute inside kernel-defined hook contexts without loading a general-purpose kernel module. That is only possible because the kernel verifier symbolically tracks program state before admitting the program. Accepted bytecode can then be interpreted or JIT-compiled into native CPU instructions and attached to networking, tracing, cgroup, security and other hooks.

USERSPACE BUILD restricted C / compiler frontend ↓ clang/LLVM BPF backend eBPF instructions + BTF/maps/relocations in ELF object ↓ libbpf loader CREATE MAPS bpf(BPF_MAP_CREATE, ...) ↓ kernel returns map file descriptors hash/array/per-CPU/ringbuf/etc. storage lives in kernel LOAD PROGRAM bpf(BPF_PROG_LOAD, insns, prog_type, map references, ...) ↓ VERIFIER — ABSTRACT INTERPRETER / STATE EXPLORER R1 initially typed pointer to program context R10 = stack/frame pointer other registers tracked as uninitialized/scalar/pointer/ref types for each reachable instruction/path: prove stack/register initialized before read track scalar min/max and known-bit ranges track pointer provenance + fixed/variable offsets prove memory access bounds/alignment prove packet access stays below data_end validate helper/kfunc argument types track acquired references/resources and require release prove loops/calls terminate within verifier/program limits merge/prune equivalent safe abstract states unsafe path? → reject BPF_PROG_LOAD with verifier log ALL PATHS ACCEPTED ↓ optional architecture JIT eBPF virtual registers/instructions → x86-64/arm64/riscv/... native code JIT hardening may blind constants/alter generated form ↓ program fd references loaded kernel object ATTACH TO HOOK examples: XDP / tc packet path socket filter tracepoint / kprobe / uprobe cgroup hooks LSM hook ↓ event occurs kernel invokes verified/JITed BPF program with typed context ↓ program reads allowed context fields calls allowed helpers/kfuncs updates BPF maps / ring buffer returns hook-specific action/value USERSPACE reads map/ringbuf state or updates configuration map eBPF IS NOT A GENERAL KERNEL ESCAPE HATCH: the verifier, program type, attach type and helper/kfunc allowlists constrain what each program can touch/do.

eBPF concept	Meaning
BPF_PROG_LOAD	bpf() command asking kernel to verify/load a BPF program and return a program fd.
verifier	Kernel static/abstract analysis proving program safety properties before execution.
register type	Verifier metadata such as scalar, context pointer, map-value pointer, packet pointer, stack pointer or refcounted object.
tnum	Verifier representation tracking bits known 0/1 versus unknown in a scalar value.
state pruning	Avoids re-exploring a program path when a previously accepted abstract state safely subsumes the current one.
program type	Defines hook context, permitted helper set and semantic contract, e.g. XDP, socket filter, tracing, LSM.
helper	Kernel function exposed to selected BPF program types under verifier-checked argument/return contracts.
kfunc	Kernel function exported to BPF through BTF/kfunc mechanisms and verifier-aware type rules.
BPF map	Kernel-managed key/value or specialized storage shared between BPF programs and/or userspace.
BTF	BPF Type Format metadata describing types/functions used by CO-RE, tracing and typed kernel interfaces.
JIT	Architecture backend translating accepted eBPF bytecode into native machine instructions.
JIT hardening	Optional transformations reducing abuse of predictable JITed immediate/constants at some performance cost.
bpffs pinning	Holding BPF program/map/link objects in a filesystem namespace so lifetime can outlast one process fd.
CO-RE	Compile Once – Run Everywhere relocations adapting BPF programs to compatible kernel type layouts using BTF.

The verifier is the critical privilege boundary. Current kernel verifier documentation describes path exploration, register/pointer typing, stack initialization, bounds checks and reference tracking. If any reachable path cannot be proven safe under the program-type rules, the load is rejected before JIT/execution.

eBPF OBSERVATION — NO PROGRAM LOADING REQUIRED # JIT configuration/status where exposed cat /proc/sys/net/core/bpf_jit_enable 2>/dev/null cat /proc/sys/net/core/bpf_jit_harden 2>/dev/null # bpftool inspection, if installed and permitted bpftool prog show 2>/dev/null | head -80 bpftool map show 2>/dev/null | head -80 # mounted BPF filesystem mount | grep ' type bpf ' || true find /sys/fs/bpf -maxdepth 2 -type f -o -type d 2>/dev/null | head -100 # To learn the verifier safely, compile tiny programs and intentionally make one # out-of-bounds/uninitialized access in a disposable dev environment, then inspect # the BPF_PROG_LOAD verifier log. Privilege policy varies by distro/kernel.

KERNEL DOCSLinux eBPF verifier

Current verifier internals: all-path simulation, typed pointers/registers, scalar ranges/tnums, stack initialization, packet bounds, reference tracking and state pruning.

https://docs.kernel.org/bpf/verifier.html

KERNEL DOCSLinux BPF documentation index

Current hub for verifier, instruction set, program types, helpers/kfuncs, maps, BTF, libbpf and testing/debugging.

https://docs.kernel.org/bpf/index.html

KERNEL DOCSLinux BPF maps

Current map API/storage overview: kernel/user shared data structures created and manipulated through bpf() and helpers.

https://docs.kernel.org/bpf/maps.html

KERNEL DOCSLinux bpf_jit_enable sysctl

Current documentation: after BPF_PROG_LOAD passes the verifier, enabled architecture JITs translate accepted eBPF programs into native CPU instructions; JIT hardening is separately configurable.

https://docs.kernel.org/admin-guide/sysctl/net.html

CURRENT PUBLIC MANUALbpf(2) — current Linux man-pages

Current syscall reference for BPF_MAP_CREATE/BPF_PROG_LOAD, verifier logs, program fds, maps and JIT execution model.

https://man7.org/linux/man-pages/man2/bpf.2.html

Linux permission checks are extensible: LSM hooks stack security policy onto VFS, sockets, exec, ptrace and more

Linux Security Modules (LSM) are not one policy. They are a framework of hooks placed at security-relevant kernel operations. Core kernel code performs ordinary ownership/capability checks and invokes security_* hooks; enabled LSMs such as SELinux, AppArmor, Landlock, Yama, Smack, BPF LSM and others can add additional restrictions.

PROCESS CALLS openat(AT_FDCWD, '/srv/secret', O_RDONLY) ↓ VFS pathname lookup / inode found ↓ ordinary DAC checks: owner/group/mode bits ACLs capabilities that override selected DAC checks ↓ LSM HOOKS AT RELEVANT POINTS e.g. security_inode_permission(...) security_file_open(...) ↓ LSM dispatcher calls registered hook functions SELinux example: current task security ID/type inode/file security label/type class + requested permission ↓ policy lookup / AVC ↓ allow or -EACCES AppArmor example: task's loaded profile path/object/action mediation ↓ allow or deny Landlock example: unprivileged process previously restricted itself with ruleset ↓ access allowed by normal DAC/SELinux/AppArmor BUT denied by Landlock domain ↓ final operation denied STACKING RULE security modules normally ADD restrictions; one LSM allowing an operation does not override another LSM's denial. OTHER HOOK FAMILIES exec transitions / binary loading ptrace / process inspection socket create/connect/bind/send/recv mount/superblock operations file ioctl/fcntl/mmap/mprotect IPC kernel module / firmware / key operations BPF and io_uring related security points AUDIT / LOGGING policy denial may emit audit/kernel logs depending on module and configuration.

LSM concept	Meaning
LSM hook	Security-sensitive callback site in core kernel code invoked before/around an operation.
security_* wrapper	Common kernel dispatcher function calling enabled LSM hook implementations.
LSM stacking	Multiple compatible security modules can be enabled together; restrictions combine rather than one universal policy replacing all others.
security blob	Per-object LSM state attached to credentials, inodes, sockets and other kernel objects.
SELinux	Label/type-based mandatory-access-control system implemented as an LSM.
AppArmor	Task/profile-centered mandatory-access-control system implemented as an LSM.
Landlock	Stackable unprivileged self-restriction LSM for scoped filesystem/network access.
Yama	LSM providing selected system-wide DAC hardening such as ptrace_scope.
BPF LSM	Mechanism for attaching verified BPF programs to supported LSM hooks.
AVC	SELinux Access Vector Cache reducing repeated policy-lookup cost for security decisions.
/proc/<pid>/attr/current	Userspace-visible current LSM security context for supporting modules.
audit denial	Policy-denial record emitted through audit/logging paths when module policy/configuration requests it.

LSMs do not replace Unix permissions. They are additional policy hooks. A file can pass mode-bit/capability checks yet still be denied by SELinux/AppArmor/Landlock; conversely, an LSM generally cannot turn a normal DAC denial into permission unless the core policy itself permits that path.

LSM OBSERVATION # enabled/ordered LSMs cat /sys/kernel/security/lsm 2>/dev/null # current process security context where supported cat /proc/self/attr/current 2>/dev/null # AppArmor state on supporting systems cat /sys/module/apparmor/parameters/enabled 2>/dev/null # SELinux state on supporting systems getenforce 2>/dev/null # Yama ptrace hardening cat /proc/sys/kernel/yama/ptrace_scope 2>/dev/null # Landlock support is best detected by its syscalls/API; kernel logs may also # report 'landlock: Up and running' on enabled systems. # Security policy is system-specific. Observe before changing profiles/policy.

KERNEL DOCSLinux Security Module Development

Current hook-interface reference and development guide for LSM security_* dispatch points throughout the kernel.

https://docs.kernel.org/security/lsm-development.html

KERNEL DOCSLinux Security Modules userspace API

Current description of LSM process security attributes and /proc/<pid>/attr interfaces for modules such as SELinux, Smack and AppArmor.

https://docs.kernel.org/next/userspace-api/lsm.html

KERNEL DOCSLandlock LSM

Current August 2026 documentation: Landlock lets unprivileged processes add scoped restrictions and is designed to compose with existing DAC/LSM policy rather than weaken it.

https://docs.kernel.org/security/landlock.html

Linux Audit is an accountability pipeline: kernel event → rule/filter → audit record set → Netlink → auditd → durable log

The Linux Audit subsystem records selected security-relevant activity for later inspection. It is not the same thing as SELinux, seccomp or ordinary application logging: those mechanisms can allow/deny or describe application behavior, while Audit provides a kernel-originated record of configured events such as syscalls, file accesses, identity changes and LSM decisions. Userspace loads audit rules; the kernel evaluates events and sends records to the audit daemon, which writes them to persistent logs.

CONFIGURATION auditctl / boot rules / augenrules ↓ Netlink control messages kernel Audit subsystem installs rule/filter state EVENT process calls openat()/execve()/setuid()/... ↓ syscall + task credentials + object/path/LSM context become available ↓ Audit filters/rules decide whether event is auditable ↓ one logical event may produce MULTIPLE records SYSCALL + PATH + CWD + PROCTITLE + AVC/... ↓ records share event/serial identity kernel audit backlog ↓ Audit Netlink channel auditd ↓ /var/log/audit/audit.log (typical) / configured dispatcher/plugins ↓ ausearch / aureport / SIEM or forensic tooling ENFORCEMENT IS ELSEWHERE SELinux/seccomp/DAC may deny an operation; Audit records what happened when policy/rules request it.

Audit concept	Meaning
audit rule	Kernel-side filter describing which syscalls, paths, task attributes or other events should generate audit information.
`auditctl`	Userspace tool for viewing/loading kernel audit rules and subsystem settings.
`auditd`	Userspace daemon that receives kernel audit records and writes/dispatches them according to configuration.
audit record	One typed record such as SYSCALL, PATH or AVC; one logical event can consist of several related records.
event serial	Identifier used to correlate records belonging to the same audited event.
backlog	Kernel queue buffering audit records while userspace consumes them; overflow behavior matters on high-assurance systems.
`audit=1`	Boot-time option commonly used when early processes must be marked auditable before `auditd` starts.
LSM audit record	Audit output generated for policy decisions such as SELinux AVC denials when configured.

Audit is evidence, not a permission system. A rule that records openat() does not itself prevent the open. Conversely, a denied SELinux operation can produce an Audit record because enforcement and accountability are separate layers. High-volume rules also have real cost and backlog implications, so production rule sets should be deliberate rather than “log every syscall forever.”

OBSERVATION LAB (requires appropriate privileges) # daemon status systemctl status auditd 2>/dev/null || true # kernel audit configuration / loaded rules auditctl -s 2>/dev/null auditctl -l 2>/dev/null # search recent records ausearch -ts recent 2>/dev/null | less # typical persistent log location ls -lh /var/log/audit/audit.log 2>/dev/null # Do not install broad syscall rules on a production host merely as an experiment.

PUBLIC MANUALauditd(8) — Linux Audit daemon

Current upstream userspace manual describing auditd as the component that receives/writes Audit records, with rule loading handled by auditctl/augenrules.

https://man7.org/linux/man-pages/man8/auditd.8.html

PUBLIC MANUALauditctl(8) — configure the kernel Audit system

Rule/filter syntax and subsystem controls used to configure what the kernel records.

https://man7.org/linux/man-pages/man8/auditctl.8.html

PUBLIC MANUALaudit.rules(7) — persistent rule-file format

Explains rule-file structure used by Linux Audit tooling, including syscall/file-system rule forms and startup loading conventions.

https://man7.org/linux/man-pages/man7/audit.rules.7.html

SELinux turns LSM hook points into label-and-policy decisions: subject context + object context + class + permission → allow or deny

The LSM framework supplies hook locations; SELinux supplies one concrete mandatory-access-control policy engine. Tasks run in security contexts (commonly called domains/types), filesystem objects carry labels—normally in extended attributes—and each mediated operation has an object class and requested permissions. SELinux policy computes whether that subject may perform that operation on that labeled object, then caches many decisions in an Access Vector Cache (AVC).

PROCESS DOMAIN / SUBJECT CONTEXT user_u:role_r:httpd_t:s0 ↓ open("/srv/site/private") VFS resolves inode ↓ normal Unix DAC / ACL / capability rules still apply ↓ SELinux LSM hook ↓ SUBJECT SID / context: httpd_t OBJECT SID / context: private_content_t OBJECT CLASS: file REQUESTED PERMISSION: read / open / getattr ... ↓ AVC lookup ├── cached decision exists → use it └── miss → security server evaluates loaded policy ↓ ALLOW rule permits requested access? ├── yes → SELinux permits its part of the check └── no → -EACCES / denial + AVC/audit record depending on configuration ENFORCING MODE policy denial blocks operation PERMISSIVE MODE denial is logged but SELinux does not block it (useful for policy development; DAC and other LSMs still matter) EXEC TRANSITION EXAMPLE unconfined_t executes labeled program entrypoint ↓ policy transition rule new task domain/type may become service_t ↓ future accesses evaluated under new subject context

SELinux concept	Meaning
security context	Label such as user:role:type:level. Type/domain is central to ordinary Type Enforcement policy decisions.
subject / domain	Security context of an executing task used as the actor in access checks.
object label	Security context associated with a file, socket, IPC object or other mediated kernel object; filesystem labels are commonly stored in xattrs.
object class	Kind of protected object—file, dir, process, socket and so on—whose permission names are class-specific.
Type Enforcement	Policy model describing which subject types/domains may access which object types and with which permissions.
AVC	Access Vector Cache storing computed SELinux access decisions so every hook need not recompute policy from scratch.
enforcing / permissive	Enforcing applies SELinux denials; permissive records would-be denials without SELinux itself blocking them.
relabel	Change/restore an object’s security context so it matches policy expectations; wrong labels can cause correct policy to deny access.

SELinux does not replace Unix permissions and it does not “grant around” a DAC failure. Think of it as an additional mandatory policy layer. For a protected operation to succeed, ordinary kernel permission rules and every relevant stacked security mechanism must all allow the operation.

PUBLIC MANUALselinux(8) — SELinux architecture and file labeling

Current upstream SELinux userspace manual describing mandatory-access-control policy models, runtime policy configuration and filesystem security labels stored in extended attributes.

https://man7.org/linux/man-pages/man8/selinux.8.html

PUBLIC MANUALavc_has_perm(3) — SELinux access-vector decisions

Documents SELinux AVC permission queries, security IDs, object classes and requested access vectors, including denial semantics.

https://man7.org/linux/man-pages/man3/avc_has_perm.3.html

AppArmor turns LSM hooks into task-profile confinement: executable attachment → profile rules → allow, deny or log

AppArmor is a concrete Linux Security Module built around profiles associated with tasks. A profile is loaded from userspace and can mediate file access, capabilities, networking, signals, ptrace, mount operations and other supported kernel actions. Programs with no applicable profile can run unconfined, while a confined task is subject to both ordinary Unix/DAC rules and its AppArmor policy.

POLICY AUTHOR / PACKAGE /etc/apparmor.d/... profile text ↓ apparmor_parser / policy loader compiled policy loaded into kernel AppArmor LSM PROGRAM EXECUTION execve(path) ↓ profile attachment / exec-transition rule selects confinement ↓ task carries AppArmor security context/profile LATER OPERATION open(), connect(), mount(), ptrace(), capability use, signal ... ↓ normal kernel permission logic + LSM hook ↓ AppArmor rule lookup for current profile ├── allowed → operation may continue if every other check also allows └── denied → -EACCES/-EPERM + optional audit/log record PROFILE MODES enforce: matching denials are enforced complain: violations are logged for policy development but generally not blocked EXEC TRANSITIONS policy can keep current profile, enter another profile/child profile, or deny execution depending on the rule.

Concept	Role
profile	Named set of mandatory rules applied to a task/program.
attachment	Rule associating executable paths/conditions with a profile during execution.
file rule	Grants/denies operations such as read, write, execute, link and locking for matching paths/objects.
capability rule	Controls use of Linux capabilities in addition to the normal capability model.
exec transition	Determines whether a new executable inherits confinement, enters another profile/child profile or is refused.
complain mode	Policy-development mode that records would-be violations instead of enforcing most denials.
unconfined task	Task with no enforcing AppArmor profile; still subject to DAC, capabilities and any other active LSMs.

AppArmor does not replace Unix permissions and is not the same policy model as SELinux. Both use LSM hooks and can impose mandatory restrictions, but AppArmor is task/profile-centered and commonly written around executable/path-oriented rules, while SELinux centers decisions on labeled subjects/objects and type policy. On stacked-LSM systems, a denial from another security module still wins.

READ-ONLY APPARMOR LAB cat /sys/module/apparmor/parameters/enabled 2>/dev/null cat /proc/self/attr/current 2>/dev/null aa-status 2>/dev/null # Policy commonly lives under /etc/apparmor.d on distributions using it. # Changing enforcement profiles can break services; inspect before editing.

KERNEL DOCSAppArmor

Current kernel documentation describes AppArmor as a MAC-style LSM using task profiles loaded from userspace.

https://docs.kernel.org/admin-guide/LSM/apparmor.html

PROJECT DOCSOfficial AppArmor documentation

Current upstream documentation hub for AppArmor concepts, releases, profile authoring and administration.

https://www.apparmor.net/

PROJECT DOCSAppArmor profile-language quick reference

Current upstream reference for profiles, complain mode, child profiles/hats, file/capability/network/ptrace rules, globbing and execute transitions.

https://www.apparmor.net/reference/profiles-quick-reference/

Landlock lets an unprivileged process remove some of its own ambient rights: ruleset → rules → restrict_self → inherited sandbox

The generic LSM section explains where Linux security hooks run. Landlock is a concrete use of that framework designed for application self-sandboxing. A process can construct a ruleset describing the access classes it wants to restrict, add allowed scopes such as selected filesystem hierarchies or supported network ports, then permanently enter that Landlock domain. Its children inherit the restrictions.

PROCESS STARTS WITH ORDINARY ACCESS DAC / ACL / capabilities / SELinux/AppArmor/etc. still apply ↓ query supported Landlock ABI ↓ landlock_create_ruleset(...) declare which access rights this layer handles ↓ landlock_add_rule(...) examples: allow selected filesystem actions beneath directory A allow selected network bind/connect ports where supported ↓ set no_new_privs when required by the userspace contract ↓ landlock_restrict_self(ruleset_fd) ↓ CURRENT THREAD ENTERS LANDLOCK DOMAIN ↓ future access check normal Linux permission/security checks AND every enforced Landlock layer must allow the operation ↓ child processes inherit the domain ↓ process may add another Landlock layer later, but cannot use Landlock to regain rights removed by an earlier layer. PRE-OPENED OBJECT CAVEAT Landlock primarily constrains later kernel operations in its supported scope; file descriptors opened before sandboxing can have important semantics of their own, so robust sandbox setup closes/unshares resources it does not intend to retain.

Property	Meaning
unprivileged self-restriction	Ordinary processes can sandbox themselves when the kernel has Landlock enabled; the interface is designed not to require a privileged policy daemon.
stackable LSM	Landlock adds restrictions alongside DAC and other LSMs; it does not override a denial from another policy layer.
ruleset	Kernel object declaring access-right classes that this Landlock policy layer handles.
rule	Associates allowed operations with a supported object/scope, such as a filesystem hierarchy or supported network port rule.
domain	Security state entered by restrict_self(); descendants inherit it.
monotonic restriction	Further Landlock layers can remove additional rights; a process cannot use Landlock to grant itself new authority.
ABI version	Userspace should query the running Landlock ABI/features rather than infer capability only from the kernel version.
scope limits	Landlock intentionally mediates a defined set of operations; it is not a universal replacement for namespaces, seccomp or system-wide MAC policy.

Landlock is allow-list-style self-confinement, not privilege acquisition. If ordinary Unix permissions, capabilities or another LSM deny an operation, a Landlock rule cannot make it succeed. Its useful direction is one-way: reduce what the application and its descendants can reach after initialization.

DEVELOPER CHECKLIST 1. Query the Landlock ABI/features at runtime. 2. Decide exactly which filesystem/network actions the program still needs. 3. Open only setup resources that truly must survive sandbox entry. 4. Create the ruleset and add narrowly-scoped rules. 5. Enter the Landlock domain before parsing untrusted work where practical. 6. Test on kernels with older ABI levels and define a deliberate fallback policy. # user-facing support can also be checked through the current kernel/man-page examples; # exact syscall structures evolve by Landlock ABI version.

KERNEL DOCSLandlock userspace API — unprivileged access control

Current kernel documentation covering rulesets, filesystem/network rules, ABI compatibility and the create/add/restrict workflow.

https://docs.kernel.org/userspace-api/landlock.html

PUBLIC MANUALlandlock(7) — Linux manual

Current manual describing Landlock's self-restriction model, supported rule families, inheritance, ABI-version checks and caveats.

https://man7.org/linux/man-pages/man7/landlock.7.html

What a Linux container actually is: one host kernel plus layered restrictions

CONTAINERIZED PROCESS CPU executes ordinary user instructions ↓ SYSCALL SAME HOST KERNEL as non-container processes Kernel decision stack can include: 1. PAGE TABLES / MMU process can only directly address its mapped virtual memory 2. CREDENTIALS + CAPABILITIES who is caller? what privilege bits exist in governing user namespace? 3. NAMESPACES which PID/mount/network/user/IPC/UTS/cgroup/time view applies? 4. CGROUP v2 how much CPU/memory/I/O/PIDs may this workload consume? 5. SECCOMP is this syscall number/argument pattern allowed to enter normal implementation? 6. LSM (SELinux/AppArmor/Landlock/etc. as configured) does mandatory/security policy permit this object/action? 7. VFS / SOCKET / DEVICE-SPECIFIC PERMISSION CHECKS mode bits, ACLs, ownership, net permissions, device policy, etc. 8. DRIVER / HARDWARE ISOLATION IOMMU/VFIO for assigned devices, normal DMA isolation for host drivers CONTAINER ≠ VM container: host kernel handles syscalls directly namespaces virtualize many kernel resource views cgroups/policy constrain host resources hardware VM: guest runs its own kernel guest kernel executes privileged instructions in virtualized CPU context nested/EPT/NPT/G-stage translation separates guest physical memory Both ultimately rely on MMU/IOMMU privilege enforcement underneath.

The useful mental model is composition. There is no single 'container mode' bit in the CPU or Linux kernel. Container runtimes assemble ordinary kernel primitives into a restricted process environment.

OverlayFS makes several directory trees look like one: lower layers supply defaults, the upper layer records changes

Linux OverlayFS presents a merged namespace built from one or more lower directory trees plus an optional writable upper tree. Reads can come directly from a lower object. When a lower-only file must be modified, OverlayFS generally copies it up into the upper tree and applies the change there. Deleting something that exists only below cannot erase a read-only lower layer, so the upper layer records a whiteout that hides the lower name.

LOWER (often read-only) UPPER (writable) /usr/bin/tool [initially absent] /etc/app.conf [initially absent] /var/data/ [upper additions later] │ │ └────────────── overlay lookup ───────────┘ ↓ MERGED MOUNT /usr/bin/tool visible /etc/app.conf visible READ lower-only /etc/app.conf merged lookup → no upper entry → lower object supplies data WRITE lower-only /etc/app.conf lookup finds lower object ↓ copy_up metadata/data as required into upper ↓ write upper copy ↓ future merged lookup sees upper version first UNLINK lower-only /usr/bin/tool cannot modify lower tree ↓ create WHITEOUT in upper for that name ↓ merged lookup hides lower /usr/bin/tool DIRECTORY PRESENT IN BOTH upper directory names + lower directory names ↓ merged directory view, with upper entries taking precedence and opaque/whiteout metadata controlling what remains visible

OverlayFS concept	Meaning
lowerdir	One or more source directory trees whose objects can appear in the merged view; lower layers need not be writable.
upperdir	Writable tree that stores new objects, copied-up versions and deletion/visibility metadata.
workdir	Private working directory required for writable overlays; it must satisfy OverlayFS placement requirements relative to the upper filesystem.
copy-up	Creates an upper representation of a lower object before a modification that cannot be represented solely by the lower object.
whiteout	Upper-layer marker meaning “this lower-layer name is deleted in the merged view.” The marker itself is hidden from ordinary merged lookup.
opaque directory	Upper directory metadata telling OverlayFS not to merge same-named lower directory contents into that directory's view.
merged dentry/inode view	The VFS-facing object can be backed by upper, lower, or both, so identity/stat behavior has OverlayFS-specific subtleties.

Container connection: a mount namespace and cgroups do not create an image-layer filesystem by themselves. OverlayFS is one Linux mechanism commonly used to give a container-like process a writable merged root while keeping base layers unchanged; the concepts are separate and can be used independently.

KERNEL DOCSLinux OverlayFS documentation

Current kernel documentation for upper/lower trees, merged directories, copy-up behavior, whiteouts, opaque directories, permission rules and advanced features.

https://docs.kernel.org/filesystems/overlayfs.html

One native x86-64 syscall: SYSCALL → swapgs/CR3/stack switch → pt_regs → sys_* → SYSRETQ or IRETQ

The x86-64 SYSCALL instruction is a hardware privilege-transfer mechanism, but it deliberately does not build the same stack frame as an interrupt gate. The CPU saves the user return RIP in RCX and RFLAGS in R11, loads the kernel entry RIP/segment state from model-specific registers, masks selected flags, and leaves RSP unchanged. Linux assembly must therefore establish kernel GS/CR3/stack state and construct a full pt_regs frame itself.

USERSPACE CALLING CONVENTION — x86-64 Linux RAX = syscall number RDI = arg1 RSI = arg2 RDX = arg3 R10 = arg4 R8 = arg5 R9 = arg6 example libc wrapper / raw syscall instruction ↓ SYSCALL instruction CPU HARDWARE DOES: RCX ← user RIP of instruction after SYSCALL R11 ← user RFLAGS RFLAGS ← RFLAGS & ~IA32_FMASK-mask kernel CS/SS selected from SYSCALL MSR configuration RIP ← IA32_LSTAR entry address RSP is NOT switched automatically LINUX entry_SYSCALL_64 ASSEMBLY swapgs # gain kernel per-CPU GS base save user RSP in per-CPU TSS scratch field SWITCH_TO_KERNEL_CR3 # KPTI/page-table transition when needed RSP ← per-CPU current kernel stack top push synthetic user SS/RSP/RFLAGS/CS/RIP push original RAX syscall number save/clear remaining GPRs into pt_regs layout ↓ do_syscall_64(regs, syscall_nr) enter common syscall bookkeeping/instrumentation bounds/ABI checks dispatch syscall table entry ↓ __x64_sys_read / __x64_sys_write / ... wrapper ↓ __se_sys_* argument sanitization/casting where generated ↓ __do_sys_* implementation / subsystem code ↓ return long result in RAX EXIT WORK signals / reschedule / tracing / audit / seccomp/task-work as required ↓ Can context be returned with fast SYSRET safely? ├── yes → restore regs, switch to user CR3, swapgs, SYSRETQ └── no → slower IRETQ-based user return path SYSRETQ restores: RIP from RCX RFLAGS from R11 user privilege/segment state from configured architectural rules IMPORTANT: a syscall is NOT automatically a context switch. The same thread normally enters kernel mode and returns on the same logical CPU unless it blocks, is preempted or the scheduler otherwise switches tasks.

x86 syscall item	Purpose
IA32_LSTAR	MSR containing native 64-bit SYSCALL target RIP.
IA32_STAR	MSR participating in kernel/user segment-selector setup for SYSCALL/SYSRET.
IA32_FMASK	MSR mask clearing selected RFLAGS bits during SYSCALL entry.
RCX	SYSCALL saves userspace return RIP here; therefore not a normal preserved syscall argument register.
R11	SYSCALL saves userspace RFLAGS here.
R10	Linux syscall ABI uses R10 for argument 4 because RCX is consumed by SYSCALL return state.
swapgs	Swaps user/kernel GS base state so Linux can access per-CPU kernel data.
kernel stack	Per-thread privileged stack used after assembly switches away from untrusted userspace RSP.
pt_regs	Kernel stack structure containing saved userspace register state for syscall/exception/ptrace/signal handling.
do_syscall_64()	Common x86-64 C-level dispatcher/bookkeeping path after low-level register save.
SYSCALL_DEFINE*	Kernel macros generating type/sanitization wrappers and the implementation entry for a syscall.
SYSRETQ	Fast return instruction used only when saved user context satisfies architectural/kernel safety conditions.
IRETQ fallback	More general user-return path used when SYSRET restrictions make a fast return unsafe.
KPTI	Kernel Page Table Isolation; entry/exit may switch CR3 between restricted user and kernel page-table views.

Native SYSCALL is different from an IDT interrupt. Current Linux entry_SYSCALL_64 explicitly saves userspace RSP, switches CR3/stack, and constructs pt_regs in software because SYSCALL itself does not push an interrupt frame.

SYSCALL LAB # trace high-level syscalls strace -e read,write,openat,close /bin/echo hello # inspect libc wrapper versus raw syscall instruction objdump -d -M intel /lib/x86_64-linux-gnu/libc.so.6 2>/dev/null | less # current syscall number table in installed headers grep -R '__NR_write' /usr/include/x86_64-linux-gnu/asm/unistd_64.h 2>/dev/null # kernel source trail: # arch/x86/entry/entry_64.S entry_SYSCALL_64 # arch/x86/entry/common.c do_syscall_64 # arch/x86/entry/syscalls/syscall_64.tbl # include/linux/syscalls.h SYSCALL_DEFINE macros # perf trace can show syscall entry/exit when permitted perf trace -e read,write /bin/echo hello 2>/dev/null # Compare strace with context-switch counters: a stream of simple getpid/write-like # syscalls does not imply one scheduler context switch per syscall.

SOURCE / FILELinux x86-64 entry_64.S

Real current native entry code: register convention, entry_SYSCALL_64, swapgs, kernel CR3/stack transition, pt_regs construction, do_syscall_64() and SYSRETQ/IRET selection.

https://github.com/torvalds/linux/blob/master/arch/x86/entry/entry_64.S

SOURCE / FILELinux x86 calling.h

Current pt_regs register-save layout and assembly helpers used by syscall/interrupt entry code.

https://github.com/torvalds/linux/blob/master/arch/x86/entry/calling.h

KERNEL DOCSAdding a new syscall — x86 tables

Current documentation showing x86 syscall_64.tbl wiring and architecture syscall-entry stubs.

https://github.com/torvalds/linux/blob/master/Documentation/process/adding-syscalls.rst

SOURCE / FILELinux SYSCALL_DEFINE implementation macros

Real macros generating __se_sys_* argument-sanitization wrappers and __do_sys_* implementation functions.

https://github.com/torvalds/linux/blob/master/include/linux/syscalls.h

Kernel pointers and user pointers are different trust domains: access_ok(), copy_from_user(), SMAP and exception fixups

A system call often receives a raw userspace address, but kernel code must not treat it like an ordinary trusted kernel pointer. Linux marks such pointers __user and uses helpers such as access_ok(), copy_from_user(), copy_to_user(), get_user() and put_user(). These helpers combine range validation, architecture access controls, fault recovery and hardening instrumentation.

userspace: write(fd, user_buf, 4096) ^ pointer belongs to process virtual address space ↓ SYSCALL kernel receives const char __user *buf KERNEL MUST COPY/ACCESS IT SAFELY copy_from_user(kernel_buf, buf, 4096) ↓ check copy size / object-size hardening might_fault() # normal copy path may page fault/sleep access_ok(buf, 4096) # address-range plausibility check barrier_nospec / architecture hardening as required ↓ x86 raw copy path STAC # temporarily allow supervisor access to user pages under SMAP REP MOVSB / optimized copy CLAC # disallow again IF USER PAGE IS PRESENT copy completes return value = 0 bytes not copied IF COPY TOUCHES A VALID BUT NOT-PRESENT USER PAGE CPU raises #PF while kernel is executing copy routine ↓ normal fault path may populate COW/file/swap/anonymous page ↓ faulting copy instruction retries and copy continues IF USER ADDRESS IS INVALID / UNRESOLVABLE kernel must NOT crash merely because user passed a bad pointer ↓ faulting instruction has an __ex_table fixup entry page-fault code recognizes kernel fault at fixup-capable instruction ↓ rewrite saved execution state to recovery label ↓ copy helper returns nonzero = number of bytes not copied syscall normally translates failure into -EFAULT IN ATOMIC / IRQ-SPINLOCK CONTEXT ordinary copy_from_user() may fault/sleep → unsafe ↓ use a design that pins/pre-faults data or appropriate inatomic helper only when its strict contract is satisfied SMAP DOES NOT MAKE USER DATA TRUSTWORTHY. It prevents accidental supervisor access when AC is clear; contents still require semantic validation after copying.

uaccess concept	Role
__user	Sparse/static-analysis annotation distinguishing userspace pointer provenance from ordinary kernel pointers.
access_ok()	Architecture range/plausibility check before selected user accesses; not a proof that every page is currently mapped.
copy_from_user()	Fault-aware/hardened copy from userspace into kernel memory; returns count of bytes not copied.
copy_to_user()	Fault-aware/hardened copy from kernel memory to userspace; returns count of bytes not copied.
get_user()/put_user()	Helpers for copying one simple scalar value to/from a userspace pointer.
might_fault()	Debug/context annotation noting that operation may cause a page fault and therefore may sleep.
SMAP	x86 Supervisor Mode Access Prevention blocks supervisor data access to user pages unless explicitly enabled.
STAC/CLAC	x86 instructions setting/clearing AC to temporarily permit/forbid SMAP-governed supervisor data access.
__ex_table	Kernel exception-fixup table mapping faulting instructions to recovery code instead of turning expected user-pointer faults into kernel oopses.
-EFAULT	Conventional syscall/API error for an invalid/unusable userspace address.
inatomic usercopy	Restricted user-access variant intended for contexts that cannot take normal sleeping page faults; caller bears stronger preconditions.
Hardened usercopy	Kernel checks that constrain copying to/from sensitive slab/object regions and detect suspicious bounds/type misuse.

access_ok() is not a dereference test. A pointer can lie within the permitted userspace address range yet still fault because its page is unmapped, swapped, COW-protected or concurrently changed. That is why the copy operation itself remains fault-aware.

SOURCE-READING LAB # generic wrappers and hardening # include/linux/uaccess.h # x86 implementation # arch/x86/include/asm/uaccess.h # arch/x86/include/asm/uaccess_64.h # arch/x86/include/asm/smap.h # exception-fixup explanation # Documentation/arch/x86/exception-tables.rst # Safe userspace experiment: # call read()/write()/ioctl-style APIs with a deliberately PROT_NONE mapping # and observe -EFAULT instead of a kernel crash where that API copies the address. # strace prints EFAULT results at syscall boundary. # Do not rely on EFAULT as universal validation: some syscalls defer or avoid # dereferencing pointers, and concurrent unmapping can race later accesses.

SOURCE / FILELinux generic uaccess implementation

Current source for copy_to/from_user wrappers, access_ok integration, might_fault(), object-size checks, speculation barriers and the contract that return values count uncopied bytes.

https://github.com/torvalds/linux/blob/master/include/linux/uaccess.h

SOURCE / FILELinux x86-64 usercopy implementation

Current x86 path uses STAC/CLAC around optimized copies and exception-table annotations so invalid user accesses return through fixup code.

https://github.com/torvalds/linux/blob/master/arch/x86/include/asm/uaccess_64.h

SOURCE / FILELinux x86 SMAP support

Current source explicitly says SMAP blocks supervisor access to user pages unless AC is set and provides stac()/clac() helpers.

https://github.com/torvalds/linux/blob/master/arch/x86/include/asm/smap.h

KERNEL DOCSLinux x86 exception tables

Detailed worked explanation of how a kernel-mode page fault during a user access is redirected through __ex_table fixup code and converted into -EFAULT behavior.

https://github.com/torvalds/linux/blob/master/Documentation/arch/x86/exception-tables.rst

The MMU enforces more than translation: U/S, R/W, NX, SMEP, SMAP and protection keys

Virtual memory is also a hardware privilege boundary. On x86, page-table entries contribute user/supervisor, writable and executable permissions; processor control bits add protections such as SMEP and SMAP; protection keys add a fast per-thread overlay for selected memory domains. These checks happen during translation/access, before ordinary kernel permission code gets a chance to run.

CPU ACCESS TO VIRTUAL ADDRESS V ↓ TLB hit OR page-table walk ↓ effective page permissions derived from paging structures PRESENT? no → #PF not-present fault CPL / U-S CHECK user-mode CPL3 accessing supervisor page? → #PF protection fault WRITE CHECK page read-only and access is write? → #PF unless privileged rules explicitly permit CR0.WP makes supervisor writes obey read-only page protection too EXECUTE CHECK NX/XD set on leaf mapping and access is instruction fetch? → #PF instruction-fetch protection fault SMEP supervisor attempts to EXECUTE instruction from user-accessible page → fault instead of treating user memory as kernel code SMAP supervisor attempts ordinary DATA access to user-accessible page while access-control state forbids it → fault Linux uaccess temporarily enables access using STAC/CLAC around controlled copies PKU / PROTECTION KEYS PTE carries 4-bit protection key ↓ per-thread PKRU has Access-Disable / Write-Disable bits per key ↓ data access allowed by page R/W/U bits BUT denied by PKRU? → #PF / userspace SIGSEGV with protection-key reason KEY DISTINCTION page-table permissions = address-space mapping policy PKRU = fast thread-local additional restriction without rewriting PTEs/TLBs RESULT only after all hardware translation/protection checks succeed does the load/store/fetch proceed into cache/memory.

Protection	Granularity/state	Prevents
U/S bit	Page-table hierarchy/leaf permissions	User-mode access to supervisor-only mappings.
R/W bit	Page-table hierarchy/leaf permissions	Writes to read-only mappings.
NX/XD bit	Leaf/page permission when NX enabled	Instruction fetch from non-executable pages.
CR0.WP	Global CPU control	Supervisor bypass of read-only page protection for normal writes.
SMEP	CR4 control + page U/S state	Supervisor execution of instructions from user-accessible pages.
SMAP	CR4 control + page U/S + AC state	Accidental supervisor data access to user pages outside explicit uaccess windows.
PKU	PTE protection-key tag + per-thread PKRU	Read/write access to selected user pages beyond ordinary page permissions.
mprotect()	Kernel page-table/VMA policy	Changes mapping R/W/X permissions, generally requiring page-table/TLB updates.
pkey_mprotect()	PTE key assignment plus ordinary permissions	Associates pages with a key so future access can be changed by PKRU register writes.

Protection keys only restrict further. A PKRU setting cannot make an NX page executable or make a read-only page writable in defiance of ordinary page-table permissions. It is an additional fast data-access filter layered on top of normal paging permissions.

READ-ONLY / SAFE LAB # CPU feature flags (names vary across vendors/kernel versions) grep -m1 '^flags' /proc/cpuinfo | tr ' ' '\n' | grep -E '^(nx|smep|smap|pku|ospke)$' # Page/VMA permissions grep -E 'r.x|rw.|---' /proc/$$/maps | head -40 # ProtectionKey field where pkeys are supported/in use grep -E '^(ProtectionKey|VmFlags):' /proc/$$/smaps | head -60 # Simple mprotect() lab: map RW, write data, switch one page RO, then handle SIGSEGV # when attempting a write. This demonstrates page permission enforcement safely. # pkey_alloc()/pkey_mprotect() can be tested in a small disposable program if # hardware/kernel support exists; never assume pkeys are universally available.

KERNEL DOCSLinux Memory Protection Keys

Current kernel documentation for x86 PKU and arm64 protection keys. On x86, PTEs carry one of 16 keys and thread-local PKRU can disable reads/writes without rewriting page tables.

https://docs.kernel.org/core-api/protection-keys.html

CURRENT PUBLIC MANUALpkeys(7)

Current userspace reference explaining pkeys as an additional restriction layered on PROT_READ/WRITE/EXEC, with SIGSEGV on violations.

https://man7.org/linux/man-pages/man7/pkeys.7.html

CURRENT PUBLIC MANUALmprotect(2)

Current mapping-permission interface for changing page protections and associating a protection key with pkey_mprotect().

https://man7.org/linux/man-pages/man2/mprotect.2.html

OFFICIAL DOCSIntel speculative-execution security features

Public description of Execute Disable, SMEP and SMAP as hardware security controls used by operating systems.

https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/hardware-behavior-related-to-speculative-execution.html

One x86-64 interrupt entry: vector → IDT gate → kernel stack → handler → IRETQ

An external interrupt is not a C function call. The Local APIC presents an interrupt vector to the core. The processor uses that vector to index the Interrupt Descriptor Table (IDT), validates the selected gate, switches stack when the gate/privilege/IST rules require it, builds an architectural return frame, and transfers RIP into low-level kernel entry code.

DEVICE EVENT NIC/NVMe/timer/etc. ↓ MSI/MSI-X / IO-APIC / local timer path LOCAL APIC chooses/delivers vector V ↓ CPU indexes IDT[V] 64-bit IDT gate is 16 bytes contains handler RIP, code selector, type/DPL/present, IST selector ↓ validate gate + privilege rules STACK SELECTION if coming from userspace CPL3 → kernel CPL0: use kernel stack pointer from privileged stack-switch machinery if IDT gate selects IST entry: use that dedicated TSS IST stack regardless of ordinary path ↓ CPU creates 64-bit interrupt return frame SS:RSP RFLAGS CS:RIP + exception error code for exceptions that architecturally supply one interrupt gate additionally clears IF while entering (trap gate does not clear IF in the same way) ↓ RIP = low-level entry stub from IDT gate ↓ Linux assembly/entry macro saves further GPR state / builds pt_regs-like frame ↓ generic IRQ / exception machinery ↓ device-specific ISR / threaded IRQ / softirq / NAPI / wakeup etc. ↓ controller acknowledgment as required e.g. Local APIC EOI for ordinary external IRQ handling ↓ exit path handles reschedule/signals/work before user return as needed ↓ restore registers IRETQ / architecture return path ↓ old RIP/RSP/RFLAGS/privilege context restored userspace resumes

x86 interrupt object	Purpose
IDTR	Privileged register holding IDT base address and limit.
IDT	Table indexed by interrupt/exception vector; 64-bit interrupt/trap gates are 16 bytes.
vector	8-bit event number selecting an IDT entry; exception vectors and external IRQ vectors share the dispatch mechanism.
interrupt gate	IDT gate transferring to handler and clearing IF on entry after saving flags.
trap gate	Similar IDT transfer but leaves IF handling different; often useful for synchronous/debug exceptions.
TSS RSP0/RSPn	Privileged stack pointers used for stack switching on privilege transitions in 64-bit mode.
IST	Interrupt Stack Table: dedicated alternate stacks selected directly by an IDT gate for events that need a known-good stack.
architectural frame	Saved SS/RSP/RFLAGS/CS/RIP plus exception error code when applicable.
pt_regs	Linux architecture-specific saved-register frame used by entry/exception/syscall code after additional software saves.
EOI	End Of Interrupt indication to the Local APIC after servicing an ordinary external interrupt.
IRETQ	64-bit interrupt return instruction restoring saved execution/privilege context.

The IDT chooses the first instruction, not the whole driver. The gate normally points to architecture entry assembly that normalizes/saves machine state. Only after that low-level entry work does Linux dispatch into generic IRQ/exception code and eventually the device driver's handler.

LINUX OBSERVATION LAB # Interrupt counts/vectors exposed by kernel cat /proc/interrupts | less # x86 IDT table symbols are privileged kernel internals; # inspect source instead of trying to dump live IDT memory on a normal machine. # Kernel source trail: # arch/x86/kernel/idt.c # arch/x86/entry/entry_64.S # arch/x86/include/asm/idtentry.h # Correlate one active device's IRQ with driver lspci -vv -s <BDF> | grep -Ei 'MSI|MSI-X|Interrupt' grep -i '<driver-or-device-name>' /proc/interrupts # perf can count/trace IRQ-related events where permissions/tracepoints allow.

DIRECT SPEC PDFIntel SDM Volume 3A — system programming guide (direct PDF)

Primary interrupt/exception reference. Its 64-bit mode chapter shows 16-byte IDT gates, fixed 8-byte stack pushes, saved SS:RSP and the IST stack-switch mechanism.

https://cdrdv2-public.intel.com/812386/253668-sdm-vol-3a.pdf

KERNEL DOCSLinux x86-specific documentation index

Current index covering kernel-level exception handling, kernel stacks and kernel-entry documentation.

https://docs.kernel.org/arch/x86/

KERNEL DOCSLinux page-table documentation — x86 page-fault entry

Concrete example of x86 IDT dispatch: the page-fault entry is defined through DEFINE_IDTENTRY_RAW_ERRORCODE() before converging on the generic VM fault machinery.

https://docs.kernel.org/mm/page_tables.html

SOURCE / FILELinux x86 IDT source

Real kernel construction of exception/system/interrupt gates and IST selections.

https://github.com/torvalds/linux/blob/master/arch/x86/kernel/idt.c

SOURCE / FILELinux x86 IDT entry macros

Actual low-level macros connecting IDT assembly stubs to C-visible exception/IRQ handlers.

https://github.com/torvalds/linux/blob/master/arch/x86/include/asm/idtentry.h

Interrupt work is split by execution context: hardirq, softirq/NAPI, threaded IRQ and workqueue

A device interrupt handler should usually do only the work that must happen immediately: identify/acknowledge the source, capture minimal state, stop an interrupt storm if necessary, and schedule the rest elsewhere. Linux has several deferred-execution mechanisms because they have different latency, CPU-affinity and sleeping rules.

DEVICE INTERRUPT ↓ x86 IDT / architecture entry ↓ Linux generic IRQ layer → primary handler HARDIRQ / PRIMARY HANDLER runs in interrupt context cannot perform operations that may sleep should be bounded/short typically reads status / masks or acknowledges source ↓ choose deferred mechanism A) SOFTIRQ per-CPU deferred interrupt context cannot sleep examples: NET_RX_SOFTIRQ, TIMER_SOFTIRQ, RCU-related work may run on return-from-interrupt or later in ksoftirqd under load B) NAPI NIC primary IRQ schedules NAPI instance and suppresses/reduces IRQs ↓ softirq invokes napi poll(budget) ↓ driver processes a bounded batch of RX/TX completions ↓ queue empty → complete NAPI → re-enable device interrupt C) THREADED IRQ request_threaded_irq(primary, thread_fn) ↓ primary handler returns IRQ_WAKE_THREAD ↓ irq/<n>-<name> kernel thread runs thread_fn ↓ thread context can sleep/use mutexes where normal kernel rules permit D) WORKQUEUE primary/softirq/thread queues work_struct ↓ kworker from managed worker pool executes function later ↓ process/thread context: work function may sleep unless workqueue/context rules forbid it The correct split depends on latency, ordering, sleep requirements, CPU locality, batching and how much work can arrive per interrupt.

Context	May sleep?	Typical use
hardirq	No	Immediate hardware acknowledgement, minimal status handling, schedule deferred work.
softirq	No	High-rate per-CPU deferred work such as network receive/transmit and selected timer/RCU processing.
NAPI poll	No in ordinary softirq mode	Bounded batch processing for networking; may also run in threaded/busy-poll modes.
threaded IRQ	Yes, subject to kernel locking rules	Longer device handling that benefits from process context and priority control.
workqueue	Yes for normal threaded workqueues	Arbitrary deferred kernel work that need not execute in interrupt context.
BH workqueue	No	Workqueue API mapped to bottom-half/softirq execution context.
ksoftirqd/N	Kernel thread	Backstop that runs softirq work when it cannot all be completed immediately.
irq_work	Usually interrupt-context mechanism; PREEMPT_RT changes execution for many items	Very-low-level deferred callback usable from contexts where normal scheduling/workqueues are unsuitable.

'Bottom half' is a family of ideas, not one modern Linux API. Softirqs, NAPI, threaded IRQs and workqueues all defer work away from the primary interrupt path, but their scheduling and sleeping properties differ substantially.

LINUX OBSERVATION LAB # interrupt counts cat /proc/interrupts | less # per-CPU softirq counters cat /proc/softirqs | less # softirq and IRQ kernel threads ps -eLo pid,tid,psr,cls,pri,stat,comm | grep -E 'ksoftirqd|irq/' # workqueue workers ps -eLo pid,tid,psr,stat,comm | grep kworker | head -60 # tracepoints available on this kernel perf list | grep -Ei 'irq:|softirq|workqueue' | less # NIC path: compare /proc/interrupts and /proc/softirqs while generating traffic. # Exact tracepoint names and PREEMPT_RT behavior depend on kernel configuration.

KERNEL DOCSLinux generic IRQ handling

Current IRQ abstraction and driver APIs including request_irq() and request_threaded_irq(); threaded handlers move selected work into a kernel thread.

https://docs.kernel.org/core-api/genericirq.html

KERNEL DOCSLinux Workqueue

Current Concurrency Managed Workqueue design: normal work executes in managed kworker pools, while BH workqueues are a convenience interface to softirq context.

https://docs.kernel.org/core-api/workqueue.html

KERNEL DOCSLinux per-CPU kthreads / ksoftirqd

Useful operational documentation identifying ksoftirqd/N, irq/%d-* threads and the softirq classes they execute under load.

https://docs.kernel.org/admin-guide/kernel-per-CPU-kthreads.html

KERNEL DOCSPREEMPT_RT execution-context differences

Important advanced caveat: PREEMPT_RT changes where hrtimers, irq_work and RCU callbacks execute, so 'interrupt context' behavior is kernel-configuration dependent.

https://docs.kernel.org/core-api/real-time/differences.html

A signal handler is entered by rewriting the user return context, not by an ordinary function call

A POSIX/Linux signal is asynchronous control transfer delivered to a process/thread according to its disposition and signal mask. When Linux is about to return a thread to userspace and notices a deliverable pending signal, it builds a signal frame in userspace containing the interrupted machine context, changes the return instruction pointer/stack so execution begins in the handler, and arranges a trampoline that later invokes rt_sigreturn to restore the saved state.

THREAD RUNNING USER CODE RIP/PC = application instruction registers + signal mask + stack state active SIGNAL BECOMES PENDING examples: kill()/tgkill() from another task terminal generates SIGINT timer expires child exits → SIGCHLD CPU fault becomes SIGSEGV/SIGFPE/SIGILL/SIGBUS as appropriate kernel chooses a deliverable unblocked signal for this thread ↓ before returning to user mode: save interrupted register/flags/context into USER-SPACE SIGNAL FRAME save old signal mask / alt-stack state optionally switch to sigaltstack() update blocked mask according to sigaction() set user instruction pointer to handler arrange return address/trampoline ↓ return from kernel to USER MODE ↓ handler(signum, siginfo_t*, ucontext_t*) runs handler returns ↓ signal trampoline executes rt_sigreturn syscall ↓ kernel validates/reads signal frame restores saved signal mask, SP, IP/PC, flags/register context ↓ returns to user mode at interrupted/resume point rt_sigreturn() never returns like a normal C function; its whole purpose is to restore an earlier execution context.

Signal concept	Meaning
disposition	Per-signal action: default, ignore, or user-installed handler via sigaction().
pending	Signal has been generated for the process/thread but not yet delivered.
blocked/masked	Delivery is deferred while signal is present in the thread's signal mask.
standard signal	Traditional signal generally not queued multiple times; multiple instances can collapse while pending.
real-time signal	Queued signal class with ordering/associated siginfo behavior and a per-user queue limit.
sigaction()	Preferred API for installing handlers and controlling masks/flags.
SA_SIGINFO	Requests three-argument handler receiving siginfo_t and a pointer to saved user context.
SA_RESTART	Requests automatic restart of certain interrupted blocking syscalls when supported by that syscall/interface.
sigaltstack()	Provides alternate user stack used for handlers marked SA_ONSTACK, useful for stack-overflow/fault handling.
signal frame	Architecture-specific user-stack object containing saved register/context/mask information for the interrupted thread.
signal trampoline	Userspace/vDSO/libc code reached after handler returns; invokes rt_sigreturn.
rt_sigreturn	Linux syscall restoring the pre-handler user context from the signal frame.
SIGKILL / SIGSTOP	Special signals whose disposition cannot be caught, blocked or ignored in the ordinary way.

Signals are not the same as hardware interrupts. A hardware IRQ enters the kernel through CPU interrupt machinery. A Unix signal is a kernel-managed userspace notification/control-transfer abstraction; an IRQ handler, timer, syscall, exception or another process can eventually cause a signal to become pending.

SIGNAL OBSERVATION LAB # terminal-generated SIGINT and signal-related syscalls strace -e trace=signal,sigaction,rt_sigaction,rt_sigprocmask,rt_sigreturn,sigaltstack ./program # show signal dispositions/masks/pending sets from /proc grep -E 'Sig(Q|Pnd|Blk|Ign|Cgt):' /proc/$$/status # send a signal explicitly kill -USR1 <PID> # in GDB, stop at a signal handler and inspect registers/stack # then single-step the handler return/trampoline if your libc/kernel/debugger exposes it clearly # Write handlers conservatively: most library functions are NOT async-signal-safe. # signal-safety(7) lists interfaces POSIX requires to be safe in a handler.

PUBLIC MANUALsignal(7) — current Linux signal overview

Current 2026 manual describes pending/blocked signals and the complete delivery sequence: kernel builds a user stack frame, enters the handler, returns through a signal trampoline and restores context with sigreturn.

https://man7.org/linux/man-pages/man7/signal.7.html

PUBLIC MANUALsigreturn(2) / rt_sigreturn — current manual

The crucial low-level path: kernel saves status/registers/mask/stack state in a user frame; rt_sigreturn restores that context so execution resumes where interrupted.

https://man7.org/linux/man-pages/man2/sigreturn.2.html

PUBLIC MANUALsigaction(2)

Preferred signal-disposition API including SA_SIGINFO, SA_ONSTACK, SA_RESTART, masking and the signal-restorer/trampoline interface.

https://man7.org/linux/man-pages/man2/sigaction.2.html

PUBLIC MANUALsigaltstack(2)

Shows how a handler can run on a separately allocated stack instead of the interrupted normal user stack.

https://man7.org/linux/man-pages/man2/sigaltstack.2.html

PUBLIC MANUALsignal-safety(7)

Important implementation reality: asynchronous handlers can interrupt libc in arbitrary state, so only async-signal-safe operations are portable/safe from a signal handler.

https://man7.org/linux/man-pages/man7/signal-safety.7.html

KERNEL DOCSLinux x86 kernel entries documentation

Use beside signals to distinguish kernel entry/return mechanics from the higher-level POSIX signal frame that Linux constructs before returning to user mode.

https://docs.kernel.org/arch/x86/entry_64.html

A signal can interrupt a blocking syscall, but “interrupted” does not always mean userspace sees EINTR

When a thread is sleeping inside a blocking kernel operation and a signal handler must run, Linux has to decide what happens to the interrupted operation. For some interfaces, a handler installed with SA_RESTART lets the kernel/library make the call appear to continue after the handler returns. Other interfaces deliberately return -1 with errno=EINTR. Still others can return a short successful result if some work completed before interruption. Correct code therefore reasons about the particular interface, partial progress and timeouts instead of blindly retrying every EINTR.

thread blocks in syscall ↓ interruptible sleep / wait queue ↓ signal becomes deliverable kernel prepares signal frame and returns to handler ↓ handler eventually rt_sigreturn() what should happen to interrupted operation? ├─ restartable + SA_RESTART applies │ → kernel arranges restart → userspace may never observe EINTR ├─ interface is never/conditionally restarted │ → return -1, errno = EINTR ├─ partial I/O already completed │ → return short byte count instead of EINTR └─ special timed wait needing elapsed-time correction → internal restart path can use restart_syscall()

Case	Correct mental model
`SA_RESTART`	Affects only selected interrupted interfaces. It is not a global “signals never interrupt syscalls” switch.
`EINTR`	The operation did not complete in the ordinary way because signal handling intervened. The caller decides whether retrying is semantically correct.
short read/write	If bytes were transferred before the signal, many I/O calls return the positive byte count. Retrying must begin after that progress, not from the original buffer offset.
timeouts	Naively repeating a relative-time wait can accidentally extend the total deadline. Absolute deadlines or kernel restart machinery avoid that bug.
`restart_syscall()`	An internal Linux mechanism used for selected operations whose remaining timeout must account for time spent stopped; ordinary applications normally do not call it.

“Retry on EINTR” is not a universal rule. A retry can duplicate side effects, extend a timeout, or ignore partial progress. Robust loops are interface-specific and usually recompute what remains to be done.

SEE EARLIERPOSIX/Linux signal delivery and rt_sigreturn

The earlier signal section shows how the kernel constructs a userspace signal frame and later restores the interrupted context; this section focuses on what happens to a blocked operation around that detour.

PUBLIC MANUALsigaction(2) — SA_RESTART and handler flags

Current Linux manual describing SA_RESTART and the signal-handler attributes that control restart semantics. It points to signal(7) for the interface-by-interface restart rules.

https://www.man7.org/linux/man-pages/man2/rt_sigaction.2.html

PUBLIC MANUALrestart_syscall(2) — internal restart with timeout adjustment

Current Linux manual for the kernel-visible restart path used by selected timed waits such as poll/nanosleep/futex waits after stop-and-continue sequences.

https://man7.org/linux/man-pages/man2/restart_syscall.2.html

A crash can become a structured memory snapshot: fatal signal → dump policy → ELF ET_CORE → debugger

Some signals have a default disposition of “terminate and dump core.” On Linux, that does not mean the kernel blindly writes all process memory to a file named core. The kernel first applies dumpability/resource/policy checks, selects which mappings are eligible, then emits a core image—normally an ELF ET_CORE object containing memory segments plus note records for thread/register/process state. core_pattern can instead pipe the dump stream to a userspace collector.

fault / abort / externally generated fatal signal ↓ signal disposition says CORE ↓ terminate path decides whether a dump is permitted ├── process dumpable state ├── RLIMIT_CORE / filesystem conditions ├── kernel CONFIG_COREDUMP support └── security/policy constraints ↓ select mappings to include ├── /proc/PID/coredump_filter bitmask └── MADV_DONTDUMP can exclude selected mappings ↓ construct ELF core image ├── ELF header: e_type = ET_CORE ├── PT_LOAD-style memory image segments as selected └── PT_NOTE / ELF notes carrying register/process/thread metadata ↓ /proc/sys/kernel/core_pattern ├── filename template → kernel writes core file └── leading pipe ('|') → stream dump to userspace collector ↓ crashing task completes termination ↓ parent/reaper can later collect wait status ↓ debugger combines core + executable/shared-library symbols ↓ inspect registers, stacks and mapped memory from crash time

Control / object	Purpose
`RLIMIT_CORE`	Per-process soft resource limit that can restrict the size of a generated core file.
`core_pattern`	Kernel template for dump filenames or a pipe-to-program collector command.
`coredump_filter`	Per-process bitmask choosing categories of memory mappings to include.
`MADV_DONTDUMP`	Mapping-level instruction that memory should be excluded from core dumps.
ELF `ET_CORE`	File type used for core images; notes can carry register/process metadata while loadable segments represent selected memory contents.
debug symbols / executable	Core data gives addresses and state; debuggers use executable/library metadata and symbols to turn addresses into functions, variables and source locations.

A core dump is a filtered snapshot, not a replay recording. It captures selected state at one termination point. It does not contain every historical instruction or every mapping, and modern systems may route dumps through a collector instead of leaving a traditional ./core file.

PUBLIC MANUALcore(5) — Linux core-dump policy and format controls

Current reference for signals that can create dumps, reasons a dump may be suppressed, core_pattern, piped collectors, coredump_filter and systemd integration.

https://www.man7.org/linux/man-pages/man5/core.5.html

PUBLIC MANUALelf(5) — ET_CORE and ELF notes

Explains that core files are ELF objects with e_type=ET_CORE and documents the note structures used for architecture/OS-specific process state.

https://www.man7.org/linux/man-pages/man5/elf.5.html

PUBLIC MANUAL/proc/PID/coredump_filter — per-process dump-selection control

Direct entry point for the per-process mapping filter described in core(5).

https://man7.org/linux/man-pages/man5/proc_pid_coredump_filter.5.html

A kernel panic can preserve the crashed kernel for a second kernel to inspect: panic → crash kexec → /proc/vmcore

A userspace process crash and a kernel panic are different failure domains. A fatal userspace signal can usually be contained to one process; a panic means the running kernel has decided the machine cannot safely continue. Linux can optionally prepare a second, small dump-capture kernel in memory reserved by crashkernel=. When a qualifying panic occurs, the crash path transfers control to that preloaded kernel with the kexec mechanism instead of relying on the damaged kernel to write its own dump.

NORMAL BOOT firmware → bootloader → production kernel ↓ reserves crashkernel memory load dump-capture kernel with KEXEC_ON_CRASH ↓ ordinary workload runs KERNEL PANIC / configured fatal condition ↓ stop/coordinate CPUs and enter crash path ↓ kexec jumps into preloaded dump-capture kernel (no normal firmware reboot is required for this handoff) ↓ dump kernel boots from reserved memory ↓ old production-kernel RAM is exposed as /proc/vmcore ↓ makedumpfile / cp / collector writes a vmcore ↓ crash, drgn, GDB or other tooling analyzes memory + symbols

Mechanism	What it means
oops	A serious kernel fault report. Depending on context and policy, Linux may attempt to continue or escalate to panic.
panic	Kernel declares execution cannot safely continue; panic policy decides whether to halt, reboot, or enter crash-dump handling.
`crashkernel=`	Boot-time reservation that keeps memory available for the dump-capture kernel and its minimal runtime.
kexec	Loads/transfers to another kernel without the ordinary firmware/bootloader path. Crash kexec is the panic-specific use.
`/proc/vmcore`	ELF-format view of the crashed kernel's preserved physical-memory image as exposed by the dump kernel.
userspace core dump	Snapshot of one process after a fatal signal; it is not the same object or failure path as a kernel vmcore.

Important: kdump must be prepared before the crash. The capture kernel and reserved memory exist precisely because the panicking kernel is no longer trusted to perform a complex filesystem/network dump path itself.

KERNEL DOCSLinux kdump — kexec-based crash dumping

Current kernel documentation for crashkernel reservation, loading the dump-capture kernel, panic handoff, /proc/vmcore, makedumpfile and post-crash analysis.

https://docs.kernel.org/admin-guide/kdump/kdump.html

PUBLIC MANUALkexec_load(2) / kexec_file_load(2)

Linux system-call interface for preloading a later kernel, including the KEXEC_ON_CRASH/KEXEC_FILE_ON_CRASH modes used with a reserved crash-kernel region.

https://man7.org/linux/man-pages/man2/kexec_load.2.html

“The kernel is stuck” has several meanings: soft lockup, hard lockup, hung task and RCU stall detectors watch different progress signals

Linux has multiple watchdog-like diagnostics because a machine can stop making progress in different ways. A CPU can keep taking interrupts but fail to schedule tasks; it can stop taking ordinary interrupts entirely; one task can remain blocked in uninterruptible sleep; or RCU can fail to reach the quiescent states needed to finish a grace period. The corresponding detectors observe different heartbeats, so one warning is evidence about a particular failure mode rather than a generic verdict that “the kernel froze.”

SOFT LOCKUP CPU still receives timer/interrupt activity but watchdog scheduling progress is missing too long → warning / stack trace / optional panic HARD LOCKUP CPU stops producing ordinary hrtimer heartbeat NMI/perf watchdog (or buddy CPU where supported) can still observe it → warning / stack trace / optional panic HUNG TASK a task remains in uninterruptible D state beyond configured timeout → task stack/state report / optional panic RCU STALL an RCU grace period cannot observe required quiescent progress → CPU/task/grace-period diagnostics These can overlap, but they are not synonyms.

Detector	Progress signal it watches	Typical clue
softlockup	Whether the watchdog scheduling job gets CPU time within the threshold.	Long kernel loop/preemption-disabled region while interrupts may still arrive.
hardlockup / NMI watchdog	Whether ordinary hrtimer interrupt heartbeats advance.	CPU spinning with interrupts disabled or otherwise no longer taking normal interrupts.
hung-task detector	How long a task has remained in uninterruptible sleep without being scheduled out of that state.	Blocked storage/device/kernel wait; the CPU itself may be healthy.
RCU stall detector	Whether an in-progress RCU grace period receives required quiescent-state progress.	Interrupt/preemption-disabled loops, starved RCU threads, timing problems or other conditions preventing grace-period completion.
hardware watchdog	An external/platform heartbeat expected from the running system.	Last-resort recovery by reset when software no longer services the watchdog.

Detection and recovery are separate decisions. These kernel detectors normally emit diagnostics and can optionally panic; a panic can feed kdump/pstore, and a hardware watchdog or panic timeout can eventually reboot. Turning every detector into an immediate reset can destroy the very evidence needed to diagnose the problem.

SEE EARLIERHardware watchdog recovery

The earlier watchdog section covers the independent platform timer that can reset a machine. The kernel lockup/stall detectors here are primarily software diagnostics and may optionally escalate to panic.

KERNEL DOCSLinux softlockup and hardlockup detectors

Current kernel documentation for the scheduler-based softlockup detector and the hardlockup detector using NMI/perf heartbeats or supported buddy-CPU monitoring.

https://docs.kernel.org/admin-guide/lockup-watchdogs.html

KERNEL DOCSUsing RCU’s CPU stall detector

Current kernel documentation for causes, thresholds and interpretation of RCU stall warnings, including starvation, disabled interrupts/preemption and timer problems.

https://docs.kernel.org/RCU/stallwarn.html

When the kernel cannot write a normal log, pstore can leave a small message for the next boot: panic/oops → persistent backend → /sys/fs/pstore

Kdump preserves a large crashed-kernel memory image, but sometimes the useful artifact is much smaller: the last kernel messages before a panic, reboot or shutdown failure. Linux pstore provides a filesystem-facing framework for records that survive a reset because a backend stores them somewhere persistent enough. ramoops is a common backend that reserves a RAM region which firmware/platform behavior leaves intact across a reboot.

NORMAL RUNNING KERNEL printk / console / ftrace state ↓ serious event: oops, panic, shutdown hang, explicit dump trigger ↓ pstore/kmsg dump path ↓ backend chosen by platform/configuration ├── ramoops / persistent reserved RAM ├── EFI-variable-backed pstore (where supported) ├── pstore/blk backend └── other platform backend ↓ SYSTEM RESETS / REBOOTS ↓ next kernel mounts pstore ↓ /sys/fs/pstore/ dmesg-ramoops-* console-ramoops-* ftrace-ramoops-* (when configured) ↓ userspace archives/clears records (e.g. systemd-pstore on systems that use it)

Mechanism	What it preserves
pstore	Kernel framework/filesystem presenting persistent diagnostic records produced by one of several storage backends.
ramoops	pstore backend using a reserved persistent-RAM region, commonly for oops/panic/console/ftrace records.
kmsg dump	Path that snapshots kernel log data to a registered dumper during configured failure/shutdown events.
`/sys/fs/pstore`	Userspace-visible files recovered from persistent records after the next boot.
kdump	Separate mechanism that boots a capture kernel and exposes a full crashed-kernel memory image as `/proc/vmcore`.

pstore is intentionally small and simple compared with kdump. It is excellent for “what did the kernel say immediately before reset?” and for machines where a full vmcore is impractical. It is not a substitute for a complete memory dump when you need arbitrary crashed-kernel data structures.

READ-ONLY PSTORE LAB mount | grep ' /sys/fs/pstore ' 2>/dev/null ls -l /sys/fs/pstore 2>/dev/null # Kernel configuration if exposed zgrep -E 'CONFIG_PSTORE|CONFIG_PSTORE_RAM' /proc/config.gz 2>/dev/null # Platform/module details vary; inspect before changing reserved-memory or # panic/reboot policy on a production system.

KERNEL DOCSLinux ramoops persistent oops/panic logger

Current kernel documentation for reserved persistent RAM, record sizing, panic/oops logs and retrieval through the pstore filesystem.

https://docs.kernel.org/admin-guide/ramoops.html

KERNEL DOCSDebugging shutdown hangs with pstore

Concrete current example of using pstore backends to retain kernel messages across a reset and recover them on the next boot.

https://docs.kernel.org/power/shutdown-debugging.html

A kernel log message has its own data path: printk → kernel ring buffer → /dev/kmsg/dmesg → journald/console/persistent logs

Kernel diagnostics do not begin as ordinary userspace files. printk() and the pr_*() helpers append records with severity metadata to the kernel log ring buffer. Console policy decides which records are also emitted immediately to configured consoles. Userspace can read the kernel stream through /dev/kmsg; dmesg is the conventional inspector, while a logging daemon such as systemd-journald can ingest kernel records alongside service stdout/stderr, syslog and other sources and optionally persist them on disk.

KERNEL CODE pr_info / pr_warn / printk(...) ↓ record + timestamp/sequence/severity metadata ↓ KERNEL LOG RING BUFFER ├── console policy → serial/VGA/other kernel console └── /dev/kmsg → userspace readers ├── dmesg └── systemd-journald ├── volatile journal under /run └── persistent journal under /var (configuration dependent) PANIC / EARLY-BOOT EDGE CASE normal journal/storage path may be unavailable ↓ pstore/ramoops can preserve a small independent crash log across reset

Layer	Role
`printk()` / `pr_*`	Kernel logging API; records carry severity levels such as emergency, error, warning, info and debug.
kernel log ring buffer	In-memory ordered store of kernel messages; it exists independently of a userspace logging daemon.
console log level	Controls which severities are emitted to active kernel consoles; a message can remain in the ring buffer without being printed to the console.
`/dev/kmsg`	Userspace interface to the kernel message stream.
`dmesg`	Utility for examining/controlling the kernel ring buffer rather than a persistent log database.
`systemd-journald`	Userspace collector that can merge kernel messages with service/stdout/syslog/audit sources and write journal files.

The console, dmesg and the journal are different views/layers. Suppressing a low-priority message from the console does not necessarily remove it from the kernel ring buffer, and persisting it across reboot requires a userspace journal or a special persistent crash backend such as pstore.

KERNEL DOCSLinux — message logging with printk

Current kernel documentation for printk/pr_* severity levels, the kernel ring buffer, /dev/kmsg, console log-level filtering and deferred console output.

https://docs.kernel.org/core-api/printk-basics.html

CURRENT MANUALdmesg(1) — inspect/control the kernel ring buffer

Current util-linux manual describing dmesg as the userspace tool for reading and controlling the kernel ring buffer.

https://man7.org/linux/man-pages/man1/dmesg.1.html

CURRENT MANUALsystemd-journald — journal collection and storage

Current upstream manual describing journald inputs including /dev/kmsg, syslog-compatible sockets and service streams, plus volatile/persistent journal handling.

https://www.man7.org/linux/man-pages/man8/systemd-journald.8.html

Sleeping and waking are state transitions: runqueues, EEVDF selection, preemption and migration

The scheduler does not 'run every process a little bit' in one central loop. Each CPU has scheduling state/runqueue machinery. Tasks move between states such as running, runnable and sleeping. A device completion, futex wake, timer or other event can make a sleeping task runnable; the scheduler then decides which CPU should host it and whether it should preempt what is already running.

RUNNING TASK A on CPU 3 ↓ A calls blocking read()/futex wait/sleep ↓ kernel changes A to a sleeping / non-runnable state dequeue from runnable scheduling set ↓ pick_next_task() chooses another runnable task B ↓ context switch A → B LATER: event completes device IRQ / timer / futex wake / pipe data / network packet ↓ try_to_wake_up()-style wakeup path ↓ task A becomes RUNNABLE ↓ choose suitable CPU considering affinity/topology/load/capacity/policy ↓ enqueue A on that CPU's scheduling structure ↓ should A preempt current task? ├── no → waits runnable until selected later └── yes → reschedule / context switch NORMAL FAIR-SCHEDULING DIRECTION IN CURRENT LINUX DOCS: EEVDF task virtual runtime → compute lag (owed vs over-served CPU time) eligible task: lag >= 0 eligible tasks get virtual deadlines pick earliest eligible virtual deadline Other classes (deadline/real-time/etc.) have different rules and can outrank fair tasks.

Scheduler concept	Meaning
running	Task currently executing instructions on a logical CPU.
runnable	Task is eligible to run but may be waiting in scheduling structures for CPU time.
sleeping / blocked	Task is not runnable until an event/condition wakes it.
enqueue_task	Scheduler-class operation placing a newly runnable task into its scheduling structure.
dequeue_task	Removes task when it stops being runnable or moves elsewhere.
wakeup CPU selection	Chooses a CPU for a newly runnable task, respecting affinity and considering topology/load/capacity/policy.
wakeup preemption	Checks whether a waking task should cause the current task to yield CPU soon/immediately.
context switch	Saves/restores enough execution state and changes which task is running.
CPU migration	Task becomes/runs on a different logical CPU; can improve load balance but disturb cache locality.
virtual runtime	Fair-scheduling accounting of CPU service in a virtual-time model.
EEVDF lag	Difference indicating whether a fair task is owed CPU time (positive/nonnegative eligibility) or has received more than its share.
virtual deadline	EEVDF scheduling value used to prioritize among eligible fair tasks; earlier eligible deadline is selected.
scheduler tick	Periodic/event-driven accounting/preemption opportunity; modern scheduling is not limited to fixed old-style time slices.

Linux is transitioning its normal fair scheduler from the older CFS selection model toward EEVDF. Current kernel documentation describes EEVDF as using task lag for eligibility and earliest virtual deadline for selection. The older CFS documentation remains valuable for concepts such as virtual runtime, enqueue/dequeue hooks and scheduler-class structure.

LINUX OBSERVATION LAB # show scheduling-related task events/counters perf stat -e context-switches,cpu-migrations,task-clock <command> # CPU affinity taskset -pc $$ # process/thread scheduling information ps -eLo pid,tid,psr,cls,rtprio,pri,ni,stat,comm | less # watch a process migrate among CPUs (psr column) watch -n 0.5 'ps -o pid,psr,stat,pri,ni,comm -p <PID>' # trace scheduler events when perf tracepoints are available perf list | grep -E 'sched:sched_(switch|wakeup)' # exact available tracepoints/tools depend on distro/kernel permissions.

KERNEL DOCSLinux EEVDF Scheduler — current documentation

Current kernel docs describe Linux's transition toward EEVDF: lag determines eligibility and the earliest virtual deadline among eligible tasks is selected.

https://docs.kernel.org/scheduler/sched-eevdf.html

KERNEL DOCSLinux CFS Scheduler design

Still useful for scheduler architecture: enqueue_task, dequeue_task, wakeup_preempt, pick_next_task and task_tick hooks plus virtual-runtime concepts.

https://docs.kernel.org/scheduler/sched-design-CFS.html

KERNEL DOCSLinux Scheduler Statistics

Current documentation exposes wakeup, balancing and migration counters, including try_to_wake_up and load-balancing activity.

https://docs.kernel.org/scheduler/sched-stats.html

KERNEL DOCSLinux CPUSETS / scheduler-domain behavior

Concrete topology/load-balancing discussion: waking tasks may move to idle sibling CPUs and idle CPUs can pull tasks from busier scheduling domains.

https://docs.kernel.org/admin-guide/cgroup-v1/cpusets.html

KERNEL DOCSLinux sched_ext — scheduling cycle

Modern scheduler-framework documentation with an especially clear wakeup→CPU selection→dispatch→run cycle, even though sched_ext itself is an extensible alternative class.

https://docs.kernel.org/scheduler/sched-ext.html

RCU lets readers run almost untouched while updaters defer reclamation until old readers are gone

Read-Copy Update (RCU) is a synchronization technique optimized for read-mostly structures. Readers traverse published pointers inside lightweight read-side critical sections. Updaters publish a replacement/removal using proper ordering, then wait for a grace period before freeing the old object—long enough that every reader that could still hold the old pointer has finished.

INITIAL STRUCTURE global pointer ─→ object A READER R1 rcu_read_lock() p = rcu_dereference(global) use object A through p ... rcu_read_unlock() UPDATER U allocate/copy object B modify B privately ↓ publish B with rcu_assign_pointer(global, B) NOW: new readers → B old R1 may STILL legally hold A WRONG: publish B kfree(A) immediately ↓ R1 dereferences freed memory → use-after-free SYNCHRONOUS RCU RECLAMATION publish B synchronize_rcu() waits for a grace period: every RCU read-side critical section that began before the grace period has completed ↓ kfree(A) is now safe with respect to those old readers ASYNCHRONOUS RCU RECLAMATION publish/remove A call_rcu(&A->rcu, free_callback) ↓ updater continues without blocking for the grace period ↓ later, after grace period RCU invokes callback → free A QUIESCENT STATE IDEA RCU tracks CPUs/tasks until it knows each pre-existing reader can no longer be inside the old read-side critical section. Depending on RCU flavor/configuration, context switch, user mode, idle, offline transitions and explicit read-side tracking participate in that proof.

RCU primitive/concept	Meaning
rcu_read_lock()/unlock()	Marks a read-side critical section whose referenced RCU-protected objects cannot be reclaimed underneath it.
rcu_dereference()	Loads a published RCU pointer with ordering/compiler rules needed for dependent reads.
rcu_assign_pointer()	Publishes a new pointer with ordering so initialized object state is visible before readers see the pointer.
grace period	Interval after which all RCU read-side critical sections that began before it have completed.
synchronize_rcu()	Blocks the updater until a grace period completes.
call_rcu()	Queues a callback to run after a future grace period without blocking the updater now.
kfree_rcu()	Convenience mechanism for freeing an RCU-protected allocation after the required grace period.
quiescent state	Observation proving a CPU/task is not still inside a relevant pre-existing read-side critical section.
SRCU	Sleepable RCU flavor whose reader rules differ from ordinary kernel RCU and permit sleeping read-side sections.
RCU stall warning	Diagnostic indicating a grace period is waiting too long for one or more CPUs/tasks/readers to report progress.
rcu_barrier()	Waits for already-posted RCU callbacks to finish; it is not simply another name for synchronize_rcu().

RCU does not make writers disappear. Updaters still need whatever locking/atomic discipline is required to serialize with other updaters. RCU's special trick is allowing old readers and a newly published version to coexist temporarily, then delaying destruction of the old version until it is safe.

KERNEL DOCSLinux RCU Handbook

The central current RCU documentation index, including the 'What is RCU?' overview, list examples, grace-period design and debugging material.

https://docs.kernel.org/RCU/index.html

KERNEL DOCSA Tour Through RCU's Requirements

Current high-level specification: the grace-period guarantee waits for all pre-existing RCU read-side critical sections while new readers may run concurrently.

https://docs.kernel.org/RCU/Design/Requirements/Requirements.html

KERNEL DOCSLinux RCU list example

Concrete list traversal/removal examples using rcu_read_lock(), list_for_each_entry_rcu(), list_del_rcu(), synchronize_rcu() and call_rcu().

https://docs.kernel.org/RCU/listRCU.html

KERNEL DOCSLinux Tree RCU data structures

Deep implementation material on rcu_node trees, CPU tracking, grace-period sequencing and callback management.

https://docs.kernel.org/RCU/Design/Data-Structures/Data-Structures.html

Worked example: the same ideas inside a modern small SoC

Modern chips hide the wires inside silicon, but the same categories still exist. RP2040 is a good modern example because its documentation openly shows the bus fabric, SRAM banks, ROM, flash interface, DMA, peripherals, clocks, resets and power.

reference crystal / clock inputs ↓ clock generation + PLLs + clock domains ↓ CPU core 0 ─┐ CPU core 1 ─┼──→ AHB-Lite crossbar / bus fabric ─→ ROM DMA read ──┤ ├→ XIP flash interface DMA write ──┘ ├→ SRAM banks ├→ high-speed peripherals └→ APB bridge → lower-speed peripherals GPIO / UART / SPI / I²C / USB / PIO ←→ physical pins and external devices

Old/simple-computer idea	RP2040 equivalent
Shared address/data path	On-chip AHB-Lite crossbar routes requests and responses
CPU is the only bus master	Two CPUs plus separate DMA read/write masters can generate transactions
External RAM chips	Multiple on-chip SRAM banks with dedicated crossbar ports
External ROM/PROM	On-chip ROM plus external QSPI flash through execute-in-place interface
Discrete address decoder	Crossbar splitters decode addresses and route transactions
Bus arbitration logic	Crossbar arbiters resolve simultaneous requests
Separate peripheral ICs	Many peripherals are integrated on-chip and exposed as memory-mapped register blocks
Clock can be a simple oscillator	Multiple clock sources/domains and PLL-based clock generation

OFFICIAL DOCSRaspberry Pi RP2040 document portal

Public official document index. Useful if datasheet revisions move; it lists the current RP2040 datasheet, hardware-design PDF and KiCad reference files.

https://pip.raspberrypi.com/categories/814-rp2040

A modern desktop motherboard: what is actually connected to what

There is no single universal PC topology, but a common modern two-chip desktop pattern is: the processor package contains CPU cores, caches, the integrated memory controller and some high-speed PCIe; a separate Platform Controller Hub (PCH) provides much of the slower/general I/O and connects back to the processor over DMI.

ATX / PSU │ ├── 12 V ─→ motherboard VRMs ─→ CPU core / SoC / memory-related rails └── other rails ───────────────→ board devices ┌────────────── PROCESSOR PACKAGE ──────────────┐ DDR5 DIMMs ⇄ memory bus ⇄ │ integrated memory controller │ │ CPU cores ⇄ caches ⇄ internal fabric │ GPU / NVMe ⇄ PCIe lanes ⇄ │ PCIe root ports │ └────────────────┬──────────────────────────────┘ │ DMI (point-to-point link) ↓ ┌──────────────── PCH ─────────────────────────┐ │ extra PCIe root ports │ │ USB host controller │ │ SATA controller │ │ SPI / eSPI / SMBus / I²C / GPIO / RTC │ │ interrupt / timer / platform-management logic │ └──────┬──────────┬──────────┬──────────────────┘ │ │ │ USB ports SATA SPI flash (UEFI/firmware) Fans / sensors / legacy I/O may hang from SMBus, eSPI, Super-I/O or embedded controllers.

OFFICIAL DOCSIntel 700 Series PCH overview and public datasheet

Public documentation. The table of contents alone is instructive: PCIe, SATA, USB, SPI/eSPI, GPIO, SMBus, clocks, RTC, power management, audio and DMI all live around the PCH.

https://edc.intel.com/content/www/us/en/design/products-and-solutions/processors-and-chipsets/700-series-chipset-family-platform-controller-hub-datasheet-volume-1-of/overview/

OFFICIAL DOCSIntel processor ↔ PCH DMI connection

Official public processor documentation explicitly stating that DMI connects the processor and the PCH, with the actual high-speed point-to-point link characteristics.

https://edc.intel.com/content/www/us/en/design/products/platforms/details/arrow-lake-s/core-ultra-200s-series-processors-datasheet-volume-1-of-2/direct-media-interface-dmi/

DIRECT PDFIntroduction to Intel Architecture — public white paper (PDF)

Older, but pedagogically useful because its block diagrams visibly separate cores, LLC, integrated memory controller, PCIe, graphics and the PCH.

https://cdrdv2-public.intel.com/786255/786255_330119_ia-introduction-basics-paper.pdf

DIRECT PDFIntel 9 Series PCH datasheet (PDF)

A large real chipset datasheet with clocks, SPI firmware, PCIe, SATA, USB, power/reset signals and platform block diagrams. Older than current chipsets, easier to study as a concrete PC example.

https://www.intel.com/content/dam/www/public/us/en/documents/datasheets/9-series-chipset-pch-datasheet.pdf

Storage as hardware: DRAM, flash and SSDs

RAM and storage are not interchangeable abstractions. DRAM stores charge that must be sensed and refreshed; NAND flash stores charge in nonvolatile cells and requires erase/program management, ECC and wear handling.

DIRECT PDFMicron — Introduction to Memory presentation (PDF)

Free manufacturer material covering memory hierarchy and semiconductor memory. The NAND section includes an actual floating-gate cell cross-section and explains program/erase/read at the device level.

https://www.micron.com/content/dam/micron/educatorhub/intro-to-memory/micron-intro-to-memory-presentation.pdf

DIRECT PDFMicron — NOR NAND Flash Guide (PDF)

Manufacturer guide distinguishing NOR from NAND, raw versus managed NAND, SLC/MLC/TLC/QLC, ECC, bad-block management and wear leveling.

https://assets.micron.com/adobe/assets/urn:aaid:aem:e98bc653-e42d-45a7-9b8c-03cb3d488f05/original/as/nor-nand-flash-guide.pdf

TECH ARTICLEKen Shirriff — reverse engineering the MK4116 DRAM

Die-level explanation of a classic 16K DRAM: one-transistor cells, bit lines, word lines, sense amplifiers, row/column decoding and refresh. One of the best links in the whole collection for understanding DRAM physically.

https://www.righto.com/2020/11/reverse-engineering-classic-mk4116-16.html

How a bit is physically stored: latches, SRAM cells and bitlines

A processor register and an SRAM array are not mystical 'boxes that remember.' At transistor level, static storage is built from positive feedback: typically cross-coupled inverters, plus access transistors controlled by wordlines. Vertical bitlines carry data into and out of the selected row.

TECH ARTICLEReverse engineering the Intel 386 register cell — Ken Shirriff

Exceptional explanation of real static storage. Starts with two cross-coupled inverters, builds the standard 6-transistor SRAM cell and an 8-transistor two-port register cell, then maps those circuits onto the actual 386 silicon layout.

https://www.righto.com/2023/11/reverse-engineering-intel-386.html

DIRECT PDFSRAM design — David Harris (direct PDF)

Public slides on SRAM arrays, cells, wordlines/bitlines, read and write operation, precharge, sensing and the surrounding circuitry needed to turn individual bistable cells into a usable memory.

https://pages.hmc.edu/harris/class/e158/16/lect19.pdf

Before semiconductor RAM: delay lines, CRT memory, drums and magnetic core

Early computer designers understood registers and addressable storage before they had a cheap semiconductor RAM chip. They therefore built memory out of sound pulses, electrostatic charge on CRTs, rotating magnetic surfaces and tiny magnetic rings. These technologies are worth studying because each makes storage, addressing, refresh and access time physically visible.

Memory technology	How one stores information	Important consequence
Mercury delay line	Bits are acoustic pulses travelling through a tube of mercury and continuously recirculated.	Serial access: where the desired bit is physically located in the travelling pulse train affects when it becomes available.
Williams-Kilburn CRT tube	Electron beam writes charge patterns on a CRT surface; pickup circuitry senses charge.	Random access, but charge leaks and must be regenerated/refreshed.
Magnetic drum	Magnetic regions on a rotating cylinder pass fixed read/write heads.	Rotational position creates latency; drums served as both memory and secondary storage.
Magnetic core	Tiny ferrite toroid magnetization direction stores a bit; intersecting wires select/read/write cores.	Random access, nonvolatile, reliable; destructive reads often require restoring the bit.
SRAM	Cross-coupled transistor feedback stores a stable logic state while powered.	Fast random access; larger cell area than DRAM.
DRAM	Charge on a tiny capacitor represents a bit and must be sensed/restored.	Very dense; needs refresh and elaborate row/bank timing.

1940s storage problem ├── circulating pulses → acoustic delay-line memory ├── surface charge → Williams/Kilburn CRT memory ├── rotating magnetic coating → drum memory └── magnetic ferrite rings + selection wires → core memory ↓ dominant main-memory technology through much of 1950s–early 1970s ↓ semiconductor SRAM / DRAM integrated circuits

MUSEUM EXPLAINERComputer History Museum — memory technology overview

Excellent compact historical overview: mercury acoustic delay lines, CRT charge storage, magnetic drums, core memory and later integrated-circuit memory.

https://www.computerhistory.org/brochures/memory/

MUSEUM EXPLAINEREDSAC mercury delay-line memory

Explains the physical loop: electronic pulse → piezoelectric transducer → acoustic pulse through mercury → receiving transducer → amplification → recirculation. EDSAC used this as main memory.

https://www.computerhistory.org/storageengine/edsac-computer-employs-delay-line-storage/

MUSEUM EXPLAINERWilliams-Kilburn CRT memory

Shows how CRT-screen charge became a random-access memory, why it leaked and why stored bits needed continuous rewriting.

https://www.computerhistory.org/storageengine/williams-demonstrates-crt-storage/

MUSEUM EXPLAINERMagnetic drum storage

Explains rotating magnetic drums, fixed read/write heads and their use as early immediate-access memory and storage.

https://www.computerhistory.org/storageengine/tauschek-patents-magnetic-drum-storage/

MUSEUM EXPLAINERWhirlwind debuts magnetic-core memory

Clear explanation of ferrite-core arrays and coincident selection wires. Useful bridge from magnetic material to addressable binary RAM.

https://www.computerhistory.org/storageengine/whirlwind-computer-debuts-core-memory/

PRIMARY ARCHIVEMIT Project Whirlwind technical report archive

Large public archive of original engineering reports covering Whirlwind circuitry, storage, input/output and the development of magnetic-core memory.

https://dome.mit.edu/handle/1721.3/37456

PRIMARY ARCHIVEWhirlwind: 16×16 metallic-core memory report index

The archive contains the 1952 report 'The 16 x 16 Metallic-Core Memory Array, Model 1', including development history and technical details of early coincident-current core memory.

https://dome.mit.edu/handle/1721.3/37455/browse?type=title

DIRECT PDFWhirlwind physical artifact — technical paper (direct PDF)

Detailed engineering treatment of the machine as a physical object: room-scale racks, bit-slice organization, storage racks, power equipment and hardware layout.

https://annals-extras.org/ww/Whirlwind-The-Physical-Artifact.pdf

DRAM is not a byte array electrically: channels, ranks, banks, rows, columns and timing

Software sees memory as addresses. DRAM hardware is organized hierarchically and must be commanded through stateful operations. A memory controller translates queued CPU requests into legal DRAM command sequences while trying to exploit bank-level parallelism and open rows.

CPU load/store request ↓ integrated memory controller ↓ memory CHANNEL ↓ DIMM / soldered DRAM devices ↓ RANK (set of chips responding together) ↓ BANK GROUP → BANK ↓ ROW of DRAM cells ──→ sense amplifiers / ROW BUFFER ↓ COLUMN selection ──→ burst of DQ data on memory bus

Operation / timing	Meaning
PRECHARGE	Closes/prepares a bank so another row can be activated; bitlines are returned to their precharge state.
ACTIVATE	Selects a row and connects its cells to bitlines/sense amplifiers, effectively opening the row in that bank.
READ / WRITE	Column command transferring a selected part of the open row through the DRAM I/O interface.
REFRESH	Periodically restores charge because DRAM cells leak; the controller/device must reserve time for this.
tRCD	Minimum row-to-column delay: ACTIVATE → READ/WRITE.
CL / read latency	Delay associated with a READ command before read data is returned; the exact definition varies with DRAM generation/mode.
tRP	Row-precharge time: delay needed after PRECHARGE before a new ACTIVATE to that bank.
tRAS	Minimum time a row must remain active before it can be precharged.
tRC	Row-cycle time; commonly tied to activate-to-activate timing for different rows in the same bank.
row hit	Requested data is in the row already open in that bank; avoids another activate/precharge sequence.
row conflict	A different row is open, so the controller must close it and activate the requested row.

Important: timings printed as numbers such as CL are often expressed in clock cycles, while the time you experience depends on the actual clock period. Bandwidth and latency therefore need to be reasoned about separately.

DIRECT PDFUniversity of Illinois — DRAM organization (direct PDF)

Public lecture PDF with the hierarchy DIMM → rank → bank → array → row buffer. Good concise visual bridge between a memory module and the cells inside it.

https://courses.grainger.illinois.edu/CS433/sp2025/slides/chapter2-part3-post-lecture.pdf

OFFICIAL DOCSAMD/Xilinx — Bank Machines and DRAM command scheduling

Public hardware documentation that explicitly describes ACTIVATE, column READ/WRITE, PRECHARGE, open-row reuse and command reordering inside a real memory controller.

https://docs.amd.com/r/en-US/ug586_7Series_MIS/Bank-Machines

OFFICIAL DOCSIntel Core Ultra memory controller

Current public Intel documentation stating that the integrated memory controller transfers data between processor and DRAM and performs DRAM maintenance, with multiple DDR5 channels.

https://edc.intel.com/content/www/us/en/design/products/platforms/details/arrow-lake-s/core-ultra-200s-series-processors-datasheet-volume-1-of-2/memory-controller-mc/

DIRECT PDFDDR5 architecture summary (PDF)

Compact public PDF showing DDR5 banks/bank groups, burst length, on-die ECC, termination, training and the relationship between clock rate, MT/s and theoretical bandwidth.

https://media.kingston.com/kingston/articles/MKF_954-DDR5-Collateral_us.pdf

FREE PDFWhat Every Programmer Should Know About Memory (PDF)

Older but exceptionally detailed public paper on DRAM, caches, memory controllers, NUMA and the software-visible consequences of the memory hierarchy. Some specific platform details are dated; the physical concepts remain valuable.

https://www.akkadia.org/drepper/cpumemory.pdf

The DRAM controller reorders requests to exploit open rows, bank parallelism and bus direction

A memory controller does not necessarily service CPU misses in simple arrival order. It has queues of pending reads/writes and tracks which row is open in every bank. The scheduler can prefer commands that are legal now, hit an already-open row, use another bank while one bank waits on timing, or group reads/writes to reduce costly data-bus turnarounds.

Suppose bank 0 currently has ROW 12 open. pending requests: A: READ bank0 row19 col2 ← ROW CONFLICT B: READ bank0 row12 col8 ← ROW HIT C: READ bank1 row7 col1 ← other bank Strict arrival order would choose A first: PRECHARGE bank0 row12 wait tRP ACTIVATE bank0 row19 wait tRCD READ col2 A locality-aware scheduler can often choose B first: READ bank0 col8 immediately when timing allows while also using bank1 for C when its command timing permits then later close row12 / activate row19 for A CONTROLLER STATE TRACKS per-bank open row ACT/PRE/READ/WRITE timing constraints refresh deadlines read queue / write queue request age / QoS / starvation limits bank-group restrictions data-bus read↔write turnaround Goal is not merely minimum latency for one request: maximize useful bandwidth while preserving ordering/QoS and never violating DRAM timing.

Term	Effect
row hit	Requested row is already active in target bank; can proceed to column READ/WRITE when timing permits.
row empty	No row active; needs ACTIVATE then tRCD before column command.
row conflict	Different row active in same bank; requires PRECHARGE + tRP + ACTIVATE + tRCD before access.
bank-level parallelism	Different banks can have different rows active and overlap portions of command latency.
bank group	DDR4/DDR5-style grouping introduces additional timing constraints depending on whether consecutive commands target same/different groups.
open-page policy	Leaves a row active hoping for future hits.
close-page/autoprecharge	Closes row after access when future locality is unlikely.
request reordering	Selects a ready/locality-efficient request rather than blindly following arrival order.
aging/starvation control	Raises priority of old requests so locality optimization cannot postpone them indefinitely.
read/write batching	Groups same-direction transfers to avoid frequent DRAM data-bus turnaround penalties.
refresh	Periodically blocks/limits ordinary access so DRAM cells can be restored before charge leaks away.

Address mapping changes performance. Which physical-address bits become row, bank, bank-group and column bits determines whether sequential or multi-threaded access patterns create row hits, bank parallelism or conflicts. The memory controller and OS/allocator usually hide this mapping from ordinary programs, but the physical consequences are real.

OFFICIAL DOCSAMD Zynq 7000 — DDR Core and Transaction Scheduler (2026)

Current public TRM explicitly describes pending read/write queues and a scheduler that selects transactions for the DDR PHY.

https://docs.amd.com/r/en-US/ug585-zynq-7000-SoC-TRM/DDR-Core-and-Transaction-Scheduler-DDRC

OFFICIAL DOCSAMD Zynq DDR controller — arbitration, page match, aging, write combine

Current 2026 documentation exposes the controller's real decision machinery: arbitration stages, page-match preference, aging counters, read priority and write combining.

https://docs.amd.com/r/en-US/ug585-zynq-7000-SoC-TRM/DDR-Controller

OFFICIAL DOCSAMD Versal DDR4 — open-page/autoprecharge policy

Public documentation stating that the controller defaults to an open-page policy and closes rows for refresh, page misses or explicit autoprecharge.

https://docs.amd.com/r/en-US/pg353-versal-acap-soft-ddr4-mem-ip/Autoprecharge

OFFICIAL DOCSAMD Versal NoC — DRAM addressing and bank groups

Current public explanation of rows, banks, bank groups, concurrent activate/precharge opportunities and same-bank-group timing penalties.

https://docs.amd.com/r/en-US/pg313-network-on-chip/DRAM-Addressing

OFFICIAL DOCSAMD Versal NoC — address mapping efficiency

Concrete 2026 example showing that mapping concurrent streams onto separate banks can dramatically change measured DRAM efficiency.

https://docs.amd.com/r/en-US/pg313-network-on-chip/Multi-Thread-Linear-Read/Write

DRAM forgets unless refreshed: retention, tREFI/tRFC, self-refresh and row-disturbance defense

A DRAM cell stores charge on a tiny capacitor-like structure and leaks over time. Memory controllers therefore issue periodic refresh operations that restore cell charge before data becomes unreliable. Refresh is not free: banks/ranks become temporarily unavailable, so real controllers schedule refresh around ordinary traffic and can vary refresh policy with memory type and temperature.

WHY REFRESH EXISTS DRAM cell stores charge representing bit ↓ leakage with time / temperature sense margin gradually shrinks ↓ controller must issue refresh often enough ↓ DRAM internally activates/restores rows according to refresh command TWO IMPORTANT TIMING IDEAS tREFI = average interval/budget between required refresh opportunities tRFC = time a refresh operation occupies the affected DRAM resources ALL-BANK REFRESH precharge required banks REF command all targeted banks unavailable during refresh window PER-BANK / SAME-BANK FINE-GRANULARITY OPTIONS (generation-dependent) refresh one bank/limited resource other banks may continue useful work ↓ can reduce visible stall but introduces more scheduling/timing complexity SELF-REFRESH controller places DRAM into low-power self-refresh mode DRAM maintains its own refresh internally host/controller clocks can often be reduced/stopped ↓ exit self-refresh → restore normal command scheduling ROW DISTURBANCE / ROWHAMMER CONCEPT repeated ACTIVATE/PRECHARGE activity to a physical DRAM row ↓ electrical disturbance/coupling in susceptible devices neighboring/victim row retention margin can degrade ↓ bit flip may occur before ordinary refresh would have corrected retention loss DEFENSE IN DEPTH DRAM-internal row-disturbance protection / TRR-like logic memory-controller refresh management / targeted refresh temperature-appropriate refresh rate ECC / stronger ECC for residual errors firmware memory-reference-code updates/configuration validated DIMMs/timings; avoid unsupported reduced-margin overclocking OS/hypervisor isolation/monitoring where appropriate No single mitigation should be assumed to eliminate all Rowhammer variants.

Refresh/reliability term	Meaning
retention time	How long a DRAM cell can preserve enough charge to be read correctly under specified conditions before refresh/restoration.
tREFI	Refresh-interval timing parameter/budget controlling how frequently required refresh commands are distributed.
tRFC	Refresh-cycle time during which affected DRAM resources are unavailable for normal accesses.
all-bank refresh	Refresh operation blocking all banks in the targeted rank/device organization for its refresh-cycle interval.
per-bank refresh	Refreshes a selected bank while potentially allowing other banks to continue accesses, where supported.
fine-granularity refresh	Memory-generation-specific modes trading more frequent shorter refresh operations against less frequent longer ones.
self-refresh	Low-power DRAM state in which the memory device maintains refresh internally.
temperature-compensated refresh	Adjusting refresh behavior because high temperature can reduce cell retention margin.
row disturbance	Electrical interference where repeated row activation can perturb data stored in nearby rows.
Rowhammer	DRAM disturbance phenomenon in which repeated activation of aggressor rows can induce bit flips in victim rows on susceptible memory.
TRR-like mitigation	Target-row-refresh family of mechanisms that attempts to identify heavily activated aggressors and refresh likely victim rows.
RFM / ARFM / DRFM	Modern refresh-management mechanisms coordinating memory-controller/device responses to high activation/disturbance risk; details vary by DDR generation/platform.
on-die ECC	ECC inside a DRAM device; useful for device reliability but not equivalent to end-to-end/system-level ECC visibility/protection.
system ECC	Memory-controller-visible ECC protecting data across the external memory interface and exposing corrected/uncorrected error information to platform RAS.

Refresh is both a reliability mechanism and a performance event. Current DDR controllers explicitly schedule refresh automatically to maintain data integrity, while newer controllers expose all-bank/per-bank/fine-granularity choices because refresh steals command/bank availability from ordinary traffic.

CURRENT OFFICIAL DOCSAMD Integrated DDR5/LPDDR5 Memory Controller — Refresh (Aug. 2026)

Current August 2026 controller documentation: refresh is issued automatically to maintain DRAM data integrity; DDR5 and LPDDR5/5X support several refresh granularities and temperature-driven interval adjustment.

https://docs.amd.com/r/en-US/pg456-integrated-mc/Refresh

CURRENT OFFICIAL DOCSAMD LPDDR5/5X refresh options — all-bank vs per-bank

Concrete current timing/performance example: all-bank refresh makes all banks unavailable during tRFC, while per-bank refresh can leave other banks available.

https://docs.amd.com/r/en-US/pg456-integrated-mc/LPDDR5/5X-Refresh-Options

CURRENT OFFICIAL DOCSAMD Zynq/MPSoC DDR controller — self refresh

Current July 2026 description of self-refresh and power-down: DRAM can retain contents while controller-side clocks/power are reduced.

https://docs.amd.com/r/en-US/ug1137-zynq-ultrascale-mpsoc-swdev/DDR-Controller

CURRENT SECURITY GUIDANCEIntel — Reducing Exposure to Rowhammer (July 2026)

Current defense-in-depth guidance explaining row-disturbance bit flips and layered mitigation across DRAM, memory controller, firmware and operations; explicitly warns against relying on one mitigation.

https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/best-practices/reducing-exposure-to-rowhammer.html

VENDOR BULLETINAMD — Phoenix DDR5 Rowhammer response

Vendor example showing that Rowhammer resilience can require platform-initialization/BIOS updates and altered refresh behavior rather than an OS-only fix.

https://www.amd.com/en/resources/product-security/bulletin/amd-sb-7048.html

KERNEL DOCSLinux RAS documentation

Connects any residual memory bit errors that escape prevention to the existing ECC/CE/UE reporting and recovery layers.

https://docs.kernel.org/6.18/admin-guide/RAS/main.html

DDR PHY training: why a memory controller cannot just 'start reading RAM'

At DDR data rates, PCB/package skew is comparable to the valid timing window. The memory controller therefore includes a PHY with programmable delay elements, VREF controls and calibration logic. During boot it sends known patterns and adjusts timing so DQS/data sampling lands inside a usable eye for each byte lane—or even each bit.

firmware configures controller/PHY with DRAM geometry + timing ↓ JEDEC-style DRAM initialization / mode-register programming ↓ WRITE LEVELING adjust outgoing DQS relative to forwarded CK / board skew ↓ READ DQS GATE TRAINING find when returning DQS burst actually arrives ↓ READ PER-BIT DESKEW / READ EYE CENTERING move sampling point away from data-eye edges ↓ WRITE DESKEW / WRITE EYE CENTERING / VREF training (generation dependent) ↓ PHY reports calibrated / ready ↓ normal memory traffic can begin reliably

Training operation	What is being adjusted/measured
write leveling	Delay outgoing DQS so it aligns correctly with CK at each DRAM device despite fly-by/package/PCB skew.
read-gate training	Find the time window in which returning read DQS is present so receiver gating opens at the correct time.
per-bit deskew	Compensate small arrival-time differences between DQ bits within a lane.
read-eye centering	Move DQS/sample point toward the center of the valid incoming DQ eye.
write-eye centering	Adjust outgoing DQ/DQS timing for maximum receiver margin at the DRAM.
VREF training	Tune receiver reference voltage so HIGH/LOW decision threshold sits near the best vertical eye opening.
VT tracking	Compensate later variation due to voltage/temperature/process conditions after initial training.

Why DRAM initialization is firmware-visible: before training succeeds, external DRAM may be unusable as ordinary stack/heap memory. That is why boot ROM/SPL/romstage code often runs from ROM, cache-as-RAM, SRAM or another temporary environment first.

OFFICIAL DOCSAMD Zynq 7000 TRM — DRAM training (2026)

Current 2026 public hardware documentation spelling out write leveling, read DQS gate training and read-data eye training, including that training compensates board delay/skew.

https://docs.amd.com/r/en-US/ug585-zynq-7000-SoC-TRM/DRAM-Training

OFFICIAL DOCSAMD Zynq — write leveling in detail

Explains the physical feedback loop: the controller shifts DQS relative to CK, the DRAM samples CK with DQS and feeds back the observed transition on DQ.

https://docs.amd.com/r/en-US/ug585-zynq-7000-SoC-TRM/Write-Leveling

OFFICIAL DOCSAMD DDR4 PHY — read-leveling calibration overview

Public description of per-bit DQ deskew and DQS centering using programmable delay elements with picosecond-scale taps.

https://docs.amd.com/r/en-US/pg353-versal-acap-soft-ddr4-mem-ip/Read-Leveling-Calibration-Overview

OFFICIAL DOCSAMD DDR4 PHY architecture

Exposes the actual PHY delay-control machinery behind training rather than treating 'memory controller' as one opaque block.

https://docs.amd.com/r/en-US/pg353-versal-acap-soft-ddr4-mem-ip/Overall-PHY-Architecture

ECC memory: detecting and correcting bit errors

Memory bits can be corrupted by electrical faults, radiation-induced upsets, marginal timing, failing cells or signal problems. Error-correcting memory stores redundant check information so the memory controller can detect—and for certain patterns, correct—corruption.

Mechanism	What it protects / can do
Parity	Adds a check bit that can detect some error patterns, but ordinarily cannot identify which bit to correct.
SECDED-style ECC	Common system-memory scheme: Single Error Correct, Double Error Detect for a protected codeword.
Side-band ECC	Extra DRAM/check bits travel alongside normal data and protect the memory path/codeword seen by the memory controller.
Inline ECC	ECC metadata is stored inside part of the normal memory-address space rather than on a physically wider side-band interface.
DDR5 on-die ECC	Internal to each DDR5 DRAM chip. It improves reliability inside the chip but is not equivalent to end-to-end/system ECC across the DIMM/channel.
Scrubbing	Controller periodically reads memory, corrects correctable errors, and may write repaired data back before errors accumulate.

OFFICIAL DOCSAMD integrated DDR5 controller — ECC

Current public hardware documentation describing side-band versus inline ECC, correction/detection behavior, error logging and latency costs in a real DDR5 controller.

https://docs.amd.com/r/en-US/pg456-integrated-mc/ECC

OFFICIAL DOCSAMD/Xilinx — classic SECDED ECC block

Public documentation explicitly describing Single Error Correct, Double Error Detect behavior and the ECC encoder/decoder around a DRAM controller.

https://docs.amd.com/r/en-US/ug586_7Series_MIS/Error-Correcting-Code

KERNEL DOCSLinux EDAC documentation

Shows the operating-system side: corrected, uncorrected, deferred and fatal memory-controller errors as reported by real hardware.

https://docs.kernel.org/driver-api/edac.html

PLAIN WEBDDR5 on-die ECC versus module/system ECC — Kingston

Useful clarification that DDR5 on-die ECC is internal to DRAM chips and does not replace the additional end-to-end ECC used by server/workstation memory platforms.

https://www.kingston.com/en/blog/pc-performance/ddr5-overview

From one flipped DRAM bit to an OS error report: ECC, syndrome, scrubbing, machine checks and page poisoning

ECC memory is only the first layer of a Reliability, Availability and Serviceability (RAS) path. The memory controller generates/checks ECC code bits, classifies errors as corrected or uncorrected, records location/syndrome information, and the CPU/firmware/kernel decides whether to log, scrub, retire a page, signal a process, reset a component or stop the machine.

WRITE TO ECC-PROTECTED MEMORY 64 data bits (illustrative common width) ↓ ECC encoder in memory controller data + check bits / syndrome codeword ↓ DRAM stores codeword LATER: ONE STORED BIT CHANGES cosmic particle / electrical disturbance / failing cell / link fault / etc. READ DRAM returns data + ECC bits ↓ ECC checker recomputes syndrome ↓ SECDED-LIKE EXAMPLE no error → deliver data normally single-bit error → identify/correct bit → deliver corrected data → CE report/counter double-bit error → detect but cannot correct → UE / machine-check/RAS path CORRECTED ERROR PATH controller may record channel/rank/DIMM/address/syndrome ↓ EDAC / firmware / machine-check infrastructure logs event ↓ background patrol scrub may later read/correct/rewrite memory repeated CEs can trigger maintenance thresholds/repair policy UNCORRECTABLE ERROR PATH (platform-dependent) hardware reports poisoned/corrupt physical address ↓ kernel identifies affected physical page / mapping ↓ recoverable case: mark page HWPoison, unmap/isolate, notify affected task (e.g. SIGBUS) fatal/unrecoverable context: machine check may panic/reset The exact response depends on CPU, memory controller, firmware, kernel and where the bad data was consumed.

RAS term	Meaning
ECC syndrome	Pattern produced by parity/check equations indicating whether/where a protected codeword is inconsistent.
CE	Corrected Error: hardware was able to recover the intended data under the implemented ECC scheme.
UE	Uncorrected/Uncorrectable Error: detected corruption beyond the code's correction capability or otherwise not safely correctable.
SECDED	Single-Error Correction, Double-Error Detection—common Hamming-derived ECC capability; actual server schemes can be stronger.
Chipkill-like protection	Stronger memory-protection organization intended to survive failures larger than one individual bit; implementation-specific.
patrol scrub	Background reads of memory that cause ECC checking and correction/rewrite before latent errors accumulate.
EDAC	Linux Error Detection And Correction subsystem for collecting/reporting memory/cache/interconnect hardware errors.
MCA / machine check	CPU hardware-error reporting architecture; exact banks/register semantics are processor-specific.
RAS daemon	Userspace service such as rasdaemon collecting/decoding kernel hardware-error trace events.
HWPoison	Linux VM state marking a physical page as corrupted so it can be isolated and mappings/users handled.
page offlining	Removing a faulty physical page from future allocation/use.
syndrome/location data	Controller/CPU metadata used to identify failing bit/channel/rank/DIMM/address where the platform can provide it.

Corrected does not mean irrelevant. The CPU may receive correct data and continue normally, but a rising corrected-error rate can be an early warning of a degrading DIMM/channel/device. Linux RAS/EDAC infrastructure therefore tracks CEs separately from UEs and exposes them for monitoring/maintenance.

LINUX OBSERVATION LAB (hardware/driver support varies) # EDAC devices/counters find /sys/devices/system/edac -maxdepth 4 -type f 2>/dev/null | sort | less # kernel RAS / machine-check messages dmesg | grep -Ei 'EDAC|MCE|machine check|hardware error|corrected|uncorrected' | less # rasdaemon if installed rasdaemon --status # inspect memory-controller devices ls -R /sys/devices/system/edac/mc 2>/dev/null | less # Do NOT use error-injection interfaces on a useful machine merely to experiment. # Injected UE/MCE paths are intentionally capable of crashing systems or killing processes.

KERNEL DOCSLinux EDAC API — current development documentation

Real EDAC structures carry CE/UE severity, page-frame number, offset, syndrome and memory-controller topology/location information.

https://docs.kernel.org/next/driver-api/edac.html

KERNEL DOCSLinux x86 machine-check documentation

Describes CPU machine-check banks and the distinction between corrected error log entries and uncorrected machine-check conditions.

https://docs.kernel.org/arch/x86/x86_64/machinecheck.html

KERNEL DOCSLinux RAS error decoding

Current guidance for decoding x86 hardware errors; AMD systems use rasdaemon for SMCA decoding.

https://docs.kernel.org/admin-guide/RAS/error-decoding.html

KERNEL DOCSLinux HWPoison

Shows the VM recovery layer for corrupted physical pages: poison/isolate pages and deal with affected page-cache or process mappings after hardware reports unrecoverable memory corruption.

https://docs.kernel.org/mm/hwpoison.html

KERNEL DOCSLinux EDAC scrub control

Current background/on-demand scrubbing controls, including ECC checking/correction and threshold-oriented RAS behavior for modern memory devices.

https://docs.kernel.org/edac/scrub.html

APEI/GHES adds firmware to the hardware-error path: error source → CPER record → Linux RAS handling

Machine checks, ECC controllers and PCIe AER can report hardware failures directly, but some platforms route or enrich errors through firmware. ACPI's Platform Error Interfaces (APEI) define standard tables and records for that path. GHES (Generic Hardware Error Source) is one APEI mechanism: firmware/platform logic places a structured error status block in memory and notifies the OS, which consumes a CPER-formatted record and feeds normal Linux RAS/logging/recovery machinery.

HARDWARE DETECTS AN ERROR CPU / memory controller / PCIe / platform RAS logic ↓ ACPI HEST describes error sources and ownership ↓ DIRECT OS-FIRST path? ── yes → native MCA/AER/etc. handler │ no / FIRMWARE_FIRST or generic source ↓ platform firmware / RAS controller handles first ↓ GHES error-status block in memory ↓ standardized Generic Error Data / CPER-style sections OS notification: SCI / NMI / interrupt / polling / platform mechanism ↓ Linux GHES/APEI copies and decodes record ├── severity: corrected / recoverable / fatal / informational ├── processor / memory / PCIe / vendor-defined section └── physical address / FRU / syndrome-like metadata where available ↓ printk / trace / EDAC / memory-failure / PCIe recovery path as applicable ↓ log only | page poison/offline | device recovery | process SIGBUS | panic/reset OTHER APEI PIECES BERT → boot-time record left by previous/early failure ERST → firmware interface for persistent error records EINJ → controlled error injection for validation/testing

APEI object	Role
HEST	Hardware Error Source Table: tells the OS what error sources exist and how they are signaled/handled, including firmware-first cases.
GHES/GHESv2	Generic Hardware Error Source structures describing a memory-resident error status block and notification/acknowledgement mechanism.
CPER	Common Platform Error Record format used to represent structured processor, memory, PCIe and other hardware-error data across firmware/OS boundaries.
BERT	Boot Error Record Table: points at error data preserved for the OS to consume during boot.
ERST	Error Record Serialization Table: standard firmware interface for persistent platform error-record storage.
EINJ	Error Injection Table/interface used to test RAS paths. Deliberately capable of provoking serious hardware-error handling.
FIRMWARE_FIRST	HEST policy bit indicating firmware gets control first for that error source before reporting standardized information onward.

APEI does not perform ECC and GHES does not replace machine checks or AER. It standardizes how platform firmware and the OS describe, route, persist and sometimes enrich hardware-error information. The physical detector/corrector is still in the processor, memory controller, link/device or other hardware.

OFFICIAL SPECACPI 6.6 — Platform Error Interfaces (APEI)

Normative APEI chapter defining HEST, BERT, ERST, EINJ, GHES/GHESv2, firmware-first handling and the relationship to Common Platform Error Records.

https://uefi.org/specs/ACPI/6.6/18_Platform_Error_Interfaces.html

KERNEL DOCSLinux — APEI error output format

Shows the concrete Linux-visible generic hardware-error record fields, including severity and processor/memory/PCIe section data.

https://docs.kernel.org/firmware-guide/acpi/apei/output_format.html

KERNEL DOCSLinux — APEI EINJ error injection

Public documentation for the firmware-backed error-injection interface used to validate RAS handling. Useful for understanding the mechanism; do not inject fatal errors on a useful machine.

https://docs.kernel.org/firmware-guide/acpi/apei/einj.html

What if a bit flips on the way? Parity, CRC, ECC, retry and end-to-end integrity

Digital logic assumes discrete values, but the physical world is noisy. High-speed links and memories therefore add redundant information so corruption can be detected or corrected. Different mechanisms solve different problems: a CRC usually detects corrupted transmission; ECC can identify/correct certain bit errors; a replay protocol retransmits a damaged packet; end-to-end integrity protects a transaction across several internal stages rather than only one wire segment.

DETECTION ONLY payload → CRC/FCS generator → payload + check bits → noisy channel ↓ receiver recomputes check → mismatch → corrupted packet detected LINK WITH RETRY sender stores unacknowledged packet in replay buffer ↓ transmit + sequence/LCRC receiver validates ├── good → ACK → sender discards replay copy └── bad/missing → NAK/timeout → sender retransmits ECC MEMORY data + check bits → decoder/syndrome ├── no error ├── correctable pattern → repair + report └── uncorrectable pattern → report/fault END-TO-END ON-CHIP INTEGRITY transaction fields + integrity code travel through crossbar/adapters ↓ destination checks that address/control/data were not silently corrupted

Mechanism	Typical capability	What it does not automatically guarantee
parity	Detects all odd-numbered bit flips in protected word; very cheap.	Correction or detection of every multi-bit pattern.
CRC / FCS	Strong burst-error detection for transmitted frames/packets.	Correction; receiver usually drops/retries or reports error.
SECDED ECC	Corrects one bit and detects two-bit errors per protected codeword.	Protection from all larger error patterns.
link replay/retry	Recovers transient transmission corruption by retransmitting.	Protection from faulty data that was already wrong before checksum generation.
end-to-end integrity	Carries/checks protection across several fabric stages.	Full semantic correctness of the requester/device or software.
timeout/watchdog	Detects transaction that never completes.	Identifying whether root cause was dead hardware, lost request, clock/reset fault, etc.

OFFICIAL DOCSAMD PCIe — Data Link Layer reliability

Current public PCIe implementation documentation: the Data Link Layer provides reliable TLP exchange, error detection/recovery, DLLPs, acknowledgments and replay buffers.

https://docs.amd.com/r/en-US/pg346-cpm-pcie/Data-Link-Layer

PUBLIC DOCSOpenTitan TL-UL bus specification

OpenTitan's on-chip fabric adds command/response integrity metadata to TL-UL and routes it with transactions through the interconnect.

https://opentitan.org/book/hw/ip/tlul/index.html

PUBLIC DOCSOpenTitan SPI host — end-to-end bus integrity

Concrete peripheral example declaring TL-UL bus integrity as a security countermeasure and raising a fatal alert on detected bus-integrity faults.

https://opentitan.org/book/hw/ip/spi_host/doc/interfaces.html

PUBLIC DOCSOpenTitan TL-UL protocol checker

Shows both protocol correctness assertions and integrity/error checks around requests and responses.

https://opentitan.org/earlgrey_1.0.0/book/hw/ip/tlul/doc/TlulProtocolChecker.html

ROM, EPROM, EEPROM and flash: how bits survive when power is removed

SRAM and DRAM forget when power disappears. Nonvolatile semiconductor memories encode information in structures that remain physically different without power. Classic EPROM and much flash technology use a floating-gate MOS transistor: electrons trapped on an insulated gate shift the transistor's threshold voltage, and the read circuitry senses that difference.

DIRECT PDFKIOXIA — NOR versus NAND flash (PDF)

Explains the architectural difference: NOR cells connected for fast random reads versus NAND strings optimized for density, plus why boot/code storage and bulk flash behave differently.

https://americas.kioxia.com/content/dam/kioxia/en-us/business/memory/slc-nand/asset/KIOXIA_NOR_to_NAND_Tech_Brief.pdf

TECH ARTICLEIBM — EEPROM mechanism

Accessible technical overview of floating gate, control gate, oxide insulation, threshold shift and electrical erase/write behavior.

https://www.ibm.com/think/topics/eeprom

Cache anatomy: tag, index, offset, ways, dirty bits and miss types

A cache does not search all of RAM every time. The address is split into fields that select a candidate set, identify which memory block is present, and select the byte within that block. In a typical set-associative cache, the index chooses a set, several ways are checked in parallel, and each way compares its stored tag against the address tag.

EXAMPLE: 32 KiB cache, 64-byte lines, 8-way set associative, 64-bit addresses data capacity = 32 KiB = 32768 bytes line size = 64 bytes = 2^6 → OFFSET = 6 bits number of lines = 32768 / 64 = 512 number of sets = 512 / 8 ways = 64 = 2^6 → INDEX = 6 bits TAG bits = 64 - 6 - 6 = 52 bits 64-bit byte address: +--------------------------------------------------+------+------+ | TAG (52) |INDEX |OFFSET| +--------------------------------------------------+------+------+ 6 6 lookup: INDEX → select one of 64 sets → compare requested TAG against 8 stored tags in parallel → one valid tag matches? HIT → use OFFSET to select byte(s) inside 64-byte cache line → no match? MISS → obtain line from next hierarchy level

Cache field/state	Purpose
valid bit	Says whether a cache-line slot contains meaningful data/tag state.
tag	Identifies which memory block currently occupies a selected cache way.
index	Selects one set in a direct-mapped or set-associative cache.
offset	Selects byte/word position inside the cache line.
dirty bit	In a write-back cache, records that cached data differs from lower memory and must be written back before eviction.
replacement state	Chooses a victim way on a miss: true/pseudo LRU, FIFO, random or another policy.
write-through	A store updates cache and lower memory immediately.
write-back	A store updates cache; lower memory is updated later when a dirty line is evicted.
write-allocate	On a store miss, fetch/allocate the line into cache before updating it.
no-write-allocate	On a store miss, send the write toward lower memory without allocating that line.

Three classic miss causes: compulsory/cold misses happen on first access; capacity misses happen because the working set exceeds cache capacity; conflict misses happen because multiple useful blocks compete for the same set even when other cache space exists.

PUBLIC NOTESCornell CS3410 (2026) — caches

Excellent current public notes covering direct-mapped, fully associative and set-associative caches, replacement, write-through/write-back and write allocation.

https://www.cs.cornell.edu/courses/cs3410/2026sp/notes/caches.html

PUBLIC NOTESCS61C — direct-mapped cache with tag/index/offset

Plain no-login walkthrough showing exactly how an address is split into tag, index and block offset and how hit/miss/replacement works.

https://notes.cs61c.org/content/caches-ii/direct-mapped/

PUBLIC NOTESCS61C — fully associative cache, dirty bits and write policy

Clear explanation of tag matching, valid/dirty state, write-through versus write-back and replacement policies.

https://notes.cs61c.org/content/caches-ii/fully-associative/

PUBLIC NOTESCS61C — set-associative caches

Shows sets, ways and the balance between a cheap direct-mapped lookup and an expensive fully associative search.

https://notes.cs61c.org/content/caches-ii/set-associative/

DIRECT PDFMIT 6.004 cache worksheet — worked address-field exercises (direct PDF)

Direct PDF with concrete exercises on 2-way set-associative write-back caches, LRU, address tag/index/offset bits and average memory-access time.

https://ocw.mit.edu/courses/6-004-computation-structures-spring-2017/e3780374f88b6201126fccda717635f7_caches_worksheet.pdf

VIPT L1 caches hide TLB latency by indexing the cache before translation finishes

A load begins with a virtual address, but coherent cache tags normally need physical identity. A Virtually Indexed, Physically Tagged (VIPT) L1 cache exploits one crucial fact: page-offset bits are unchanged by address translation. It can use those low virtual bits to start the SRAM lookup at the same time as the TLB lookup, then compare the resulting cache tags against the translated physical address.

EXAMPLE L1 DATA CACHE capacity = 32 KiB associativity = 8 ways line size = 64 B base page = 4 KiB sets = 32 KiB / (8 × 64 B) = 64 sets line offset bits = log2(64) = 6 set index bits = log2(64) = 6 4 KiB page offset = 12 bits index + line offset = 6 + 6 = 12 bits Virtual address: ┌──────── virtual page number ────────┬─ set ─┬ offset ┐ │ ... │ 6 bit │ 6 bit │ └─────────────────────────────────────┴───────┴────────┘ │ │ │ ├→ index L1 tag/data SRAM immediately ↓ │ DTLB lookup │ ↓ │ physical page number │ └──────┬───────────┘ ↓ physical address/tag compare ↓ cache HIT/MISS Because bits [11:0] are identical in VA and PA for a 4 KiB page, the cache set can be selected before the physical page number is known. COMMON NON-ALIASING VIPT SIZE RULE number_of_sets × line_size ≤ smallest_page_size equivalently: cache_capacity ≤ associativity × smallest_page_size Here: 8 ways × 4 KiB = 32 KiB maximum without using translated bits in the set index.

Cache organization	Index	Tag	Main consequence
PIPT	physical	physical	Simple physical identity but translation must provide index before lookup.
VIVT	virtual	virtual	Very early lookup but suffers homonym/synonym/coherence complications.
VIPT	virtual page-offset bits	physical	Can overlap TLB and cache-array access while retaining physical tag identity.
ASID/PCID-tagged TLB	TLB-specific	virtual translation key + address-space ID	Allows translations from several address spaces to coexist without flushing them all on every context switch.
huge/superpage	more untranslated page-offset bits	physical	Larger page offset can relax the set-index constraint for accesses known to use that page size.

Synonyms are the danger. If set-index bits extend above the unchanged page offset, two different virtual addresses mapping the same physical page could select different cache sets and create multiple cached copies of one physical line. Designs either constrain cache geometry, detect aliases, use software coloring/maintenance, or apply other architecture-specific mechanisms.

SOURCE / FILERocket Chip ICache — real VIPT source

Exceptionally concrete source comments: the Rocket instruction cache is VIPT, accesses tag/data SRAM in parallel, and explicitly notes that with 4 KiB pages and 64 sets × 64-byte lines it uses virtual address bits inside the unchanged page offset.

https://github.com/chipsalliance/rocket-chip/blob/master/src/main/scala/rocket/ICache.scala

PUBLIC DOCSCVA6 MMU documentation

Real open CPU translation path showing virtual-address requests, ITLB/DTLB lookup, ASID/VMID matching, physical page-number generation, shared TLB and hardware page-table walking.

https://cva6.readthedocs.io/en/latest/03_cva6_design/MMU.html

PUBLIC PAPERVESPA — VIPT Enhancements for Superpage Accesses

Architecture paper focused on the fundamental VIPT constraint: parallel TLB/L1 lookup relies on page-offset bits, so associativity rather than number of sets commonly grows as L1 capacity increases.

https://arxiv.org/abs/1701.03499

A TLB miss is a memory transaction too: shared TLBs, page-walk caches and hardware walkers

A virtual-memory lookup does not always end with one TLB check. Real CPUs can have separate L1 instruction/data TLBs, a larger shared translation cache, and dedicated page-table walkers. If all translation caches miss, the walker performs ordinary memory accesses to fetch page-table entries, often benefiting from normal caches and dedicated page-walk caches.

LOAD VA = 0x0000_7f12_3456_7890 ↓ L1 DTLB lookup ├── HIT → PPN + page offset → physical address └── MISS ↓ shared/L2 TLB lookup (if implementation has one) ├── HIT → refill L1 DTLB → continue load └── MISS ↓ PAGE TABLE WALKER (PTW) Sv39-style example: satp.PPN gives root page-table physical page VA provides VPN[2] / VPN[1] / VPN[0] / 12-bit offset walk level 2: PTE address = root_pa + VPN[2] × sizeof(PTE) ↓ memory/cache access for PTE valid non-leaf? → extract next-level PPN walk level 1: next_table_pa + VPN[1] × sizeof(PTE) ↓ fetch PTE walk level 0: next_table_pa + VPN[0] × sizeof(PTE) ↓ fetch leaf PTE ↓ permission / accessed / dirty / privilege checks ↓ valid leaf translation refill shared TLB / L1 DTLB form PA = PPN || page_offset retry/continue original load If a required PTE is invalid or permissions fail: PTW raises PAGE FAULT instead of silently allocating memory ↓ OS fault handler decides whether the access can be repaired PAGE-WALK CACHE IDEA upper-level PTEs are reused by many nearby virtual pages ↓ cache intermediate page-table entries / walk state ↓ later TLB misses may skip some memory references

Translation structure	Typical role
ITLB	Small fast cache of instruction-fetch translations.
DTLB	Small fast cache of load/store translations.
shared/L2 TLB	Larger translation cache serving L1 ITLB/DTLB misses before a full walk.
superpage TLB entry	Translation covering a larger page; reduces pressure because one entry maps more bytes.
page-walk cache	Caches intermediate page-table entries or walk-derived state to reduce repeated upper-level PTE fetches.
hardware PTW	State machine that calculates PTE addresses, fetches entries, checks leaf/nonleaf rules and refills TLBs.
software-filled TLB	Some architectures/implementations instead trap and let privileged software perform/refill translations.
ASID/PCID	Tags cached translations by address-space identity, reducing flushes on context switches.
TLB shootdown	Invalidates stale translations on other CPUs when page tables change.
PTE accessed/dirty handling	Architecture-specific mechanism for recording/validating whether a mapping was read or written.

A page-table walk competes for the memory hierarchy. PTEs live in ordinary physical memory. A walker therefore generates cache/memory traffic of its own, and a cache miss in a PTE fetch can make a TLB miss dramatically more expensive than the number of page-table levels alone suggests.

SOURCE / DESIGN DOCCVA6 MMU — real hardware page-table walker

Concrete open RISC-V implementation: L1 ITLB/DTLB, optional shared TLB, dedicated seven-state hardware PTW, PTE-address calculation and page-fault propagation.

https://github.com/openhwgroup/cva6/blob/master/docs/03_cva6_design/MMU.rst

PUBLIC DOCSCVA6 Execute Stage — PTW overview

Shows the page-table walker listening to ITLB/DTLB misses, prioritizing DTLB misses and returning page-fault exceptions through the MMU.

https://docs.openhwgroup.org/projects/cva6-user-manual/03_cva6_design/ex_stage.html

SOURCE / FILERocket Chip TLB source

Real source showing L1 TLB misses requesting a PTW/L2-TLB path, superpage entries, associativity, refill and SFENCE invalidation behavior.

https://github.com/chipsalliance/rocket-chip/blob/master/src/main/scala/rocket/TLB.scala

One x86 page fault: MMU detects the access, #PF enters Linux, the VM either repairs it or signals the process

A page fault is a synchronous CPU exception caused by the faulting memory access itself. On x86 the processor records the faulting linear address in CR2 and supplies a page-fault error code describing properties such as present/protection versus not-present, read versus write, user versus supervisor, instruction fetch and selected protection-key/reserved-bit conditions. Linux then decides whether that fault is a legitimate demand-paging event or an invalid access.

USER INSTRUCTION mov (%rax), %rbx # CPU tries to load virtual address V ↓ TLB miss? → hardware page-table walk ↓ translation/access check fails examples: PTE not present write to read-only mapping instruction fetch from NX mapping protection-key denial ↓ CPU raises x86 exception vector 14: #PF CR2 = faulting linear address V error code describes access/fault class ↓ IDT page-fault entry → exc_page_fault() ↓ Linux x86 do_user_addr_fault()-style path FAST VMA LOOKUP find vm_area_struct covering V ↓ no VMA? → invalid address → SIGSEGV / SEGV_MAPERR VMA exists, but access violates VMA permissions? → SIGSEGV / SEGV_ACCERR (or pkey-specific signal code) VMA + access are valid ↓ handle_mm_fault(vma, V, FAULT_FLAGS, regs) COMMON REPAIR CASES 1) anonymous demand-zero missing private anonymous page → allocate/map zeroed page (or shared zero-page read path) 2) copy-on-write write to read-only shared COW PTE → allocate/copy private page → replace PTE writable 3) file-backed page → page-cache lookup → cache miss may initiate storage I/O/readahead → install PTE after folio becomes available 4) swapped anonymous page → locate swap entry → read/decompress page → install present PTE 5) THP opportunity → may allocate/collapse large page where policy permits REPAIR SUCCEEDS update page table / TLB state as needed ↓ return from exception ↓ CPU retries/resumes faulting instruction REPAIR FAILS VM_FAULT_OOM → OOM handling VM_FAULT_SIGBUS/HWPOISON → SIGBUS path invalid mapping/access → SIGSEGV Minor fault = serviced without backing-storage I/O. Major fault = fault handling required backing-storage I/O.

Page-fault item	Meaning
#PF / vector 14	x86 page-fault exception raised synchronously by a failed translation/protection check.
CR2	x86 control register containing the linear address that caused the most recent page fault.
X86_PF_PROT	Kernel interpretation of the error-code bit distinguishing protection violation from not-present translation.
X86_PF_WRITE	Faulting access was a write.
X86_PF_USER	Fault arose from an access treated as user-mode for page-fault checking.
X86_PF_INSTR	Fault arose during instruction fetch.
vm_area_struct	Linux VMA describing a contiguous virtual-address range and its permissions/backing object/policy.
handle_mm_fault()	Generic Linux VM fault engine handling page-table population, COW, file/swap/THP cases and retry/error results.
VM_FAULT_MAJOR	Fault result flag indicating backing-store I/O was required.
VM_FAULT_RETRY	Fault path dropped/rearranged locking and asks architecture code to retry the fault handling.
SEGV_MAPERR	SIGSEGV reason meaning the address is not mapped by a suitable VMA.
SEGV_ACCERR	SIGSEGV reason meaning a mapping exists but access permissions reject the operation.
SIGBUS	Can report file truncation/storage/memory-poison style faults where an address mapping exists but cannot be satisfied normally.

The CPU does not know about malloc(), files, swap or COW. Hardware reports a failed virtual-memory access. Linux interprets that address using the process's VMAs and page-table/software metadata to decide what higher-level event the fault represents.

PAGE-FAULT LAB # Count minor/major faults perf stat -e page-faults,minor-faults,major-faults ./program # Process-wide fault counters also appear in /proc/<pid>/stat # fields include minflt/cminflt/majflt/cmajflt. # VMA map and permissions cat /proc/<PID>/maps cat /proc/<PID>/smaps | less # Experiments: # 1. mmap anonymous 256 MiB but touch no pages; compare RSS/faults. # 2. touch one byte per page; observe minor faults/RSS increase. # 3. map a file and first-touch it; warm vs cold major-fault behavior differs. # 4. fork(), then write private pages; observe COW minor faults. # Do not deliberately dereference arbitrary kernel addresses; a userspace SIGSEGV # experiment is enough to study invalid-address behavior safely.

SOURCE / FILELinux x86 page-fault implementation

Current x86 source distinguishes user/supervisor, instruction/write/read and not-present/protection faults; invalid user addresses are converted to SIGSEGV while valid VMAs flow into handle_mm_fault().

https://github.com/torvalds/linux/blob/master/arch/x86/mm/fault.c

SOURCE / FILELinux generic VM fault implementation

The real current demand-paging/COW/page-table machinery behind handle_mm_fault(). This is where architecture faults become generic Linux VM operations.

https://github.com/torvalds/linux/blob/master/mm/memory.c

SOURCE / FILELinux RISC-V page-fault implementation

Useful comparison showing the same architecture→VMA→handle_mm_fault→signal design on another ISA, despite different trap registers/cause encoding.

https://github.com/torvalds/linux/blob/master/arch/riscv/mm/fault.c

The MMU leaves breadcrumbs for the OS: accessed/young and dirty bits connect hardware page walks to reclamation

A page-table entry is not only an address translation and permission record. Many architectures also maintain state describing whether the mapped page has been accessed and whether it has been written. Linux abstracts these ideas as young/accessed and dirty page-table state. They let the kernel learn something about memory usage without trapping on every ordinary load and store.

LEAF PAGE-TABLE ENTRY virtual page → physical page + permissions + A/D-style state CPU reads / executes from page ↓ accessed / young state becomes set ↓ kernel can later test + clear "young" ↓ if the mapping is touched again, state can become young again ↓ reclaim code gets a recency hint CPU stores to page ↓ dirty state becomes set ↓ kernel learns that the mapped page was written ↓ that information is propagated into the VM/filesystem bookkeeping needed to preserve modified data IMPORTANT DISTINCTION PTE dirty bit = translation-level evidence that a mapped page was written page-cache dirty folio = filesystem/cache state requiring storage writeback cache-line dirty = CPU-cache coherence/writeback state These are related concepts at different layers, not one universal bit.

State/helper	Meaning to Linux
`pte_young()`	Tests whether a mapping has recently been accessed according to the architecture's page-table semantics.
`ptep_test_and_clear_young()`	Observes and clears that recency state so a later access can make use visible again.
`pte_dirty()`	Tests whether the page-table mapping records a write.
`pte_mkdirty()` / `pte_mkclean()`	Architecture-independent MM helpers for manipulating dirty state where appropriate.
hardware-managed A/D	The MMU/page-table walker updates the PTE as memory is used.
fault-managed A/D	An architecture may instead fault when accessed/dirty state must be established, letting software update it.

Accessed is usually a hint; dirty state participates in correctness. Replacement/reclaim algorithms can tolerate approximate recency information, while modified data must not be discarded as though it were clean. Architecture rules therefore make different guarantees about accessed and dirty updates.

KERNEL DOCSLinux architecture page-table helpers

The generic MM contract for pte_young, pte_dirty, ptep_test_and_clear_young and the corresponding PMD/PUD helpers used across architectures.

https://docs.kernel.org/next/mm/arch_pgtable_helpers.html

SEE EARLIERRISC-V Sv39 page-table walk and privileged specification

The earlier Sv39 section links the current privileged architecture specification. Its leaf-PTE definition gives a concrete architecture-level example of A (accessed) and D (dirty) semantics and the permitted update schemes.

# User-space page-table visibility is intentionally constrained for security reasons. # Useful high-level signals instead: cat /proc/meminfo | grep -E 'Active|Inactive|Dirty|Writeback' # Kernel code uses architecture page-table helpers rather than assuming # every MMU encodes young/dirty state in the same bit positions.

userfaultfd lets userspace participate in page-fault handling: fault → fd event → UFFDIO_* resolution

Normally, a CPU page fault enters the kernel and the kernel VM resolves it. Linux userfaultfd adds a controlled exception to that model: a process can register selected virtual-memory ranges so particular faults become messages on a file descriptor. A userspace handler can then supply a page, map an already-existing page, zero-fill it, or manage write protection before the faulting thread continues. The MMU still detects the fault and the kernel still mediates the page tables; userspace is being delegated part of the policy and data-supply work.

APPLICATION / VMM SETUP create userfaultfd ↓ UFFDIO_API feature negotiation ↓ UFFDIO_REGISTER virtual-address range modes can include missing / minor / write-protect ↓ handler thread poll()/read()s the userfaultfd FAULTING THREAD CPU load/store touches registered address ↓ normal MMU translation/protection machinery ↓ page fault kernel recognizes userfaultfd-managed condition ↓ queue uffd_msg event on userfaultfd ↓ faulting thread waits when synchronous handling is required USERSPACE HANDLER read fault address + flags ↓ choose resolution UFFDIO_COPY → install supplied page contents UFFDIO_ZEROPAGE → map a zero-filled page UFFDIO_CONTINUE → continue with an existing cached page UFFDIO_WRITEPROTECT / related modes → track writes/access ↓ kernel updates mapping / wakes waiter ↓ faulting instruction resumes

Mechanism	What it means
missing fault	The registered address has no usable page yet; userspace can provide contents or a zero page before execution resumes.
minor fault	Backing data already exists, such as in a page cache, but the mapping is not installed; userspace can inspect/modify state before continuing.
write-protect tracking	Write protection can turn the first write into a userfaultfd event, useful for dirty-page tracking and migration/checkpoint logic.
pollable fd	Fault notifications fit the ordinary Linux fd event model and can be handled by a dedicated manager thread/process.
post-copy migration	A VM/container may resume before all memory arrives; missing guest pages are fetched on demand when the guest first touches them.

userfaultfd is not a userspace page-table writer. Userspace receives events and asks the kernel to resolve registered faults through defined ioctls. The kernel still owns validation, PTE installation, wakeups and access-control policy.

USERFAULTFD INSPECTION IDEAS # API and flags are Linux-specific; read the current interface first. man 2 userfaultfd # In a demonstration program, watch the handler and faulting threads: strace -f -e userfaultfd,ioctl,poll,read,mmap ./uffd-demo # Useful mental experiment: # reserve 1 GiB VA with mmap # register it as missing # touch one page at a time # handler supplies each page only when first touched # This makes demand paging policy visible in userspace.

KERNEL DOCSLinux userfaultfd — design, API and fault-resolution modes

Current kernel documentation for creating/registering a userfaultfd, receiving fault messages, resolving missing/minor faults, write-protection tracking and modern access-control details.

https://docs.kernel.org/admin-guide/mm/userfaultfd.html

PUBLIC MANUALuserfaultfd(2) — userspace page-fault handling interface

Current Linux man-page reference for the syscall, file-descriptor behavior, history, privilege restrictions and checkpoint/post-copy migration use cases.

https://man7.org/linux/man-pages/man2/userfaultfd.2.html

What happens after a cache miss: MSHRs, line fills, write buffers and prefetchers

A simple blocking cache can stop while one miss goes to the next level. Modern caches are often non-blocking: they remember outstanding misses in Miss Status Holding Registers (MSHRs) so unrelated hits—and often additional misses—can proceed while earlier lines are still in flight.

LOAD address A ↓ L1 tag lookup MISS: line A not present ↓ allocate MSHR for cache-line address A ├── remember original load as a target/waiter └── send line request to L2 / coherence fabric while A is outstanding: LOAD address B → L1 HIT → can often complete = HIT UNDER MISS LOAD address C → L1 MISS → second MSHR if available = MISS UNDER MISS another LOAD to A before fill returns: ↓ find existing MSHR for same cache line merge/coalesce second waiter into that MSHR do NOT necessarily issue a duplicate lower-level read eventually line A returns ↓ line-fill buffer / fill path choose cache way / evict victim if needed ├── dirty victim → enqueue WRITEBACK └── clean victim → discard install tag + data + coherence state ↓ wake/satisfy all loads waiting in MSHR for A free MSHR entry If all MSHRs are occupied, a new miss may have to STALL even though the cache data/tag RAM itself is otherwise available.

Structure/mechanism	Purpose
MSHR	Tracks an outstanding line miss, its address/state and one or more waiting CPU/prefetch requests.
line-fill buffer	Temporary path/storage for a cache line arriving from the next hierarchy level before/while it is installed.
write buffer	Queues dirty evictions or uncached writes so the cache does not always block on lower-level write completion.
hit under miss	Serve a cache hit while a previous miss is outstanding.
miss under miss	Launch another independent miss while earlier miss(es) are still outstanding.
miss coalescing	Attach multiple accesses to the same absent cache line to one outstanding line request.
critical-word first / early restart	Some designs can return the requested word to CPU before the rest of a full cache line has finished arriving.
prefetcher	Predicts future line accesses and requests them before an architectural demand misses.
prefetch accuracy	Fraction of prefetched lines that become useful before eviction.
prefetch coverage	Fraction of otherwise-demand misses eliminated/anticipated by useful prefetches.
late prefetch	Prefetch that predicts the right line but does not arrive before the demand access needs it.
memory-level parallelism	Number/degree of independent memory misses a core/cache hierarchy can overlap.

More MSHRs do not make DRAM intrinsically faster. They expose memory-level parallelism by letting several independent misses overlap. Performance improves only when downstream caches/interconnect/memory have enough parallelism and bandwidth to service that concurrency.

SIMPLE STRIDE PREFETCH IDEA observed demand lines: 100, 101, 102, 103 ... predict next line 104 before CPU asks ↓ prefetch line 104 into cache / prefetch buffer best case: later demand 104 hits → latency hidden bad case: CPU never uses 104 → bandwidth/cache capacity wasted Aggressive prefetch can also evict useful lines or compete with demand traffic.

PUBLIC DOCSgem5 — Classic caches (current July 2026)

Current public documentation explicitly describes gem5's default cache as non-blocking with MSHRs and a write buffer, plus optional prefetching and configurable replacement/indexing policies.

https://www.gem5.org/documentation/general_docs/memory_system/classic_caches/

PUBLIC DOCSgem5 — memory system: MSHR and write-buffer queues

Explains that MSHRs hold outstanding cached read/write misses and uncached reads, while the write buffer holds uncached writes and dirty evictions.

https://www.gem5.org/documentation/general_docs/memory_system/gem5_memory_system/

SOURCE DOCSgem5 — current MSHR class/source

Real simulator implementation reference showing allocation of an MSHR by cache-block address and attached target requests.

https://doxygen.gem5.org/develop/classgem5_1_1MSHR.html

PUBLIC DOCSBOOM — detailed open out-of-order pipeline

Open RISC-V OoO core documentation showing the execution/memory pipeline context around load/store queues, caches and miss handling.

https://docs.boom-core.org/en/latest/sections/intro-overview/boom-pipeline.html

PUBLIC DOCSBOOM documentation — complete front-end/backend overview

The project overview exposes load/store queues, data cache, configurable MSHRs and line-fill structures in a real open superscalar design.

https://docs.boom-core.org/

One modern LOAD instruction: from register bits to DRAM and back

On a simple 8-bit computer, a load may be almost literally 'put address on bus, read RAM.' A modern CPU preserves that architectural meaning while inserting translation, caches, speculation, queues, coherence, and a memory controller underneath.

instruction decoder / execution unit ↓ computes virtual address TLB lookup ├── hit → physical address known └── miss → page-table walk → fill TLB ↓ L1 data-cache lookup ├── hit → data returned quickly └── miss ↓ L2 / L3 / coherence machinery ├── another cache may own newer data └── otherwise request memory ↓ memory controller ↓ DRAM command/address/data signaling ↓ cache line filled upward ↓ requested bytes delivered to CPU register

Important: this is a conceptual path, not a promise that every CPU performs the operations serially in exactly this order. Real microarchitectures overlap work aggressively and may translate, speculate, prefetch, merge or reorder requests.

DIRECT PDFOSTEP — Paging: Introduction (direct PDF)

Direct free book chapter on virtual pages, physical frames and page tables. Good preparation before TLBs and hardware page-table walking.

https://pages.cs.wisc.edu/~remzi/OSFEP/vm-paging.pdf

FREE BOOKOSTEP — free chapter index

The whole book remains free chapter-by-chapter. Relevant sequence: address translation → paging → TLBs → smaller/multilevel page tables → swapping.

https://pages.cs.wisc.edu/~remzi/OSTEP/

OPEN NOTESMIT 6.823 — cache / virtual-memory architecture notes

No enrollment. Public PDF index covering caches, virtual memory, out-of-order execution, branch prediction, synchronization and cache coherence.

https://ocw.mit.edu/courses/6-823-computer-system-architecture-fall-2005/pages/lecture-notes/

DIRECT PDFIntel system-programming manual — caches and TLBs (direct PDF)

Real architecture manual. Chapter 12 covers internal caches, TLBs, buffers and cache-control behavior. Dense but authoritative.

https://cdrdv2-public.intel.com/825758/253668-sdm-vol-3a.pdf

A real page-table walk: RISC-V Sv39 virtual address → physical address

Virtual memory is not an abstract dictionary hidden in the OS. On a TLB miss, hardware may literally perform several dependent memory reads to walk a page-table tree. RISC-V Sv39 is unusually good for learning this because the architecture specifies a clean three-level structure.

Sv39 uses a 39-bit effective virtual-address structure inside RV64: VA bits 38........30 29........21 20........12 11........0 VPN[2] VPN[1] VPN[0] page offset 9 bits 9 bits 9 bits 12 bits page size = 2^12 = 4096 bytes each page-table level has 2^9 = 512 entries each PTE is 8 bytes one page table = 512 × 8 = 4096 bytes = exactly one page satp register → root page-table physical page number ↓ root + VPN[2] * 8 → read level-2 PTE ↓ if pointer/non-leaf next table + VPN[1] * 8 → read level-1 PTE ↓ if pointer/non-leaf next table + VPN[0] * 8 → read level-0 PTE ↓ leaf PTE PPN from PTE + original 12-bit page offset → physical address Then: physical address → cache hierarchy → RAM/device

Event	What happens
TLB hit	Cached VPN→PPN translation is found; no page-table memory walk is needed for that translation.
TLB miss	Hardware/software page walker reads page-table entries from memory; successful result is commonly cached in the TLB.
invalid/nonpermitted PTE	Translation cannot legally complete; a page-fault exception is raised to privileged software.
page fault	Kernel decides what to do: allocate/map memory, load a page from backing storage, grow a mapping, deliver a fault to process, etc.
context switch	OS changes page-table context (e.g. satp/ASID); TLB entries may need tagging or invalidation according to architecture rules.
SFENCE.VMA	RISC-V instruction used to order/invalidate address-translation state after page-table updates where required.

DIRECT SPEC PDFRISC-V privileged specification — direct PDF

Direct specification PDF. Search for 'Sv39' and 'SFENCE.VMA' when you want the normative page-table-walk algorithm and translation-cache rules.

https://docs.riscv.org/reference/isa/v20240411/_attachments/riscv-privileged.pdf

PUBLIC NOTESCornell CS3410 (2026) — virtual memory

Current public notes explaining VPN/PPN/offset, page tables, satp and TLBs without requiring a course account.

https://courses.cs.cornell.edu/cs3410/2026sp/notes/vm.html

PUBLIC NOTESCS61C — address translation

Simple worked VA→VPN→PTE→PPN→PA examples before you attack Sv39's three-level version.

https://notes.cs61c.org/content/vm/address-translation/

PUBLIC NOTESCS61C — page-table design

Covers page-table entries, permissions/status bits, process isolation, replacement and hierarchical page tables.

https://notes.cs61c.org/content/vm/page-table/

PUBLIC NOTESCS61C — TLB + cache + memory hierarchy together

One of the best conceptual bridges showing TLB lookup/page-table walk followed by physical-address cache lookup and eventual RAM access.

https://notes.cs61c.org/content/vm/memory-hierarchy-full/

TLB shootdown: changing one page table means every CPU using it may need to forget

Page-table memory and TLB state can diverge. If CPU 0 changes a PTE while CPU 1 still holds the old translation in its TLB, CPU 1 may continue using the stale mapping. A multiprocessor OS therefore needs TLB shootdown: update page tables, order the update, notify relevant CPUs, invalidate their cached translation state, and wait as required before reusing/freing the old mapping.

Initial state: page table: VA X → physical page P CPU0 TLB: X → P CPU1 TLB: X → P CPU0 unmaps / changes X ↓ write new PTE into memory ↓ ordering rule / page-table barrier as architecture requires invalidate CPU0 local translation ↓ which other CPUs may have run this address space? ↓ send IPI / architecture shootdown notification to relevant CPUs ↓ CPU1 interrupt handler performs local TLB invalidation ↓ CPU1 acknowledges completion ↓ CPU0 may safely proceed with operation that required stale references gone RISC-V example: page-table store → data fence if needed → IPI remote hart → remote SFENCE.VMA → acknowledgement Optimization: if address space never executed on CPU7, no reason to shoot down CPU7.

Operation	Why invalidation may be needed
munmap()	Old VA→PA translation must not remain usable after mapping disappears.
mprotect()	TLB may cache old writable/executable permission bits.
COW remap	Old page mapping/protection must not outlive page-table update.
page migration	Translation must eventually point to new physical frame, not old page.
fork/exec address-space operations	Large portions or entire address-space translation state can change.
kernel mapping update	Global/kernel translations may need broad invalidation according to architecture.
ASID/PCID reuse	Identifier reuse must not let stale translations from an old address space alias a new one.

This is why a page-table write alone is not enough. TLBs are intentionally caches. Architectures therefore provide invalidation/fence mechanisms, and SMP kernels add cross-CPU coordination on top.

KERNEL DOCSLinux — Cache and TLB Flushing

Direct kernel explanation of flush_tlb_all/mm/range/page and the SMP requirement that relevant CPUs observe page-table modifications; also notes optimizations based on which CPUs actually used an address space.

https://docs.kernel.org/core-api/cachetlb.html

KERNEL DOCSLinux x86 TLB documentation

Public architecture-specific documentation for x86 TLB behavior, flush tradeoffs and when targeted invalidation can beat a full TLB flush.

https://docs.kernel.org/arch/x86/tlb.html

A context switch no longer has to flush every translation: x86 PCID and per-CPU cached address spaces

Classic x86 code often treated writing a new CR3 page-table root as implicitly discarding old process translations. With Process-Context Identifiers (PCID), TLB entries can be tagged by address-space context, allowing translations for several processes to coexist. Linux uses a small per-CPU dynamic ASID/PCID cache rather than assigning one permanent PCID to every process forever.

CPU 4 CURRENTLY RUNS PROCESS A CR3 = A page-table root + PCID 3 TLB contains: PCID3: A VA 0x400000 → PA ... PCID3: A VA 0x7f... → PA ... PCID1: older process C translations still cached SCHEDULER SWITCHES A → B ↓ switch_mm() / x86 TLB context code ↓ Does CPU's small ASID cache already contain B's mm with current generation? ├── yes │ select B's cached ASID/PCID │ load CR3 with NOFLUSH semantics where valid │ old A entries remain tagged PCID3 and ignored while B runs │ └── no ↓ choose/recycle one per-CPU dynamic ASID slot ↓ if recycled slot may contain stale translations, flush that context ↓ associate slot with B's mm + current TLB generation ↓ load B CR3 + corresponding PCID B accesses a page whose translation survived from B's earlier run on this CPU ↓ TLB hit immediately; no page walk required PAGE TABLE FOR B CHANGES unmap/mprotect/COW/etc. ↓ B's mm TLB generation advances / invalidation initiated ↓ local invalidation + remote shootdown for CPUs that may use B ↓ INVPCID/INVLPG/CR3 reload strategy chosen by range/feature/context INVPCID can target: one address within one PCID one entire PCID context all non-global contexts all contexts including globals PCID saves work ONLY when retained entries are still valid.

Translation-context term	Meaning
PCID	x86 Process-Context Identifier carried in CR3 and associated with non-global TLB entries.
ASID	Generic/RISC-style term for address-space identifier; Linux x86 source calls its software slot IDs ASIDs and maps them to hardware PCIDs.
CR3	x86 page-table-root register; with PCID enabled also carries PCID and a no-flush control bit in defined cases.
CR3 no-flush	PCID-enabled CR3 load mode preserving compatible TLB entries instead of flushing that context automatically.
INVPCID	x86 instruction invalidating translations by PCID/address or broader context without switching page-table roots.
tlb_gen	Linux generation counter tracking whether a CPU's cached translations for an mm are current.
per-CPU dynamic ASID cache	Linux cache of a small number of recent mm contexts on each CPU for cheap switch_mm() reuse.
global TLB entry	Translation marked global so ordinary address-space switches do not discard it; special invalidation rules apply.
KPTI PCID pair	With page-table isolation, Linux may use separate user/kernel PCID spaces for one mm.
TLB shootdown	IPI/remote invalidation process required when another CPU might retain stale translations.

PCID is a performance optimization, not a relaxation of page-table correctness. A translation may remain cached across a context switch only while Linux can prove that the PCID still names the same address space and that its TLB generation is not stale.

LINUX/x86 OBSERVATION LAB # CPU advertises PCID / INVPCID? grep -m1 -oE '\b(pcid|invpcid)\b' /proc/cpuinfo | sort -u # kernel boot may mention PCID/PTI/IOMMU-related features dmesg | grep -Ei 'PCID|page table isolation|PTI|INVPCID' | less # context-switch and TLB PMU events vary by processor perf list | grep -Ei 'tlb|dtlb|itlb|context-switch' | less # compare a workload pinned to one CPU versus migrating across CPUs; # retained TLB state is per CPU, so migration can change translation locality. # Kernel source trail: # arch/x86/mm/tlb.c # arch/x86/include/asm/tlbflush.h # arch/x86/include/asm/invpcid.h

SOURCE / FILELinux x86 TLB context implementation

Current source explicitly explains PCID versus software ASID naming and Linux's small per-CPU cache of recently used mm contexts for cheaper process switches.

https://github.com/torvalds/linux/blob/master/arch/x86/mm/tlb.c

SOURCE / FILELinux x86 tlbflush.h

Real current state structure: loaded mm, per-ASID contexts, TLB generation tracking and user-PCID flush masks.

https://github.com/torvalds/linux/blob/master/arch/x86/include/asm/tlbflush.h

SOURCE / FILELinux x86 INVPCID helpers

Small readable source defining x86 INVPCID operations for one address/context, one full PCID, all nonglobal contexts or all contexts including globals.

https://github.com/torvalds/linux/blob/master/arch/x86/include/asm/invpcid.h

SOURCE / FILELinux x86 PCID setup

Current initialization code enables CR4.PCIDE on capable x86-64 CPUs and shows that PCID support is an explicit architecture feature rather than automatic behavior.

https://github.com/torvalds/linux/blob/master/arch/x86/mm/init.c

Kernel virtual addresses are not all the same: direct map, vmalloc, vmemmap and ioremap

The kernel itself executes with virtual addresses. On 64-bit Linux, ordinary RAM is commonly accessible through a large direct/linear map whose virtual-to-physical relationship is simple. But the kernel also reserves separate virtual regions for dynamically mapped pages, metadata and device MMIO. A pointer's address alone therefore does not tell you whether it represents normal RAM, vmalloc pages or device memory.

ILLUSTRATIVE x86-64 KERNEL ADDRESS REGIONS (exact bases vary with 4/5-level paging, KASLR and kernel config) USER VIRTUAL SPACE low canonical addresses ... user VMAs / stacks / libraries ... ↓ canonical-address hole KERNEL HALF DIRECT / LINEAR MAP kernel_va = page_offset_base + physical_address [conceptually] ↓ all ordinary physical RAM reachable through a predictable offset mapping struct page-backed memory from buddy/slab/kmalloc normally lives here VMALLOC / VMAP SPACE contiguous KERNEL VIRTUAL range ↓ page tables physical page 9 physical page 10482 physical page 400 physical page 77 ↓ physically scattered pages appear virtually adjacent IOREMAP SPACE / VMALLOC AREA device BAR physical/bus MMIO address ↓ ioremap()/pci_iomap() kernel __iomem token / virtual mapping ↓ readl()/writel() access DEVICE REGISTERS not ordinary cacheable RAM VMEMMAP large virtual mapping containing struct page metadata one struct page object describes each relevant physical page/PFN range WHY vmalloc EXISTS need 8 MiB virtually contiguous buffer buddy allocator may not have an 8 MiB physically contiguous block ↓ vmalloc allocates many individual pages ↓ installs PTEs making them adjacent in kernel VA space WHY vmalloc IS NOT FOR DMA BY DEFAULT device usually needs physical/DMA segments, not the CPU's private vmalloc VA. DMA API must map the underlying pages appropriately.

Kernel mapping	Physical continuity	Typical use
direct/linear map	Mirrors ordinary physical RAM with a simple architecture-defined offset relationship	Most normal kernel RAM and page/slab allocations.
vmalloc()	Virtually contiguous, physically noncontiguous pages allowed	Large kernel buffers where physical contiguity is unnecessary.
vmap()	Maps caller-supplied page array into one contiguous kernel VA range	Temporary/constructed virtual mapping of existing pages.
ioremap()	Maps bus/device physical MMIO, not normal allocated RAM	Device control/status registers and MMIO apertures.
ioremap_wc()	Device memory with write-combining semantics where architecture/resource permits	Framebuffer/device memory optimized for streaming writes.
vmemmap	Metadata mapping rather than user payload	Virtual array of struct page descriptors for physical pages.
fixmap	Fixed-address slots with mappings changed underneath	Early boot, APIC/architecture helpers and specialized temporary mappings.
memremap()	Maps certain system/physical memory ranges as normal memory semantics	Persistent/system RAM-like ranges outside ordinary linear map use cases.
__iomem	Sparse type annotation/token, not ordinary C RAM pointer semantics	Forces drivers toward readl/writel/memcpy_toio-style accessors.

Do not dereference MMIO as if it were ordinary RAM. Linux's ioremap() returns an __iomem access token; portable drivers use readl()/writel() and related accessors because cacheability, ordering, endianness and even the meaning of the token vary by architecture.

LINUX KERNEL-VIRTUAL-MEMORY OBSERVATION # vmalloc/vmap/ioremap areas, usually root-readable cat /proc/vmallocinfo 2>/dev/null | less # high-level physical memory and kernel accounting grep -E 'MemTotal|Vmalloc|Slab|SReclaimable|SUnreclaim|DirectMap' /proc/meminfo # x86 kernel map bases are randomized on many systems, # so use kernel docs/source rather than assuming textbook constants. # /proc/iomem shows physical resource ownership but may redact addresses cat /proc/iomem | less # Do not mmap /dev/mem or poke MMIO registers just to inspect the mapping model.

KERNEL DOCSLinux x86-64 memory map

Current architecture map distinguishes the direct physical-memory mapping, vmalloc/ioremap space and vmemmap; it also notes KASLR can randomize their bases.

https://docs.kernel.org/arch/x86/x86_64/mm.html

KERNEL DOCSLinux Memory Allocation Guide

Current guide distinguishes kmalloc/SLAB objects, alloc_pages and large virtually contiguous vmalloc allocations.

https://docs.kernel.org/core-api/memory-allocation.html

KERNEL DOCSLinux vmalloc APIs

Current API explicitly defines vmalloc_node() as allocating pages from the page allocator and mapping them into contiguous kernel virtual space.

https://docs.kernel.org/core-api/mm-api.html

The kernel does not waste a 4 KiB page on every inode: kmalloc and SLUB carve pages into reusable objects

The buddy/page allocator works in page-sized and power-of-two page blocks, which is too coarse for the kernel's enormous number of small objects. Linux therefore uses slab-family object caches. On typical modern configurations the implementation is SLUB: pages are obtained from the page allocator, divided into same-sized object slots, and fast per-CPU freelists satisfy many allocations without entering the buddy allocator each time.

kmalloc(120, GFP_KERNEL) ↓ round/select kmalloc size class example conceptual cache: kmalloc-128 ↓ CURRENT CPU has free object in active SLUB slab? ├── yes │ pop object from per-CPU freelist │ return 128-byte slot (caller requested 120) │ └── no ↓ obtain/refill a slab for this cache ↓ node partial-slab list has one? ── yes → adopt/refill │ no ↓ buddy/page allocator allocates one or more physical pages ↓ initialize slab metadata / object freelist ↓ allocate one object kfree(ptr) ↓ determine owning slab/cache/object slot ↓ return slot to appropriate freelist ↓ if whole slab becomes empty and policy allows slab pages can eventually return to page allocator DEDICATED OBJECT CACHE kmem_cache_create("dentry-like", sizeof(struct foo), ...) ↓ all slots have same size/alignment/layout ↓ kmem_cache_alloc(cache, GFP_KERNEL) WHY OBJECT CACHES HELP less per-allocation metadata reuse already initialized/aligned slots strong locality for common kernel structures per-CPU fast paths reduce global lock contention GFP FLAGS MATTER GFP_KERNEL may enter reclaim/sleep GFP_ATOMIC cannot perform the same sleeping reclaim path ↓ allocation context constrains what the allocator is allowed to do.

SLAB/SLUB term	Meaning
kmalloc	General kernel small-object allocator choosing among size-class caches.
kzalloc	kmalloc-family allocation whose requested region is zero initialized.
kmem_cache	Cache describing objects of one fixed size/layout/alignment and its slab bookkeeping.
slab	One or more physical pages assigned to one object cache and subdivided into object slots.
object	One allocation slot inside a slab.
per-CPU freelist	Fast local list of reusable objects allowing many alloc/free operations without global contention.
partial slab	Slab containing both allocated and free objects; reusable for future allocations.
full slab	Slab with no free object slots.
empty slab	Slab with no live objects; may be retained for reuse or returned to page allocator.
GFP_KERNEL	Normal kernel allocation context allowed to sleep/reclaim where needed.
GFP_ATOMIC	Restricted non-sleeping allocation context for interrupt/atomic paths; success is less assured under pressure.
SLAB_HWCACHE_ALIGN	Object-cache flag requesting cacheline-oriented alignment.
SLAB_RECLAIM_ACCOUNT	Marks cache objects/pages as reclaimable for memory accounting/reclaim policy.
SLAB_TYPESAFE_BY_RCU	Delays freeing slab pages, not arbitrary object reuse; users must still validate object identity correctly.
slab poisoning/red zones	Debugging features that fill/check freed/guard memory to catch use-after-free/overflow bugs.

SLUB sits on top of the page allocator; it does not replace it. A cache miss/refill eventually needs physical pages, so slab pressure can propagate downward into buddy allocation, reclaim, compaction and ultimately OOM behavior.

LINUX SLAB OBSERVATION LAB # cache totals / object counts head -40 /proc/slabinfo # named cache objects and debugging/accounting controls ls /sys/kernel/slab | head -80 # overall slab memory grep -E 'Slab|SReclaimable|SUnreclaim' /proc/meminfo # if slabtop is installed slabtop -o # kernel allocation tracepoints, if tracefs/perf permissions allow perf list | grep -Ei 'kmem:kmalloc|kmem:kmem_cache|kmem:mm_page_alloc' | less # Keep slab_debug/failslab/fault-injection controls read-only on a useful machine.

KERNEL DOCSSLUB users/debug guide

Public guide to SLUB debugging, /sys/kernel/slab, poisoning, red zones, allocation tracking and cache-specific diagnostics.

https://docs.kernel.org/admin-guide/mm/slab.html

KERNEL DOCSKernel kmem tracepoints

Current tracepoint guide explicitly separates kmalloc, typed slab-cache allocation, page allocation, per-CPU allocator activity and external fragmentation.

https://docs.kernel.org/trace/events-kmem.html

SOURCE / FILELinux SLUB source

Real allocator implementation: per-CPU slab/freelist fast paths, node partial lists, page allocation, object freeing and debug/hardening hooks.

https://github.com/torvalds/linux/blob/master/mm/slub.c

KERNEL DOCSLinux fault-injection infrastructure

Explains failslab and fail_page_alloc; useful for understanding allocator error paths, though injection should be confined to disposable test systems.

https://docs.kernel.org/fault-injection/fault-injection.html

Below SLUB is the physical page allocator: zones, per-CPU page lists, buddy orders, splitting and coalescing

The kernel ultimately needs physically backed pages. Linux organizes free physical memory by NUMA node and zone, then by buddy order. An order-N allocation is 2^N physically contiguous base pages. Order-0 traffic often uses per-CPU page lists first to avoid taking a heavily contended zone lock; larger or refill allocations descend into the buddy allocator.

4 KiB BASE PAGE EXAMPLE order 0 = 1 page = 4 KiB order 1 = 2 pages = 8 KiB order 2 = 4 pages = 16 KiB order 3 = 8 pages = 32 KiB ... alloc_pages(GFP_KERNEL, order=0) ↓ select NUMA node / memory policy / zone ↓ per-CPU page list has suitable free page? ├── yes → pop local page → fast path └── no ↓ refill from zone buddy allocator under zone synchronization BUDDY FREE AREAS zone->free_area[order][migratetype] request order 2 (4 contiguous pages) free_area[2] empty free_area[3] contains one 8-page block ↓ remove order-3 block split into two order-2 buddies ↓ return one 4-page half put the other 4-page half on free_area[2] FREEING free order-2 block at PFN P buddy PFN = P XOR (1 << order) ↓ is buddy free, same compatible order/type, and mergeable? ├── no → place block on order-2 free list └── yes ↓ remove buddy combine into order 3 ↓ repeat upward while matching free buddy exists MIGRATETYPES / PAGEBLOCKS free lists are partitioned by mobility/use classes such as MOVABLE / UNMOVABLE / RECLAIMABLE / CMA / HIGHATOMIC ↓ goal: keep compatible allocations together so future high-order contiguous allocations are less likely to be destroyed by fragmentation IF CONTIGUOUS HIGH-ORDER ALLOCATION FAILS kernel may reclaim pages kernel may compact movable pages fallback between migratetypes may occur ↓ if nothing can satisfy allocation under allowed GFP policy → failure/OOM path

Page-allocator concept	Meaning
order	Exponent describing physically contiguous block size: 2^order base pages.
buddy	The uniquely paired same-order block that can be merged with a freed block when both are free.
free_area[]	Per-zone arrays/lists of free blocks grouped by order and migratetype.
per-CPU page list (PCP)	CPU-local cache of free pages reducing zone-lock contention for common small allocations.
zone	Physical-memory allocation class/range such as DMA/DMA32/Normal, maintained per NUMA node as applicable.
migratetype	Free-page classification intended to reduce fragmentation by separating movable, unmovable, reclaimable and special-use allocations.
pageblock	Larger grouping whose migration type guides compaction/CMA/anti-fragmentation policy.
split	Break one higher-order free block into smaller buddies until requested order is reached.
coalesce	Merge a freed block with its matching free buddy, recursively producing a larger order.
compaction	Moves movable pages to assemble larger physically contiguous free ranges.
watermark	Per-zone free-memory threshold controlling allocation/reclaim behavior.
compound page	Allocation of multiple contiguous pages represented by one head plus tail pages.

Physical fragmentation is different from userspace allocator fragmentation. A process may have plenty of virtual address space while the buddy allocator still cannot find a large physically contiguous block. Conversely, vmalloc() can create one contiguous kernel virtual range from many scattered physical pages.

LINUX PAGE-ALLOCATOR OBSERVATION # free buddy blocks by zone/order cat /proc/buddyinfo # free blocks by migratetype/order cat /proc/pagetypeinfo | less # zone watermarks / per-zone accounting cat /proc/zoneinfo | less # page-allocation tracepoints on kernels exposing them perf list | grep -Ei 'kmem:mm_page_(alloc|free|pcpu)' | less # Watch /proc/buddyinfo while allocating/freeing large memory in a test program. # Do not force compaction/OOM/fault injection on an important machine just for this lab.

SOURCE / FILELinux page allocator source: mm/page_alloc.c

The real current buddy implementation. Source comments explicitly describe free lists by order, splitting smaller allocations from larger blocks and recursive buddy coalescing when both halves become free.

https://github.com/torvalds/linux/blob/master/mm/page_alloc.c

KERNEL DOCSLinux pagemap flags — BUDDY/compound pages

Documents BUDDY free blocks and compound-page order semantics visible through kernel page metadata interfaces.

https://docs.kernel.org/admin-guide/mm/pagemap.html

KERNEL DOCSLinux page allocation failure diagnostics

Current physical-memory/zone documentation for free-area organization, watermarks and page-allocation behavior.

https://docs.kernel.org/mm/physical_memory.html

Free memory can still be too fragmented: page migration lets compaction rebuild large buddy blocks

The buddy allocator can have plenty of free base pages yet fail a higher-order allocation because those pages are scattered between allocated pages. Memory compaction attacks that external fragmentation by relocating movable pages so free pages collect into larger physically contiguous runs. The virtual addresses seen by processes need not change: Linux can temporarily replace mappings with migration entries, copy page contents to new physical pages, then repair the mappings.

FRAGMENTED PHYSICAL MEMORY A = allocated/movable page, . = free page A . A . A . A . A . A . A . ↓ high-order allocation wants one contiguous run but free pages are separated ↓ COMPACTION scan one region for movable allocated pages scan another region for free destination pages ↓ for a migratable page: isolate + lock old page prepare destination page replace user PTE references with migration entries copy data / metadata redirect mappings to destination release old physical page ↓ repeat so allocated pages cluster one way, free pages cluster the other way ↓ . . . . . . . . A A A A A A A A ↓ buddy allocator can coalesce adjacent free pages into higher-order blocks

Operation	What changes	Primary goal
reclaim	Removes reclaimable memory contents from RAM (for example clean cache pages, or anonymous pages after swap-out).	Create more free memory.
compaction	Moves movable allocated pages away from selected regions and groups free pages together.	Create physically contiguous free blocks without necessarily reducing total used memory.
NUMA page migration	Moves a page to another physical NUMA node while preserving the process virtual address.	Improve locality or enforce placement policy.
THP collapse/allocation	Needs sufficiently contiguous physical memory for a large folio/huge mapping.	Reduce TLB pressure; may trigger or benefit from compaction.
pinned / unmovable pages	Cannot simply be relocated at that moment.	Can obstruct compaction and contribute to fragmentation.

# Trigger system-wide compaction manually (root; diagnostic action) echo 1 | sudo tee /proc/sys/vm/compact_memory # Current compaction policy / aggressiveness cat /proc/sys/vm/compaction_proactiveness cat /proc/sys/vm/extfrag_threshold # NUMA placement clues for one process cat /proc/<PID>/numa_maps | less # Overall buddy free-block distribution by order cat /proc/buddyinfo

Compaction is not free. It consumes CPU time, copies memory and can create latency spikes. Linux therefore uses heuristics and supports proactive/background compaction rather than continuously packing every page as tightly as possible.

KERNEL DOCSLinux VM sysctls — compaction controls

Documents compact_memory, proactive compaction and the external-fragmentation threshold used when choosing compaction versus reclaim.

https://kernel.org/doc/html/latest/admin-guide/sysctl/vm.html

PUBLIC MANUALmigrate_pages(2)

User-visible NUMA page migration: move a process's pages between node sets without changing its virtual address space.

https://man7.org/linux/man-pages/man2/migrate_pages.2.html

Two processes can temporarily share one anonymous physical page: KSM deduplicates identical contents and restores privacy with COW

Copy-on-write after fork() begins with pages that are already known to be shared. Kernel Samepage Merging (KSM) solves the harder reverse problem: discover independently allocated anonymous pages whose bytes happen to be identical, replace them with one shared write-protected page, and split them again if somebody writes. Applications opt eligible mappings into this mechanism; the ksmd scanner then spends CPU time comparing candidate pages to save RAM.

PROCESS A anonymous page PA = [same bytes] PROCESS B anonymous page PB = [same bytes] ↓ both VM ranges marked MADV_MERGEABLE ↓ ksmd scans candidate anonymous pages ↓ contents identical? no → keep separate candidates yes ↓ replace PTEs so both mappings reference ONE KSM page ↓ shared physical page is write-protected ↓ READ from A or B → same physical contents later PROCESS B writes ↓ write-protection fault ↓ allocate private page + copy contents ↓ B writes its private copy A keeps original shared page Result: memory saved while equal; semantic independence preserved on write.

Concept	What KSM actually does
eligibility	Targets selected anonymous/private memory ranges, not ordinary page-cache file pages.
`MADV_MERGEABLE`	Userspace opt-in telling the kernel this range may be scanned for KSM merging.
`ksmd`	Background scanner that searches registered ranges and compares page contents.
stable tree	Tracks already merged, write-protected KSM pages whose contents are stable enough to compare reliably.
unstable tree	Tracks unmerged candidates whose bytes can still change underneath the scanner.
COW break	A write fault recreates a private page so merging never changes the process-visible memory semantics.
tradeoff	Lower memory use versus CPU scanning cost, extra write faults and possible NUMA/locality/security-policy considerations.

KSM is not filesystem deduplication. It merges equal anonymous RAM pages after comparing their current contents. A reflink, by contrast, makes two files share filesystem extents intentionally. Both rely on copy-on-write to preserve independent writers, but they live at different layers and have different lifetime/accounting rules.

KERNEL DOCSLinux Kernel Samepage Merging design

Current kernel documentation for KSM's stable/unstable trees, write-protected merged pages, scanner behavior and internal design.

https://docs.kernel.org/mm/ksm.html

PUBLIC MANUALmadvise(2) — MADV_MERGEABLE / MADV_UNMERGEABLE

Current userspace memory-advice interface, including the Linux-specific controls applications use to make anonymous ranges eligible for KSM.

https://man7.org/linux/man-pages/man2/madvise.2.html

One TLB entry can map megabytes: Transparent Huge Pages, khugepaged, splitting and HugeTLB

A huge page reduces page-table depth/overhead and TLB pressure by mapping much more memory per translation. Linux has two major mechanisms with different semantics: Transparent Huge Pages (THP), which the VM can promote/demote automatically, and HugeTLB, which manages a more explicit reserved pool of huge pages.

ASSUME 4 KiB BASE PAGE + 2 MiB PMD-SIZED HUGE PAGE 512 × 4 KiB PTE mappings 512 translation granules potentially many TLB entries ↓ promote 1 × 2 MiB huge mapping one PMD leaf mapping covers same 512 base pages ↓ fewer TLB entries + fewer page-table entries/walks THP FAULT-TIME PATH anonymous VMA allows THP faulting address falls in suitable aligned hugepage region ↓ try high-order huge-page allocation / zeroing ├── success → install huge PMD/PTE-size mapping └── fail/fragmented → gracefully fall back to ordinary pages KHUGEPAGED BACKGROUND COLLAPSE process already has many populated 4 KiB pages ↓ khugepaged scans eligible VMAs ↓ find sufficiently suitable/aligned region ↓ allocate/collapse into huge folio/page replace many small PTE mappings with one huge mapping SPLITTING / DEMOTION operation cannot handle huge mapping directly or reclaim/migration/COW/mprotect pattern needs finer granularity ↓ split huge mapping and/or huge folio into smaller mappings/pages TRADEOFF TLB/walk win vs larger fault allocation + zero/copy cost internal memory waste if only tiny portion is touched more difficult contiguous allocation under fragmentation HUGETLB explicit huge-page pool/reservations mmap via hugetlbfs or MAP_HUGETLB ↓ pages are managed separately from ordinary pageable memory/THP policy ↓ useful when application wants deterministic huge-page availability/control

Huge-page concept	Meaning
THP	Transparent Huge Page: VM-managed large mapping/folio that can be allocated, promoted, split or demoted without explicit hugetlb reservation.
PMD-sized THP	Common huge mapping at page-middle-directory level; often 2 MiB on x86 with 4 KiB base pages.
PTE-mapped THP	Large folio whose individual base-page PTEs remain installed rather than one huge PMD entry.
khugepaged	Background kernel thread that scans eligible mappings and collapses suitable smaller pages into THPs.
collapse	Promotion/replacement of many smaller populated pages/mappings with a larger huge page/folio.
split	Break a huge mapping and/or huge folio into smaller units when fine-grained VM operations require it.
madvise MADV_HUGEPAGE	Application hint favoring THP for a mapping under applicable system policy.
madvise MADV_NOHUGEPAGE	Application hint asking the kernel not to back a mapping with THP.
HugeTLB	Explicit huge-page subsystem backed by reserved/persistent pools and hugetlbfs/MAP_HUGETLB mappings.
hugetlbfs	Pseudo-filesystem used to create mappings backed by HugeTLB pages.
TLB reach	Total memory addressable by current TLB entries; huge pages increase reach per entry.
internal fragmentation	Memory wasted inside a large page when the application uses only part of the mapped huge region.

THP is designed to fail gracefully. Current Linux documentation explicitly says fragmentation-driven huge-page allocation failure should fall back to regular pages without userspace failure, and khugepaged can later promote suitable regions when resources become available.

LINUX HUGE-PAGE OBSERVATION # THP global policy cat /sys/kernel/mm/transparent_hugepage/enabled 2>/dev/null cat /sys/kernel/mm/transparent_hugepage/defrag 2>/dev/null # khugepaged controls/stats grep . /sys/kernel/mm/transparent_hugepage/khugepaged/* 2>/dev/null | head -80 # one process's mappings grep -E '^(Size|Rss|Pss|AnonHugePages|ShmemPmdMapped|FilePmdMapped|THPeligible|VmFlags):' /proc/<PID>/smaps | less # explicit HugeTLB pool grep -E '^Huge|Hugetlb' /proc/meminfo # madvise(MADV_HUGEPAGE) in a test program lets you compare TLB/fault behavior # without changing global policy.

KERNEL DOCSLinux Transparent Hugepage Support

Current THP design and controls: automatic huge-page use, graceful fallback, khugepaged promotion, splitting/demotion, sysfs policy and madvise behavior.

https://docs.kernel.org/admin-guide/mm/transhuge.html

KERNEL DOCSLinux THP design principles

Current internal design notes emphasizing graceful fallback when huge allocation fails and background relocation/collapse through khugepaged.

https://docs.kernel.org/mm/transhuge.html

KERNEL DOCSLinux HugeTLB Pages

Current explicit huge-page subsystem documentation, including architecture page sizes, reserved pools, hugetlbfs and mmap usage.

https://docs.kernel.org/admin-guide/mm/hugetlbpage.html

What malloc() really gives you: virtual space first, physical pages on demand

malloc() is a user-space allocator interface, not a CPU instruction and not necessarily a system call per allocation. An allocator manages chunks inside larger regions obtained from the kernel with mechanisms such as brk() and mmap(). Those mappings define where the process may access memory; physical pages can be populated lazily when the process actually touches them.

application: p = malloc(1 GiB) ↓ userspace allocator searches free chunks / arena metadata ↓ if it needs more virtual memory brk() grows heap and/or mmap(MAP_ANONYMOUS) creates a mapping ↓ kernel records a VMA: virtual range + permissions + backing type ↓ malloc() may return BEFORE every page has a private physical frame FIRST READ of untouched anonymous page CPU VA → TLB/page tables → not yet privately populated ↓ page fault kernel may map a shared read-only ZERO PAGE ↓ read returns zeros FIRST WRITE store to page → write/protection fault ↓ kernel allocates physical page (PCP/buddy allocator path) ↓ zeroes/private-copies contents as required ↓ installs writable PTE for this process ↓ instruction retries → store now succeeds This is DEMAND / LAZY physical allocation.

Layer	Object / responsibility
C allocator	Splits/coalesces chunks, arenas/bins and decides when it needs more address space from kernel.
VMA	Kernel metadata for a virtually contiguous process range with common permissions/backing.
page table / PTE	Hardware-consumed mapping from virtual page to physical frame plus protection/status bits.
page fault handler	Kernel resolves legitimate missing/protected mappings or delivers a fault such as SIGSEGV when access is invalid.
zero page	Shared physical all-zero page that can satisfy untouched anonymous reads without allocating a private page.
physical page allocator	Chooses free page frames from per-CPU page caches and/or buddy-managed zone/node free lists.
NUMA policy	Influences which node supplies the physical page when first-touch allocation occurs.
overcommit policy	Controls how aggressively kernel promises virtual memory relative to RAM+swap capacity.

Virtual address space, committed memory and resident RAM are different quantities. A process can have a very large mapping while only a subset of its pages are resident in physical RAM. This is why tools such as /proc/<pid>/maps and /proc/<pid>/smaps report different kinds of size/usage.

KERNEL DOCSLinux kernel — Process Addresses / VMAs

Current kernel documentation: each user-space memory range is tracked as a VMA in an mm_struct address space, independently from whether every virtual page currently has a PTE/physical page.

https://docs.kernel.org/mm/process_addrs.html

KERNEL DOCSLinux kernel — memory-management concepts

Current overview of anonymous memory, page cache, nodes, zones, huge pages, reclaim and the shared zero-page optimization for untouched anonymous mappings.

https://docs.kernel.org/admin-guide/mm/concepts.html

PUBLIC MANUALmalloc(3) — glibc/Linux behavior

Current manual page. Explains that glibc typically obtains allocator arenas with brk()/mmap() and that Linux uses optimistic memory allocation by default.

https://man7.org/linux/man-pages/man3/malloc.3.html

PUBLIC MANUALmmap(2) — anonymous/file mappings

Current Linux man page for creating virtual mappings, permissions and MAP_ANONYMOUS behavior.

https://man7.org/linux/man-pages/man2/mmap.2.html

KERNEL DOCSLinux — Physical Memory / buddy allocator

Detailed current documentation for NUMA nodes, zones, per-CPU pagesets and buddy free_area blocks, including recursive split/merge behavior.

https://docs.kernel.org/next/mm/physical_memory.html

Between malloc() and mmap(): chunks, arenas, thread caches and fragmentation

The kernel allocates/maps pages; malloc() usually manages much smaller objects inside those mapped regions. A user-space allocator therefore keeps metadata describing chunks, reuses freed blocks, splits larger blocks, coalesces neighbors where possible and maintains size-segregated free structures. Multithreaded allocators also use per-thread caches and/or multiple arenas to reduce lock contention.

PROCESS VIRTUAL ADDRESS SPACE arena / heap mapping obtained with brk()/mmap() ┌──────────────────────────────────────────────────────────────┐ │ chunk A: allocated 24 B │ metadata/alignment │ │ ├─────────────────────────┴────────────────────┤ │ │ free chunk B: reusable space │ │ ├──────────────────────────────────────────────┤ │ │ chunk C: allocated 4 KiB │ │ ├──────────────────────────────────────────────┤ │ │ top/remainder/free space │ │ └──────────────────────────────────────────────┴──────────────┘ malloc(80) ↓ round for alignment + allocator metadata/size class check per-thread cache / appropriate free structure ├── suitable free chunk exists → remove/split → return user pointer └── no suitable chunk → refill from arena or request more virtual memory free(pointer) ↓ recover allocator chunk metadata place in thread cache/free structure possibly coalesce adjacent free chunks possibly trim/release sufficiently large top/mmap regions to kernel INTERNAL FRAGMENTATION request 33 B → allocator may reserve a larger aligned/size-class chunk unused bytes INSIDE allocated chunk EXTERNAL FRAGMENTATION free space total = 1 MiB, but scattered as many small holes a single 512 KiB request may still not fit one reusable contiguous allocator chunk RSS DOES NOT HAVE TO FALL WHEN free() RETURNS allocator can keep freed chunks cached/inside an arena for future mallocs; only memory actually unmapped/trimmed/advised away becomes immediately reclaimable by the kernel in the strongest sense.

Allocator concept	Meaning
chunk	Allocator-managed block containing user payload plus implementation-specific metadata/alignment.
arena	Allocator heap state/free structures and backing mappings; multiple arenas can reduce cross-thread lock contention.
tcache	glibc per-thread cache for selected freed chunk sizes, avoiding arena locking on many fast paths.
size class/bin	Grouping of free chunks by size/range so suitable blocks can be found without scanning all free memory.
split	Use part of a larger free chunk for a request and leave a smaller remainder free.
coalesce	Merge adjacent free chunks to make a larger reusable region.
internal fragmentation	Space reserved inside an allocated chunk but unused by the requested payload.
external fragmentation	Free memory exists but is divided among holes/chunks that do not satisfy a large request efficiently.
mmap threshold	Allocator policy where sufficiently large requests can use separate mmap-backed regions instead of the normal arena heap.
trim threshold	Policy controlling when releasable top-of-arena memory may be returned to the operating system.
allocator contention	Multiple threads serializing on shared allocator metadata; arenas/thread caches reduce but can increase retained memory.
malloc_info()	glibc diagnostic API exporting allocator/arena state as XML for inspection.

Do not memorize one glibc bin layout as the definition of malloc. Allocator internals and defaults change by release and other libc/allocators use different designs. The stable abstraction is that malloc manages user-space sub-page allocations on top of OS virtual-memory mechanisms.

GLIBC OBSERVATION LAB # trace when allocator actually asks kernel for address-space changes strace -e brk,mmap,munmap,madvise ./allocation_test # compare many tiny allocations with one huge allocation # malloc calls will vastly outnumber mmap/brk syscalls in normal allocator use # malloc_info() can dump current arena state as XML # mallinfo2() exposes summary counters, but not every allocation is represented perfectly # Compare /proc/<pid>/smaps RSS before/after free(). # Freed user objects can remain in allocator caches/arenas rather than instantly disappearing from RSS.

CURRENT GLIBC DOCSGNU C Library — current malloc tunables

Current manual documents arenas, mmap/trim thresholds and per-thread tcache controls. It is useful precisely because it also shows which details are implementation/version tunables rather than malloc ABI guarantees.

https://sourceware.org/glibc/manual/latest/html_node/Memory-Allocation-Tunables.html

SOURCE / FILEglibc malloc.c — browsable current source

Real allocator source exposing malloc_state arenas, per-thread tcache structures, bin logic, chunk metadata, mmap accounting, splitting/coalescing and corruption checks.

https://codebrowser.dev/glibc/glibc/malloc/malloc.c.html

PUBLIC MANUALmalloc_info(3)

Current 2026 Linux manual page for exporting glibc allocator state, including information about all arenas, as XML.

https://man7.org/linux/man-pages/man3/malloc_info.3.html

PUBLIC MANUALmallinfo2(3)

Current allocation-summary interface and caveats; explicitly recommends malloc_info() when per-arena visibility is needed.

https://man7.org/linux/man-pages/man3/mallinfo.3.html

fork() does not copy every byte of RAM: copy-on-write makes the illusion cheap

After fork(), parent and child must behave as if they have separate memory spaces with identical initial contents. Linux normally does not eagerly duplicate every anonymous page. Instead, both processes initially reference the same physical pages while mappings are arranged so a later write triggers copy-on-write (COW).

BEFORE fork() parent VA 0x4000 ──PTE──→ physical page P [value = 7] AFTER fork() parent VA 0x4000 ──PTE (not privately writable) ─┐ ├→ same physical page P [7] child VA 0x4000 ──PTE (not privately writable) ─┘ reference/map counts record sharing Both READ → 7 with no copy child WRITES 9 ↓ write-protection page fault kernel recognizes legitimate COW mapping ↓ allocate new physical page Q copy old page P → Q change child's PTE → Q, writable retry store → Q now contains 9 parent PTE → P [7] child PTE → Q [9] The processes now diverge only on pages actually written.

Operation	Memory consequence
fork()	Creates child task/address-space metadata and copies page tables/related structures; physical data pages are largely shared COW initially.
read shared COW page	No private copy required.
write shared COW page	Fault allocates/copies a private physical page for the writing process.
MAP_SHARED page	Writes are intended to remain shared; COW semantics differ from MAP_PRIVATE.
execve() soon after fork()	Old inherited address space is discarded, which is why fork+exec can avoid copying most user memory.
page-table copy cost	Even with COW data pages, duplicated/managed page-table structures and task metadata still cost time/memory.

PUBLIC MANUALfork(2) — current Linux manual

Current man page explicitly states that Linux fork uses copy-on-write pages; initial cost is chiefly duplicating page tables and creating the child task structure rather than copying all process RAM.

https://man7.org/linux/man-pages/man2/fork.2.html

When RAM fills up: reclaim, page cache eviction, swap, huge pages and the OOM killer

Physical RAM is a cache/resource managed continuously by the kernel. When free-page watermarks fall, Linux tries to recover reusable pages: clean file-backed cache can often be dropped, dirty file pages may be written back, and anonymous memory may be swapped/compressed depending on configuration. If the kernel cannot free enough usable memory, allocation eventually fails or the OOM machinery selects a process to terminate.

memory allocation request ↓ per-CPU pages / buddy free lists have enough? ── yes → allocate │ no ↓ memory pressure / watermarks ↓ background kswapd and/or direct reclaim ↓ identify colder/reclaimable memory ├── clean file cache → drop page; reload from file later if needed ├── dirty file cache → write back then reclaim ├── anonymous page → swap/zswap if configured and eligible └── movable page → migrate/compact for contiguous allocation ↓ sufficient memory recovered? ── yes → retry allocation │ no ↓ OOM handling / allocation failure according to context and policy LATER ACCESS TO RECLAIMED PAGE PTE says not resident / swap entry / file mapping ↓ page fault reload page from backing file or swap ↓ install mapping + continue

Mechanism	Benefit	Cost/tradeoff
file page-cache eviction	Quickly frees clean cached file data because backing copy already exists.	Future access causes storage read/page fault.
writeback	Makes dirty cached file data reclaimable.	Consumes storage bandwidth and adds latency.
swap	Moves anonymous/private memory contents out of DRAM.	Future fault may require slow storage I/O.
zswap/compressed memory	Trades CPU/compressed RAM for fewer backing-device swap writes.	Still consumes memory and compression CPU.
memory compaction	Creates larger physically contiguous free extents.	Moves pages and consumes CPU/memory bandwidth.
Transparent Huge Pages	Larger translations reduce TLB pressure and page-table overhead for suitable mappings.	Fault/copy/fragmentation costs can be larger; not universally beneficial.
OOM kill	Frees memory when reclaim cannot satisfy demands.	Terminates a process; last-resort survival mechanism.
overcommit	Allows sparse/lazy applications to reserve more virtual memory than immediate physical capacity.	Successful allocation call does not always guarantee future physical memory availability.

A page fault is not inherently an error. Demand-zero allocation, file-backed demand paging, copy-on-write and swap-in all deliberately use faults as normal control transfers into the kernel. A segmentation fault is the outcome when the access cannot be legally resolved, not another name for every page fault.

KERNEL DOCSLinux Multi-Gen LRU

Current reclaim implementation documentation describing modern working-set aging and eviction behavior.

https://docs.kernel.org/admin-guide/mm/multigen_lru.html

KERNEL DOCSLinux VM sysctls — overcommit and OOM controls

Current documentation for vm.overcommit_memory and OOM policy, including strict versus permissive overcommit behavior.

https://docs.kernel.org/next/admin-guide/sysctl/vm.html

PUBLIC MANUALLinux pagemap — inspect page presence/swap state

Low-level no-login interface showing whether virtual pages are present, swapped, file/shared, exclusive or soft-dirty, subject to kernel security restrictions.

https://man7.org/linux/man-pages/man5/proc_pid_pagemap.5.html

malloc() can promise address space before RAM exists: Linux overcommit separates virtual-memory commitment from current physical residency

Demand paging means a successful allocation need not immediately consume one physical page for every virtual page requested. Linux therefore has a separate notion of commit accounting: how much potentially writable private/anonymous memory the system has promised processes versus how much backing capacity policy allows it to promise. This is different from RSS, free RAM, page-cache size and the amount of swap currently occupied.

process: malloc(4 GiB) ↓ libc may extend heap / create anonymous mapping kernel creates virtual address range ↓ commit-accounting policy checked as required ↓ allocation call succeeds pages may still have NO physical RAM behind them yet ↓ first write to one page page fault → allocate/zero physical page → install writable PTE ↓ RSS grows only for pages actually made resident vm.overcommit_memory = 0 heuristic policy (default) vm.overcommit_memory = 1 always allow overcommit until resources run out vm.overcommit_memory = 2 strict commit accounting against CommitLimit

Quantity	Meaning
virtual address space	Ranges present in a process's address map; a range can exist without every page being resident.
RSS	Pages currently resident for the process; this is not the same as total address space or committed promise.
`Committed_AS`	Kernel accounting estimate of memory committed to satisfy allocations if processes actually use the promised writable memory.
`CommitLimit`	System commit ceiling used by strict mode; derived from configured RAM contribution plus swap, with HugeTLB reservations accounted for.
OOM	Actual inability to satisfy an allocation after reclaim and other mechanisms; OOM policy is related to but not identical with commit accounting.
cgroup memory limit	A separate scoped resource limit; a process can hit a memory-cgroup limit even when global commit accounting would allow more.

“malloc succeeded” is not always a physical-memory reservation. In the default/always-overcommit modes, a process may receive a large virtual mapping and fail later when touching pages if the system cannot provide backing. Strict mode 2 moves more failures to allocation time, but it does not turn every memory failure into a simple CommitLimit comparison: cgroup limits, kernel memory needs and other resource constraints still exist.

OBSERVATION LAB cat /proc/sys/vm/overcommit_memory cat /proc/sys/vm/overcommit_ratio cat /proc/meminfo | grep -E 'CommitLimit|Committed_AS|MemAvailable|Swap' # Compare address space with resident pages for a process: grep -E 'VmSize|VmRSS' /proc/<PID>/status cat /proc/<PID>/smaps_rollup

KERNEL DOCSLinux — Overcommit Accounting

Authoritative description of overcommit modes 0/1/2, strict commit accounting, which mapping types consume commit and the role of vm.overcommit_memory.

https://docs.kernel.org/mm/overcommit-accounting.html

PUBLIC MANUALproc_meminfo(5) — CommitLimit and Committed_AS

Current Linux manual for the memory counters exported in /proc/meminfo, including the strict-overcommit CommitLimit and the system-wide Committed_AS promise.

https://man7.org/linux/man-pages/man5/proc_meminfo.5.html

Swapping an anonymous page replaces a RAM mapping with a location token: swap entries, zswap and zram are different layers

Anonymous memory has no file to reread after eviction, so reclaim needs somewhere to preserve its contents. Linux can assign the page a swap entry—conceptually a swap type plus an offset—write the page to a swap backend, and replace the process PTE with non-present metadata that identifies that entry. A later access faults, the kernel finds or reads the saved contents, installs a resident page again, and resumes the instruction. Compression can intercept this path, but zswap and zram do so differently.

MEMORY PRESSURE ↓ reclaim selects cold anonymous folio ↓ allocate SWAP ENTRY = {swap type, swap offset} ↓ try to preserve page contents ├── zswap enabled + page accepted │ compress into RAM pool │ map swap entry → compressed object │ backing swap slot still defines the identity │ └── ordinary swap write submit I/O to swap file / swap block device ↓ replace resident PTE with non-present swap entry physical page can now be reclaimed LATER CPU ACCESS ↓ page fault PTE identifies swap entry ↓ look for cached/in-flight folio ↓ zswap hit? → decompress into RAM │ no ↓ read swap slot from backing device ↓ restore PTE → retry instruction ZRAM CASE /dev/zramN is itself a compressed RAM-backed BLOCK DEVICE ↓ it can be formatted as swap ↓ normal swap I/O goes to that compressed in-memory device (no separate disk backing is required)

Mechanism	Where data lives	Important distinction
swap entry	Metadata encoding a swap type and offset.	It is not a physical pointer; it identifies where an evicted anonymous page can be recovered.
swap file / partition	Persistent or block-backed storage.	Acts as the ordinary backing store for swapped anonymous pages.
swap cache/table state	Kernel metadata associating a swap entry with an in-memory or in-flight folio/shadow state.	Coordinates swap-in/swap-out and avoids treating the disk slot as the only state.
zswap	Compressed pool in RAM in front of a real swap backend.	A cache for pages on their way to swap; when its pool fills, entries can be evicted to the backing swap device.
zram	Compressed RAM exposed as a block device such as /dev/zram0.	Can itself be used as a swap device. It is not merely a cache in front of a disk swap device.
swapoff	Moves/invalidates pages so the selected swap area can be disabled.	Can require substantial free RAM because swapped pages must become resident or migrate elsewhere.

Swap does not extend physical address space. The CPU still executes only against resident RAM mappings. Swap provides backing storage so the kernel can evict recoverable anonymous contents and fault them back later.

KERNEL DOCSLinux swap table and swap-entry internals

Current kernel documentation describing swap entries as swap type + offset and the swap-cache/table states used while pages move between memory and a swap backend.

https://docs.kernel.org/mm/swap-table.html

KERNEL DOCSLinux zswap — compressed cache for swap pages

Explains how zswap intercepts swap-out, compresses accepted pages into an in-memory pool, maps swap entries to compressed objects, and evicts to the backing swap device when needed.

https://docs.kernel.org/admin-guide/mm/zswap.html

KERNEL DOCSLinux zram — compressed RAM-backed block devices

Defines /dev/zram devices and their compressed in-memory storage model. A zram block device can be configured as swap, which is distinct from zswap's cache-in-front-of-swap design.

https://docs.kernel.org/admin-guide/blockdev/zram.html

CURRENT PUBLIC MANUALswapon(8)/swapoff(8) — enable and inspect swap areas

Current util-linux manual for enabling swap files/devices, assigning priorities, showing active swap and disabling swap areas.

https://man7.org/linux/man-pages/man8/swapon.8.html

7. Memory hierarchy: SRAM/DRAM concepts, caches and virtual memory

PUBLIC NOTESBerkeley CS61C — cache terminology and locality

Explains locality, cache blocks/lines, hits, misses, tags, and why small fast memory in front of larger slow memory works.

https://notes.cs61c.org/content/caches-ii/

PUBLIC NOTESBerkeley CS61C — virtual memory and pages

Explains virtual versus physical addresses, address translation, pages, protection, and the role of main memory relative to storage.

https://notes.cs61c.org/content/vm/

What a storage drive is doing below the filesystem

A filesystem's 'block' is an abstraction. In a hard disk, the controller must position a magnetic head over rotating media and recover a coded signal. In an SSD, the controller maps logical block addresses onto NAND pages/blocks, performs error correction, wear leveling and garbage collection, and moves data between flash channels and host interfaces.

DIRECT PDFIBM Redbook — hard disk mechanics (PDF)

A freely available IBM book with a concise but concrete disk-mechanics section: moving head, spinning platter, sequential transfer, seek time and rotational latency.

https://www.redbooks.ibm.com/redbooks/pdfs/sg245050.pdf

DIRECT PDFIBM technical paper — magnetic recording channel (PDF)

Treats write head + magnetic medium + read head as a communications channel. Explains binary write-current waveforms, magnetization and readback/data detection.

https://dominoweb.draco.res.ibm.com/reports/rz3456.pdf

WHITE PAPERIBM — hard-drive media and defect management

Describes HDA, spindle motor, platters, heads, actuator, bit cells, areal density, manufacturing defects and defect remapping.

https://www.ibm.com/support/pages/understanding-hard-drive-media-defects-white-paper-servers

TECH ARTICLEWhat the Flash Translation Layer does

Clear description of SSD logical-to-physical mapping, NAND die/plane/block/page organization, garbage collection, wear leveling and bad-block management.

https://rossmanngroup.com/technical-reference/what-the-flash-translation-layer-does

SCSI is a command/status model, not one cable: initiator → CDB → target/LUN → status + sense

SCSI defines a client/server command model used by many storage stacks. An initiator sends a Command Descriptor Block (CDB) to a target and a selected Logical Unit (LUN). Data may move with the command, and the target returns status. On a CHECK CONDITION result, structured sense data explains what happened. Different transports can carry this same conceptual command model, which is why Linux has a SCSI midlayer independent of any single physical bus.

FILESYSTEM / BLOCK LAYER REQUEST ↓ SCSI DISK UPPER LAYER (sd) translate request into SCSI command ↓ CDB: opcode + LBA/range + flags ↓ SCSI MIDLAYER queueing / routing / timeout + error handling ↓ LOW-LEVEL TRANSPORT DRIVER examples: SAS HBA, USB mass-storage/UAS path, iSCSI transport ↓ TARGET → selected LUN ↓ command phase optional DATA-OUT (write) or DATA-IN (read) ↓ STATUS GOOD → request completes CHECK CONDITION→ inspect returned SENSE DATA ↓ sense key + additional sense information may classify not-ready, medium error, illegal request, unit attention, etc. ↓ upper layers decide retry / fail / reset / report error

SCSI object	Meaning
initiator	Endpoint that originates SCSI commands.
target	Server-side SCSI endpoint that exposes one or more logical units.
LUN	Logical Unit Number selecting a logical device/object behind a target.
CDB	Command Descriptor Block carrying the operation and parameters such as logical block address and transfer length.
status	Final command-level outcome such as GOOD or CHECK CONDITION.
sense data	Structured diagnostic information returned for a failed/exceptional command, richer than a generic “I/O error.”
SCSI midlayer	Linux layer between upper device classes and transport-specific low-level drivers; manages commands, queueing and error handling.

Do not equate SCSI with the old parallel ribbon cable. The SCSI architecture model survived because the command/status abstraction can ride over very different transports. iSCSI, for example, carries SCSI commands over TCP rather than a local storage cable.

KERNEL DOCSLinux SCSI interfaces guide

Current kernel documentation for the three-layer SCSI design: upper device classes, the SCSI midlayer and transport/hardware-specific low-level drivers.

https://docs.kernel.org/driver-api/scsi.html

KERNEL DOCSLinux SCSI midlayer ↔ low-level driver interface

Detailed command structure, result/status fields, DMA direction, CHECK CONDITION handling and sense-buffer/autosense behavior.

https://docs.kernel.org/scsi/scsi_mid_low_api.html

KERNEL DOCSLinux SCSI error handling

Shows how timed-out or failed SCSI commands move through completion and error-recovery machinery instead of simply becoming an opaque error code.

https://docs.kernel.org/scsi/scsi_eh.html

STANDARDS TRACK RFCRFC 7143 — iSCSI: carrying SCSI over TCP

Concrete example proving that SCSI is a protocol model rather than a cable: initiator/target commands, data and status are mapped into iSCSI PDUs over TCP.

https://www.rfc-editor.org/info/rfc7143/

SD and eMMC are managed block-storage devices: host controller → card protocol → internal flash controller

An SD card or eMMC package does not expose raw NAND cells directly to the operating system. The host talks a standardized command/data protocol through an MMC/SD host controller. Linux discovers card capabilities, negotiates bus parameters, creates block devices such as /dev/mmcblk0, and submits logical-block requests. Inside the device, a controller manages flash translation, ECC, bad blocks, wear and other media-specific work—similar in principle to an SSD, though with different interfaces and feature sets.

APPLICATION / FILESYSTEM ↓ Linux block layer ↓ mmc block driver → /dev/mmcblkN ↓ MMC/SD core identify card + read capability registers choose bus width / signaling / clock supported by host+card ↓ HOST CONTROLLER (for example SDHCI-class hardware) command register + DMA/data engine ↓ pins / package interface CLK + CMD + DATA lines ↓ SD CARD or eMMC DEVICE protocol engine ↓ logical sectors / special partitions ↓ internal flash controller FTL-like mapping + ECC + bad-block/wear management ↓ NAND flash arrays eMMC commonly also exposes device-defined areas such as: user area boot partitions (boot0 / boot1) RPMB authenticated replay-protected area and configuration state readable through EXT_CSD.

Term	What it means
SD	Removable card family using the SD/MMC-style command/data interface and card-defined capability registers.
eMMC	Embedded managed flash package using the MMC protocol family, typically soldered to a board and exposing logical blocks plus device-management features.
MMC/SD host controller	SoC/PCI hardware that generates command/clock/data signaling and usually DMA-transfers payloads between system RAM and the card/device.
CID/CSD/SCR	Identification/capability registers used to learn card identity and supported behavior; exact register set differs across SD/MMC families.
EXT_CSD	Extended eMMC configuration/status data containing device capabilities and controls such as partitioning, cache and reliability-related features.
boot partitions	Special eMMC logical areas often used by early boot firmware; Linux exposes them separately and protects writes by default.
RPMB	Replay Protected Memory Block: authenticated storage with a write counter, intended for small security-sensitive state rather than general files.

The block number is not a NAND address. Software submits logical sectors. The managed-flash controller decides where those bytes physically live and can move them later for wear leveling, garbage collection or bad-block handling.

KERNEL DOCSLinux MMC/SD/SDIO subsystem documentation

Current public kernel documentation for the MMC/SD stack, including block-device attributes, partitions, asynchronous requests and tooling.

https://docs.kernel.org/driver-api/mmc/index.html

KERNEL DOCSLinux SD/MMC device partitions

Explains separately exposed MMC boot partitions such as /dev/mmcblkXboot0/boot1 and the protective read-only policy around boot-critical contents.

https://docs.kernel.org/driver-api/mmc/mmc-dev-parts.html

KERNEL DOCS / TOOL GUIDEmmc-utils guide — EXT_CSD, partitions, cache, RPMB and device state

Practical public guide to inspecting and controlling eMMC features including EXT_CSD, boot/general-purpose partitions, cache, sanitize, write reliability and RPMB operations.

https://docs.kernel.org/driver-api/mmc/mmc-tools.html

Software RAID is a block-device state machine: stripe/mirror data, survive selected failures, then rebuild redundancy

Linux md can combine several component block devices into one logical /dev/md* device. Filesystems above it issue ordinary block reads and writes; md maps those logical sectors onto mirrors, stripes and parity according to the RAID level. The important mental model is that RAID changes placement and redundancy below the filesystem. It does not know which files are important, and it is not a substitute for backups or filesystem-level crash-consistency rules.

FILESYSTEM / DATABASE / RAW BLOCK USER ↓ logical block I/O /dev/md0 — Linux md array ↓ map logical sector according to RAID layout RAID1 example WRITE block X ├── member A: X └── member B: X READ can use either healthy mirror RAID5 example (conceptual stripe) member A member B member C DATA D0 DATA D1 PARITY P = D0 XOR D1 small overwrite of D0 may require: read old D0 + old P ↓ compute new parity from delta write new D0 + new P ONE MEMBER FAILS ↓ array becomes DEGRADED if level still has enough redundancy reads reconstruct missing data from surviving members/parity ↓ replacement/spare added REBUILD / RECOVERY scans stripes ↓ reconstruct missing member contents onto replacement ↓ array returns to fully redundant state

Layout / mechanism	Core tradeoff
RAID0	Stripes data across members for capacity/parallelism but provides no redundancy; losing one member loses array data.
RAID1	Mirrors the same data on multiple members; usable capacity is reduced but reads can come from any valid mirror.
RAID5	Distributed single parity; can normally tolerate one failed member, but small writes may need read-modify-write parity work.
RAID6	Two independent parity syndromes; can normally tolerate two failed members at the cost of more capacity and parity computation.
RAID10	Combines mirroring and striping; avoids parity updates but consumes mirror capacity and has topology-dependent failure tolerance.
degraded array	Array is still operating with one or more missing/failed members while enough redundancy remains to satisfy I/O.
rebuild / recovery	Reads surviving members and reconstructs missing redundancy onto a replacement/spare; this creates substantial background I/O and temporarily increases exposure to another failure.
write-intent bitmap	Tracks regions dirtied while redundancy may be incomplete so recovery after interruption can focus on affected areas rather than always resyncing everything.
RAID4/5/6 write hole	Crash/power loss during separate data/parity updates can leave a stripe internally inconsistent. md supports mechanisms such as journals or RAID5 PPL to reduce/close this hazard.
array metadata / superblock	Records membership, layout, event counters and array identity so mdadm/kernel can assemble the correct devices and detect stale members.

RAID availability is not backup. Mirroring/parity can keep a machine running through selected device failures, but deletion, ransomware, filesystem corruption, software bugs and many controller/site failures can affect every live member. Independent backup copies solve a different problem.

KERNEL DOCSLinux md RAID array documentation

Kernel documentation for md array states, metadata, synchronization/recovery controls, degraded operation and consistency policies such as bitmap, journal and PPL.

https://docs.kernel.org/admin-guide/md.html

KERNEL DOCSLinux RAID4/5/6 cache and write-hole handling

Explains parity-array write-through/write-back journaling, why non-full-stripe writes can require old-data reads, and how the cache closes the RAID write hole.

https://docs.kernel.org/driver-api/md/raid5-cache.html

CURRENT PUBLIC MANUALmdadm(8) — create, assemble, manage and reshape Linux MD arrays

Current userspace management reference for RAID levels, component/spare devices, degraded assembly, rebuilds, layout changes and array reshape operations.

https://man7.org/linux/man-pages/man8/mdadm.8.html

One SATA read through AHCI: command slot → FIS → PRDT → DMA → interrupt

AHCI is the standardized host-controller interface traditionally used by software to drive SATA devices. The controller is a PCI function with MMIO registers, while most command and data descriptions live in host RAM. Each port points to a Command List with 1–32 command slots; each command header points to a Command Table containing a command FIS and a Physical Region Descriptor Table (PRDT) for scatter/gather DMA.

ONE AHCI SATA READ (simplified) filesystem / block layer ↓ Linux libata / AHCI driver gets request ↓ choose free command slot/tag 0..31 ↓ COMMAND HEADER[slot] in host RAM ├── CFL = command-FIS length ├── W=0 for device → host read ├── PRDTL = number of scatter/gather entries └── CTBA = DMA address of COMMAND TABLE ↓ COMMAND TABLE ├── CFIS: Register Host-to-Device FIS │ contains ATA read command, LBA, count, tag/protocol fields └── PRDT[]: host DMA address + byte count for each segment ↓ driver sets PxCI bit for command slot for NCQ, PxSACT/tag state also participates ↓ AHCI HBA DMA-fetches command header/table/PRDs ↓ HBA emits SATA Register H2D FIS ↓ device reads media / schedules NCQ work ↓ SATA Data FIS(es) return sectors ↓ AHCI HBA DMA-writes data into PRDT-listed host pages ↓ completion/status FIS + HBA state update ↓ interrupt libata completes block request

AHCI/SATA object	Role
ABAR	PCI BAR containing AHCI global/per-port MMIO registers.
PxCLB/PxCLBU	Per-port pointer to the Command List in system memory.
Command List	Host-memory array of 1–32 command headers per port.
Command Header	Direction/type, PRDT length and pointer to the command's Command Table.
Command Table	Command FIS, optional ATAPI command area and PRDT.
PRDT	Scatter/gather list of DMA memory regions for transfer payload.
FIS	SATA Frame Information Structure carrying commands, data, setup and status.
PxCI	Port Command Issue bitmap; setting a slot bit issues that command.
PxSACT	SATA Active bitmap used for NCQ-tagged active commands.
Received FIS area	Host-memory area where the HBA deposits received device FISes.
NCQ	Native Command Queuing: tagged SATA command mechanism allowing multiple outstanding commands and device reordering.

AHCI descriptors are host-memory data structures; FISes are SATA protocol structures. The HBA bridges those worlds by fetching descriptors with PCI/DMA and generating/receiving SATA FIS traffic on the device link.

OFFICIAL SPECIntel — AHCI 1.3.1 specification page

Official public AHCI reference. AHCI is a PCI-class data-movement interface for SATA and supports per-port command lists and NCQ.

https://www.intel.com/content/www/us/en/io/serial-ata/serial-ata-ahci-spec-rev1-3-1.html

DIRECT SPEC PDFIntel — AHCI 1.3.1 direct PDF

Sections 4.2–4.2.3 show the Received FIS area, 1–32-entry Command List, Command Headers and Command Tables/PRDTs.

https://www.intel.com/content/dam/www/public/us/en/documents/technical-specifications/serial-ata-ahci-spec-rev1-3-1.pdf

KERNEL DOCSLinux libATA Developer's Guide

Shows how ATA requests are DMA-mapped, scatter/gather tables prepared, commands issued and completions propagated.

https://docs.kernel.org/driver-api/libata.html

SOURCE / FILESLinux ATA/AHCI source

Real ahci.c/libahci.c plus libata implementation for command issue, PxCI/NCQ handling, interrupts and completion.

https://github.com/torvalds/linux/tree/master/drivers/ata

Why NVMe looks different from AHCI: many queues close to CPUs instead of one shallow SATA queue

AHCI was designed around SATA/ATA devices and a shallow command queue. NVMe was designed for highly parallel nonvolatile memory over PCIe. Its software interface makes Submission Queues (SQs) and Completion Queues (CQs) first-class DMA structures so operating systems can map multiple CPU/block-layer hardware queues to multiple device queues.

AHCI / SATA CPU cores → Linux block layer → libata/AHCI ↓ per SATA port: one Command List with up to 32 slots ↓ AHCI HBA ↔ SATA link ↔ drive ↓ NCQ tags let the device reorder outstanding commands NVMe / PCIe CPU0 ─ blk-mq hctx ─→ I/O SQ0 ─┐ CPU1 ─ blk-mq hctx ─→ I/O SQ1 ─┼→ NVMe controller → flash channels/dies CPU2 ─ blk-mq hctx ─→ I/O SQ2 ─┘ NVMe submit path: host writes command into Submission Queue memory ↓ orders memory then writes SQ-tail doorbell MMIO ↓ controller fetches command + PRP/SGL-described data ↓ controller executes work in parallel ↓ controller writes Completion Queue entry ↓ MSI-X interrupt or host polling ↓ host consumes completion and advances CQ-head doorbell Linux blk-mq mirrors this: per-CPU/per-node software staging queues ↓ hardware dispatch queues ↓ driver/device submission queues

Property	AHCI/SATA	NVMe/PCIe
transport	SATA serial link	PCI Express for local NVMe; NVMe also defines other transports
command abstraction	AHCI command list + ATA/FIS protocol	Submission Queue entries + NVMe command sets
queue model	Up to 32 NCQ commands per SATA device/port	Many host-created I/O SQ/CQ pairs; actual count/depth negotiated with controller
completion	Port/FIS/status state + interrupt	Completion Queue entry + MSI/MSI-X or polling
scatter/gather	PRDT	PRP lists or SGLs
CPU scaling	Shallow per-port model	Designed to distribute queue pairs across CPUs/vectors
Linux path	libata/SCSI translation + block layer	native NVMe driver + blk-mq hardware queue mapping

NVMe's advantage is not merely higher PCIe line rate. The queue/interface model was redesigned around parallel solid-state media and multicore hosts. Linux blk-mq was likewise designed to remove single shared block-I/O queue bottlenecks.

CURRENT OFFICIAL SPECNVM Express — current Base Specification 2.4

Current official NVMe page: Base Specification 2.4 was ratified July 31, 2026 and released in the August 4, 2026 specification set.

https://nvmexpress.org/specification/nvm-express-base-specification/

CURRENT OFFICIAL SPECNVM Express — current NVMe over PCIe Transport 1.4

Current official PCIe transport page; revision 1.4 was ratified July 31, 2026.

https://nvmexpress.org/specification/nvme-over-pcie-transport-specification/

KERNEL DOCSLinux blk-mq

Explains the move from one locked request queue toward per-CPU software queues and multiple hardware dispatch queues for parallel SSDs.

https://docs.kernel.org/block/blk-mq.html

KERNEL DOCSLinux NVMe PCI endpoint target

Concrete queue mechanics: fetch Submission Queue commands, parse PRP/SGL lists, post Completion Queue entries and signal interrupts.

https://docs.kernel.org/next/nvme/nvme-pci-endpoint-target.html

SOURCE / HEADERSPDK NVMe specification definitions

Readable open-source definitions mirroring queue limits, commands, completions, controller registers and PRP/SGL formats.

https://github.com/spdk/spdk/blob/master/include/spdk/nvme_spec.h

NBD turns remote bytes into a local block device: filesystem → block layer → NBD requests → TCP → userspace server

A network filesystem such as NFS sends file-oriented RPC operations. Network Block Device (NBD) sits lower: Linux exposes a remote byte-addressable export as a local-looking block device such as /dev/nbd0. The client can then place an ordinary filesystem, partition table, LVM stack or other block consumer on top. From those upper layers, reads and writes look like normal block I/O; the NBD client translates them into protocol requests sent to a server.

APPLICATION read("/mnt/remote-disk/file") ↓ local VFS + local filesystem (ext4/xfs/etc. metadata is interpreted on CLIENT) ↓ local page cache / filesystem block mapping ↓ Linux block layer request to /dev/nbd0 ↓ NBD client driver ↓ request: READ offset=X length=Y ↓ socket / TCP NETWORK ↓ userspace NBD server ↓ server backing export may be: regular file local block device logical/virtual storage backend ↓ reply payload / status ↓ TCP → NBD client → block completion ↓ filesystem receives completed block I/O WRITE path is symmetric: filesystem dirty data → block write → NBD WRITE request → remote backing store Optional negotiated operations can include FLUSH, TRIM/discard and WRITE_ZEROES, depending on client/server/protocol capabilities.

Layer	NBD behavior
filesystem location	Usually on the client: ext4/XFS/etc. sees /dev/nbd0 as its block device and interprets inode/extent metadata locally.
server view	Exports a range of bytes; it need not understand the filesystem stored inside that range.
protocol unit	Offset/length block-style operations rather than pathname/open/read-directory RPCs.
transport	The classic Linux NBD client uses socket connections; the documented TCP version carries protocol negotiation and requests over TCP.
flush/FUA	Durability semantics must be negotiated and propagated correctly through the server/backing device; merely reaching the server process is not necessarily durable media persistence.
disconnect	A network/server failure becomes a storage failure from the mounted filesystem's perspective: I/O may stall, time out or fail.
multi-client access	NBD itself does not magically make an ordinary local filesystem safe for simultaneous independent mounting by multiple clients.
contrast with NFS	NFS server understands exported filesystem objects and file operations; NBD exports raw storage semantics and leaves the client filesystem to interpret bytes.

NBD moves the block-device boundary across the network. That means failures cross the boundary too. Packet loss/reconnect/server failure can surface as block I/O latency or errors, and filesystem crash-consistency assumptions still depend on correct flush/order behavior all the way through the remote storage stack.

OBSERVATION LAB (requires NBD userspace tools/server) # loaded NBD module/device nodes lsmod | grep '^nbd' ls -l /dev/nbd* 2>/dev/null # inspect block properties once connected lsblk -o NAME,TYPE,SIZE,FSTYPE,MOUNTPOINTS /dev/nbd0 cat /sys/block/nbd0/size 2>/dev/null # block-layer activity appears like other devices cat /sys/block/nbd0/stat 2>/dev/null # the server and negotiation tooling define the actual export, # access mode, timeout, flush/discard support and reconnect policy.

KERNEL DOCSLinux Network Block Device — kernel documentation

Concise kernel-side overview explaining how a remote server can appear to Linux as a block device whose read requests are sent over TCP and answered remotely.

https://www.kernel.org/doc/html/latest/admin-guide/blockdev/nbd.html

OPEN PROTOCOLNetwork Block Device protocol specification

Maintained protocol documentation covering negotiation, exports, requests/replies and modern NBD extensions.

https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md

A loop device makes bytes in a regular file look like sectors on a block device

A filesystem image is normally just a regular file: the VFS reads and writes it by pathname and file offset. A Linux loop device inserts a block-device facade in front of that file. Upper layers can then treat /dev/loopN like other block storage: scan a partition table, run mkfs, mount a filesystem, feed it to device mapper, or issue block I/O. The loop driver translates sector requests back into I/O against the backing file (or another block device).

regular file: disk.img (stored inside some backing filesystem) ↓ LOOP_CONFIGURE / losetup /dev/loopN ← block-device interface ↓ optional partition parser / dm layer ↓ filesystem mounted on /dev/loopN ↓ read/write filesystem blocks loop driver maps sectors → offsets in disk.img ↓ backing file I/O → backing filesystem/page cache/direct-I/O mode ↓ underlying block device → controller → storage

Thing	Not the same as a loop device
bind mount	Creates another pathname view of an existing mounted tree; it does not turn a file into a block device.
tmpfs/ramdisk	Provides memory-backed storage; a loop device can instead be backed by an ordinary file on any suitable filesystem.
NBD	Maps a remote block export to a local block device; loop maps a local file or block object.
`-o loop` in mount	User-space convenience that allocates/configures a loop device and then mounts it.
`LO_FLAGS_AUTOCLEAR`	Allows the loop association to disappear after the final opener closes it, simplifying lifecycle management.

Layering matters: a filesystem mounted from an image file can sit above a loop block device while that image file itself sits in another filesystem. That means caching, writeback, durability and error behavior can involve both the inner filesystem and the backing filesystem.

PUBLIC MANUALloop(4) — Linux loop block devices

Kernel/userspace ABI for associating a loop block device with a backing file, including modern LOOP_CONFIGURE, block size, autoclear, partition scanning and direct-I/O flags.

https://man7.org/linux/man-pages/man4/loop.4.html

PUBLIC MANUALlosetup(8) — configure and inspect loop devices

Current util-linux interface for finding free loop devices, associating backing files, setting offset/size/direct-I/O parameters, rescanning size and detaching devices.

https://man7.org/linux/man-pages/man8/losetup.8.html

NVMe: what happens when software asks an SSD to read blocks

NVMe is a storage command protocol designed around PCIe and host-memory queues. The controller is not handed one byte at a time. The host prepares command descriptors in RAM, rings a memory-mapped doorbell, the controller DMA-fetches work, transfers data, writes a completion entry, and may interrupt the CPU.

filesystem / block layer ↓ NVMe driver builds command in host RAM ↓ Submission Queue entry (SQE) ↓ CPU writes PCIe BAR doorbell register ↓ NVMe controller DMA-fetches command ↓ SSD controller translates logical blocks → NAND locations / internal flash operations ↓ controller DMA-writes requested data into host RAM ↓ Completion Queue entry (CQE) written to RAM ↓ MSI-X interrupt or polling ↓ driver consumes completion and advances CQ head doorbell

DIRECT SPEC PDFNVMe PCIe Transport Specification 1.0c (direct PDF)

Official free specification. Its command-processing figure directly shows host submission queue, controller doorbells, DMA fetching, completion queue and MSI-X interrupt flow.

https://nvmexpress.org/wp-content/uploads/NVM-Express-PCIe-Transport-Specification-1.0c-2022.10.03-Ratified.pdf

SOURCE / NOTESSPDK — NVMe command path

Open source documentation explaining 64-byte commands, submission queues, doorbells, phase bits and completion processing in implementation terms.

https://github.com/spdk/spdk/blob/master/doc/nvme_spec.md

DIRECT PDFCornell OS notes — NVMe read I/O path (direct PDF)

Public lecture PDF with a clear host-memory Submission/Completion Queue diagram and command/doorbell/interrupt sequence.

https://www.cs.cornell.edu/courses/cs4410/2021fa/assets/material/lecture24_blk_layer.pdf

NVMe Zoned Namespaces expose placement constraints to the host: random reads remain easy, but sequential-write zones have write pointers

A conventional block device pretends every logical block can be overwritten independently in arbitrary order. NAND flash cannot actually do that: erase granularity, garbage collection and relocation sit beneath the LBA interface. NVMe Zoned Namespaces (ZNS) deliberately expose a zoned interface instead. The namespace is divided into contiguous zones; sequential-write zones track a device-maintained write pointer, and hosts must write at the allowed position (or use zone-append semantics) before resetting a zone for reuse.

NVMe namespace +--------------------+--------------------+--------------------+ | zone 0 | zone 1 | zone 2 | ... +--------------------+--------------------+--------------------+ SEQUENTIAL-WRITE ZONE start LBA → written data → write pointer → unwritten capacity ^ next ordinary write belongs here random READs: allowed random overwrite behind write pointer: not the normal model reset zone: return write pointer to start / make space reusable zone append: device chooses the exact final LBA at the current append point Host software now participates in placement/lifetime organization that a conventional SSD would hide behind its FTL.

Interface	Where placement complexity lives
Conventional NVMe NVM namespace	Host issues ordinary block reads/writes; the SSD FTL absorbs random updates, mapping and garbage-collection work internally.
ZNS namespace	Host sees zones and sequential-write constraints, enabling software to align write streams/lifetimes with device behavior.
zone write pointer	Tracks the next permitted sequential write position for a sequential-write-required zone.
zone reset	Makes a zone reusable by returning it to an empty state rather than overwriting arbitrary old LBAs in place.
zonefs / zoned-aware filesystem or application	Provides software structures that obey/expose zone rules instead of pretending the device is an unrestricted random-write disk.

ZNS is not merely a faster NVMe queue. It changes the storage contract. The point is to reduce hidden device-side work such as write amplification, over-provisioning pressure and garbage-collection unpredictability by letting host software make more of the placement/reuse policy explicit.

OFFICIAL SPECNVM Express Zoned Namespaces Command Set

Official ZNS specification landing page. As of August 2026, the current ratified ZNS command-set revision is 1.5 (ratified 31 July 2026), alongside NVMe Base 2.4.

https://nvmexpress.org/specification/nvme-zoned-namespaces-zns-command-set-specification/

KERNEL DOCSLinux zonefs and zoned-block-device model

Concrete Linux explanation of conventional versus sequential zones, write pointers, reset/finish behavior and a filesystem that exposes each zone as a file rather than hiding the sequential-write constraint.

https://docs.kernel.org/filesystems/zonefs.html

NVMe over Fabrics keeps the NVMe queue/command model but moves the controller across a network fabric

Local NVMe normally reaches a controller over PCIe. NVMe over Fabrics (NVMe-oF) separates the NVMe subsystem/controller model from that local transport: a host connects to a remote NVMe subsystem identified by an NQN, creates an administrative controller relationship and I/O queues, then sends NVMe commands over a fabric transport such as TCP or RDMA. To the upper Linux block stack, the resulting namespaces can still look like NVMe block devices.

APPLICATION / FILESYSTEM ↓ Linux block layer / blk-mq ↓ NVMe host driver builds NVMe command ↓ NVMe Fabrics controller + queue pair ↓ transport mapping ├─ NVMe/TCP → TCP socket/network stack/NIC └─ NVMe/RDMA → RDMA queue pairs + registered memory ↓ NETWORK FABRIC ↓ REMOTE NVMe TARGET transport receives command capsule / data ↓ NVMe target core → subsystem → namespace backing ↓ completion travels back over fabric ↓ host NVMe completion queue → blk-mq completion DISCOVERY host contacts discovery controller ↓ learns subsystem NQN + transport addresses/services ↓ nvme connect -t tcp|rdma -n <subsystem NQN> -a <address> ... ↓ remote controller and namespaces appear on host

Object	Role
NQN	NVMe Qualified Name identifying a host or NVMe subsystem independent of an IP address or PCI bus location.
discovery controller	Special controller used to return records describing available NVMe subsystems and fabric endpoints.
NVMe/TCP	Maps NVMe-oF capsules/data onto ordinary TCP connections, allowing deployment on standard IP networks.
NVMe/RDMA	Maps NVMe-oF onto RDMA transports so command/data movement can use registered memory and RDMA operations.
remote namespace	Namespace exported by the target subsystem; after connection the host exposes it through the normal NVMe/block-device stack.
keep-alive / reconnect policy	Detects failed controller relationships and governs how the host attempts to restore a fabric connection.

NVMe-oF is not “a network filesystem.” It exports a remote block/storage controller interface. The client can place a local filesystem on the namespace, use it as a raw block device, or feed it to another storage layer. NFS/SMB instead export file-level operations and their own coherence semantics.

OFFICIAL SPECNVMe over TCP Transport Specification

Official current transport specification page. As of August 2026, NVMe/TCP Revision 1.3 is the ratified current revision and maps NVMe-oF onto TCP.

https://nvmexpress.org/specification/tcp-transport-specification/

OFFICIAL SPECNVMe over RDMA Transport Specification

Official transport specification for carrying NVMe over RDMA fabrics; the current specification family separates transport mappings from the NVMe base/controller model.

https://nvmexpress.org/specification/rdma-transport-specification/

PUBLIC MANUALnvme-connect(1) — create an NVMe Fabrics controller

Practical host-side interface showing transport type, subsystem NQN, transport address/service, queue counts, keep-alive, reconnect policy, authentication and TLS-related connection options.

https://manpages.debian.org/unstable/nvme-cli/nvme-connect.1.en.html

open() resolves a pathname once, then software holds a file descriptor to an open file description

A pathname and a file descriptor are different kinds of name. open()/openat() walk directory components through the VFS and dentry cache to select an inode/file object. On success the process receives a small integer file descriptor that indexes its descriptor table. That descriptor refers to an open file description (struct file) holding state such as the current file offset and status flags.

openat(AT_FDCWD, "/home/a/data.txt", O_RDONLY) ↓ choose starting point: filesystem root because path is absolute ↓ walk components: 'home' → dentry lookup/cache → inode must be directory 'a' → dentry lookup/cache → inode must be directory 'data.txt' → final-component lookup + permissions/open rules ↓ filesystem/VFS opens inode into struct file OPEN FILE DESCRIPTION f_path → mount + dentry f_inode → underlying inode f_pos → current read/write offset f_flags → O_APPEND/O_NONBLOCK/etc. status flags f_op → file_operations methods ↓ install pointer into process FILE-DESCRIPTOR TABLE fd table: 0 → terminal/open file description A 1 → same/different terminal description 2 → stderr description 3 → struct file for data.txt ← open() returns integer 3 read(3, buf, 100) fd=3 → table lookup → struct file → f_op/read_iter read starts at shared f_pos; successful read advances f_pos dup(3) → fd 4 fd3 ─┐ ├→ SAME open file description → SAME f_pos/status flags fd4 ─┘ fork() parent fd3 ─┐ ├→ SAME open file description child fd3 ─┘ shared offset/status flags separate open("data.txt") → NEW open file description with independent f_pos unlink("/home/a/data.txt") removes directory entry/name fd3 can still refer to the already-open inode/file until last reference closes execve() descriptors survive unless FD_CLOEXEC/O_CLOEXEC says close-on-exec

VFS/file object	Meaning
pathname	Sequence of directory components interpreted relative to root, cwd or a dirfd.
dentry	In-memory VFS object caching one directory-name component and its link to an inode.
inode	Filesystem object identity/metadata for file, directory, device node, etc.; does not itself contain a pathname.
mount/vfsmount	Identifies which mounted filesystem instance a dentry belongs to.
struct file / open file description	One opened instance holding current offset, status flags, path and operations.
file descriptor	Per-process small integer indexing a reference to an open file description.
fd flags	Descriptor-local flags such as FD_CLOEXEC; not the same as shared open-file status flags.
file status flags	Open-file-description state such as O_APPEND/O_NONBLOCK shared by dup/fork references.
openat()	Resolves a relative pathname from a supplied directory file descriptor rather than process cwd.
openat2()	Linux extension adding explicit path-resolution restrictions such as RESOLVE_BENEATH/IN_ROOT/NO_SYMLINKS.
RCU-walk	Fast pathname walk using RCU and lockless-ish cached dentry traversal where possible.
REF-walk	Reference-counted/locked pathname walk used when slow paths or unstable conditions require stronger synchronization.

An inode does not have one permanent pathname. Hard links can give one inode several directory names; a file can be renamed while open; it can even be unlinked completely while existing file descriptors continue using the open object.

LINUX FILE-DESCRIPTOR LAB # shell's current descriptors ls -l /proc/$$/fd cat /proc/$$/fdinfo/0 2>/dev/null # process-wide limits/count ulimit -n # watch pathname/open/dup behavior strace -e openat,openat2,close,dup,dup2,dup3,read,lseek ./program # demonstration idea: # open a file, unlink it from another shell, then keep reading via fd. # /proc/<pid>/fd/<n> will often display '(deleted)' while the open reference survives. # compare two separately opened fds versus dup()ed fds by lseek/read; # dup()ed descriptors share one current offset.

KERNEL DOCSLinux pathname lookup — current kernel documentation

Deep VFS path walk: absolute/relative roots, components, dentries, RCU-walk, REF-walk, symlinks and openat2 resolution restrictions.

https://docs.kernel.org/7.1/filesystems/path-lookup.html

KERNEL DOCSLinux VFS overview

Current VFS object model. A struct file represents an opened file/open file description and connects to file_operations.

https://docs.kernel.org/filesystems/vfs.html

PUBLIC MANUALopen(2) — current Linux manual

Current 2026 manual explicitly distinguishes file descriptor from open file description and documents shared file offsets/status flags across dup/fork references.

https://man7.org/linux/man-pages/man2/open.2.html

PUBLIC MANUALdup(2) — descriptor duplication

Shows that duplicated descriptor numbers point to the same open file description and therefore share current file offset/status flags but not descriptor-local close-on-exec state.

https://man7.org/linux/man-pages/man2/dup.2.html

A pathname is a lookup request, not a stable object: dirfd-relative lookup and openat2() make resolution constrainable

A pathname such as /srv/uploads/user/file is a sequence of names that the VFS must walk. Between a preliminary check and a later open(), another process can rename directories, replace a component with a symbolic link, or move a mount. That is why security-sensitive code should avoid “check this string, then open the same string” designs. The *at() interfaces let software anchor a relative path to an already-open directory file descriptor, and Linux openat2() adds explicit rules that constrain how the walk may proceed.

FRAGILE CHECK-THEN-USE SHAPE lstat("/srv/uploads/user/file") ↓ appears safe attacker renames / replaces a path component ↓ open("/srv/uploads/user/file") ↓ second pathname walk may reach a DIFFERENT object DIRFD-ANCHORED SHAPE open trusted directory → dirfd ↓ openat2(dirfd, "user/file", how) ↓ one constrained pathname walk VFS walks dentries / mountpoints relative to dirfd ↓ RESOLVE_* policy checked during the walk ↓ returns FILE DESCRIPTOR to the resolved open file description ↓ subsequent read/write/fstat operate on that opened object, not by re-resolving the original pathname string

Mechanism	What it constrains or stabilizes
directory file descriptor	Provides a stable starting directory object for relative `*at()` operations, avoiding dependence on the process current working directory and repeated prefix lookup.
`RESOLVE_BENEATH`	Rejects resolution that escapes above the supplied directory through absolute paths or parent traversal; useful when interpreting an untrusted relative path beneath a trusted tree.
`RESOLVE_IN_ROOT`	Treats the supplied directory as a temporary root for this lookup, including absolute path handling, without changing the process-wide root directory.
`RESOLVE_NO_SYMLINKS`	Rejects any symbolic-link component. This is stronger than merely refusing a final symlink.
`RESOLVE_NO_MAGICLINKS`	Rejects procfs-style “magic links” whose resolution semantics are more powerful than ordinary symlink text.
`RESOLVE_NO_XDEV`	Rejects crossing mount points, including bind mounts, during the lookup.
`RESOLVE_CACHED`	Requires the lookup to complete from cached VFS information; returns `EAGAIN` if blocking/revalidation would be needed.
returned file descriptor	Names the opened file description after lookup. Later renames/unlinks can change directory names without making that descriptor “follow” a new pathname target.

openat2() does not freeze the filesystem. It constrains one pathname walk and returns a stable handle to the object that was opened. Other names, metadata and directory structure can continue changing concurrently; robust code should keep using file descriptors for later operations instead of repeatedly converting trusted objects back into pathname strings.

PUBLIC MANUALopenat2(2) — current Linux manual

Current Linux-specific open interface with resolution controls including RESOLVE_BENEATH, RESOLVE_IN_ROOT and cached/no-symlink restrictions.

https://man7.org/linux/man-pages/man2/openat2.2.html

CURRENT PUBLIC MANUALpath_resolution(7) — how Linux walks pathname components

Current manual describing roots, dirfd-relative paths, symlink traversal, mount-point crossings, final-component rules and the openat2 resolution restrictions.

https://man7.org/linux/man-pages/man7/path_resolution.7.html

KERNEL DOCSLinux VFS pathname lookup — dentries, RCU-walk and rename races

Implementation-level explanation of the dcache, REF-walk/RCU-walk, concurrent rename protection and how openat2-style beneath/in-root constraints are defended during lookup.

https://docs.kernel.org/filesystems/path-lookup.html

“A file lock” can mean three different Linux ownership models: flock, POSIX record locks and OFD record locks

File locking sounds simple until fork(), dup(), threads and byte ranges enter the picture. Linux exposes multiple advisory-lock families whose crucial difference is what kernel object owns the lock. BSD-style flock() locks whole files and follows an open file description; traditional POSIX fcntl() record locks are process-associated byte-range locks; open-file-description (OFD) fcntl() locks keep byte ranges but attach ownership to the open file description.

pathname ↓ open() file descriptor ──────────────┐ ↓ │ dup()/fork() may create another fd OPEN FILE DESCRIPTION <───────┘ ↓ inode/file flock(fd, LOCK_EX) → whole-file advisory lock → associated with open file description → duplicate fds sharing that description share the lock → released when explicitly unlocked or last reference closes fcntl(fd, F_SETLK/F_SETLKW, byte range) → traditional POSIX process-associated record lock → byte-range read/write locks → surprising rule: closing a descriptor for that file can release process-associated locks on that file fcntl(fd, F_OFD_SETLK/F_OFD_SETLKW, byte range) → OFD byte-range lock → associated with open file description → fork/dup sharing naturally follows the same lock ownership all of the above are normally ADVISORY cooperating programs honor them; ordinary read()/write() is not blocked merely because a peer ignores the convention

Lock family	Granularity	Ownership / lifetime model
`flock()`	Whole file: shared or exclusive	Open-file-description based on native Linux local filesystems; duplicate descriptors referring to the same description share the lock.
POSIX `F_SETLK`	Byte ranges; read or write locks	Process-associated. Historically subtle around `fork()` and especially `close()`.
OFD `F_OFD_SETLK`	Byte ranges; read or write locks	Open-file-description associated, giving semantics that compose more naturally with threads, dup and fork.
`/proc/locks`	Observation	Kernel view showing current FLOCK, POSIX and OFDLCK entries, useful for seeing which model is actually active.

Advisory means “coordination protocol,” not “access-control wall.” A process with permission to modify the file can generally ignore another process's advisory lock. Also, network filesystems can translate/emulate lock families differently, so do not project local-filesystem interaction rules onto NFS/SMB without checking that filesystem's semantics.

PUBLIC MANUALfcntl_locking(2) — POSIX and OFD byte-range locks

Current Linux man-pages reference for F_SETLK/F_SETLKW and F_OFD_SETLK/F_OFD_SETLKW, including ownership, inheritance and close semantics.

https://www.man7.org/linux/man-pages/man2/fcntl_locking.2.html

PUBLIC MANUALflock(2) — BSD-style whole-file locking

Explains shared/exclusive locks, open-file-description ownership, fork/dup behavior, advisory semantics and filesystem-specific NFS/SMB caveats.

https://man7.org/linux/man-pages/man2/flock.2.html

PUBLIC MANUAL/proc/locks — inspect live kernel locks

Shows how Linux exposes FLOCK, POSIX and OFDLCK entries, lock mode, owner information and byte ranges.

https://man7.org/linux/man-pages/man5/proc_locks.5.html

Filesystem changes can become pollable events: VFS operation → fsnotify mark → inotify queue → read()/epoll

Applications such as editors, build tools and file indexers often need to learn that a pathname or directory changed without repeatedly rescanning it. Linux inotify exposes the kernel's filesystem-notification machinery as a file descriptor. The application creates an inotify instance, installs watches with event masks, then reads variable-length event records or waits for the descriptor through poll/epoll.

inotify_init1() → inotify instance fd ↓ inotify_add_watch(path, mask) ↓ kernel associates a watch descriptor (wd) with watched object + mask ↓ some filesystem/VFS operation occurs ├── create / delete ├── open / close / access / modify ├── attribute change └── move / rename ↓ fsnotify/inotify matching logic generates event ↓ instance event queue ↓ fd becomes readable ↓ read(fd) returns one or more struct inotify_event records ├── wd ├── mask ├── cookie (used to correlate paired move events where available) └── optional child name for watched directories ↓ application updates its own model QUEUE PRESSURE producer outruns reader → queue may overflow → IN_Q_OVERFLOW application must treat this as lost information and resynchronize state

Property	Consequence
file-descriptor API	The same readiness loop can wait for filesystem changes alongside sockets, timers, signalfd/eventfd and other descriptors.
directory watch	Events can include names of children changed inside the watched directory; this does not make inotify recursively watch an entire tree automatically.
watch descriptor	Small integer identifying the watch within one inotify instance; it is not a file descriptor for the watched file.
move cookie	Helps pair `IN_MOVED_FROM` and `IN_MOVED_TO` events generated for a rename/move when both sides are visible to the instance.
queue overflow	Notification streams are not an infallible transaction log; once events are lost, the consumer may need a full rescan.
pathname race	A notification reports that something changed; by the time userspace follows a name, the namespace may have changed again. Normal race-resistant file-opening rules still apply.

inotify reports events; it does not freeze the namespace. Treat the notification as a prompt to inspect current state, not as a durable guarantee that every intermediate pathname state can still be opened later.

PUBLIC MANUALinotify(7) — filesystem event monitoring

Complete public description of watch masks, returned events, move cookies, queue overflow, watch lifetime and important race/rename caveats.

https://man7.org/linux/man-pages/man7/inotify.7.html

PUBLIC MANUALinotify_init1(2) — create the pollable instance

Defines the inotify instance file descriptor and flags such as nonblocking and close-on-exec behavior.

https://man7.org/linux/man-pages/man2/inotify_init.2.html

PUBLIC MANUALinotify_add_watch(2) — install or modify a watch

Direct system-call semantics for binding a pathname and event mask to a watch descriptor.

https://www.man7.org/linux/man-pages/man2/inotify_add_watch.2.html

fanotify can watch broader filesystem scopes and can pause selected accesses for a userspace allow/deny decision

fanotify is another Linux interface built on filesystem notification machinery, but it targets different use cases than inotify. A fanotify group can place marks on files, directories, mounts or entire filesystems, receive event metadata through a file descriptor, and—when configured for supported permission events—temporarily block an operation while a userspace monitor replies FAN_ALLOW or FAN_DENY.

fanotify_init(flags, event_f_flags) ↓ fanotify group fd ↓ fanotify_mark(...) ├── inode/file mark ├── directory mark ├── mount mark └── filesystem mark ↓ VFS/filesystem operation occurs ↓ fsnotify matching logic ↓ fanotify event queue ↓ read(fanotify_fd) ↓ metadata identifies event ├── mask ├── triggering process information where available ├── object fd in traditional notification mode, or └── file-handle/FID records with FAN_REPORT_* modes NOTIFICATION-ONLY EVENT monitor observes event → closes event fd / updates state PERMISSION EVENT (supported masks/modes) target operation waits ↓ monitor inspects policy/context ↓ write response to fanotify fd ├── FAN_ALLOW → operation may continue └── FAN_DENY → operation fails If the event queue overflows, FAN_Q_OVERFLOW tells the monitor that observation is incomplete and state may need resynchronization.

inotify	fanotify
Convenient pathname/directory change watching for editors, build tools and similar applications.	Designed for broader monitoring/security-style use cases, including mount/filesystem scope.
Events identify the inotify watch descriptor and may include a child name.	Can report an opened object fd or file-handle/FID records depending on initialization/report flags.
Observation only; it does not provide a general “ask userspace before allowing this open” path.	Selected permission events can synchronously ask the listener to allow or deny the operation.
Recursive tree monitoring normally requires managing watches across directories.	Mount/filesystem marks can cover a much broader namespace scope, subject to API/event limitations.

Permission events are in the I/O path. A slow or dead policy monitor can therefore delay operations it is asked to arbitrate. fanotify is not a replacement for kernel access-control mechanisms such as DAC, capabilities or LSMs; it is an event/mediation interface with its own privilege and coverage rules.

PUBLIC MANUALfanotify(7) — API overview

Current overview of notification groups, marks, event records, FID reporting, permission events, queue behavior and API limitations.

https://man7.org/linux/man-pages/man7/fanotify.7.html

PUBLIC MANUALfanotify_init(2)

Creates the fanotify group and selects notification/reporting behavior and event-file status flags.

https://man7.org/linux/man-pages/man2/fanotify_init.2.html

PUBLIC MANUALfanotify_mark(2)

Defines how files, directories, mounts or filesystems are marked and which event masks are requested or ignored.

https://man7.org/linux/man-pages/man2/fanotify_mark.2.html

A filename is a directory entry, not the file itself: hard links, unlink(), rename() and open-file lifetime

The VFS section distinguishes pathnames, dentries, inodes and open file descriptions. The important consequence is that changing the namespace does not necessarily destroy the underlying object. A hard link creates another directory name for the same inode; unlink() removes one name; rename() moves/replaces names. Existing open file descriptions keep referring to the same file object even if every pathname to it disappears.

INITIAL DIRECTORY NAMESPACE /home/u/report.txt ──directory entry──→ inode 4812 link count = 1 open("/home/u/report.txt") fd 3 → struct file/open file description → inode 4812 link("report.txt", "report.backup") report.txt ─┐ ├──→ inode 4812 link count = 2 report.backup ─┘ unlink("report.txt") report.txt name disappears report.backup ───────────────→ inode 4812 link count = 1 fd 3 still works unlink("report.backup") no directory name remains link count = 0 fd 3 ───────────────────────→ inode/file still alive read/write through fd 3 can continue close(3) last open reference disappears ↓ filesystem may finally reclaim inode/data blocks when no other references remain RENAME CASE rename("tmp.new", "config") ↓ namespace change is atomic with respect to seeing the destination absent: new destination refers to replacement object after the rename completes open fds that referenced the old destination keep referring to that old object

Operation / object	What changes
hard link	Adds another directory entry naming the same inode. It is not a second copy of the file's contents.
inode link count	Counts hard links represented in the filesystem namespace; it is different from the number of open descriptors.
`unlink()`	Removes one directory entry and decrements link count. Last-name removal does not invalidate already-open file descriptions.
`rename()`	Changes a directory name/location and can atomically replace an existing destination within the supported filesystem constraints.
`RENAME_NOREPLACE`	Linux `renameat2()` mode that fails rather than overwriting an existing destination.
`RENAME_EXCHANGE`	Linux `renameat2()` mode that atomically swaps two existing pathnames.
open file description	Kernel reference that survives pathname rename/unlink and keeps the opened object alive until its references are released.

“Deleted” can mean “unreachable by pathname,” not “bytes already reclaimed.” This explains the common Unix behavior where a process continues writing to a log that has been unlinked: disk space is not recovered until the last live reference is closed. It also explains why replacing a file with rename does not retarget processes that already opened the old file.

LINK / UNLINK / RENAME LAB printf 'old data ' > demo.txt ln demo.txt second-name ls -li demo.txt second-name # same inode number, link count 2 exec 3<>demo.txt # shell holds an open descriptor rm demo.txt second-name ls -l /proc/$$/fd/3 # commonly displayed with '(deleted)' cat <&3 # descriptor still refers to the opened object exec 3>&- # Atomic replacement pattern used by many programs: printf 'new data ' > config.tmp mv config.tmp config # rename within one filesystem # Observe namespace syscalls strace -e link,linkat,unlink,unlinkat,rename,renameat,renameat2 mv a b

PUBLIC MANUALunlink(2) / unlinkat(2)

Current Linux manual explicitly describing last-link removal and delayed reclamation while a process still has the file open.

https://man7.org/linux/man-pages/man2/unlinkat.2.html

PUBLIC MANUALrename(2) / renameat2(2)

Current Linux manual covering atomic destination replacement plus Linux-specific RENAME_NOREPLACE and RENAME_EXCHANGE.

https://man7.org/linux/man-pages/man2/renameat2.2.html

PUBLIC MANUALlink(2) / linkat(2) — create another hard link

Shows that two hard-link names refer to the same underlying file/inode and therefore share content, ownership and permissions.

https://man7.org/linux/man-pages/man2/link.2.html

PUBLIC MANUALinode(7) — inode number, link count and metadata

Current concise reference for inode identity and metadata, including st_nlink, the hard-link count visible through stat()/statx().

https://man7.org/linux/man-pages/man7/inode.7.html

A file copy does not always copy blocks immediately: reflinks share extents until one file writes

A normal byte-for-byte copy creates independent destination storage as data is written. A filesystem that supports reflink can instead create another inode whose logical range initially points at the same physical filesystem blocks. The operation can therefore be very fast and space-efficient. The shared extents are marked for copy-on-write: when either file modifies a shared block, the filesystem allocates new storage for the modified version while the other inode keeps seeing the original data.

BEFORE CLONE file A inode → extent E → physical blocks [100..199] FICLONE / FICLONERANGE ↓ file A inode ─┐ ├→ shared extent E → blocks [100..199] file B inode ─┘ read A or B ↓ same underlying bytes write B at one shared block ↓ filesystem COW allocate new block for B's changed range copy old contents as needed apply B's write ↓ file A → original block file B → new modified block DEDUPLICATION A range and B range already exist independently ↓ compare / verify identical FIDEDUPERANGE ↓ replace eligible duplicate extents with shared COW storage

Interface/idea	Semantics
`FICLONE`	Ask a supporting filesystem to clone the source file's data mappings into the destination using shared copy-on-write storage.
`FICLONERANGE`	Clone only a selected byte range.
`FIDEDUPERANGE`	Share storage only when source and destination ranges already contain identical data.
`copy_file_range()`	Requests file-to-file copying; a filesystem may implement it with reflink, server-side copy or another acceleration, but callers should rely on copy semantics rather than a particular physical layout.
hard link	Two directory entries name the same inode. A reflink instead creates/uses distinct inodes that may share only data extents.
COW write	Allocates private storage for the modified range so later changes to one inode do not alter the other's file contents.

Reflink is a storage optimization with normal file semantics, not a promise that blocks remain shared forever. Writes, preallocation, filesystem policy, snapshots or other operations may break sharing. Also, “copy offload” and “reflink” are related but not identical: a remote or device-side copy can avoid CPU data movement while still producing fully independent destination blocks.

PUBLIC MANUALcopy_file_range(2) — in-kernel file-to-file copy

Companion interface for copying file ranges without forcing payload data through a userspace buffer; filesystems may optimize or offload the operation.

https://man7.org/linux/man-pages/man2/copy_file_range.2.html

PUBLIC MANUALFICLONE / FICLONERANGE — share file data with copy-on-write

Current Linux man-page documentation for cloning whole files or ranges so separate inodes share underlying storage until a later write causes copy-on-write.

https://man7.org/linux/man-pages/man2/FICLONE.2const.html

PUBLIC MANUALFIDEDUPERANGE — filesystem block deduplication

Current interface for asking a supporting filesystem to replace already-identical file ranges with shared underlying storage.

https://man7.org/linux/man-pages/man2/FIDEDUPERANGE.2const.html

Btrfs makes copy-on-write a filesystem-wide design: trees, checksums, shared extents, snapshots and scrub

Reflink shows that two files can share extents, but a copy-on-write filesystem can build much more around the same idea. Btrfs stores filesystem metadata in copy-on-write B-trees and normally writes changed data to new extents rather than overwriting the old blocks in place. That makes subvolume snapshots cheap because the original and snapshot initially reference the same unchanged extents. Checksums over data and metadata let reads detect corruption, while redundant profiles can give scrub/read-repair a known-good copy to use.

NORMAL WRITE TO EXISTING FILE RANGE old inode/extent metadata → old data extent D0 ↓ application changes bytes allocate new extent D1 ↓ write new data + checksum write new COW metadata path pointing at D1 ↓ commit transaction/root update old references remain valid until no tree/subvolume needs D0 SNAPSHOT OF SUBVOLUME A subvolume A root ───────→ shared metadata/data extents ↓ snapshot subvolume B root ───────┘ ↓ later write in A COW only modified paths/extents ↓ A sees new blocks; B still sees old blocks SCRUB walk allocated extents ↓ read block + verify checksum checksum good → continue checksum bad + verified redundant copy available ↓ repair damaged replica SEND/RECEIVE read-only snapshot A2 compared with parent A1 ↓ encode filesystem operations/extents stream → another Btrfs filesystem → reconstruct subvolume

Btrfs concept	What it means
subvolume	Independent file/directory tree inside one Btrfs filesystem that shares the same underlying storage pool and can be mounted separately.
snapshot	New subvolume whose initial state shares existing extents through COW. It is cheap locally, but it is not an independent backup of the underlying media.
checksums	Metadata and normally file data carry checksums so corruption can be detected when blocks are read.
scrub	Online pass that reads allocated data/metadata and validates checksums; with redundant good copies it can repair damaged replicas.
reflink/shared extent	Multiple inode/subvolume references can point to the same physical data until a write forces COW.
send/receive	Serializes a read-only subvolume or incremental difference as filesystem operations so another Btrfs filesystem can reconstruct it.
NOCOW/NODATASUM	Per-file/mount choices can opt selected workloads out of normal data COW/checksumming behavior; those choices remove some of the properties described above.

A snapshot is not a backup. A local snapshot can survive accidental file edits because it preserves old logical references, but the original and snapshot may still share the same physical device and many of the same blocks. Device loss, controller failure or corruption outside recoverable redundancy can damage both. A real backup needs an independent failure domain.

FILESYSTEM DOCSBtrfs filesystem manual

Current reference for Btrfs mount/filesystem behavior, including data copy-on-write, checksumming and snapshot/subvolume terminology.

https://btrfs.readthedocs.io/en/latest/btrfs-man5.html

FILESYSTEM DOCSBtrfs subvolumes and snapshots

Explains that snapshots are subvolumes sharing unchanged extents through copy-on-write and explicitly warns that a snapshot is not itself a backup.

https://btrfs.readthedocs.io/en/stable/btrfs-subvolume.html

FILESYSTEM DOCSbtrfs scrub — checksum verification and repair

Current scrub semantics: verify data/metadata checksums and, on replicated profiles, repair a bad copy from a verified good replica when possible.

https://btrfs.readthedocs.io/en/latest/btrfs-scrub.html

FILESYSTEM DOCSbtrfs send — full and incremental snapshot streams

Shows how read-only subvolume snapshots can be serialized as full or incremental operation streams for replication with btrfs receive.

https://btrfs.readthedocs.io/en/latest/btrfs-send.html

epoll turns file wait queues into one ready list: interest, callback, wakeup and drain-to-EAGAIN

epoll is not a kernel thread continuously checking every descriptor. An epoll instance maintains an interest list of watched open-file descriptions and a ready list populated when the underlying file/socket/pipe wakeup machinery reports a readiness transition or condition.

SETUP epfd = epoll_create1(...) ↓ epoll_ctl(epfd, ADD, sockfd, EPOLLIN) ↓ kernel epoll item references target FILE / open-file-description identity ↓ target file's poll() method reports current readiness and registers epoll callback on one or more WAIT QUEUES NO DATA AVAILABLE application calls epoll_wait(epfd, ...) ↓ epoll ready list empty ↓ calling task sleeps on epoll's own wait queue scheduler runs something else NETWORK PACKET ARRIVES NIC IRQ/NAPI → TCP/IP → socket receive queue gains data ↓ socket code wakes its wait queue ↓ epoll callback runs ↓ if watched event is ready, epoll item is linked/marked on READY LIST ↓ wake task sleeping in epoll_wait() epoll_wait returns event(s) ↓ application read(sockfd, ...) LEVEL-TRIGGERED (default) if bytes remain readable, fd continues to be reported ready EDGE-TRIGGERED (EPOLLET) notification is tied to readiness changes best practice: nonblocking fd + read/write until EAGAIN otherwise unread data can remain while no new edge arrives EPOLLONESHOT after one delivered event, item is disabled application must epoll_ctl(MOD) to rearm it

epoll concept	Meaning
epoll instance	In-kernel object represented to userspace by an epoll file descriptor.
interest list	Registered files/open-file descriptions and event masks to monitor.
ready list	Subset/references whose watched operations are currently reportable as ready.
wait queue callback	Hook registered with the underlying file's poll mechanism so epoll learns when readiness may have changed.
EPOLLIN / EPOLLOUT	Readable/writable readiness masks, not guarantees that a future blocking operation can never race/change.
level-triggered	Reports readiness while the condition remains true.
edge-triggered	Reports transitions/change notifications; callers normally use nonblocking I/O and drain until EAGAIN.
EPOLLONESHOT	Disables item after an event is reported until userspace rearms it.
EPOLLEXCLUSIVE	Reduces thundering-herd wakeups for selected multi-waiter use cases.
epoll_wait()	Sleeps until ready-list events, a signal or timeout; timeout uses CLOCK_MONOTONIC.
EAGAIN	Nonblocking operation cannot make progress now; in edge-triggered loops it commonly marks the point where the fd has been drained.

epoll tracks open-file identity more deeply than the integer fd alone. A file descriptor is only a per-process table index; duplicated descriptors can refer to the same underlying open file description, which matters for registration, close/dup behavior and event semantics.

NO-ACCOUNT EPOLL LAB # trace a small event-loop program strace -f -e epoll_create1,epoll_ctl,epoll_wait,epoll_pwait2,accept4,read,write ./server # compare level-triggered and EPOLLET behavior using a pipe/socket # write 2048 bytes, wake once, then intentionally read only 1024 bytes. LT: next epoll_wait still reports readable because 1024 bytes remain ET: if you fail to drain to EAGAIN, you can stall waiting for another edge # use nonblocking descriptors for the ET experiment # current epoll watch limit cat /proc/sys/fs/epoll/max_user_watches 2>/dev/null # Kernel implementation source: fs/eventpoll.c

CURRENT PUBLIC MANUALepoll(7) — current man-pages 6.19

Current September 2026 rendering explicitly defines the interest list and dynamically populated ready list, plus level-triggered versus edge-triggered semantics and the drain-to-EAGAIN rule.

https://man7.org/linux/man-pages/man7/epoll.7.html

CURRENT PUBLIC MANUALepoll_wait(2) — current man-pages 6.19

Current manual: epoll_wait fetches from the ready list and blocks until an event, signal or timeout; its timeout is measured using CLOCK_MONOTONIC.

https://www.man7.org/linux/man-pages/man2/epoll_wait.2.html

PUBLIC MANUALepoll_ctl(2)

Defines ADD/MOD/DEL, EPOLLET, EPOLLONESHOT and other registration flags.

https://www.man7.org/linux/man-pages/man2/epoll_ctl.2.html

SOURCE / FILELinux fs/eventpoll.c — real implementation

Current source showing epoll items, wait-queue entries/callback removal, ready-list handling, sleeping/wakeup and event collection.

https://github.com/torvalds/linux/blob/master/fs/eventpoll.c

PUBLIC MANUALpoll(2)

Useful contrast: poll waits on an explicit array supplied on each call, whereas epoll keeps persistent in-kernel registration state.

https://man7.org/linux/man-pages/man2/poll.2.html

Linux turns more events into file descriptors: eventfd and signalfd plug synchronization/signals into epoll

Linux's file-descriptor model extends beyond files and sockets. eventfd wraps a kernel-maintained 64-bit counter in one fd, providing a cheap event notification primitive. signalfd redirects selected blocked signals into a readable fd containing structured signal records. Both become first-class epoll sources.

EVENTFD efd = eventfd(0, EFD_NONBLOCK|EFD_CLOEXEC) kernel object: uint64 counter = 0 worker/thread/device-emulation path: write(efd, uint64_t{5}, 8) ↓ counter 0 → 5 ↓ eventfd wait queue wakes epoll ready list marks efd readable event loop: epoll_wait(...) ↓ eventfd EPOLLIN read(efd, &v, 8) ↓ normal mode: v = entire counter; counter → 0 EFD_SEMAPHORE: read returns 1 and decrements counter by 1 eventfd is useful when the payload is just 'N events happened', not an arbitrary byte stream like a pipe. SIGNALFD sigemptyset(&mask) sigaddset(&mask, SIGTERM) sigaddset(&mask, SIGCHLD) pthread_sigmask(SIG_BLOCK, &mask, NULL) ↓ selected signals are BLOCKED from normal asynchronous handler delivery sfd = signalfd(-1, &mask, SFD_NONBLOCK|SFD_CLOEXEC) epoll_ctl(epfd, ADD, sfd, EPOLLIN) SIGCHLD becomes pending ↓ signalfd becomes readable ↓ read(sfd, struct signalfd_siginfo[]) ↓ consume pending signal(s) synchronously in normal event-loop control flow THIS CHANGES PROGRAM STRUCTURE traditional: async signal handler interrupts arbitrary code handler limited to async-signal-safe operations signalfd: selected signals are blocked main event loop reads records like another fd ordinary code/locking/allocation is possible after read returns timerfd + eventfd + signalfd + sockets + pipes ↓ one epoll instance can multiplex timers, software notifications, signals and I/O readiness using a uniform fd interface.

FD event primitive	Kernel state	Read semantics
eventfd	Unsigned 64-bit counter	8-byte counter value; normal read resets to 0.
eventfd + EFD_SEMAPHORE	Unsigned 64-bit counter	Returns 1 and decrements counter by one.
signalfd	Selected pending signals for a blocked signal mask	One or more signalfd_siginfo records.
timerfd	Timer expiration count	8-byte number of expirations since prior read.
pipe	Bounded page-backed byte stream	Arbitrary bytes up to requested count.

signalfd does not magically intercept an unblocked signal. The selected signals should be blocked with sigprocmask/pthread_sigmask so they remain pending for signalfd to consume instead of taking the ordinary handler/default-delivery path.

FD-EVENT LOOP LAB # Source-code exercise: # create epoll + eventfd + timerfd + signalfd(SIGINT,SIGTERM) # add all three fds plus stdin or a socket # print which fd wakes and the structured value read from it # inspect fd-specific kernel state cat /proc/<PID>/fdinfo/<EVENTFD_FD> 2>/dev/null cat /proc/<PID>/fdinfo/<SIGNALFD_FD> 2>/dev/null cat /proc/<PID>/fdinfo/<TIMERFD_FD> 2>/dev/null # syscall trace strace -e eventfd2,signalfd4,timerfd_create,timerfd_settime,epoll_ctl,epoll_wait,read,write ./event_loop # Compare this with an async SIGINT handler to see how signalfd moves # signal handling back into ordinary synchronous event-loop code.

CURRENT PUBLIC MANUALeventfd(2) — current Linux man-pages

Current eventfd contract: kernel-maintained 64-bit counter, semaphore mode, read/write blocking rules and direct poll/epoll integration.

https://www.man7.org/linux/man-pages/man2/eventfd.2.html

CURRENT PUBLIC MANUALsignalfd(2) — current Linux man-pages

Current signalfd interface for consuming blocked signals as structured records through a file descriptor usable with poll/epoll.

https://www.man7.org/linux/man-pages/man2/signalfd.2.html

A pipe is a bounded page-backed ring with reader/writer wait queues and backpressure

A Unix pipe is not a hidden temporary file and not a byte array in userspace. Linux represents an anonymous pipe with a pipe_inode_info containing a power-of-two ring of pipe_buffer entries. Each entry references a page plus offset/length/operations. Separate reader and writer wait queues provide blocking and readiness notification.

pipe2(fds, O_CLOEXEC) ↓ kernel creates anonymous pipe inode + pipe_inode_info ↓ two struct file objects / descriptors: fds[0] = read end fds[1] = write end PIPE RING head/tail are monotonically increasing indices physical slot = index & (ring_size - 1) slot 0: pipe_buffer → page P17, offset 0, len 4096 slot 1: pipe_buffer → page P92, offset 0, len 1300 slot 2: empty ... write(fd_w, userbuf, 5000) ↓ pipe mutex ↓ if last buffer is mergeable and has room, append there ↓ otherwise obtain/allocate page copy bytes from user into page publish new pipe_buffer at head advance head ↓ if pipe transitioned empty → nonempty wake readers / EPOLLIN waiters PIPE FULL blocking writer: sleep on wr_wait until reader frees space nonblocking writer: return EAGAIN if no progress can be made read(fd_r, dst, n) ↓ consume bytes from tail pipe_buffer(s) advance offset/len; release empty buffer/page advance tail ↓ if pipe was full and now has room wake writers / EPOLLOUT waiters NO WRITERS + EMPTY read returns 0 = EOF NO READERS write generates SIGPIPE unless suppressed/ignored and fails with EPIPE PIPE_BUF ATOMICITY POSIX-size writes ≤ PIPE_BUF are atomic relative to other writers under the documented blocking/nonblocking cases; larger writes can be split/interleaved. splice()/tee()/vmsplice() can manipulate/reference pipe_buffer pages so data sometimes moves between file/socket/user memory and pipe without the same copy path as read()+write().

Pipe concept	Meaning
pipe_inode_info	Kernel state for a pipe: ring metadata, readers/writers, limits, mutex and read/write wait queues.
pipe_buffer	One ring descriptor referencing a page, byte offset/length and operations for ownership/release/steal semantics.
pipe ring	Power-of-two array of pipe_buffer descriptors indexed by monotonic head/tail counters.
rd_wait	Wait queue used when readers need data or readiness callbacks need notification.
wr_wait	Wait queue used when writers need buffer space or writable readiness changes.
PIPE_BUF	POSIX atomic-write threshold: qualifying writes at or below it are not interleaved with other writers.
pipe capacity	Finite buffering; Linux exposes get/set controls through F_GETPIPE_SZ/F_SETPIPE_SZ subject to limits.
EAGAIN	Nonblocking read/write cannot currently make progress.
SIGPIPE/EPIPE	Result when writing after all read-end references are gone.
EOF	read returns zero after all writers are gone and the pipe buffer is empty.
splice()	Moves/references data between a pipe and another file descriptor using kernel pipe-buffer machinery.
tee()	Duplicates references from one pipe to another without consuming the source pipe data.
vmsplice()	Maps/copies user iovecs into pipe-buffer machinery; true splice semantics are strongest in the user→pipe direction.

Pipe backpressure is scheduler-visible flow control. When the ring fills, a blocking writer goes to sleep; when a reader frees space, the writer is awakened. That same readiness state feeds poll/epoll, connecting the pipe implementation directly to the wait-queue and epoll sections.

PIPE LAB # shell pipeline creates real kernel pipes sleep 60 | cat >/dev/null & PID=$! # pipe fds appear as pipe:[inode] links ls -l /proc/$PID/fd 2>/dev/null # capacity controls are easiest from a tiny C/Python program using fcntl # F_GETPIPE_SZ / F_SETPIPE_SZ # global/user pipe limits grep . /proc/sys/fs/pipe-* 2>/dev/null # trace blocking/wakeup-visible syscalls strace -f -e pipe2,read,write,splice,tee,vmsplice,poll,epoll_wait <program> # experiment: one slow reader + fast writer; observe writer block when capacity fills. # repeat with O_NONBLOCK and observe EAGAIN instead.

CURRENT PUBLIC MANUALpipe(7) — current Linux man-pages 6.19

Current September 2026 manual covering finite capacity, blocking/nonblocking behavior, PIPE_BUF atomicity, SIGPIPE/EPIPE and F_GETPIPE_SZ/F_SETPIPE_SZ.

https://man7.org/linux/man-pages/man7/pipe.7.html

SOURCE / FILELinux fs/pipe.c — real current implementation

Current source shows power-of-two head/tail ring indexing, page-backed pipe_buffer entries, rd_wait/wr_wait queues, blocking/EAGAIN and reader/writer wakeups.

https://github.com/torvalds/linux/blob/master/fs/pipe.c

PUBLIC MANUALpipe(2)

Current system-call interface for creating the two file descriptors and flags such as O_CLOEXEC/O_NONBLOCK.

https://man7.org/linux/man-pages/man2/pipe.2.html

CURRENT PUBLIC MANUALsplice(2)

Current zero/low-copy-style interface moving data to/from pipes; its implementation is built around the same kernel pipe-buffer abstraction.

https://www.man7.org/linux/man-pages/man2/splice.2.html

PUBLIC MANUALtee(2)

Current interface duplicating pipe-buffer data from one pipe to another without consuming the source.

https://man7.org/linux/man-pages/man2/tee.2.html

PUBLIC MANUALvmsplice(2)

Current interface connecting user iovecs to pipe buffers; the manual notes that true splicing is primarily supported in the user→pipe direction.

https://man7.org/linux/man-pages/man2/vmsplice.2.html

A shell pipeline is a process graph: pipe → fork → dup2 → exec → close → wait

The pipe section describes the kernel buffer itself. The missing piece is how a shell turns text such as producer | filter | consumer into several concurrent processes connected by file descriptors. Conceptually, the shell creates the pipes first, creates children, rewires each child's standard input/output with descriptor duplication, closes descriptors that child does not need, and then executes the requested programs.

COMMAND producer | filter | consumer SHELL CREATES TWO PIPES pipe A: Aread ←──────────── Awrite pipe B: Bread ←──────────── Bwrite CHILD 1: producer fork/clone child dup2(Awrite, STDOUT_FILENO=1) close Aread/Awrite/Bread/Bwrite copies not needed after duplication execve("producer", ...) │ bytes written to fd 1 ▼ PIPE A BUFFER │ CHILD 2: filter dup2(Aread, STDIN_FILENO=0) dup2(Bwrite, STDOUT_FILENO=1) close unused original descriptors execve("filter", ...) │ PIPE B BUFFER │ CHILD 3: consumer dup2(Bread, STDIN_FILENO=0) close unused original descriptors execve("consumer", ...) PARENT SHELL close its copies of pipe ends it no longer needs wait/waitpid for foreground pipeline according to shell semantics collect statuses → print next prompt BACKPRESSURE slow consumer → pipe B fills → filter blocks in write → pipe A may then fill → producer blocks in write EOF consumer reads 0 only after every descriptor referring to Bwrite is closed.

Operation	Why the shell needs it
`pipe()/pipe2()`	Create kernel byte-stream channels with separate read and write file descriptors.
`fork()` / process creation	Create execution contexts that inherit the shell's open descriptors.
`dup2()/dup3()`	Make a pipe end become conventional fd 0 (stdin) or fd 1 (stdout) before the new program starts.
`close()`	Remove unused copies. Correct closing is part of the protocol because lingering write descriptors can prevent downstream readers from ever seeing EOF.
`execve()`	Replace the child process image with the utility while preserving non-close-on-exec descriptors such as the rewired stdin/stdout.
`wait()/waitpid()`	Let a foreground shell wait for child state changes, collect exit status and reap terminated children.

A pipeline is concurrent, not “run command 1 completely, then command 2.” The processes normally overlap in time. The finite pipe buffers create backpressure that naturally couples their rates, while the scheduler chooses which runnable process executes at each moment.

POSIX.1-2024POSIX Shell Command Language — pipelines

The current Issue 8 shell specification states that each command's standard output is connected to the next command's standard input as if by creating a pipe, and defines foreground-pipeline waiting semantics.

https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V3_chap02.html

CURRENT PUBLIC MANUALwait(2), waitpid(2), waitid(2)

Explains how the shell observes child termination/stops/continues and why an exited child remains a zombie until its parent collects the state change.

https://man7.org/linux/man-pages/man2/wait.2.html

# Watch a real shell construct a three-stage pipeline: strace -f -e trace=process,pipe2,dup2,dup3,close,wait4 sh -c 'printf "alpha\nbeta\n" | tr a-z A-Z | wc -l' # Then compare with the existing sections on: # pipe ring buffers, file descriptors, fork/COW, execve and TTY/job control.

Unix-domain sockets can carry bytes, datagrams and kernel object references between local processes

AF_UNIX/AF_LOCAL sockets use the familiar socket API without sending packets through an IP network. They are a local IPC mechanism with stream, datagram and sequenced-packet forms. Their most distinctive feature is ancillary data: with sendmsg()/recvmsg(), one process can transfer credentials or references to already-open file descriptions to another process.

PROCESS A PROCESS B open('/data/file') → fd 7 create/connect AF_UNIX socket ─────────────── socket endpoint │ │ sendmsg() │ normal payload byte(s) │ cmsg level = SOL_SOCKET │ cmsg type = SCM_RIGHTS │ data = [7] ├────────────────────────────────────→ recvmsg() ↓ receives new fd, e.g. 11 ↓ fd 7 in A and fd 11 in B refer to the SAME underlying open file description (as if duplicated): shared file status/offset semantics apply. No pathname for the underlying object has to be reopened in B. The transferred object might be a file, pipe end, socket, eventfd, device fd, etc.

Feature	What it provides
`SOCK_STREAM`	Reliable ordered local byte stream with connection semantics, analogous at the API level to a stream socket but without IP routing.
`SOCK_DGRAM`	Local datagram/message transport preserving message boundaries.
`SOCK_SEQPACKET`	Connection-oriented, reliable ordered messages whose record boundaries are retained.
`socketpair()`	Creates an already-connected pair of local sockets, useful for parent/child or service IPC.
pathname / abstract address	Linux supports filesystem-visible socket pathnames and a Linux-specific abstract namespace. Filesystem socket names have directory/permission and cleanup semantics.
`SCM_RIGHTS`	Passes references to open file descriptions. The receiver gets new descriptor numbers; it does not inherit the sender's integer fd numbers as identities.
`SO_PEERCRED` / `SCM_CREDENTIALS`	Allows local peers to obtain or receive kernel-checked PID/UID/GID credential information under the documented semantics.

Descriptor passing is capability-like delegation. Instead of telling another process “open this pathname and hope permissions/names still identify the intended object,” a process can hand over an already-resolved kernel reference. This connects directly to the open-file-description model described elsewhere on this page.

# See Unix-domain endpoints on a Linux machine ss -x -a cat /proc/net/unix | head # Useful experiments: # 1. socketpair(AF_UNIX, SOCK_STREAM, 0, sv) # 2. open a file in process A # 3. send its fd with sendmsg()+SCM_RIGHTS # 4. recvmsg() in process B and read through the received fd # 5. compare file offsets to observe shared open-file-description state

CURRENT PUBLIC MANUALunix(7) — Unix-domain sockets, credentials and SCM_RIGHTS

Current Linux man page covering pathname/abstract addressing, stream/datagram/seqpacket behavior, peer credentials and ancillary messages for file-descriptor passing.

https://man7.org/linux/man-pages/man7/unix.7.html

CURRENT PUBLIC MANUALcmsg(3) — constructing and parsing ancillary control messages

Practical API reference for the cmsghdr, CMSG_* macros and control-message buffers used by sendmsg()/recvmsg().

https://man7.org/linux/man-pages/man3/cmsg.3.html

Processes can share pages without naming a disk file: memfd turns anonymous memory into a file-descriptor-backed object

memfd_create() creates an anonymous, volatile, file-like object and returns a file descriptor. Because it behaves like a regular file descriptor, it can be resized with ftruncate(), mapped with mmap(), inherited across fork(), and transferred to another process with SCM_RIGHTS. The mapped pages then become a shared-memory transport whose identity and lifetime are represented by ordinary kernel file references.

PROCESS A memfd_create("buffer", MFD_ALLOW_SEALING) ↓ fd 7 ftruncate(fd, 1 MiB) ↓ mmap(..., MAP_SHARED, fd, 0) ↓ write structured data into shared pages ↓ optional fcntl(fd, F_ADD_SEALS, ...) ↓ send fd 7 over Unix-domain socket with SCM_RIGHTS ↓ PROCESS B receives a new fd referring to THE SAME kernel file object ↓ mmap(..., MAP_SHARED, received_fd, 0) ↓ A and B address the same underlying shmem pages LIFETIME close one fd? object remains while another fd or mapping still references it last reference disappears? volatile memfd object can be reclaimed automatically

Mechanism	What it provides
`memfd_create()`	Anonymous file descriptor suitable for file operations and memory mapping without choosing a persistent pathname.
`MAP_SHARED`	Maps the same file-backed pages into multiple address spaces so writes can become visible across processes.
`SCM_RIGHTS`	Transfers a reference to the memfd through a Unix-domain socket; the receiver gets its own descriptor for the same kernel object.
`F_SEAL_GROW` / `F_SEAL_SHRINK`	Prevent size changes that could invalidate a peer's assumptions about the shared object.
`F_SEAL_WRITE`	Prevents writes once the documented conditions for applying the seal are satisfied; useful when publishing immutable shared data.
`F_SEAL_FUTURE_WRITE`	Blocks future writable mappings/writes while allowing existing writable shared mappings to continue under its documented semantics.
POSIX shared memory	`shm_open()` is another file-descriptor-backed shared-memory API, normally using names in a special shared-memory namespace.

Shared memory does not provide synchronization by itself. Two processes can map the same bytes, but ordering and race freedom still require atomics, mutexes/futexes, sequence counters or another protocol. File seals constrain mutation of the backing object; they are not a replacement for a concurrent-data algorithm.

PUBLIC MANUALmemfd_create(2) — anonymous file plus sealing

Current Linux manual with the complete lifecycle: create, resize, map, populate, apply seals and hand the descriptor to another process.

https://man7.org/linux/man-pages/man2/memfd_create.2.html

PUBLIC MANUALF_ADD_SEALS / F_GET_SEALS — file sealing rules

Focused Linux manual for adding/querying seals, including the fact that seals are inode properties, are shared by descriptors for the same object and can only be added.

https://man7.org/linux/man-pages/man2/F_GET_SEALS.2const.html

SEE EARLIERUnix-domain sockets and SCM_RIGHTS

The previous section shows how an already-open kernel object such as a memfd can be transferred between local processes without inventing a pathname-based rendezvous.

POSIX shared memory is a named kernel object mapped into multiple address spaces: shm_open → ftruncate → mmap(MAP_SHARED)

Processes do not need to copy bytes through a pipe or socket when they can safely share pages. POSIX shared memory provides a pathname-like name for obtaining a file descriptor; the descriptor can then be sized and mapped with the ordinary virtual-memory interfaces. On Linux these objects are implemented on a dedicated tmpfs, normally visible under /dev/shm, but applications should think in terms of the POSIX shared-memory object rather than depending on a persistent disk file.

PROCESS A PROCESS B shm_open("/telemetry", O_CREAT|O_RDWR, 0600) ↓ ftruncate(fd, 1 MiB) ↓ mmap(..., MAP_SHARED, fd, 0) shm_open("/telemetry", O_RDWR, 0) │ ↓ │ mmap(..., MAP_SHARED, fd, 0) └──────────── SAME BACKING PAGES ────────────┘ ↓ writes can be visible in both address spaces shm_unlink("/telemetry") ↓ removes NAME existing open descriptors/mappings can continue to reference object ↓ last reference gone → object storage can disappear

Step	What it establishes
`shm_open()`	Creates/opens the named shared-memory object and returns a file descriptor. A new object starts at length zero.
`ftruncate()`	Sets the object size before clients map/use the desired range.
`mmap(..., MAP_SHARED,...)`	Maps the same backing pages into a process so stores can become visible to other mappings.
`close(fd)`	Releases the descriptor; an established mapping can remain valid independently of that descriptor.
`shm_unlink()`	Removes the name, analogous to unlinking a file; existing references are not retroactively invalidated.
mutex/semaphore/atomics	Provide synchronization. Shared mapping alone gives common bytes, not a race-free protocol.

Shared bytes and synchronized access are separate problems. If two processes modify shared data concurrently, they still need a defined interprocess synchronization scheme. A process-shared pthread mutex, POSIX semaphore, futex-based protocol or carefully designed atomics can provide ordering; mmap(MAP_SHARED) by itself does not.

PUBLIC MANUALshm_overview(7) — POSIX shared memory

Compact overview of shm_open(), mmap(), shared-memory object lifetime and the synchronization primitives normally paired with shared mappings.

https://man7.org/linux/man-pages/man7/shm_overview.7.html

PUBLIC MANUALshm_open(3) — named POSIX shared-memory objects

Current Linux manual for creating a shared-memory object, sizing it and mapping it; it explicitly notes that Linux implements POSIX shared memory on a dedicated tmpfs normally mounted at /dev/shm.

https://man7.org/linux/man-pages/man3/shm_open.3.html

POSIX semaphores and message queues are named IPC objects: one synchronizes; the other carries prioritized messages

Shared memory is the fastest way for processes to see the same bytes, but it does not tell them when those bytes are ready. POSIX semaphores provide a count-based blocking synchronization primitive. POSIX message queues provide a kernel-managed queue of discrete messages with priorities and optional asynchronous notification. Both can be named independently of a process's ordinary file-descriptor namespace, and both have kernel-managed lifetime rules.

POSIX SEMAPHORE named: sem_open("/ready", ...) unnamed: sem_init(&sem, pshared=1, ...) ↓ ↓ kernel named object sem_t stored in shared memory ↓ ↓ sem_wait() → decrement if count > 0; otherwise block sem_post() → increment and potentially wake waiter ↓ NO MESSAGE PAYLOAD — the count is synchronization state POSIX MESSAGE QUEUE mq_open("/jobs", ...) → mqd_t ↓ mq_send(payload, priority) ↓ kernel queue keeps discrete message boundaries + priority ↓ mq_receive() / mq_timedreceive() ↓ receiver gets exactly one queued message Linux-specific conveniences: named semaphores commonly appear as /dev/shm/sem.* POSIX message queues are exposed through the mqueue filesystem (commonly /dev/mqueue) Linux implements mqd_t using a file descriptor, so poll/epoll can monitor it ↓ that fd behavior is useful but not required by portable POSIX.

Mechanism	Semantics	Typical use
named semaphore	`sem_open()` creates/opens a named count; `sem_wait()` blocks at zero; `sem_post()` increments; `sem_unlink()` removes the name.	Cross-process rendezvous or bounded-resource counting without designing a message format.
unnamed process-shared semaphore	`sem_init(..., pshared != 0,...)` places semaphore state in memory accessible to multiple processes.	Synchronization embedded beside shared-memory data structures.
POSIX message queue	`mq_open()` names a kernel queue; sends preserve message boundaries and attach a priority.	Small discrete command/event messages where sender and receiver should not share the payload buffer itself.
`mq_notify()`	One process can request notification when a previously empty queue receives a message.	Event-driven consumers that do not want a permanently blocked receive thread.
IPC namespace	On Linux, POSIX message-queue state is namespaced; containers can have distinct queue namespaces/limits.	Isolation of named IPC objects between container/process environments.

A semaphore is not an event history. Multiple posts can accumulate only as a count; there is no attached payload or identity. A message queue stores distinct payloads, but it is still a bounded kernel object, so producers can block or fail when capacity is exhausted. For bulk data, shared memory plus synchronization is usually a different design point.

PUBLIC MANUALsem_overview(7) — POSIX semaphores with process-shared memory

Shows the synchronization half of the design: unnamed process-shared semaphores live in shared memory, while named semaphores provide a separate named rendezvous mechanism.

https://man7.org/linux/man-pages/man7/sem_overview.7.html

PUBLIC MANUALmq_overview(7) — POSIX message queues

Current Linux man-pages overview of named queues, message priorities, persistence, notification and Linux's file-descriptor implementation of message queue descriptors.

https://man7.org/linux/man-pages/man7/mq_overview.7.html

PUBLIC MANUALipc_namespaces(7) — isolating IPC objects

Explains the Linux IPC namespace boundary, including POSIX message queues and their per-namespace limits.

https://man7.org/linux/man-pages/man7/ipc_namespaces.7.html

PUBLIC MANUALsem_open(3) — named POSIX semaphores

Concrete API reference for creating/opening a named semaphore, initial count, permissions, close and unlink lifetime.

https://man7.org/linux/man-pages/man3/sem_open.3.html

tmpfs is a filesystem backed by virtual memory: inode + page cache/shmem pages + optional swap, with no disk filesystem underneath

A file does not have to live on a block device. Linux tmpfs stores file contents in virtual-memory pages managed by the kernel. Those pages consume RAM while resident and may be moved to swap under memory pressure unless swapping is disabled for the mount. There is no ext4/XFS-style on-disk block mapping underneath: directory names and file metadata are real VFS objects, but the data pages belong to the shmem/tmpfs memory subsystem.

create/write /dev/shm/example ↓ VFS pathname + tmpfs inode ↓ file offset → shmem/tmpfs page/folio ↓ first write or page fault allocate memory page ↓ CPU reads/writes ordinary RAM ↓ memory pressure? ├── no → page remains resident └── yes → eligible tmpfs/shmem page may be swapped out ↓ later access swap-in → resident page again NO NORMAL BLOCK-FILESYSTEM DATA PATH no ext4 extent → bio → NVMe mapping for the file's contents RELATED INTERNAL USES MAP_SHARED anonymous memory / SysV shared memory / memfd ↓ kernel internal shmem/tmpfs machinery

Mechanism	Where bytes live / key distinction
tmpfs	Virtual-memory-backed filesystem with mount size/inode limits; resident pages use RAM and can normally use swap.
shmem	Kernel shared-memory machinery underlying tmpfs and several anonymous/shared-memory interfaces.
/dev/shm	Conventionally a tmpfs mount used by POSIX shared-memory objects and related IPC facilities.
memfd	Anonymous file-descriptor-backed object implemented on shmem; can be mmaped and passed between processes.
ramfs	Older/simple memory filesystem that lacks tmpfs-style size controls and swap support.
RAM block device	A block device backed by RAM; a conventional filesystem can be placed on top. This still traverses a block-device abstraction, unlike tmpfs.

# Find tmpfs mounts findmnt -t tmpfs # Inspect /dev/shm capacity and use df -h /dev/shm # Kernel accounting for tmpfs/shared-memory pages grep -E '^(Shmem|ShmemHugePages|ShmemPmdMapped|Swap)' /proc/meminfo # Create a small throwaway tmpfs and inspect it sudo mount -t tmpfs -o size=64M tmpfs /mnt # ... create files ... sudo umount /mnt

“Memory-backed” does not mean “always physically in RAM.” tmpfs is integrated with virtual memory, so eligible pages can be swapped. It is also not automatically persistent: unmounting the filesystem or rebooting discards its contents.

KERNEL DOCSLinux tmpfs documentation

Authoritative explanation of tmpfs, shmem accounting, swap behavior, sizing, huge-page support and the kernel's internal shared-memory mount.

https://docs.kernel.org/filesystems/tmpfs.html

PUBLIC MANUALtmpfs(5)

Current Linux manual page for tmpfs mount behavior, memory policy, limits and its relationship to /dev/shm, shared anonymous mappings and memfd.

https://man7.org/linux/man-pages/man5/tmpfs.5.html

“Zero copy” usually means avoiding the userspace bounce buffer: sendfile(), splice() and page references

A conventional file-to-socket loop calls read() into a userspace buffer and then write()/send() from that buffer. That path can cross the user/kernel boundary twice and may copy the same bytes between page-cache memory and userspace only to hand them straight back to the kernel. Linux provides interfaces that keep the transfer inside kernel-managed buffers instead.

ORDINARY FILE → SOCKET LOOP storage DMA → page-cache folio ↓ copy_to_user() userspace byte buffer ↓ copy_from_user() socket / skb data ↓ NIC DMA network wire SENDFILE-STYLE PATH storage DMA → page-cache folio ↓ kernel transfers/references data directly socket / skb path ↓ NIC DMA network wire SPLICE-STYLE PATH file / socket / other splice-capable fd ↓ PIPE BUFFER = references to kernel pages/folios ↓ splice to another fd The exact implementation may still copy in places; the key property is that userspace does not have to receive and retransmit every payload byte.

Interface	Useful mental model
`sendfile(out, in, ...)`	Ask the kernel to transfer file data directly between descriptors. On Linux it avoids the explicit userspace read/write bounce and can feed a socket efficiently.
`splice()`	Moves data between a pipe and another descriptor without copying between kernel and userspace; pipe buffers can carry page references.
`tee()`	Duplicates pipe-buffer references so the same underlying data can flow to another pipe without consuming the original stream.
`vmsplice()`	Maps/grafts user iovecs into a pipe-oriented transfer path; direction and lifetime rules matter.
`copy_file_range()`	Requests an in-kernel file-to-file copy and may let a filesystem/device use specialized copy/offload mechanisms.

Do not interpret “zero copy” literally without naming the boundary. Avoiding a page-cache→userspace→kernel memcpy does not mean no bytes ever move: storage and NIC DMA still move data, checksumming/encryption/protocol transformations may require touching bytes, and a kernel implementation may fall back to copying when page references cannot be reused safely.

PUBLIC MANUALsendfile(2) — kernel-side descriptor-to-descriptor transfer

Current Linux man page explicitly contrasts sendfile() with a read()+write() loop and documents file/socket/pipe behavior and partial transfers.

https://man7.org/linux/man-pages/man2/sendfile.2.html

PUBLIC MANUALsplice(2) — move data through pipe buffers

Explains the Linux pipe-buffer machinery, including the page-reference model behind splice/tee/vmsplice and the fact that SPLICE_F_MOVE is only a hint.

https://man7.org/linux/man-pages/man2/splice.2.html

How kernel code actually sleeps: wait queues, task states, wake_up() and completions

Many kernel operations cannot busy-loop waiting for hardware or another thread. Linux therefore uses wait queues to connect a condition to sleeping tasks. The waiter must publish its sleep state and queue membership before checking/sleeping in a race-safe pattern; the producer changes the condition and calls a wakeup primitive. Completions are a focused wrapper for the common 'wait until one event has happened' case.

WAIT FOR DEVICE/FIRMWARE/THREAD CONDITION DECLARE_WAIT_QUEUE_HEAD(wq) bool ready = false WAITER wait_event_interruptible(wq, ready) ↓ expands conceptually into a loop prepare_to_wait(..., TASK_INTERRUPTIBLE) ↓ task state is now visible as sleeping-capable re-check condition 'ready' ├── true → finish_wait() → TASK_RUNNING → continue └── false ↓ schedule() ↓ task removed from current CPU execution some other task runs PRODUCER / IRQ / WORKER write data/result first ↓ ready = true ↓ required locking/memory ordering wake_up_interruptible(&wq) ↓ scheduler wakeup path marks eligible waiter RUNNABLE ↓ waiter eventually executes and checks condition AGAIN WHY RECHECK? wake_up() means 'something may have changed', not 'you own the resource'. multiple waiters/races can make the condition false again before one task runs. COMPLETION — ONE-SHOT EVENT TOKEN struct completion done init_completion(&done) # done = 0 + internal swait queue CPU0 / process context CPU1 / IRQ / worker wait_for_completion(&done) do asynchronous work ↓ if done==0, sleep ↓ complete(&done) ↓ done++ + wake waiter ← scheduler wakes CPU0 task consume one completion token → continue complete() BEFORE wait_for_completion() is legal: done already contains a token, so later waiter consumes it without sleeping.

Primitive/state	Meaning
wait_queue_head	Queue plus lock/bookkeeping for tasks or callbacks interested in a condition/event.
wait_queue_entry	One waiter entry linking a task or callback into a wait queue.
TASK_INTERRUPTIBLE	Sleeping task can be awakened by the condition or an unblocked signal.
TASK_UNINTERRUPTIBLE	Sleep is not interrupted by ordinary signal delivery; used only when that semantic is justified.
TASK_KILLABLE	Sleep can be interrupted by fatal/kill signals but not every ordinary signal.
prepare_to_wait()	Queues the current task and sets its sleep state before scheduling.
finish_wait()	Removes waiter if needed and restores TASK_RUNNING state.
wake_up()	Walks the wait queue and attempts to make eligible waiting tasks runnable.
exclusive waiter	Wake-one style queue entry used to avoid waking every waiter for one unit of work.
completion	Small done-counter plus simple wait queue for 'event finished' synchronization.
complete()	Adds one completion token and wakes a waiter; safe from IRQ/atomic context.
complete_all()	Marks completion satisfied for all current/future waiters until reinitialized.
spurious/competitive wakeup	A task can wake yet still find its predicate false; condition must be checked in a loop.

The condition is the truth; the wakeup is only a hint/event. Correct wait-queue code changes the state/condition under the appropriate synchronization, wakes afterward, and rechecks the predicate after every return to execution.

SOURCE / OBSERVATION LAB # See sleeping/runnable task states ps -eLo pid,tid,psr,stat,wchan:32,comm | less # wchan often identifies the kernel function where a task sleeps cat /proc/<PID>/wchan 2>/dev/null # blocked-I/O / mutex examples often show wait-related stack frames cat /proc/<PID>/stack 2>/dev/null # Read kernel waitqueue/completion API declarations and implementations: # include/linux/wait.h # kernel/sched/wait.c # kernel/sched/completion.c # /proc/<pid>/stack may require root and can be restricted by kernel security settings.

KERNEL DOCSLinux completions

Current detailed guide: completions are built on scheduler waitqueue/wakeup infrastructure and contain a done counter plus swait queue; complete() can safely run in IRQ/atomic context.

https://docs.kernel.org/scheduler/completion.html

KERNEL DOCSLinux scheduler wakeup API

The same current reference documents wake_up_process() as moving an eligible sleeping task into the runnable set, tying wait queues directly to scheduler state transitions.

https://docs.kernel.org/driver-api/basics.html

SOURCE / FILELinux waitqueue source

Real implementation of wait-queue preparation, wakeup walking, exclusive wake semantics and cleanup.

https://github.com/torvalds/linux/blob/master/kernel/sched/wait.c

SOURCE / FILELinux completion source

Small enough to read end-to-end: completion waiting, done-token consumption and wakeups.

https://github.com/torvalds/linux/blob/master/kernel/sched/completion.c

One file read: pathname → inode → page cache → extent → block I/O → DMA → userspace

Normal Linux file reads and file-backed mappings meet in the page cache. read() asks the VFS/filesystem for bytes; mmap() may defer the actual storage read until a page fault. If the required file folio is already cached and uptodate, no storage I/O is needed. Otherwise the filesystem maps the file offset to backing blocks/extents and submits I/O.

BUFFERED read(fd, buf, 128 KiB) ↓ syscall / VFS file operations file offset → inode/address_space/page-cache index ↓ lookup required folio(s) in PAGE CACHE ├── HIT + uptodate │ ↓ │ copy bytes to userspace buffer │ ↓ │ advance file position / return byte count │ └── MISS ↓ allocate/cache folio(s) ↓ readahead logic may request adjacent future folios too ↓ filesystem maps FILE OFFSET → logical/physical storage extent ↓ bio / blk-mq request ↓ NVMe/AHCI/etc. driver builds hardware command + DMA descriptors ↓ storage controller/device transfers data into RAM ↓ interrupt or polling completion folio marked uptodate + unlocked ↓ waiting read copies bytes from page cache to userspace FILE-BACKED mmap() mmap(fd) creates VMA but need not read file data immediately ↓ later CPU load from mapped VA PTE absent/not present → PAGE FAULT ↓ filemap_fault()-style path ↓ page-cache HIT? ── yes → install PTE mapping cached folio │ no ↓ same filesystem/read_folio/readahead/storage path as above ↓ install PTE → retry faulting instruction A file page read once through read() can later satisfy mmap(), and vice versa, because both normally use the same page cache.

Object/mechanism	Role in a read
dentry	Caches pathname-component lookup and connects a directory name to an inode.
inode	Filesystem object metadata plus mapping from file logical offsets toward storage allocation/extent structures.
address_space	Kernel object connecting an inode/file to its page-cache folios and filesystem read/write operations.
folio	Current page-cache memory-management unit; may contain one base page or a larger group of pages.
read_folio()	Filesystem operation used when one cached folio must be filled from backing storage.
readahead()	Asynchronously/synchronously asks for nearby uncached file folios before demand reaches them.
extent	Compact mapping describing a consecutive range of file logical blocks backed by consecutive physical filesystem blocks.
bio / blk-mq request	Block-layer representation/dispatch of storage I/O below the filesystem.
DMA completion	Controller/device places read data into RAM and signals or exposes completion.
major file fault	File-backed page fault that requires storage I/O; terminology/counters distinguish it from a minor fault satisfied without storage.
O_DIRECT	I/O mode that can bypass ordinary page-cache buffering under alignment/filesystem/device constraints.

mmap() is not automatically 'zero I/O.' It can avoid an explicit kernel→userspace copy for file data because the process's PTEs map page-cache memory directly, but cache misses still require page faults and storage DMA before the CPU can use absent file pages.

LINUX READ-PATH LAB # Identify file extents (filesystem support required) filefrag -v largefile 2>/dev/null | less # System calls and page faults strace -e read,pread64,mmap,munmap,madvise ./reader perf stat -e page-faults,minor-faults,major-faults ./reader # Per-process mappings / residency clues cat /proc/<PID>/maps cat /proc/<PID>/smaps | less # Block device queue/topology lsblk -o NAME,TYPE,FSTYPE,MODEL,TRAN,MOUNTPOINTS # Drop-caches changes are system-wide and distort other workloads; # don't use them casually on a shared/useful machine just to force a cold read.

KERNEL DOCSLinux — Page Cache

Current kernel documentation: ordinary reads, writes and mmaps normally go through the page cache, whose memory-management unit is the folio.

https://docs.kernel.org/next/mm/page_cache.html

KERNEL DOCSLinux iomap — buffered reads and readahead

Modern filesystem helper path: buffered I/O is cached in pagecache; iomap_readahead and iomap_read_folio fill cache folios from filesystem mappings.

https://docs.kernel.org/filesystems/iomap/operations.html

KERNEL DOCSext4 — extent tree layout

Concrete on-disk mapping: ext4 extent-tree leaves map file logical ranges to physical data blocks, with the tree root stored in inode.i_block.

https://docs.kernel.org/filesystems/ext4/ifork.html

A file can be logically huge without owning blocks for every byte: sparse holes read as zeros until real storage is allocated

A regular file has a logical byte range described by its size, but the filesystem does not have to allocate physical blocks for every part of that range. An unallocated range inside the file is a hole. Reads through the filesystem return zeros for the hole even though no ordinary data blocks need exist there. This is why a VM image, database or core file can report a large apparent size while consuming much less storage.

logical file offsets 0 16 MiB | DATA extent | HOLE | DATA extent | HOLE | | blocks own | no data blocks| blocks own | no data blocks | read(hole range) ↓ filesystem knows no extent backs these logical offsets ↓ returns zero-filled bytes to caller lseek(fd, far_past_EOF, SEEK_SET) write(fd, "X", 1) ↓ file grows; gap can become a sparse hole fallocate(... PUNCH_HOLE | KEEP_SIZE ...) ↓ whole covered filesystem blocks are deallocated partial edge blocks are zeroed as required ↓ logical file size can stay unchanged FIEMAP / SEEK_DATA / SEEK_HOLE ↓ inspect where data/extents and holes appear

Concept	What it means
logical size	Highest logical file offset plus one; this is what `stat.st_size` reports and what a process sees as the byte length.
allocated blocks	Filesystem storage actually reserved for file contents/metadata; can be much smaller than logical size for a sparse file.
hole	Logical range with no ordinary backing data extent; normal reads synthesize zeros.
unwritten extent	Storage may be physically allocated yet logically read as zeros until written. It is therefore not the same thing as a hole.
`SEEK_DATA`/`SEEK_HOLE`	Portable-style interfaces for finding data/hole regions where the filesystem supports meaningful reporting; reporting can be conservative rather than an exact physical map.
`FALLOC_FL_PUNCH_HOLE`	Requests deallocation of a file range while keeping file size; support/alignment depend on the filesystem.
FIEMAP	Linux ioctl that reports extent mappings and flags such as delayed-allocation, unwritten, encoded or shared extents.

A hole is not a reflink. A sparse hole owns no ordinary data extent and reads as zeros. A reflinked range points at real shared extents containing actual data. Likewise, writing explicit zero bytes does not necessarily create a hole; whether zero ranges consume blocks depends on how the file was created and what filesystem operations were used.

PUBLIC MANUALfallocate(2) — allocation, hole punching and range operations

Current Linux interface for preallocation and deallocation, including FALLOC_FL_PUNCH_HOLE plus the required KEEP_SIZE behavior.

https://man7.org/linux/man-pages/man2/fallocate.2.html

PUBLIC MANUALlseek(2) — SEEK_DATA and SEEK_HOLE

Explains sparse gaps created by writes beyond EOF and the interfaces applications can use to discover logical data/hole regions.

https://www.man7.org/linux/man-pages/man2/lseek.2.html

KERNEL DOCSLinux FIEMAP ioctl — extent mapping

Kernel documentation for querying filesystem extents and interpreting flags such as DELALLOC, UNWRITTEN, ENCODED and SHARED.

https://docs.kernel.org/filesystems/fiemap.html

Filesystem quotas account allocations by identity and can refuse new blocks or inodes even when free space remains

Ordinary Unix permissions answer may this credential modify this object? A disk quota answers a different question: how much filesystem space or how many inodes may this user, group or project consume on this filesystem? Linux filesystems can maintain quota accounting and optionally enforce limits. A hard limit is not exceedable; a soft limit may be exceeded temporarily until its grace period expires.

process creates/writes file ↓ VFS/filesystem permission checks pass ↓ filesystem needs another inode or data/metadata allocation ↓ identify quota subjects ├── user ID ├── group ID └── project ID (filesystem/project-tree policy) ↓ charge proposed allocation against quota counters ├── below limit → allocate space/inode ├── above soft limit → allow during grace period + mark overquota └── hard limit / expired grace → reject allocation (for example EDQUOT) filesystem may still have free blocks globally even when this subject is denied.

Quota concept	Meaning
user quota	Accounts/enforces filesystem usage associated with a UID.
group quota	Accounts/enforces usage associated with a GID.
project quota	Associates inodes with a project ID, commonly inherited through a directory tree; useful when ownership does not match the desired accounting boundary.
space limit	Constrains charged filesystem storage allocation; it is not the same thing as a file's logical byte length.
inode/file limit	Constrains the number of filesystem objects charged to the quota subject.
soft limit + grace	Temporary overage is possible until a deadline; after the grace period it behaves like an enforced ceiling.
hard limit	Immediate enforced ceiling for the relevant quota resource.

Quotas are storage policy, not memory policy. They do not replace cgroup memory limits, RLIMIT_AS, filesystem permissions or free-space accounting. They add another allocation gate inside the filesystem.

PUBLIC MANUALquotactl(2) / quotactl_fd(2) — Linux quota control

Current system-call reference for enabling, querying and setting user/group/project quota limits and status, including soft/hard limits and XFS quota-manager operations.

https://www.man7.org/linux/man-pages/man2/quotactl.2.html

PUBLIC MANUALquota(1) — inspect user, group and project usage/limits

Practical userspace view of space/inode usage, grace periods and project quotas across supported filesystems.

https://man7.org/linux/man-pages/man1/quota.1.html

FUSE lets an ordinary process implement filesystem operations while the kernel still presents normal VFS syscalls

With FUSE (Filesystem in Userspace), applications still call ordinary interfaces such as open(), stat(), read() and write(). The kernel's FUSE client translates filesystem operations into requests for a userspace filesystem daemon. The daemon supplies metadata/data or errors, and the kernel turns its reply back into the result of the original syscall. This makes the filesystem implementation a process without making every application learn a new API.

APPLICATION open("/mnt/example/file", O_RDONLY) ↓ syscall VFS pathname walk ↓ mount identifies FUSE filesystem ↓ kernel FUSE client builds request ├── operation opcode ├── node/object identifier ├── credentials/context └── operation-specific arguments ↓ request queued on FUSE connection ↓ classic path: userspace daemon read()s /dev/fuse (newer kernels also document an io_uring transport path) ↓ FILESYSTEM DAEMON ├── looks up metadata ├── talks to remote service, archive, database, etc. if needed └── constructs success/error reply ↓ reply returned to kernel FUSE client ↓ VFS completes lookup/open ↓ application receives fd or errno READ PATH DEPENDS ON MODE cached mode: page-cache hit → kernel can satisfy data without a daemon round trip miss → FUSE READ request → daemon → page cache/userspace direct-io mode: ordinary page-cache buffering/readahead is bypassed for that I/O path DAEMON STALL requests accumulate → calling tasks can block waiting for filesystem service.

Layer/object	Role
VFS	Keeps the normal Linux pathname/file-descriptor interface; callers do not need to know that implementation logic lives in userspace.
fuse.ko / kernel FUSE client	Represents FUSE inodes/files to the VFS, serializes operations into the FUSE protocol and matches replies to waiting kernel operations.
FUSE connection	Kernel↔daemon communication context that lives until the connection is torn down; control information is exposed through fusectl when mounted.
/dev/fuse	Traditional file-descriptor transport by which a daemon receives kernel requests and writes replies.
filesystem daemon	Userspace process implementing policy and data/metadata operations—possibly by translating them to some completely different backing service.
cached I/O	Allows normal page-cache behavior, including readahead and optional writeback caching according to negotiated mode.
direct I/O	Bypasses ordinary page-cache reads/writes for the FUSE file path and changes mmap/readahead behavior.

Userspace implementation changes failure modes. A crash, deadlock or overloaded FUSE daemon can make filesystem operations fail or stall even though the kernel itself is healthy. The kernel exposes connection controls and limits because the daemon is part of the filesystem's live data path.

KERNEL DOCSLinux FUSE technical documentation

Current documentation hub covering the FUSE overview, I/O modes, io_uring transport and passthrough support.

https://docs.kernel.org/filesystems/fuse/index.html

KERNEL DOCSFUSE overview — kernel/userspace interface

Defines filesystem daemon, connection lifetime, non-privileged mounts, fusectl and the classic request/reply interface.

https://www.kernel.org/doc/html/latest/filesystems/fuse/fuse.html

KERNEL DOCSFUSE I/O modes

Explains direct-io versus cached mode, write-through and writeback-cache semantics, readahead and mmap implications.

https://kernel.org/doc/html/latest/filesystems/fuse/fuse-io.html

virtio-fs shares host files with a VM without pretending they are a block disk or a network server

A guest sometimes needs direct access to a host directory tree. Exporting a virtual disk gives the guest blocks, while NFS/SMB introduce a network-filesystem protocol. virtio-fs instead exposes a file-level paravirtual device based on the FUSE request protocol: the guest kernel is the FUSE client, requests travel through virtqueues, and a host-side implementation such as virtiofsd performs operations against the exported host filesystem.

GUEST PROCESS open/read/write/mmap("/mnt/shared/...") ↓ guest VFS ↓ virtiofs filesystem client ↓ FUSE request virtio-fs request virtqueue ↓ VMM / vhost-user transport ↓ virtiofsd or other host backend ↓ host VFS + exported directory tree ↓ host filesystem/storage completion returns through virtqueue ↑ guest FUSE/virtiofs client completes syscall or page fault priority path: requests that must bypass a busy normal queue can use the hiprio virtqueue

Layer	What virtio-fs changes
guest API	Applications still use ordinary pathname/VFS syscalls; they do not speak a special userspace sharing protocol.
FUSE protocol	Request semantics are based on FUSE, but the transport is a virtio file-system device rather than the ordinary `/dev/fuse` userspace interface inside the guest.
virtqueues	Carry requests and responses between guest driver and host backend; a separate high-priority queue avoids starvation of selected requests.
host backend	`virtiofsd` commonly runs as a vhost-user backend and translates guest requests into host filesystem operations under configured sandbox/security rules.
DAX mode	Selected file mappings can use direct-access-style shared mappings to avoid duplicating host file data in an ordinary guest page-cache copy when supported/configured.
networking	No IP network or remote-filesystem server is inherently required; this is intended for co-located host/guest sharing.

virtio-fs is file sharing, not block sharing. The guest does not get ownership of raw sectors and then mount its own independent filesystem on them. It sends filesystem operations to a host-side file server using FUSE semantics over virtio, so caching, permissions, locking and coherency have to be understood at that file-operation boundary.

KERNEL DOCSLinux virtiofs — host↔guest shared filesystem

Current kernel overview explaining guest FUSE-client integration, virtqueue request transport, normal versus hiprio queues and the basic mount model.

https://docs.kernel.org/filesystems/virtiofs.html

PROJECT DOCSvirtio-fs project documentation

Primary project documentation for architecture, host-side configuration and virtiofsd-based sharing between a host and virtual machines.

https://virtio-fs.gitlab.io/

NFS makes a remote filesystem look local by translating VFS operations into network RPCs while keeping a client-side page cache

An NFS mount is not a remote block device. Applications still call ordinary open(), read(), write() and mmap() against the local Linux VFS. The NFS client filesystem translates the operations that cannot be satisfied locally into NFS RPC requests sent to a server. File data can be cached in the client's page cache, while the server may independently use its own filesystem cache and storage stack.

CLIENT PROCESS read(fd, buf, ...) ↓ VFS → NFS inode/address-space state ↓ client page-cache hit? ── yes → copy/map cached bytes │ no ↓ construct NFS READ operation / RPC ↓ TCP/IP (commonly) → client NIC → network → server NIC ↓ server RPC/NFS service (nfsd) ↓ server VFS → server filesystem → server page cache / block I/O / storage ↓ NFS reply carries data + protocol metadata/status ↓ client fills/updates cached folio(s) ↓ waiting read completes WRITE, simplified: client modifies cached file data → pages become dirty ↓ NFS WRITE RPC(s) → server acknowledges according to stability semantics ↓ COMMIT may be required for previously unstable writes before durability claim Namespace / metadata operations use protocol filehandles and NFS operations, not client-side knowledge of the server's raw disk sectors.

Layer	What is local vs remote
pathname/VFS	The client has a normal mount point, dentries/inodes and file descriptors representing remote objects through the NFS client filesystem.
client page cache	Frequently read data can be served without a new RPC while cached state remains valid under NFS coherency rules.
NFS protocol	Filesystem operations are encoded as protocol requests/replies using stable filehandles and NFS state rather than exposing the server block layout.
RPC transport	Moves NFS operations across the network; modern NFS deployments commonly use TCP, with version/transport policy selected by mount negotiation/options.
server VFS/storage	The server performs operations on its local exported filesystem and may itself hit page cache or issue block I/O to disks/SSDs.
coherency/state	NFS uses attribute validation plus protocol state such as opens/locks/delegations; it is not equivalent to coherent shared RAM between client and server.

A cache hit can hide the network; a cache miss exposes the whole distributed path. This is why the latency and failure model of an NFS file is fundamentally different from a local ext4/XFS file even though both are reached through the same POSIX/VFS system calls.

CURRENT PUBLIC MANUALnfs(5) — Linux NFS client mount semantics

Current nfs-utils manual covering supported NFS versions, transports, client caching/data-metadata coherency, mount behavior and security-related options.

https://man7.org/linux/man-pages/man5/nfs.5.html

KERNEL DOCSLinux kernel — NFS documentation index

Current kernel documentation entry point for Linux NFS client/server implementation topics and protocol-facing filesystem machinery.

https://docs.kernel.org/filesystems/nfs/index.html

IETF STANDARDRFC 8881 — NFSv4.1 protocol

Current standards-track NFSv4.1 specification (obsoleting RFC 5661), including sessions, state, filehandles, locking and the protocol foundation used by later NFSv4 minor versions.

https://www.rfc-editor.org/rfc/rfc8881.html

IETF STANDARDRFC 7862 — NFSv4.2 extensions

NFSv4.2 extensions on top of NFSv4.1, including server-side copy, I/O advice, sparse-file operations, space reservation and related modern remote-filesystem features.

https://www.rfc-editor.org/rfc/rfc7862.html

SMB3 makes a remote share look like a mounted filesystem: VFS calls → CIFS client → SMB requests → server object store

Linux can mount a Windows/Samba/NAS share through the kernel CIFS/SMB client. Applications still use ordinary pathname, open(), read(), write(), mmap() and metadata operations, while the client translates those operations into SMB2/SMB3 requests over the network. Unlike a local filesystem, caching and open-handle lifetime must be coordinated with a server that may also have other clients.

mount -t cifs //server/share /mnt/share ... ↓ mount helper supplies server/share + authentication + options ↓ Linux cifs.ko negotiates SMB dialect/capabilities/session ↓ VFS mount is attached at /mnt/share PROCESS read("/mnt/share/file") ↓ pathname/VFS/page-cache path local cached data valid? ── yes → satisfy locally │ no / revalidation needed ↓ SMB CREATE/READ/QUERY/etc request over TCP ↓ server share / remote filesystem ↓ SMB response → Linux client/page cache → process Leases/oplocks can authorize client caching. Lease break → client must reduce/flush/invalidate caching as required. Durable/persistent handles can help an open survive selected reconnect/failover cases.

SMB concept	Why it exists
dialect negotiation	Client/server choose a mutually supported SMB2/SMB3 protocol version and capabilities; modern Linux normally negotiates SMB2.1 or later rather than legacy SMB1.
session / tree connect	Authenticates a security context and connects that session to a particular exported share.
lease / oplock	Lets the server grant read/write/handle caching rights and later break those rights when another access requires tighter coherence.
durable/persistent open	Protocol state allowing selected file handles to be re-established across a temporary disconnect or supported clustered failover.
signing / encryption	SMB3 can protect message integrity and, when negotiated/configured, confidentiality independently of the application using the mounted tree.
Linux inode/page cache	Local VFS objects still exist, but their freshness/coherency is constrained by protocol state and server behavior rather than only local storage.

NFS and SMB solve a similar “remote tree through the VFS” problem with different protocols and semantics. Neither turns remote storage into a local block device; that is a different model such as NBD or iSCSI/NVMe-oF.

KERNEL DOCSLinux CIFS/SMB3 client — usage and semantics

Current kernel documentation for the SMB3-capable CIFS VFS client, dialect negotiation, mount behavior, security features and Linux interoperability details.

https://docs.kernel.org/admin-guide/cifs/usage.html

PUBLIC MANUALmount.cifs / mount.smb3(8)

Current userspace mount-helper manual covering //server/share attachment, dialect selection, authentication, cache/coherency options and SMB3.1.1 support.

https://man7.org/linux/man-pages/man8/mount.smb3.8.html

Buffered I/O versus O_DIRECT: where the page cache is bypassed and where coherence still matters

Ordinary Linux file I/O is normally buffered: reads and writes interact with the page cache, which decouples application calls from storage latency and enables caching, readahead and delayed writeback. O_DIRECT asks the filesystem to minimize those cache effects and transfer between storage and the application's memory more directly. It is a specialized interface, not a universally faster mode.

BUFFERED READ read(fd, user_buf, 128 KiB) ↓ VFS/filesystem page-cache lookup ├── HIT → copy cached bytes to user_buf └── MISS → submit storage I/O ↓ fill page-cache folios/pages ↓ copy bytes to user_buf BUFFERED WRITE write(fd, user_buf, 128 KiB) ↓ copy data into page cache mark cache pages dirty ↓ application may return before media is updated background/writeback or fsync/fdatasync ↓ filesystem maps extents → block layer → storage O_DIRECT READ/WRITE open(..., O_DIRECT) ↓ filesystem validates direct-I/O support + alignment ↓ pin/map user-memory segments as needed ↓ issue I/O between storage path and user buffer ↓ page cache is bypassed for the data transfer COHERENCE PROBLEM same file/range also has cached pages? ↓ filesystem/kernel must coordinate flushing/invalidation; mixing buffered and direct I/O on overlapping ranges is costly and discouraged.

I/O mode	Key property
buffered read	Uses page cache; hot data can be served without a storage request.
buffered write	Copies into page cache and dirties memory; persistence is a separate question handled by writeback/fsync semantics.
O_DIRECT	Attempts file I/O directly between storage and userspace buffers, bypassing the page cache for the transfer.
STATX_DIOALIGN	Lets applications query direct-I/O memory/offset alignment requirements when the filesystem supports reporting them.
alignment	Requirements vary by filesystem/kernel/device; misaligned requests may fail or be handled differently depending on implementation.
mixed buffered/direct I/O	Especially on overlapping ranges, requires cache coherence work and is generally best avoided in application design.

Direct does not mean durable. Bypassing the page cache is a data-path choice. Durability still depends on filesystem, block-device/controller caches and the synchronization semantics you request. Likewise, buffered I/O is not “fake I/O”: the kernel eventually writes dirty cache state through the same lower storage stack.

SEE EARLIERiomap operations reference already linked in the page-cache section

The earlier page-cache section links the same current kernel iomap operations reference; its direct-I/O subsection defines the bypass path and cache-coherence steps, so the external resource is not duplicated here.

SEE EARLIERopen(2) reference already linked in the pathname/file-descriptor section

The existing open(2) resource contains the O_DIRECT caveats and alignment discussion; this section reuses it through an internal link instead of adding a second external copy.

CURRENT PUBLIC MANUALstatx(2) — STATX_DIOALIGN

Shows the modern interface for querying required user-memory and file-offset alignment for direct I/O where supported.

https://man7.org/linux/man-pages/man2/statx.2.html

# Compare behavior experimentally on a disposable test file. # Buffered path: dd if=test.img of=/dev/null bs=1M status=progress # Direct path (support/alignment depends on filesystem/device): dd if=test.img of=/dev/null bs=1M iflag=direct status=progress # Do not infer storage-device latency from one run alone: # buffered runs may be page-cache hits, while direct I/O has different alignment, # queueing and readahead behavior.

DAX can map storage directly into a process: filesystem extent → PFN → CPU load/store without the page cache

The page cache normally stands between a file and a process mapping: file data is copied/read into RAM pages and those pages are mapped into userspace. Linux Direct Access (DAX) exists for storage that is directly byte-addressable by the CPU. For a DAX file mapping, the filesystem can map storage-backed page frames directly into the process instead of creating ordinary page-cache copies.

ORDINARY FILE mmap() virtual address ↓ PTE page-cache RAM page ↓ filesystem/block I/O when cache fill/writeback is needed storage device DAX FILE mmap() virtual address ↓ page fault / filesystem extent lookup DAX translates file offset toward storage page frame (PFN) ↓ install mapping CPU load/store ↓ directly addressed storage media / DAX-capable backing FS-DAX filesystem still owns names, inodes, permissions, extents, allocation but file data mapping bypasses the page cache DEV-DAX character-device style direct mapping of a device address range without a normal filesystem namespace

Concept	Important distinction
page-cache I/O	File bytes are represented by cache pages/folios in ordinary RAM; read/writeback moves data between those pages and storage.
O_DIRECT	Direct-I/O system calls bypass ordinary page-cache buffering for transfers, but application accesses still occur through a user buffer in memory.
filesystem DAX (fsdax)	A DAX-capable filesystem maps file-backed storage directly into userspace while preserving normal file/inode/extents semantics.
device DAX (devdax)	Maps a DAX device address range directly through a character device; it is not a normal filesystem file hierarchy.
persistent memory	Memory-like nonvolatile media can make DAX especially useful, but persistence ordering/flush rules remain a separate programming concern.

DAX is not a synonym for durability. It primarily changes the data path by removing page-cache copies. Software that needs crash persistence must still obey the platform/media/filesystem persistence and ordering model; a CPU store becoming visible is not automatically the same event as becoming power-fail durable.

DAX INSPECTION IDEAS # Filesystems can expose active DAX state through statx attributes. # On a machine with appropriate hardware/configuration, inspect mount options, # block topology and /dev/pmem* or /dev/dax* devices. findmnt lsblk -o NAME,TYPE,SIZE,FSTYPE,MOUNTPOINTS # Compare the mental path: # buffered mmap → page-cache folio → storage I/O # DAX mmap → storage-backed PFN mapped into the process

KERNEL DOCSLinux — Direct Access (DAX) for files

Current kernel documentation explaining why DAX removes page-cache copies, how DAX file mappings work, filesystem support, direct-access driver requirements and important limitations.

https://docs.kernel.org/filesystems/dax.html

PUBLIC DOCUMENTATIONLinux NVDIMM / persistent-memory documentation

Practical map of persistent-memory namespaces, including raw/sector/fsdax/devdax modes and the distinction between filesystem DAX and device DAX.

https://nvdimm.docs.kernel.org/index.html

Buffered writes need backpressure: dirty folios, background writeback and throttling keep RAM from becoming an unlimited write queue

A buffered write() can finish after copying data into the page cache, long before the storage device has persisted those bytes. That is useful because RAM absorbs bursts and lets the filesystem merge or delay work, but it creates a resource-control problem: writers could otherwise dirty memory faster than the device can drain it. Linux therefore tracks dirty/writeback state, starts background writeback at lower thresholds, and eventually throttles tasks that keep dirtying memory when the system is out of balance.

APPLICATION STREAMS BUFFERED WRITES write() copies bytes into page-cache folios ↓ folios/inode become DIRTY ↓ dirty memory grows BELOW BACKGROUND THRESHOLD application can keep dirtying quickly BACKGROUND THRESHOLD CROSSED ↓ flusher/writeback work starts selecting dirty mappings/folios ↓ filesystem maps dirty file ranges to storage ↓ dirty state transitions into WRITEBACK state ↓ block I/O → controller → device ↓ completion writeback state clears; errors are recorded for later reporting DIRTYING OUTRUNS STORAGE ↓ balance_dirty_pages()-style control observes dirty state + writeback progress ↓ writer is paced / may sleep while storage catches up ↓ production rate is pushed toward a sustainable drain rate fsync()/fdatasync() ↓ explicitly starts/waits for required file writeback and reports relevant errors ↓ filesystem/device durability rules still determine when media state is safe

Term	Role
dirty folio	Page-cache memory modified relative to its backing file/storage state.
writeback folio	Cached file data for which storage I/O has been started and is still in progress.
background dirty threshold	Lower control point at which kernel flusher activity begins before writers must be strongly throttled.
dirty limit	Higher control point beyond which tasks generating dirty data are slowed so RAM does not fill with unpersisted writes.
`balance_dirty_pages()`	Core writeback-control mechanism used to pace dirtying tasks against dirty state and writeback progress.
`fsync()`	Explicit synchronization boundary that drives/waits for required writeback and surfaces relevant writeback errors.

Three different bottlenecks can coexist. Page-cache dirty throttling controls how much modified file data occupies RAM; the block layer separately queues requests; and the drive can have its own volatile write cache and flash translation layer. “write() returned” therefore says much less about physical persistence than many first-time systems programmers assume.

KERNEL DOCSLinux VM dirty/writeback controls

Documents dirty_background_bytes/ratio, dirty_bytes/ratio, dirty_expire_centisecs and periodic writeback controls—the user-visible policy knobs around dirty-memory pacing.

https://docs.kernel.org/admin-guide/sysctl/vm.html

SEE EARLIERVFS/page-cache internals referenced in the file-read section

The existing VFS and memory-management references define folio dirty/writeback state, writeback_control, file mappings and dirty-balancing APIs; this section connects them into the write-pressure control loop instead of duplicating those external links.

SEE EARLIERCrash consistency and fsync

Continue there for the separate question of ordering metadata/data, journal commits, cache flushes/FUA and what an application must do to make updates recoverable after power loss.

# Observe dirty/writeback memory in real time: grep -E '^(Dirty|Writeback):' /proc/meminfo # Inspect current policy values: sysctl vm.dirty_background_bytes vm.dirty_background_ratio vm.dirty_bytes vm.dirty_ratio vm.dirty_expire_centisecs vm.dirty_writeback_centisecs # On a test machine, vmstat can show blocked tasks and I/O pressure while # a sustained buffered writer exceeds storage throughput: vmstat 1

Below bio is blk-mq: per-CPU staging, request merging, tags, hardware queues and out-of-order completion

The block layer does not hand every bio directly to a storage controller one at a time. blk-mq turns bios into struct request objects, may merge adjacent work, optionally runs an I/O scheduler, then maps software submission contexts to one or more hardware dispatch queues that correspond closely to real device queues.

FILESYSTEM / SWAP / RAW BLOCK USER submit_bio(bio) ↓ blk-mq creates/merges into struct request FAST CASE request can go directly to driver/hardware queue ↓ blk_mq_try_issue_directly() STAGING CASE merge opportunity or active I/O scheduler ↓ per-CPU/per-node software context: struct blk_mq_ctx ↓ merge adjacent sectors / plug short bursts I/O scheduler may reorder for latency/fairness/performance SOFTWARE CONTEXT MAPPING CPU0 ctx ─┐ CPU1 ctx ─┼→ hardware context hctx0 → NVMe SQ0 CPU2 ctx ─┘ CPU3 ctx ─┐ CPU4 ctx ─┼→ hardware context hctx1 → NVMe SQ1 CPU5 ctx ─┘ DISPATCH allocate DRIVER TAG from hctx tag set ↓ driver queue_rq(request) ↓ driver maps request payload to DMA descriptors/PRPs/SGLs ↓ write device SQ entry ring doorbell ↓ blk_mq_start_request() starts timeout accounting DEVICE RESOURCE SHORTAGE queue_rq returns resource/busy status ↓ request may sit on hctx->dispatch / queue is rerun later COMPLETION device finishes tag 27 before tag 19 ↓ IRQ / poll driver identifies request from device/block-layer tag ↓ blk_mq_complete_request() / blk_mq_end_request() ↓ bio end_io callbacks → filesystem/page-cache waiter completion Neither blk-mq nor device protocols promise submission-order completion; filesystem/barrier/FUA logic must express ordering where correctness requires it.

blk-mq object	Role
bio	Block-I/O description from higher layers containing operation, sector and memory segments.
struct request	One driver-facing request, potentially containing/merging multiple bios.
blk_mq_ctx	Software staging/submission context normally associated with a CPU.
blk_mq_hw_ctx	Hardware dispatch context mapping software work toward one device/hardware queue.
request tag	Integer identifying an in-flight request without a linear completion search.
scheduler tag	Tag allocated while request is owned by an I/O scheduler rather than yet dispatched to the driver.
plugging	Temporarily collecting I/O so adjacent requests can be merged before dispatch.
request merge	Combines compatible adjacent block ranges to reduce commands and overhead.
I/O scheduler	Optional blk-mq policy reordering work for fairness, latency or device-specific performance goals.
hctx dispatch list	Temporary holding list for requests ready to go but rejected because hardware/driver resources are unavailable.
queue_rq()	Low-level block-driver callback asked to submit a request to hardware.
poll()	Driver/block path for checking completion without waiting for an interrupt where supported.
timeout	blk-mq tracking used when a started request fails to complete within the configured period.

One hardware queue is not necessarily one CPU. blk-mq maps many software CPU contexts onto the hardware contexts the driver/device actually supports. NVMe often exposes many hardware submission queues; SATA/AHCI exposes far less parallelism.

BLOCK-QUEUE OBSERVATION # scheduler selected for a block device cat /sys/block/nvme0n1/queue/scheduler 2>/dev/null # queue limits grep . /sys/block/nvme0n1/queue/{nr_requests,nomerges,max_segments,max_sectors_kb,logical_block_size} 2>/dev/null # multiqueue sysfs structure find /sys/block/nvme0n1/mq -maxdepth 2 -type f -o -type d 2>/dev/null | head -100 # block tracepoints perf list | grep -Ei 'block:(bio|rq|block_rq|block_bio)' | less # iostat -x / pidstat -d can show latency/queueing at a higher level if installed. # Do not change schedulers/queue depths on production storage casually.

KERNEL DOCSLinux queue sysfs documentation

Current observable queue properties including scheduler, request count, merge policy, segment/sector limits and polling-related settings.

https://docs.kernel.org/block/queue-sysfs.html

KERNEL DOCSLinux block tracepoints

Generic tracepoint documentation for following request lifecycle through the running kernel when block trace events are enabled.

https://docs.kernel.org/trace/events.html

An I/O scheduler sits between blk-mq request creation and driver dispatch: reorder enough for latency/fairness, then let the device schedule again

blk-mq can dispatch requests directly, but it can also pass them through an elevator/I/O scheduler. The scheduler only sees the block requests Linux has built; it can merge, delay, reorder or prioritize those requests before they reach the driver. Modern devices then have their own queues, firmware and media schedulers, so the Linux I/O scheduler is one policy layer—not the final authority over physical execution order.

processes / filesystems / writeback ↓ bios blk-mq request creation / merge opportunities ↓ OPTIONAL I/O SCHEDULER ├── none │ minimal scheduler-layer reordering; dispatch toward blk-mq/hardware ├── mq-deadline │ sector-order batching + age/deadline pressure to bound starvation/latency ├── BFQ │ per-process/group budgeted proportional-share scheduling + latency heuristics └── Kyber (where built/available) feedback/throttling toward read/synchronous-write latency targets ↓ blk_mq_hw_ctx / driver queue_rq() ↓ NVMe/SCSI/SATA controller queue(s) ↓ DEVICE FIRMWARE MAY REORDER AGAIN ↓ flash channels / NAND FTL / disks / remote storage ↓ completion may arrive out of submission order

Choice	What it is trying to control
`none`	No full elevator policy at this layer; useful when software/device queueing already provides the desired behavior or scheduler overhead is unwanted.
`mq-deadline`	Balances locality/batching with deadline-style aging so requests—especially reads—do not wait indefinitely behind a stream of other I/O.
BFQ	Budget-based proportional sharing with strong fairness and interactive/soft-real-time latency goals; can trade throughput for service guarantees.
Kyber	Latency-oriented scheduler that throttles in-flight work toward target read and synchronous-write latencies where available.
I/O priority	Per-task/process-class priority information interpreted only by schedulers that support it; current kernel docs identify BFQ and mq-deadline support.
device queue depth	Amount of concurrent work exposed to hardware; more parallelism can improve throughput but can also increase tail latency.
internal controller scheduling	Firmware/device-level ordering below Linux; especially important for NVMe and managed flash.

“Best scheduler” is workload- and device-dependent. A rotational disk, a SATA SSD, a many-queue NVMe SSD, a virtual disk and a remote block device expose very different latency/parallelism behavior. Also, selecting none does not mean requests execute instantly or strictly FIFO—the driver, transport and device may still queue and reorder them.

OBSERVATION / TEST LAB # Current scheduler is in brackets; available names depend on kernel/device cat /sys/block/nvme0n1/queue/scheduler 2>/dev/null # Scheduler-specific tunables if an elevator exposes them find /sys/block/nvme0n1/queue/iosched -maxdepth 1 -type f -print 2>/dev/null # Queue depth and request limits cat /sys/block/nvme0n1/queue/nr_requests 2>/dev/null # Per-process I/O priority if ionice is installed ionice -p $$ 2>/dev/null # Measure before changing scheduler policy on production storage.

KERNEL DOCSLinux block-layer documentation index

Current entry point for blk-mq I/O scheduler documentation, including BFQ, deadline/Kyber tunables, I/O priorities and scheduler switching.

https://docs.kernel.org/block/index.html

KERNEL DOCSLinux block I/O priorities

Defines I/O priority classes and notes which blk-mq schedulers currently interpret them, clarifying that priority semantics depend on the active scheduler.

https://docs.kernel.org/block/ioprio.html

KERNEL DOCSBFQ (Budget Fair Queueing)

Detailed current documentation for BFQ's budgeted proportional-share model, latency heuristics, group scheduling and throughput/fairness tradeoffs.

https://docs.kernel.org/block/bfq-iosched.html

The same file page can become either shared dirty page-cache data or a private anonymous COW page

MAP_SHARED and MAP_PRIVATE may initially fault in the same file-backed page-cache folio. The difference becomes decisive on writes. A shared writable mapping dirties the file-backed page so changes can propagate to the underlying file. A private writable mapping instead uses copy-on-write: the process gets a private anonymous page, leaving the file and other private mappers unchanged.

FILE data.bin contains page F0 Process A: mmap(..., PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0) Process B: mmap(..., PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, 0) FIRST READ FAULT IN A OR B VMA says file-backed mapping ↓ page cache contains F0? ├── yes → map cached folio/page └── no → read F0 from storage into page cache, then map At this point both processes can read the SAME cached file contents. A WRITES THROUGH MAP_SHARED CPU store hits write-protected/shared-write fault path as required ↓ filesystem/page_mkwrite-style checks allocate/prepare blocks if needed ↓ file-backed page-cache folio becomes dirty ↓ other coherent MAP_SHARED mappings can observe new bytes ↓ later writeback/msync/fsync semantics underlying file is updated B WRITES THROUGH MAP_PRIVATE PTE is not writable to the original shared file page ↓ write fault allocate a new anonymous page copy original file-page contents into it apply B's write ↓ replace B's PTE with writable private anonymous page ↓ B now diverges from file/page-cache copy Process B /proc/<pid>/numa_maps or smaps can report private COW pages as anonymous. FORK AFTER MAP_PRIVATE parent and child can initially share B's private page read-only ↓ first writer triggers another ordinary anonymous COW split MAP_PRIVATE therefore combines file-backed demand paging with anonymous COW on write.

Mapping type	Read miss	Write behavior	File changes?
MAP_SHARED	Fault/load file page through page cache	Write dirties shared file-backed memory after filesystem write-fault checks	Yes, subject to writeback/msync/fsync semantics.
MAP_PRIVATE	Fault/load file page through page cache	First private write creates anonymous COW copy	No; mapper's private writes are not carried back to underlying file.
read-only mapping	Uses page cache	Write faults with protection violation because PROT_WRITE absent	No write possible through mapping.
anonymous MAP_PRIVATE	Demand-zero/shared-zero-page style start is possible	Write allocates/private anonymous page	No backing file exists.

MAP_PRIVATE does not mean the file is copied at mmap() time. The mapping can share clean page-cache data for reads and allocate private anonymous memory only for pages the process actually writes.

MAPPING/COW LAB Create a multi-page file and map it twice in a small C program: one MAP_SHARED one MAP_PRIVATE touch every page for reading ↓ inspect /proc/<PID>/smaps and /proc/<PID>/numa_maps write one page through MAP_PRIVATE ↓ look for Anonymous / Private_Dirty growth in that VMA write another page through MAP_SHARED ↓ observe Shared_Dirty/file-backed state and verify file content after msync()/fsync as appropriate Useful tracing: perf stat -e page-faults,minor-faults,major-faults ./mmap_test strace -e mmap,msync,madvise,munmap ./mmap_test

CURRENT PUBLIC MANUALnuma(7) — file mappings and private COW pages

Current manual notes that private file mappings can generate copy-on-write pages and that those pages appear as anonymous memory in numa_maps output.

https://www.man7.org/linux/man-pages/man7/numa.7.html

io_uring: userspace and kernel communicate through shared SQ/CQ rings

io_uring is a Linux asynchronous-I/O interface whose primary communication mechanism is shared memory rather than one blocking syscall per operation. Userspace writes Submission Queue Entries (SQEs), advances a shared producer tail, and notifies the kernel when necessary. The kernel consumes those submissions and writes Completion Queue Entries (CQEs) back into a separate shared ring.

SETUP fd = io_uring_setup(entries, &params) ↓ mmap shared SQ ring metadata mmap SQE array mmap shared CQ ring metadata/CQEs SUBMIT READ userspace chooses free SQE sqe->opcode = IORING_OP_READ sqe->fd = file fd sqe->addr = user buffer sqe->len = bytes sqe->off = file offset sqe->user_data = application token ↓ publish SQE index into SQ array store-release new SQ tail ↓ io_uring_enter() notifies kernel [ordinary mode] or SQPOLL kernel thread notices new tail [SQPOLL mode] KERNEL reads SQ head/tail ↓ consumes SQE → builds internal io_kiocb/request ↓ operation follows VFS/socket/block/device async path ↓ may complete inline, asynchronously, via worker thread, IRQ/poll, etc. COMPLETE kernel writes CQE: user_data = original token res = bytes or -errno flags = completion metadata ↓ kernel publishes new CQ tail ↓ userspace observes CQE handles result advances CQ head WHY RINGS HELP batch many operations per syscall shared producer/consumer metadata avoids copying a syscall argument struct each time SQPOLL can remove submission syscalls while busy registered files/buffers can avoid repeated lookup/pinning setup But the operation still obeys filesystem/network/block/device semantics underneath.

io_uring object/feature	Role
SQE	Submission Queue Entry describing one requested operation such as read/write/accept/connect/timeout.
SQ	Shared submission ring: userspace produces work; kernel consumes it.
CQE	Completion Queue Entry containing result/error plus application-chosen user_data.
CQ	Shared completion ring: kernel produces results; userspace consumes them.
head/tail	Monotonic producer/consumer indices whose masked low bits select a physical ring slot.
ring mask	Power-of-two wrap mask mapping monotonically increasing indices into ring-array positions.
io_uring_enter()	Syscall that can submit queued work and/or wait for completions in ordinary operation.
SQPOLL	Dedicated kernel thread polls the SQ so active applications can submit without an enter syscall.
IOPOLL	Completion polling mode for supported storage devices instead of interrupt-driven completion.
registered/fixed files	Pre-register file references to reduce repeated per-I/O fd-table lookup/ref overhead.
registered/fixed buffers	Pre-register user memory so repeated operations can avoid some pin/map setup.
linked SQEs	Submission dependency/chaining mechanism allowing one request to start only after another completes.
multishot operation	One SQE capable of producing multiple CQEs for repeated events where supported.
zero-copy receive	Modern networking path where supported NIC/kernel configuration can place payload data directly in registered userspace memory.

io_uring is not kernel bypass. The kernel still validates and executes the operations and the normal VFS/TCP/block/device code paths still matter. The rings mainly change how requests/completions cross the userspace↔kernel boundary and how much setup/batching/polling overhead can be amortized.

OBSERVATION / BUILD LAB # If liburing examples are installed or cloned, trace setup/mapping/enter calls strace -e io_uring_setup,io_uring_enter,io_uring_register,mmap,munmap ./uring_program # Compare syscall counts against repeated synchronous pread()/pwrite() strace -c ./sync_program strace -c ./uring_program # perf context-switch/system-call counters can help quantify batching/SQPOLL tradeoffs perf stat -e context-switches,cpu-migrations ./uring_program # Read the ring head/tail code in io_uring(7): # producer publishes an SQ tail with release semantics; # consumer similarly advances shared heads. # SQPOLL consumes a CPU thread while active/idle-window polling; # it is not automatically the best choice for low-rate workloads.

PUBLIC MANUALio_uring(7) — full programming model

Excellent public manual explaining shared SQ/CQ mappings, SQEs/CQEs, head/tail publication, batching and SQPOLL.

https://man7.org/linux/man-pages/man7/io_uring.7.html

PUBLIC MANUALio_uring_setup(2)

Defines queue sizes/offsets, mmap layout, head/tail/ring masks and setup modes such as SQPOLL/IOPOLL.

https://man7.org/linux/man-pages/man2/io_uring_setup.2.html

PUBLIC MANUALio_uring_enter(2)

Current operation submission/wait syscall, including linked/drained request behavior and submission/completion controls.

https://man7.org/linux/man-pages/man2/io_uring_enter.2.html

CURRENT TOOL DOCSio_uring setup flags — current upstream liburing manual

Current detailed flag reference for single-issuer optimization, SQPOLL, IOPOLL, queue sizing and task-run behavior.

https://www.man7.org/linux/man-pages/man7/io_uring_setup_flags.7.html

KERNEL DOCSio_uring zero-copy network receive

Current kernel feature showing how shared rings extend down to modern NIC receive: payload can land directly in userspace registered memory while the kernel TCP stack still processes headers.

https://docs.kernel.org/networking/iou-zcrx.html

SOURCE / EXAMPLESliburing source/examples

Official userspace helper library and examples; useful for moving from the raw io_uring_setup/mmap interface to practical programs.

https://github.com/axboe/liburing

One buffered file write: write() → page cache → filesystem → block layer → NVMe → flash

A successful write() usually does not mean the SSD has already programmed NAND. With ordinary buffered I/O, Linux commonly copies/modifies data in the page cache, marks it dirty, and returns. Later writeback converts dirty file ranges into storage I/O. Durability requires an explicit synchronization contract such as fsync() plus correct filesystem/device behavior.

userspace write(fd, buf, len) ↓ system call / VFS filesystem maps file offset → logical filesystem blocks/extents ↓ copy data into PAGE CACHE folio(s) ↓ mark folio/range DIRTY ↓ write() may return here later: background writeback OR fsync()/memory pressure ↓ filesystem allocates/maps blocks + prepares metadata as needed ↓ build BIO(s) describing block I/O ↓ blk-mq software queue → hardware dispatch queue ↓ NVMe driver creates SQ command in host RAM ↓ doorbell → NVMe controller DMA-fetches command/data ↓ SSD FTL maps logical blocks to NAND pages/blocks ↓ NAND program + ECC/controller bookkeeping ↓ NVMe completion queue entry → interrupt/poll ↓ BIO/writeback completion → dirty/writeback accounting cleared ↓ fsync() can report completion/error according to filesystem/device guarantees

Layer/object	What it abstracts
VFS	Common file API over many concrete filesystems.
page cache	RAM-backed cache of file contents used for buffered reads/writes.
dirty folio/page	Memory copy has been modified and must eventually be persisted to backing storage.
filesystem extent/block mapping	Maps file offsets to allocated blocks and manages metadata/consistency rules.
bio	Block-I/O description for ranges/pages headed to a block device.
blk-mq	Linux multiqueue block layer mapping software submission queues onto hardware dispatch queues.
NVMe SQ/CQ	Device-visible host-memory submission/completion rings.
FTL	SSD controller mapping from logical block addresses to NAND placement, erase blocks, wear management and garbage collection.
fsync	Requests synchronization according to filesystem semantics; also surfaces delayed writeback errors.

Crash consistency: write() success, filesystem consistency and durable application state are not the same guarantee

Storage has several buffers and reorder points: page cache, filesystem transactions, block queues, controller DRAM/SRAM and device flash/media. A filesystem journal mainly protects filesystem metadata consistency; it does not automatically make every recently returned application write durable. Applications that need a crash-safe update protocol must use the filesystem's persistence primitives correctly.

EXT4/JBD2 METADATA TRANSACTION (conceptual) application changes file / directory ↓ page cache + in-memory ext4 metadata become dirty ↓ JBD2 groups metadata changes into a transaction ↓ journal descriptor + journaled metadata blocks ↓ storage flush/order as required ↓ JOURNAL COMMIT RECORD ↓ transaction is recoverable after crash ↓ later CHECKPOINT same metadata reaches its normal/home filesystem locations CRASH DURING CHECKPOINT? journal still has committed transaction → replay after reboot CRASH BEFORE VALID COMMIT? incomplete transaction is not replayed as committed state SAFE FILE-REPLACEMENT PATTERN (conceptual POSIX/Linux) write new.tmp fsync(new.tmp) # persist file contents + needed file metadata rename(new.tmp, final) # atomically switch directory name within filesystem semantics fsync(parent_directory) # persist directory entry/rename itself Exact guarantees depend on filesystem, mount mode and storage-stack correctness.

Operation/concept	Guarantee / non-guarantee
write()	Typically copies/modifies buffered file data and can return before durable media persistence.
ext4 journal	Primarily makes metadata transactions recoverable/consistent across crashes; default ordered mode does not journal ordinary file data itself.
data=ordered	Ext4 writes associated file data to final location before committing related metadata transaction.
data=journal	Journals data as well as metadata; stronger but slower.
data=writeback	Metadata journaling without ordered data-before-metadata guarantee.
fsync(file)	Requests file data and associated metadata be flushed through storage caches as required; blocks until device reports completion.
fdatasync(file)	Like fsync but may omit metadata not needed to retrieve file data correctly.
fsync(directory)	Needed when the durability requirement includes a newly created/renamed directory entry.
device volatile write cache	Can acknowledge writes before nonvolatile media; filesystems use flush/FUA mechanisms for persistence ordering.
FUA / preflush	Block-layer/device commands constraining cache persistence/order without globally disabling a fast write-back cache.
journal replay	Reapplies complete committed transactions after an unclean shutdown.
checkpoint	Moves committed journaled changes to their ordinary filesystem home locations so journal space can be reused.

Filesystem consistency is weaker than application durability. A journal can make the filesystem structurally recoverable while a recently written application's bytes are still legitimately lost after power failure. For important state, reason explicitly about the sequence of write, fsync/fdatasync, rename/link operations and directory fsync.

KERNEL DOCSLinux ext4 — JBD2 journal internals

Current kernel documentation explains descriptor/data/revocation/commit blocks, checkpointing and replay; committed transactions can be replayed after a crash.

https://www.kernel.org/doc/html/latest/filesystems/ext4/journal.html

KERNEL DOCSLinux ext4 — data modes and write barriers

Current ext4 documentation distinguishes data=ordered, data=writeback and data=journal and explains that write barriers enforce ordering around volatile disk write caches.

https://cdn.kernel.org/doc/html/latest/admin-guide/ext4.html

KERNEL DOCSLinux Journalling API

Deep public JBD2 interface documentation covering transaction handles, commits, callbacks, checkpoint/flush behavior and fast commits.

https://www.kernel.org/doc/html/latest/filesystems/journalling.html

PUBLIC MANUALfsync(2) — current Linux manual

Current man page: fsync flushes file data and metadata through disk caches and waits for completion, but does not by itself guarantee the containing directory entry is durable.

https://man7.org/linux/man-pages/man2/fsync.2.html

KERNEL DOCSLinux — volatile write-back cache control

Explains why a device can report completion before nonvolatile persistence and how REQ_PREFLUSH/FUA give filesystems ordering and durability control.

https://www.kernel.org/doc/html/v5.18/block/writeback_cache_control.html

KERNEL DOCSLinux ext4 — atomic block writes

Advanced current material showing how torn-write prevention differs from ordinary journaling and requires specific allocation/I/O constraints.

https://www.kernel.org/doc/html/latest/filesystems/ext4/atomic_writes.html

A concrete ext4 buffered write: page cache first, delayed block placement later, metadata consistency through JBD2

The generic VFS/page-cache path becomes more concrete in ext4. A buffered write() can dirty page-cache folios before ext4 has committed to final physical block locations. With delayed allocation, the filesystem can wait until writeback to choose larger, more contiguous extents. The file's logical-to-physical mapping is represented by an extent tree, while metadata updates participate in JBD2 journal transactions so a crash does not leave filesystem metadata half-updated. These mechanisms improve placement and recoverability, but they do not turn write() into a durability guarantee.

write(fd, bytes) ↓ VFS / ext4 buffered-write path copy data into page-cache folios mark folios dirty ↓ DELAYED ALLOCATION: logical dirty range may not yet have final disk blocks ↓ writeback / fsync / memory pressure / commit activity ext4 allocator chooses physical space ↓ preferably contiguous runs extent tree maps file logical blocks → physical filesystem blocks ↓ submit data I/O through block layer ↓ metadata changes join a JBD2 transaction ↓ data=ordered default required data ordering before metadata transaction commit ↓ JBD2 descriptor/metadata + commit record reach journal ↓ later checkpoint metadata blocks reach their home locations ↓ crash recovery replays complete committed journal transactions

ext4 mechanism	What problem it solves
block groups	Partition filesystem metadata/data into locality domains and give allocators manageable regions.
extent tree	Represent a long contiguous mapping as a range instead of one pointer per filesystem block.
delayed allocation	Delay exact physical placement until more of the write pattern is known, improving extent size/locality and reducing fragmentation.
unwritten extent	Reserve disk space while guaranteeing reads return zeros until the extent is converted after successful data I/O.
JBD2 transaction	Make groups of metadata updates recoverable/atomic with respect to crashes by logging a transaction and commit record before checkpointing home blocks.
`data=ordered`	The common mode: file data itself is not generally journaled, but required data writes are ordered before the metadata transaction that makes them reachable.
`fsync()`	Requests the durability needed by the application; ordinary buffered `write()` success only means bytes were accepted into the kernel's write path.

Delayed allocation is about placement, not about promising that data is durable. A dirty page can exist in RAM with no final disk block chosen yet. The separate writeback, journal, cache-flush and fsync() rules determine what survives a crash.

SEE EARLIERPage cache → extent → block I/O

The earlier read-path section links the ext4 extent-tree documentation and shows the reverse direction: a file offset is resolved through cached filesystem metadata toward block I/O and storage DMA.

KERNEL DOCSext4 block and inode allocation policy

Current kernel documentation for the multiblock allocator, delayed allocation, block-group locality and the point at which dirty logical ranges are assigned physical storage.

https://docs.kernel.org/filesystems/ext4/allocators.html

KERNEL DOCSext4 JBD2 journal and transaction layout

Current ext4 documentation for metadata journaling, commit records, checkpointing, replay, data modes and fast commits.

https://docs.kernel.org/filesystems/ext4/journal.html

Why SSDs need an FTL: NAND pages can be programmed, but erase happens in much larger blocks

An SSD pretends to be a simple array of logical blocks, but NAND flash has awkward physical rules. Data is read/programmed in pages, while erasure happens at the larger erase-block level; cells also wear with program/erase cycling. The controller therefore maintains a Flash Translation Layer (FTL) that remaps host LBAs to changing physical locations.

HOST VIEW LBA 100 → 'sector/block 100' SSD CONTROLLER / FTL L2P mapping table says: LBA 100 → NAND die 2 / block 81 / page 37 UPDATE LBA 100 old physical page cannot simply be overwritten in place ↓ program NEW physical page elsewhere ↓ update L2P mapping: LBA 100 → new page ↓ old page becomes INVALID/stale GARBAGE COLLECTION choose erase block containing valid + invalid pages ↓ copy remaining valid pages elsewhere ↓ erase ENTIRE block ↓ block becomes free for future programming WEAR LEVELING choose placements/migrations so erase cycles are not concentrated TRIM / DISCARD filesystem tells device certain LBAs are no longer needed ↓ FTL can mark associated physical data invalid earlier, reducing pointless copying during later garbage collection

Term	Physical meaning / consequence
page	Basic NAND read/program unit; many pages belong to one erase block.
erase block	Large collection of pages erased together.
L2P table	Metadata mapping host logical addresses to current physical NAND location.
out-of-place update	New version written elsewhere; old physical page invalidated instead of overwritten in place.
garbage collection	Relocates still-valid pages so a whole block can be erased/reused.
wear leveling	Distributes program/erase stress so a small subset of blocks does not wear out prematurely.
bad-block management	Retires unusable/failing blocks and substitutes reserved capacity.
over-provisioning	Physical flash capacity reserved/not exposed as normal host LBAs, giving controller working space for GC/replacement/endurance.
write amplification	Physical NAND writes exceed host logical writes because metadata, migration and garbage collection also write data.
SLC / MLC / TLC / QLC	One/two/three/four information bits per flash cell; more levels increase density but narrow voltage margins and intensify ECC/endurance challenges.

Host LBA stability is an illusion created by controller metadata. The physical NAND page holding a logical block may move repeatedly during writes, garbage collection, wear leveling or bad-block retirement while the host continues using the same LBA.

DIRECT PDFKIOXIA — Logical Block Address → Physical Block Address Translation (direct PDF)

Excellent no-login four-page brief showing L2P mapping tables, physical relocation while LBA remains unchanged, metadata overhead and write-amplification factor.

https://americas.kioxia.com/content/dam/kioxia/shared/business/memory/ufs/asset/productbrief/KIOXIA_Understanding_Logical_Block_Address_Tech_Brief.pdf

DIRECT PDFKIOXIA — Garbage Collection in NAND Flash (direct PDF)

Shows why valid pages must be copied out before an erase block can be reclaimed, and why GC both consumes bandwidth and adds flash wear.

https://americas.kioxia.com/content/dam/kioxia/shared/business/memory/ufs/asset/productbrief/KIOXIA_Understanding_Garbage_Collection_Tech_Brief.pdf

DIRECT PDFKIOXIA — Wear Leveling in NAND Flash (direct PDF)

Explains static versus dynamic wear leveling and why controllers distribute write/erase cycles across available blocks.

https://americas.kioxia.com/content/dam/kioxia/shared/business/memory/ufs/asset/productbrief/KIOXIA_Understanding_Wear_Leveling_Tech_Brief.pdf

DIRECT PDFKIOXIA — ECC in NAND Flash (direct PDF)

Explains increasing raw bit-error pressure, spare-area ECC, controller-side correction and the progression from simple codes toward BCH/LDPC-class correction.

https://americas.kioxia.com/content/dam/kioxia/shared/business/memory/ufs/asset/productbrief/KIOXIA_Understanding_ECC_Tech_Brief.pdf

DIRECT PDFKIOXIA — Bad Block Management (direct PDF)

Shows initial and runtime bad blocks, reserved replacements and why real NAND is designed around imperfect media rather than assuming every block remains usable forever.

https://americas.kioxia.com/content/dam/kioxia/en-us/business/memory/mlc-nand/asset/KIOXIA_Managed_Flash_BOS_P2_Understanding_Bad_Block_Management_Tech_Brief.pdf

PUBLIC MANUALLinux fstrim(8) — filesystem discard/TRIM

Current no-login manual. fstrim tells a supporting block device which filesystem ranges are unused so SSD/thin-storage layers can treat that logical space as discardable.

https://man7.org/linux/man-pages/man8/fstrim.8.html

Deleting a file does not tell NAND which cells are useless unless free-space information travels down the stack: discard/TRIM closes that gap

A filesystem knows when logical blocks stop containing live file data, but an SSD normally sees only block reads and writes. Without another signal, the flash translation layer cannot know whether an old logical block still matters or merely contains stale data. A discard operation carries that information down the block stack. SATA commonly calls the device operation TRIM; NVMe exposes deallocation through Dataset Management. The device may use that hint to reduce future garbage-collection work and reclaim flash more efficiently.

unlink()/truncate()/filesystem block becomes free ↓ FILESYSTEM FREES LOGICAL EXTENT ↓ metadata update alone does NOT necessarily tell SSD immediately ↓ DISCARD POLICY ├── periodic FITRIM: submit filesystem free extents in batches └── online discard: issue discards as frees occur (filesystem/mount dependent) ↓ block layer forms discard ranges and respects device limits/granularity ↓ controller command ├── SATA/ATA: DATA SET MANAGEMENT / TRIM concept └── NVMe: Dataset Management with Deallocate attribute ↓ SSD FTL marks corresponding logical ranges as no longer needing old data ↓ later garbage collection can move fewer still-live NAND pages before erase IMPORTANT discard ≠ guaranteed immediate physical erase discard ≠ cryptographic secure erase discard says previous logical-block contents no longer need preservation

Layer/object	What it knows
filesystem allocator	Which logical filesystem extents are free and no longer assigned to live file data.
FITRIM / periodic trim	Asks a mounted filesystem to submit currently unused ranges for discard in batches.
block discard request	Generic Linux block-layer representation of ranges whose previous contents need not be preserved.
`discard_granularity`	Alignment/granularity constraint exported by a block device; zero indicates no discard support in the documented sysfs ABI.
`discard_max_bytes`	Software cap used to limit discard request size, often to control latency.
NVMe Deallocate	Dataset Management attribute indicating that listed logical-block ranges may be deallocated by the NVM subsystem.
FTL	Can treat deallocated LBAs as not requiring preservation when selecting/moving valid pages during flash garbage collection.

TRIM is a liveness hint, not a sanitization primitive. The host is saying “the old contents of these logical blocks no longer matter.” It is not promising that every underlying NAND cell is erased immediately or that forensic recovery is impossible. Secure data destruction is a different contract.

PUBLIC MANUALblkdiscard(8) — issue discard directly to a block device

Current util-linux manual for sending discard ranges at the raw block-device layer, distinct from filesystem-aware fstrim.

https://man7.org/linux/man-pages/man8/blkdiscard.8.html

KERNEL DOCSLinux block queue sysfs ABI — discard limits and granularity

Current kernel ABI documentation for discard_granularity, discard_max_bytes and discard_max_hw_bytes, including how unsupported discard is represented.

https://docs.kernel.org/admin-guide/abi-stable.html

OFFICIAL SPECNVM Express — current NVM Command Set specification

The current NVMe NVM command-set family. Dataset Management includes the Deallocate attribute used to tell an NVM subsystem that specified logical-block ranges may be discarded.

https://nvmexpress.org/specification/nvm-command-set-specification/

SEE EARLIERNAND pages, erase blocks, FTL and garbage collection

The earlier FTL section explains the physical reason discard is useful: NAND erase operates on large erase blocks, so controllers otherwise spend work preserving live pages mixed with stale ones.

Device Mapper turns sector ranges into a programmable block graph; LVM adds persistent allocation policy and metadata

A filesystem only needs a block device with sectors; those sectors do not have to map one-to-one onto a physical disk. Linux Device Mapper (DM) creates virtual block devices whose logical sector ranges are routed through one or more mapping targets. The target might simply add an offset, stripe across devices, encrypt data, implement thin provisioning or create snapshots. LVM2 is the higher-level userspace volume manager that records physical-volume/volume-group/logical-volume metadata and programs the corresponding Device Mapper tables in the kernel.

PHYSICAL STORAGE /dev/nvme0n1p3 /dev/nvme1n1p3 ↓ pvcreate ↓ pvcreate PV A PV B \ / \ LVM metadata / Volume Group (VG) extents pooled from PVs ↓ lvcreate / lvextend / pvmove ↓ userspace LVM computes mappings + loads DM table ↓ /dev/mapper/vg-data (logical block device) ↓ DEVICE-MAPPER TABLE, conceptually: logical sectors 0..N → linear PV A sectors X.. logical sectors N+1..M → linear PV B sectors Y.. ↓ filesystem / database / raw block user WRITE TO LOGICAL SECTOR L ↓ block layer Device Mapper finds table target covering L ↓ target translates / transforms request underlying block device + physical-sector offset ↓ controller / media TARGET STACKING filesystem ↓ dm-crypt ↓ dm-linear / LVM LV ↓ md RAID or physical device ↓ NVMe/SATA Different stacks are possible; ordering changes semantics.

Layer/object	Role
Device Mapper core	Kernel block-layer framework exposing mapped `/dev/dm-`/`/dev/mapper/` devices and dispatching each I/O through table targets.
DM table	Ordered logical-sector ranges: start, length, target type and target-specific parameters.
linear target	Maps a logical sector range onto a contiguous range of another block device; the simplest building block for logical volumes.
thin target/pool	Allocates physical blocks on demand from a shared pool and supports efficient snapshots through persistent mapping metadata.
snapshot target	Preserves an origin view using copy-on-write chunks; different from a filesystem snapshot because it operates on blocks.
PV	LVM Physical Volume: a disk/partition/block device initialized with LVM label/metadata participation.
VG	Volume Group: LVM allocation pool assembled from one or more PVs and divided into extents.
LV	Logical Volume: virtual block device allocated from a VG; LVM realizes it through one or more DM mappings/targets.
`dmsetup`	Low-level userspace tool that directly loads/changes Device Mapper tables; useful for seeing what LVM automates.

LVM is not a filesystem and Device Mapper is not RAID by definition. An LV is still a block device that normally needs a filesystem (or database/raw-block consumer). LVM can construct many layouts through DM targets, while Linux md is a separate software-RAID subsystem. They can be stacked in either sensible direction depending on the design, but each layer adds its own metadata, failure modes and recovery rules.

Resizing is a multi-layer operation. Making an LV larger changes the virtual block device size; the filesystem above it still needs its own grow operation. Shrinking is more dangerous because the filesystem must first move/validate data so no live blocks remain beyond the new block-device boundary.

CURRENT MANUALlvm(8) — LVM2 architecture and commands

Current upstream LVM manual defining PVs, VGs and LVs and explicitly explaining that LV blocks are stored on PVs according to mappings implemented by the kernel Device Mapper.

https://man7.org/linux/man-pages/man8/lvm.8.html

CURRENT MANUALdmsetup(8) — direct Device Mapper table control

Low-level interface for creating logical devices by loading tables that assign a target to each logical sector range. Excellent for seeing the mechanism underneath LVM.

https://man7.org/linux/man-pages/man8/dmsetup.8.html

KERNEL DOCSLinux dm-linear target

The simplest Device Mapper target: map one logical range onto a linear range of another block device. The kernel docs explicitly describe it as a basic building block of logical volume managers.

https://docs.kernel.org/admin-guide/device-mapper/linear.html

KERNEL DOCSLinux Device Mapper thin provisioning

Shows how a persistent metadata device and data pool map many virtual thin devices, allocating blocks on demand and supporting internal/external snapshots.

https://docs.kernel.org/admin-guide/device-mapper/thin-provisioning.html

Full-disk encryption can be a block-layer transform: LUKS unlocks keys; dm-crypt encrypts sectors on every I/O

Linux can present an encrypted partition as an ordinary virtual block device. Userspace tools such as cryptsetup parse LUKS metadata, obtain a passphrase/key/token and activate a Device Mapper mapping. After that, normal filesystems issue block I/O to /dev/mapper/…; the kernel's dm-crypt target transforms data between plaintext sectors above the mapping and ciphertext sectors on the underlying device. Applications and the filesystem do not need to encrypt individual files themselves.

BOOT / UNLOCK TIME passphrase / key file / hardware-backed credential ↓ cryptsetup reads LUKS metadata + keyslots ↓ derive / unwrap volume key ↓ create Device Mapper dm-crypt mapping ↓ /dev/mapper/cryptroot appears as a normal block device WRITE PATH filesystem / page cache / direct I/O ↓ BIO containing plaintext sectors Device Mapper → dm-crypt ↓ choose cipher/mode + sector-dependent IV/tweak ↓ ENCRYPT DATA using kernel crypto API ↓ ciphertext BIO underlying NVMe/SATA/virtual block device ↓ media stores ciphertext READ PATH media ciphertext → block layer → dm-crypt ↓ decrypt plaintext BIO → filesystem → process LUKS metadata is about key management and volume setup. dm-crypt is the runtime per-I/O block transformation.

Layer	Responsibility
LUKS	On-disk metadata format for encrypted-volume setup, including keyslots and parameters used to unlock the volume key.
cryptsetup	Userspace management tool that reads volume metadata, obtains credentials and asks Device Mapper to create/remove mappings.
Device Mapper	Kernel framework that stacks virtual block devices over other block devices using targets such as crypt, integrity, linear, thin and others.
dm-crypt	Kernel target that encrypts writes and decrypts reads, using the kernel crypto API and sector-dependent IV/tweak construction.
filesystem	Normally sees the decrypted virtual block device and manages files/directories without knowing the physical device holds ciphertext.
dm-integrity / authenticated modes	Can provide per-sector integrity metadata/authentication. Encryption alone does not automatically prove that ciphertext was not modified.
discard through encryption	Optional policy can pass discard/deallocation information downward, but doing so may reveal which logical regions are unused even though their contents remain encrypted.

Encryption and integrity are different properties. Conventional dm-crypt confidentiality prevents someone without the key from reading plaintext sectors, but unauthenticated encryption by itself is not a general tamper-detection mechanism. dm-integrity or authenticated encryption modes add separate integrity metadata/checks.

KERNEL DOCSLinux dm-crypt target documentation

Current kernel documentation for the Device Mapper crypt target: cipher/mode syntax, keys, IV generation, sector sizing, workqueues, discards and integrity-related options.

https://docs.kernel.org/admin-guide/device-mapper/dm-crypt.html

CURRENT PUBLIC MANUALcryptsetup(8) — LUKS and dm-crypt management

Current manual describing how cryptsetup creates dm-crypt mappings, the distinction between plain dm-crypt and LUKS metadata, and common open/close/status management operations.

https://man7.org/linux/man-pages/man8/cryptsetup.8.html

KERNEL DOCSLinux dm-integrity — per-sector integrity metadata

Shows the complementary integrity layer, including journaled metadata and the mode where dm-integrity combines with dm-crypt for authenticated disk encryption.

https://docs.kernel.org/admin-guide/device-mapper/dm-integrity.html

fscrypt encrypts selected directory trees inside a filesystem: policy + master key → per-file keys → encrypted contents and filenames

fscrypt is Linux's filesystem-level encryption framework used by supporting filesystems. Unlike dm-crypt, which transforms sectors for an entire block device, fscrypt can apply different encryption policies/keys to different directory trees on the same filesystem while leaving other files unencrypted. The filesystem/VFS still sees normal files and directories once the appropriate key is present.

SETUP / POLICY empty directory ↓ FS_IOC_SET_ENCRYPTION_POLICY / management tool fscrypt policy stored with directory • policy version / modes / flags • master-key identifier ↓ new child files/directories inherit policy UNLOCK userspace obtains strong master key (passphrase must be stretched/wrapped in USERSPACE; kernel expects a real key) ↓ FS_IOC_ADD_ENCRYPTION_KEY master key retained by filesystem/fscrypt context ↓ per-file nonce + KDF ↓ derived per-file/per-mode keys READ storage ciphertext → page/buffer I/O ↓ decrypt contents with file key ↓ page cache / application sees plaintext DIRECTORY LOOKUP on-disk encrypted filename ↓ key available filename decryption / matching ↓ normal dentry/pathname visible to process LOCK / KEY REMOVAL userspace requests master-key removal ↓ unused unlocked inodes/derived keys can be evicted ↓ without key, protected regular-file data cannot be opened normally and names may be exposed only in encrypted/no-key form according to fscrypt semantics.

Question	fscrypt answer
Granularity	Filesystem directory-tree policy; different protected trees can use different keys on one filesystem.
Contents	Regular-file contents are transparently encrypted/decrypted.
Filenames	Filename encryption is supported so directory entry names are not stored as plaintext when protected.
Other metadata	Most filesystem metadata such as file sizes, permissions and timestamps is not hidden merely by fscrypt.
Key derivation	Modern v2 policies derive subkeys/per-file keys from a master key; userspace is responsible for generating/stretching/wrapping secrets safely.
Page cache	fscrypt is integrated into supporting filesystems rather than stacking a second filesystem, avoiding a second encrypted+decrypted page-cache copy.
Inline crypto	Supported block devices/filesystems may use blk-crypto/inline-encryption hardware; otherwise CPU cryptography can perform the transform.
Integrity	Confidentiality encryption does not by itself authenticate all filesystem data/metadata. fs-verity/dm-verity solve different integrity problems.
Supported filesystems	Support is filesystem-specific; current kernel documentation lists ext4, F2FS, UBIFS and CephFS among implementations.

Three layers, three different jobs: dm-crypt hides block-device sector contents below the filesystem; fscrypt selectively hides file contents and filenames inside a supporting filesystem; fs-verity authenticates immutable file contents with a Merkle tree. They can be combined because confidentiality, granularity and authenticity are separate requirements.

NO-ACCOUNT INSPECTION / LEARNING LAB # fscrypt userspace tool (when installed) fscrypt status fscrypt status /path/to/mount # low-level tool can inspect/set policies on supported filesystems # fscryptctl get_policy /encrypted/directory # kernel API is ioctl-based; management tools are safer than inventing # your own passphrase/KDF/key-lifecycle design. # compare layers conceptually: lsblk -f # block/filesystem view; dm-crypt appears here as mapped block layer mount | less # filesystem mounted above it # fscrypt policy then applies to selected directories within that filesystem.

KERNEL DOCSLinux filesystem-level encryption (fscrypt)

Authoritative kernel documentation covering threat model, v2 policies, key hierarchy/KDFs, filename/content modes, key ioctls, inline encryption and access semantics.

https://docs.kernel.org/filesystems/fscrypt.html

OPEN-SOURCE TOOLfscrypt — high-level Linux native filesystem-encryption manager

Practical management tool for protectors, policy keys, PAM integration, setup, locking/unlocking and encrypted directories on filesystems that support the kernel API.

https://github.com/google/fscrypt

OPEN-SOURCE TOOLfscryptctl — low-level fscrypt policy/key tool

Small tool exposing the lower-level v2 policy/key operations directly; useful for understanding the kernel API while the full fscrypt tool remains the recommended general-purpose interface.

https://github.com/google/fscryptctl

Encryption hides bytes; verity proves which bytes you received: dm-verity and fs-verity use Merkle trees for read-time integrity

Confidentiality and integrity are separate properties. dm-crypt can make sectors unintelligible without a key, but plain encryption does not by itself prove that ciphertext blocks were not maliciously replaced. Linux verity mechanisms solve a different problem: hash data in a Merkle tree, trust a small root digest through some external chain of trust, and verify blocks or file pages as they are read.

DM-VERITY: WHOLE READ-ONLY BLOCK DEVICE trusted root hash ↓ Merkle-tree root ↓ verify hash block intermediate hash block(s) ↓ verify leaf digest data block read from backing device ↓ hash matches? ── yes → return data upward └─ no → corruption/error policy ↓ filesystem sees a read-only verified block device FS-VERITY: SELECTED FILE trusted/authenticated file digest ↓ per-file Merkle tree stored by filesystem ↓ page/file block read verify path to root digest ↓ valid bytes enter page cache / mmap consumer

Mechanism	Granularity	Typical use	Important limitation
dm-crypt	block device	confidentiality for writable or read-only storage	Encryption alone is not authenticated integrity.
dm-verity	read-only block device	verified root/system image	The root hash itself must be authenticated by something else.
fs-verity	individual read-only file	independently updated executables/assets on a writable filesystem	It protects enabled files, not arbitrary filesystem metadata.
Merkle tree	hierarchical hashes	verify only the path needed for blocks actually read	Integrity does not provide secrecy.

The root digest is the trust anchor, not magic metadata. Verity detects data that disagrees with the trusted digest; it does not tell you whether an attacker also replaced an unauthenticated root hash. Secure/Measured Boot, signed metadata, trusted userspace, or another authenticated channel must establish what digest is expected.

KERNEL DOCSLinux dm-verity — verified read-only block devices

Kernel documentation for the Device Mapper verity target, including data/hash devices, block sizes, digest/salt parameters, corruption handling and on-disk Merkle-tree layout.

https://docs.kernel.org/admin-guide/device-mapper/verity.html

KERNEL DOCSLinux fs-verity — per-file authenticity/integrity protection

Explains per-file Merkle trees, file digests, read-time verification, signatures and why fs-verity complements rather than replaces dm-verity.

https://docs.kernel.org/filesystems/fsverity.html

Read the real hardware documentation

Once gates and buses make some sense, original datasheets stop looking like hieroglyphics. They teach signal names, electrical limits, timing diagrams, truth tables and how chips are actually wired together.

DATASHEETTexas Instruments SN7400 — real NAND-gate datasheet

Official TI page for the classic 7400-family quad 2-input NAND gate, with PDF/HTML datasheet. Read the pinout, VIH/VIL voltage thresholds, output current limits and propagation-delay tables.

https://www.ti.com/product/SN7400

FREE MANUALMCS6500 Family Hardware Manual — clean remaster

Freely downloadable remaster of the original 1976 MOS Technology hardware manual. Useful when the old scan is hard to read; covers the processor family as a physical system rather than only as an instruction set.

https://pickledlightprojects.com/documents/mcs6500-hardware/

PLAIN HTMLMCS6500 Family Programming Manual — HTML transcription

Plain HTML transcription of the original manual. The opening explicitly connects internal microprocessor architecture, available components and how those components are interconnected.

https://lbaeza.neocities.org/mcs6500/6500_ch01

Worked example: trace the Apple-1 as a complete computer

The Apple-1 is ideal for this exercise because the processor is a separate 6502 chip, the RAM is separate, the I/O adapter is separate, and the video terminal is implemented in hardware rather than hidden inside a modern SoC.

POWER / REGULATION │ ├──────────────→ +5 V logic rail │ TIMING / TERMINAL LOGIC ─────→ ~1 MHz CPU clock │ ↓ MOS 6502 CPU ├── address bus ───────────→ RAM / PROM / PIA selection ├── data bus ↔──────────→ RAM / PROM / PIA data └── R/W + timing/control ──→ read/write coordination RAM ←→ CPU working memory PROM → 256-byte monitor firmware PIA ← keyboard ASCII input PIA → terminal/display character output terminal circuitry → video timing + character generation → display

What to trace in the schematic

Find the 6502. Identify A0–A15, D0–D7, R/W, clock, RESET, IRQ/NMI, VCC and GND.
Follow the address lines into the decoding logic. Ask which address ranges enable RAM, PROM and the PIA.
Follow the shared data bus. Notice that multiple chips connect to it, but only the selected device is allowed to drive it during a read.
Find the two 4 KB DRAM banks. The original machine's memory is dynamic, so refresh has to occur even while the CPU wants bus time.
Find the tiny 256-byte monitor PROM. Reset ultimately causes the 6502 to fetch a reset vector and begin executing firmware from ROM.
Find the PIA. Its keyboard-facing registers appear to software as memory-mapped addresses, so a CPU load can actually mean 'read a keyboard register'.
Trace the display path separately. The Apple-1 terminal hardware generates video independently of the CPU; the CPU mostly feeds it characters.
Finally follow the clock. The Apple-1's processor timing is derived from the terminal timing circuitry, which is a beautiful example of one clock source serving several subsystems.

PLAIN WEBApple-1 system overview and block diagram

Plain technical overview of the complete machine. It explicitly identifies the 6502, two 4 KB DRAM banks, 256-byte ROM, PIA, keyboard/terminal connection, refresh and the origin of the 1 MHz CPU clock.

https://www.sbprojects.net/projects/apple1/a1block.php

PLAIN WEBApple-1 terminal circuitry

Detailed explanation of the hardware terminal, including timing, counters, character generation and video-related blocks.

https://www.sbprojects.net/projects/apple1/terminal.php

SOURCE / NOTESWoz Monitor memory map and PIA register addresses

Shows the software side of memory-mapped hardware: keyboard at $D010/$D011 and display at $D012/$D013, tying assembly code directly to physical PIA registers.

https://www.sbprojects.net/projects/apple1/wozmon.php

Same computer concepts, different eras

Concept	Apple-1 / 1970s board	Modern SoC
Processor	Separate 6502 IC	One or more CPU cores on same silicon as much of the system
RAM	Separate DRAM chips on PCB	Often on-chip SRAM plus external DRAM/flash as needed
ROM / firmware	Tiny external PROM	On-chip boot ROM plus external flash/firmware storage
Address decoding	Discrete logic chips	Integrated interconnect/crossbar/NoC logic
I/O	Separate PIA and terminal logic	Integrated UART/SPI/I²C/USB/GPIO/etc.
Clock	Board-level timing logic	Oscillator/crystal reference plus PLLs/dividers and several clock domains
Bus	Visible parallel PCB traces	Mostly on-chip interconnect; high-speed serial links externally
Debugging	Probe individual pins/traces	Registers, debug ports, trace hardware, logic analyzers/oscilloscopes at external interfaces
Power	Few rails, simple linear regulation	Multiple voltage domains, regulators, sequencing, decoupling and power-management logic

The stored-program idea: why putting instructions in memory changed everything

Early electronic computers could perform general calculations without having a modern stored-program architecture. The crucial conceptual step was allowing instructions to live in the same kind of addressable memory as data. Then changing the program no longer required rewiring the machine's control structure; the CPU could fetch an instruction word, decode it, update state, advance/replace its instruction address and fetch the next one.

PRE-STORED-PROGRAM STYLE operator configures switches / plugboards / function tables ↓ machine's sequencing follows that physical configuration STORED-PROGRAM STYLE memory[address] contains instruction bits ↓ PC / instruction address selects memory word ↓ instruction register captures bits ↓ decoder/control executes operation ↓ PC changes → another memory word becomes next instruction Program and data can now be loaded, copied and changed as information.

UNIVERSITY ARCHIVEManchester Baby — University of Manchester technical history

Excellent primary-institution page. The Baby ran a stored program on 21 June 1948, used a 32×32-bit Williams-tube store, had a 32-bit word and just seven original instructions. The page also explains its accumulator, instruction state and input/output.

https://curation.cs.manchester.ac.uk/computer50/www.computer50.org/mark1/new.baby.html

EMULATORSManchester Baby / SSEM emulators — Computer Conservation Society

No-login emulator index for the Manchester Baby, Ferranti Mark I, EDSAC and other historic systems. Useful for actually running old instruction sets rather than only reading dates.

https://www.computerconservationsociety.org/emu/index.htm

HISTORICAL SIMULATOREDSAC Simulator — University of Warwick

Faithful software evocation of the 1949 EDSAC with original controls, displays, programs, subroutines and debugging software. Explicitly intended as a tutorial introduction to classic stored-program architecture.

https://www.dcs.warwick.ac.uk/~edsac/Home.html

DOWNLOADSEDSAC simulator downloads

Direct public download page for Linux, Raspberry Pi, Windows and macOS versions plus an open-source Qt implementation and tutorial guide.

https://www.dcs.warwick.ac.uk/~edsac/Software/Software1.html

A short implementation timeline: same concepts, radically different hardware

Machine / era	Switching technology	Memory / program storage	Why study it
Relay computers / 1930s–40s	Electromechanical relays	Relays, mechanical media, tape/cards depending on machine	Logic gates and state are physically visible.
ENIAC / 1946	Vacuum tubes + electronic counters/accumulators	Program configuration originally external to ordinary data storage	Shows electronic speed before convenient stored-program architecture.
Manchester Baby / 1948	Vacuum tubes	Williams-Kilburn CRT stores both program and data	Minimal stored-program CPU with only a handful of instructions.
EDSAC / 1949	Vacuum tubes	Mercury delay-line main memory + paper tape input	Classic serial stored-program machine and early software library.
Whirlwind / early 1950s	Vacuum tubes	CRT then magnetic-core main memory	Real-time computing, parallel architecture, interactive display and core-memory development.
IBM 1401 / 1959	Discrete/transistorized logic modules	Magnetic core; punched cards, tape and later disk I/O	Business data processing with visible electromechanical peripherals.
PDP-1 / 1959–61	Discrete transistor logic	Magnetic core	Interactive computing, paper tape, CRT graphics/light pen.
Intel 4004 / 1971	PMOS large-scale integration	External ROM/RAM chip family	CPU collapses into a single small IC but still needs a complete surrounding system.
Apple-1 / 1976	6502 NMOS CPU + TTL/MOS support ICs	DRAM + PROM/ROM	Whole personal computer remains traceable on a few schematics.
Modern SoC	CMOS VLSI	Caches, SRAM, external DDR, flash/SSD	Same state/control/interconnect concepts integrated at extreme density and speed.

8. Historical computers and original documentation

PUBLIC WEBApple I — original manual, schematics and ROM source downloads

Scans of the original Apple I manual including schematics, the Apple Cassette Interface manual, BASIC manual, video-chip material, and Woz Monitor / cassette-interface source.

https://www.sbprojects.net/projects/apple1/download.php

INTERACTIVEVisual6502

A transistor-level preservation and simulation project for the MOS 6502 and other chips. Lets you move below block diagrams and inspect an actual historical microprocessor die.

https://visual6502.org/

SOURCE / FILESVisual6502 — source code

The JavaScript transistor-level 6502 simulator and chip data. Valuable if you want to inspect how the simulator represents nodes, transistor connections, and die geometry.

https://github.com/trebonian/visual6502

PUBLIC WEBIntel 4004 — 50th Anniversary Project

A deep historical/technical archive around the first commercial microprocessor family: verified schematics, mask artwork, simulators, calculator firmware, replicas, and commentary.

https://www.4004.com/

PUBLIC WEBIntel 4004 — original schematics

Federico Faggin's original 4004 schematics with explanations of the memory block, control logic, arithmetic unit, internal bus, timing, MOS loads, and physical design constraints.

https://www.intel4004.com/4004_original_schematics.htm

ARCHIVEBitsavers — PDP-8 handbook archive

A whole archive of original DEC small-computer and PDP-8 handbooks. Excellent historical primary-source material.

https://bitsavers.trailing-edge.com/pdf/dec/pdp8/handbooks/

ARCHIVEBitsavers — MITS / Altair archive

Original documentation around the MITS Altair ecosystem. Useful for understanding early personal-computer buses, front panels, bootstrapping, boards, and software.

https://bitsavers.org/pdf/mits/

PUBLIC WEBVirtual Apollo Guidance Computer

A serious preservation/emulation project for the Apollo Guidance Computer, with documentation, emulators, software, hardware information, and original program material.

https://virtualagc.github.io/

SOURCE / FILESVirtual AGC — source repository

Actual source transcriptions, assembler/emulator tools, peripheral simulation, schematics branches, and Apollo mission software. A rare chance to study a complete historically significant computer stack.

https://github.com/virtualagc/virtualagc

DIRECT PDFIBM PC 5150 Technical Reference — complete hardware manual (PDF)

Original-style technical reference for the IBM PC architecture: 8088 system board, memory, BIOS, expansion bus, adapter interfaces, signal names and full schematics. A superb example of a whole computer documented down to connectors and timing.

https://www.minuszerodegrees.net/manuals/IBM_5150_Technical_Reference_6322507_APR84.pdf

DIRECT PDFApple II Reference Manual — 1978 (PDF)

Original Apple II reference material with hardware/firmware details and schematics. Read beside the Apple-1 manual to see how Wozniak moved from a bare board computer to a more complete expandable personal computer.

https://www.applelogic.org/files/AIIREF.pdf

MANUAL ARCHIVEAltair 8800 — original manual archive

Plain page of original Altair PDFs: operator manual, theory of operation, serial interface, floppy disk, cassette interface, interrupt/clock board and Intel 8080 material. No account.

https://altairclone.com/altair_manuals.html

DIRECT FILESAltair 8800 — theory, assembly and schematics file index

Direct directory listing containing the operator manual, theory of operation, assembly manual and full schematics. Excellent for the S-100 bus, front panel, CPU board and power supply.

https://deramp.com/downloads/altair/hardware/altair_8800_computer/

Historical architecture manuals worth reading as engineering documents

These are not nostalgia links. Older machines are pedagogically useful because the buses, registers, timing and control are often documented explicitly and are small enough to reason about.

PRIMARY ARCHIVEIntel MCS-4 / 4004 original document archive — Bitsavers

Directory of original Intel MCS-4 documents, including the 4004/4001/4002/4003 user manual, data sheet, assembly manual and hardware simulator documentation. The MCS-4 is especially good for studying a complete multi-chip computer set with a 4-bit shared data bus.

https://www.bitsavers.org/components/intel/MCS4/

DIRECT PDFPDP-11 Architecture Handbook — DEC (direct PDF)

Original DEC architecture handbook. The PDP-11 is historically important for its register model, orthogonal addressing modes and UNIBUS-style system organization. Read it as a contrast to single-chip microprocessors.

https://www.bitsavers.org/pdf/dec/pdp11/handbooks/EB-23657-18_PDP-11_Architecture_Handbook_1983.pdf

DIRECT PDFCRAY-1 Hardware Reference Manual — Cray Research (direct PDF)

Original 1977 hardware reference for a radically different machine: vector registers, functional units, pipelines, large register files and extremely aggressive physical packaging. Useful after a simple CPU because it shows how architecture changes when throughput dominates.

https://www.bitsavers.org/pdf/cray/CRAY-1/2240004C_CRAY-1_Hardware_Reference_Nov77.pdf

A useful comparison exercise

Machine / CPU	What to inspect	Why it is useful
Intel 4004 / MCS-4	4-bit bus, two-phase clock, ROM/RAM/I/O chip set	A whole microcomputer system at extremely small scale
MOS 6502 / Apple-1	CPU bus, RAM, ROM/monitor, PIA, terminal/video hardware	Early personal computer with understandable glue logic
Z80	Memory vs I/O cycles, refresh, interrupts, clocked bus timing	Shows a richer 8-bit microprocessor interface
Motorola 68000	16-bit data bus, asynchronous handshake, arbitration, exceptions	Bridge from simple micros to more capable processor buses
PDP-11	Registers, UNIBUS, addressing modes, minicomputer organization	Shows pre-microprocessor system architecture
CRAY-1	Vector registers, functional-unit pipelines, physical packaging	Shows how extreme performance changes the architecture

How people actually operated computers before screens, mice and USB

The human interface used to expose far more of the machine. Operators loaded paper tape or decks of punched cards, set console switches, watched indicator lamps, mounted magnetic tapes, and read printed output. Studying these systems makes I/O devices, buffers, device commands and machine state much less abstract.

PRIMARY PDFIBM 1401 Reference Manual — direct PDF

One of the best primary documents for understanding a complete early-1960s business computer. Covers processor instructions, core storage, console and devices including the 1402 card reader/punch and 1403 printer.

https://www.bitsavers.org/pdf/ibm/1401/A24-1403-5_1401_Reference_Apr62.pdf

RESTORATION ARCHIVEIBM 1401 restoration — Computer History Museum

Rich public restoration archive with IBM manuals, timing charts, programming material, theory-of-operation work and photos from a functioning transistorized 1401 system.

https://ibm1401.computerhistory.org/

PRIMARY ARCHIVEIBM 1401 document index — Bitsavers

Curated list of original 1401 system summaries, operator guides, card reader/punch, printer, disk/tape and programming manuals.

https://www.bitsavers.org/1401/1401-docs.html

RESTORATIONPDP-1 restoration — Computer History Museum

Shows a complete restoration including power supplies, CPU, paper-tape reader/punch, typewriter, CRT display and light pen. Excellent reminder that I/O used to be enormous electromechanical subsystems.

https://www.computerhistory.org/pdp-1/restoration/

MUSEUM EXPLAINERPDP-1 specifications and peripherals

Concrete specs for magnetic-core memory, 18-bit word, paper tape, typewriter, display and light pen, plus operating speed and physical size.

https://www.computerhistory.org/pdp-1/specifications/

MUSEUM EXPLAINERPDP-1 graphics and light pen

Explains the point-plotting CRT display and light pen. Useful historical ancestor to modern interactive graphics pipelines.

https://www.computerhistory.org/pdp-1/graphics/

No-login tools for actually watching a computer change state

Reading is necessary, but computer architecture becomes much easier when you can pause one cycle, inspect every register and wire, then advance exactly one event. These tools are free/open and can be used without joining a course.

Tool	Best use	What you can observe
Digital	Build logic from gates upward	Logic values, buses, clocks, FSMs, ROM/RAM, processors, truth tables, timing and high-Z states.
Logisim-evolution	Large visual digital circuits and CPUs	Gate/register/memory state, chronograms, buses, TTL components and hierarchical circuits.
Ripes	Understand CPU datapaths/pipelines/caches	PC, instructions in pipeline stages, register file, control/data paths, cache hits/misses, MMIO and CPI/IPC.
GTKWave	Inspect HDL simulation timing	Every traced signal against time: clocks, reset, state machines, bus handshakes, glitches, setup sequences.
Visual6502	Go below gates into a real historic CPU	Individual transistor/node state while the 6502 executes machine instructions.
Falstad	Electrical rather than ideal digital behavior	Analog voltages/currents, MOSFET conduction, capacitors, oscillators and dynamic transitions.

OPEN-SOURCE TOOLDigital — open-source logic designer/simulator

Free GPL digital-logic simulator. It is specifically designed for educational circuit work and supports gates, FSMs, RAM/ROM, processors, test cases, high-impedance states and Verilog generation. No account is needed to download/view the public repository.

https://github.com/hneemann/Digital

DOWNLOADSDigital — current releases

Direct release page. Current releases provide downloadable packages; no course account or enrollment is involved.

https://github.com/hneemann/Digital/releases

DOWNLOADSLogisim-evolution — official releases

Compiled downloads for Windows/macOS/Linux/JAR. This is the project release page, not a sign-up service.

https://github.com/logisim-evolution/logisim-evolution/releases

OPEN-SOURCE TOOLRipes — official RISC-V visual architecture simulator

Open-source graphical processor simulator. It visualizes single-cycle and pipelined RISC-V processors, registers, memory, MMIO and caches, and can run assembly or compiled C.

https://github.com/mortbopet/Ripes

DOWNLOADSRipes — releases

Prebuilt binaries for Linux, Windows and macOS. Ripes currently includes multiple processor models, pipeline stepping and cache simulation.

https://github.com/mortbopet/Ripes/releases

PUBLIC DOCSRipes documentation

Direct documentation index covering introduction/tutorial, cache simulation, C compilation, memory-mapped I/O and command-line operation.

https://github.com/mortbopet/Ripes/blob/master/docs/README.md

PUBLIC DOCSVerilator — waveform tracing

Public documentation showing how Verilator emits VCD/FST waveforms. Pair this with GTKWave after writing or modifying real Verilog/SystemVerilog.

https://verilator.org/guide/latest/faq.html

A no-account lab: inspect the computer you are already using

If you have Linux—or a Linux VM—you can inspect many of the abstractions in this page directly. The commands below are observational by default. Some verbose hardware details may require root privileges; do not use register-writing tools such as setpci unless you know exactly what you are changing.

# 1) CPU topology, caches, virtualization, NUMA lscpu lscpu -e # 2) PCI / PCIe devices and tree topology lspci lspci -t lspci -vv # 3) USB hierarchy lsusb lsusb -t # 4) Block devices / NVMe/SATA-visible storage hierarchy lsblk -o NAME,TYPE,SIZE,MODEL,TRAN,FSTYPE,MOUNTPOINTS # 5) Inspect your shell/program's ELF file readelf -h /bin/ls readelf -l /bin/ls readelf -S /bin/ls objdump -d /bin/ls | less # 6) Watch program → kernel system-call boundary strace -o /tmp/trace.txt -f /bin/echo hello less /tmp/trace.txt # 7) Count hardware/software events for a workload perf stat /bin/ls >/dev/null perf stat -e cycles,instructions,branches,branch-misses,cache-references,cache-misses sleep 1 # 8) See what events your CPU/kernel exposes perf list # 9) Pin a program to one logical CPU taskset -c 0 <command> # 10) On NUMA hardware, inspect topology/policy (if numactl installed) numactl --hardware numactl --show # Permissions and exact event support vary by distro/kernel/CPU.

Command	What concept from this page becomes visible
lscpu	Logical CPUs, cores, sockets, threads/core, cache sizes and NUMA nodes.
lspci -t	PCIe hierarchy: root ports, bridges and endpoints.
lspci -vv	BAR regions, link width/speed, MSI/MSI-X and PCIe capabilities where readable.
lsusb -t	USB host-controller/root-hub/device hierarchy, interface drivers and negotiated speeds.
lsblk	Kernel block-device graph and storage transport/model metadata.
readelf -l	Executable loadable segments that the OS loader maps into a process.
readelf -S	ELF sections such as .text/.data/.rodata and symbol/debug-oriented structure.
objdump -d	Actual ISA instruction bytes/disassembly in an executable.
strace	System call names, arguments, return values, signals and process interactions with the kernel.
perf stat	Retired instructions, cycles, IPC, branches, branch misses, cache events, page faults, context switches.
taskset	OS CPU affinity over logical CPUs.
numactl	NUMA nodes, distances and process memory-placement/CPU-binding policy.

Extra memory-observation commands

# Current shell's virtual mappings cat /proc/$$/maps # Per-mapping resident/private/shared accounting cat /proc/$$/smaps | less # System-wide RAM/cache/commit/swap accounting cat /proc/meminfo # Compact process map pmap -x $$ # Watch major/minor faults and RSS for a command /usr/bin/time -v <command> # perf software events include faults/context switches perf stat -e page-faults,minor-faults,major-faults,context-switches <command>

Extra timekeeping observations

# Which hardware counter is Linux using as its clocksource? cat /sys/devices/system/clocksource/clocksource0/current_clocksource cat /sys/devices/system/clocksource/clocksource0/available_clocksource # Battery/platform RTC state cat /proc/driver/rtc # Timer-related interrupts grep -Ei 'timer|hpet|rtc|lapic' /proc/interrupts # x86 TSC-related CPU flags (if x86) grep -m1 -oE 'constant_tsc|nonstop_tsc' /proc/cpuinfo # Compare wall and monotonic time APIs with a tiny clock_gettime program # or the clock_gettime(2) example linked in the timekeeping section.

Extra PCIe enumeration observations

# Identify a PCI function lspci -nn # Draw bridge/endpoint topology lspci -t # Inspect one function in detail lspci -s 03:00.0 -vv # Read kernel-assigned BAR resource ranges cat /sys/bus/pci/devices/0000:03:00.0/resource # Read-only config-space dump when permissions allow lspci -s 03:00.0 -xxxx # Do not modify BARs/config registers on a live system merely to experiment.

Extra filesystem/storage observations

# Identify filesystem and mount options findmnt -T . -o TARGET,SOURCE,FSTYPE,OPTIONS # ext4 superblock/journal features on an unmounted or appropriate filesystem sudo tune2fs -l /dev/<device> | less # Watch actual write/fsync/rename calls made by a program strace -f -e write,fsync,fdatasync,rename,renameat,renameat2 <command> # Block devices and transport lsblk -o NAME,TYPE,SIZE,FSTYPE,MODEL,TRAN,MOUNTPOINTS # Use read-only observation first. Do not run filesystem repair/mkfs tools on live data merely as an experiment.

Extra branch/cache measurements

# High-level branch/cache event counts (event support varies) perf stat -e cycles,instructions,branches,branch-misses,cache-references,cache-misses <command> # List model-specific branch / cache / TLB events perf list | less # Compare predictable vs unpredictable branches in your own benchmark # while keeping compiler optimization and input size controlled. # Record samples around expensive misses or branch events when supported perf record -e branch-misses <command> perf report # PMU event semantics differ by CPU model; read the actual event description.

Extra USB and packetized-I/O observations

# USB topology and descriptors lsusb lsusb -t lsusb -v # On newer usbutils: lsusb.py -i -e # Kernel descriptor/topology view (debugfs/root may be needed) cat /sys/kernel/debug/usb/devices # USB URB trace facility (raw capture can expose sensitive input!) sudo modprobe usbmon ls /sys/kernel/debug/usb/usbmon # PCIe capability/flow details visible at config level lspci -vv # Network neighbor and routing tables ip route ip neigh # Observe Ethernet/IP/TCP headers on a test interface if tcpdump is installed sudo tcpdump -ni <iface> -e -vv

Extra firmware / Secure Boot / TPM observations

# Is this Linux system booted through EFI? test -d /sys/firmware/efi && echo EFI || echo no-EFI-directory # Inspect Secure Boot-related EFI variables (read-only) ls /sys/firmware/efi/efivars 2>/dev/null | grep -E 'SecureBoot|PK-|KEK-|db-|dbx-' # TPM devices exposed by kernel ls -l /dev/tpm* /dev/tpmrm* 2>/dev/null ls /sys/class/tpm 2>/dev/null # If tpm2-tools is installed tpm2_pcrread # Firmware messages often reveal SPI/EFI/TPM initialization dmesg | grep -Ei 'efi|secure boot|tpm|spi|flash' | less # Observation only: changing EFI Secure Boot keys or TPM state can make a machine unbootable # or invalidate disk-encryption/attestation policies.

Extra synchronization and scheduling observations

# Trace userspace locks that fall into futex slow paths strace -f -e futex ./multithreaded_program # Count context switches and CPU migrations perf stat -e context-switches,cpu-migrations,task-clock ./multithreaded_program # Show thread → CPU placement ps -eLo pid,tid,psr,cls,pri,ni,stat,comm | less # Current process CPU affinity taskset -pc $$ # Scheduler tracepoints if enabled/allowed perf list | grep -E 'sched:sched_(switch|wakeup)' # Compare an uncontended lock benchmark to a contended one; # the latter should make blocking/wakeup/context-switch behavior much easier to see.

Extra USB/storage-controller observations

# Host controllers and storage controllers lspci -nnk | grep -A3 -Ei 'USB controller|SATA|AHCI|Non-Volatile|NVMe' # USB topology above xHCI lsusb -t # Block transport/type lsblk -o NAME,MODEL,TRAN,TYPE,SIZE,ROTA # Linux multi-queue objects for d in /sys/block/*/mq; do echo ===$d===; ls "$d" 2>/dev/null; done # Kernel driver binding for an identified controller readlink /sys/bus/pci/devices/0000:BB:DD.F/driver 2>/dev/null # Do not issue raw destructive ATA/NVMe commands to a device containing useful data.

Extra cache/RAS/thermal/allocator observations

# Cache and TLB geometry exposed by Linux lscpu -C 2>/dev/null || lscpu getconf LEVEL1_DCACHE_SIZE 2>/dev/null getconf LEVEL1_DCACHE_ASSOC 2>/dev/null getconf LEVEL1_DCACHE_LINESIZE 2>/dev/null getconf PAGESIZE # Hardware-error / EDAC reporting when supported find /sys/devices/system/edac -maxdepth 4 -type f 2>/dev/null | head -80 dmesg | grep -Ei 'EDAC|MCE|machine check|hardware error' | tail -50 # Thermal sensors for z in /sys/class/thermal/thermal_zone*; do printf '%s ' "$(cat "$z/type" 2>/dev/null)" cat "$z/temp" 2>/dev/null done # Observe malloc's kernel-facing VM activity strace -e brk,mmap,munmap,madvise ./allocation_test # Never deliberately inject ECC/MCE faults or disable thermal protection on a useful system.

Extra linker, boot and signal observations

# Dynamic dependencies, interpreter and relocations readelf -l /bin/ls | grep -A2 INTERP readelf -d /bin/ls | grep NEEDED readelf -rW /bin/ls | less objdump -d /bin/ls | less # Dynamic-loader diagnostics LD_DEBUG=libs,reloc /bin/true 2>&1 | less # Boot handoff and PID 1 cat /proc/cmdline ps -p 1 -o pid,ppid,comm,args dmesg | head -100 # Signal state for current shell grep -E 'Sig(Q|Pnd|Blk|Ign|Cgt):' /proc/$$/status # Trace signal syscalls/delivery in a test program strace -e trace=signal ./program

Extra file-read, IRQ and PCIe-error observations

# File mapping / page-fault behavior perf stat -e page-faults,minor-faults,major-faults ./reader strace -e read,pread64,mmap,madvise ./reader # Interrupt sources and rates cat /proc/interrupts | less # PCIe Advanced Error Reporting capability and logs lspci -vv | grep -A18 -i 'Advanced Error Reporting' | less dmesg | grep -Ei 'AER|PCIe Bus Error|Uncorrected|Corrected' | tail -100 # DRAM refresh timing is normally hidden behind the integrated memory controller; # inspect firmware/vendor controller docs rather than changing refresh knobs on a useful machine. # Avoid AER/MCE/Rowhammer fault injection on production or valuable systems.

Extra device-virtualization, fd and io_uring observations

# IOMMU/VFIO/SR-IOV visibility find /sys/kernel/iommu_groups -maxdepth 2 -type l 2>/dev/null | sort | head -100 find /sys/bus/pci/devices -name sriov_totalvfs -print -exec cat {} \; 2>/dev/null # Current process file descriptors/open-file state ls -l /proc/$$/fd for f in /proc/$$/fdinfo/*; do echo ===$f===; head -20 "$f"; done # io_uring syscall visibility in a program strace -e io_uring_setup,io_uring_enter,io_uring_register,mmap ./uring_program # Device shared-virtual-memory features are hardware/driver-specific; # inspect lspci -vv and driver docs rather than assuming PASID/ATS/PRI are enabled. # Keep SR-IOV/VFIO experiments read-only on a machine whose networking/display/storage matters.

Extra translation, epoll and Ethernet-PHY observations

# TLB/cache events available on this CPU perf list | grep -Ei 'tlb|page.walk|dtlb|itlb' | less # Page/TLB-related workload counters where your PMU supports them perf stat -e page-faults,minor-faults,major-faults <command> # epoll syscalls in an event-driven server strace -f -e epoll_create1,epoll_ctl,epoll_wait,epoll_pwait2,read,write <server> # Current Ethernet negotiation and PHY-facing state ethtool eth0 ethtool -i eth0 # Don't assume a PMU event name exists on every CPU, or that every NIC exposes PHY/cable/FEC stats.

Extra threads, RCU and MSI-X observations

# Threads and TIDs ps -T -p <PID> -o pid,tid,psr,stat,comm ls /proc/<PID>/task # Thread creation/join slow paths strace -f -e clone,clone3,futex ./thread_program # MSI/MSI-X capability + current IRQ vectors lspci -vv -s <BDF> | grep -A12 -Ei 'MSI-X|MSI:' ls /sys/bus/pci/devices/0000:BB:DD.F/msi_irqs 2>/dev/null cat /proc/interrupts | less # RCU state/debug files depend on kernel config/debugfs dmesg | grep -i 'rcu.*stall' | tail -50 # Don't alter a thread's FS base or device MSI-X table manually on a normal live system.

Extra IRQ, timer, DVFS and CPU-idle observations

# hardirq / softirq activity cat /proc/interrupts | less cat /proc/softirqs | less # clocksources cat /sys/devices/system/clocksource/clocksource0/current_clocksource # CPUFreq for p in /sys/devices/system/cpu/cpufreq/policy*; do echo ===$p=== cat "$p"/scaling_driver "$p"/scaling_governor "$p"/scaling_cur_freq 2>/dev/null done # CPUIdle cat /sys/devices/system/cpu/cpuidle/current_governor_ro 2>/dev/null for s in /sys/devices/system/cpu/cpu0/cpuidle/state*; do printf '%s ' "$(cat "$s/name" 2>/dev/null)" cat "$s/usage" "$s/time" 2>/dev/null | tr '\n' ' '; echo done # Read-only observation is sufficient; forcing governors/idle states can alter # power, latency and thermals across the whole machine.

Extra kernel-wait, allocator and startup observations

# sleeping tasks + wait channels ps -eLo pid,tid,stat,wchan:32,comm | less # slab object caches head -40 /proc/slabinfo grep -E 'Slab|SReclaimable|SUnreclaim|Vmalloc' /proc/meminfo # kernel virtual mappings head -80 /proc/vmallocinfo 2>/dev/null # startup entry and constructors of a dynamically linked executable readelf -h /bin/ls | grep 'Entry point' readelf -SW /bin/ls | grep -E 'init|fini|interp|dynamic' objdump -d /bin/ls | grep -A20 '<_start>' # Keep kernel allocator fault-injection and raw MMIO experiments off useful systems.

Extra page allocator, mmap, networking and PCID observations

Extra locking, DMA and MMIO observations

# IOMMU/SWIOTLB boot state dmesg | grep -Ei 'IOMMU|DMAR|AMD-Vi|swiotlb|bounce' | less # Device BARs / resources / current driver lspci -vv -s <BDF> | less cat /sys/bus/pci/devices/0000:BB:DD.F/resource 2>/dev/null # Lockdep warnings, when CONFIG_PROVE_LOCKING/lockdep are enabled dmesg | grep -Ei 'lockdep|possible circular locking dependency|BUG: sleeping function' | tail -100 # Kernel config availability differs by distro grep -E 'CONFIG_(LOCKDEP|PROVE_LOCKING|PREEMPT_RT|SWIOTLB)=' /boot/config-$(uname -r) 2>/dev/null # Keep MMIO writes and DMA/IOMMU configuration read-only unless you are developing # a driver on disposable/recoverable hardware.

Extra driver, module, udev and syscall observations

# Driver/device binding lspci -nnk | less find /sys/bus/pci/devices -maxdepth 2 -name driver -type l -ls 2>/dev/null | head -60 # Module aliases / dependencies / loaded modules modinfo <module> 2>/dev/null | less modprobe --show-depends <module> 2>/dev/null cat /proc/modules | less # Live device events udevadm monitor --kernel --udev --property # System call activity strace -f -c <program> perf trace <program> 2>/dev/null # Avoid sysfs bind/unbind/remove/rescan and module insertion/removal on # devices that provide your active storage/network/display.

Extra GPU, block-queue, huge-page and pipe observations

# DRM/GPU scheduler trace events, if exposed perf list | grep -Ei 'drm_sched_job_(queue|run|done|add_dep)' | less # blk-mq cat /sys/block/nvme0n1/queue/scheduler 2>/dev/null find /sys/block/nvme0n1/mq -maxdepth 2 2>/dev/null | head -80 # THP / HugeTLB cat /sys/kernel/mm/transparent_hugepage/enabled 2>/dev/null grep -E '^Huge|Hugetlb' /proc/meminfo # pipe limits grep . /proc/sys/fs/pipe-* 2>/dev/null # These are intentionally read-only; GPU resets, storage queue tuning and # global THP policy changes can affect the whole machine.

Extra page-fault, usercopy, I-cache and network-TX observations

# Page faults perf stat -e page-faults,minor-faults,major-faults <program> # VMAs cat /proc/<PID>/maps # Network TX queuing/offload state tc -s qdisc show dev eth0 ethtool -k eth0 | grep -Ei 'tx|tso|gso|checksum' # Source inspection for fault-aware user copies # include/linux/uaccess.h # arch/x86/include/asm/uaccess_64.h # JIT/cache-coherence behavior is ISA-specific. Prefer compiler/runtime helpers # such as __builtin___clear_cache over hand-coding cache maintenance unless # you are deliberately studying one ISA's exact rules.

Extra credentials, namespace, cgroup and seccomp observations

# credentials/capabilities grep -E '^(Uid|Gid|Groups|CapInh|CapPrm|CapEff|CapBnd|CapAmb|NoNewPrivs|Seccomp):' /proc/$$/status # namespaces lsns ls -l /proc/$$/ns # cgroup membership + available controllers cat /proc/self/cgroup cat /sys/fs/cgroup/cgroup.controllers 2>/dev/null # process seccomp state grep -E '^(NoNewPrivs|Seccomp|Seccomp_filters):' /proc/$$/status # File capabilities getcap -r /usr/bin /usr/sbin 2>/dev/null | head -80 # These interfaces can change privilege/resource limits. Keep this top-level lab # read-only; use an unprivileged disposable namespace/test process for experiments.

Extra hardware and policy protection observations

# CPU protection features grep -m1 '^flags' /proc/cpuinfo | tr ' ' '\n' | grep -E '^(nx|smep|smap|pku|ospke|shstk|ibt)$' # Enabled Linux security modules / current context cat /sys/kernel/security/lsm 2>/dev/null cat /proc/self/attr/current 2>/dev/null # ASLR state cat /proc/sys/kernel/randomize_va_space # CET properties on an executable readelf -n /bin/ls 2>/dev/null | grep -A4 -Ei 'SHSTK|IBT|x86 feature' # Keep this read-only on a normal machine: disabling ASLR/LSMs or changing # security policy affects system-wide protection assumptions.

Extra device-PM, NUMA, fd-event and BPF observations

# Runtime PM (replace BDF) grep . /sys/bus/pci/devices/0000:BB:DD.F/power/{control,runtime_status,autosuspend_delay_ms} 2>/dev/null # NUMA numactl --hardware 2>/dev/null head -60 /proc/$$/numa_maps # eventfd/signalfd/timerfd objects show type-specific fdinfo fields ls -l /proc/$$/fd # BPF/JIT cat /proc/sys/net/core/bpf_jit_enable 2>/dev/null bpftool prog show 2>/dev/null | head -40 # Keep this read-only on an important machine: power policy, NUMA pinning and # BPF attachment can materially change latency/performance/security behavior.

PUBLIC MANUALlscpu(1) — current manual

Current util-linux manual: lscpu gathers CPU architecture, threads, cores, sockets and NUMA-node information from sysfs/proc and architecture-specific sources.

https://man7.org/linux/man-pages/man1/lscpu.1.html

PUBLIC MANUALGNU readelf

Current GNU Binutils documentation. readelf exposes ELF headers, program headers/segments, sections, symbols, relocations, dynamic data and notes.

https://www.sourceware.org/binutils/docs/binutils/readelf.html

PUBLIC MANUALGNU objdump

Current GNU Binutils documentation. objdump can disassemble machine code and intermix source where debugging information is available.

https://www.sourceware.org/binutils/docs/binutils/objdump.html

PUBLIC MANUALstrace(1) — current manual

Current public manual for tracing system calls/signals and their arguments/return values. Excellent for discovering how mundane programs actually use the kernel.

https://man7.org/linux/man-pages/man1/strace.1.html

PUBLIC MANUALperf stat(1) — current manual

Current manual with examples reporting cycles, instructions, instructions-per-cycle, branches, branch misses, task time, page faults and context switches.

https://man7.org/linux/man-pages/man1/perf-stat.1.html

PUBLIC MANUALtaskset(1) — CPU affinity

Current util-linux manual for launching/pinning processes on specified logical CPUs.

https://man7.org/linux/man-pages/man1/taskset.1.html

PUBLIC MANUALnumactl(8) — NUMA policy

Public manual for inspecting NUMA hardware and running processes with local, bind, preferred or interleave placement policies.

https://man7.org/linux/man-pages/man8/numactl.8.html

Performance counters: ask the CPU what actually happened

Modern processors contain Performance Monitoring Units (PMUs): programmable hardware counters that count events such as cycles, instructions retired, branches, mispredictions, cache events, TLB events and implementation-specific pipeline behavior. These are measurements—not guesses from wall-clock time—but interpreting them still requires care.

Example: $ perf stat -e cycles,instructions,branches,branch-misses ./program 4,000,000,000 cycles 5,200,000,000 instructions 900,000,000 branches 27,000,000 branch-misses IPC = instructions / cycles = 1.30 branch-miss rate = 27M / 900M = 3% A low IPC is a symptom, not a diagnosis. Possible causes: cache/TLB misses, branch recovery, serial dependencies, front-end starvation, execution-port contention, memory bandwidth, synchronization, etc.

Counter/event	Question it helps answer
cycles	How many counted CPU/core clock cycles elapsed for this measurement context?
instructions	How many architectural instructions retired?
IPC/CPI	How effectively did the workload convert cycles into retired instructions?
branches / branch-misses	How much control flow occurred and how often prediction failed?
cache references / misses	Is cache behavior plausibly contributing to stalls? Exact semantics can be CPU/event-specific.
page faults	How often did software-visible virtual-memory faults occur? This is not the same as a TLB miss.
context switches	How often did the OS switch running tasks during the measurement?
CPU migrations	Did the scheduler move the task between logical CPUs?
raw/model-specific PMU events	Can expose front-end, execution-port, load/store, cache, uncore and other microarchitectural details.

Caution: PMU events are not perfectly portable. Generic names can map differently across processor families, some events count speculative activity while others count retirement, counters can multiplex when too many events are requested, and security settings may restrict access. Always read the event definition for the actual CPU.

KERNEL DOCSLinux perf security and data exposure

Explains what PMU/perf data can expose, why distributions restrict performance-counter access, and how perf_event permissions are controlled.

https://docs.kernel.org/admin-guide/perf-security.html

DIRECT PDFIntel Optimization Reference Manual (direct PDF)

Use performance counters beside this manual's front-end/cache/branch discussion to turn microarchitecture concepts into measured hypotheses.

https://cdrdv2-public.intel.com/821612/248966-Optimization-Reference-Manual-V1-050.pdf

Linux observability has several layers: tracepoints → tracefs/ftrace buffers → perf event file descriptors

Performance counters answer “how many?” while tracing can answer “when, where and in what sequence?” Linux exposes static tracepoints at selected kernel events, ftrace/tracefs machinery for enabling and recording trace data, and the perf_event_open() ABI for counters or sampled events delivered through file descriptors and memory-mapped ring buffers. These facilities overlap, but they are not the same mechanism.

STATIC KERNEL EVENT scheduler / IRQ / syscall / block / networking code ↓ executes TRACEPOINT normally-off fast check ↓ when enabled tracepoint payload fields captured ↓ trace event / ftrace ring buffer ↓ tracefs files under /sys/kernel/tracing ↓ userspace reads formatted/raw trace data FUNCTION TRACING compiler/kernel instrumentation hook at function entry/exit ↓ ftrace function / function_graph tracer ↓ per-CPU trace buffers PERF EVENT ABI perf_event_open(attr, pid, cpu, group_fd, flags) ↓ returns fd counting mode: read() accumulated counts sampled mode: PMU/software/tracepoint event overflows ↓ mmap perf ring buffer ↓ userspace profiler consumes samples

Layer	Best mental model
tracepoint	A statically defined hook and structured event site in kernel code. Disabled tracepoints are designed to have very small overhead.
trace event	Infrastructure that records enabled tracepoint/event data into tracing buffers and exposes controls through tracefs.
ftrace function tracer	Function-level instrumentation used to record call activity; function_graph can show call/return nesting and durations.
PMU event	Hardware counter/sample source such as cycles, instructions or cache/branch events.
perf_event_open()	Kernel ABI that turns one counter/sampling source into a file descriptor; sampled records can be consumed from an mmap ring buffer.

Tracing changes the thing being measured. Buffer writes, stack collection, high event rates and function tracing all add overhead. Start with the narrowest event set that can answer the question, and distinguish timestamps/event ordering from hardware counter totals.

TRACEFS / PERF LAB # Discover trace events (requires tracefs mounted and permissions): cat /sys/kernel/tracing/available_events | less # Example event family: grep '^sched:' /sys/kernel/tracing/available_events | head # perf can use PMU counters and many software/tracepoint sources: perf stat -e cycles,instructions ./program perf list | less # On a test system, use trace-cmd/perf/ftrace controls carefully; # tracing high-frequency events system-wide can produce large overhead/data.

KERNEL DOCSLinux kernel tracepoints — runtime hooks and probe semantics

Explains what a tracepoint is, how disabled/enabled hooks behave and how probes attach to statically defined kernel instrumentation sites.

https://docs.kernel.org/trace/tracepoints.html

KERNEL DOCSftrace — Linux function tracer and tracefs controls

Comprehensive current documentation for tracers, per-CPU buffers, tracefs control files, function/function-graph tracing and event interaction.

https://docs.kernel.org/trace/ftrace.html

PUBLIC MANUALperf_event_open(2) — counting and sampled event ABI

Current userspace ABI reference for creating performance-event file descriptors, event grouping, counting versus sampling, permissions and mmap ring-buffer delivery.

https://man7.org/linux/man-pages/man2/perf_event_open.2.html

A build ladder: 18 projects from one transistor to a small computer

Do these in a simulator first. Repeat selected stages on a breadboard or in Verilog later. The point is not completing worksheets; each project introduces one new abstraction while preserving everything underneath it.

1. NMOS switch

Drive an NMOS gate and observe drain current/output voltage with a resistor or load. Understand cutoff versus conduction.

2. CMOS inverter

One PMOS pull-up + one NMOS pull-down. Sweep the input and observe logic inversion, transition region and load-capacitance delay.

3. NAND and NOR

Build CMOS NAND/NOR transistor networks. Derive AND/OR using inversion and DeMorgan's laws.

4. XOR + half-adder

Produce SUM=A XOR B and CARRY=A AND B. Verify all four input combinations.

5. Full adder

Add A, B and carry-in. Chain four of them into a 4-bit ripple-carry adder and watch carry propagation.

6. Multiplexer

Build a 2:1 mux, then 4:1. Use muxes to choose ALU operands/results.

7. SR latch

Make feedback store one bit. Understand why state is possible at all.

8. D latch / D flip-flop

Turn raw feedback into controlled, clock-related state storage. Inspect setup/hold assumptions.

9. 8-bit register

Eight flip-flops plus load-enable/clear. Put LEDs/probes on every bit.

10. Counter / program counter

Build increment/load/reset behavior and watch binary state advance each clock.

11. ALU

Select ADD/SUB/AND/OR/XOR/shift operations and generate zero/carry/overflow-style flags.

12. Shared bus

Connect several registers/ALU through muxes or tri-state drivers. Deliberately create bus contention once so you understand why enables matter.

13. RAM + address decoder

Map RAM and a tiny ROM into address ranges. Use high address bits to generate chip-selects.

14. Instruction register + decoder

Define a tiny instruction word and decode opcode/register/immediate fields.

15. Control sequencer

Implement fetch/decode/execute as an FSM or microcode ROM. Single-step every micro-operation.

16. Complete tiny CPU

Run load/store/add/jump instructions from ROM/RAM. Inspect PC, IR, registers, ALU, bus and control signals every cycle.

17. Memory-mapped output

Map an LED/display/output register to an address. A CPU STORE should now cause a physical/simulated output change.

18. Interrupt-capable system

Add a timer or input device that raises IRQ. Save/redirect/handle/return so external hardware can divert instruction flow.

Useful discipline: before moving to the next project, be able to point at every input/output/control signal and say what causes it to change, what voltage/logic state means, whether the path is combinational or stored, and on which clock/handshake event state updates.

9. Projects that make the connections concrete

PUBLIC WEBMagic-1 HomebrewCPU

A complete home-built computer whose custom CPU uses roughly 200 74-series TTL chips rather than an off-the-shelf processor. It runs Minix and exposes the machine at a comprehensible physical scale.

https://www.magic-1.org/

PUBLIC WEBMegaprocessor

A room-scale processor built from thousands of discrete transistors and LEDs. The point is visibility: you can literally see logic state and data movement that would normally be microscopic.

https://www.megaprocessor.com/

SOURCE / FILESFrom the Transistor to the Web Browser

An ambitious project outline that connects Verilog hardware, a CPU, assembler, compiler, MMU, OS, networking, and a browser. Not a polished textbook, but excellent for understanding what a truly vertical computer-stack project entails.

https://github.com/geohot/fromthetransistor

PUBLIC WEBProject F — FPGA & RISC-V tutorials

Practical Verilog/FPGA tutorials including digital design, graphics, arithmetic, and RISC-V. Good after logic simulators when you want circuits to become synthesizable hardware.

https://projectf.io/tutorials/

PUBLIC WEBProject F — Verilog library

Documented real modules for clocks, clock-domain crossing, RAM/ROM, UART, display timing, graphics, and arithmetic, with testbenches.

https://projectf.io/verilog-lib/

PUBLIC WEBYosys documentation

Open-source RTL synthesis documentation. Use this after learning Verilog to understand how HDL becomes a netlist of logic elements rather than imagining Verilog as ordinary sequential software.

https://yosyshq.readthedocs.io/projects/yosys/en/latest/

Verilog is not software execution: HDL → logic → placed/routed hardware

A synthesizable HDL description describes hardware structure/behavior. The synthesis tool elaborates that description, recognizes registers, memories and arithmetic, optimizes the Boolean network, and maps it to the resources of a target technology. For an FPGA, those resources include LUTs, flip-flops, block RAM, carry chains and DSPs. For an ASIC, synthesis maps into a standard-cell library; physical-design tools then place cells, build the clock tree and route wires before signoff/manufacturing.

SystemVerilog / Verilog RTL ↓ parse + elaborate processes / expressions / memories / FSMs ↓ synthesis + optimization generic muxes / gates / FFs / arithmetic cells ↓ technology mapping FPGA: LUTs + FFs + BRAM + DSP + carry chains ASIC: library NAND/NOR/AOI/OAI/FF/buffer/etc. standard cells ↓ placement ↓ clock-tree synthesis ↓ routing + parasitic extraction + timing checks ↓ FPGA bitstream OR ASIC physical layout / masks

Critical distinction: two adjacent lines in an always_comb block do not necessarily become two operations that happen one after another in time. They can become one combinational circuit whose gates all exist at once. A clocked always_ff block describes state updates at clock events.

PUBLIC DOCSYosys — synthesis primer

Current public Yosys documentation. Its synthesis primer starts from behavioral Verilog and follows conversion into RTL, logical gates or physical target gates.

https://yosyshq.readthedocs.io/

PUBLIC DOCSYosys — guided synthesis walkthrough

Detailed public walkthrough of loading Verilog, AST/elaboration, process conversion, memory/FSM handling, optimization and mapping to an iCE40 FPGA.

https://yosyshq.readthedocs.io/projects/yosys/en/latest/getting_started/example_synth.html

PUBLIC DOCSYosys — gate-level technology mapping

Explains mapping generic flip-flops and combinational logic into the actual cells described by a target Liberty library.

https://yosyshq.readthedocs.io/projects/yosys/en/v0.55/using_yosys/synthesis/techmap_synth.html

PUBLIC DOCSOpenROAD — clock-tree synthesis

Makes clock distribution concrete: buffers and routing are deliberately constructed while controlling slew, capacitance and skew.

https://openroad.readthedocs.io/en/latest/main/src/cts/README.html

SOURCE / FILESPicoRV32 — small open RISC-V CPU in Verilog

A complete small open hardware CPU core with one main Verilog file, memory interfaces, interrupt option, synthesis scripts and example SoC. GitHub viewing does not require an account.

https://github.com/YosysHQ/picorv32

SOURCE FILEPicoRV32 — the actual processor Verilog

Roughly three thousand lines of real synthesizable processor RTL. Useful after toy CPUs because it shows what a compact production-oriented RV32 core looks like in source.

https://github.com/YosysHQ/picorv32/blob/main/picorv32.v

Read a real open CPU core: Ibex is small enough to trace and real enough to matter

Toy CPUs are ideal for first principles, but eventually you should read a core that has actually been synthesized, verified and taped out. Ibex is a production-quality open-source 32-bit RISC-V core written in SystemVerilog. It is small enough to understand without immediately drowning in the complexity of a desktop out-of-order processor.

instruction memory / bus ↓ instr_* interface instruction fetch + prefetch / optional I-cache ↓ IF pipeline stage ↓ ID/EX stage ├── decoder ├── register file ├── controller ├── ALU ├── branch logic └── multiplier/divider (configuration dependent) ↓ optional WB stage data_* load/store interface ↔ memory / MMIO irq_* → exceptions / interrupts debug_* → external debug support clock/reset/fetch-enable → execution state control

PUBLIC DOCSIbex documentation — production-quality open 32-bit RISC-V core

Public docs explicitly describe Ibex as production-quality, heavily parameterizable SystemVerilog with multiple tape-outs. No account required.

https://ibex-core.readthedocs.io/en/latest/

PUBLIC DOCSIbex pipeline details

Concrete two-stage pipeline documentation: IF fetches via a prefetch buffer, ID/EX decodes/executes/register-reads/writes, and multicycle instructions stall the execution stage.

https://ibex-core.readthedocs.io/en/latest/03_reference/pipeline_details.html

PUBLIC DOCSIbex core integration and external signals

Especially useful for your original 'how each component is connected' question: documents instruction-fetch, load/store, interrupt, debug, fetch-enable, sleep and configuration interfaces.

https://ibex-core.readthedocs.io/en/latest/02_user/integration.html

PUBLIC DOCSIbex control/status and hardware debug registers

Real CSR documentation including hardware trigger registers, debug control/status, debug PC and single-step state.

https://ibex-core.readthedocs.io/en/latest/03_reference/cs_registers.html

SOURCE / FILESIbex source repository

Actual Apache-licensed SystemVerilog source: ALU, decoder, controller, IF/ID stages, load-store unit, multiplier/divider, register files and cache. Reading source requires no GitHub account.

https://github.com/lowRISC/ibex

Read a real open SoC: CPU, memories, peripherals, interrupts, buses and pins in one design

Once a standalone CPU core makes sense, the next useful object is a complete System-on-Chip. OpenTitan exposes the integration level that commercial SoCs normally hide: CPU, ROM/SRAM/nonvolatile memory, UART/I²C/SPI/GPIO, timers, interrupt controller, debug module, crossbars, clock/reset/power domains, memory map and chip pins.

chip pins / pads ↕ pinmux + peripheral I/O Ibex RISC-V CPU ├── instruction/data requests ├── interrupt input ← PLIC ← peripheral interrupt sources └── debug ← RISC-V Debug Module ← JTAG pins ↓ high-speed TileLink-UL crossbar ├── ROM / SRAM / nonvolatile memory ├── cryptographic / security blocks └── bridge → lower-speed peripheral crossbar ├── UART ├── GPIO ├── SPI ├── I²C ├── timers └── other MMIO peripherals software sees those peripherals at fixed memory-mapped addresses

Concept from simpler computers	OpenTitan manifestation
Address decoder	Generated TileLink crossbars route transactions according to the SoC address map.
Peripheral register	MMIO CSR block at a documented base address, e.g. UART/GPIO/I²C.
IRQ wire	Many peripheral sources feed a PLIC that prioritizes/routes interrupts to the RISC-V core.
ROM boot code	Boot ROM is a mapped on-chip memory and begins the secure boot chain.
Debug header	JTAG pins feed a RISC-V debug module capable of halting/injecting/accessing the core/system.
Clock/reset wires	Clock and reset managers distribute controlled domains rather than one global ideal signal.
Glue logic	Generated top-level RTL and standardized inter-module interfaces replace much discrete PCB glue logic.

PUBLIC DOCSOpenTitan Earl Grey design documentation

Extremely valuable complete-SoC documentation: JTAG/debug, PLIC interrupts, SRAM/NVM, secure boot, peripherals, TileLink-UL bus network, memory map and chip I/O.

https://opentitan.org/book/hw/top_earlgrey/doc/design/index.html

PUBLIC DOCSOpenTitan Earl Grey memory map

A real memory map with actual UART, GPIO, SPI, I²C, timer and other peripheral base addresses. This is what 'memory-mapped I/O' looks like in a production-class open chip.

https://opentitan.org/book/hw/top_earlgrey/doc/memory_map.html

PUBLIC DOCSOpenTitan top-level design generation

Shows how top-level SystemVerilog, power-domain wrappers, crossbars, memories, peripheral instances, interrupts, clocks and resets are generated and connected from structured design descriptions.

https://opentitan.org/book/hw/top_earlgrey/

SOURCE / FILESOpenTitan source repository

Actual open SoC RTL/software repository. The project is large, but documentation and generated top-level files let you trace from an address-map entry to the peripheral RTL and then to chip-level connections.

https://github.com/lowRISC/opentitan

Inside an SoC, blocks need a protocol too: ready/valid, AXI, TileLink and backpressure

A CPU, DMA engine, GPU, SRAM controller and UART cannot simply connect all of their wires together and hope for the best. Modern SoCs use structured transaction protocols and interconnect fabrics. A common handshake pattern is VALID/READY: the sender says a payload is valid; the receiver says it is ready; the transfer occurs on a clock edge only when both are asserted.

READY / VALID HANDSHAKE source destination payload --------------------------→ VALID --------------------------→ READY ←-------------------------- At rising clock edge: VALID=1, READY=1 → one transfer happens VALID=1, READY=0 → NO transfer; source must keep payload/VALID valid VALID=0, READY=1 → destination is willing, but there is nothing to transfer BACKPRESSURE slow destination deasserts READY ↓ upstream queue fills / stalls ↓ backpressure propagates toward transaction source CROSSBAR EXAMPLE CPU0 ─┐ CPU1 ─┼─→ arbitration + address decode + routing ─┬→ SRAM DMA ─┤ ├→ UART MMIO GPU ─┘ ├→ SPI controller └→ memory controller Responses travel back to the correct originator using routing state / IDs.

Concept	What it solves
ready/valid handshake	Decouples producer and consumer timing; either side can stall without dropping a transaction.
backpressure	Allows a downstream block with no capacity to stop upstream traffic safely.
address decode	Routes a memory/MMIO transaction to the device that owns the requested address range.
arbitration	Chooses which requester gets a contested destination/link when several request simultaneously.
crossbar	Permits multiple independent host→device paths to operate concurrently when they do not conflict.
outstanding transaction	Request has been accepted but its response has not yet returned.
transaction ID / source ID	Lets responses be matched and routed when multiple requests are in flight.
burst	Transfers several adjacent beats under one address/control transaction, reducing overhead.
ordering rule	Defines when responses/operations may be reordered and what ordering software/hardware can rely on.
flow-control buffer	Temporary queue/FIFO absorbs rate differences and prevents combinational timing paths from spanning an entire fabric.

AXI is not one giant bidirectional bus. AXI4 separates write-address, write-data, write-response, read-address and read-data channels, each with its own VALID/READY handshake. That separation is why reads and writes can overlap and why transaction IDs/outstanding requests matter.

OFFICIAL SPECArm AMBA AXI Protocol Specification — latest Version L

Official current AXI specification landing page. The channel-handshake rules define transfers at clock edges where VALID and READY are both asserted.

https://support.arm.com/documentation/ihi0022/l/

DIRECT SPEC PDFArm AXI/ACE specification — direct non-confidential PDF

Older Issue H but directly downloadable and excellent for study. Chapter A3 shows AWVALID/AWREADY, WVALID/WREADY, BVALID/BREADY, ARVALID/ARREADY and RVALID/RREADY timing.

https://developer.arm.com/-/media/Arm%20Developer%20Community/PDF/IHI0022H_amba_axi_protocol_spec.pdf

OFFICIAL SPECSiFive TileLink specification 1.9.3

Current public TileLink specification from SiFive. TileLink is an open on-chip protocol used by Rocket Chip/Chipyard and related RISC-V systems.

https://www.sifive.com/document-file/tilelink-spec-1.9.3

PUBLIC DOCSChipyard TileLink and Diplomacy reference

Public explanation of how caches, memories, MMIO peripherals and DMA devices are connected with TileLink in an actual open SoC generator ecosystem.

https://chipyard.readthedocs.io/en/1.12.2/TileLink-Diplomacy-Reference/

The most useful exercise on this entire page: trace one signal across abstraction levels

Pick one observable event and refuse to accept any unexplained box. Move downward or upward until every transition has an electrical or architectural cause.

Choose one event: CPU reads RAM, UART transmits a byte, keyboard interrupt arrives, GPIO LED turns on, or reset is released.
Find the software-visible operation: instruction, MMIO address, CSR, queue descriptor, or interrupt vector.
Find the hardware block that owns that operation in the memory map or CPU documentation.
Find the RTL/schematic input and output signals of that block.
Trace how address/control logic selects it and how data returns or propagates onward.
Find the clock edge or handshake condition that makes state change.
Find the physical pin/pad if the event leaves the chip.
Find the board trace, pull-up/termination/transceiver/connector and the external device.
Use a logic analyzer or oscilloscope on a low-voltage project and compare the measured waveform/timing with the schematic and datasheet.
Then walk back upward: explain how those voltages become bits, registers, an instruction result, a driver event and finally application-visible behavior.

If you can do this for one complete event, you understand more than someone who has memorized a hundred component definitions. Computer architecture is mostly disciplined composition: state elements, combinational transformations, protocols, timing and physical interfaces.

How you actually observe a running computer: multimeter, oscilloscope, logic analyzer

Schematics tell you what should happen. Instruments tell you what is happening. For slow breadboard CPUs and historical parallel buses, probing CLK, RESET, address/data lines and chip-select signals is one of the fastest ways to make the abstractions real.

Instrument	Good for	What it throws away
Multimeter	DC rail voltage, resistance/continuity, static logic levels	Fast timing and waveform shape
Oscilloscope	Actual voltage vs time: edge shape, ringing, overshoot, rise/fall time, clock quality, analog noise	Usually fewer simultaneous channels than a logic analyzer
Logic analyzer	Many digital channels at once, bus timing, triggers, hexadecimal values, UART/SPI/I²C decoding	Analog detail: it reduces voltage to logic states using thresholds

Probe ground matters. Your measuring instrument becomes electrically connected to the circuit. On mains-referenced or otherwise hazardous hardware, careless grounding can short nodes together or create a shock/fire hazard. The breadboard/low-voltage computer projects in this page are a much safer place to learn probing.

PUBLIC PRIMERTektronix — oscilloscope fundamentals

The primer is readable publicly without registering; login is only advertised for saving material to a library. Covers waveform shape, triggering, bandwidth, sample rate and basic measurements.

https://www.tek.com/en/documents/primer/xyzs-oscilloscopes-primer

PUBLIC MANUALDigilent WaveForms logic-analyzer reference

Public manual with digital channels, buses and built-in SPI/I²C/UART interpreters. Useful as an example of how captured edges become decoded transactions.

https://digilent.com/reference/software/waveforms/waveforms-3/reference-manual

Hardware debug: JTAG, scan chains, breakpoints and single-stepping

A software debugger such as GDB cannot magically read CPU registers while the CPU is crashed before the OS works. Many processors contain dedicated hardware debug logic. A debug probe speaks a physical debug transport such as JTAG; the on-chip debug module can halt a core, inspect registers/memory, install hardware triggers, single-step and resume execution.

GDB / debugger UI on host PC ↓ remote-debug protocol OpenOCD / debug server ↓ USB / Ethernet to debug probe debug probe / adapter ↓ JTAG electrical pins TCK → test clock TMS → TAP state-machine control TDI → serial data into target scan chain TDO ← serial data out of target scan chain nRESET→ optional target reset control VREF ← tells probe target logic voltage ↓ JTAG TAP / Debug Transport Module ↓ Debug Module Interface on-chip Debug Module ↓ halt/resume core, read/write GPRs/CSRs, memory/system-bus access, breakpoint/trigger control, reset debugging, program buffer

Term	Role
TAP	JTAG Test Access Port and its standard state machine/register access mechanism.
IR	Instruction Register: selects which JTAG data-register function the TAP exposes.
DR	Data Register: serial register selected by the current JTAG instruction.
scan chain	One or more serially connected scan/TAP elements through which test/debug bits are shifted.
boundary scan	Using scan cells around chip I/O to test board-level interconnections without needing normal functional execution.
hardware breakpoint / trigger	Comparator/debug logic that can halt execution when a PC/address/event matches.
single step	Resume execution for one architecturally defined instruction/event and re-enter debug mode.
system-bus access	Debug hardware accesses memory/peripherals independently of ordinary CPU instruction execution.

JTAG has two related but distinct uses: board/chip test via scan/boundary-scan, and processor/system debugging. The same TAP transport can expose very different data registers and on-chip logic depending on the selected instruction.

OFFICIAL SPECRISC-V Debug Specification v1.0 — ratified public specification

Current ratified RISC-V debug architecture (v1.0, February 2025). Defines external debug, Debug Module, DMI, JTAG DTM, triggers and debugger behavior.

https://docs.riscv.org/reference/debug/index.html

OFFICIAL SPECRISC-V JTAG Debug Transport Module

The useful physical/transport section: TAP basics, JTAG IR/DR access and the recommended connector with TMS, TCK, TDI, TDO, VREF, GND and nRESET.

https://docs.riscv.org/reference/debug/v1.0/dtm.html

OFFICIAL SPECRISC-V Debug Module

Specifies the logic behind the transport: halt/resume, register access, reset, program buffer, memory access and optional system-bus access.

https://docs.riscv.org/reference/debug/debug_module.html

PUBLIC MANUALOpenOCD User's Guide

Plain public manual for a real open-source debug server. Covers adapters, reset, TAP declaration, CPU targets, flash, JTAG, boundary scan and GDB.

https://openocd.org/doc/html/

PUBLIC MANUALOpenOCD — JTAG Commands

Explains JTAG scan chains, shared TMS/TCK operation, instruction/data registers, BYPASS and shifting bits through multiple TAPs.

https://www.openocd.org/doc/html/JTAG-Commands.html

PUBLIC MANUALOpenOCD — GDB and OpenOCD

Shows how a normal source-level debugger connects to the hardware-debug server in an actual embedded workflow.

https://openocd.org/doc/html/GDB-and-OpenOCD.html

DIRECT PDFCMOS VLSI — Design for Testability / boundary scan (direct PDF)

Direct public slides covering scan design and boundary-scan/JTAG pins TCK, TMS, TDI, TDO and optional TRST.

https://pages.hmc.edu/harris/class/e158/04/lect17.pdf

Computers where almost nothing is hidden

These machines are unusually good teaching objects because the CPU is not a mysterious modern chip. You can trace registers, ALU functions, buses, instruction decoding, clocking and control through relays or ordinary logic ICs.

SCHEMATICSHarry Porter's Relay Computer — circuit diagrams

Scanned circuit and wiring diagrams in roughly the order the machine was built. Includes both abstract circuit drawings and concrete relay-contact wiring diagrams, which makes the difference between logical design and physical wiring explicit.

https://web.cecs.pdx.edu/~harry/Relay/thirdPage.html

PAPER / FILESHarry Porter's Relay Computer — long technical paper and programs

Links to a 92-page technical paper, HTML version, sample machine-language programs, videos and circuit diagrams. No login; it is a plain university-hosted archive.

https://web.cecs.pdx.edu/~harry/Relay/secondPage.html

OPEN HARDWAREGigatron TTL microcomputer — a computer with no microprocessor

An 8-bit computer built from ordinary 7400-series TTL logic rather than a CPU chip. It generates video in software while also running programs. The design, PCB files, schematics and software were released as open source.

https://gigatronttl.eu/

SCHEMATICS / FILESGigatron TTL — project files and schematics

Direct project file index containing the full circuit schematic PDF, KiCad/gerber hardware files, instruction-set material and technical presentations. Useful for tracing how a CPU can be assembled from fewer than forty logic ICs.

https://hackaday.io/project/20781/files

SOURCE / FILESGigatron ROM, assembler and system source

Public source repository for the Gigatron's native code, assembler, ROM, kernel, apps and virtual CPU. Particularly interesting because hard real-time video/audio generation, I/O and computation all have to coexist on a very small TTL CPU.

https://github.com/kervinck/gigatron-rom

SCHEMATIC ARTICLENibbler — custom 4-bit CPU schematic and control

A complete 4-bit CPU and I/O system whose whole schematic fits on one page. Except for RAM and ROM it uses common 7400-series logic. The article walks through program ROM, registers, ALU, control and how the blocks connect.

https://www.bigmessowires.com/2013/08/27/custom-4-bit-cpu-schematic-and-control/

FREE WEB BOOKPutting the “You” in CPU

A free, plain-web systems explainer for the layer above CPU hardware: what actually happens when a program runs, system calls, interrupts, multitasking, ELF executables, paging and process creation. Best read after you understand a simple processor.

https://cpu.land/

From transistor layout to a packaged chip soldered onto a PCB

A schematic transistor symbol is eventually realized as patterned regions and conductors on silicon. The chip does not end at the transistor: the fabricated wafer is diced, each die is packaged, tiny on-die pads or bumps connect through the package to external pins/balls, and those connect to copper traces, planes, capacitors, connectors and other chips on the PCB.

RTL / transistor schematic / layout ↓ photomasks / process layers ↓ silicon wafer ├── oxidation / dielectric formation ├── photolithography ├── etch ├── ion implantation / diffusion (doping) ├── deposition / polysilicon / contacts / vias └── multiple metal interconnect layers ↓ wafer test → dice wafer into individual dies ↓ die attached inside/on package ↓ wire bonds OR flip-chip bumps ↓ package substrate / leadframe ↓ package pins / BGA balls / LGA lands ↓ PCB pads + vias + copper traces / power & ground planes ↓ other ICs, memory, connectors, regulators, crystals, passives

Layer of reality	What matters there
Transistor/layout	Gate length/width, source/drain diffusion, wells, contacts, local capacitances and resistances.
On-chip interconnect	Metal resistance/capacitance, coupling, repeaters, clock routing, IR drop and electromigration.
I/O pad	Large drivers, input buffers, Schmitt behavior, voltage domains, ESD structures and level conversion.
Package	Pin/ball assignment, bond/bump parasitics, package power distribution, thermal path and mechanical constraints.
PCB	Trace geometry, return-current path, characteristic impedance, crosstalk, vias, connectors and termination.
Power network	Regulators, planes, decoupling capacitors, transient current delivery and ground/reference integrity.

DIRECT PDFCMOS VLSI — Circuits & Layout (direct PDF)

Direct public slides that connect transistor schematics to actual CMOS layout layers. Useful for seeing diffusion, polysilicon, contacts, wells and metal as geometry rather than symbols.

https://pages.hmc.edu/harris/cmosvlsi/4e/lect/lect1.pdf

DIRECT PDFCMOS VLSI — Design Flow, fabrication and packaging (direct PDF)

Public technical slides connecting design/tapeout to fabrication, wafer dicing and packaging, including bond-wire connection from die I/O pads to the package.

https://pages.hmc.edu/harris/class/e158/10/lect2.pdf

DIRECT PDFCMOS VLSI — introductory fabrication steps (direct PDF)

Contains a compact bottom-up CMOS fabrication sequence: oxide, patterned openings, dopant implant/diffusion, well formation and subsequent process layers.

https://pages.hmc.edu/harris/cmosvlsi/4e/lect/lect0.pdf

DIRECT PDFCMOS VLSI — I/O pads, level conversion and ESD (direct PDF)

Direct public slides showing that a chip pin is not wired straight into tiny core logic. Input/output pads include large drivers, level conversion, Schmitt/noise filtering and electrostatic-discharge protection.

https://pages.hmc.edu/harris/class/e158/16/lect23.pdf

DIRECT PDFTI — High-Speed Layout Guidelines (direct PDF)

Excellent no-login application report on clocks, microstrip/stripline, impedance, reflections, termination, return-current paths, crosstalk, decoupling and practical PCB routing.

https://www.ti.com/lit/an/scaa082a/scaa082a.pdf

Modern processors are often systems-in-a-package: chiplets, die-to-die links and CXL memory

A 'CPU package' no longer necessarily contains one monolithic die. Designers can split cores, cache, I/O, memory controllers or accelerators into chiplets connected by an on-package die-to-die fabric. This improves modularity and manufacturing economics, but introduces another layer of links, clocking, power delivery, routing, coherency and failure/debug considerations between what software still perceives as one processor/system.

MULTI-DIE PROCESSOR PACKAGE compute chiplet 0 ─┐ compute chiplet 1 ─┼─→ on-package coherent fabric / die-to-die PHY compute chiplet 2 ─┤ ↓ I/O die / tile ─┘ package substrate/interposer ├→ DDR memory channels ├→ PCIe root ports ├→ CXL-capable ports └→ platform I/O UCIe-style chiplet link concept: protocol/transaction layer ↓ adapter/link management ↓ die-to-die PHY ↓ microbumps / package traces / interposer / 3D bonding neighbor chiplet CXL MEMORY-EXPANSION CONCEPT CPU / host bridge ↓ CXL link/fabric CXL Type-3 memory device ↓ host-managed device memory ↓ HDM decoders map system physical address range to device physical address Linux may expose that capacity as DAX or online it as System RAM

Concept	Meaning
monolithic die	Most major functions integrated onto one piece of silicon.
chiplet	Smaller die intended to be combined with other dies inside a package/system-in-package.
interposer / package substrate	Physical routing medium connecting dies, power and external package pins/balls.
die-to-die PHY	Electrical transmitter/receiver circuitry optimized for very short on-package links.
coherent fabric	Interconnect carrying transactions plus rules/messages needed to keep caches/memory views coherent.
CXL.io	CXL protocol used for discovery/configuration and ordinary I/O-style access, closely related to PCIe mechanisms.
CXL.cache	Allows a capable device to coherently cache/access host memory.
CXL.mem	Allows the host CPU to coherently access/cache memory attached to a CXL device.
HDM decoder	Host-Managed Device Memory decoder mapping a host/system physical address range through the CXL topology to device physical memory.
memory tier	OS grouping/classification of NUMA memory nodes by performance characteristics; CXL memory may form slower/different tiers from local DRAM.

Access-policy note: the CXL Consortium's current CXL 4.0 evaluation-specification download is form/terms gated, so this guide intentionally does not make it a recommended learning link. The Linux kernel's CXL documentation below is direct, public and technically detailed.

KERNEL DOCSLinux CXL — devices and three core protocols

Very useful public overview of CXL.io, CXL.cache and CXL.mem plus Type-1/2/3 devices, switches, memory expanders and dynamic-capacity concepts.

https://docs.kernel.org/driver-api/cxl/devices/device-types.html

KERNEL DOCSLinux CXL driver operation — real fabric topology

Shows actual Linux objects for roots, ports, endpoints, memory devices, decoders, interleaved regions and DAX exposure, with concrete sysfs examples.

https://docs.kernel.org/driver-api/cxl/linux/cxl-driver.html

KERNEL DOCSLinux CXL theory of operation

Explains how PCIe enumeration, ACPI objects and CXL.mem HDM decode topology are assembled into host physical-address regions.

https://docs.kernel.org/driver-api/cxl/theory-of-operation.html

KERNEL DOCSLinux CXL — DAX and System RAM exposure

Shows how a CXL memory region can become a directly mmap-able DAX device or be converted into ordinary System RAM for the kernel page allocator.

https://docs.kernel.org/driver-api/cxl/linux/dax-driver.html

KERNEL DOCSLinux CXL early boot and memory tiers

Connects EFI/ACPI memory maps, NUMA nodes, CXL memory and kernel memory-tier creation during boot.

https://docs.kernel.org/driver-api/cxl/linux/early-boot.html

DIRECT PDFAMD — CXL memory expansion solution brief (direct PDF)

Public vendor overview showing CXL as a coherent memory/accelerator interconnect and explicitly separating CXL.io, CXL.mem and CXL.cache capabilities.

https://www.amd.com/content/dam/amd/en/documents/products/adaptive-socs-and-fpgas/versal/cxl-solution-brief-versal-premium-series-gen-2.pdf

PUBLIC OVERVIEWUCIe Consortium — current chiplet-standard overview

No-login overview of the UCIe generations. UCIe 3.0 is the current listed specification family and supports 48/64 GT/s plus newer manageability features; full spec downloads are not needed for this learning path.

https://www.uciexpress.org/specifications

PUBLIC ARTICLEUCIe 3.0 technical overview

Public no-login summary of standardized die-to-die interconnect, 48/64 GT/s rates, package-level connectivity, runtime recalibration and manageability.

https://www.uciexpress.org/post/ucie-3-0-specification-redefining-chiplet-interconnects

DIRECT PDFUCIe 2.0 technical presentation (direct PDF)

Direct public presentation explaining UCIe's role as a high-bandwidth, low-latency on-package chiplet interconnect and the motivation for standardized multi-die systems.

https://www.uciexpress.org/_files/ugd/0c1418_b6481ec611e24c6e91f1beb743b0c860.pdf

When transistors and wires stop being ideal: real CMOS physical design

This is the layer where 'a gate has a delay' gets unpacked into transistor current, parasitic capacitance, wire resistance/capacitance, crosstalk, clock skew, power distribution and layout. It is difficult material and belongs here.

DIRECT PDFWires / on-chip interconnect — David Harris (direct PDF)

Thirty-one pages on metal layers, resistance, capacitance, RC delay, crosstalk, energy, repeaters and why modern ICs are as much a wiring problem as a transistor problem.

https://pages.hmc.edu/harris/class/e158/16/lect14.pdf

PUBLIC PDF INDEXCMOS VLSI public lecture-PDF index — Harvey Mudd

Plain public index; no account or enrollment needed. Direct PDFs cover transistor theory, nonideal devices, transient response, power, combinational/sequential circuits, wires, SRAM, ROM/CAM/PLA, packaging, clocks, PLL/DLL and I/O.

https://pages.hmc.edu/harris/class/e158/16/index.html

OPEN NOTESMIT 6.374 digital integrated-circuit notes index

OpenCourseWare means no signup. The technical sequence includes inverter delay/power, pass-transistor and dynamic logic, arithmetic, latches/registers, metastability, interconnect, clock skew/jitter, clock distribution, SRAM, DRAM and power distribution.

https://ocw.mit.edu/courses/6-374-analysis-and-design-of-digital-integrated-circuits-fall-2003/pages/lecture-notes/

TECH REPORTPower and Ground Requirements for a High-Speed 32-Bit Computer Chip Set — Berkeley

Free Berkeley technical report from 1986. Connects faster clocks and wider buses to inductance, CMOS noise margins, supply separation, pad loading/delay and package power/ground behavior.

https://www2.eecs.berkeley.edu/Pubs/TechRpts/1986/6073.html

At the silicon level — actual transistors inside real chips

These are especially valuable after you know what an ALU, register, bus and control signal are. They connect schematic symbols back to doped silicon, polysilicon, metal layers and individual MOS transistors.

TECH ARTICLE6502 overflow flag at the transistor level — Ken Shirriff

Walks from a 6502 die photograph to individual NMOS enhancement/depletion transistors, then reconstructs the Boolean logic that produces the overflow flag. Excellent transistor → gate → CPU behavior bridge.

https://www.righto.com/2013/01/a-small-part-of-6502-chip-explained.html

TECH ARTICLE8085 ALU reverse-engineered from the die — Ken Shirriff

Shows how an actual arithmetic/logic unit is recovered from a transistor netlist. Useful for understanding that an ALU is a deliberately arranged network of gates, buses and control signals.

https://www.righto.com/2013/01/inside-alu-of-8085-microprocessor.html

TECH ARTICLE8086 ALU reverse-engineered from die photos — Ken Shirriff

Detailed 16-bit ALU analysis including dynamic NMOS circuits, control signals, carry handling and physical layout. Much closer to real implementation than a textbook block diagram.

https://www.righto.com/2020/08/reverse-engineering-8086s.html

TECH ARTICLE8086 address/data pin circuitry — Ken Shirriff

Explains how an external CPU pin connects to the internal address/data bus, including MOSFET I/O stages, bus precharge, multiplexed address/data signals and physical pad circuitry.

https://www.righto.com/2023/07/8086-pins.html

TECH ARTICLEInside the Apple-1's MOS clock driver chip — Ken Shirriff

Explains a real Apple-1 support chip that generated the unusual high-current, two-phase clocks required by its shift-register video memory. Good evidence that a 'clock' is an electrical circuit, not just a symbol.

https://www.righto.com/2022/03/inside-apple-1s-unusual-mos-clock.html

From reset vector to operating system: boot and firmware

Powering a CPU does not somehow produce an operating system. Reset forces defined processor state; execution begins at an architecturally defined address/vector; firmware initializes enough hardware to find and load the next stage.

OPEN DOCScoreboot documentation

Open-source firmware documentation. coreboot deliberately focuses on the fundamental job: initialize hardware enough to make the machine usable, then hand control to a payload. This makes firmware less mysterious than opaque vendor BIOS images.

https://doc.coreboot.org/

OPEN DOCScoreboot FAQ — what firmware actually does

Plain explanation of hardware initialization, payloads and the distinction between coreboot and UEFI.

https://doc.coreboot.org/getting_started/faq.html

SPEC HTMLUEFI Specification 2.11 — introduction/overview

Public HTML specification describing the software-visible interface between platform firmware, OS loader and operating system. Dense, but authoritative.

https://uefi.org/specs/UEFI/2.11/01_Introduction.html

SPEC HTMLUEFI 2.11 — Boot Manager

The actual standard boot-manager sequence: NVRAM boot variables, loading firmware drivers/applications and launching an OS loader.

https://uefi.org/specs/UEFI/2.11/03_Boot_Manager.html

Do not confuse these: socket, core, logical CPU, NUMA node and memory channel

Term	Physical/logical meaning	Can there be several?
socket	Motherboard/package attachment position for one processor package.	A server can have multiple sockets.
processor package	Physical packaged silicon assembly in a socket; may contain several dies/chiplets/tiles.	One per populated socket in common systems.
die / chiplet / tile	Piece of silicon inside the package implementing cores, cache, I/O or other functions.	Modern packages can contain many.
physical CPU core	Execution core with its own pipeline/execution resources and architectural-thread capacity.	Usually many per package.
logical CPU / hardware thread	Architectural execution context exposed to the OS scheduler; SMT may expose >1 per physical core.	Potentially two or more per SMT-capable core.
NUMA node	Locality domain grouping CPUs and/or memory with similar access cost.	One socket may expose one or several; memory-only nodes can also exist.
memory controller	Hardware scheduling/issuing DDR or other memory transactions.	Several per package/socket are common.
memory channel	Independent physical memory interface attached to a controller.	Many server CPUs expose multiple channels per socket.
cache domain	Set of CPUs sharing some cache level, often LLC slices or cluster cache.	May not line up exactly with NUMA/socket boundaries.

example hierarchy (illustrative, not universal): SYSTEM ├── socket 0 / package 0 │ ├── NUMA node 0 │ │ ├── cores 0..7 │ │ │ └── 2 SMT threads each → 16 logical CPUs │ │ └── memory channels A/B → DIMMs │ └── NUMA node 1 │ ├── cores 8..15 → 16 more logical CPUs │ └── memory channels C/D → DIMMs └── socket 1 / package 1 └── analogous nodes/cores/channels OS scheduler sees logical CPUs. Memory allocator sees NUMA nodes/pages. Hardware coherence/interconnect ties the whole machine together.

A logical CPU can leave the scheduler while the machine keeps running: CPU hotplug migrates work, tears down per-CPU state and can later bring it back

Linux distinguishes CPUs that are possible, physically/logically present, and currently online. Taking a logical CPU offline is not equivalent to putting it in a deep idle state. Before the kernel can clear that CPU from the online set, runnable tasks, interrupts, timers and subsystem-specific per-CPU state must be moved or torn down in a coordinated order. Bringing the CPU back reverses the process through a hotplug state machine and architecture-specific startup.

CPU N ONLINE scheduler may run tasks there IRQs may target it timers / per-CPU work / RCU / perf / drivers may own state ↓ echo 0 > /sys/devices/system/cpu/cpuN/online ↓ CPU hotplug teardown callbacks ↓ migrate runnable tasks away retarget/migrate interrupts migrate timers and per-CPU work where required subsystems dismantle CPU-local state ↓ architecture disables CPU execution path ↓ clear CPU N from cpu_online_mask ↓ CPU N OFFLINE (no normal scheduling or device IRQ delivery there) later: echo 1 > .../cpuN/online ↓ PREPARE callbacks on control CPU ↓ architecture starts secondary CPU ↓ STARTING callbacks on that CPU ↓ ONLINE callbacks rebuild per-CPU subsystem state ↓ set/restore usable online state

Mask/state	Meaning
`possible`	CPU IDs for which the kernel has provisioned enough resources that they could potentially become available.
`present`	CPUs currently known to exist in the system; physical hot-add/remove can affect this set on supporting platforms.
`online`	CPUs currently participating in scheduling and normal kernel work.
PREPARE hotplug states	Callbacks executed on a control CPU before startup or after the outgoing CPU is already unusable.
STARTING states	Low-level callbacks executed on the hotplugged CPU with interrupts disabled during early bring-up/late teardown.
ONLINE states	Subsystem callbacks that run on the hotplugged CPU with normal execution facilities available.

CPU hotplug is not C-state idle and not DVFS. An online CPU may enter deep idle states thousands of times per second while remaining a scheduling target. DVFS changes operating frequency/voltage while the CPU remains online. Hotplug removes the logical CPU from ordinary OS service and requires cross-subsystem lifecycle callbacks. Physical socket/CPU hot-remove is an additional platform/firmware problem layered on top.

KERNEL DOCSLinux CPU hotplug state machine

Authoritative documentation for possible/present/online masks, sysfs online/offline control, task/IRQ/timer migration and PREPARE/STARTING/ONLINE callbacks.

https://docs.kernel.org/core-api/cpu_hotplug.html

KERNEL DOCSLinux CPU topology and exported CPU masks

Explains the sysfs topology view and the meanings of kernel_max, offline, online, possible and present CPU sets.

https://docs.kernel.org/admin-guide/cputopology.html

NUMA: all RAM is addressable, but not all RAM is equally close

Non-Uniform Memory Access (NUMA) appears when a system has multiple CPU/memory domains. Every CPU may be able to access the whole coherent physical address space, but memory attached to the local socket/node usually has lower latency and higher effective bandwidth than remote memory reached across a socket/interconnect link.

TWO-SOCKET ccNUMA EXAMPLE Socket / NUMA node 0 Socket / NUMA node 1 +-------------------------+ +-------------------------+ | CPU cores + LLC |<-- coherent --> | CPU cores + LLC | | integrated mem ctrl 0 | fabric/link | integrated mem ctrl 1 | +-----------+-------------+ +-------------+-----------+ | | DDR DDR | | local RAM 0 local RAM 1 Core on node 0 → RAM 0 = LOCAL access Core on node 0 → RAM 1 = REMOTE access across inter-socket fabric same physical address space / hardware cache coherence different latency + bandwidth depending on source and target locality

Policy / behavior	What it means
local allocation	Allocate a page from memory near the CPU/node performing the allocation when possible.
bind	Restrict allocations to one or more specified NUMA nodes.
preferred	Prefer one node but allow fallback.
interleave	Spread page allocations across selected nodes to distribute bandwidth/capacity.
automatic NUMA balancing	Kernel samples memory access and may migrate pages/tasks to reduce remote-access cost.
CPU affinity	Keep a thread on selected CPUs; useful only if its important memory is also placed sensibly.
memory migration	Move an already allocated page from one NUMA node to another to improve locality.
remote access	Load/store serviced by memory physically attached to another node/socket, usually adding fabric traversal and contention.

NUMA is not the same as cache coherence. ccNUMA systems normally keep caches coherent across nodes, so software still sees one shared-memory address space. 'Non-uniform' means the cost of reaching that shared memory depends on topology.

KERNEL DOCSLinux — What is NUMA?

Excellent kernel explanation of ccNUMA: all memory remains visible, caches/interconnect maintain coherence, but remote memory has worse latency/bandwidth than local memory.

https://docs.kernel.org/6.2/mm/numa.html

KERNEL DOCSLinux — NUMA Memory Policy

Current detailed documentation for local/default, bind, preferred, interleave and weighted-interleave allocation policies, plus mbind()/set_mempolicy().

https://docs.kernel.org/admin-guide/mm/numa_memory_policy.html

KERNEL DOCSLinux — NUMA Memory Performance

Current documentation for memory initiators/targets, read/write latency and bandwidth attributes, heterogeneous memory and ACPI HMAT-exposed topology.

https://docs.kernel.org/admin-guide/mm/numaperf.html

NUMA placement happens when pages are allocated: first touch, mempolicy, migration and automatic balancing

A VMA does not physically live on a NUMA node merely because mmap() created it. For demand-paged anonymous memory, physical placement usually happens when a CPU first faults/touches each page. Linux memory policy can bias, bind or interleave those allocations, and already-populated pages can later be migrated.

TWO-SOCKET / TWO-NUMA-NODE SYSTEM CPU0..15 near Node0 DRAM CPU16..31 near Node1 DRAM thread runs on CPU3 (Node0) p = mmap(NULL, 1 GiB, MAP_PRIVATE|MAP_ANONYMOUS, ...) ↓ only VIRTUAL range created; most physical pages not allocated yet FIRST TOUCH CPU3 writes p[0] ↓ page fault default/local memory policy ↓ allocate physical page from Node0 if possible ↓ install PTE Later thread migrates to CPU20 (Node1) existing page remains on Node0 ↓ every access from CPU20 can become REMOTE NUMA traffic unless scheduler/task moves back or page is migrated MEMORY POLICY OPTIONS MPOL_LOCAL / default allocate near CPU performing the fault/allocation when possible MPOL_BIND {1} allocate only from Node1 set; failure/fallback semantics follow policy rules MPOL_PREFERRED {1} try Node1 first, then allowed fallback nodes if needed MPOL_PREFERRED_MANY {0,1} prefer a set of nodes, allowing fallback elsewhere under pressure MPOL_INTERLEAVE {0,1} spread new allocations across nodes rather than localizing all to one mbind(addr,len,...) installs policy on a VMA/range optional MPOL_MF_MOVE/STRICT-style flags can migrate/check existing pages move_pages(pid,...) query or explicitly move selected pages between nodes AUTOMATIC NUMA BALANCING kernel samples memory locality via hinting/protection faults ↓ detect task ↔ memory mismatch ↓ migrate task and/or pages toward better locality when worthwhile CROSS-LAYER RULE CPU affinity and memory policy are coupled: pinning threads to Node1 CPUs while their hot pages remain on Node0 can be worse than leaving scheduling unconstrained.

NUMA policy/tool	Meaning
first touch	Demand-paged physical page is allocated when first faulted/touched, under the policy active for that allocation.
MPOL_DEFAULT	Remove explicit policy and fall back to the next applicable scope/system default.
MPOL_BIND	Restrict allocation to a specified node set.
MPOL_PREFERRED	Prefer one node but allow fallback according to policy/system constraints.
MPOL_PREFERRED_MANY	Prefer a nodemask rather than one node, with fallback beyond it under pressure.
MPOL_INTERLEAVE	Distribute new pages over a selected set of NUMA nodes.
set_mempolicy()	Sets calling thread/task memory policy for future allocations not governed by a more-specific VMA policy.
mbind()	Sets memory policy for a virtual-address range and can optionally request migration/checking of existing pages.
move_pages()	Queries current node placement or migrates individual pages.
numa_maps	/proc view of policy and page counts per NUMA node for each process mapping.
automatic NUMA balancing	Kernel mechanism sampling access locality and migrating tasks/pages to reduce remote-memory cost.
cpuset.mems	Administrative cgroup/cpuset restriction limiting which NUMA nodes a task's allocations/policies may use.

Changing policy does not automatically move old pages. Linux's NUMA policy documentation explicitly notes that task/VMA policy normally governs pages allocated after the policy is installed; migration flags or move_pages()/automatic balancing are separate mechanisms for already-populated memory.

NUMA PLACEMENT LAB numactl --hardware 2>/dev/null numactl --show 2>/dev/null numastat -p $$ 2>/dev/null cat /proc/$$/numa_maps | less # Compare first-touch placement: # 1. pin a test program to one NUMA node's CPUs # 2. allocate a large anonymous mapping # 3. initialize every page from that thread # 4. inspect numa_maps / numastat # Then move computation to a remote node and compare memory latency/bandwidth. # Use a disposable benchmark; NUMA pinning can reduce performance if policy and workload disagree.

KERNEL DOCSLinux NUMA overview

Connects scheduler topology, CPU affinity, default local memory allocation and NUMA-aware application policy.

https://docs.kernel.org/mm/numa.html

CURRENT PUBLIC MANUALmove_pages(2) — current Linux man-pages

Current API for querying/migrating selected pages between NUMA nodes and its interaction with cpusets/memory policy.

https://man7.org/linux/man-pages/man2/move_pages.2.html

KERNEL DOCSLinux Memory Management index

Current MM documentation hub linking NUMA policy/performance, Multi-Gen LRU, THP, DAMON and related placement/reclaim mechanisms.

https://docs.kernel.org/admin-guide/mm/

Why multicore makes memory harder: cache coherence

If CPU core A and core B both cache the same memory block and A writes it, B must not continue indefinitely using a stale value. Cache coherence protocols maintain rules about which cached copies are valid, shared, modified, or owned. This is distinct from the broader question of memory consistency, which constrains what ordering of reads/writes software may observe.

OPEN NOTES / VIDEOMIT 6.004 — cache coherence

Public undergraduate lecture/transcript on cache coherence. No sign-up; video can also be downloaded.

https://ocw.mit.edu/courses/6-004-computation-structures-spring-2017/resources/cache-coherence-9-30-/

DIRECT PDFMIT 6.823 — directory-based coherence protocol (direct PDF)

Direct PDF defining cache-line states and the messages/assumptions of a directory protocol. Good after the basic idea is clear.

https://ocw.mit.edu/courses/6-823-computer-system-architecture-fall-2005/6620f8baa4395cc18ba2c03ecfcecc4c_handout12.pdf

PUBLIC PDF INDEXCMU 15-740 architecture handouts

Plain public index of PDFs including memory hierarchy, small/large-scale cache coherence, memory consistency, synchronization, interconnection networks, virtual memory and pipelining.

https://www.cs.cmu.edu/afs/cs/academic/class/15740-f12/www/handouts.html

OPEN TEXTBOOKMIT Principles of Computer System Design — open textbook

Legally free open textbook. Later chapters cover atomicity, consistency and cache coherence from a broader systems perspective.

https://ocw.mit.edu/courses/res-6-004-principles-of-computer-system-design-an-introduction-spring-2009/pages/open-textbook/

Worked cache coherence: one cache line moving through MESI states

Cache coherence is easiest to understand by following one cache line. MESI names four states: Modified, Exclusive, Shared and Invalid. The exact implementation can be snooping, directory-based, inclusive/non-inclusive, and much more complicated internally, but the state-machine view captures the ownership rules that software-visible coherence depends on.

Start: memory contains X = 0 Core A cache: I Core B cache: I 1) Core A LOAD X → A obtains line, no other cached copy known → A: E (Exclusive), B: I, memory still 0 2) Core B LOAD X → coherence machinery observes another reader → A: S, B: S, memory 0 3) Core A STORE X = 1 → A must gain write ownership → invalidate B's copy → A: M, B: I, memory may still contain old 0 in a write-back cache 4) Core B LOAD X → B cannot use its invalid copy → coherence obtains newest data from A / ownership path → both may end S with value 1; lower memory becomes or remains coherent as required Important: Modified means the cache has the newest version and lower memory is stale. Exclusive means clean + only this cache has a copy. Shared means clean + other caches may have copies. Invalid means this cache line slot is not a usable copy.

State	Valid?	Memory up to date?	Other caches may have copy?	Local write
M — Modified	Yes	No	No	Write locally; line remains owned/dirty.
E — Exclusive	Yes	Yes	No	Can transition to Modified without first invalidating another copy.
S — Shared	Yes	Yes	Yes	Must obtain write ownership and invalidate peers before modifying.
I — Invalid	No	—	Maybe elsewhere	Must fetch/obtain line before using it.

False sharing: coherence works at cache-line granularity, not C-variable granularity. If two cores repeatedly write different variables that happen to occupy the same cache line, ownership can ping-pong even though the source-level variables are logically independent.

DIRECT PDFIntel SDM Volume 3A — MESI cache-control protocol (direct PDF)

Intel's system-programming manual explicitly defines Modified, Exclusive, Shared and Invalid cache-line states and what they imply about memory freshness, peer copies and writes.

https://cdrdv2-public.intel.com/835754/253668-sdm-vol-3a.pdf

PUBLIC NOTESCornell CS3410 — cache coherence and false sharing

Public notes covering snooping, VI/MSI protocols and false sharing. Useful before the more implementation-specific Intel material.

https://www.cs.cornell.edu/courses/cs3410/2025fa/notes/cachecoherency.html

Beyond broadcast MESI: directories and snoop filters remember who might have each cache line

A simple snooping system can broadcast every coherence request to all caches, but broadcast traffic scales poorly. A directory protocol keeps metadata describing the owner and/or sharers of each coherent cache line so requests can be routed only to caches that might actually hold a copy. Large coherent meshes often combine home nodes, directories, last-level caches and snoop filters.

DIRECTORY ENTRY FOR CACHE LINE X state: Shared sharers: {CPU0, CPU3, CPU7} owner: none CPU5 LOAD X L1 miss → GetS/ReadShared request to HOME/DIRECTORY for X ↓ directory sees shared clean copies ↓ data may come from LLC/memory or another cache depending protocol ↓ directory adds CPU5 to sharer set CPU5 STORE X needs exclusive/write permission → GetM/GetX/ReadUnique ↓ directory knows EXACT candidate sharers ↓ send invalidations to CPU0, CPU3, CPU7 ↓ each cache serializes invalidation and returns Inv-Ack ↓ after all required acknowledgments: directory state = Modified/Exclusive owner CPU5 CPU5 may write line IF CPU3 HAD DIRTY/OWNED DATA home may forward request to CPU3 CPU3 supplies latest data directly to CPU5 or through home and downgrades/invalidates according to protocol SNOOP FILTER metadata says which upstream/private caches MIGHT hold a line ↓ request for unrelated line does not need to probe every core ↓ less broadcast traffic / lower power DIRECTORY CAPACITY PROBLEM if directory/snoop-filter entry for X must be evicted while private caches still hold X ↓ protocol may need back-invalidation/probes or a conservative fallback so it never forgets a hidden sharer incorrectly.

Coherence concept	Meaning
snooping protocol	Coherence requests are observed/broadcast across a shared ordered interconnect or snoop domain.
directory protocol	A home/directory tracks sharer/owner metadata and sends targeted coherence messages.
home node	Serialization/point-of-coherency agent responsible for a physical address range.
sharer vector	Bitset or compressed structure indicating which caches/nodes currently hold a shared copy.
owner	Node believed to hold the unique/latest dirty copy when memory/LLC may be stale.
GetS / ReadShared	Request read/shared permission and valid data for a line.
GetM/GetX / ReadUnique	Request exclusive/write permission; existing sharers must normally be invalidated.
invalidation acknowledgment	Confirms a cache has serialized an invalidation; writer cannot assume exclusive permission until required acks complete.
snoop filter	Directory-like metadata used to avoid sending probes to caches that cannot contain a line.
point of serialization	Agent/location where conflicting requests for one coherence block are ordered.
direct cache transfer	Latest data moves cache-to-cache without necessarily round-tripping through DRAM.
transient coherence state	Temporary controller state while requests, invalidations, data and acknowledgments are still in flight.
directory eviction/back-invalidation	Removing metadata may require invalidating upstream copies so the system does not lose track of sharers.

Directory coherence does not eliminate snoops; it targets them. The directory's value is knowing which nodes are relevant to a particular line, so invalidations/data probes can be sent selectively instead of broadcast to every coherent cache.

SIMULATOR / SOURCE LAB Use gem5 Ruby's current MSI/MESI directory protocols: read MSI-cache.sm, MSI-dir.sm and MSI-msg.sm ↓ identify message networks for requests, forwards and responses ↓ trace a line through: CPU0 GetS CPU1 GetS CPU2 GetM ↓ record directory sharer state and every invalidation/ack Then compare: directory protocol: targeted messages to known sharers broadcast/snoop protocol: every relevant cache must observe request Useful stress case: many cores repeatedly read one line, then one core writes it → watch invalidation fan-out and acknowledgment traffic. gem5 Ruby includes random testers because coherence state machines have many transient races that are easy to get wrong.

CURRENT PUBLIC DOCSgem5 — Cache Coherence Protocols (July 2026)

Current protocol vocabulary for GETS, GETX, INV, ACK/NACK, writebacks and the message-driven state machines used by Ruby coherence models.

https://www.gem5.org/documentation/general_docs/ruby/cache-coherence-protocols/

CURRENT PUBLIC DOCSgem5 — MESI Two Level directory protocol

Current documentation for a private-L1/shared-L2 MESI hierarchy whose on-chip coherence is explicitly maintained through a directory co-located with L2 lines.

https://www.gem5.org/documentation/general_docs/ruby/MESI_Two_Level/

CURRENT PUBLIC DOCSgem5 — MSI directory tutorial

Current step-by-step tutorial for implementing a three-hop directory protocol and understanding the state machines/messages rather than only memorizing MESI letters.

https://www.gem5.org/documentation/learning_gem5/part3/MSIintro/

CURRENT PUBLIC DOCSgem5 — MSI Directory implementation

Concrete directory-controller SLICC implementation with request, forward and response networks.

https://www.gem5.org/documentation/learning_gem5/part3/directory/

CURRENT PUBLIC DOCSgem5 — CHI coherent hierarchy

Current scalable coherent-interconnect model: home nodes act as points of coherency/serialization and can include LLC plus a directory for targeted snoops.

https://www.gem5.org/documentation/general_docs/ruby/CHI/

FREE BOOK EXCERPTSorin/Hill/Wood — Directory Coherence Protocols excerpt

Free directory-protocol chapter excerpt with sharer vectors, GetS/GetM flows, invalidations and explicit acknowledgement messages.

https://www.gem5.org/pages/static/external/Sorin_et-al_Excerpt_8.2.pdf

PUBLIC TOOL DOCSgem5 Ruby Random Tester

Current testing tool for stressing coherence state machines and their races across cache/directory/DMA controllers.

https://www.gem5.org/documentation/general_docs/debugging_and_testing/directed_testers/ruby_random_tester/

Atomicity is not ordering: LR/SC, AMOs, acquire/release and memory fences

In a multicore machine, it is not enough to say 'each core executes its program in order.' Compilers and CPUs may reorder, speculate, buffer and combine memory operations. Atomicity means an operation appears indivisible with respect to competing accesses; memory ordering controls which operations other cores/devices are allowed to observe before or after others.

SPINLOCK IDEA shared lock = 0 Core A: atomic acquire lock 0→1 ↓ acquire ordering read/write protected shared data ↓ release ordering atomic release lock 1→0 Core B: repeats atomic attempt until lock becomes available RISC-V mechanisms: LR = load-reserved SC = store-conditional (succeeds only if reservation still valid) AMO = atomic memory operation such as swap/add/and/or/xor/min/max aq = acquire bit rl = release bit FENCE= explicit ordering constraint between selected classes of accesses

Property	Meaning
atomic RMW	Read + modify + write behaves as one atomic memory operation relative to competing operations.
acquire	Later memory operations may not be observed as happening before the acquire in the prohibited direction.
release	Earlier memory operations may not be observed as happening after the release in the prohibited direction.
sequentially consistent atomic	Provides stronger ordering constraints than relaxed/acquire/release forms, at possible hardware/compiler cost.
memory barrier / fence	Restricts reordering/visibility of selected memory operations across the barrier.
cache coherence	Makes shared cached copies converge according to a coherence protocol; does not by itself define all legal ordering of independent memory operations.
memory consistency model	Architectural contract describing which values/orderings loads and stores may legally observe across threads/harts.

Important distinction: a coherent cache system can still implement a weak memory-consistency model. Coherence primarily answers 'what happens to multiple cached copies of one location?'; consistency answers broader ordering questions across multiple locations and operations.

OFFICIAL SPECRISC-V A extension — atomic instructions

Ratified spec for LR/SC and atomic memory operations. It explicitly defines aq/rl acquire/release semantics and gives lock/CAS examples.

https://docs.riscv.org/reference/isa/unpriv/a-st-ext.html

OFFICIAL SPECRISC-V RVWMO memory consistency model

Normative RISC-V weak-memory-ordering model. Dense, but this is the actual definition of which load/store executions are legal.

https://docs.riscv.org/reference/isa/unpriv/rvwmo.html

KERNEL DOCSLinux atomic types and ordering

Concrete mapping of relaxed/acquire/release/fully ordered atomic operations and barrier augmentation in real systems code.

https://docs.kernel.org/core-api/wrappers/atomic_t.html

Stores can retire before becoming globally visible: store buffers, load queues and forwarding

Modern out-of-order cores separate architectural retirement from the moment a store reaches the coherent cache/memory system. Committed stores can sit in a store queue/buffer while the core continues. Younger loads may execute early, but they must compare against older stores and either forward matching data, wait, or be replayed if speculation was wrong.

PROGRAM ORDER S1: store [A] = 42 L2: load r1 = [B] L3: load r2 = [A] OUT-OF-ORDER EXECUTION S1 decoded → allocate STORE QUEUE entry address A may become known before data 42, or vice versa ↓ S1 eventually retires architecturally ↓ store is now safe to make externally visible but may still wait in STORE BUFFER/QUEUE for cache port/coherence ownership L2 address B becomes ready compare B against older unresolved/known store addresses if definitely independent → issue load early L3 address A becomes ready while S1 is still buffered address match with older S1 ├── S1 data ready → STORE-TO-LOAD FORWARD 42 directly └── S1 data not ready → put L3 to sleep/retry later MEMORY DISAMBIGUATION SPECULATION younger load executes before every older store address is known ↓ later an older store resolves to SAME address ↓ ordering violation detected ↓ replay load / flush younger dependent work restore rename/speculation state as implementation requires STORE BUFFER DRAIN obtain cache-line coherence permission/ownership merge/write bytes into L1 cache or downstream memory system ↓ other harts can eventually observe store according to architecture memory model FENCE / release / acquire rules constrain when selected accesses may pass each other; they do not mean the CPU stops being internally out of order.

LSU structure/mechanism	Purpose
Load Queue (LDQ)	Tracks in-flight loads, addresses, ordering/replay metadata and sometimes observed coherence events.
Store Queue (STQ)	Tracks older/in-flight stores, their addresses/data and commit state before they drain to memory.
store buffer	General term for buffering stores after/around retirement so the core does not stall on every cache/coherence write.
store-to-load forwarding	Returns data from an older matching buffered store directly to a younger load without waiting for cache update.
memory disambiguation	Predicts/checks whether younger loads are independent of older unresolved stores.
replay	Re-executes a load or dependent work after discovering ordering/alias speculation was invalid.
store mask/dependency mask	Per-load metadata marking which older stores still need checking before the load is unquestionably safe.
cache-coherence probe	External observation that can make otherwise-hidden load reordering architecturally visible and require replay/order enforcement.
retired store	Store whose instruction can no longer be squashed architecturally, though its bytes may still be buffered before global visibility.
fence	ISA ordering operation constraining specified memory/I/O operations across the fence.

Your own load can see your own buffered store before anyone else can. That is why architectural memory models explicitly account for store-buffer forwarding: a later load to the same address may return the new value even while another core still sees the old globally visible value.

PUBLIC DOCSBOOM — Load/Store Unit

Excellent real OoO design: explicit LDQ/STQ, early load issue, address comparison against older stores, store-to-load forwarding, sleeping loads and expensive replay/flush on memory-ordering failures.

https://docs.boom-core.org/en/latest/sections/load-store-unit.html

OFFICIAL SPECRISC-V RVWMO explanatory material — store-buffer forwarding

Official explanatory material explicitly uses store buffers to justify store→load reordering and describes a load forwarding from a buffered store before that store becomes globally visible.

https://docs.riscv.org/reference/isa/unpriv/mm-eplan.html

Real-time scheduling changes the contract: FIFO/RR priorities and DEADLINE reservations can outrank ordinary fair tasks

Linux does not use one scheduling rule for every runnable thread. Normal interactive/server work is handled by the fair-scheduling class, but applications with explicit latency or periodic-computation requirements can request SCHED_FIFO, SCHED_RR or SCHED_DEADLINE. These policies are powerful because a runnable real-time/deadline task can preempt ordinary fair tasks; misconfigured code can therefore starve the rest of the machine.

NORMAL FAIR TASKS EEVDF/fair scheduler → share CPU according to fairness/weight SCHED_FIFO static RT priority 1..99 on Linux highest runnable RT priority wins same priority: run in queue order running task keeps CPU until it: • blocks / sleeps • is preempted by a higher RT priority • explicitly yields NO ordinary time slice SCHED_RR same static-priority ordering as FIFO + per-thread round-robin quantum among equal-priority RR peers SCHED_DEADLINE userspace supplies: runtime = CPU budget available per activation/period deadline = relative deadline period = replenishment period constraint: runtime ≤ deadline ≤ period ↓ kernel admission control ↓ EDF-style deadline selection + CBS bandwidth enforcement ↓ over-budget task is throttled/replenished rather than consuming CPU forever CLASS ORDER (conceptual user-visible direction) runnable DEADLINE work ↓ runnable FIFO/RR real-time work ↓ normal fair work PREEMPT_RT IS DIFFERENT: it changes kernel preemptibility/locking/IRQ behavior to reduce worst-case latency; it is not a replacement name for SCHED_FIFO/RR/DEADLINE.

Mechanism	What it means	Failure mode / caveat
SCHED_FIFO	Fixed-priority real-time policy with no normal time slice. Highest-priority runnable thread executes until block, preemption or yield.	A CPU-bound runaway thread can starve lower-priority work.
SCHED_RR	FIFO semantics plus a round-robin quantum among runnable threads at the same static priority.	Still outranks ordinary tasks; the quantum only arbitrates equal-priority RR peers.
SCHED_DEADLINE	Reservation specified as runtime/deadline/period; Linux uses EDF-style selection with Constant Bandwidth Server accounting.	Admission can fail when requested utilization is not schedulable; overruns are throttled.
static RT priority	Linux FIFO/RR priority from 1 (low RT) through 99 (high RT).	Do not hard-code portable assumptions; POSIX guarantees fewer distinct levels than Linux exposes.
RLIMIT_RTPRIO / CAP_SYS_NICE	Controls who may raise a thread into privileged real-time scheduling ranges.	Real-time policy is intentionally restricted because it can deny CPU to the rest of the system.
RT runtime throttling	Kernel-wide/group bandwidth controls reserve some CPU time so non-RT work can still make progress.	Disabling safeguards makes lockups from runaway RT loops easier.
PREEMPT_RT	Kernel configuration/patch set that makes more kernel execution preemptible and turns many interrupt/locking paths into schedulable contexts.	It improves latency determinism but does not automatically make an application correctly designed for real time.

Real-time does not mean “runs faster.” It means the scheduler provides a different ordering/budget contract intended to bound latency or reserve CPU service. A low-priority normal task can have excellent throughput; a real-time task is about when work runs, not how many instructions the CPU can execute.

OBSERVATION LAB # scheduling class, RT priority, CPU and state ps -eLo pid,tid,psr,cls,rtprio,pri,ni,stat,comm | less # show current process scheduling policy/priority chrt -p $$ # run a command under RR or FIFO (requires appropriate privilege/limits) chrt -r 20 command chrt -f 20 command # deadline uses sched_setattr()-style interfaces; inspect documentation first # system RT bandwidth controls cat /proc/sys/kernel/sched_rt_period_us cat /proc/sys/kernel/sched_rt_runtime_us # never test an unbounded high-priority busy loop on a machine you cannot recover.

PUBLIC MANUALsched(7) — Linux scheduling policies and priorities

Current manual covering normal policies, SCHED_FIFO, SCHED_RR, SCHED_DEADLINE, privileges, RT limits and the distinction between policy and priority.

https://man7.org/linux/man-pages/man7/sched.7.html

KERNEL DOCSLinux SCHED_DEADLINE — EDF + Constant Bandwidth Server

Kernel documentation for runtime/deadline/period parameters, replenishment, admission control and deadline scheduling behavior.

https://docs.kernel.org/scheduler/sched-deadline.html

KERNEL DOCSLinux real-time group scheduling and bandwidth controls

Explains RT period/runtime bandwidth, cgroup allocation and the safeguards used to keep real-time workloads from consuming all CPU service.

https://docs.kernel.org/scheduler/sched-rt-group.html

A mutex connects CPU atomics to the scheduler: the futex fast path and slow path

On Linux, a common mutex design keeps the lock word in ordinary shared user memory. The uncontended fast path can acquire/release it with atomic instructions entirely in userspace. Only when a thread must actually sleep or wake a sleeper does the kernel need to participate through a futex operation.

UNCONTENDED LOCK shared lock word = 0 # unlocked Thread A: atomic compare/exchange 0 → 1 ↓ success acquire ordering ↓ critical section No syscall. Kernel may not know this lock exists. CONTENDED LOCK Thread B tries atomic 0 → 1 ↓ fails because value says locked mark/observe contended state as implementation requires ↓ futex(FUTEX_WAIT, expected_locked_value) ↓ kernel atomically checks: does futex word STILL equal the expected value? ├── no → return immediately (state changed; retry in userspace) └── yes → enqueue waiter + block task ↓ scheduler removes B from runnable execution UNLOCK WITH WAITER Thread A publishes protected writes with release semantics ↓ changes lock word to unlocked / handoff state ↓ futex(FUTEX_WAKE, 1 or more) ↓ kernel marks waiter runnable ↓ scheduler eventually selects B on some allowed CPU ↓ B retries/acquires the user-space lock WHY FUTEX_WAIT COMPARES BEFORE SLEEPING B saw 'locked' A unlocks + wakes B has not slept yet ↓ if B blindly slept now, wakeup could be lost ↓ atomic compare-and-block prevents that race.

Layer/mechanism	What it contributes
atomic compare-and-exchange	Changes lock state only if it still has the expected value; fast uncontended ownership transition.
acquire/release ordering	Makes protected memory accesses obey the synchronization relationship around successful lock/unlock.
futex word	32-bit shared user-memory value used to connect userspace lock state to kernel wait/wake operations.
FUTEX_WAIT	Atomically compare the futex word with expected value and sleep only if it still matches.
FUTEX_WAKE	Make one or more tasks waiting on that futex address eligible to run again.
futex queue	Kernel bookkeeping that associates blocked waiters with a futex key/address.
scheduler	Removes blocked task from CPU eligibility and later chooses when/where a woken task runs.
priority-inheritance futex	Special futex/rt-mutex path intended to reduce priority inversion for PI mutexes.
robust mutex/futex	Provides recovery conventions for a thread dying while owning a lock.

A mutex is not inherently a syscall. The design goal is that the normal uncontended case stays in userspace, while the kernel is used for expensive scheduling actions such as blocking and waking contended threads.

LINUX OBSERVATION LAB # trace futex activity from a multithreaded program strace -f -e futex ./program # an uncontended lock may produce no futex syscall at all # contention/blocking usually makes futex waits/wakes visible # perf can show context switches while the threads contend perf stat -e context-switches,cpu-migrations ./program # compare: # 1 thread / uncontended lock # 2+ threads heavily contending on same lock # then correlate futex syscalls with context-switch growth.

PUBLIC MANUALfutex(2) — current Linux manual

Current manual explains the key design: synchronization is normally done in userspace; the syscall is used when a thread needs to block/wake. FUTEX_WAIT performs an atomic compare-and-block operation.

https://man7.org/linux/man-pages/man2/futex.2.html

PUBLIC MANUALFUTEX_WAIT(2const) — lost-wakeup protection

Current manual spells out why the expected-value comparison exists: the value check and going to sleep are atomic/ordered so a concurrent wake cannot be silently lost.

https://man7.org/linux/man-pages/man2/FUTEX_WAIT.2const.html

PUBLIC MANUALfutex(7) — futex semantics

Concise userspace-first overview of fast noncontended atomic operations and kernel FUTEX_WAKE use when waiters exist.

https://man7.org/linux/man-pages/man7/futex.7.html

PUBLIC MANUALpthreads(7) — Linux NPTL implementation

Current manual notes that Linux NPTL synchronization primitives such as mutexes and joins are implemented using futex operations underneath.

https://man7.org/linux/man-pages/man7/pthreads.7.html

PUBLIC MANUALpthread_mutex_lock(3p)

POSIX-visible mutex semantics and mutex types; useful to distinguish the API contract from Linux's futex-based implementation strategy.

https://man7.org/linux/man-pages/man3/pthread_mutex_lock.3p.html

A robust mutex can detect that its owner died, but the next owner must repair the protected state: robust list → OWNER_DIED → EOWNERDEAD → consistent

An ordinary futex-backed mutex is fast because the kernel may know nothing about an uncontended lock. That becomes a problem if a thread dies while holding a process-shared lock: there may be no kernel waiter queue from which to infer ownership. Linux robust futexes solve this by having each participating thread register a userspace robust list of locks it currently owns. On thread exit, the kernel can walk that list, mark owned futexes as having a dead owner, and wake a waiter.

ROBUST MUTEX SETUP pthread_mutexattr_setrobust(..., PTHREAD_MUTEX_ROBUST) ↓ pthread_mutex_init(...) ↓ thread/library registers its per-thread robust-list head with kernel OWNER ACQUIRES MUTEX userspace atomic fast path ↓ library links mutex into owner's robust list ↓ critical section mutates protected shared state OWNER DIES BEFORE UNLOCK exit path sees registered robust list ↓ kernel walks list carefully ↓ marks matching futex word FUTEX_OWNER_DIED ↓ wakes waiter if needed NEXT OWNER pthread_mutex_lock() acquires mutex ↓ returns EOWNERDEAD new owner now owns the mutex BUT protected data may be inconsistent ↓ application repairs/checks transaction state ↓ pthread_mutex_consistent() ↓ unlock normally IF RECOVERY IS IMPOSSIBLE unlock without making it consistent ↓ future lock attempts fail ENOTRECOVERABLE

State/API	Meaning
`PTHREAD_MUTEX_ROBUST`	Requests owner-death recovery semantics for the mutex.
robust list	Per-thread userspace linked list maintained by the threading library and registered with the kernel.
`FUTEX_OWNER_DIED`	Kernel-visible bit used during owner-death cleanup to mark a robust futex whose owning task exited.
`EOWNERDEAD`	Successful lock acquisition plus a warning: the previous owner died while holding the mutex, so the protected data must be checked/repaired.
`pthread_mutex_consistent()`	Declares that the new owner has restored the protected state to a usable invariant.
`ENOTRECOVERABLE`	Mutex was left inconsistent/unrecoverable; later lockers cannot simply continue.

Robustness repairs lock ownership, not your data structure. The kernel can tell the next thread that the former owner vanished; it cannot know whether that owner had updated half of a linked list, debit/credit pair or shared metadata transaction. The application must define a recovery invariant or decide that the protected object cannot be recovered.

KERNEL DOCSLinux robust futex documentation

Kernel explanation of ordinary futex locking plus recovery concerns when a thread dies while owning a userspace lock.

https://docs.kernel.org/locking/robust-futexes.html

KERNEL DOCSLinux robust futex ABI

Kernel ABI details for the per-thread robust list, task-exit processing and how userspace and kernel cooperate without adding a syscall to every uncontended lock operation.

https://docs.kernel.org/locking/robust-futex-ABI.html

PUBLIC MANUALpthread_mutexattr_setrobust(3)

Current Linux man-page semantics for PTHREAD_MUTEX_ROBUST, EOWNERDEAD and the transition to an unrecoverable mutex.

https://www.man7.org/linux/man-pages/man3/pthread_mutexattr_setrobust.3.html

PUBLIC MANUALpthread_mutex_consistent(3)

Explains how a new owner marks a robust mutex consistent after repairing state following an EOWNERDEAD acquisition.

https://man7.org/linux/man-pages/man3/pthread_mutex_consistent.3.html

A condition variable waits for a predicate without losing the mutex handoff: check → atomic unlock-and-sleep → wake → recheck

A mutex answers “who may inspect/change this shared state now?” A condition variable answers a different question: “when might the shared state have changed enough that I should check again?” The protected predicate lives in ordinary shared variables. pthread_cond_wait() atomically releases the associated mutex and blocks with respect to condition signaling, then reacquires the mutex before returning. Because wakeups may be spurious and several waiters may race for the same state, correct code tests the predicate in a loop.

CONSUMER / WAITER pthread_mutex_lock(&m) ↓ while (!predicate(shared_state)) { ↓ pthread_cond_wait(&cv, &m) ↓ conceptually atomically release m + become a cv waiter ↓ BLOCK — other threads can now acquire m ↓ signal / broadcast / spurious wakeup / cancellation semantics re-acquire m before returning ↓ loop re-checks predicate while holding m } consume/update shared_state pthread_mutex_unlock(&m) PRODUCER / SIGNALER pthread_mutex_lock(&m) modify shared_state so predicate may become true ↓ pthread_cond_signal(&cv) # wake at least one waiter candidate or pthread_cond_broadcast(&cv) # wake all waiters ↓ pthread_mutex_unlock(&m) WHY THE WAIT OPERATION MUST RELEASE+SLEEP ATOMICALLY bad design: waiter checks predicate=false waiter unlocks mutex producer sets predicate=true + signals waiter goes to sleep afterward ← signal already gone condition-variable wait closes that lost-wakeup window, but the predicate is still the source of truth.

Concept	Correct interpretation
predicate	Boolean condition over shared state protected by the mutex; this, not “a signal happened,” determines whether the thread may proceed.
`pthread_cond_wait()`	Requires the mutex locked, releases it as part of blocking, and returns with that mutex reacquired.
signal	Makes at least one waiter eligible to wake; it is not a persistent queued token that future waiters can consume.
broadcast	Wakes all current waiters, which then contend for the mutex and independently recheck the predicate.
spurious wakeup	A wait may return even though the application predicate is false; therefore use `while`, not `if`.
timed wait	Same predicate/mutex discipline with an absolute timeout; timeout racing with a state change still requires checking the predicate.
implementation	POSIX specifies semantics, not Linux's exact internal algorithm; glibc may use futex operations and sequence bookkeeping underneath.

# Watch a pthread program for blocking/wakeup syscalls strace -f -e futex ./condvar-demo # A useful experiment: # 1. one producer, one consumer # 2. several consumers + pthread_cond_signal() # 3. several consumers + pthread_cond_broadcast() # Observe that waking != immediately running: the mutex and scheduler still decide progress.

Never treat a condition variable as the state itself. If no thread is waiting, pthread_cond_signal() does not bank a future wakeup. Store the real event/state in protected memory, update it while following the mutex protocol, and always re-evaluate the predicate after waking.

POSIX STANDARDPOSIX.1-2024 — pthread_cond_wait / timedwait / clockwait

Current normative condition-variable semantics: the mutex release/block operation, mutex reacquisition, predicate rechecking and permitted spurious wakeups.

https://pubs.opengroup.org/onlinepubs/9799919799/functions/pthread_cond_clockwait.html

PUBLIC MANUALpthread_cond_wait(3) — Linux manual

Practical Linux/Pthreads overview of condition initialization, wait, signal and broadcast operations and their association with a mutex.

https://www.man7.org/linux/man-pages/man3/pthread_cond_wait.3.html

Kernel locking beyond mutexes: spinlocks, seqlocks and lockdep

A mutex is appropriate when the caller may sleep while waiting. A spinlock protects short critical sections where sleeping is forbidden or undesirable. A sequence counter/seqlock takes a different approach: readers do not lock at all, but retry if a writer changed the data while it was being copied. Linux's lockdep validator tracks lock classes and observed acquisition order to detect deadlock patterns and IRQ-context misuse.

SPINLOCK CPU0: spin_lock(&L) atomic acquire succeeds ↓ short critical section ↓ spin_unlock(&L) CPU1: spin_lock(&L) acquire fails ↓ spins / loops waiting for ownership ↓ must NOT sleep while holding or waiting under raw spin semantics SEQCOUNT / SEQLOCK READER do { seq = read_seqcount_begin(&seqc); # requires even/stable start local_a = shared.a; local_b = shared.b; } while (read_seqcount_retry(&seqc, seq)); WRITER serialize writers with appropriate lock/preemption rules write_seqcount_begin(&seqc); # sequence becomes odd shared.a = ...; shared.b = ...; write_seqcount_end(&seqc); # sequence becomes next even value If reader saw writer overlap, start/end sequence differs or is unstable → retry. LOCKDEP DEADLOCK EXAMPLE path 1: lock(A) → lock(B) path 2: lock(B) → lock(A) lockdep builds dependency graph: A → B B → A ↓ cycle means possible ABBA deadlock → warning/splat LOCKING ALSO ORDERS MEMORY unlock on CPU0 publishes protected writes later lock acquisition on CPU1 must observe the lock-protected state according to the kernel memory-model locking rules.

Primitive	Readers	Writers	Can sleep?	Best fit
mutex	lock	lock	Yes	Longer process-context critical sections and contended blocking.
spinlock_t	lock	lock	Normally no; PREEMPT_RT changes implementation semantics	Short shared-data critical sections, including IRQ-related synchronization.
raw_spinlock_t	lock	lock	No	Low-level scheduler/IRQ/timer code requiring traditional non-preemptible spin semantics.
seqcount_t	lockless retry	externally serialized	Writer must obey non-preemptibility/context rules	Read-mostly scalar snapshots such as timekeeping-style data.
seqlock_t	lockless retry or optional locking-reader path	embedded spinlock serializes	No on traditional writer path	Read-mostly data with cheap readers and serialized writers.
rwlock/rwsem	shared lock	exclusive lock	rwsem may sleep; raw/spin rwlock does not	Multiple readers need protected traversal rather than retry semantics.
RCU	very cheap read-side section	copy/publish + delayed reclamation	Flavor-dependent	Read-mostly pointer structures with version coexistence.

Sequence counters are not for arbitrary pointer graphs. A reader may be forced to retry after it already followed a pointer that the writer invalidated. Linux's seqlock documentation explicitly warns that plain sequence counters are unsuitable when protected data contains pointers that writers can invalidate.

KERNEL DOCSLinux sequence counters and sequential locks

Current reference for seqcount_t and seqlock_t: lockless retry readers, odd/even writer sequence transitions, writer serialization rules and pointer-lifetime caveats.

https://docs.kernel.org/locking/seqlock.html

KERNEL DOCSLinux kernel memory model — locking

Current locking-memory-order rule: acquiring a lock observes changes made before the previous release of that same lock.

https://docs.kernel.org/dev-tools/lkmm/docs/locking.html

KERNEL DOCSLinux lockdep design

Current runtime lock validator documentation: lock classes, IRQ-context usage state and dependency-order graph validation.

https://docs.kernel.org/locking/lockdep-design.html

KERNEL DOCSLinux kernel hacking — locking guide

Practical kernel locking guide covering spinlocks, IRQ context, lock ordering, deadlocks and the distinction between sleepable and atomic contexts.

https://docs.kernel.org/kernel-hacking/locking.html

Why spin_lock_irqsave() exists: the deadlock between process context and its own interrupt

If the same lock is used from process context and from a hard interrupt handler on the same CPU, plain locking can deadlock locally: the process holds the lock, an interrupt preempts it, and the interrupt spins forever waiting for a lock only the interrupted code can release. The classic non-RT solution is to disable local interrupts before acquiring that lock.

BROKEN ON A TRADITIONAL NON-RT KERNEL process on CPU2: spin_lock(&dev->lock) ↓ holds lock [hardware IRQ arrives on CPU2] ↓ CPU2 enters hardirq handler IRQ handler: spin_lock(&dev->lock) ↓ spins forever The owner cannot run because the interrupt preempted it. The interrupt cannot finish because it wants the owner's lock. CLASSIC FIX process context: unsigned long flags; spin_lock_irqsave(&dev->lock, flags); local IRQ delivery disabled on this CPU acquire lock critical section spin_unlock_irqrestore(&dev->lock, flags); hardirq context on that CPU cannot interrupt the protected process-side section. WHY 'irqsave' INSTEAD OF JUST 'irq' caller may itself already have IRQs disabled ↓ irqsave records previous interrupt-enable state irqrestore restores exactly that state on unlock SMP STILL MATTERS disabling LOCAL interrupts does not stop CPU3 ↓ the spinlock itself still excludes other CPUs SOFTIRQ/BOTTOM-HALF SHARING spin_lock_bh() disables local bottom-half processing while holding lock PREEMPT_RT CAVEAT most interrupts are threaded spinlock_t changes semantics and does not behave like classic raw spinning raw_spinlock_t remains the low-level primitive when actual IRQ-disabled/raw semantics are required.

Sharing pattern	Traditional non-RT tool
process context ↔ process context only	mutex if sleeping is acceptable; spinlock for truly short atomic sections.
process context ↔ hardirq on same data	spin_lock_irqsave()/irqrestore() around process-side critical section.
hardirq ↔ hardirq	spinlock; local IRQ state may already be disabled depending on path, but nested/source rules matter.
process ↔ softirq/bottom half	spin_lock_bh() or appropriate BH-disabling primitive.
NMI ↔ ordinary context	Requires specially NMI-safe primitives/design; ordinary spinlock use can still deadlock.
PREEMPT_RT ordinary driver lock	Use documented RT-safe spinlock/mutex patterns; don't mechanically assume non-RT atomic-context semantics.

pthread_create() builds a new execution context: stack, TLS, TID and shared process state

Linux POSIX threads are a 1:1 threading model: each userspace thread corresponds to a kernel scheduling entity, but threads in one process deliberately share most process-wide state. glibc's NPTL implementation allocates a stack/thread descriptor/TLS area, then enters the kernel with clone-family flags that request shared address space, file-descriptor table, signal dispositions and thread-group membership.

pthread_create(&thr, &attr, start_routine, arg) ↓ glibc/NPTL allocate/reuse THREAD STACK mapping guard region / stack size according to attributes allocate struct pthread / TCB-related state allocate/initialize TLS image for this thread __thread / _Thread_local variables get per-thread storage ↓ prepare clone arguments typical NPTL sharing flags include: CLONE_VM → same virtual address space / mm CLONE_FS → same cwd/root/umask state CLONE_FILES → same fd table CLONE_SIGHAND → same signal dispositions CLONE_THREAD → same thread group / TGID CLONE_SYSVSEM → shared SysV semaphore adjustment state CLONE_SETTLS → install new thread pointer/TLS base CLONE_PARENT_SETTID CLONE_CHILD_CLEARTID ↓ kernel creates task_struct + scheduler state + TID but points task at shared mm/files/sighand structures where flags requested sharing ↓ new thread begins on ITS OWN user stack with ITS OWN TLS/thread pointer and signal mask/scheduling state ↓ glibc start_thread trampoline runtime initialization call start_routine(arg) x86-64 TLS IDEA FS base → thread-control/TLS region compiler turns __thread variable access into FS-relative address calculation context switch restores each thread's thread-pointer state THREAD EXIT / JOIN thread returns from start_routine or calls pthread_exit() ↓ kernel exit clears child_tid location because CLONE_CHILD_CLEARTID ↓ kernel wakes futex waiters on that address ↓ pthread_join() waiting in userspace/glibc observes termination, collects return value and releases thread resources Threads share a process, but they do NOT share one stack or one TLS instance.

Thread object/state	Shared or per-thread?	Purpose
virtual address space (mm)	shared	All POSIX threads normally see the same mappings and ordinary process memory.
file-descriptor table	shared	open()/close()/dup() effects are visible across threads in the process.
signal dispositions	shared	sigaction() handler/default/ignore policy is process-wide.
signal mask	per-thread	Each thread can block a different set of signals.
user stack	per-thread	Independent call frames, local automatic variables and return-address chain.
TLS / thread pointer	per-thread	Implements __thread/_Thread_local variables and libc/thread runtime state.
TID	per-thread	Kernel-visible unique thread identifier.
TGID / getpid()	shared thread-group identity	All NPTL threads in one process report the same process ID/TGID.
scheduler state / affinity	per-thread	Each thread can be runnable, sleeping, scheduled and affinity-constrained independently.
errno	logically per-thread	libc typically implements errno using TLS so threads do not overwrite each other's error state.
pthread_t / struct pthread	thread-library identity	Userspace descriptor tracking stack, TLS, join/detach and runtime bookkeeping.

CLONE_THREAD alone does not define a POSIX thread. glibc deliberately combines several clone flags because POSIX requires a particular mixture of shared and private state. The userspace library also supplies stacks, TLS, cancellation, joining and other semantics the raw clone system call does not provide by itself.

LINUX THREAD/TLS LAB # see process TGID vs per-thread TIDs ps -T -p <PID> -o pid,tid,psr,stat,comm ls /proc/<PID>/task # trace thread creation and futex join/locking behavior strace -f -e clone,clone3,futex,exit,exit_group ./thread_program # inspect stacks/mappings cat /proc/<PID>/maps | grep -E 'stack|libc|ld-linux' # in C, print: # getpid(), syscall(SYS_gettid), pthread_self(), &errno, # and address of a _Thread_local variable from several threads. # getpid() should match; TID/TLS addresses should differ. # On x86-64, ARCH_GET_FS can expose FS base for experiments, # but DO NOT overwrite FS base in a glibc thread: libc itself relies on it.

SOURCE / FILEglibc NPTL pthread_create.c — current implementation

Real source showing NPTL's clone flags: CLONE_VM/FS/FILES/SIGHAND/THREAD/SETTLS/PARENT_SETTID/CHILD_CLEARTID, stack allocation and start_thread trampoline.

https://codebrowser.dev/glibc/glibc/nptl/pthread_create.c.html

CURRENT PUBLIC MANUALclone(2) — current man-pages 6.19

Current clone/clone3 reference. CLONE_VM shares memory, CLONE_FILES shares the fd table, CLONE_THREAD joins the same thread group, CLONE_SETTLS installs TLS, and CHILD_CLEARTID clears+wakes a futex on exit.

https://man7.org/linux/man-pages/man2/clone.2.html

CURRENT PUBLIC MANUALpthread_create(3) — current man-pages 6.19

Current API behavior plus Linux/NPTL stack-size defaults and joinable/detached thread semantics.

https://man7.org/linux/man-pages/man3/pthread_create.3.html

PUBLIC MANUALarch_prctl(2) — x86-64 FS/GS thread state

Current x86-64 interface for reading/setting FS/GS bases; explicitly warns that FS is normally owned by the threading library.

https://www.man7.org/linux/man-pages/man2/arch_prctl.2.html

PUBLIC MANUALset_thread_area(2) — architecture TLS support

Cross-architecture background on kernel support for thread-local-storage pointers/descriptors.

https://www.man7.org/linux/man-pages/man2/set_thread_area.2.html

Restartable sequences make tiny per-CPU userspace critical sections fast by letting the kernel abort them when scheduling invalidates the assumption

Per-CPU data is attractive because a thread can often update the data belonging to the CPU it is currently running on without contending on one global cache line. The hard part is migration: the scheduler can move the thread between CPUs. Linux restartable sequences (rseq) provide a per-thread kernel/userspace ABI that marks a small userspace instruction sequence as restartable. If preemption, migration or a relevant signal occurs at the wrong point, the kernel redirects execution to an abort path instead of allowing the sequence to commit under a stale CPU assumption.

thread has an rseq area registered with the kernel ↓ read current CPU / choose that CPU's userspace data ↓ enter registered rseq critical section ↓ perform small update ↓ commit instruction / point of no return IF NO DISRUPTION: update commits on the intended CPU IF scheduler migration / preemption / relevant interruption happens while execution is in the restartable region: ↓ kernel recognizes the registered critical section ↓ resume at abort handler instead of the interrupted instruction ↓ retry with fresh CPU-dependent state

Technique	Cost / purpose
mutex	General mutual exclusion; can involve atomics, cache-line contention and kernel blocking under contention.
atomic read-modify-write	Useful for shared counters/state but still participates in cache coherence and can contend across CPUs.
rseq	Optimizes carefully structured per-CPU userspace operations by turning migration/preemption into abort-and-retry rather than requiring a heavyweight atomic protocol.
`sched_getcpu()`	Reports the CPU at an instant; by itself it does not make a later multi-instruction CPU-local update migration-safe.

rseq is not a general transaction system. The critical section must obey the ABI's strict structure, remain very small, and tolerate restart. Its strength is exactly the narrow case where libraries/allocators/runtime code want extremely cheap per-CPU operations.

KERNEL DOCSLinux restartable sequences userspace API

Current kernel documentation describing the per-thread rseq ABI, userspace restartable sequences, fast CPU/node identification and newer scheduler-related rseq features.

https://kernel.org/doc/html/latest/userspace-api/rseq.html

PUBLIC MANUALsched_getcpu(3) — current CPU query

Useful contrast: querying the current CPU is easy, but the value can become stale when the scheduler migrates the thread; rseq provides the abort/restart machinery needed around selected CPU-local critical sections.

https://www.man7.org/linux/man-pages/man3/sched_getcpu.3.html

Compiler ordering and CPU ordering are separate layers: volatile is not an atomic or a memory barrier

A source program passes through two independent reorderers: the compiler can move/eliminate/combine operations while preserving language semantics, and the CPU can execute/load/store speculatively or out of order while preserving its architectural memory model. Correct concurrent code therefore needs language-level synchronization that the compiler maps to appropriate machine instructions/fences for the target architecture.

SOURCE CODE data = 42; ready = 1; Without a synchronization contract: compiler may reorder / register-promote / remove accesses ↓ machine instructions ↓ CPU store buffer / speculation / memory system may expose effects in architecture-permitted order ↓ other thread/device CORRECT THREAD-TO-THREAD IDEA producer: ordinary writes to data atomic_store_explicit(&ready, 1, release) consumer: if (atomic_load_explicit(&ready, acquire)) read data compiler knows acquire/release semantics + backend selects target-appropriate instructions/fences ↓ language happens-before contract becomes machine ordering VOLATILE forces/retains accesses according to language/compiler rules for volatile objects but is NOT, by itself, a portable inter-thread atomic or general memory barrier.

Mechanism	Compiler constraint?	CPU/hardware ordering?	Atomicity?
ordinary C/C++ load/store	Only language semantics	Target/compiler decides within model	No inter-thread atomic guarantee
volatile access	Preserves required volatile access behavior	Not a general portable hardware fence	No
compiler barrier	Stops selected compiler motion	No hardware ordering by itself	No
C/C++ atomic relaxed	Yes; atomic object semantics	Atomic operation, minimal inter-thread ordering	Yes for that atomic operation
acquire/release atomic	Yes	Maps to needed target ordering	Yes
seq_cst atomic/fence	Strong language ordering	Typically strongest required mapping for target	Atomic operation if atomic access
kernel smp_* barrier	Compiler + architecture implementation	Yes according to kernel memory model	Barrier itself is not an RMW
MMIO accessor	Prevents inappropriate compiler treatment and uses architecture/device-I/O semantics	Yes according to accessor contract	Not general shared-RAM atomicity

Use volatile for the problems it actually solves. GCC explicitly states that non-volatile memory accesses are not ordered with respect to volatile accesses, so a volatile object cannot be used as a generic memory barrier. Linux likewise warns that volatile does not replace locks, atomics or memory barriers for shared data.

COMPILER DOCSGCC — When is a Volatile Object Accessed?

Current GCC documentation explicitly states that non-volatile accesses are not ordered with volatile accesses and volatile cannot be used as a memory barrier for ordinary memory.

https://gcc.gnu.org/onlinedocs/gcc/Volatiles.html

COMPILER DOCSGCC — __atomic builtins

Documents relaxed, acquire, release, acquire-release and sequentially-consistent atomic orders and how GCC maps atomic operations to target hardware or library routines.

https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html

KERNEL DOCSLinux — Why volatile should not be used for synchronization

Clear kernel explanation: volatile suppresses some optimization but does not provide the synchronization guarantees locks, atomics and memory barriers provide.

https://docs.kernel.org/next/process/volatile-considered-harmful.html

COMPILER DOCSGCC — optimization options

Useful proof that the compiler actively rewrites/reorders code: block/function reordering, propagation, dead-code removal, store-forwarding avoidance and many other transformations are documented here.

https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html

Five Ripes experiments that make CPU architecture visible

Ripes is useful because you can compare multiple microarchitectures while keeping the same RISC-V program. That lets you separate ISA behavior from implementation behavior.

Run a three- or four-instruction arithmetic sequence on a single-cycle core. Trace PC → instruction memory → register file → ALU → write-back and note that one instruction occupies the whole datapath per clock.
Run the same sequence on a five-stage pipeline. Watch different instructions simultaneously occupy IF, ID, EX, MEM and WB.
Create a RAW dependency such as ADD followed immediately by an instruction using its result. Compare a processor without forwarding to one with forwarding/hazard detection; watch stalls or bypass paths appear.
Use a taken branch. Observe when the branch resolves, which younger instruction(s) entered the pipeline incorrectly, and what flush/recovery does.
Enable the cache simulator. Access an array repeatedly, then alter cache size/line size/associativity. Watch address bits split into tag/index/offset and correlate misses with execution time.

Same RISC-V instructions │ ├── single-cycle datapath → 1 long clock / instruction ├── 5-stage pipeline → overlap instructions └── dual-issue model → potentially >1 retired instruction/cycle Architectural result should match. Cycles, hazards, forwarding, cache behavior and throughput can differ.

PUBLIC DOCSRipes processor-model documentation

The project documentation explicitly describes visual single-cycle and five-stage processor models, their wire layouts, validation and how architecture state is compared between implementations.

https://github.com/mortbopet/Ripes/blob/master/docs/new_processor_models.md

PUBLIC DOCSRipes command-line pipeline reporting

Public docs showing reports for cycles, instructions retired, CPI, IPC, pipeline state and register values—useful after visual experiments.

https://github.com/mortbopet/Ripes/blob/master/docs/cli.md

Classic pipeline hazards: forwarding fixes many RAW dependencies, but not every one

A five-stage pipeline overlaps instructions: IF → ID → EX → MEM → WB. That creates hazards whenever the next instruction needs information that has not reached the place it normally reads from yet. The important mechanisms are hazard detection, forwarding/bypassing, stalls/bubbles and flushes.

5-STAGE PIPELINE IF = fetch ID = decode/register read EX = ALU/address MEM = data memory WB = register write-back RAW DEPENDENCY THAT FORWARDING CAN FIX I1: add x5, x6, x7 I2: sub x8, x5, x9 cycle: 1 2 3 4 5 6 I1 IF ID EX MEM WB I2 IF ID EX MEM WB I2 needs x5 in its EX stage during cycle 4. I1's ADD result already exists after EX in cycle 3, but has not reached WB. FORWARD/BYPASS EX/MEM result directly into I2's ALU input. LOAD-USE HAZARD I1: lw x5, 0(x6) I2: add x8, x5, x9 I1 load data normally appears only after MEM. I2 wants it in EX one cycle too early. Typical simple pipeline: detect rd(I1)==rs1/rs2(I2) and I1 is load → hold PC/IF-ID state → inject bubble/stall → forward loaded value when it finally exists CONTROL HAZARD branch predicted/fetched path turns out wrong → younger wrong-path instructions are FLUSHED → fetch restarts from correct PC

Hazard	Example	Typical remedy
RAW data hazard	instruction reads a register an older in-flight instruction will write	Forward result from later pipeline register; stall if data does not exist soon enough.
load-use hazard	consumer immediately follows a load of its operand	Usually at least one bubble in a simple 5-stage design because load data arrives after memory access.
control hazard	branch/jump changes next PC after younger instructions were fetched	Prediction plus flush/recovery; simpler CPUs may stall until target/decision is known.
structural hazard	two stages need the same non-multiported hardware resource in one cycle	Duplicate/port resource, arbitrate, or stall.
WAW/WAR name hazard	out-of-order execution lets younger operations overtake older ones	Register renaming in modern OoO designs; not normally present in simple in-order 5-stage pipelines.

Forwarding does not make data exist earlier. It removes the delay of waiting for normal register-file write-back when a result already exists elsewhere in the pipeline. If the producer has not actually computed/received the value yet—as in a classic load-use case—the consumer must still wait.

PUBLIC NOTESCornell CS3410 — pipelining and hazards

Public notes introducing pipelining, stage registers, latency/throughput and control hazards in a RISC-V-style five-stage datapath.

https://www.cs.cornell.edu/courses/cs3410/2025fa/notes/pipelining.html

PUBLIC PROJECTCornell Mini-RISC project — explicit forwarding and load-stall conditions

Concrete no-login design notes deriving EX/MEM and MEM/WB forwarding conditions and explaining why a load in the memory stage still requires a stall.

https://www.cs.cornell.edu/courses/cs3410/2019sp/projects/mini-risc/

Modern CPU front ends: instruction bytes, µops, decoded caches and speculative supply

An ISA instruction is the architecturally visible unit, but a modern implementation may internally translate it into one or more micro-operations (µops). The front end's job is to predict where execution is going, fetch instruction bytes, find instruction boundaries, decode them, and continuously feed µops into the out-of-order engine fast enough that execution units do not starve.

architectural instruction stream ↓ branch predictor predicts next fetch address ↓ L1 instruction cache supplies instruction BYTES ↓ predecode / instruction-boundary detection (especially hard for variable-length x86) ↓ decoders translate instructions → one or more µops │ ├── simple instructions → normal decoders ├── complex instructions → microcode sequencer / ROM-like µop flow └── hot already-decoded code → decoded-µop cache on implementations that have one ↓ µop queue ↓ rename architectural registers → physical registers/tags ↓ allocate reorder/load/store/scheduler resources ↓ issue ready µops to execution ports ↓ ALU / branch / load / store / multiply / vector / FP execution ↓ write result / wake dependents ↓ retire architectural instructions in the required order

Structure	What bottleneck it addresses
instruction cache	Avoids repeatedly fetching hot code bytes from slower cache/memory levels.
branch predictor / BTB	Chooses likely next control-flow target before branch execution resolves.
decoder	Turns ISA instruction encoding into internal operations/control.
microcode sequencer	Produces longer internal operation sequences for complex architectural instructions.
decoded µop cache	Avoids re-decoding hot instructions by caching their already-decoded µops.
rename map	Removes false register-name dependencies by assigning physical registers.
scheduler / reservation station	Holds µops until dependencies and execution resources are ready.
load queue	Tracks in-flight loads, ordering, forwarding/replay and memory-dependency constraints.
store queue/buffer	Tracks pending stores and can decouple retirement from later cache/memory visibility.
reorder buffer	Tracks speculative in-flight work and permits precise ordered retirement/recovery.

Do not universalize Intel names. Terms such as DSB/Decoded ICache, MITE, MSROM, ROB and particular queue sizes are implementation-specific. The general ideas—fetch/decode, prediction, internal operations, dependency tracking, scheduling, speculation and ordered architectural commitment—apply much more broadly.

DIRECT PDFIntel Optimization Manual Volume 2 — decoded µop cache and microcode structures (direct PDF)

Public microarchitecture tables identify instruction cache, legacy decode pipeline, decoded instruction/µop cache, MSROM, branch prediction and µop queue structures on documented Intel cores.

https://cdrdv2-public.intel.com/821614/356477-Software-Optimization-Manual_V2.pdf

OFFICIAL ARTICLEIntel — Decoded ICache / µops explanation

Public technical article with a concise diagram and explanation: legacy decoder emits µops, the decoded instruction cache stores already-decoded µops, and the µop queue feeds execution.

https://www.intel.com/content/www/us/en/developer/articles/technical/enhance-vm-workloads-performance-with-pgo.html

DIRECT PDFMIT 6.823 — Out-of-Order Execution & Register Renaming (direct PDF)

Still one of the best no-login architecture references for the generic machinery behind rename, scheduling, speculative execution and precise retirement.

https://ocw.mit.edu/courses/6-823-computer-system-architecture-fall-2005/a984df22afeb4bd732058005861a70a4_l12_ooo_pipes.pdf

Branch predictors are hardware state machines: counters, BTBs, history, RAS, GShare and TAGE

A branch predictor tries to answer two different questions early: will control flow redirect? and where will it go? Direction predictors estimate taken/not-taken behavior; a Branch Target Buffer (BTB) caches likely target addresses; a Return Address Stack (RAS) specializes in function returns.

2-BIT SATURATING COUNTER FOR ONE CONDITIONAL BRANCH 00 Strongly Not Taken 01 Weakly Not Taken 10 Weakly Taken 11 Strongly Taken actual TAKEN → increment toward 11 actual NOT TAKEN → decrement toward 00 prediction uses high bit: 0x → predict NOT TAKEN 1x → predict TAKEN Why 2 bits? A single unusual outcome does not immediately reverse a strongly learned prediction. BTB fetch PC → tag/index lookup → predicted branch/jump TARGET RAS CALL: push predicted return PC RET: pop predicted return PC GSHARE index = hash(PC, Global History Register), often conceptually PC XOR GHR index → table of saturating prediction counters TAGE PC + different lengths of branch history ↓ ↓ ↓ ↓ tagged predictor tables with geometrically increasing history lengths ↓ longest-history matching useful entry wins branch executes later ├── prediction correct → continue + train predictor └── wrong → redirect fetch, squash younger work, repair speculative history/snapshots, train predictor

Predictor structure	Predicts / stores
bimodal counter table	Per-index taken/not-taken tendency, often with 2-bit saturating counters.
BTB	Whether an instruction is a known control-flow site and its predicted target address.
RAS	Likely return addresses for nested CALL/RET behavior.
global history register	Recent taken/not-taken outcomes used to correlate current branch with earlier branches.
GShare	Combines PC and global history to index a direction-prediction counter table.
TAGE table	Tagged prediction entries indexed using progressively longer folded branch histories.
usefulness counter	TAGE-like metadata indicating whether an entry is valuable enough to retain.
predictor snapshot/checkpoint	Speculative predictor/history state needed so misprediction recovery can restore a known-correct history context.
FTQ / prediction metadata queue	Tracks prediction information alongside in-flight fetch blocks/instructions until resolution/commit.

A BTB and a direction predictor are not the same thing. A conditional branch can be predicted 'taken' but still need a target address. Conversely, a BTB hit can tell the front end a likely target without independently proving that the conditional branch should be taken.

PUBLIC DOCSBOOM — Branch Prediction

Open RISC-V superscalar-core documentation. BOOM uses a fast Next-Line Predictor plus a more complex Backing Predictor and explains prediction, update and recovery.

https://docs.boom-core.org/en/latest/sections/branch-prediction/index.html

PUBLIC DOCSBOOM — Next-Line Predictor: BTB + bimodal table + RAS

Concrete hardware description of a fast front-end predictor combining a fully associative BTB, bimodal table and return-address stack.

https://docs.boom-core.org/en/latest/sections/branch-prediction/nl-predictor.html

PUBLIC DOCSBOOM — Backing Predictor: two-bit counters, GShare and TAGE

Deep implementation-oriented explanation of GShare and TAGE. The TAGE section describes tagged tables with geometrically increasing history lengths, usefulness counters and recovery snapshots.

https://docs.boom-core.org/en/latest/sections/branch-prediction/backing-predictor.html

PUBLIC DOCSBOOM — branch-prediction terminology

Concise definitions of BTB, RAS, global-history register, GShare, TAGE, FTQ and prediction snapshots in one real implementation.

https://docs.boom-core.org/en/latest/sections/terminology.html

DIRECT PDFMIT 6.823 — branch prediction (direct PDF)

No-login architecture lecture covering branch prediction as a performance mechanism; useful conceptual companion to BOOM's concrete implementation.

https://ocw.mit.edu/courses/6-823-computer-system-architecture-fall-2005/3993f2698825866156870dc6196825f2_l13_brnchpred.pdf

Speculation can be squashed architecturally yet still leave microarchitectural footprints

Out-of-order processors predict future control/data behavior so they can do useful work before all older conditions are resolved. If a prediction is wrong, speculative instructions are squashed: their architectural register/memory results are not committed. But speculative execution can still perturb microarchitectural structures such as caches, TLBs, predictors and shared execution resources.

architectural program: if (index < array_length) value = array[index]; CPU predicts branch before comparison fully resolves ↓ predicted path may execute transiently/speculatively ↓ loads can touch cache/TLB/predictor/resource state ↓ bounds check eventually resolves FALSE ↓ speculative instructions SQUASHED architectural registers/memory do not commit their wrong-path results BUT: cache fill / predictor update / resource timing may remain measurable ↓ microarchitectural side channel can reveal information Architectural state: what the ISA promises software can observe directly Microarchitectural state: caches, predictors, queues, TLBs, replacement state, execution contention and implementation details

Mechanism	Performance purpose	Security relevance
conditional branch prediction	Keep fetch/decode/execution busy before branch resolves.	Wrong-path transient loads may change cache state.
indirect branch prediction	Predict CALL/JMP targets early.	Mistraining can steer transient execution toward unintended gadgets.
return-stack buffer	Predict RET targets efficiently.	Shared/history state may require mitigation around privilege/domain changes.
store-to-load speculation	Allow younger loads to proceed before all older store relationships are known.	Speculative stale-value consumption can create side channels on affected designs.
speculation barrier/control	Intentionally restrict speculative execution in sensitive sequences/domains.	Mitigates classes of transient-execution attack, often with performance cost.
cache timing	Not a security feature; cache exists for performance.	Latency difference between hit/miss can reveal whether prior activity touched a cache line.

Precise retirement protects architectural correctness, not secrecy of every internal timing effect. This distinction is why an instruction that never retires can still matter to a side-channel observer.

OFFICIAL ARTICLEIntel — Analysis of Speculative Execution Side Channels

Explains the crucial distinction: speculative operations can be discarded architecturally while still modifying microarchitectural state such as caches, TLBs, predictors and prefetchers.

https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/analysis-speculative-execution-side-channels.html

OFFICIAL DOCSIntel — Speculative Execution Side Channel Mitigations

Technical description of IBRS, STIBP, IBPB, LFENCE-based bounds-check bypass mitigation and speculative-store-bypass controls.

https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/speculative-execution-side-channel-mitigations.html

KERNEL DOCSLinux — Spectre vulnerability/mitigation documentation

Public kernel documentation describing Spectre variants, branch-predictor effects and OS/compiler/hardware mitigations on Linux.

https://docs.kernel.org/admin-guide/hw-vuln/spectre.html

Inside a modern CPU core: prediction, renaming, out-of-order execution and retirement

A simple CPU executes one instruction after another in an obvious sequence. A modern superscalar core usually tries to keep many instructions in flight at once. The architectural illusion is still a sequential instruction stream, but internally the core predicts future control flow, renames registers, wakes instructions when operands are ready, executes them on different functional units, and then commits results in a controlled order.

instruction-cache / fetch ↓ branch predictor guesses next PC ↓ decode instructions / split into internal operations ↓ register renaming maps ISA register names → physical storage/tags ↓ dispatch into issue queues / reorder structures ↓ ready instructions issue when operands + functional unit are available ↓ ALU / load-store / multiply / vector / other units execute OUT OF ORDER ↓ results wake dependent instructions ↓ retirement / commit makes completed work architecturally visible in order ↓ branch mispredict / exception? discard younger speculative work and recover

Why register renaming exists: the ISA may name only a small fixed set of registers, but the processor can maintain many more internal physical registers. Renaming eliminates false name dependencies such as WAR and WAW hazards while preserving true RAW data dependencies.

DIRECT PDFMIT 6.823 — Advanced Superscalar Architectures (direct PDF)

Direct public PDF covering register management, issue queues, memory dependencies, speculative loads/stores, load paths and branch-mispredict recovery.

https://ocw.mit.edu/courses/6-823-computer-system-architecture-fall-2005/de5d7f798f868802112e1b6cdbc454cc_l14_superscalar.pdf

OPEN NOTESMIT 6.004 — simplified modern out-of-order processor

Public annotated slides with a compact modern-core diagram: multiple-wide fetch/decode, branch prediction, register renaming, queues, functional units and out-of-order execution.

https://ocw.mit.edu/courses/6-004-computation-structures-spring-2017/pages/c21/c21s1/

PUBLIC PDF INDEXCMU 15-740 architecture schedule and direct lecture PDFs

Plain public page linking lecture PDFs on pipelining, out-of-order execution, branch prediction, caches, virtual memory, coherence, vectors/GPUs and accelerators.

https://www.cs.cmu.edu/afs/cs/academic/class/15740-s18/www/schedule.html

SMT / Hyper-Threading: two architectural threads sharing one physical core

Simultaneous multithreading (SMT) does not create a second full physical core. The processor exposes multiple architectural/logical CPU contexts that share substantial execution resources inside one core. If one thread is stalled on a cache miss, branch recovery, dependency or under-used execution mix, the other may use otherwise idle resources.

ONE PHYSICAL CORE WITH 2-WAY SMT logical CPU 0 architectural state logical CPU 1 architectural state PC / GPRs / control state / interrupts PC / GPRs / control state / interrupts \ / \ / shared front end / predictors ↓ shared rename / queues ↓ shared execution resources ALUs / load-store / vector / FP etc. ↓ shared portions of caches ↓ core ↔ uncore / memory Two logical CPUs can make forward progress in the same cycle, but they compete for resources that a second physical core would have separately.

Thing	Separate per SMT thread?	Shared in some substantial form?
architectural registers / PC	Yes	No
interrupt/APIC architectural context	Logically yes	Core/platform delivery resources still interact
execution ports / ALUs / load-store machinery	No	Yes
front end / decode bandwidth	Not fully separate	Yes
some predictor/cache structures	Implementation dependent	Often shared or partitioned
power/thermal budget	No	Yes

Why SMT speedup varies: two threads help most when they use complementary resources or one frequently stalls. If both saturate the same execution units, cache capacity, memory bandwidth or power limit, SMT can provide little gain or even hurt a particular workload.

OFFICIAL GUIDEIntel Hyper-Threading architecture overview

Public Intel technical guide with diagrams contrasting separate physical processors and two logical processors sharing one core's execution engine, caches and system interface.

https://www.intel.com/content/www/us/en/developer/articles/guide/hyper-thread-tuning-guide-for-video-ai-workload.html

OFFICIAL EXPLAINERIntel — what Hyper-Threading means

Simpler official explanation of logical threads versus physical CPU cores. Useful as a quick first read before the optimization manual.

https://www.intel.com/content/www/us/en/gaming/resources/hyper-threading.html

GPU architecture: why it looks different from a CPU

A GPU is still a computer: instructions, registers, ALUs, caches, memory controllers and schedulers. The architectural emphasis is different. CPUs spend large amounts of silicon making a few instruction streams fast and low-latency; GPUs devote much more hardware to throughput across many parallel threads and hide latency by keeping many warps/wavefronts ready.

CPU host program ↓ submits work / command buffers PCIe / coherent system interconnect ↓ GPU front end / work distribution ↓ many Streaming Multiprocessors (SMs) / compute units ├── warp schedulers ├── large register files ├── integer / floating-point execution units ├── load-store units └── local/shared-memory + L1 structures ↓ shared L2 / on-chip interconnect ↓ memory controllers → GDDR/HBM device memory A 'warp' in CUDA = 32 threads grouped for SIMT execution.

CPU bias	GPU bias
Large sophisticated cores optimized for low single-thread latency	Many throughput-oriented execution resources
Aggressive branch prediction and out-of-order machinery	Massive thread-level parallelism and fast switching among ready warps
Relatively few hardware threads per core	Many resident threads/warps used to cover long execution/memory latency
Large general-purpose cache hierarchy per small number of cores	Huge register files plus shared/local memory and caches feeding many parallel lanes

OFFICIAL DOCSCUDA Programming Guide — current public documentation

Official no-login documentation. The programming model and advanced execution sections explain thread blocks, 32-thread warps, SIMT, warp divergence, hardware multithreading and the GPU memory hierarchy.

https://docs.nvidia.com/cuda/cuda-programming-guide/

DIRECT PDFCUDA Programming Guide (direct PDF)

Same official material as a direct PDF; useful as a durable long-form reference.

https://docs.nvidia.com/cuda/cuda-programming-guide/pdf/cuda-programming-guide.pdf

DIRECT PDFNVIDIA Ampere A100 architecture white paper (direct PDF)

Excellent block diagrams of a real GPU Streaming Multiprocessor: warp scheduler, dispatch unit, register files, integer/FP units, tensor units, load/store units and L1/shared memory.

https://images.nvidia.com/aem-dam/en-zz/Solutions/data-center/nvidia-ampere-architecture-whitepaper.pdf

DIRECT PDFNVIDIA Turing architecture white paper (direct PDF)

Detailed public GPU architecture reference showing GPC/TPC/SM hierarchy, caches, register files, execution units, memory controllers and graphics-specific hardware.

https://images.nvidia.com/aem-dam/Solutions/design-visualization/technologies/turing-architecture/NVIDIA-Turing-Architecture-Whitepaper.pdf

A GPU job is queued work plus dependencies: command submission, hardware rings and dma_fence completion

Modern graphics/compute APIs do not usually make the CPU wait for each draw or kernel. Userspace builds command buffers and submits them to a kernel driver, which validates/locks the referenced buffer objects, records synchronization dependencies, places a job on a scheduler entity, and eventually emits that job to a hardware queue/ring. Completion is represented by a fence, often signaled from an interrupt or firmware-completion path.

USER MODE DRIVER / GRAPHICS API record commands into command buffer(s) draw / dispatch / copy / barriers ↓ buffer objects referenced: command buffer shader code vertex/index data textures/images render target / output buffer ↓ submit ioctl / driver-specific exec API KERNEL DRIVER lock/validate GEM/TTM buffer objects ensure objects are resident / GPU-addressable resolve implicit and/or explicit synchronization dependencies reserve/install output fence slots ↓ create drm_sched_job attach to one drm_sched_entity (client/context queue) ↓ push job into software scheduler runqueue DRM GPU SCHEDULER wait until dependency fences are signaled respect priority / scheduler credit limits ↓ run_job() ↓ driver emits command-stream start/ring entries ↓ MMIO doorbell / firmware queue / hardware ring tail update GPU HARDWARE fetches command packets from GPU virtual/physical memory executes shader/compute/copy work writes output buffers through GPU memory hierarchy ↓ writes completion/fence value or raises completion interrupt KERNEL COMPLETION PATH IRQ / firmware event / polling notices job complete ↓ dma_fence_signal(hardware_fence) ↓ scheduler's finished fence becomes signaled ↓ wake fence waiters / unblock dependent jobs ↓ userspace can poll/wait on sync_file, syncobj, driver fd, or API primitive HANG/TIMEOUT scheduler timeout expires before fence signals ↓ driver timeout recovery stops scheduling / resets engine or GPU ↓ pending fences receive error status so waiters are not stuck forever

GPU/DRM object	Role
command buffer	GPU-readable memory containing packets/instructions that tell a specific engine what work to perform.
GEM/TTM buffer object	Kernel-managed GPU memory object backing commands, shaders, images and other resources.
GPU virtual address	Address in the GPU's own MMU context at which a buffer object is mapped.
drm_sched_entity	Per-client/context software queue feeding one or more DRM GPU schedulers.
drm_sched_job	Kernel software representation of one schedulable GPU submission.
hardware ring/queue	Device-consumed command queue, often in memory with head/tail pointers or firmware-managed submission.
doorbell	MMIO or memory notification telling the device/firmware that new queue work is available.
dma_fence	Kernel asynchronous completion primitive signaled when hardware work reaches its completion point.
dma_resv	Reservation object collecting read/write fences associated with a shared buffer for implicit synchronization.
sync_file	Userspace file-descriptor wrapper around a fence for explicit synchronization.
drm_syncobj	DRM synchronization object whose underlying fence can be replaced/advanced over time.
scheduler credit	Current DRM scheduler flow-control unit limiting how much work can be in flight on a scheduler.
hang recovery	Driver-specific timeout/reset path that restores progress and reports errors on affected fences/contexts.

A fence is a completion fact, not the work itself. The command buffer tells hardware what to execute; the fence only says whether a particular asynchronous point has completed. This separation is what allows later GPU jobs, display operations or another device sharing a dma-buf to depend on prior work without forcing the CPU to busy-wait.

KERNEL DOCSLinux DRM GPU scheduler

Current scheduler documentation: userspace-facing entities queue jobs to software runqueues, dependencies are tracked, jobs are emitted to hardware runqueues, and a hardware fence drives finished-fence completion.

https://docs.kernel.org/gpu/drm-mm.html

KERNEL DOCSBroadcom V3D DRM driver — concrete scheduler example

A readable real driver example: each DRM file gets a scheduler entity, jobs are serialized onto hardware queues, and completion interrupts signal the corresponding fence.

https://docs.kernel.org/gpu/v3d.html

KERNEL DOCSAMDGPU ring-buffer documentation

Concrete current GPU-ring material showing jobs, fences, cache flushes and hardware submission isolation/debug behavior.

https://docs.kernel.org/gpu/amdgpu/ring-buffer.html

GPU memory is managed as movable buffer objects: GEM/TTM handles → VRAM or system-memory placement → GPU virtual mappings → eviction/rebind

A GPU application usually does not program a physical VRAM address directly. The DRM driver represents commands, textures, render targets and other allocations as buffer objects (BOs). GEM supplies common object/handle/mmap infrastructure; drivers for devices with dedicated memory often use TTM to manage placement and eviction. A BO may live in system RAM or device VRAM, be mapped into one or more GPU virtual-address spaces, move when memory pressure requires eviction, and carry synchronization metadata so movement or reuse does not race outstanding GPU work.

USER MODE DRIVER / API create image/buffer ↓ DRM driver creates BUFFER OBJECT (BO) userspace receives per-file handle ↓ choose/validate backing placement ├── system memory └── device-local VRAM (when available) ↓ map object into GPU virtual address space GPU VA → GPU page tables / GTT → system RAM or VRAM ↓ command buffer references GPU virtual address ↓ GPU job executes against BO ↓ pressure / placement requirement changes ↓ TTM/driver may EVICT or MOVE BO VRAM ↔ system memory / temporary placement ↓ wait/order against dma_resv/dma_fence dependencies ↓ update/rebind GPU virtual mappings before next use A GEM handle names a kernel object for one DRM file; it is not the physical address of the pixels.

Graphics-memory concept	Meaning
buffer object (BO)	Kernel-managed allocation representing GPU-accessible data such as commands, textures, vertex data or render targets.
GEM	DRM Graphics Execution Manager infrastructure for object lifetime, per-file handles, mmap and common driver helpers.
TTM	Translation Table Manager used by many DRM drivers to manage placement, movement and eviction across memory regions.
VRAM	Device-local graphics memory on discrete GPUs; often higher-bandwidth for the GPU but distinct from ordinary CPU RAM.
GPU virtual address	Address used by GPU commands after the BO is bound into a GPU MMU/address-space mapping.
eviction	Moving/removing an object from a preferred region so another allocation can use that capacity; later use may require validation/rebind.
`dma_resv` / fence	Synchronization state used to order asynchronous users of a BO and prevent movement/reuse while device work still depends on it.

“VRAM allocation,” “GEM object” and “GPU virtual mapping” are three different things. The object is the lifetime/identity container; placement says where backing memory currently resides; the GPU virtual mapping says where commands address it. Any of those relationships can change while the userspace object handle remains the same.

KERNEL DOCSLinux Xe — concrete TTM buffer-object placement and eviction

A current discrete-GPU example showing BO creation, system-memory/VRAM placement masks, TTM validation, runtime eviction/movement and GPU-address-space rebind behavior.

https://docs.kernel.org/gpu/xe/xe_mm.html

KERNEL UAPIDRM driver userspace APIs — GEM/BO creation and mappings

Current DRM driver-uAPI documentation with concrete buffer-object creation, CPU caching, VRAM/system-memory placement and VM binding examples.

https://docs.kernel.org/gpu/driver-uapi.html

A camera frame is a queued DMA buffer, not a stream of pixels copied one byte at a time: sensor → media pipeline → V4L2/VB2 → userspace or DMA-BUF consumer

Linux camera capture commonly uses Video4Linux2 (V4L2). A sensor and its supporting bridge/receiver/scaler blocks are represented as V4L2/media entities or sub-devices; userspace negotiates format and controls, allocates or imports a queue of frame buffers, queues those empty buffers to the driver, then starts streaming. Hardware captures a frame into one queued buffer by DMA, completion moves that buffer to the done queue, and userspace dequeues it. The application then requeues the buffer after consuming or forwarding the frame.

IMAGE SENSOR photodiodes → sensor readout / ISP / bridge / CSI receiver (platform dependent) ↓ V4L2 / media pipeline configured resolution + pixel format + crop/controls ↓ /dev/videoX ↓ VIDIOC_REQBUFS / CREATE_BUFS or imported DMA-BUFs ↓ QUEUE OF EMPTY FRAME BUFFERS userspace: VIDIOC_QBUF ↓ VIDIOC_STREAMON ↓ CAPTURE ENGINE DMA writes next frame into queued buffer ↓ interrupt / hardware completion videobuf2 marks buffer DONE + timestamp/sequence metadata ↓ userspace VIDIOC_DQBUF ↓ CPU processing OR pass same dma-buf fd to codec/GPU/display ↓ VIDIOC_QBUF again → buffer returns to capture queue Several buffers stay in flight so sensor capture and userspace processing can overlap.

V4L2 object/API	Role
`/dev/videoX`	Character-device interface for a capture/output function; complex hardware can expose several related nodes.
V4L2 sub-device	Represents supporting blocks such as camera sensors, muxes, decoders or controllers that form a larger media pipeline.
`VIDIOC_S_FMT`	Negotiates frame dimensions, pixel format and related capture layout.
videobuf2 (VB2)	Kernel buffer-queue framework implementing common streaming-buffer state and MMAP/USERPTR/DMA-BUF memory models.
`QBUF` / `DQBUF`	Transfers ownership of empty/complete buffers between application and driver without requiring a fresh allocation per frame.
DMA-BUF import/export	Lets the captured image buffer be shared with a codec, GPU or display subsystem without a mandatory CPU memcpy.

V4L2 controls the capture contract; it does not imply one fixed physical camera bus. A USB webcam, PCI capture card and MIPI-CSI camera can all appear through V4L2 while their lower hardware paths differ substantially. Buffer queueing is the common bridge from asynchronous hardware capture to userspace.

KERNEL UAPILinux V4L2 — video capture interface

Current userspace specification for /dev/videoX, capability discovery, image-format negotiation and capture I/O.

https://docs.kernel.org/userspace-api/media/v4l/dev-capture.html

KERNEL DOCSLinux videobuf2 — streaming buffer queues

Driver-side documentation for VB2 buffer memory models and queue/dequeue state handling, including MMAP, USERPTR and DMA-BUF backed buffers.

https://docs.kernel.org/driver-api/media/v4l2-videobuf2.html

KERNEL UAPIVideo4Linux2 userspace API

Complete V4L2 userspace API reference covering common elements, streaming I/O, buffer formats, capture/output interfaces and ioctls.

https://docs.kernel.org/userspace-api/media/v4l/v4l2.html

One image buffer can cross camera, GPU and display without becoming three separate copies: DMA-BUF

Modern media/graphics pipelines often have several independent devices that all need access to the same pixels. Linux DMA-BUF turns a hardware-accessible buffer into a shareable kernel object represented to userspace by a file descriptor. One driver exports the allocation; another imports that same object, attaches its device, maps the backing scatter/gather memory into the device's DMA address space and schedules work against it.

CAMERA / VIDEO DECODER DRIVER allocates or receives backing memory ↓ exports dma_buf ↓ userspace receives ordinary fd dma-buf fd ↓ pass fd between APIs/processes GPU DRIVER imports same dma_buf attach GPU device map attachment → device DMA addresses render / transform pixels asynchronously ↓ GPU completion represented by dma_fence ↓ implicit dma_resv fence or explicit sync_file fd DISPLAY DRIVER imports same dma_buf waits for required producer fence maps buffer for display engine scanout reads those same backing pages / device memory ↓ no mandatory CPU memcpy merely to cross subsystem boundaries

Object	Role
`dma_buf`	Shared-buffer object; userspace normally handles it as an opaque file descriptor.
exporter	Driver/subsystem that owns allocation policy and exposes the buffer to others.
attachment	Relationship between one importing device and the shared buffer; mapping produces that device's DMA-visible scatterlist.
`dma_resv`	Reservation object associated with the buffer that can carry implicit synchronization fences.
`dma_fence`	Completion primitive for asynchronous device work such as rendering, decoding or scanout dependencies.
`sync_file`	File-descriptor carrier for explicit fences passed through userspace APIs.
format / modifier	Describes pixel layout and device-specific tiling/compression constraints. Sharing the memory object alone does not guarantee that every device understands its layout.

Buffer sharing, synchronization and cache coherency are separate problems. DMA-BUF lets devices refer to one allocation; fences order asynchronous access; CPU/device cache-maintenance rules still have to be followed where the architecture is not fully coherent. A buffer can also be physically contiguous, scatter/gather-backed, system RAM or device-specific memory depending on the exporter.

KERNEL DOCSLinux — exchanging pixel buffers

Practical userspace-facing explanation of negotiating formats/modifiers, allocating a compatible buffer and exchanging DMA-BUF file descriptors between graphics/media components.

https://docs.kernel.org/userspace-api/dma-buf-alloc-exchange.html

KERNEL DOCSLinux — DMA-BUF heaps

Documents userspace allocation from system and contiguous heaps, showing that DMA-BUF describes sharing while allocation policy can come from a separate heap interface.

https://docs.kernel.org/userspace-api/dma-buf-heaps.html

KERNEL DOCSLinux — sync_file explicit fencing

Shows how a dma_fence can cross a userspace boundary as a file descriptor so a consumer waits until the producer has finished with the shared buffer.

https://docs.kernel.org/driver-api/sync_file.html

Shared virtual memory for GPUs/accelerators: PASID, ATS, PRI and HMM

Traditional accelerators often require special device-memory allocations and explicit copies. Shared Virtual Addressing (SVA/SVM) aims for something much easier to program: a pointer meaningful to the CPU can also be meaningful to a device. The hardware/software machinery underneath is substantial: process identifiers, IOMMU translation, device translation caches, page-request faults, invalidation and sometimes migration between system RAM and device-private memory.

PROCESS VIRTUAL ADDRESS SPACE CPU pointer p = 0x00007f12.... │ ├→ CPU load/store │ ↓ CPU TLB / page tables │ ↓ │ system RAM physical page │ └→ accelerator work descriptor includes SAME virtual address p + PASID identifies which process address space ↓ device issues DMA/memory access tagged with PASID ↓ device translation cache hit? ├── yes → translated access proceeds └── no → ATS request to IOMMU for translation ↓ mapping present? ├── yes → cache translation, access memory └── no/not-present → PRI page request ↓ OS handles page request/fault ↓ allocate/migrate/map page ↓ respond to device ↓ device retries ATS/access CPU PAGE TABLE CHANGES munmap/mprotect/COW/migration ↓ Linux MMU notifier / IOMMU invalidation path ↓ device-side cached translation must be invalidated before stale use HMM DEVICE-PRIVATE MEMORY system page can migrate to GPU/device-private RAM ↓ CPU PTE becomes special migration/device entry ↓ CPU touches that VA later → CPU page fault ↓ driver/kernel migrate page back to CPU-accessible memory Same virtual address ≠ same physical location forever.

Mechanism	Problem solved
PASID	Tags device transactions with a process/address-space identity in addition to the PCI Requester ID.
ATS	Lets a PCIe device request and cache IOMMU translations instead of translating every access from scratch.
PRI	Lets a device request that the OS establish/fault-in a translation/page it needs.
IOMMU SVA binding	Associates a device/PASID with a process memory context and coordinates DMA/page-request routing.
MMU notifier	Tells secondary/device MMUs that CPU page-table mappings are changing so stale translations can be removed.
HMM page-table mirroring	Helps device drivers mirror process mappings into a device-specific MMU while tracking invalidation.
ZONE_DEVICE / device-private memory	Represents accelerator-local memory inside Linux VM accounting/migration machinery.
migrate_vma_*()	Kernel helpers for migrating pages between system RAM and device-private memory while coordinating PTE/device-MMU state.
SVA page fault	Device access to an absent valid VA can cause page-request handling rather than requiring all memory be permanently pinned.
device TLB	Translation cache in/near accelerator; must obey invalidation rules just like CPU TLBs.

Shared virtual address is a naming contract, not a bandwidth promise. A GPU may still run far faster from its own local memory than from host DRAM over PCIe, so runtime/driver policy may migrate hot pages while preserving the process-visible pointer.

KERNEL DOCSLinux Heterogeneous Memory Management (HMM)

Current public HMM design: shared virtual memory, CPU page-table mirroring, device-private pages, migration to/from accelerator memory and MMU notifier coordination.

https://docs.kernel.org/6.15/mm/hmm.html

KERNEL DOCSLinux x86 Shared Virtual Addressing with PASID/ATS/PRI

Concrete SVA description: PASID tags process context; ATS supplies device-side translations; PRI requests missing pages; IOMMU/OS invalidations keep device TLB state coherent.

https://docs.kernel.org/next/x86/sva.html

KERNEL DOCSLinux page migration

Shows that process virtual addresses can stay constant while physical pages move between memory nodes; links directly to HMM migration for device-private memory.

https://docs.kernel.org/next/mm/page_migration.html

KERNEL DOCSLinux IOMMU userspace API

Virtualization-facing description of guest PASID binding, IOMMU cache invalidation, page-request servicing and guest SVA/IOVA use cases.

https://docs.kernel.org/6.0/userspace-api/iommu.html

Linux display control is a hardware transaction: DRM/KMS planes → CRTC → connector → vblank

Rendering and displaying are separate jobs. A GPU can finish drawing an image while the display controller is still scanning out the previous image. Linux Kernel Mode Setting (KMS) exposes the display pipeline as objects—framebuffers, planes, CRTCs and connectors—and modern drivers apply coordinated changes through atomic commits so a complete display state can be validated before hardware is touched.

RENDER SIDE GPU renders into GEM/VRAM/system-memory buffer ↓ dma-buf / GEM object contains finished image ↓ optional fence says rendering is complete DISPLAY / KMS SIDE DRM framebuffer object describes pixels in buffer ↓ plane source rectangle / format / scaling / position ↓ CRTC blends planes + owns display mode/timing ↓ encoder / bridge (driver/hardware routing) ↓ connector HDMI / DP / eDP / DSI / etc. ↓ monitor/panel ATOMIC UPDATE userspace assembles new plane + CRTC + connector properties ↓ DRM_MODE_ATOMIC_TEST_ONLY can validate without changing hardware ↓ atomic commit ↓ driver waits on input fences / prepares scanout state ↓ program hardware registers / page-flip address ↓ vertical blank boundary ↓ new framebuffer becomes visible without mid-frame tearing ↓ vblank/page-flip event + output fence signal completion

KMS object	Role
framebuffer	KMS metadata describing pixel format, pitches, offsets and backing buffer objects used for scanout.
plane	One independently positionable/scalable image layer; common hardware has primary, cursor and overlay planes.
CRTC	Logical display pipeline that combines planes and owns mode timing/scanout state.
connector	Userspace-visible physical/logical display endpoint such as HDMI, DisplayPort or an embedded panel connection.
mode	Resolution and timing parameters: pixel clock, active dimensions, blanking and sync intervals.
atomic state	Prospective set of object-property changes checked together before being committed to hardware.
page flip	Change the framebuffer/address a plane scans out, commonly synchronized to vblank.
vblank	Vertical blanking interval/event between displayed frames; useful synchronization point for tear-free state changes.
in-fence	Dependency proving rendering into an input framebuffer has completed before scanout reads it.
out-fence / flip event	Completion indication that a display commit/page flip has reached its synchronization point.

KMS does not render the scene. Rendering creates pixel buffers; KMS decides which buffers the display engine will scan, where planes are placed, which mode the connector uses and when a state transition becomes visible. This is why a system can use GPU rendering, CPU rendering or even a simple “dumb” buffer with the same display-control model.

DRM/KMS INSPECTION LAB # DRM nodes ls -l /dev/dri 2>/dev/null # Connector state exposed by DRM for f in /sys/class/drm/card*-*/status; do [ -e "$f" ] && echo "$f: $(cat "$f")"; done # Modes advertised for connected outputs for f in /sys/class/drm/card*-*/modes; do [ -s "$f" ] && { echo "--- $f"; cat "$f"; }; done # If libdrm tests are installed, modetest is the canonical object inspector: # modetest -c # connectors # modetest -p # CRTCs / planes / properties # Avoid setting modes on the display serving your current session unless you # understand how your compositor/display server will react.

KERNEL DOCSLinux Kernel Mode Setting (KMS)

Canonical object model for framebuffers, planes, CRTCs, connectors, display modes, atomic state and vblank handling.

https://docs.kernel.org/gpu/drm-kms.html

KERNEL DOCSDRM atomic modesetting helpers

Implementation-side view of validating and committing coordinated display state, waiting for fences/vblanks and cleaning up old framebuffers after a flip.

https://docs.kernel.org/gpu/drm-kms-helpers.html

KERNEL DOCSDRM userspace API — atomic commits, properties and events

Stable userspace-facing DRM rules and atomic KMS flags/events, including test-only validation, visible-artifact constraints and vblank-timestamped completion.

https://docs.kernel.org/gpu/drm-uapi.html

One pixel end to end: framebuffer → scanout timing → display signal

Graphics software may render an image into a framebuffer, but displaying it requires a hardware scanout process. A display engine repeatedly reads pixels in raster order according to a pixel clock and timing counters, creates active-video/blanking/sync state, then a transmitter/PHY encodes the stream for VGA, DVI/HDMI, DisplayPort or an embedded panel link.

CPU / GPU renders image ↓ framebuffer in RAM / VRAM ↓ DMA / dedicated display fetch display controller / scanout engine ├── horizontal pixel counter ├── vertical line counter ├── pixel clock └── blanking / sync / data-enable generation ↓ pixel format conversion / colour processing ↓ VGA DAC OR DVI/HDMI/DisplayPort transmitter/PHY ↓ cable / connector ↓ monitor timing receiver + panel electronics ↓ physical subpixels emit/transmit light

PUBLIC PROJECTProject F — Beginning FPGA Graphics

One of the clearest public hardware-level display explanations: pixel clock, horizontal/vertical timing, blanking, sync, RGB values and actual Verilog that emits VGA/DVI-compatible signals.

https://projectf.io/posts/fpga-graphics/

PUBLIC PROJECTProject F — Framebuffers

Shows bitmap pixels stored in memory being fetched and scanned onto a display. Includes synthesizable hardware and simulation examples.

https://projectf.io/posts/framebuffers/

PUBLIC PROJECTProject F — Display Signals

Goes deeper into registered timing, screen coordinates, line/frame signals, palettes and active versus blanking periods.

https://projectf.io/posts/display-signals/

PUBLIC PROJECTProject F — Hardware Sprites

Useful historical/architectural contrast: graphics need not always be a CPU drawing every pixel into RAM; dedicated sprite logic can generate pixels directly during scanout.

https://projectf.io/posts/hardware-sprites/

10. Advanced follow-ons once a simple CPU makes sense

PUBLIC NOTESBerkeley CS61C — pipelining and performance

Introduces latency, throughput, critical paths, clock frequency, and why pipelining can increase instruction throughput without making an individual instruction magically instantaneous.

https://notes.cs61c.org/content/pipeline/

PUBLIC NOTESBerkeley CS61C — pipeline summary and hazards

Use after building a simple processor. Covers the five-stage pipeline and points toward data/control hazards, forwarding, stalls, and more realistic CPU organization.

https://notes.cs61c.org/content/pipeline/summary/

SPECRISC-V specifications library

The official set of ratified RISC-V architecture specifications. After RV32I, explore privileged architecture, extensions, vectors, atomics, compressed instructions, and other real-world ISA machinery.

https://docs.riscv.org/

Compact jargon map

These are intentionally short reminders, not substitutes for the linked material. Their purpose is to make schematics and architecture diagrams less opaque.

VDD	Positive supply rail in MOS/CMOS notation; historically named for the voltage associated with transistor drains.
GND / VSS	Reference potential, usually treated as 0 V. VSS is common MOS notation for the lower supply rail.
NMOS	N-channel MOSFET. In digital CMOS it is typically used in the pull-down network, conducting strongly when its gate is high relative to its source.
PMOS	P-channel MOSFET. In digital CMOS it is typically used in the pull-up network, conducting when its gate is low relative to its source.
CMOS	Complementary MOS: NMOS and PMOS networks are paired so static logic normally draws very little steady-state current.
Propagation delay	Time between an input transition and the corresponding valid output transition through a gate or combinational path.
Setup / hold	Intervals around a clock edge during which a flip-flop's data input must remain stable.
Clock period / frequency	Period is time per cycle; frequency is cycles per second. Maximum usable frequency is constrained by the slowest register-to-register path plus timing margins.
Register	A group of flip-flops storing a multi-bit word, usually updated on a clock edge.
ALU	Arithmetic Logic Unit: combinational hardware for arithmetic, bitwise logic, comparisons, shifts, etc.
Bus	A group of conductors/signals that carry a multi-bit value among modules. Buses normally need rules determining which source drives them.
Program counter (PC)	Register holding the address of the current or next instruction, depending on the architecture's definition.
Control unit	Logic that decodes instructions and timing/state into control signals that steer the datapath.
ISA	Instruction Set Architecture: the programmer-visible contract between software and processor implementation.
Datapath	Registers, ALU, muxes, buses, memory interfaces, and related paths through which instruction data flows.
Microarchitecture	A particular hardware implementation of an ISA: pipeline depth, caches, execution units, branch prediction, control organization, etc.
SRAM / DRAM	Two major RAM families. SRAM stores state in bistable cells; DRAM stores charge in capacitive cells and requires refresh.
Tri-state	An output mode with 0, 1, and high-impedance Z, historically useful for sharing buses among multiple possible drivers.
Address decoding	Logic that examines address bits and selects the RAM, ROM, peripheral, or register mapped to that address range.
Critical path	The slowest relevant combinational path between state elements; often the path that limits maximum clock speed.
MMIO	Memory-mapped I/O: device control/status registers occupy addresses in an address space and are accessed with load/store-like operations.
DMA	Direct Memory Access: a device or DMA engine transfers data to or from RAM without the CPU executing a load/store for every byte or word.
Interrupt	A hardware/software event that diverts execution to an interrupt handler so the CPU can react to a device or other event.
PCIe	PCI Express: a packet-based, point-to-point serial interconnect used to connect many modern peripherals and bridges; it is not a single old-style shared parallel bus.
BAR	PCI Base Address Register: tells system software how much I/O or memory address space a PCI device needs and where its device registers/resources are mapped.
IOMMU	I/O Memory Management Unit: translates and restricts device-visible DMA addresses, analogous in spirit to an MMU for device memory accesses.
Memory controller	Hardware that schedules and translates CPU/memory requests into the electrical command/address/data protocol required by DRAM.
Clock domain	A region of synchronous logic driven by a particular clock. Crossing between unrelated clock domains requires synchronization or asynchronous buffering.
VCC	Positive supply notation historically common with bipolar/TTL logic; in mixed literature you will see VCC where MOS texts may use VDD.
Decoupling capacitor	A capacitor placed close to an IC supply pin to provide local transient current and reduce supply/ground noise caused by fast switching.
Reset	A signal or condition that forces state elements or a processor into a defined initial state so execution can begin predictably.
Oscillator	A circuit that generates a periodic signal without requiring an external clock waveform; a crystal is often the frequency-selective element, not the complete oscillator by itself.
Jitter	Short-term variation of clock edge timing from its ideal positions. Excessive jitter reduces timing margin.
Chip select / enable	A control signal that determines whether a memory or peripheral is active/responding. Often generated by decoding address bits.
Firmware	Software stored in nonvolatile memory that initializes hardware and/or provides low-level services before or beneath the main operating system.
Reset vector	The architecturally defined address or mechanism that determines where a processor begins fetching instructions after reset.
Floating gate	An electrically isolated conductive region used by classic flash/EEPROM cells to retain charge and therefore data without power.
VIH / VIL	Guaranteed receiver input-voltage thresholds: voltages at or above VIH are recognized as HIGH; at or below VIL are recognized as LOW. The gap provides noise margin.
VOH / VOL	Guaranteed output-voltage levels for HIGH and LOW under specified load/current conditions.
Fan-out	How many receiver inputs an output can drive while still meeting voltage/current and timing specifications. Modern CMOS is often limited by capacitance/timing as much as DC input current.
Pull-up / pull-down	A resistor or active device that biases a signal toward a defined logic level when nothing stronger is driving it.
Open-drain	An output that can actively pull a line low but does not actively drive it high; an external pull-up supplies the high level. Multiple open-drain devices can safely share a line under the protocol rules.
Bus contention	Two outputs drive the same net to conflicting levels at the same time, potentially causing excessive current, corrupted voltage and damage.
Rise / fall time	Time a signal edge takes to move between defined low/high voltage percentages. Fast edges contain high-frequency energy and make PCB interconnect behavior more demanding.
Baud rate	Symbol rate of a serial link; in simple binary UART usage it is commonly numerically equal to bits per second, but baud and bit/s are not universally identical concepts.
SCLK / CS	Common SPI signals: serial clock and chip select. CS chooses the peripheral; SCLK establishes when serial data is shifted or sampled.
SDA / SCL	The two principal I²C lines: serial data and serial clock. They are shared open-drain/open-collector-style signals with pull-ups.
VGS / VDS	MOSFET terminal-voltage differences: gate-to-source and drain-to-source. They determine whether a MOS transistor is cut off, operating approximately resistively, or in saturation.
Threshold voltage (Vt)	Approximate gate-to-source voltage at which a MOS channel begins to form strongly enough for useful conduction; not the same thing as a digital logic threshold.
Bitline	Column wire in a memory/register array that carries data to or from selected storage cells.
Wordline	Row-select wire in a memory/register array that turns on access transistors for a selected word or row.
Precharge	Driving a dynamic node or memory bitline to a known initial voltage before evaluation/readout.
Sense amplifier	Circuit that detects and amplifies the small voltage difference produced on memory bitlines during a read.
Clock skew	Difference in arrival time of nominally the same clock edge at different state elements.
Clock jitter	Variation of clock-edge timing from its ideal periodic position over time.
PLL	Phase-Locked Loop: feedback system using phase/frequency comparison and a controlled oscillator to generate or align clocks/frequencies.
DLL	Delay-Locked Loop: feedback system that adjusts a delay line to align clock phase without using a free-running oscillator in the same way as a PLL.
tCQ	Clock-to-Q delay: time from a flip-flop's active clock edge until its output Q becomes valid.
Bus arbitration	Rules/circuitry deciding which potential bus master may control a shared bus at a given time.
Chip select	Control signal enabling a particular memory or peripheral after address decoding determines that the current transaction targets it.
BJT	Bipolar Junction Transistor. Current-controlled transistor family used heavily in TTL logic; NPN BJTs are the common active devices in classic 7400-series TTL.
TTL	Transistor-Transistor Logic: bipolar digital logic family historically built mostly from NPN transistors and resistors, commonly powered from 5 V.
Logic family	A compatible set of logic ICs sharing electrical characteristics such as supply range, input thresholds, output drive, delay and power behavior.
Microcode	Low-level control words/routines inside some CPUs that sequence register transfers, ALU operations, memory cycles and other internal actions needed to implement machine instructions.
Control store	ROM/RAM-like storage holding microinstructions in a microcoded control unit.
Crystal	Piezoelectric resonator used as a stable frequency-determining element in an oscillator circuit; it is not by itself a complete logic clock source.
Power-on reset (POR)	Circuit/function that holds digital logic in reset until supply voltage and related conditions are sufficiently valid for deterministic startup.
FTL	Flash Translation Layer: SSD/controller firmware that maps host logical block addresses to physical NAND locations and manages erase blocks, wear, garbage collection and bad blocks.
Seek time	Mechanical time required for a hard-disk actuator to move the read/write head to the target track.
Rotational latency	Delay waiting for the desired hard-disk sector to rotate under the head after the head has reached the correct track.
Boot ROM	Nonvolatile code available immediately after reset, often responsible for validating/loading later firmware or a bootloader.
Trap	Synchronous or asynchronous transfer of control into a privileged handler caused by an exception, interrupt, or explicit system-call mechanism.
Exception	Condition associated with instruction execution that causes special architectural handling, such as illegal instruction, page fault, divide error, or breakpoint.
System call	Controlled entry from an application into operating-system kernel code, normally implemented through a trap/instruction plus a defined calling convention.
TLB	Translation Lookaside Buffer: cache of recent virtual-to-physical address translations used to avoid walking page tables on every memory access.
Page table	Memory-resident translation data structure mapping virtual pages to physical frames and carrying permissions/status bits.
Page-table walk	Hardware or software traversal of page-table structures after a required translation is absent from the TLB.
Cache line	Fixed-size block transferred/stored as a unit in a cache, commonly tens of bytes in modern CPUs.
Cache coherence	Mechanisms/protocol rules keeping multiple cached copies of shared memory sufficiently consistent after writes.
Memory consistency model	Architectural rules describing which orderings of memory operations may be observed by concurrent processors/software.
IRQ	Interrupt Request: signal or message indicating that a device/controller requests processor attention.
Interrupt vector	Identifier/address-selection mechanism used to choose the appropriate interrupt or trap handler.
Logic analyzer	Instrument that samples many signal lines against digital thresholds and displays their timing/decoded values.
Oscilloscope	Instrument that displays measured voltage versus time, preserving analog waveform shape rather than reducing it to binary states.
Trigger	Measurement condition that tells an oscilloscope or logic analyzer when to anchor/capture an event of interest.
Superscalar	Microarchitecture capable of issuing/executing more than one instruction or micro-operation per cycle using multiple parallel resources.
Out-of-order execution	Executing ready instructions before older stalled instructions while preserving the architecture's required observable behavior.
Register renaming	Mapping architectural register names to a larger set of physical registers/tags to eliminate false WAR/WAW name dependencies.
ROB	Reorder Buffer: structure associated with tracking in-flight instructions and supporting ordered retirement, recovery and precise architectural state.
Branch predictor	Hardware that predicts branch direction and/or target so instruction fetch can continue before the branch is resolved.
Speculative execution	Executing work before it is known to be architecturally required, with machinery to discard/recover if the prediction or assumption was wrong.
Retirement / commit	Point where a completed instruction's effects become part of architecturally visible state in the required order.
PCIe Root Complex	Host-side PCI Express component connecting the CPU/memory system to the PCIe fabric and endpoints.
TLP	Transaction Layer Packet: PCIe packet carrying requests/completions such as memory reads, memory writes and messages.
PCI configuration space	Standard register space used to identify/configure PCI/PCIe functions and discover capabilities/resources.
MSI / MSI-X	Message-Signaled Interrupts: PCI/PCIe mechanisms where a device signals an interrupt by issuing a specially addressed write/message.
USB endpoint	Unidirectional logical source or sink of USB traffic within a device, identified by endpoint number and direction.
USB enumeration	Host-driven process of discovering a newly attached USB device, reading descriptors, assigning an address and selecting configuration/interfaces.
MAC	Media Access Control block: in Ethernet, the digital link-layer hardware that forms/receives frames and interfaces toward a PHY.
PHY	Physical-layer transceiver translating digital interface data to/from the electrical/optical signaling used on the physical medium.
MII / RMII / GMII / RGMII	Families of digital interfaces connecting an Ethernet MAC to an Ethernet PHY at different speeds and pin counts.
MDIO / MIIM	Management interface used by software/MAC-side logic to read and write Ethernet PHY registers, separate from the packet-data interface.
MT/s	Megatransfers per second: interface transfer rate, not necessarily the same number as oscillator/clock frequency in MHz.
GT/s	Gigatransfers per second, commonly used for serial interconnect lane rates such as PCI Express.
Bandwidth	Amount of information that can be transferred per unit time; distinct from latency.
DIMM	Dual Inline Memory Module: circuit board carrying DRAM devices plus identification/control components and contacts for a memory socket.
Memory channel	Independent data/control path between a memory controller and one or more memory devices/modules.
Rank	Group of DRAM chips selected together to provide the full data width for a memory transaction.
DRAM bank	Semi-independent subarray within a DRAM device with its own currently open row/row-buffer state.
Row buffer	Sense-amplifier structure holding the contents of an activated DRAM row while column accesses occur.
tRCD	Minimum DRAM delay between ACTIVATE and a subsequent column READ/WRITE to that bank.
tRP	Minimum DRAM precharge time before a different row can be activated in a bank.
tRAS	Minimum time an activated DRAM row must remain active before precharge.
CAS latency / CL	DRAM read-latency parameter expressed in clock cycles for a selected operating mode; cycles must be converted using the actual clock period to obtain time.
SECDED	Single Error Correct, Double Error Detect: common class of error-correcting code used in system memory.
On-die ECC	ECC performed internally by a memory chip; in DDR5 this does not by itself provide end-to-end ECC protection across the memory channel.
PCH	Platform Controller Hub: Intel term for a chipset component supplying many I/O/platform functions and linked to the processor, commonly over DMI.
DMI	Direct Media Interface: Intel point-to-point link between processor and Platform Controller Hub in two-chip platform designs.
VRM	Voltage Regulator Module/circuitry that converts a supply rail into tightly regulated low-voltage, high-current rails required by processors and other chips.
P-state	Processor operating-performance state/point associated with a frequency and voltage target or range.
C-state	Processor idle state; deeper states save more power but generally require more time/energy to exit.
Clock gating	Suppressing clock transitions in inactive synchronous logic to reduce dynamic switching power.
Power gating	Disconnecting or collapsing a power domain to reduce leakage when a block is inactive.
NVMe	NVM Express: storage protocol built around submission/completion queues and transports such as PCI Express.
Submission Queue	Host-memory circular queue into which software places NVMe commands before ringing the corresponding doorbell.
Completion Queue	Host-memory circular queue into which an NVMe controller writes completion records.
Doorbell register	Memory-mapped register written by software to notify hardware that a queue pointer/state has changed.
SM	Streaming Multiprocessor: NVIDIA GPU execution block containing schedulers, register files, execution units and shared/L1 resources.
Warp	CUDA/NVIDIA group of 32 threads scheduled/executed together under the SIMT programming/execution model.
SIMT	Single Instruction, Multiple Threads: GPU execution model where groups of threads execute a common instruction stream while retaining per-thread state and masking/divergence behavior.
Active low	A signal convention where the asserted/true function corresponds to logic 0; often marked with #, /, an overbar, or an n-prefix.
Endianness	Convention defining byte order of multi-byte values in byte-addressed memory; commonly little-endian or big-endian.
Two's complement	Dominant signed-integer encoding where an n-bit pattern has range −2^(n−1) through 2^(n−1)−1 and ordinary binary addition hardware naturally supports signed addition.
IEEE 754	Widely used floating-point standard defining binary formats, rounding behavior, infinities, NaNs, subnormals and arithmetic semantics.
Opcode	Instruction bit field identifying the operation or broad instruction class to the CPU decoder.
Immediate	Constant value encoded directly within an instruction rather than read from a separate register or memory location.
MUX	Multiplexer: combinational circuit selecting one of several inputs according to select/control bits.
DQ	Generic memory-interface notation for a data input/output signal.
DQS	Data strobe used by DDR memories to time data transfers relative to DQ signals.
Power-good	Signal indicating that a supply rail or platform power condition has reached an acceptable operating range.
Half-adder	Combinational circuit adding two one-bit inputs and producing sum and carry outputs.
Full-adder	One-bit adder with A, B and carry-in inputs plus sum and carry-out outputs; basic building block of wider adders.
Ripple-carry adder	Multi-bit adder where carry propagates sequentially from lower to higher bit positions, creating delay proportional to carry-chain length.
Carry lookahead	Adder technique computing generate/propagate information so carries can be determined faster than simple ripple propagation.
Partial product	Shifted/intermediate product term generated from subsets/bits of multiplier inputs before reduction into the final product.
Relocation	Object-file/linker record identifying encoded data/instructions that must be adjusted after symbol addresses and final layout become known.
ELF	Executable and Linking Format: common Unix/Linux object, executable, shared-library and core-file format.
Section	Object/link-time grouping such as .text, .data, .bss or .rodata.
Program header / segment	ELF loader-oriented description of file ranges to map into process memory with specific sizes, addresses and permissions.
Linker script	File controlling output section placement, memory regions, symbols and related executable/firmware layout decisions.
Synthesis	Transformation of HDL/RTL behavior into an optimized network of implementable hardware primitives/cells.
Technology mapping	Replacing generic synthesized logic with the actual LUTs, flip-flops, standard cells or other resources supported by a target technology.
LUT	Lookup Table: small programmable truth-table resource forming the main combinational-logic primitive in many FPGAs.
Standard cell	Pre-designed ASIC logic/layout building block such as NAND, AOI, flip-flop or buffer with characterized area/timing/power.
Place and route	Physical-design process that chooses circuit-cell locations and constructs actual wire routes between them.
Scanout	Display-engine process that continuously reads pixel data and emits it according to display raster/timing requirements.
Framebuffer	Memory containing pixel/image data used as a source for display scanout or rendering.
HID report	USB/Bluetooth/I²C HID data packet containing input/output/feature values according to a device's report descriptor.
EV_KEY	Linux input-event type representing keyboard keys, buttons and similar binary/key-like controls.
Photolithography	Semiconductor-manufacturing process using patterned light-sensitive material to define where later etch, implant, deposition or other process steps act.
Ion implantation	Process that accelerates dopant ions into selected semiconductor regions to alter their electrical properties.
Die	Individual integrated-circuit piece cut from a fabricated semiconductor wafer.
Wire bond	Fine wire connecting a die pad to a package lead/substrate connection in many IC packages.
Flip-chip	Packaging method where the active die face connects through solder/microbumps directly to a package/substrate rather than long peripheral bond wires.
BGA	Ball Grid Array: package with a two-dimensional array of solder balls underneath for electrical/mechanical connection to a PCB.
LGA	Land Grid Array: package with flat contact lands that mate with a socket or corresponding contacts rather than attached solder balls.
I/O pad	Large on-die circuit/physical structure connecting tiny core logic to external package connections, often including drivers, input buffers, ESD and level shifting.
ESD protection	Structures intended to shunt/limit electrostatic-discharge energy so external pin events do not destroy thin internal transistor gates/junctions.
TAP	JTAG Test Access Port: standard state machine and serial register-access mechanism controlled by TCK/TMS/TDI/TDO.
TCK	JTAG test clock driven by the debug/test adapter.
TMS	JTAG Test Mode Select signal controlling TAP state-machine transitions.
TDI / TDO	JTAG serial Test Data In and Test Data Out signals used to shift instruction/data-register contents through a TAP or scan chain.
Debug Module	On-chip hardware implementing operations such as halt/resume, register access, memory access and reset/debug control.
Boundary scan	Scan cells associated with chip I/O that allow interconnects/pins to be controlled and observed for board manufacturing/debug tests.
Hardware trigger	On-chip comparator/event logic that enters debug or takes another action when an instruction address, memory address or defined event matches.
Ibex	Open-source production-quality lowRISC 32-bit RISC-V CPU core written in SystemVerilog, useful as a manageable real implementation to study.
TileLink-UL	Uncached lightweight TileLink profile used by OpenTitan to connect processors, memories and memory-mapped peripherals.
Crossbar	Interconnect that routes transactions between multiple initiators/targets and arbitrates when paths/resources conflict.
Relay	Electromechanical switch whose contacts are moved by an energized coil; can implement Boolean logic and stored state.
Vacuum tube / valve	Electronic device controlling electron flow in vacuum; triodes provided gain/switching for early electronic computers.
Triode	Three-electrode vacuum tube with cathode, control grid and anode/plate; grid voltage controls electron flow.
Williams-Kilburn tube	Cathode-ray-tube random-access memory storing bits as electrostatic charge patterns that must be regenerated.
Delay-line memory	Serial memory that stores data as travelling acoustic or electrical pulses recirculated through a delay medium.
Magnetic drum	Rotating magnetic cylinder with one or more fixed read/write heads used as early main memory and/or secondary storage.
Magnetic core memory	RAM built from magnetized ferrite rings threaded by selection/read wires; nonvolatile and dominant before semiconductor main memory.
Destructive read	Memory read operation that destroys or changes the stored state and therefore requires the value to be rewritten/restored afterward.
Stored-program computer	Computer in which machine instructions are represented as data in addressable memory and fetched by the processor.
Paper tape	Long punched medium encoding data/program bits or characters in rows of holes, read by electromechanical/optical readers.
Punched card	Stiff card with hole positions encoding records/instructions; historically a dominant batch data/program medium.
Console switches	Physical operator controls for setting addresses/data, stepping execution, resetting or examining/modifying machine state.
Indicator lamps	Front-panel lights connected to selected machine-state signals so operators can observe registers, buses or status.
Light pen	CRT input device detecting screen illumination at a pointed position; used on early interactive graphics systems such as the PDP-1.
Karnaugh map	Gray-code-arranged truth table used to visually minimize small Boolean logic functions by grouping adjacent equal outputs.
Minterm	Boolean product/AND term that is true for one particular input combination.
Maxterm	Boolean sum/OR term that is false for one particular input combination.
Moore machine	Finite-state machine whose outputs depend on current stored state.
Mealy machine	Finite-state machine whose outputs can depend on both current stored state and current inputs.
Logic hazard	Temporary incorrect output transition caused by unequal propagation delays through logically reconvergent paths.
High impedance / Z	Output condition where a driver is effectively disconnected from the bus so another device may control the line.
Open drain	Output structure that actively pulls low but relies on an external pull-up (or other bias network) for the high level.
Push-pull	Output stage that actively drives both high and low logic states.
Differential signalling	Encoding information in the voltage difference between two related conductors rather than one conductor relative to ground alone.
Common-mode voltage	Voltage component shared by both conductors of a differential pair relative to a reference.
SerDes	Serializer/Deserializer: circuitry converting between parallel internal data and one/few high-speed serial streams.
CDR	Clock and Data Recovery: receiver circuitry that derives sampling timing/clock phase from an incoming serial data stream.
Equalization	Signal-processing/circuit technique compensating channel frequency loss and distortion to improve high-speed receiver margins.
Eye diagram	Overlay of many serial bit intervals used to visualize amplitude/timing opening, jitter, noise and intersymbol interference.
Bootblock	Very early firmware stage placed where processor reset can reach it; responsible for establishing enough machine state to load/enter later stages.
SPI flash	Serial nonvolatile flash commonly used to store PC/embedded firmware images.
UEFI	Unified Extensible Firmware Interface: standardized firmware/OS-loader interface defining boot/runtime services, protocols and data structures.
VCD	Value Change Dump: standard textual waveform format recording digital signal changes from HDL simulation.
FST	Fast Signal Trace: compact binary waveform format commonly used with GTKWave and supported by simulators such as Verilator.
Waveform	Representation of signal value versus time; essential for reasoning about clocks, protocols, delays and ordering.
Hazard detection	Pipeline logic that detects data/control situations requiring stalls, forwarding or other intervention to preserve correct execution.
Forwarding / bypassing	Routing a just-computed pipeline result directly to a dependent instruction before normal register-file write-back.
Pipeline stall	Deliberately preventing one or more pipeline stages from advancing for a cycle so a required condition/data becomes available.
Pipeline flush	Discarding younger/speculative instructions from pipeline stages, commonly after a branch misprediction or exception.
Chronogram	Timing chart showing several digital signals or states versus time, often used by logic simulators.
Logic simulator	Software that evaluates digital circuits and state transitions without requiring the circuit to be physically built.
Cache set	Group of cache ways selected by the address index; a block mapped to that set may occupy one of its ways.
Cache way	One candidate cache-line slot within a set in a set-associative cache.
Cache tag	Upper address information stored with a cache line and compared on lookup to identify which memory block occupies that slot.
Cache offset	Low-order address bits selecting byte/word position within a cache line.
Dirty bit	Cache state indicating that a write-back line has been modified and differs from lower-level memory.
Write-through	Cache policy writing modified data to both cache and lower memory immediately.
Write-back	Cache policy postponing lower-memory update until a modified/dirty line is evicted.
Write-allocate	Store-miss policy that first brings the missed line into cache and then updates it.
Compulsory miss	Cache miss caused by the first access to a block not previously loaded.
Capacity miss	Cache miss caused because the active working set exceeds total usable cache capacity.
Conflict miss	Cache miss caused by placement restrictions forcing useful blocks to compete for the same cache set.
VPN	Virtual Page Number: high-order portion of a virtual address identifying its virtual page.
PPN	Physical Page Number: high-order portion of a physical address identifying a physical page/frame.
PTE	Page Table Entry: mapping/control record containing a physical page number or pointer plus validity/permission/status information.
satp	RISC-V Supervisor Address Translation and Protection register selecting the active page-table root/mode and related context information.
SFENCE.VMA	RISC-V instruction used to synchronize address-translation state with page-table updates according to architecture rules.
LR/SC	Load-Reserved / Store-Conditional atomic pair: read value and establish reservation, then conditionally store only if reservation remains valid.
AMO	Atomic Memory Operation: atomic read-modify-write instruction such as swap, add, xor or min/max.
Acquire	Memory-ordering constraint preventing relevant later operations from being observed before the acquire.
Release	Memory-ordering constraint preventing relevant earlier operations from being observed after the release.
Memory fence	Architectural instruction/primitive restricting visibility or reordering of selected memory operations.
CDC	Clock Domain Crossing: transfer of information between logic controlled by different clock domains.
Synchronizer	Register chain or related circuit used to reduce the probability that metastability propagates into destination-domain logic.
Asynchronous FIFO	FIFO with independent write/read clocks, used to move multi-bit streams safely between unrelated clock domains.
Gray code	Encoding in which adjacent values differ by only one bit; useful when synchronizing counters/pointers across clock domains.
FPU	Floating-Point Unit: hardware implementing floating-point arithmetic, conversions, comparisons and IEEE-defined status behavior.
FMA	Fused Multiply-Add: computes a×b+c as one fused operation with a single final rounding.
Subnormal	IEEE-754 floating-point value with minimum exponent encoding and no implicit leading 1, enabling gradual underflow near zero.
Guard/round/sticky bits	Extra internal bits retained during floating-point arithmetic to decide the correctly rounded representable result.
MESI	Cache-coherence protocol/state model whose line states are Modified, Exclusive, Shared and Invalid.
Exclusive state	MESI clean cache-line state indicating this cache has the only cached copy and memory is up to date.
False sharing	Performance problem where independent variables used by different cores occupy the same cache line and therefore cause unnecessary coherence invalidations/ownership transfers.
IOVA	I/O Virtual Address: device-visible DMA address translated/protected by an IOMMU.
IOTLB	IOMMU Translation Lookaside Buffer caching recent I/O address translations.
PASID	Process Address Space ID used by PCIe/IOMMU mechanisms to associate device requests with a process/address-space context.
ATS	PCIe Address Translation Services, allowing capable devices to request/cache address translations under system IOMMU control.
PRI	PCIe Page Request Interface, allowing capable devices to request servicing of address-translation/page faults.
DDR PHY	Physical-layer circuitry between memory controller logic and high-speed DRAM pins, including drivers/receivers, delay elements, VREF and training/calibration machinery.
Write leveling	DDR calibration procedure adjusting write DQS timing relative to CK to compensate board/package skew.
Read leveling	DDR calibration process adjusting receiver delay/sampling to capture incoming DQ/DQS near the center of the valid eye.
VREF training	Calibration of receiver reference voltage to improve vertical eye margin on high-speed memory interfaces.
LTSSM	PCI Express Link Training and Status State Machine controlling link detection, training, configuration, recovery, power states and normal L0 operation.
TS1 / TS2	PCIe training ordered sets exchanged during link initialization/recovery.
L0	Normal active PCIe link state in which higher-layer traffic can flow.
VM exit	Hardware transfer from guest execution to hypervisor caused by a configured/sensitive event, exception or intercept.
VM entry	Hardware transition from hypervisor into a configured guest virtual-machine context.
EPT	Intel Extended Page Tables: second-stage translation from guest physical to host/system physical addresses.
NPT	AMD Nested Page Tables: hardware second-stage guest-physical to host-physical address translation.
G-stage translation	RISC-V hypervisor second-stage translation from guest physical addresses to supervisor/system physical addresses.
VMCS	Intel Virtual Machine Control Structure holding guest/host state and VMX execution controls.
VMCB	AMD Virtual Machine Control Block holding guest state and virtualization intercept/control information.
VALID/READY	Two-signal flow-control handshake where a transfer occurs on a sampling clock edge only when source VALID and destination READY are both asserted.
Backpressure	Flow-control effect where a receiver that cannot accept more work stalls an upstream producer, often by deasserting READY or withholding credits.
Outstanding transaction	Accepted request whose completion/response has not yet returned.
Transaction ID	Identifier carried/tracked so responses can be associated with the correct in-flight request when multiple operations overlap.
AXI	Arm AMBA Advanced eXtensible Interface family of on-chip protocols with independent read/write address, data and response channels.
TileLink	Open cache-coherent/noncoherent on-chip interconnect protocol used in Rocket Chip/Chipyard and other RISC-V systems.
TL-UL	TileLink Uncached Lightweight profile used by OpenTitan for memory-mapped request/response traffic.
NAPI	Linux network event-processing mechanism that combines interrupts with bounded polling to process packet RX/TX completions efficiently.
sk_buff	Linux kernel packet metadata/data-buffer object used throughout the conventional networking stack.
RSS	Receive Side Scaling: NIC mechanism hashing packet flows across multiple hardware receive queues/CPUs.
RX ring	Circular descriptor/completion structure coordinating receive buffers between a network driver and NIC hardware.
VFS	Linux Virtual Filesystem layer providing common file operations over multiple concrete filesystems.
Page cache	Kernel RAM cache of file contents used by buffered file I/O.
Dirty page / folio	Cached file-memory region modified in RAM but not yet persisted to its backing storage.
Writeback	Kernel process of converting dirty cached file data into storage I/O and completing persistence bookkeeping.
bio	Linux block-layer object describing block-device I/O over one or more memory regions.
blk-mq	Linux multiqueue block layer designed to feed multiple hardware queues efficiently on modern storage devices.
FCS	Frame Check Sequence: checksum/CRC field appended to a link-layer frame for corruption detection.
CRC	Cyclic Redundancy Check: polynomial error-detection code particularly effective at detecting transmission burst errors.
Replay buffer	Sender-side storage of unacknowledged link packets so damaged/lost transmissions can be resent.
End-to-end integrity	Protection scheme in which integrity metadata is maintained/checked across multiple internal transport stages rather than only one physical hop.
SMT	Simultaneous Multithreading: one physical core exposes multiple hardware-thread contexts that share execution resources and can issue work in the same time period.
Logical processor / logical CPU	Hardware execution context exposed to software/OS as a schedulable CPU; SMT may provide multiple logical CPUs per physical core.
Physical core	Actual execution core containing pipeline/front-end/execution resources; may support one or multiple hardware threads.
NUMA	Non-Uniform Memory Access: shared-memory architecture where access latency/bandwidth varies with CPU-to-memory locality.
NUMA node	CPU/memory locality domain in which some processors and memory ranges are closer to one another than to other nodes.
Local memory	Memory attached/closest to the NUMA node of the accessing CPU.
Remote memory	Memory reached from a CPU through another NUMA node/socket/fabric path, usually with higher latency or lower effective bandwidth.
SRAT	ACPI System Resource Affinity Table associating processors, memory and initiators with proximity/NUMA domains.
SLIT	ACPI System Locality Information Table providing relative distance values among NUMA/system localities.
MADT	ACPI Multiple APIC Description Table describing processor and interrupt-controller topology, including Local APIC/x2APIC and I/O APIC structures on x86.
Local APIC	Per-logical-processor x86 interrupt controller handling local interrupts, vectors, timers and inter-processor interrupts.
I/O APIC	x86 platform interrupt controller routing external line-based device interrupts to processor APIC destinations.
IPI	Inter-Processor Interrupt sent by one processor to another for coordination such as rescheduling, TLB invalidation or CPU startup.
BSP	Bootstrap Processor: initial logical processor used to bootstrap an x86 multiprocessor system.
AP	Application Processor: additional processor brought online after the bootstrap processor begins system initialization.
SIPI	Startup Inter-Processor Interrupt used in classic x86 multiprocessor startup to begin execution on an application processor.
ACPI	Advanced Configuration and Power Interface: firmware/OS specification for platform topology, devices, power, thermal control, NUMA, interrupts and configuration methods/tables.
OSPM	Operating System-directed configuration and Power Management; ACPI model in which the OS generally chooses policy using interfaces described by firmware.
FADT	ACPI Fixed ACPI Description Table containing fixed platform information and power-management/control fields.
DSDT	Differentiated System Description Table containing the primary ACPI AML definition block for the platform namespace.
SSDT	Secondary System Description Table adding ACPI AML namespace objects/methods to the base platform description.
AML	ACPI Machine Language: bytecode stored in ACPI definition blocks and interpreted by the operating system's ACPI subsystem.
CPPC	ACPI Collaborative Processor Performance Control: abstract interface for requesting/reporting processor performance capabilities rather than exposing only discrete legacy P-states.
µop / micro-op	Internal microarchitectural operation into which a CPU may decode/decompose an architectural instruction.
µop cache / decoded instruction cache	Microarchitectural cache holding already-decoded operations so hot code can bypass some ordinary instruction decoding.
Microcode sequencer	Control mechanism generating internal operation sequences for architecturally complex instructions or exceptional flows.
BTB	Branch Target Buffer: predictor structure caching likely branch/jump targets so fetching can continue before the branch executes.
Reservation station / issue queue	Structure holding decoded/renamed operations until source operands and a suitable execution unit are ready.
Load queue	Structure tracking in-flight loads and supporting ordering, dependency checking, replay and forwarding interactions.
Store buffer / store queue	Structure tracking pending stores, often allowing architectural retirement before the store becomes globally visible.
Context switch	Operating-system change from one running task/thread context to another on a CPU.
SYSCALL instruction	x86 fast system-call instruction transferring control from user execution to an OS-configured privileged entry point.
ECALL	RISC-V environment-call instruction causing an exception into the configured execution environment/privilege handler.
SRET	RISC-V supervisor return-from-trap instruction restoring privilege/interrupt state according to supervisor CSRs.
PMU	Performance Monitoring Unit: processor hardware providing programmable counters for cycles, instructions and microarchitectural events.
Hardware performance counter	Special counter register incremented by selected processor/system events for profiling and diagnosis.
IPC	Instructions Per Cycle: retired instructions divided by elapsed processor cycles over a measurement interval.
CPU affinity	Scheduler constraint specifying which logical CPUs a task is allowed/preferred to execute on.
Disassembly	Translation/display of encoded machine-code bytes as human-readable assembly instructions.
VIH	Minimum input voltage guaranteed to be interpreted as logic HIGH under specified conditions.
VIL	Maximum input voltage guaranteed to be interpreted as logic LOW under specified conditions.
VOH	Guaranteed output-HIGH voltage under stated supply/current/load conditions.
VOL	Guaranteed output-LOW voltage under stated supply/current/load conditions.
Absolute maximum rating	Stress boundary beyond which permanent damage may occur; not a recommended functional operating condition.
Recommended operating condition	Supply, temperature, timing or other range within which normal specified operation is intended/guaranteed.
ABI	Application Binary Interface: machine-level software contract covering calling convention, register use, stack alignment, object formats and related binary compatibility rules.
Caller-saved register	Register whose value a caller must preserve itself if needed after calling another function.
Callee-saved register	Register a called function must restore before returning if it modifies that register.
Stack frame	Per-call region/conventionally organized stack storage holding return/control data, spills, locals, saved registers and arguments as needed.
Red zone	ABI-defined memory below the current x86-64 SysV stack pointer that qualifying leaf code may use temporarily without adjusting RSP.
L2P mapping	Logical-to-physical table mapping a host logical block address to its current physical flash location.
Garbage collection	Flash-controller process relocating still-valid pages so an erase block containing stale pages can be erased and reused.
Wear leveling	Flash-management strategy distributing program/erase cycles across blocks to avoid premature wear concentration.
Write amplification	Ratio/phenomenon in which physical flash writes exceed host logical writes due to relocation, garbage collection, metadata and related work.
Over-provisioning	Physical flash capacity withheld from normal host-addressable space to provide replacement/working room for flash management.
TRIM / discard	Host operation informing a storage device that specified logical blocks no longer contain data the host needs.
Chiplet	Die designed as a modular component of a larger package/system rather than as the entire monolithic chip.
Interposer	Intermediate package-level substrate providing dense routing among dies/chiplets and often external package connections.
Die-to-die PHY	Electrical physical layer implementing short-reach communication between dies inside one package.
UCIe	Universal Chiplet Interconnect Express: industry die-to-die interconnect standard for interoperable chiplets in system-in-package designs.
CXL	Compute Express Link: coherent interconnect family for I/O, accelerator caching of host memory and host access to device-attached memory.
CXL.io	CXL protocol for configuration and I/O access, based closely on PCIe-style mechanisms.
CXL.cache	CXL protocol enabling a device to coherently access/cache host memory.
CXL.mem	CXL protocol enabling a host to coherently access/cache memory attached to a CXL device.
HDM decoder	CXL Host-Managed Device Memory address decoder mapping host/system physical ranges through fabric endpoints to device physical addresses.
Memory tier	Class of memory nodes grouped by performance characteristics such as latency/bandwidth; an OS may treat local DRAM and expansion memory differently.
Static Timing Analysis (STA)	Graph-based timing verification of a digital netlist using characterized cell/interconnect delays and timing constraints, without enumerating functional input sequences.
Setup time	Minimum interval before a capture clock edge during which input data must already be stable.
Hold time	Minimum interval after a capture clock edge during which input data must remain stable.
Timing slack	Difference between required and actual timing; negative slack indicates a timing violation.
PVT corner	Process, Voltage and Temperature condition used to characterize or verify circuit timing/power across manufacturing and operating variation.
Liberty file	Standard-cell library timing/power model format used by synthesis and static timing tools.
SDC	Synopsys Design Constraints format for clocks, I/O delays, false paths, multicycle paths and related timing constraints.
SPEF	Standard Parasitic Exchange Format describing extracted interconnect resistance/capacitance for post-layout timing analysis.
IR drop	Voltage reduction caused by current flowing through finite electrical resistance in a power-delivery path.
PDN	Power Distribution Network: regulator, capacitors, board/package/on-die conductors and related structures delivering power to circuits.
Target impedance	Maximum desired PDN impedance over a frequency range, commonly estimated from allowed voltage deviation divided by transient current demand.
ESR	Equivalent Series Resistance: nonideal resistance associated with a capacitor/inductor or other reactive component.
ESL	Equivalent Series Inductance: parasitic inductance that limits a capacitor's high-frequency effectiveness.
Self-resonant frequency	Frequency where a capacitor's capacitance and parasitic inductance resonate, typically producing its minimum impedance.
ADC	Analog-to-Digital Converter: circuit that samples/quantizes an analog quantity into a digital code.
DAC	Digital-to-Analog Converter: circuit that converts digital code/sample values into analog voltage or current.
Quantization	Mapping a continuous or finely varying amplitude to one of a finite set of representable digital levels.
Aliasing	Sampling ambiguity where frequency content above the uniquely representable band appears as lower-frequency content.
Anti-alias filter	Analog filter preceding an ADC that attenuates frequencies likely to alias into the sampled band.
Reconstruction filter	Analog output filter after a DAC that suppresses images/steps outside the intended output band.
SAR ADC	Successive-Approximation-Register ADC that performs a comparator/DAC-assisted binary search to determine each sample code.
ENOB	Effective Number Of Bits: converter performance metric expressing measured noise/distortion as an equivalent ideal resolution.
PCM	Pulse-Code Modulation: sequence of numeric sample amplitudes representing an analog waveform.
I²S	Inter-IC Sound: synchronous serial digital-audio interface carrying sample data with bit/frame timing signals.
BCLK	Bit clock for a serial digital-audio interface.
LRCLK / word select	Digital-audio framing signal marking sample/channel boundaries, often left-versus-right channel in stereo I²S.
DAI	Digital Audio Interface connecting SoC, codec or DSP using I²S/TDM/PCM-style serial framing.
XRUN	Audio buffer overrun or underrun caused when producer/consumer timing fails to keep up with the continuous sample stream.
VMA	Virtual Memory Area: kernel object describing a contiguous process virtual-address range with common permissions/backing attributes.
mm_struct	Linux kernel object representing a process/shared-thread-group virtual address space and its VMA/page-table context.
Demand paging	Technique of creating/loading physical page mappings only when an access fault shows the page is actually needed.
Demand-zero page	Anonymous page that logically contains zeros and can initially be represented by a shared zero page until a private write occurs.
Zero page	Read-only physical page filled with zeros that may be mapped into many processes to satisfy untouched anonymous reads efficiently.
Copy-on-write (COW)	Sharing a physical page until a write occurs, at which point the writer receives a private copy.
Resident Set Size (RSS)	Amount of a process's mapped memory currently resident in physical RAM, with caveats around sharing/accounting.
Proportional Set Size (PSS)	Memory-accounting metric charging a process its private pages plus a proportional share of shared pages.
Minor page fault	Page fault resolved without reading page contents from backing storage, such as demand-zero or many COW cases.
Major page fault	Page fault requiring storage I/O to obtain needed page data under the operating system's accounting definition.
Overcommit	Policy allowing virtual-memory commitments to exceed immediately available physical RAM under controlled assumptions.
Buddy allocator	Physical-page allocator managing free memory in power-of-two contiguous blocks that split and merge with their buddies.
Memory reclaim	Kernel process of freeing reusable physical pages by dropping cache, writing dirty data, swapping/migrating pages or similar actions.
kswapd	Linux background kernel thread responsible for reclaim activity when memory-zone watermarks indicate pressure.
Swap	Backing store used to preserve evicted anonymous/private page contents outside ordinary DRAM.
Transparent Huge Page (THP)	Kernel-managed use of larger memory pages for eligible mappings to reduce TLB/page-table overhead without explicit hugetlb allocation.
OOM killer	Linux last-resort mechanism selecting process(es) to terminate when memory demands cannot be satisfied through reclaim/other recovery.
PT_LOAD	ELF program-header type describing a loadable segment the OS loader maps into a process.
PT_INTERP	ELF program-header type naming the userspace program interpreter/dynamic linker for a dynamically linked executable.
Auxiliary vector	AT_* key/value data placed by the kernel in a new ELF process image to convey page size, interpreter/base addresses, hardware/platform details and other startup information.
Dynamic linker	Userspace loader such as ld-linux.so that maps required shared libraries, resolves relocations/symbols and transfers control to program startup.
vDSO	Virtual Dynamic Shared Object: kernel-provided code mapped into userspace so selected operations can avoid a full system-call transition.
DMA coherency	Property determining whether CPU caches and device DMA accesses automatically observe mutually consistent memory contents without explicit cache maintenance.
Streaming DMA mapping	Temporary/ownership-oriented DMA mapping optimized for device transfers, with direction and synchronization semantics.
Coherent DMA mapping	DMA-visible memory for which CPU and device memory accesses are kept mutually coherent by the platform, though ordering barriers may still be required.
Cache clean	Cache-maintenance operation writing dirty cached data toward lower memory so another noncoherent agent can observe the latest bytes.
Cache invalidate	Cache-maintenance operation discarding cached copies so subsequent CPU reads obtain newer data written by another agent.
Posted MMIO write	Device register write that can be buffered by CPU/bus fabric and considered complete by the issuer before the target device has actually received it.
dma-fence	Kernel synchronization primitive representing completion of asynchronous DMA/GPU/device work on a shared resource.
Clocksource	Kernel abstraction for a monotonically advancing hardware counter used as the base system timeline.
Clockevent device	Kernel abstraction for programmable timer hardware capable of generating an interrupt at a selected future time.
TSC	x86 Time Stamp Counter read by RDTSC/RDTSCP; modern invariant implementations advance at a constant reference rate independent of core DVFS.
HPET	High Precision Event Timer: x86 platform timer with a fixed-rate main counter and programmable interrupt comparators.
RTC	Real-Time Clock: low-power calendar/timekeeping device that usually continues while the main computer is powered off.
CLOCK_REALTIME	POSIX/Linux wall-clock timeline that can be set/adjusted and therefore may experience discontinuous corrections.
CLOCK_MONOTONIC	Nonsettable monotonic Linux clock suitable for elapsed-time measurement; excludes suspended duration.
CLOCK_BOOTTIME	Linux monotonic clock that includes time spent suspended.
Instruction encoding	Assignment of opcode, register, immediate and function information to specific bit positions in a machine instruction.
funct3 / funct7	RISC-V instruction fields refining operation selection within a major opcode family.
rd	RISC-V destination-register field.
rs1 / rs2	RISC-V source-register fields.
Immediate generator	Decode hardware that extracts, rearranges and sign/zero-extends constant fields from an instruction encoding.
Illegal instruction	Instruction encoding unsupported/reserved/invalid for the current ISA configuration, causing an architecture-defined exception/trap.
ECAM	PCI Express Enhanced Configuration Access Mechanism mapping extended PCI configuration space into memory.
Bus Master Enable	PCI command bit permitting a function to originate memory transactions, including DMA.
Resizable BAR	PCIe capability allowing a supported memory BAR aperture to be resized among device-advertised sizes.
TLB shootdown	Cross-CPU protocol ensuring processors discard stale address translations after shared page-table state changes.
PCID	x86 Process-Context Identifier used to tag TLB translations so address-space switches need not discard unrelated translations.
ASID	Address-Space Identifier tagging TLB entries with an address-space context to reduce global flushing.
Compiler barrier	Construct preventing specified compiler reordering across a point without necessarily emitting a hardware memory-fence instruction.
Volatile	Language qualifier requiring observable volatile accesses according to implementation/language rules; not a portable substitute for atomic synchronization.
Happens-before	Language-level ordering relationship used by concurrency memory models to define when one thread's effects are guaranteed visible to another.
Architectural state	Processor state whose behavior is defined by the ISA/software contract, such as registers, architecturally committed memory and control state.
Microarchitectural state	Implementation-internal state such as caches, TLBs, predictors, queues and replacement/history information not directly specified as ordinary ISA state.
Squash	Discard speculative operations/results after discovering that their control/data speculation was incorrect.
Transient execution	Short-lived speculative execution that does not retire architecturally but can still perturb microarchitectural state.
IBRS	Intel Indirect Branch Restricted Speculation control limiting how indirect-branch predictions can be influenced across privilege/predictor domains.
IBPB	Intel Indirect Branch Predictor Barrier command preventing prior software's indirect-branch prediction history from controlling later software as specified.
STIBP	Intel Single Thread Indirect Branch Predictors mechanism restricting sibling SMT-thread influence on indirect-branch predictions.
RAW hazard	Read-After-Write dependency where a younger instruction needs a value an older instruction has not yet made available through the normal path.
Structural hazard	Pipeline conflict caused when multiple simultaneous operations require the same non-duplicated hardware resource.
Bubble	Intentionally empty/no-op pipeline slot inserted to delay dependent work while preserving correctness.
Bypassing / forwarding network	Datapath muxes/comparators routing recent results directly from later pipeline stages to waiting consumers.
Load-use hazard	Dependency where an instruction immediately needs a value being loaded from memory, often too early for simple forwarding to avoid a stall.
Row hit	DRAM access targeting the row already active in the selected bank.
Row conflict	DRAM access targeting a different row than the one currently active in the same bank, requiring precharge/activate sequence.
Open-page policy	Memory-controller policy that leaves a DRAM row active after access in hopes of later row hits.
Close-page policy	Policy that precharges/closes a DRAM row after access when locality is not expected.
Bank-level parallelism	Ability to overlap useful work across independent DRAM banks with separately active rows/timing state.
Request aging	Memory/interconnect scheduling mechanism that increases priority of old requests to prevent starvation.
Reference plane	PCB plane conductor forming the nearby high-frequency return path and electromagnetic reference for a signal trace.
Controlled impedance	PCB interconnect geometry designed to maintain a specified characteristic impedance.
Microstrip	PCB transmission-line geometry with a surface trace referenced primarily to a plane beneath it.
Stripline	PCB transmission-line geometry with a trace embedded between reference planes.
Stitching via	Reference-plane via placed near a signal layer transition to provide a short high-frequency return-current path.
Return-path discontinuity	Gap or abrupt reference change that forces high-frequency return current away from the corresponding signal route.
Stub	Branch or unused length of transmission line capable of reflecting high-frequency signal energy.
JBD2	Linux journaling layer used by ext4 (and ocfs2) to group and commit recoverable filesystem metadata transactions and replay completed transactions after a crash.
Journal replay	Crash-recovery process applying complete committed journal transactions to restore consistent filesystem metadata state.
Journal checkpoint	Process of writing committed journaled changes to their normal filesystem locations so journal space can be reused.
FUA	Force Unit Access: storage request attribute requiring data to reach nonvolatile media rather than remain only in a volatile device write-back cache.
REQ_PREFLUSH	Linux block-I/O flag requiring prior volatile device-cache writes to be flushed before a new request proceeds.
Crash consistency	Property describing which filesystem/application states can remain after an unexpected crash or power loss.
Torn write	Partially persisted multi-sector/block update containing a mixture of old and new data after interruption.
MSHR	Miss Status Holding Register: cache structure tracking an outstanding cache-line miss and one or more requests waiting for that line.
Non-blocking cache	Cache able to continue servicing some hits and/or additional misses while earlier misses remain outstanding.
Hit under miss	Ability to complete a cache hit while another request is waiting on a cache miss.
Miss under miss	Ability to launch another independent cache miss while an earlier miss is still outstanding.
Miss coalescing	Combining multiple requests for the same absent cache line into one lower-level fetch plus multiple waiting targets.
Line-fill buffer	Temporary structure/path holding an incoming cache line while it is installed or forwarded to requesters.
Prefetcher	Hardware/software mechanism predicting future memory accesses and fetching cache lines before demand requests need them.
Memory-level parallelism	Degree to which a processor/cache system overlaps multiple independent memory accesses or cache misses.
Bimodal predictor	Branch-direction predictor indexed by branch address and storing simple taken/not-taken tendency counters.
Saturating counter	Finite counter that stops at minimum/maximum rather than wrapping, commonly used as branch-prediction hysteresis.
RAS	Return Address Stack: branch-predictor stack specialized for predicting function-return targets.
GHR	Global History Register: encoded recent branch outcomes used to correlate future branch predictions with prior control flow.
GShare	Branch predictor combining branch PC and global history, commonly through XOR/hash, to index a direction-prediction table.
TAGE	TAgged GEometric history-length branch predictor using multiple tagged tables indexed by progressively longer branch histories.
Reset tree	Distribution structure producing reset signals with appropriate scope/timing for many clock and power domains.
Reset-domain crossing (RDC)	Design/verification problem involving reset assertion/deassertion relative to sequential logic in one or more clock domains.
Warm reset	Reset that restarts selected logic while preserving more state than a cold/power-on reset.
Cold reset	Broad reset approximating initial power-on state across most or all relevant system domains.
Watchdog reset	Automatic hardware reset caused when software fails to service a watchdog timer within its required interval.
PLA	Programmable Logic Array with programmable AND and programmable OR planes implementing sum-of-products logic.
PAL	Programmable Array Logic with programmable AND terms feeding fixed OR terms, historically common programmable glue logic.
SPLD	Simple Programmable Logic Device, typically PAL/PLA-like logic and macrocells in a small device.
CPLD	Complex Programmable Logic Device combining multiple programmable-logic blocks/macrocells with programmable routing.
FPGA LUT	Small configurable lookup table whose stored truth-table bits implement a Boolean function of its inputs.
INIT bits	FPGA LUT configuration value specifying output truth-table contents for each input combination.
Carry chain	Dedicated fast FPGA routing/logic path specialized for arithmetic carry propagation among logic elements.
Posted request	PCIe request class that does not require a Transaction-Layer completion response; Memory Write is the classic example.
Non-Posted request	PCIe request class that requires a Completion response; Memory Read is a common example.
Completion TLP	PCIe response packet returning status and optionally data for a Non-Posted request.
PCIe tag	Requester-assigned identifier used to associate Completion traffic with one of several outstanding Non-Posted requests.
Flow-control credit	Receiver-advertised PCIe buffer-capacity unit that limits how much header/data traffic a transmitter may send.
PH/PD/NPH/NPD/CplH/CplD	PCIe Posted, Non-Posted and Completion Header/Data flow-control credit pools.
Endpoint 0 / EP0	Mandatory default bidirectional USB control endpoint used for enumeration and standard control requests.
USB control transfer	Structured USB transfer consisting of SETUP, optional DATA and STATUS stages.
USB descriptor	Standard byte structure describing a USB device, configuration, interface, endpoint, string, capability or class-specific property.
USB interface	Logical function within a USB configuration to which an operating-system class/device driver can bind.
USB configuration	Device-selected collection of interfaces/endpoints and associated attributes activated by SET_CONFIGURATION.
URB	USB Request Block: Linux asynchronous request object submitted to a USB endpoint through usbcore/HCD.
ARP	Address Resolution Protocol mapping an IPv4 next-hop protocol address to a local-link hardware address such as Ethernet MAC.
EtherType	Ethernet frame field identifying the protocol carried in the frame payload, such as IPv4 or IPv6.
Next hop	Immediate router/neighbor to which an IP packet is handed on the current link according to routing.
TCP sequence number	Byte-stream position used by TCP for ordered delivery, acknowledgment and retransmission.
Programmed I/O (PIO)	CPU-driven device data transfer using explicit loads/stores or I/O instructions rather than a DMA engine moving bulk data.
Busy polling	Repeatedly checking a device/completion queue instead of sleeping for an interrupt, trading CPU/energy for potentially lower latency.
IRQ coalescing	Device/driver strategy combining multiple events before interrupting the CPU, reducing interrupt overhead at the cost of some added latency.
SPI NOR	Nonvolatile NOR flash accessed through a serial peripheral interface, commonly used for firmware/boot images.
QSPI / Quad I/O	Serial-flash mode using four data lines per clock phase/transfer to raise throughput over single-bit SPI.
SFDP	Serial Flash Discoverable Parameters: JEDEC-standard table format allowing software to discover flash erase/read/program capabilities.
Page Program	NOR-flash command programming one bounded page span from erased 1s toward 0s.
Sector Erase	Flash command erasing a larger sector/block back to all-1 state before it can be reprogrammed.
WEL	Write Enable Latch in serial flash; must be set before accepted erase/program operations on typical devices.
XIP	Execute In Place: executing code directly from a nonvolatile memory mapping rather than copying all code to RAM first.
Root of trust	Minimal trusted hardware/software/key material from which later verification or security decisions are anchored.
Secure Boot	Boot policy verifying that executable firmware/loader images are authorized before allowing them to run.
Measured Boot	Boot process that hashes measured components into protected registers/logs so later software/verifiers can inspect what ran.
TPM	Trusted Platform Module implementing protected cryptographic keys, PCRs, sealed objects and attestation functions.
PCR	Platform Configuration Register whose value is extended with ordered measurements and can be quoted by a TPM.
PCR extend	Cumulative hash update conceptually PCR_new = Hash(PCR_old \|\| new_digest), making final value dependent on measurement order/history.
UEFI db	Secure Boot authorized signature database containing hashes/certificates permitted to validate UEFI images.
UEFI dbx	Secure Boot forbidden/revocation signature database overriding otherwise trusted images/signers.
KEK	UEFI Secure Boot Key Exchange Key used to authorize updates to signature databases.
Platform Key (PK)	UEFI Secure Boot platform ownership key at the top of the Secure Boot variable authorization hierarchy.
Rollback protection	Security policy preventing installation/boot of an older vulnerable but still cryptographically authentic software/firmware version.
W1C / RW1C	Register-bit behavior where writing 1 clears a bit and writing 0 preserves it.
W1S / RW1S	Register-bit behavior where writing 1 sets a bit and writing 0 preserves it.
RC / read-clear	Register field whose read operation itself clears or consumes the state.
Self-clearing register bit	Command/control bit software sets and hardware automatically clears after recognizing/completing the request.
REGWEN	Register-write-enable/lock field controlling whether protected configuration registers may still be modified.
Reserved bit	Register bit not currently assigned ordinary software semantics; specification defines whether/how software must preserve or write it.
UART	Universal Asynchronous Receiver/Transmitter hardware serializes/deserializes framed asynchronous data without a shared clock line.
8N1	UART framing shorthand for 8 data bits, no parity and one stop bit.
Framing error	UART receive error where expected stop-bit timing/level is invalid for the configured frame.
Oversampling	Receiving technique sampling a serial input several times per bit interval to locate stable bit centers and tolerate clock mismatch/noise.
SPI	Synchronous Serial Peripheral Interface using a controller-supplied clock and usually CS#, MOSI and MISO signals.
CPOL	SPI clock polarity: defines idle level of SCK.
CPHA	SPI clock phase: defines whether sampling occurs on first or second clock transition after selection.
MOSI	SPI Controller/Master Out, Peripheral/Slave In data signal.
MISO	SPI Controller/Master In, Peripheral/Slave Out data signal.
Chip select / CS#	Signal enabling one SPI peripheral and commonly delimiting a serial transaction.
I²C	Two-wire open-drain synchronous serial bus using SDA data, SCL clock, addressing, ACK/NACK and START/STOP conditions.
SDA	I²C serial data line.
SCL	I²C serial clock line.
I²C START	Condition where SDA transitions high-to-low while SCL is high.
I²C STOP	Condition where SDA transitions low-to-high while SCL is high.
Repeated START	I²C START issued without an intervening STOP, preserving control of the bus while starting a new address/direction phase.
ACK / NACK	I²C ninth-clock receiver response: low ACK acknowledges byte; released/high NACK declines/terminates according to context.
Clock stretching	I²C mechanism where a device holds SCL low to delay progress until ready.
Protocol decoder	Software state machine turning captured electrical/logic transitions into higher-level frames, fields and events.
Probe loading	Measurement probe's resistance/capacitance/inductance altering the circuit being observed, potentially changing edge shape or operation.
Futex	Fast userspace mutex mechanism: a 32-bit shared user-memory word plus kernel wait/wake operations used when synchronization requires blocking or waking.
FUTEX_WAIT	Linux futex operation atomically checking a futex word against an expected value and blocking only while it still matches.
FUTEX_WAKE	Linux futex operation making one or more tasks waiting on a futex key/address runnable.
Lost wakeup	Synchronization race where a wake event occurs just before a waiter sleeps and would be missed without an atomic condition-check-and-block protocol.
Priority inversion	Situation where a high-priority task waits on a resource held by a lower-priority task while intermediate-priority work delays the owner.
Priority inheritance	Locking mechanism temporarily boosting the lock owner's effective priority to reduce priority inversion.
Runnable	Task state meaning eligible for CPU execution but not necessarily currently running.
Blocked / sleeping task	Task temporarily removed from runnable execution until a required event/condition occurs.
Runqueue	Per-CPU or scheduler-class runnable-task data structure/state from which work is selected for execution.
Wakeup preemption	Scheduler decision that a newly runnable task should cause the currently running task to be preempted.
EEVDF	Earliest Eligible Virtual Deadline First fair-scheduling approach using service lag for eligibility and virtual deadlines for selection.
Scheduler lag	EEVDF/fair-scheduling accounting indicating whether a task is owed CPU service relative to its fair share.
Virtual deadline	EEVDF value used to select among eligible tasks; the earliest eligible virtual deadline is favored.
Mechanical switch bounce	Rapid repeated contact transitions during switch press/release before the electrical state settles.
Debounce	Filtering/state logic that accepts a mechanical or noisy input transition only after it remains stable for a required interval/criterion.
Schmitt trigger	Input circuit with hysteresis: rising and falling switching thresholds differ, reducing chatter from slow/noisy edges.
Hysteresis	Dependence of switching threshold/state on transition direction/history, providing noise margin between rising and falling decisions.
GPIO	General-Purpose Input/Output pin/peripheral software can configure for digital input/output and often edge/level interrupt detection.
Input filter	Hardware logic requiring a signal to remain stable for multiple samples/cycles before propagating the new digital state.
xHCI	eXtensible Host Controller Interface: standardized host-controller/software interface used for USB 2.0-and-later devices on modern systems.
TRB	xHCI Transfer Request Block: fixed-size descriptor used in command, transfer and event rings.
xHCI Command Ring	Host-produced circular ring containing host-controller management commands such as Enable Slot and Configure Endpoint.
xHCI Transfer Ring	Host-produced per-endpoint/stream ring containing Transfer TRBs describing USB work and DMA buffers.
xHCI Event Ring	Controller-produced ring containing command completions, transfer events and other xHCI event TRBs.
Transfer Descriptor (TD)	One logical xHCI/USB transfer represented by one or more chained Transfer TRBs.
Cycle bit	Circular-ring generation/ownership bit used by xHCI to distinguish newly produced entries as a ring wraps.
xHCI Doorbell	MMIO register write used by host software to tell xHCI that new command or endpoint transfer work has been enqueued.
AHCI	Advanced Host Controller Interface: standardized PCI/MMIO host-controller interface for SATA devices.
AHCI HBA	AHCI Host Bus Adapter acting as the data-movement/command engine between system memory and SATA links/devices.
FIS	SATA Frame Information Structure used for commands, register/status exchange, setup and data traffic on the SATA protocol.
AHCI Command List	Per-port host-memory array containing up to 32 AHCI command headers.
AHCI Command Table	Per-command host-memory structure containing the command FIS, optional ATAPI command and PRDT.
PRDT	AHCI Physical Region Descriptor Table: scatter/gather list of DMA memory regions for a SATA command.
PxCI	AHCI per-port Command Issue register bitmap; setting a slot bit submits that command slot to the HBA.
PxSACT	AHCI per-port SATA Active bitmap representing NCQ-active command tags.
NCQ	SATA Native Command Queuing: tagged command mechanism allowing multiple commands to be outstanding and reordered by the drive.
NVMe Submission Queue	Host-memory queue whose entries are commands submitted to an NVMe controller.
NVMe Completion Queue	Host-memory queue into which an NVMe controller writes command completion status entries.
NVMe doorbell	MMIO register by which host software reports new Submission Queue tail or consumed Completion Queue head positions.
PRP	NVMe Physical Region Page pointer/list format describing command data buffers in host memory.
SGL	Scatter-Gather List descriptor format describing one or more memory/data segments for DMA-capable protocols such as NVMe.
VIPT	Virtually Indexed, Physically Tagged cache: selects a set using untranslated virtual/page-offset bits while validating hits with physical tags.
PIPT	Physically Indexed, Physically Tagged cache: both set selection and tag comparison use a translated physical address.
VIVT	Virtually Indexed, Virtually Tagged cache: fast pre-translation lookup but with substantial alias/homonym/coherence complexity.
Cache synonym	Two different virtual addresses mapping the same physical cache line, potentially creating duplicate cache copies in alias-prone organizations.
Cache homonym	Same virtual address value in different address spaces mapping to different physical memory.
Page coloring	OS allocation/mapping technique controlling selected physical/virtual index bits to avoid cache aliases or partition cache usage.
ECC syndrome	Check result derived from a protected codeword indicating whether and potentially where an error occurred.
CE	Corrected Error: hardware detected corruption and recovered the intended data.
UE	Uncorrected/Uncorrectable Error: detected error beyond the implemented correction capability or otherwise not safely corrected.
Patrol scrub	Background reading/checking of memory so ECC can detect/correct latent errors and refresh corrected contents.
Machine check	Processor hardware-error reporting mechanism for conditions such as cache, memory, interconnect or internal execution failures.
HWPoison	Linux VM mechanism marking a physical memory page as corrupted and isolating it from normal future use.
Page offlining	Removing a suspect/failing physical page from the allocator and active mappings where possible.
Tjunction	Temperature at the semiconductor junction/die, typically the key device thermal-protection quantity.
TjMax	Device-specific maximum junction/thermal-control temperature threshold or reference.
Thermal resistance	Temperature rise per unit dissipated power between two thermal points, typically expressed in °C/W or K/W.
θJA	Junction-to-ambient thermal resistance measured under defined conditions.
θJC	Junction-to-case thermal resistance measured under defined conditions.
TIM	Thermal Interface Material between package/IHS and cooling hardware, reducing contact thermal resistance.
Thermal capacitance	Heat-storage property causing temperature to change over time rather than instantaneously.
Thermal trip point	Temperature threshold triggering a cooling, throttling, shutdown or other thermal-management action.
Allocator chunk	Allocator-managed memory block containing user payload plus implementation-specific size/alignment/metadata state.
Allocator arena	Independent/shared heap-management state and free structures used by an allocator, often one of several in a multithreaded process.
tcache	glibc per-thread cache of selected freed chunk sizes used to accelerate common allocation/free paths.
Internal fragmentation	Unused space inside an allocated chunk due to alignment, metadata or size-class rounding.
External fragmentation	Free memory split among separated holes/chunks that cannot efficiently satisfy larger allocations.
Coalescing	Merging adjacent free allocator chunks into a larger free region.
Allocator trimming	Returning suitably positioned/releasable free arena memory to the operating system.
GOT	Global Offset Table: table of runtime addresses/data used by position-independent code and dynamic linking.
PLT	Procedure Linkage Table: code stubs commonly used to dispatch external function calls through GOT/resolver machinery.
GOTPLT	GOT slots specifically associated with PLT-mediated function resolution in common ELF implementations.
DT_NEEDED	ELF dynamic entry naming a shared object dependency that the runtime linker must load.
R_X86_64_JUMP_SLOT	x86-64 dynamic relocation used for PLT/GOT function binding in the canonical ELF model.
RELRO	Relocation Read-Only: ELF hardening that makes selected data read-only after runtime relocations are applied.
Lazy binding	Deferring applicable external-function symbol resolution until the first time a function is called.
Eager binding	Resolving applicable dynamic symbols during object/program loading, e.g. via BIND_NOW or -z now.
Symbol interposition	Dynamic-link lookup behavior where a definition from one object can override a reference/definition from another under applicable ELF rules.
IFUNC	GNU indirect function symbol whose runtime resolver selects the final implementation address.
bzImage	Common x86 Linux compressed bootable kernel image format containing setup/entry/decompressor plus kernel payload.
boot_params	Linux/x86 boot-protocol structure carrying loader-provided parameters into the kernel; historically called the zero page.
initramfs	Early-userspace cpio archive unpacked into the kernel's initial rootfs; its `/init` can discover or assemble the real root filesystem and hand control to the final init.
rootfs	Kernel's initial root filesystem instance, populated by initramfs and used before/through transition to a persistent root filesystem.
initcall	Kernel function registered into an ordered initialization level and invoked during boot for built-in subsystems/drivers.
__init	Linux kernel annotation for code/data needed only during initialization so its memory can later be discarded/reclaimed.
kthreadd	Early Linux kernel thread that participates in creation/management ancestry of many subsequent kernel threads.
PID 1	First userspace process on Linux; init system with special boot/service and child-reaping responsibilities.
Signal disposition	Per-signal action specifying default behavior, ignore, or a user-installed handler.
Pending signal	Signal generated for a process/thread but not yet delivered.
Signal mask	Per-thread set of blocked signals whose delivery is temporarily deferred.
Signal frame	Architecture-specific user-stack structure containing saved execution context for a signal handler invocation.
Signal trampoline	Userspace code executed after a signal handler returns, normally invoking rt_sigreturn.
rt_sigreturn	Linux system call restoring registers, signal mask and stack/context from the signal frame after a handler.
SA_SIGINFO	sigaction flag requesting a three-argument handler with siginfo_t and user-context information.
SA_RESTART	`sigaction()` flag that requests transparent restart for selected interrupted interfaces; it does not apply to every syscall.
sigaltstack	Alternate user stack that can host signal handlers, especially useful when the ordinary stack is damaged/exhausted.
Async-signal-safe	Function/operation guaranteed safe to call from a signal handler even when it interrupts another operation at an arbitrary point.
Folio	Linux memory-management object representing one or more physically contiguous base pages, used extensively by page-cache code.
address_space	Kernel object connecting an inode/file to cached folios and filesystem page-cache operations.
read_folio	Filesystem address-space operation that fills one page-cache folio with data from backing storage.
Readahead	Reading likely-future file folios into the page cache before an application explicitly demands them.
Extent	Compact mapping representing a contiguous range of logical file blocks backed by contiguous physical filesystem blocks.
Major file fault	File-backed page fault requiring storage I/O before the missing file page can be mapped.
IDTR	x86 privileged register containing the Interrupt Descriptor Table base address and limit.
IDT	x86 Interrupt Descriptor Table indexed by an interrupt/exception vector.
Interrupt gate	x86 IDT descriptor transferring control to an interrupt handler while applying interrupt-gate flag semantics.
Trap gate	x86 IDT descriptor similar to an interrupt gate but with different IF behavior.
IST	x86-64 Interrupt Stack Table providing dedicated stack pointers selectable by IDT gates.
IRETQ	x86-64 interrupt-return instruction restoring saved instruction pointer, flags, stack/privilege state as applicable.
EOI	End Of Interrupt acknowledgment telling an interrupt controller that servicing of the active interrupt is complete.
AER	PCI Express Advanced Error Reporting capability for standardized link/transaction error status, source logging and recovery.
Correctable PCIe error	PCIe error hardware/protocol can recover without loss of function/data, though it may be logged.
Non-fatal PCIe error	Uncorrectable error invalidating a transaction/function state while the link/hierarchy can remain usable.
Fatal PCIe error	Uncorrectable PCIe error making the link/hierarchy unreliable and normally requiring reset/recovery.
PCI error recovery callback	Driver callbacks such as error_detected(), mmio_enabled(), slot_reset() and resume() used during bus error recovery.
FLR	PCIe Function Level Reset targeting one function without resetting an entire upstream bus when supported.
Retention time	Time a DRAM cell can preserve readable stored charge under specified conditions before restoration/refresh is needed.
tREFI	DRAM refresh-interval timing parameter describing the required refresh cadence/budget.
tRFC	DRAM refresh-cycle time during which affected banks/ranks cannot service ordinary commands.
Self-refresh	Low-power DRAM mode in which the memory device internally maintains refresh.
Fine-granularity refresh	DRAM refresh mode trading shorter/more-frequent refresh operations against longer/less-frequent ones.
Row disturbance	DRAM reliability effect where frequent activation of a row electrically perturbs neighboring rows.
Rowhammer	Class of DRAM row-disturbance behavior where repeated aggressor-row activations can cause victim-row bit flips on susceptible memory.
TRR	Target Row Refresh family of mitigation techniques intended to refresh likely victim rows near heavily activated aggressors.
RFM	Refresh Management mechanism coordinating extra disturbance-oriented refresh behavior between memory controller and DRAM in newer standards.
SVA	Shared Virtual Addressing: device and CPU use the same process virtual addresses under coordinated IOMMU/MMU translation.
SVM	Shared Virtual Memory: broader accelerator programming concept in which CPU and device can access a shared pointer/address space.
Device TLB	Device-side cache of IOMMU/virtual-address translations requiring coherent invalidation when mappings change.
HMM	Linux Heterogeneous Memory Management infrastructure for shared address spaces, device page-table mirroring and device-memory migration.
ZONE_DEVICE	Linux VM representation for memory associated with devices, including device-private memory variants.
PF	SR-IOV Physical Function: full PCI function controlling and managing one or more Virtual Functions.
VF	SR-IOV Virtual Function: lighter PCIe function intended for isolated datapath/virtualization use.
VFIO	Linux framework exposing direct userspace/VMM device access under IOMMU protection.
IOMMU group	Set of devices Linux treats as one minimum isolation unit for safe direct assignment.
vfio-pci	Generic Linux PCI driver exposing a PCI function through VFIO rather than a normal host device driver.
ACS	PCIe Access Control Services capability helping control peer/upstream transaction routing and isolation.
File descriptor	Per-process small integer referencing an open file description.
Open file description	Kernel open-file object holding state such as current offset and status flags; Linux represents it with struct file.
Dentry	VFS directory-entry cache object representing a pathname component/name-to-inode relationship.
RCU-walk	Linux pathname-resolution fast path using RCU-based traversal of stable cached dentries.
REF-walk	Linux pathname-resolution mode taking normal references/locks when RCU-walk cannot safely continue.
FD_CLOEXEC	Descriptor-local flag causing that descriptor to be closed across a successful execve().
O_CLOEXEC	open-time flag atomically creating a descriptor with close-on-exec enabled.
SQE	io_uring Submission Queue Entry describing one asynchronous operation.
CQE	io_uring Completion Queue Entry containing the result of a submitted operation.
SQPOLL	io_uring mode using a kernel thread to poll the Submission Queue so active submissions can avoid enter syscalls.
IOPOLL	io_uring mode polling supported device completions rather than relying on normal interrupt completion.
Fixed/registered buffer	User memory registered with io_uring to reduce repeated setup/pinning overhead for compatible operations.
Fixed/registered file	File reference registered in an io_uring instance to reduce repeated descriptor lookup/reference overhead.
Multishot io_uring operation	One submission capable of producing multiple completion entries for repeated events.
Page-table walker (PTW)	Hardware or privileged software mechanism that reads page-table entries after TLB misses and refills translation caches.
Page-walk cache	Translation-side cache of intermediate page-table entries or walk state that reduces memory references during later TLB misses.
Shared/L2 TLB	Larger second-level translation cache serving misses from smaller L1 instruction/data TLBs.
Store buffer	Microarchitectural queue holding stores before they become globally visible, allowing the core to continue execution.
Load Queue (LDQ)	Structure tracking in-flight loads and ordering/replay state in an out-of-order load/store unit.
Store Queue (STQ)	Structure tracking in-flight store addresses/data/commit state before stores drain into the memory hierarchy.
Store-to-load forwarding	Supplying a younger load directly from an older matching buffered store instead of waiting for the cache/memory copy.
Memory disambiguation	Prediction/checking that determines whether younger loads can execute before all older store addresses are fully resolved.
Memory-order replay	Re-execution/flush required when speculative load/store ordering is later discovered to have violated a true dependency.
epoll	Linux scalable readiness-notification facility maintaining persistent interest and ready sets for open file descriptions.
epoll interest list	Set of files/event masks registered for monitoring in an epoll instance.
epoll ready list	Subset/references currently reportable as ready to epoll_wait().
EPOLLET	epoll edge-triggered mode; callers generally use nonblocking I/O and drain until EAGAIN.
EPOLLONESHOT	epoll mode disabling a registration after one delivered event until explicitly rearmed.
EPOLLEXCLUSIVE	epoll registration mode intended to reduce excessive multi-waiter wakeups in supported scenarios.
PCS	Ethernet Physical Coding Sublayer performing speed-specific digital coding/alignment and sometimes in-band negotiation.
PMA	Ethernet Physical Medium Attachment layer handling serializer/deserializer and physical attachment/timing functions.
PMD	Ethernet Physical Medium Dependent layer implementing medium-specific electrical/optical signaling.
RGMII	Reduced Gigabit Media Independent Interface: common parallel-style on-board connection between Gigabit Ethernet MAC and PHY.
SGMII	Serial Gigabit Media Independent Interface: serial MAC↔PHY/PCS interface often carrying in-band link information.
MDIO	Management Data Input/Output interface used to configure/read Ethernet PHY registers, normally with MDC clock.
Ethernet auto-negotiation	Link-partner protocol advertising capabilities and resolving a mutually supported speed/duplex/pause operating mode.
Link training	High-speed PHY adaptation/calibration process used by selected Ethernet/backplane/copper link types after or alongside negotiation.
NPTL	Native POSIX Threads Library: modern glibc Linux pthread implementation using a 1:1 kernel-thread model.
TCB	Thread Control Block: per-thread runtime structure associated with TLS and thread-library state.
Thread pointer	Architecture register/state pointing into a thread's TLS/TCB area, such as FS base on x86-64.
CLONE_THREAD	Linux clone flag placing a new task in the same thread group/TGID as the caller.
CLONE_SETTLS	Linux clone flag installing architecture-specific thread-local-storage state for the new task.
CLONE_CHILD_CLEARTID	Linux clone flag causing thread exit to clear a userspace TID word and futex-wake waiters.
RCU	Read-Copy Update synchronization technique allowing very cheap read-side access and deferring reclamation until a grace period.
RCU grace period	Interval after which all relevant read-side critical sections that began before it have completed.
RCU quiescent state	Observation establishing that a CPU/task cannot still be inside a relevant old RCU read-side critical section.
rcu_dereference	RCU pointer-load primitive providing the compiler/memory-ordering semantics required for published objects.
rcu_assign_pointer	RCU publication primitive ordering object initialization before a new pointer becomes visible to readers.
call_rcu	Kernel API scheduling a callback after a future RCU grace period.
SRCU	Sleepable RCU flavor that supports sleeping read-side critical sections under its own tracking rules.
MSI-X table	PCIe BAR-mapped array containing per-vector interrupt message address, data and mask control.
PBA	MSI-X Pending Bit Array with one pending bit per vector.
IRTE	Interrupt Remapping Table Entry mapping/validating a remappable device interrupt to CPU destination/vector policy.
Interrupt remapping	IOMMU/APIC security/routing mechanism translating and validating device-generated interrupt messages.
IRQ affinity	Kernel policy assigning an interrupt to one or more CPUs.
Snoop protocol	Cache-coherence design where relevant caches observe broadcast/ordered coherence requests.
Directory protocol	Coherence design maintaining sharer/owner metadata so requests can target only relevant caches.
Sharer vector	Directory metadata bitset or compressed representation identifying caches/nodes holding shared copies.
Home node	Coherence agent responsible for serializing transactions and directory/memory actions for an address range.
Snoop filter	Directory-like metadata structure that suppresses probes to caches known not to contain a line.
GetS	Directory-coherence request for shared/read permission and valid cache-line data.
GetM / GetX	Directory-coherence request for exclusive/modified write permission, usually requiring invalidation of existing sharers.
Invalidation acknowledgment	Message confirming a cache has processed/serialized an invalidation before a writer gains exclusive permission.
Transient coherence state	Temporary controller state while coherence request/data/invalidation/ack messages are in flight.
Hardirq	Linux hard interrupt context entered directly from architecture IRQ handling; cannot sleep and should remain bounded.
Softirq	Linux deferred interrupt-context mechanism executed per CPU for classes such as networking, timers and selected RCU processing.
ksoftirqd	Per-CPU kernel thread used to execute pending softirqs when immediate interrupt-exit processing is insufficient or deferred.
Threaded IRQ	IRQ handling mode where a short primary interrupt handler wakes a dedicated kernel thread to perform longer/sleepable work.
Workqueue	Linux asynchronous kernel work mechanism backed by managed worker pools/kworker threads.
kworker	Kernel worker thread executing queued workqueue items.
BH workqueue	Workqueue mode executing work in bottom-half/softirq context rather than normal worker-thread context.
hrtimer	Linux high-resolution timer object maintained in nanosecond time and ordered by expiration.
timerfd	Linux file descriptor representing timer expiration events, usable with read/poll/epoll.
Timer wheel	Kernel data structure optimized for large numbers of coarse timeout-style timers rather than precise ordered deadlines.
NO_HZ	Linux dynamic-tick mode reducing unnecessary periodic scheduler tick interrupts during idle or selected full-dynticks operation.
DVFS	Dynamic Voltage and Frequency Scaling: coordinated adjustment of operating voltage and frequency according to performance/power constraints.
Clock divider	Hardware dividing an input/reference clock to produce a lower output frequency.
Clock mux	Multiplexer selecting one of several clock sources for a clock domain.
Integrated clock-gating cell	Clock-control primitive designed to enable/disable a clock without producing illegal runt/glitch pulses.
CPUFreq	Linux CPU performance-scaling subsystem containing core policy infrastructure, governors and hardware-specific scaling drivers.
schedutil	Linux CPUFreq governor using scheduler utilization to request CPU performance.
HWP	Intel Hardware-Managed P-states mechanism where processor hardware selects performance within software-provided policy/hints.
CPUIdle	Linux subsystem choosing and entering processor idle states when a logical CPU has no runnable work.
Idle target residency	Minimum expected idle duration required for a deeper idle state to repay its entry/exit energy overhead.
Idle exit latency	Worst-case time from a wake event until the CPU can resume executing instructions from an idle state.
CPUIdle governor	Policy algorithm selecting an idle state based on predicted idle duration and latency constraints.
CPUIdle driver	Platform-specific code that requests the selected hardware idle state.
Package C-state	Idle state affecting power/clock resources shared by multiple cores in a processor package.
Wait queue	Linux scheduler-linked queue of tasks/callbacks waiting for a condition or event.
wait_queue_head	Kernel wait-queue object containing queue/list and synchronization state.
wait_queue_entry	One waiter/callback entry attached to a Linux wait queue.
TASK_INTERRUPTIBLE	Linux sleeping task state in which an unblocked signal can wake/interupt the wait.
TASK_UNINTERRUPTIBLE	Linux sleeping state not interrupted by normal signal delivery.
TASK_KILLABLE	Linux sleep state interruptible by fatal signals while ignoring ordinary nonfatal signal wakeups.
Completion	Linux done-counter plus wait queue used to signal that an asynchronous activity reached a required point.
complete()	Kernel completion operation posting one completion token and waking a waiter.
complete_all()	Kernel completion operation satisfying all current and future waiters until the object is reinitialized.
Direct map	Kernel virtual range mapping ordinary physical RAM with a simple architecture-defined linear/offset relationship.
Linear map	Another name for the kernel direct mapping of normal physical memory on architectures that provide one.
vmalloc	Kernel allocator providing a contiguous virtual range backed by pages that need not be physically contiguous.
vmap	Kernel mechanism mapping an array/set of existing physical pages into one contiguous kernel virtual range.
ioremap	Kernel mechanism mapping device/bus MMIO into an architecture-appropriate __iomem CPU access token.
__iomem	Linux sparse annotation/token distinguishing I/O memory from ordinary kernel RAM pointers.
vmemmap	Kernel virtual mapping holding struct page metadata describing physical memory pages.
fixmap	Reserved fixed kernel virtual-address slots whose underlying physical mappings can be installed/changed for special purposes.
kmalloc	General-purpose kernel allocator for small physically contiguous/direct-map-backed objects from size-class slab caches.
kzalloc	kmalloc-family allocation with requested bytes zero-initialized.
kmem_cache	Kernel object cache describing a fixed-size object type and its slab allocation state.
SLUB	Common modern Linux slab allocator implementation using per-CPU fast paths and page-backed object slabs.
Slab	Page or page group assigned to one slab cache and subdivided into reusable fixed-size object slots.
Partial slab	Slab containing a mixture of allocated and free object slots.
Per-CPU freelist	CPU-local list of free slab objects enabling fast allocation/free with little shared contention.
GFP_KERNEL	Normal kernel allocation flag allowing blocking/reclaim where appropriate.
GFP_ATOMIC	Non-sleeping kernel allocation flag for atomic/interrupt-sensitive contexts with more limited reclaim options.
SLAB_HWCACHE_ALIGN	Slab-cache flag requesting object alignment based on hardware cacheline considerations.
crt1.o	C runtime startup object that conventionally supplies the ELF _start entry for normal hosted executables.
_start	Low-level executable entry symbol that receives initial process ABI state and invokes runtime/libc startup rather than being main().
__libc_start_main	glibc routine coordinating startup initialization, constructors, main() invocation and normal termination.
.init_array	ELF section/array containing constructor function pointers executed before main().
.fini_array	ELF section/array containing destructor function pointers executed during normal termination/unloading.
.preinit_array	ELF initialization array for main-executable functions that run before normal init processing.
atexit	C library mechanism registering callbacks to run during normal exit() processing.
exit_group	Linux syscall terminating all threads in a process/thread group and reporting process exit status.
Buddy order	Exponent N meaning a contiguous free/allocation block contains 2^N base pages.
Per-CPU page list	CPU-local cache of free pages used to reduce contention in the zone buddy allocator for common allocations.
Migratetype	Linux page-allocation classification separating movable, unmovable, reclaimable and special-use pageblocks/free lists to reduce fragmentation.
Pageblock	Large physical-memory grouping whose migration type guides compaction, CMA and anti-fragmentation policy.
Physical compaction	Migration of movable pages so scattered free pages can be combined into larger contiguous buddy blocks.
MAP_SHARED	mmap mode where writes are shared with other mappings of the same object and file-backed changes can be written back to the file.
MAP_PRIVATE	mmap mode providing private copy-on-write behavior: writes are not propagated to the underlying file or other private mappers.
File-backed COW	Private mmap write fault that replaces a shared clean file page with a private anonymous copy for the writing process.
skb headroom	Reserved unused bytes before packet data so lower protocol layers can prepend headers cheaply.
skb fragment	Packet payload region stored in a separately referenced memory page/page fragment rather than the linear head buffer.
skb_clone	Fast packet clone creating new skb metadata while sharing underlying packet data through reference counts.
GSO	Generic Segmentation Offload: Linux represents a large packet and postpones splitting it into MTU-sized packets.
TSO	TCP Segmentation Offload: NIC hardware segments a large TCP skb into multiple wire packets.
GRO	Generic Receive Offload: Linux combines compatible received packets into larger skb representations before upper-stack processing.
INVPCID	x86 instruction invalidating TLB translations selected by PCID/address or broader context.
CR3 no-flush	PCID-enabled page-table-root switch preserving valid tagged TLB entries instead of discarding them automatically.
TLB generation	Linux version counter used to decide whether a CPU's cached translations for an address space remain current.
Spinlock	Kernel mutual-exclusion primitive that waits by spinning rather than sleeping under traditional non-RT semantics.
raw_spinlock_t	Linux spinlock preserving non-preemptible raw spin semantics even where PREEMPT_RT changes ordinary spinlock_t.
Sequence counter	Odd/even version counter allowing lockless readers to retry if a writer overlapped their snapshot.
seqlock	Sequence counter plus writer-serialization spinlock, providing retrying lockless readers and serialized writers.
Lockdep	Linux runtime lock validator tracking lock classes, IRQ-context usage and acquisition-order dependencies.
ABBA deadlock	Two paths acquire locks in opposite orders, allowing each to hold one while waiting forever for the other.
spin_lock_irqsave	Spinlock acquisition variant saving/disabling local interrupt state to prevent same-CPU interrupt deadlock.
spin_lock_bh	Spinlock acquisition variant disabling local softirq/bottom-half processing while the lock is held.
DMA mapping	Transformation from CPU-owned memory representation into device-visible DMA addresses with platform/IOMMU/cache rules.
dma_addr_t	Linux type representing a DMA address usable by a device, not an ordinary CPU pointer.
Scatterlist	Kernel list describing one logical buffer across multiple pages or memory segments.
dma_map_sg	DMA API mapping a scatterlist to device-visible segments, potentially merging entries and returning a smaller segment count.
DMA mask	Address-width/range constraint describing what DMA addresses a device can generate.
SWIOTLB	Linux software bounce-buffer layer for DMA that cannot directly target the original memory.
Bounce buffer	Temporary DMA-accessible memory copied to/from an original buffer to satisfy device/address/security constraints.
MMIO read-back flush	Safe non-posted device read used to ensure relevant prior posted writes reached the device.
writel_relaxed	Relaxed MMIO write accessor with weaker ordering against normal memory/DMA than writel on architectures where that distinction matters.
ioremap_wc	Write-combining device mapping allowing more aggressive write merge/reorder behavior for suitable apertures such as framebuffers.
mmiowb	Kernel primitive for ordering MMIO writes across critical sections on architectures requiring explicit MMIO write serialization.
Driver core	Linux infrastructure matching generic device objects to registered drivers and coordinating probe/remove, sysfs and power-management relationships.
Driver probe	Callback that verifies/initializes a matched device and returns success only after the device is ready for the driver to own.
Deferred probe	Driver-core mechanism retrying probe later after a driver returns -EPROBE_DEFER because a supplier/resource is not ready.
Device link	Kernel supplier-consumer relation used to constrain probe, runtime-power and teardown ordering.
Modalias	Device identity string mapped by module tools to one or more kernel module aliases.
devres	Linux managed device-resource framework behind many devm_* helpers, releasing resources automatically at detach.
Kernel module	Loadable privileged code image that can extend the running kernel without rebuilding/rebooting it.
.ko	Conventional filename extension for a Linux loadable kernel object/module.
vermagic	Kernel module compatibility string checked against the running kernel unless force-loading policy overrides it.
EXPORT_SYMBOL	Kernel mechanism publishing a symbol so compatible loadable modules can resolve references to it.
Module relocation	Architecture-specific patching of module code/data after final kernel virtual addresses are assigned.
Module reference count	Count/dependency state preventing ordinary unload while code/data remain in active use.
Module taint	Kernel diagnostic state recording conditions such as unsigned, out-of-tree or forced modules.
kobject	Reference-counted kernel object embedded by many exported subsystems and represented in sysfs.
uevent	Kernel notification to userspace describing an object/device add, remove, change, move or related action.
systemd-udevd	Userspace daemon consuming kernel uevents and applying udev rules/policy.
devtmpfs	Kernel-supported filesystem maintaining basic character/block device nodes under /dev.
Major number	Character/block device identifier selecting a registered kernel device-driver family.
Minor number	Device number interpreted within the major/family to select one device/instance/partition.
udev coldplug	Boot-time replay/processing of events for hardware already present before normal hotplug monitoring.
IA32_LSTAR	x86 MSR holding the native 64-bit SYSCALL entry address.
IA32_STAR	x86 SYSCALL/SYSRET MSR containing segment-selector-related configuration.
IA32_FMASK	x86 SYSCALL MSR specifying RFLAGS bits cleared on entry.
entry_SYSCALL_64	Linux x86-64 low-level assembly entry for native SYSCALL.
do_syscall_64	Linux common x86-64 syscall dispatcher after low-level register/context setup.
pt_regs	Architecture-specific kernel saved-register structure used across syscall, interrupt, exception, signal and tracing paths.
SYSRETQ	x86-64 fast privilege-return instruction used by Linux when the user context is safe for SYSRET.
GPU command buffer	GPU-readable memory containing engine-specific packets/instructions describing rendering, compute, copy or synchronization work.
drm_sched_entity	DRM GPU scheduler client/context queue whose jobs are submitted in order to one or more hardware schedulers.
drm_sched_job	Kernel representation of one schedulable GPU submission and its dependencies/fences.
dma_fence	Kernel asynchronous-completion primitive signaled when hardware/DMA work reaches a defined completion point.
dma_resv	Reservation object associating synchronization fences with a shared GPU/dma-buf buffer.
sync_file	File descriptor wrapping a synchronization fence for explicit userspace synchronization.
drm_syncobj	DRM synchronization object whose current fence may be updated or advanced.
GPU doorbell	MMIO or memory notification used to tell GPU/firmware that new queue/ring work is available.
blk_mq_ctx	blk-mq software staging/submission context, normally associated with one CPU.
blk_mq_hw_ctx	blk-mq hardware dispatch context corresponding to a device submission queue or group of such resources.
Block request tag	Integer identifying an in-flight blk-mq request for efficient completion lookup.
Block plugging	Short-term collection of block I/O so adjacent requests can be merged before device dispatch.
THP	Transparent Huge Pages: Linux VM-managed large pages/folios that can be allocated, promoted, split and demoted automatically.
khugepaged	Background Linux kernel thread scanning eligible mappings and collapsing smaller pages into THPs.
HugeTLB	Explicit Linux huge-page subsystem using separately managed/reserved pools.
hugetlbfs	Pseudo-filesystem providing mappings backed by HugeTLB pages.
TLB reach	Amount of memory that can be covered by the set of translations currently held in a TLB.
pipe_inode_info	Linux kernel object holding a pipe's buffer ring, reader/writer counts, limits, lock and wait queues.
pipe_buffer	One Linux pipe-ring entry describing a page plus offset/length and page-lifetime operations.
PIPE_BUF	POSIX threshold below which qualifying writes to a pipe/FIFO are atomic relative to other writers.
F_SETPIPE_SZ	Linux fcntl command requesting a different pipe capacity subject to per-user/system limits.
Page fault	Synchronous CPU exception raised when virtual-address translation or access permission checks cannot complete the memory operation normally.
CR2	x86 control register containing the linear address associated with a page fault.
SEGV_MAPERR	SIGSEGV si_code indicating the fault address is not mapped.
SEGV_ACCERR	SIGSEGV si_code indicating an address is mapped but the attempted access violates permissions.
VM_FAULT_MAJOR	Linux VM fault result flag marking a page fault that required backing-store I/O.
VM_FAULT_RETRY	Linux VM fault result requesting architecture fault code retry after lock/state changes.
__user pointer	Linux type-analysis annotation identifying a pointer whose pointee lives in userspace and must be accessed through uaccess rules.
access_ok	Architecture helper checking whether a user pointer/range is plausibly in the permitted userspace range.
copy_from_user	Linux fault-aware copy from userspace to kernel memory; return value counts bytes not copied.
copy_to_user	Linux fault-aware copy from kernel memory to userspace; return value counts bytes not copied.
SMAP	x86 Supervisor Mode Access Prevention preventing normal supervisor data accesses to user mappings unless explicitly enabled.
STAC	x86 instruction setting AC so kernel code can temporarily access user pages while SMAP is enabled.
CLAC	x86 instruction clearing AC and restoring SMAP protection against supervisor access to user pages.
Exception table	Kernel table mapping selected potentially faulting instructions to recovery/fixup code.
Self-modifying code	Code that writes bytes in memory that will subsequently be fetched/executed as instructions.
Cross-modifying code	One execution context modifies instructions another CPU/thread may execute, requiring inter-context synchronization.
FENCE.I	RISC-V instruction synchronizing earlier visible stores with later instruction fetches on the same hart.
Instruction-cache coherence	Mechanism ensuring newly written executable bytes become visible to instruction fetch/decode.
W^X	Write-xor-execute design principle avoiding writable and executable permissions simultaneously.
qdisc	Linux network queueing discipline between protocol stack and device transmit queue.
ndo_start_xmit	Network-driver transmit callback accepting an skb and programming device/DMA resources for eventual transmission.
TX descriptor ring	NIC-consumed ring describing transmit buffer DMA addresses, lengths and control/offload metadata.
BQL	Linux Byte Queue Limits mechanism dynamically limiting bytes outstanding in a hardware network transmit queue.
NETDEV_TX_OK	Driver return value meaning it accepted responsibility for an skb; it does not mean transmission already completed.
Effective UID	User ID consulted by many Linux permission and privilege checks.
Saved set-user-ID	Stored UID enabling controlled privilege-drop/regain semantics under setuid-family rules.
Supplementary groups	Additional group IDs considered by discretionary permission checks.
Effective capability set	Per-thread set of Linux capabilities currently active for privileged-operation checks.
Permitted capability set	Capabilities a thread may potentially make effective/inheritable under Linux capability rules.
Capability bounding set	Per-thread ceiling limiting which capabilities can be gained across execve.
Ambient capability set	Capabilities designed to persist across non-privileged execve under strict eligibility rules.
File capability	security.capability extended attribute granting selected Linux capabilities on execve.
PID namespace	Namespace providing a process-ID number hierarchy and namespace-local init/PID 1.
Mount namespace	Namespace giving a process tree its own view of mounted filesystems and mount propagation.
Network namespace	Namespace isolating network interfaces, routing, sockets/ports and associated network stack state.
UTS namespace	Namespace isolating hostname and NIS domain name.
IPC namespace	Namespace isolating System V IPC and POSIX message-queue resources.
Time namespace	Namespace virtualizing offsets of selected clocks such as boottime and monotonic.
cgroup v2	Linux unified hierarchical mechanism for organizing processes and applying resource accounting/control.
cpu.max	cgroup-v2 hard CPU-bandwidth quota/period control.
cpu.weight	cgroup-v2 relative fair-class CPU share under contention.
memory.high	cgroup-v2 memory pressure/throttling threshold intended to trigger reclaim rather than directly kill.
memory.max	cgroup-v2 hard memory limit that can lead to cgroup-local OOM handling when reclaim cannot satisfy it.
pids.max	cgroup-v2 maximum task count enforced by the pids controller.
Pressure Stall Information	Linux metrics quantifying time tasks are stalled by CPU, memory or I/O resource pressure.
seccomp	Linux mechanism restricting which system calls a process/thread may execute.
no_new_privs	Sticky task bit preventing execve from granting additional privilege via setuid/setgid/file capabilities.
SECCOMP_RET_ALLOW	Seccomp filter action allowing the syscall to continue.
SECCOMP_RET_ERRNO	Seccomp filter action suppressing syscall execution and returning a selected errno.
SECCOMP_RET_USER_NOTIF	Seccomp action forwarding a blocked syscall to a userspace supervisor notification fd.
Container	Process environment assembled from host-kernel isolation/resource/security primitives rather than a separate guest kernel.
NX / XD	Page permission preventing instruction fetch/execution from selected virtual-memory mappings.
SMEP	x86 Supervisor Mode Execution Prevention blocking supervisor execution from user-accessible pages.
CR0.WP	x86 Write Protect control making supervisor writes respect read-only page protections.
PKU	x86 protection keys for user pages: PTE key tags plus per-thread PKRU data-access restrictions.
PKRU	x86 per-thread register containing Access Disable/Write Disable bits for each userspace protection key.
LSM	Linux Security Modules framework providing stackable security hooks throughout kernel operations.
LSM hook	Security-sensitive kernel callback site at which enabled security modules can allow/deny or update security state.
SELinux	Linux Security Module implementing label/type-based mandatory access control.
AppArmor	Linux Security Module implementing profile/task-centered mandatory access control.
Landlock	Stackable Linux Security Module allowing unprivileged processes to add filesystem/network restrictions to themselves.
Yama	Linux Security Module supplying selected system-wide discretionary-access hardening such as ptrace_scope.
ASLR	Address Space Layout Randomization: per-exec randomization of userspace memory-region addresses.
KASLR	Kernel Address Space Layout Randomization: boot-time randomization of kernel code/module/layout addresses.
randomize_va_space	Linux sysctl selecting userspace ASLR mode 0, 1 or 2.
PIE	Position-Independent Executable, typically ELF ET_DYN main binary that can load at a randomized base.
CET	x86 Control-flow Enforcement Technology including shadow stack and indirect branch tracking.
Shadow Stack	Hardware-protected secondary stack of return addresses checked against the normal stack at return.
IBT	x86 Indirect Branch Tracking requiring valid indirect call/jump targets to begin at ENDBR landing pads.
ENDBR64	x86 CET instruction marking a valid 64-bit indirect-branch landing site.
#CP	x86 Control Protection exception raised for CET control-flow violations.
Runtime PM	Linux framework opportunistically suspending/resuming individual idle devices while the rest of the system remains operational.
PCI D0	Full-power PCI device state.
PCI D3hot	Low-power PCI state with main power present and configuration space accessible but normal function decoding disabled.
PCI D3cold	Lowest-power PCI condition with main device supply removed and device context generally lost.
PME	PCI/PCIe Power Management Event used for wake signaling.
ASPM	PCI Express Active State Power Management for lowering link power independently of device D-state.
NUMA first touch	Placement effect where a demand-paged physical page is allocated on/near the node of the CPU first faulting it under default policy.
MPOL_BIND	Linux NUMA policy restricting allocations to a specified node set.
MPOL_INTERLEAVE	Linux NUMA policy distributing allocations across selected nodes.
MPOL_PREFERRED	Linux NUMA policy preferring a selected node but allowing fallback.
move_pages	Linux syscall for querying or migrating individual process pages between NUMA nodes.
Automatic NUMA balancing	Linux mechanism sampling access locality and moving tasks/pages to reduce remote-memory cost.
eventfd	Linux file descriptor wrapping a kernel-maintained 64-bit event counter for wait/notify and epoll integration.
EFD_SEMAPHORE	eventfd mode causing each successful read to return 1 and decrement the event counter.
signalfd	Linux file descriptor delivering selected blocked signals as readable signalfd_siginfo records.
BPF_PROG_LOAD	bpf() operation asking the kernel to verify and install an eBPF program.
eBPF verifier	Kernel abstract interpreter proving admitted BPF programs obey memory, control-flow, type and resource-safety rules.
tnum	Verifier representation describing scalar bits known to be zero/one versus unknown.
BPF map	Kernel-managed storage object shared between BPF programs and/or userspace.
BTF	BPF Type Format metadata for typed kernel/BPF objects, functions and CO-RE relocations.
BPF JIT	Architecture backend translating verified eBPF instructions into native CPU machine code.
CO-RE	Compile Once – Run Everywhere: BTF-based BPF relocations adapting one object to compatible kernel type layouts.
PAT / memory type	x86 Page Attribute Table mechanism used to select per-page cacheability such as write-back, write-combining or uncached behavior; crucial for mapping device memory correctly.
TTY / PTY	TTY is the kernel terminal abstraction; a PTY is a master/slave virtual terminal pair used by terminal emulators, SSH and related software.
line discipline	TTY processing layer that can implement canonical line editing, echo, terminal-generated signals and related termios behavior.
O_DIRECT	Linux file-open flag requesting I/O that minimizes page-cache effects and transfers data directly between storage and userspace buffers subject to filesystem/device constraints.
System suspend	Global low-power transition that freezes normal execution and suspends devices/CPUs as a coordinated system operation, unlike per-device runtime PM.
s2idle	Linux suspend-to-idle state: userspace/devices are suspended and CPUs enter deep idle without requiring a firmware-defined deep platform suspend state.
Hibernation	System sleep method that saves a snapshot of RAM to persistent storage so memory itself can lose power and later be restored.
PTE young / accessed	Page-table state indicating use of a mapping; Linux can test and clear it to obtain recency hints for memory-management decisions.
PTE dirty	Page-table state indicating that a mapped page has been written; distinct from CPU cache-line dirty state and filesystem page-cache dirty state.
TCP listen backlog	On Linux TCP, the limit requested by listen() for fully established connections waiting in the completed accept queue, capped by somaxconn.
Accept queue	Kernel queue of established connections waiting for a server process to obtain connected socket descriptors with accept()/accept4().
Dirty throttling	Kernel feedback mechanism that slows tasks dirtying page-cache memory when writeback cannot drain modified data to storage fast enough.
Microcode update	Vendor-supplied processor patch applied through a CPU-defined update mechanism to change selected internal implementation behavior after fabrication; on x86 Linux it is preferably loaded very early during boot.
Entropy pool / CSPRNG	Kernel random state mixes unpredictable seed inputs and initializes a cryptographically secure pseudorandom generator, which then produces application random bytes such as those returned by getrandom().
cwnd	TCP congestion window: sender-side limit on how much data may be in flight according to congestion-control state; distinct from the receiver-advertised flow-control window.
RTO	TCP retransmission timeout derived from round-trip-time estimates; expiry triggers loss recovery and the timeout is backed off after repeated failures.
SACK	TCP Selective Acknowledgment: option that reports received byte ranges beyond a gap so the sender can target retransmission of missing data.
virtio	Standard paravirtual device interface in which a guest driver and device/backend exchange buffers through shared-memory virtqueues rather than emulating one specific legacy device model.
virtqueue	Virtio shared-memory queue containing buffer descriptors plus driver/device availability and completion state; split and packed formats are standardized.
DMA-BUF	Linux shared-buffer object exposed to userspace as a file descriptor so multiple devices/subsystems can access one allocation without mandatory CPU copying.
dma_fence / sync_file	Kernel asynchronous-completion primitive and its userspace file-descriptor carrier, used to order access to shared device buffers.
sendfile / splice	Linux data-movement interfaces that can avoid routing payload bytes through a userspace bounce buffer; implementations may still copy where required.
TIME-WAIT	TCP state retained by an active closer after the FIN exchange to absorb delayed duplicates and allow retransmission of the final ACK before the old connection incarnation is forgotten.
Zombie process	Terminated child whose parent/reaper has not yet collected its retained exit status and accounting record with a wait-family operation.
Subreaper	Linux process designated to adopt orphaned descendant processes before they fall back to the normal namespace reaper/init role.
Hard link / link count	A hard link is one directory name for an inode; an inode's link count tracks how many such namespace links exist, independently of open file descriptors.
unlink()	Removes a directory entry/name. If no hard links remain but the file is still open, storage reclamation is postponed until live references disappear.
DNS resolver	Software that turns DNS questions into answers. A host's stub resolver commonly asks a recursive resolver, which can answer from cache or pursue referrals toward authoritative servers.
DNS TTL	Resource-record cache lifetime indicating how long DNS data may normally be reused before it should be refreshed.
TLS handshake	Protocol exchange that negotiates cryptographic parameters, establishes shared keying material and authenticates peers as required before protected application records are exchanged.
TLS record layer	Layer that frames and applies authenticated encryption to application data using traffic keys produced by the TLS handshake.
pidfd	Linux PID file descriptor: a stable file-descriptor reference to a specific task/process, useful for polling, signaling and other operations without relying only on a reusable numeric PID.
FIB / route lookup	Forwarding Information Base and associated routing lookup logic that selects how an IP destination should be reached, including an output interface and usually a next hop.
Neighbor table	Per-link mapping/reachability state connecting a next-hop IP address to link-layer delivery information such as an Ethernet MAC address.
UDP	User Datagram Protocol: message-oriented transport over IP with ports and checksums but no built-in reliable byte stream, retransmission, ordering repair or connection handshake.
AF_UNIX / SCM_RIGHTS	Local Unix-domain sockets plus ancillary-message descriptor passing, allowing one process to transfer references to already-open kernel objects to another.
DHCP	Dynamic Host Configuration Protocol. Commonly leases an IPv4 address and supplies parameters such as subnet mask, routes and DNS resolver addresses.
Netfilter	Linux kernel packet-hook framework used by nftables and related subsystems for filtering, NAT, logging, queueing and other packet processing.
conntrack	Linux flow/connection tracking state that associates packets with bidirectional protocol flows and exposes states used by stateful firewalling and NAT.
SNAT / DNAT	Source or destination network-address translation: rewrite the source endpoint for outgoing traffic or the destination endpoint for incoming/redirected traffic.
memfd	Linux anonymous file-descriptor-backed memory object created by memfd_create(); it can be mapped, shared and optionally sealed without a persistent pathname.
file seal	Kernel-enforced restriction on later mutation of a sealable file/memfd, such as preventing growth, shrinkage or writes.
discard / TRIM	Storage-layer indication that specified logical blocks no longer contain data the host needs preserved; useful for SSD FTL reclamation and thin provisioning.
SLAAC / DAD	IPv6 Stateless Address Autoconfiguration forms addresses from advertised prefixes; Duplicate Address Detection checks a tentative address for conflicts before ordinary use.
Router Advertisement (RA)	ICMPv6 Neighbor Discovery message by which routers announce their presence and supply prefix/default-router and related configuration information to hosts.
inotify	Linux file-descriptor API for receiving filesystem change events from watched files/directories; event queues can overflow and are not a durable transaction log.
OFD lock	Open-file-description byte-range lock acquired with fcntl(); unlike traditional process-associated POSIX record locks, ownership follows the shared open file description.
core dump / ET_CORE	Filtered snapshot of a terminating process's selected memory and machine/process state, commonly encoded as an ELF ET_CORE file for debugger analysis.
QUIC	Secure multiplexed transport over UDP. QUIC supplies streams, flow control, acknowledgments, loss recovery, congestion control and TLS-derived packet protection rather than relying on UDP for reliability.
OverlayFS	Linux overlay filesystem that merges lower and upper directory trees; modifications can copy lower objects into the upper layer, while whiteouts hide deleted lower names.
fanotify	Linux filesystem-notification interface supporting broad marks and, for selected event types, synchronous userspace allow/deny permission decisions.
FUSE	Filesystem in Userspace: kernel VFS operations are translated into requests serviced by a userspace filesystem daemon, with caching/direct-I/O modes negotiated through the FUSE interface.
Path MTU (PMTU)	The smallest link MTU along a source→destination path; senders use PMTU information to choose packet sizes that do not require fragmentation.
veth	Linux virtual Ethernet pair: transmitting a frame on one endpoint causes it to be received on the peer endpoint, often across network namespaces.
Linux bridge / FDB	Kernel Layer-2 software switch and its forwarding database, which learns source MAC→port associations and forwards frames by destination MAC, optionally scoped by VLAN.
swap entry	Non-present page-table/kernel metadata identifying an evicted anonymous page by swap type and offset so a page fault can recover its contents.
zswap	Compressed RAM cache in front of a real swap backend; accepted swap-out pages can stay compressed in memory until faulted back or evicted to backing swap.
zram	Compressed RAM-backed block device. It can itself be configured as swap and does not require a disk backing device.
dm-crypt	Linux Device Mapper target that transparently encrypts block writes and decrypts block reads beneath filesystems.
LUKS	Linux Unified Key Setup metadata/key-management format commonly used to unlock a volume key and activate a dm-crypt mapping.
openat2 / RESOLVE_*	Linux pathname-open interface whose resolution flags can constrain symlinks, mount crossings and escape above a supplied directory file descriptor during one VFS lookup.
Netlink	Linux AF_NETLINK kernel↔userspace message transport used by many control/event APIs; families define typed operations and attributes, while multicast groups deliver asynchronous notifications.
XDP / AF_XDP	XDP is an early eBPF network receive hook; AF_XDP can redirect selected XDP frames into userspace descriptor rings backed by registered UMEM packet buffers.
Linux md / RAID	Kernel software RAID layer that combines multiple block devices using mirroring, striping and/or parity, exposing one logical block device while tracking degraded state and rebuild/recovery.
Devicetree / DTB	Declarative hardware description passed by firmware/bootloader to an OS; a DTB/FDT encodes nodes and properties such as compatible, reg and interrupts so non-self-enumerating platform hardware can be populated and matched to drivers.
ioctl	File-descriptor system call for device/subsystem-specific control or query operations that do not fit generic read/write/mmap semantics; the request code and payload structure form a userspace/kernel ABI.
mount namespace / propagation	Per-process-group view of filesystem mount attachments; shared/slave/private/unbindable propagation modes determine whether later mount/unmount events cross between related mount trees.
Device Mapper / LVM	Device Mapper is the kernel block-remapping framework that routes logical sector ranges through targets; LVM2 is higher-level userspace metadata/allocation policy that builds PV/VG/LV abstractions and programs DM mappings.
TUN / TAP	Linux virtual network devices backed by userspace file descriptors: TUN exchanges Layer-3 IP packets, while TAP exchanges Layer-2 Ethernet frames.
RSS / RPS / RFS / XPS	Complementary network queue/CPU steering mechanisms: hardware Receive-Side Scaling, software Receive Packet Steering, application-locality-aware Receive Flow Steering and Transmit Packet Steering.
ptrace	Linux tracer/tracee execution-control interface used by debuggers and syscall tracers to stop threads, inspect or modify execution state, and resume them under permission checks.
NFS	Network File System: a remote filesystem protocol integrated with the local VFS/page cache, translating file operations into RPCs to a server instead of exposing remote disk blocks directly.
GPT	GUID Partition Table: partition metadata with primary/backup headers, CRCs and per-partition type/unique GUIDs plus starting/ending LBAs.
ESP	EFI System Partition: firmware-readable system partition, conventionally FAT-formatted, containing UEFI executable boot files and related data.
process group	Set of processes sharing a PGID, commonly one shell job or pipeline; signals and terminal foreground/background rules can operate on the whole group.
session	Collection of process groups sharing a SID; a session can own one controlling terminal and has one foreground process group at a time.
dm-verity	Read-only Device Mapper target that verifies block-device data against a Merkle tree rooted in a trusted digest.
fs-verity	Filesystem support for read-only files whose blocks/pages are verified against a per-file Merkle tree and stable file digest.
PSI	Pressure Stall Information: Linux accounting of time workloads lose while stalled on CPU, memory or I/O resource contention.
KMS	Kernel Mode Setting: Linux DRM interface for configuring display modes, planes, CRTCs, connectors and atomic scanout state.
CRTC	KMS display-pipeline object that combines planes and owns mode timing/scanout state; historical name survives from CRT controllers.
vblank	Vertical blanking interval/event between displayed frames; a common synchronization point for page flips and display-state updates.
NTP	Network Time Protocol: measures network clock offset/delay and feeds algorithms that discipline a local clock toward reference time.
PTP / PHC	Precision Time Protocol and PTP Hardware Clock: PTP targets tighter synchronization, often using NIC hardware timestamps and a device clock such as /dev/ptp0.
RDMA	Remote Direct Memory Access: queue-based network I/O where an RDMA-capable NIC can DMA directly to/from registered application memory.
Queue Pair (QP)	RDMA endpoint containing a send queue and receive queue to which work requests are posted.
Memory Region (MR)	Application memory registered for RDMA device access, associated with permissions and local/remote access keys.
userfaultfd	Linux file-descriptor interface that lets userspace receive and resolve selected page-fault events for registered virtual-memory ranges.
DNSSEC	DNS Security Extensions: signed DNS RRsets plus DS/DNSKEY trust chaining and authenticated denial-of-existence, providing origin authentication/integrity rather than confidentiality.
DAX	Direct Access: Linux path for directly byte-addressable storage that can map file/device-backed page frames without ordinary page-cache copies.
tracepoint / ftrace	A tracepoint is a static kernel instrumentation site; ftrace/tracefs provide tracing infrastructure and function/event recording around such observability mechanisms.
perf_event_open	Linux syscall ABI that exposes hardware/software/tracepoint performance events as file descriptors for counting or sampled ring-buffer delivery.
SMM / SMI	System Management Mode / System Management Interrupt: x86 firmware execution entered outside the normal OS interrupt/privilege path, using protected management memory and returning with RSM.
memory compaction	VM operation that migrates movable pages so scattered free pages can coalesce into larger physically contiguous buddy blocks; unlike reclaim, it need not reduce used memory.
tmpfs / shmem	Linux virtual-memory-backed filesystem/shared-memory machinery whose resident data lives in RAM and may normally be swapped; no ordinary block filesystem backs the file contents.
BSS / BSSID	802.11 Basic Service Set and its identifier; in infrastructure Wi-Fi, a station associates with a specific BSS/AP instance before carrying normal network data.
CSMA/CA	Carrier Sense Multiple Access with Collision Avoidance: contention/backoff family used by Wi-Fi stations to share a radio channel rather than transmit as if on a dedicated wire.
PCIe hotplug	Runtime insertion/removal of PCIe functions. Safe removal requires software lifetime teardown; surprise removal means the link/device can disappear before that teardown completes.
Multicast / IGMP / MLD	One-to-many IP delivery model where receivers join group addresses; IGMP reports IPv4 memberships and MLD reports IPv6 memberships to neighboring multicast routers.
KSM	Kernel Samepage Merging: Linux scans opted-in anonymous pages for identical contents, maps matches to one write-protected physical page, and breaks sharing with copy-on-write on modification.
Reflink	Filesystem clone in which distinct files initially share physical extents and use copy-on-write when one file modifies a shared range.
sparse file / hole	A file may have logical ranges with no allocated data blocks; reads from those holes return zeros. This differs from an allocated unwritten extent and from reflink sharing.
MPTCP	Multipath TCP: one application-visible reliable byte stream carried across one or more ordinary TCP subflows, with connection-level data sequencing and path management.
kernel keyring	Linux key-retention object that links typed kernel keys into searchable thread/process/session/user lifetime scopes under dedicated permissions and quotas.
CPU hotplug	Coordinated logical CPU online/offline lifecycle that migrates tasks/interrupts/timers and runs ordered subsystem callbacks; distinct from idle C-states or DVFS.
real-time scheduling	Linux scheduling classes/policies such as SCHED_FIFO, SCHED_RR and SCHED_DEADLINE that prioritize deterministic latency or reserved CPU service over ordinary fair sharing; distinct from PREEMPT_RT kernel preemptibility.
VXLAN / VNI / VTEP	VXLAN carries an inner Ethernet frame across a routed IP underlay using UDP; VNI is its 24-bit overlay identifier and a VTEP encapsulates/decapsulates tunnel traffic.
NBD	Network Block Device: protocol/driver boundary that exposes a remote byte range as a local-looking block device, leaving the client to run its filesystem/block stack above it.
fscrypt	Linux filesystem-level encryption framework that applies policies and keys to selected directory trees, transparently encrypting file contents and filenames inside supporting filesystems.
VSOCK	Virtual-machine socket address family (AF_VSOCK) for host↔guest communication using virtualization-specific addressing rather than IP networking.
CID (VSOCK)	Context ID identifying a VSOCK communication domain such as a host or guest; paired with a VSOCK port to name an endpoint.
HTTP/2	Binary-framed HTTP mapping that multiplexes multiple HTTP streams over one connection and uses HPACK field compression.
HTTP/3	HTTP mapping over QUIC streams, preserving HTTP semantics while avoiding TCP's single ordered-byte-stream transport model.
HPACK	HTTP/2 field-compression format using static/dynamic tables and indexed representations.
QPACK	HTTP/3 field-compression format adapted from HPACK for QUIC's independently delivered streams.
FQ-CoDel	Linux qdisc combining per-flow fair queueing with CoDel active queue management to control persistent queue delay.
HTB	Hierarchy Token Bucket, a classful Linux qdisc for hierarchical rate guarantees, ceilings and link sharing.
access ACL	POSIX-style ACL governing discretionary access to one file or directory, extending owner/group/other mode bits with named users/groups and an ACL mask.
default ACL	ACL attached to a directory that supplies inherited initial ACL entries for newly created children.
xattr	Extended attribute: persistent name:value metadata associated with an inode, used for arbitrary user metadata and features such as ACLs, labels and file capabilities.
trust anchor	Public key/name information accepted as a root of trust by local configuration and used as the endpoint of certificate-path validation.
certification path	Ordered sequence from an end-entity certificate through issuer certificates to an acceptable trust anchor.
SAN	subjectAltName: X.509 extension carrying service identities such as DNS names; current TLS identity checking uses the appropriate SAN form rather than Common Name fallback.
OCSP	Online Certificate Status Protocol: request/response mechanism for obtaining status information about certificates.
mlock	Userspace VM operation that keeps mapped pages resident in RAM; it is not the same contract as a DMA/GUP page pin.
FOLL_PIN	Linux GUP-internal pinning mode used by pin_user_pages*() to track pages whose data is accessed under DMA/direct-I/O-style pins.
FOLL_LONGTERM	More restrictive long-duration page-pin mode layered on FOLL_PIN, used for cases such as conventional long-lived RDMA registration.
MSG_ZEROCOPY	Linux socket-send flag requesting payload copy avoidance by temporarily sharing user-backed pages with the transmit stack and reporting asynchronous release completion.
SO_ZEROCOPY	Socket option that opts a socket into the MSG_ZEROCOPY API before per-send zerocopy flags are honored.
Bluetooth LE	Low Energy portion of Bluetooth using advertising/scanning and scheduled link-layer connections; many applications exchange data through ATT/GATT rather than IP.
HCI (Bluetooth)	Host Controller Interface: standardized command/event/data boundary between host Bluetooth software and the controller.
ATT / GATT	Attribute Protocol transports operations on typed attributes; Generic Attribute Profile organizes those attributes into discoverable services, characteristics and descriptors.
IPsec	IP-layer security architecture applying policy-selected protection to IP packets using Security Associations and protocols such as ESP.
Security Association (IPsec)	Unidirectional bundle of IPsec transformation state including SPI, algorithms/keys, mode, peer and sequence/replay state.
XFRM	Linux framework for packet transformations such as IPsec, exposing policy and state objects configured through Netlink/iproute2.
user namespace	Linux namespace that remaps UID/GID identity and establishes a scoped capability domain; UID 0 inside need not be host root.
subuid / subgid	Delegated subordinate ID ranges that rootless namespace tooling can map into child user namespaces through approved helpers.
condition variable	Thread synchronization object used with a mutex to sleep until shared state may satisfy a predicate; waking requires rechecking that predicate.
spurious wakeup	Condition-variable wait returning without implying the application predicate is true; one reason waits belong in a loop.
USB Type-C	Reversible connector/cable ecosystem with CC-based attach/orientation/role detection; connector shape alone does not specify data speed or negotiated power.
CC / Configuration Channel	USB Type-C sideband connection used for attachment/orientation/current-role signaling and USB Power Delivery communication.
USB Power Delivery (PD)	Negotiation/control protocol that lets compatible Type-C partners establish power contracts and exchange capability/mode messages.
memory overcommit	Policy allowing virtual-memory promises to exceed immediately available physical backing, relying on demand paging and later resource availability.
CommitLimit	Linux system-wide commit ceiling used by strict overcommit mode; visible in `/proc/meminfo`.
Committed_AS	Linux estimate of the amount of memory promised/committed to processes, distinct from how many pages are currently resident.
POSIX shared memory	Named shared-memory objects opened with `shm_open()` and normally mapped into participating processes with `mmap(MAP_SHARED)`.
LAG	Link Aggregation Group: multiple point-to-point links presented as one logical Layer-2 link.
LACP	Link Aggregation Control Protocol used by IEEE 802.1AX aggregation peers to negotiate and maintain active member links.
bond	Linux logical network interface that combines member interfaces using policies such as active-backup or 802.3ad/LACP.
kernel panic	Kernel-level fatal condition in which Linux decides it cannot safely continue normal execution.
kexec / kdump	kexec transfers directly to another loaded kernel; kdump uses a crash-loaded capture kernel to preserve and export the crashed kernel's memory.
loop device	Linux block device whose sectors are backed by offsets in a regular file or another block object.
AF_PACKET	Linux socket family for sending and receiving link-layer packets directly at a network interface.
PACKET_MMAP / TPACKET	Memory-mapped AF_PACKET ring interface that batches packet transfer through shared ring slots instead of one receive syscall per frame.
filesystem quota	Per-filesystem accounting/enforcement of space or inode consumption by user, group or project identity.
project quota	Quota identity attached to filesystem objects/directories independently of their UID/GID ownership, commonly inherited by a directory tree.
request_firmware()	Linux driver API that obtains a named firmware/data blob so a device driver can upload it to peripheral hardware or consume device-specific configuration data.
efivarfs	Linux filesystem view of UEFI variables, normally mounted at `/sys/firmware/efi/efivars`; accesses firmware-managed variable state rather than files on the EFI System Partition.
SMB3	Modern Server Message Block protocol family used for remote file/share access, including negotiated security, caching leases and reconnect-capable open-handle mechanisms.
WireGuard	Layer-3 encrypted tunnel interface that associates peer public keys with AllowedIPs prefixes and carries authenticated encrypted packets over UDP.
ICMP	Internet-layer control protocol used for selected error reports and diagnostics; it provides feedback about IP delivery conditions but does not make IP reliable.
Echo Request / Reply	ICMP informational message pair used by ping to observe reachability and round-trip behavior.
Time Exceeded	ICMP error indicating TTL/Hop Limit expiration or related timeout; traceroute deliberately induces this hop by hop.
traceroute	Diagnostic technique/tool that varies TTL/Hop Limit and interprets ICMP responses to infer successive forwarding hops.
SSH	Secure Shell protocol family layering encrypted host-authenticated transport, user authentication and multiplexed logical channels.
SSH host key	Server identity key used by SSH transport authentication; clients commonly remember/verify it through known_hosts policy.
SSH channel	Multiplexed logical stream inside one SSH connection, used for shells, commands and forwarded connections.
SCSI	Command/status storage and peripheral architecture in which initiators issue CDBs to targets/LUNs over a chosen transport.
CDB	SCSI Command Descriptor Block containing an operation code and command parameters.
LUN	Logical Unit Number selecting a logical device/object behind a SCSI target.
sense data	Structured SCSI diagnostic information explaining CHECK CONDITION and other exceptional command outcomes.
eMMC	Embedded managed-flash device using the MMC protocol family, exposing logical block storage plus device-management features.
EXT_CSD	Extended eMMC configuration/status register space containing capabilities and controls such as partition, cache and reliability settings.
RPMB	Replay Protected Memory Block: authenticated eMMC storage with a monotonic write counter for small security-sensitive state.
idmapped mount	Linux mount with a user-namespace-derived ID mapping attached to it so VFS ownership is translated for that mount without recursively changing inode ownership on disk.
MOUNT_ATTR_IDMAP	`mount_setattr()` attribute that attaches an ID mapping, selected through a user-namespace file descriptor, to a supported mount.
rseq	Linux restartable-sequences ABI for very small userspace critical sections that can be aborted/restarted when preemption or migration invalidates a per-CPU assumption.
kTLS	Linux kernel TLS record-layer data path installed on an established TCP socket after TLS handshake state/traffic keys are available.
TLS ULP	Linux TCP Upper Layer Protocol hook used by kTLS to attach TLS record processing to a socket.
Zoned Namespace (ZNS)	NVMe namespace model divided into zones, including sequential-write zones whose next write location is constrained by a device-maintained write pointer.
zone write pointer	Per-zone position indicating where the next sequential write belongs in a sequential-write-required zoned block device.
zone reset	Zoned-storage operation that returns a zone to an empty/reusable state and resets its write pointer instead of overwriting old LBAs arbitrarily.
Livepatch	Runtime kernel update mechanism that redirects selected functions while a consistency model moves tasks safely from old to new code.
Livepatch transition	Interval in which tasks are converging to a patched or unpatched state; completion means all relevant tasks have reached the target state.
Livepatch shadow variable	Auxiliary state associated with an existing kernel object so a livepatch can carry new per-object data without changing the original structure layout in place.
Seccomp user notification	Seccomp action that blocks a matching syscall and sends a request to a userspace listener for brokered handling.
SECCOMP_IOCTL_NOTIF_ADDFD	User-notification ioctl that installs an fd supplied by the supervisor into the blocked target task.
Memory balloon	Virtual device through which a guest voluntarily removes pages from its usable RAM set so the host can reclaim backing memory.
Balloon inflate / deflate	Inflate gives guest pages to the balloon; deflate returns previously ballooned pages to the guest allocator.
Free-page reporting	Virtualization mechanism that tells the host which guest pages are already free so their backing may be reclaimed without permanently ballooning them.
NVMe-oF	NVMe over Fabrics: the NVMe controller/queue/command model carried over a fabric transport instead of only local PCIe.
NQN	NVMe Qualified Name: persistent textual identifier for an NVMe host or subsystem.
NVMe discovery controller	Controller that supplies discovery records describing NVMe subsystems and fabric endpoints a host may connect to.
NVMe/TCP vs NVMe/RDMA	Two NVMe-oF transport mappings: one uses TCP/IP sockets; the other uses RDMA transport and registered-memory mechanisms.
vhost	Host-side virtio acceleration framework that services selected virtqueues outside the ordinary QEMU userspace data path, commonly in the host kernel.
vhost-user	Protocol for connecting a virtio frontend such as QEMU to a separate userspace backend that maps shared guest memory and consumes virtqueues.
TPM sealing	Protecting data in a TPM object whose release is gated by an authorization policy, commonly including selected PCR state.
TPM quote	Signed TPM attestation over selected PCR state plus qualification data such as a verifier nonce.
Attestation Key (AK)	TPM signing key used to produce attestation statements such as PCR quotes; its public key still needs an external trust rationale.
procfs	Linux pseudo-filesystem exposing process and selected runtime kernel state, including namespace-sensitive PID views and /proc/sys controls.
sysfs	Linux pseudo-filesystem exposing kobjects, devices, buses, classes and documented kernel attributes through a structured hierarchy.
debugfs	Kernel developer/debug pseudo-filesystem intentionally not treated as a stable production userspace ABI.
configfs	Userspace-driven kernel-object configuration filesystem where mkdir/rmdir can create and destroy subsystem objects.
SELinux context	Security label commonly written user:role:type:level and attached to subjects/objects for SELinux policy decisions.
Type Enforcement	SELinux policy model expressing which subject domains/types may perform which permissions on which object types.
AVC	SELinux Access Vector Cache holding computed policy decisions for subject/object/class/permission combinations.
Confidential VM / CoCo VM	Virtual machine whose private memory/execution state receives hardware-backed protection from a host-side adversary that ordinary virtualization would normally trust.
SEV-SNP	AMD confidential-VM architecture adding encrypted guest state plus Secure Nested Paging ownership/integrity protections and attestation.
TDX	Intel Trust Domain Extensions: confidential-VM architecture using the TDX module/SEAM boundary, private/shared memory and TD attestation.
Live migration	Moving a running VM between hosts while transferring RAM, vCPU and device state with only a bounded switchover pause.
Pre-copy migration	Copies RAM while the source VM continues running, then retransmits pages dirtied after earlier copies before a final stop-and-copy phase.
Post-copy migration	Starts destination CPUs before all RAM is present; accesses to missing pages fault and fetch those pages from the source.
memcg / memory cgroup	cgroup v2 memory-controller domain that charges and controls memory usage, reclaim pressure, swap and scoped OOM behavior for a workload hierarchy.
Btrfs	Linux copy-on-write filesystem using tree-structured metadata, checksums, shared extents, subvolumes and snapshot/replication features.
Btrfs subvolume	Independent file/directory tree inside one Btrfs filesystem; subvolumes share the same storage pool and can share extents.
Btrfs scrub	Online read-and-verify pass that checks Btrfs data/metadata and can repair a bad replica when a verified redundant copy exists.
CRIU	Checkpoint/Restore In Userspace: Linux tooling that serializes supported process-tree state and reconstructs equivalent tasks/resources later.
Checkpoint/restore	Saving enough execution/resource state to stop an execution context and later rebuild it so computation resumes from the saved point.
Memory hotplug	Runtime addition or removal of physical/system RAM ranges, including separate add/remove and online/offline allocator states.
ZONE_MOVABLE	Linux page-allocation zone restricted to migration-compatible allocations so physical memory ranges have a better chance of being offlined later.
DoT	DNS over TLS: DNS messages carried through an authenticated encrypted TLS transport.
DoH	DNS over HTTPS: DNS query/response exchanges mapped into HTTPS requests and responses.
DoQ	DNS over QUIC: DNS mapped onto dedicated encrypted QUIC connections.
virtio-fs	Virtio file-system device carrying FUSE-style file operations between a guest kernel and host-side backend for shared host directories.
virtiofsd	Host-side userspace daemon commonly serving virtio-fs requests, typically through the vhost-user transport.
UEFI capsule	UEFI-defined container for passing firmware-update or other firmware-consumed payloads from an OS-present environment to platform firmware.
ESRT	EFI System Resource Table: firmware-published inventory of updateable firmware resources with GUID, version and last-attempt status information.
fwupd	Linux userspace firmware-update daemon/framework that discovers supported devices and stages vendor firmware through mechanisms such as UEFI capsules.
systemd unit	Named object managed by systemd, such as a service, socket, target, timer, mount or device, with state and dependency relationships.
service unit	systemd unit describing how a process/daemon is started, supervised, stopped and optionally restarted.
socket activation	Service-manager pattern in which the listening IPC/network endpoint is created first and the service process is launched on demand with the open descriptor passed to it.
PAM	Pluggable Authentication Modules: Linux library/API that lets privilege-granting applications invoke configurable authentication, account, credential, password and session policy modules.
PAM service	Name supplied by a PAM-aware application to select the corresponding module-stack policy.
PAM conversation	Application callback that PAM modules use for prompts/input/output independently of a specific terminal or graphical UI.
IMA	Linux Integrity Measurement Architecture: policy-driven runtime measurement and optional appraisal of files/kernel data, with measurement logs and optional TPM PCR extension.
IMA appraisal	IMA enforcement mode that validates expected integrity metadata/signatures for selected objects before allowing policy-covered use.
EVM	Extended Verification Module: Linux integrity mechanism protecting integrity-sensitive inode metadata and security extended attributes with HMACs/signatures.
security.ima	Extended attribute commonly carrying an IMA file hash or signature used by appraisal policy.
security.evm	Extended attribute carrying EVM authentication data protecting selected security metadata/xattrs.
IMA measurement list	Kernel-maintained ordered runtime log of policy-selected integrity measurements used for local inspection or remote attestation.
TEE	Trusted Execution Environment: isolated trusted software environment with a defined interface to a less-trusted host OS.
TrustZone	ARM security architecture separating secure and non-secure security states and enabling platform resources to be partitioned between them.
OP-TEE	Open-source trusted operating system commonly used as a TrustZone-based TEE on ARM platforms.
secure world	TrustZone secure security state used by trusted firmware/TEE software and secure resources according to platform design.
normal world	TrustZone non-secure security state where a conventional OS such as Linux commonly runs.
SMC	Secure Monitor Call: ARM instruction/interface used to request services that cross into secure-monitor/trusted-firmware handling.
audit rule	Linux Audit filter selecting syscall, path, task or related events for kernel audit recording.
auditd	Userspace daemon that receives Linux Audit records and writes/dispatches them according to configuration.
audit record	One typed Linux Audit record; several records can belong to one logical audited event.
I/O scheduler	Optional blk-mq policy layer that can merge, reorder, delay or prioritize block requests before driver dispatch.
mq-deadline	blk-mq I/O scheduler combining batching/locality with deadline-style request aging to reduce starvation and latency.
BFQ	Budget Fair Queueing: blk-mq scheduler providing proportional-share service and latency/fairness policies.
I/O priority	Per-task/class priority metadata used by supporting block I/O schedulers to influence service order/share.
watchdog timer	Hardware countdown that triggers reset or another recovery action unless software periodically proves liveness.
watchdog heartbeat	Keepalive action that refreshes a watchdog before its timeout expires.
nowayout	Watchdog policy preventing an armed timer from being disabled, preserving failure recovery if the supervising process dies.
pretimeout	Optional watchdog warning interval/event before final expiry, often used to collect diagnostics before reset.
GPE	ACPI General-Purpose Event: runtime/wake event source dispatched to an AML method or ACPI-aware native driver.
ACPI Embedded Controller	Platform microcontroller exposed through the ACPI EC interface and query mechanism for OEM-specific board functions.
pstore	Linux framework/filesystem exposing diagnostic records that a persistent backend preserved across reset.
ramoops	pstore backend that stores panic/oops/console/ftrace records in a reserved RAM region intended to survive reboot.
robust mutex	Mutex whose owner-death state can be reported to the next locker so application data can be repaired instead of deadlocking forever.
EOWNERDEAD	Robust-mutex lock result indicating the previous owner died while holding the lock; the new owner has the mutex but must repair protected state.
AppArmor profile	Task-centered mandatory-access-control ruleset loaded into the AppArmor LSM and associated with a confined program/task.
AppArmor complain mode	Policy-development mode that records would-be AppArmor denials instead of enforcing most of them.
orderly shutdown	Coordinated userspace and kernel teardown that stops services, flushes/unmounts storage and only then performs the final reset/power-off operation.
kernel log ring buffer	In-memory ordered store receiving printk/pr_* kernel messages independently of any userspace logging daemon.
/dev/kmsg	Linux userspace interface for reading/writing the kernel message stream; journald and other readers can consume kernel records through it.
journald	systemd userspace logging daemon that can ingest kernel messages, service streams, syslog-compatible input and other records into volatile or persistent journals.
V4L2	Video4Linux2: Linux userspace/kernel API family for video capture/output, controls, formats and streaming buffers.
videobuf2 (VB2)	Kernel media buffer-queue framework implementing common V4L2 streaming-buffer states and MMAP/USERPTR/DMA-BUF memory models.
buffer object (BO)	Kernel-managed graphics/accelerator allocation used as the identity/lifetime container for commands, images, textures and similar GPU-visible data.
GEM	DRM Graphics Execution Manager infrastructure for graphics buffer-object lifetime, handles, mmap and common driver helpers.
TTM	DRM Translation Table Manager for buffer placement, movement and eviction across device/system memory regions.
VRAM	Device-local video/graphics memory on a discrete GPU.
GPU virtual address	Address used by GPU commands after a buffer object is bound into the GPU's own MMU/address-space mapping.
switch_root	Userspace helper commonly used by initramfs to make an already-mounted real filesystem become `/`, move API mounts and execute the new init.
EINTR	Error reported when an interruptible operation returns because signal handling intervened and the interface was not transparently restarted.
restart_syscall	Linux-internal syscall restart mechanism used by selected timed waits so elapsed stopped time can be accounted for correctly.
ext4 delayed allocation	Technique that dirties logical file ranges before choosing final physical blocks, allowing the allocator to make better extent/locality decisions later at writeback.
soft lockup	Kernel condition where a CPU fails to schedule the watchdog thread for too long even though timer/interrupt activity can still occur.
hard lockup	CPU condition where ordinary interrupt heartbeat progress stops; commonly detected using an NMI/perf watchdog on supported architectures.
hung task	Task that remains in uninterruptible sleep beyond the configured detector timeout, often pointing to a stalled I/O or kernel wait.
RCU stall	Condition where an active RCU grace period cannot obtain required quiescent-state progress from CPUs/tasks within the warning threshold.
SMBIOS/DMI tables	Firmware-supplied typed platform-inventory records for system, board, processor, memory, slots and related management data; distinct from bus enumeration and from Intel's Direct Media Interface link.
APEI	ACPI Platform Error Interfaces: standardized firmware/OS mechanisms for describing, routing, persisting and testing platform hardware-error reporting.
GHES	Generic Hardware Error Source: APEI structure/mechanism in which platform firmware or a RAS controller reports structured hardware-error status to the operating system.
CPER	Common Platform Error Record: standardized structured representation for processor, memory, PCIe and other hardware-error information across firmware/OS boundaries.
nftables	Modern Linux packet-classification/rule framework programmed with nft; rules attach to Netfilter hooks and can use sets, maps, conntrack state and verdicts.
POSIX semaphore	Count-based synchronization object operated with sem_wait()/sem_post(); named semaphores have kernel-managed names/lifetime, while unnamed process-shared semaphores can live in shared memory.
POSIX message queue	Kernel-managed named queue of discrete prioritized messages accessed with mq_open()/mq_send()/mq_receive(); on Linux message-queue descriptors are pollable file descriptors.

What to actually build

Reading alone is not enough for this subject. A strong progression is: simulate a CMOS inverter; build NAND/NOR/XOR; build a half-adder and full-adder; build a mux; build an SR latch and D flip-flop; make a multi-bit register; make a counter/program counter; make an ALU; connect registers and ALU with a bus; add RAM; add an instruction register; design control signals; execute a tiny instruction set; then repeat the exercise with a real ISA or a 6502-class computer.

At that point, study timing violations, address decoding, interrupts, memory-mapped I/O, caches and pipelining. Then express some of the same circuits in Verilog and synthesize them with tools such as Yosys or onto an FPGA.

When you reach a breadboard or schematic, keep the datasheet beside it. Trace one signal at a time: where it is driven, what voltage levels mean, which clock edge matters, what enables a bus driver, where the value is stored, and what path it takes on the next cycle. That habit is how a block diagram turns into an electrical machine.