TERMS TO KNOW

Time elapsed between sending a request and receiving the first byte of response. The single most human-felt metric — users notice >100ms. Engineers target <1ms inside the data center.

The latency experienced by the slowest 1% of requests. Averages lie — P99 reveals outliers that kill user experience. Distributed systems compound tail latency: if one service is slow, every caller waits.

Number of complete database transactions committed per second. Each transaction can span multiple reads and writes. The gold-standard throughput metric for relational databases — dictates vertical scale ceiling before sharding.

Raw read or write operations per second, typically for caches or key-value stores. Broader than TPS — a single transaction can contain many ops. Caches are optimized to maximize this number since they are memory-bound, not disk-bound.

Volume of discrete messages a queue or broker can ingest and deliver per second. Differs from TPS — messages are often fire-and-forget, async. Kafka partitions are the scaling unit: more partitions = higher parallelism.

How many individual read or write operations a disk can handle per second, regardless of data size. Small random reads (databases, indexes) are IOPS-bound. SSDs vastly outperform spinning disks on this metric.

Total data transferred to/from disk per second. Matters for large sequential reads: log streaming, analytics scans, backups. Distinct from IOPS — you can have high throughput with few large I/Os, or high IOPS with many small I/Os.

Parallel execution units. More cores = more concurrent work without context-switching penalty. App servers are typically CPU-bound during request processing. Clock speed (GHz) determines single-threaded performance.

Number of simultaneous open connections an app server holds. Async runtimes (Go, Node.js, Nginx) handle far more connections per core than thread-per-request models (Apache prefork). This is often the first ceiling hit in real-time or WebSocket-heavy systems.

Maximum data transfer rate of a network link. Bandwidth is the pipe width; throughput is how full the pipe is. Cloud instance network bandwidth is a hard ceiling — exceeding it causes packet drops and queue buildup, not graceful degradation.

In message queues, the gap between the latest message produced and the last message consumed. Lag is the leading indicator of a system falling behind — it means producers are outpacing consumers. Growing lag = bandwidth or processing bottleneck downstream.

Working memory — the fastest storage tier (after CPU cache). Caches live entirely in RAM. DBs use RAM for buffer pools to avoid disk I/O. When RAM fills, systems page to disk, causing latency to spike 1000x.

Total persistent data volume a system holds. Databases use it for rows, indexes, WAL logs. Message queues retain it for replay windows (Kafka default: 7 days). Capacity planning drives replication factor × data size × retention period.