Queueing and Capacity

An order processor works well during quiet periods. During a promotion, the processing time for an individual order barely changes, but customers wait much longer for confirmation. The extra time is spent waiting to be processed.

A queue holds work that cannot start immediately. Arrival rate describes incoming work; processing capacity describes how quickly workers can complete it. The time spent in this part of the system includes both waiting time and processing time. Suppose 120 orders arrive each second while workers complete 100. The backlog grows by 20 orders per second: 1,200 additional orders after one minute. A queue absorbs the burst, but it does not create processing capacity. Even when average arrivals remain below capacity, uneven arrivals and variable processing times create waiting. As utilization approaches the bottleneck's limit, small bursts can cause large delays. Spare capacity provides room to recover. Queues have finite storage and may enforce retention limits. Monitor queue depth and the age of waiting work. Add capacity where the bottleneck is, reduce work, or limit incoming demand. Adding workers helps only while their shared dependencies can handle the extra load.