TCP Flow Control

A TCP receiver places arriving bytes in a receive buffer, from which the application reads them when it chooses. If the sender sends faster than the application reads, the buffer fills and further bytes would be lost. Flow control prevents this: in every segment the receiver advertises a receive window (rwnd), modelled here as free receive-buffer space (implementation window-allocation policies can differ), and the sender uses the latest accepted window to limit new transmissions. A reduced rwnd or cwnd can temporarily be smaller than data already in flight; it does not withdraw already-sent bytes.

This is a sliding-window protocol (Sliding Window Protocols) with two differences: the window counts bytes, not frames, and its size changes during the connection as the receiver's buffer fills and empties. Flow control protects the receiver; congestion control (TCP Congestion Control) protects the network, and the sender obeys the smaller of the two windows.

The 16-bit window and 32-bit sequence number also have limits on fast links: the window scale option and sequence-number wraparound.

For this simplified buffer model, assume all received data is in order and no out-of-order bytes are buffered. In general, actual free buffer space is RcvBuffer minus all unread buffered bytes, including out-of-order data; this is not a complete implementation window-allocation policy. With buffer size RcvBuffer, the last byte the application has read LastByteRead, and the last byte received in order LastByteRcvd:

The receiver puts rwnd in the window field of every segment it sends, together with the ACK number.

The sender keeps

where $\text{FlightSize}=\text{LastByteSent}-\text{LastByteAcked}$ in this simplified in-order model. Without congestion control, omit cwnd and use $\max(0,\text{rwnd}-\text{FlightSize})$. This is permission for new data, not an invariant on previously transmitted data after a window reduction. The window's left edge advances when ACKs arrive; its advertised right edge is $\text{ack}+\text{rwnd}$.

When rwnd = 0 the sender stops. The receiver will announce new space in a window update, but that segment carries no data and is not acknowledged; if it were lost both sides would wait forever. So a sender facing a zero window starts a persist timer and, when it fires, sends a window probe (one byte, or a segment the receiver must answer) to draw out the current window.

If the application reads one byte at a time and the receiver advertises every freed byte, the sender sends one-byte segments, each with 40 bytes of headers. Two fixes:

Delayed ACKs. A receiver may also wait before acknowledging, so one ACK can cover two segments and carry a window update or reply data; RFC 1122 requires the delay to stay under 0.5 seconds and an ACK for at least every second full-sized segment.

At most one window is delivered per round-trip time, so

The 16-bit window field allows at most 65,535 bytes, so without scaling a connection with RTT 100 ms cannot exceed $65{,}535 \times 8 / 0.1 \approx 5.24$ Mbps, however fast the link. To keep a link busy the window must be at least the bandwidth-delay product (Performance Metrics).

The window scale option (RFC 7323), sent on the SYN, gives a shift count $S$ between 0 and 14; the true window is the 16-bit field times $2^S$. The largest window is about $2^{16} \cdot 2^{14} = 2^{30}$ bytes (1 GiB). The limit 14 keeps the window below $2^{30}$, which leaves room in the $2^{32}$ sequence space for old and new data to be told apart. To support window $W$, the smallest shift is the least $S$ with $65{,}535 \cdot 2^S \ge W$.

Sequence numbers are 32-bit byte counts, so they repeat after $2^{32}$ bytes. At bandwidth $B$ bit/s the wrap time is

If $T_{\text{wrap}}$ is less than the maximum segment lifetime, an old duplicate could carry a sequence number that is valid again. RFC 9293 takes MSL as 2 minutes; with TCP timestamps, PAWS (protection against wrapped sequences, RFC 7323) rejects such old segments.

The inverse question: for bandwidth $B$ and lifetime $L$, the sequence field needs at least $\lceil \lg (B L / 8) \rceil$ bits to count every byte sent in $L$ seconds without repeating.

RcvBuffer = 4096 bytes, MSS = 1024 bytes, Clark's rule in use (threshold $\min(1024, 2048) = 1024$).

Between steps 3 and 5 the sender's persist timer may fire; the probe is answered with the current window (0 at step 4).

LastByteAcked = 5000, LastByteSent = 7000, the latest rwnd = 3000, and cwnd = 4000. The bound is $\min(3000, 4000) = 3000$ bytes outstanding; 2000 are outstanding, so 1000 more may be sent now: bytes 7001 to 8000.

A path with 1 Gbps and RTT 100 ms has bandwidth-delay product $10^9 \times 0.1 / 8 = 12{,}500{,}000$ bytes. Without scaling the window is at most 65,535 bytes, so throughput is at most about 5.24 Mbps, about 0.5% of the link. The least shift with $65{,}535 \cdot 2^S \ge 12{,}500{,}000$ is $S = 8$ ($2^7$ gives 8,388,480; $2^8$ gives 16,776,960).

A 1 Gbps link and a segment lifetime of 60 s: $10^9 \times 60 / 8 = 7.5 \times 10^9$ bytes can be sent in one lifetime. $2^{32} \approx 4.29 \times 10^9 < 7.5 \times 10^9 \le 2^{33}$, so at least 33 bits are needed; TCP's 32 bits wrap in 34.4 s at this rate, which is why PAWS exists.

A user types a character every 70 ms (at 0, 70, ..., 980 ms; 15 characters) over a connection with RTT 200 ms. Without Nagle, 15 one-byte segments are sent. With Nagle, the first character goes at once; each later segment leaves when the previous one is acknowledged:

Six segments instead of fifteen.

With MSL 2 minutes, sequence numbers must not wrap within 120 s, so at most $2^{32}$ bytes may be sent in 120 s:

about 286 Mbps. Faster connections rely on PAWS; at 1 Gbps the sequence space wraps in about 34.36 s.

Common misconceptions.