Sockets, buffering and backpressure

Why it matters

Backend and infrastructure interviews ask what happens "under load" and "when the client is slow": does memory grow, do connections pile up, does latency explode? The answers live in socket buffers, listen backlogs and how your event loop or thread pool handles partial writes. Networking research uses the same model when it analyzes end-host bottlenecks.

Core ideas

The socket API. A server calls socket, bind and listen, then accept in a loop; a client calls connect. Each accepted connection is a file descriptor. A read returns whatever bytes are available, possibly fewer than requested, and zero means the peer has finished sending. A write copies bytes into the kernel send buffer and, on a non-blocking socket, may accept only part of them.

Queues before accept. The kernel completes TCP handshakes on its own. On Linux, half-open connections wait in a SYN queue and completed ones in an accept queue whose length is bounded by the listen backlog (capped by a system-wide limit). If the application does not call accept fast enough, the accept queue fills and new handshakes are dropped or delayed, so clients see connection timeouts while the process looks healthy. The accept queue length is a direct overload signal.

Buffers and flow control. Each connection has a receive buffer and a send buffer in the kernel. If the application stops reading, the receive buffer fills, the advertised TCP window falls to zero and the sender stops: TCP flow control has carried backpressure to the peer. If the peer stops reading, your send buffer fills and writes block or return EAGAIN. Buffer sizes bound throughput (they must cover the BDP) and memory (connections times buffer size).

Backpressure in applications. A proxy that reads from a fast client and writes to a slow backend must stop reading when its outbound buffer is full; otherwise it buffers without bound in user space and TCP's flow control never engages. Event loops express this by pausing reads above a high-water mark and resuming below a low-water mark. The same principle applies above sockets: bounded queues between stages, admission control, and rejecting early (for example HTTP 429 or 503 with a retry hint) rather than letting latency grow until every request times out.

I/O models. Blocking I/O with a thread per connection is simple but costs memory and context switches at high connection counts. Non-blocking sockets with readiness notification (epoll, kqueue) let one thread serve many connections; completion-based I/O (io_uring) submits operations and collects results. All of them require correct handling of partial reads and writes.

on_readable(client):
    if pending_bytes(backend) > HIGH_WATER:
        pause_reading(client)      # kernel buffer fills, TCP window closes
    else:
        enqueue(backend, read(client))
on_drained(backend):
    if pending_bytes(backend) < LOW_WATER:
        resume_reading(client)

Common misconceptions

• "listen(fd, 1024) limits the number of connections." The backlog bounds the queue of completed connections waiting for accept, not the number of concurrent connections.
• "A returned write means the data was sent." It was copied into the kernel buffer; it may not have left the host, let alone reached the peer.
• "Bigger buffers solve overload." They hide it and add latency; overload needs bounded queues and admission control.
• "Non-blocking means fast." It means calls do not wait; you still must handle EAGAIN, partial writes and fairness between connections.
• "A zero-byte read is an error." It signals an orderly shutdown of the peer's sending direction; writing may still be possible.

How to approach it

Write a small framed echo server with non-blocking sockets and test it against a client that sends faster than it reads, watching Recv-Q and Send-Q in ss. In interviews, trace one slow client through the stack (kernel buffer fills, window closes, sender blocks) and say where your service pushes back or sheds load.