I/O models, readiness and event loops

Why it matters

Every network service waits for many clients at once, and the waiting strategy shapes its memory use, latency and failure modes. The practical versions of this topic are common: an async service that calls model APIs becomes unresponsive because one handler parses a large document synchronously; a server corrupts data because it assumed one read() returns one message; a proxy runs out of memory because it reads faster than it can write. Backend, networking and infrastructure interviews ask you to design or debug such loops.

Core ideas

Blocking threads. The simplest server dedicates a thread to each connection and lets read() block. It is easy to write and debug and works well for moderate connection counts. Costs grow with many mostly idle connections: a stack per thread, scheduler work and context switches. That limit is the classic "C10K" motivation for event-driven designs.

Nonblocking descriptors plus readiness. With O_NONBLOCK, read() and write() return EAGAIN instead of sleeping, and a readiness API tells you which descriptors can make progress. select uses fixed-size bitmaps and rescans every descriptor on each call; poll takes an array but still scans it; Linux epoll keeps the interest set in the kernel and returns only ready descriptors; BSD and macOS kqueue does the same and also delivers timers, signals and process events.

Level versus edge triggering. Level-triggered notification keeps reporting a descriptor while data remains. Edge-triggered notification reports changes, so the handler must drain the socket until EAGAIN or it may never be woken again. Edge triggering reduces repeated wakeups but punishes partial handling.

Readiness is not completion. Readiness says an operation would not block now; you still perform the copy yourself. Regular files are always considered ready (epoll refuses them outright), yet reading them can block on the disk. Event-loop runtimes therefore run file operations on a thread pool, as libuv does under Node.js, or use completion-based I/O. Linux io_uring shares a submission ring and a completion ring with the kernel: you queue operations, the kernel performs them and posts results, and many operations can be submitted with one system call.

The event loop. Wait for events, run the callbacks or resume the coroutines that were waiting, run expired timers, repeat; async/await is syntax over the same state machine. One thread owns the loop's state, so it needs no locks, but any callback that blocks or computes for long delays every other connection on that loop. Offload CPU-heavy work and blocking libraries to worker threads or processes, and monitor loop lag, the delay between when a timer should fire and when it runs. Python's asyncio debug mode, for example, reports slow callbacks.

Byte streams and partial operations. TCP delivers bytes, not messages: one read() can return half a message or three of them, so the application frames messages with a length prefix or delimiter. A nonblocking write() may accept only part of the buffer; keep the remainder in a per-connection output buffer and ask for writability notifications only while that buffer is non-empty.

Backpressure and lifetimes. If a peer reads slowly, output buffers grow; stop reading from the producing side when a buffer passes a bound instead of buffering without limit. Connections end in several ways: read() returning 0 means the peer finished sending (it may still be reading); a reset surfaces as ECONNRESET or EPIPE, plus SIGPIPE unless suppressed. Servers also need idle and request timeouts, a graceful shutdown that stops accepting and drains in-flight work, and a plan for running out of file descriptors.

Common misconceptions

• "Nonblocking and asynchronous are the same." Nonblocking calls return immediately with EAGAIN; asynchronous (completion) I/O starts an operation and tells you when it has finished.
• "epoll makes file reads nonblocking." epoll does not accept regular files; disk reads block unless offloaded or issued through io_uring.
• "One read returns one message." TCP is a byte stream; framing is your job.
• "A single-threaded event loop cannot have races." State can change across every await; check-then-act spanning an await point is a logical race.
• "Async is always faster than threads." Async wins with many mostly idle connections; for modest counts, threads with blocking I/O are often simpler and just as fast.
• "Edge-triggered is just a faster level-triggered." It changes the contract: forget to drain and the connection stalls.

How to approach it

Build the same echo server three ways: thread per connection, a poll or epoll loop, and your language's async runtime. Test it with a client that sends one byte at a time and with one that never reads its replies, and confirm that framing and backpressure hold. For interviews, be ready to sketch the loop, point to where a blocking call would hurt, and say how you would detect it in production. Senior discussions add multi-core designs (one loop per core, distributing accepts) and behavior under overload.