Processes, threads, system calls and scheduling

Why it matters

Most systems questions eventually become "what is the CPU doing, on whose behalf, and what does switching cost?" Explaining why a service with thousands of threads slows down, why a container with a CPU limit shows latency spikes, or why an async runtime stalls behind one slow call all rests on a few mechanisms: processes and threads, the user/kernel boundary and the scheduler. Infrastructure, SRE and low-level interviews expect you to trace these paths precisely; backend interviews use them to check that your performance intuitions have a mechanical basis.

Core ideas

Process versus thread. A process bundles an address space, open file descriptors, credentials and signal handling with one or more threads. A thread is an execution context: registers, program counter, stack pointer and thread-local storage. Threads of one process share memory, so communication is cheap and isolation is absent: a wild pointer in one thread corrupts all of them. On Linux both are tasks created by clone(), differing only in what they share.

Limited direct execution. User code runs directly on the CPU in user mode, where privileged instructions and device access are forbidden. The kernel regains control only through a trap: a system call (deliberate, such as syscall on x86-64 or ecall on RISC-V), an exception (page fault, illegal instruction) or an interrupt (a device or the timer). The hardware raises the privilege level and jumps to a handler the kernel registered at boot; the kernel saves user registers, validates arguments (user pointers are untrusted), does the work and returns with an instruction that drops privilege. The timer interrupt is what makes preemption possible.

A blocking call, end to end. read() on a socket with no data traps into the kernel, finds the receive queue empty, puts the thread on a wait queue and calls the scheduler, which runs another thread. When a packet arrives, the interrupt path queues the data and marks the thread runnable. After some run-queue delay the scheduler resumes it inside the kernel, which copies the data out and returns to user mode.

Context switches. A thread switch saves and restores general registers, stack pointer, program counter and, when used, floating-point and vector state. Switching to another process also changes the page-table root, so cached translations are useless unless the TLB tags entries with address-space IDs. The direct cost is small; the indirect cost of cold caches, TLB misses and lost branch history is often larger. A system call is not a context switch: it is a mode switch within the same thread, and the scheduler is involved only if the call blocks. Some calls, such as reading the clock on Linux, avoid the trap entirely through the vDSO.

Scheduling. Policies trade turnaround time (batch work: SJF, STCF) against response time (interactive work: round robin). Shorter time slices improve responsiveness but add switching overhead. MLFQ approximates shortest-job-first without knowing job lengths by watching whether a task uses its whole slice, and needs periodic priority boosts to prevent starvation. Proportional-share schedulers track the CPU each task has received: Linux's fair class used CFS (virtual runtime in a red-black tree) and began moving to EEVDF in kernel 6.6. Multiprocessor schedulers keep per-CPU run queues for cache affinity and migrate tasks to balance load.

Creating processes. fork() duplicates the caller cheaply thanks to copy-on-write; exec() replaces the program image; wait() collects the exit status, and until then the dead child stays a zombie. Separating fork from exec lets a shell rewire file descriptors in the child before the new program starts.

Common misconceptions

• "A system call is a context switch." It changes privilege, not the running thread; only blocking or preemption invokes the scheduler.
• "Threads are always cheaper than processes." Creation and switching are cheaper, but shared state brings lock and allocator contention and no fault isolation; process-per-worker designs trade memory for robustness.
• "More threads means more throughput." For CPU-bound work, threads beyond the core count add switches and cache churn; for blocking I/O they help only until another resource saturates.
• "fork() copies the whole address space." It copies page tables and marks pages copy-on-write; data is copied on first write. A large parent still pays for page-table copying and later write faults.
• "The kernel is a separate process serving system calls." Kernel code runs in the context of the calling thread, of an interrupt or of a kernel thread; there is no server to message.
• "A higher priority guarantees low latency." Fair schedulers give proportional shares, not deadlines; latency still depends on how many runnable tasks compete.

How to approach it

Practice telling the blocking-read story aloud in under two minutes, then deepen one level at a time: what the hardware saves, what the kernel saves, where the thread sleeps. Measure instead of quoting numbers: time a null system call and a pipe ping-pong between two processes pinned to one CPU, then compare with two threads. On Linux, /proc/<pid>/status shows voluntary and involuntary context switches: many involuntary switches suggest CPU contention, many voluntary ones suggest blocking. Graduate interviews usually stop at processes versus threads and fork/exec; mid-level ones probe the system-call path and scheduling tradeoffs; senior ones connect scheduler behavior to tail latency and container CPU limits.