Linux system calls process spawning Interview Preparation Guide

The fork() system call duplicates the current calling process, creating a new child process with its own copy of memory page tables and process identifiers while executing the exact same code path. In contrast, execve() does not create a new process; instead, it replaces the entire memory address space, text segment, stack, and data of the current process with a new binary executable loaded from disk, while preserving the original Process ID (PID) and open file descriptor table. In standard Unix software design, these two primitives are combined: a process calls fork() to spawn a worker context, and the child immediately calls execve() to run a distinct program.

When fork() is invoked, duplicating physical RAM page by page for large applications would introduce massive latency and waste memory. Instead, the kernel employs copy-on-write: it copies the virtual memory page table entries from parent to child but marks all shared physical memory pages as read-only in both tables. Both processes share the exact same physical RAM frames. If either process attempts to modify a shared page, the CPU generates a page fault, intercepted by the Virtual Memory Manager, which then allocates a new physical page, copies the data, updates the respective page table mapping, and permits the write. This makes process duplication instantaneous unless heavy memory modification occurs.

A zombie process is a terminated process whose execution has completed, but whose process control block (task_struct) remains in the kernel process table because the parent process has not yet read its termination status using wait() or waitpid(). While zombies consume no CPU or memory RAM, they hold onto their Process ID (PID). If a buggy parent process spawns thousands of children without reaping them, the system exhausts its available PID pool, preventing any new processes or containers from spawning and ultimately causing system lockup. Proper reaping via waitpid() instructs the kernel to release the task_struct and reclaim the PID.

Signal handlers can interrupt a process at any arbitrary instruction, including in the middle of executing library functions like malloc() or printf() that maintain internal non-atomic data structures or locks. If a signal handler re-enters malloc() while the main thread is half-way through modifying memory allocation arenas, the internal data structures become corrupted. This leads to silent memory corruption, deadlocks, or immediate segmentation faults. Consequently, signal handlers must remain strictly async-signal-safe, limiting actions to setting atomic flags (sig_atomic_t) or writing to self-pipes.

When fork() is executed, the child process inherits an exact duplicate of the parent's file descriptor table, where both parent and child file descriptors point to the exact same open file descriptions in the kernel table (sharing file offsets and status flags). When execve() is subsequently called, these inherited file descriptors remain open in the new program unless explicitly marked with the FD_CLOEXEC (close-on-exec) flag. Failing to set FD_CLOEXEC causes unintended file locks, port retention, and security vulnerabilities where subprocesses retain access to sensitive sockets.

When a process is blocked inside a system call such as read(), write(), select(), or waitpid(), and the kernel delivers a signal to that process, the system call is interrupted to allow the signal handler to execute. Upon return from the handler, the blocked system call fails, returning -1 with errno set to EINTR. Robust production code must wrap blocking system calls in retry loops that explicitly check for errno == EINTR and reissue the call, or configure signal handlers with the SA_RESTART flag to instruct the kernel to automatically restart eligible interrupted system calls.

The CFS replaces traditional priority queues with a red-black tree keyed on virtual runtime (vruntime), which measures the exact amount of CPU time a task has consumed, scaled by its nice value weight. The scheduler constantly selects the runnable task with the lowest vruntime to execute next. As a task runs, its vruntime increases; once its vruntime exceeds that of other waiting tasks, it is preempted. This ensures that CPU-bound tasks and I/O-bound interactive tasks receive fair, proportional shares of CPU time without suffering starvation.

The self-pipe trick is a robust design pattern used to bridge asynchronous signal handling with synchronous event loops (such as epoll or select). Because signal handlers cannot safely perform complex operations or manage process reaping directly without reentrancy hazards, a daemon process creates an internal non-blocking pipe. Inside the asynchronous SIGCHLD signal handler, a single byte is written into the pipe's write end. The main event loop, monitoring the pipe's read end via epoll, wakes up synchronously, safely reads the byte, and executes waitpid() to reap terminated child processes without signal race conditions.

When fork() is called in a multi-threaded program, the POSIX standard specifies that only the calling thread is duplicated in the child process; all other threads vanish instantly. If another thread happened to hold a mutex lock (e.g., inside a memory allocator or logging library) at the exact moment of the fork, that mutex remains permanently locked in the child process because the thread holding it no longer exists to release it. If the child process subsequently attempts to acquire that same mutex, it deadlocks instantly and permanently.

Container engines leverage the clone() system call—and its companion unshare() and setns() calls—with specific bitmask flags to isolate operating system resources. For instance, CLONE_NEWPID creates an isolated Process ID namespace where the container's init process becomes PID 1; CLONE_NEWNET isolates network interfaces, routing tables, and firewall rules; CLONE_NEWNS isolates mount points; and CLONE_NEWUTS isolates hostname and NIS domain identifiers. This granular virtualization underpins modern containerization without requiring full hardware hypervisors.

A standard call to wait() blocks the calling parent process unconditionally until any of its child processes terminate, halting execution flow. In contrast, calling waitpid(-1, &status, WNOHANG) checks the termination status of any child process non-blockingly. If no child has exited yet, waitpid() immediately returns 0 rather than sleeping, allowing the parent process to continue executing its primary event loop or servicing other requests while periodically polling for completed child tasks.

Excessive process spawning introduces severe hardware and kernel overhead. Each process requires its own task_struct, kernel stack, and virtual memory page tables. Under high process churn, frequent address space switches invalidate Translation Lookaside Buffers (TLB shootdowns), cache line locality drops dramatically (L1/L2 cache thrashing), and CPU time is wasted on hardware context-switching overhead rather than actual computational work. This phenomenon, known as thrashing, can cause system throughput to collapse entirely.