Linux memory management / process creation

Linux fork() and copy-on-write: private pages are shared until the first write

A process can fork a large address space quickly because Linux initially shares its physical pages. The cost arrives later, at the exact write that needs isolation: a page fault, a new page, a copy, and a carefully ordered page-table update.

Linux deep dive · Process memory and isolation

The concrete problem

Suppose a server has a 2 GiB heap and calls fork() to create a worker. A literal implementation would allocate 2 GiB for the child and copy every byte before returning. That would make process creation slow even when the child immediately calls execve() and discards the inherited heap.

Linux instead uses copy-on-write, usually abbreviated COW. After fork(), parent and child point at the same physical pages. Those pages are temporarily mapped read-only in both processes. If either process writes, the CPU traps; the kernel allocates a private replacement page, copies the old contents, and retries the write against the replacement.

What fork() actually duplicates

A process address space is a set of virtual-memory areas (VMAs) plus page-table mappings. A VMA describes a range and its permissions and flags; page tables translate individual virtual pages to physical pages. fork() creates a new memory-management object and copies the parent’s VMA metadata and page-table structure, but it does not eagerly copy every mapped page.

The child gets a distinct mm_struct, so later operations such as mmap(), munmap(), and changing VMAs do not modify the parent’s address-space metadata. For the physical pages referenced by private writable mappings, however, both address spaces temporarily share the same page. The page reference count records that more than one mapping or owner still needs the page.

Not every mapping follows this exact path. MAP_SHARED memory is intentionally shared and writes are visible through the mapping. A mapping marked VM_DONTCOPY is excluded, while madvise(MADV_DONTFORK) and MADV_WIPEONFORK) let an application choose different inheritance semantics for selected ranges.

The write-fault path

The MMU enforces the read-only PTE. User code does an ordinary store; the CPU sees a write to a non-writable translation and raises a page fault. The kernel must distinguish a legitimate COW fault from an invalid write to read-only memory.

Exact helper boundaries move as the kernel evolves, but current source still exposes the important division in mm/memory.c: do_user_addr_fault() validates and dispatches the user fault, handle_mm_fault() walks the page-table levels, and the write-protection path decides whether to reuse the old page or copy it.

The fault is not “copy every page in the process.” It is “copy this page, now.” A workload that forks and then writes only 1% of its private heap pays for roughly 1% of the page copies, plus page-table and bookkeeping work. A workload that writes the entire heap eventually pays almost the same memory-copy cost as eager duplication, only spread over time.

Why the kernel can sometimes avoid the copy

The read-only PTE is a conservative signal. If the physical page is no longer shared in a way that matters to this mapping, the kernel may be able to restore write permission in place. This is commonly called a reuse or “reuse the old page” fast path. Otherwise, the kernel allocates a fresh page and copies.

That decision is constrained by more than a simple page reference count. Other mappings, page-cache state, pinning, concurrent faults, dirty/writeback rules, and the mapping type all matter. The implementation must preserve the promise that a write by one process cannot alter what another process can read.

Important structures and functions

Use the source links below against the kernel version you are diagnosing. Function names and folio conversions change; the invariant does not: shared private pages must become independent before a successful write can modify one process’s view.

Ordering and concurrency: the dangerous part

COW is a correctness protocol, not just a memory-saving trick. The kernel must coordinate page-table locks, page references, reverse-mapping information, and TLB invalidation. A stale writable PTE would let a process modify a page that another process still believes is private.

A simplified safe sequence is:

1. Take the locks needed to inspect and modify the relevant page-table entry and mapping state.
2. Confirm that the fault is a permitted write and that the old mapping still represents the faulting page.
3. Either take the exclusive/reuse path or allocate a replacement page.
4. For a copy, copy the old contents while the old page remains valid, install the new PTE with the correct writable and dirty state, and adjust references.
5. Perform the required TLB invalidation or ordering work before another CPU can use an obsolete translation.

The real implementation has retry paths because another thread can alter the mapping while the fault is being handled. It also has to handle allocation failure, signals, memory cgroups, NUMA policy, and userfaultfd interactions. A fault that returns successfully is the end of a transaction whose intermediate states must never leak as an unsafe writable alias.

Costs that surprise people

This is why prefork servers often fork before accepting traffic, then avoid mutating inherited large heaps. Conversely, a process that forks and immediately dirties a large memory image may be better served by an explicit worker design or posix_spawn(), depending on its required inheritance semantics.

Security and isolation implications

COW is part of process isolation: a child can inherit bytes, but a normal write must not change the parent’s private view. Bugs in the write-fault or page-table invalidation path are therefore security bugs, not merely accounting errors. The historical “Dirty COW” vulnerability showed the danger of racing COW-related write access against mappings that should have remained read-only; modern kernels include fixes, but the lesson is general.

There are also availability risks. An attacker who can trigger repeated fork() plus broad writes can force page allocation and copying, consuming CPU, memory bandwidth, and memory limits. Use process limits, cgroup memory and PID limits, and appropriate overcommit policy. Do not infer isolation from MAP_PRIVATE alone: shared mappings, file descriptors, inherited capabilities, and IPC objects have their own sharing rules.

Debugging method

1. Measure the phase change. Compare time and fault counts before fork, immediately after fork, and after the child or parent dirties memory.
2. Use getrusage(), perf stat -e page-faults,minor-faults,major-faults, or a tracing tool. COW faults are normally minor faults because the backing data is already in memory.
3. Read /proc/<pid>/smaps_rollup. Rss counts resident mappings; Pss divides shared pages proportionally; private/shared dirty fields show the transition after writes.
4. Check whether transparent huge pages are involved. A small write to a huge mapping can have a very different fault and memory profile from a 4 KiB page.
5. When investigating a kernel bug, trace the faulting address and inspect page-table state with kernel debugging tools; do not rely on physical frame numbers from /proc/<pid>/pagemap, which are restricted on many systems.

# Linux observations for a running process
grep -E '^(Rss|Pss|Private_Dirty|Shared_Dirty):' /proc/$PID/smaps_rollup
perf stat -e page-faults,minor-faults,major-faults -p "$PID"
cat /sys/kernel/mm/transparent_hugepage/enabled

# Source-oriented search in a checked-out kernel tree
rg -n 'copy_page_range|do_wp_page|wp_page_copy|do_user_addr_fault' mm/

Compact reproducible lab

Goal

Observe that fork() initially shares resident pages, then watch a child’s first writes turn shared pages into private pages and produce minor faults.

Prerequisites

A Linux system or VM with a C compiler, /proc, and perf (optional). Run on an isolated host if you change memory limits. The program allocates 256 MiB and intentionally pauses so its state can be inspected.

Exact commands

cc -O0 -Wall -Wextra -o /tmp/cow-lab -x c - <<'C'
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <sys/wait.h>
#include <unistd.h>

static void report(const char *label) {
    FILE *f = fopen("/proc/self/smaps_rollup", "r");
    char line[256];
    printf("%s (pid %ld)\n", label, (long)getpid());
    if (!f) { perror("smaps_rollup"); return; }
    while (fgets(line, sizeof line, f))
        if (!strncmp(line, "Rss:", 4) || !strncmp(line, "Pss:", 4) ||
            !strncmp(line, "Private_Dirty:", 15) || !strncmp(line, "Shared_Dirty:", 14))
            fputs(line, stdout);
    fclose(f); fflush(stdout);
}

int main(void) {
    const size_t bytes = 256UL * 1024 * 1024;
    const long page = sysconf(_SC_PAGESIZE);
    unsigned char *p = mmap(NULL, bytes, PROT_READ | PROT_WRITE,
                            MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
    if (p == MAP_FAILED) { perror("mmap"); return 1; }
    for (size_t i = 0; i < bytes; i += (size_t)page) p[i] = 1;
    report("parent before fork");

    pid_t child = fork();
    if (child < 0) { perror("fork"); return 1; }
    if (child == 0) {
        report("child after fork"); sleep(2);
        for (size_t i = 0; i < bytes / 2; i += (size_t)page) p[i]++;
        report("child after writing half"); sleep(8);
        _exit(0);
    }
    sleep(4);
    report("parent while child holds private copies");
    waitpid(child, NULL, 0);
    munmap(p, bytes);
    return 0;
}
C
/tmp/cow-lab

# In another shell, while the program sleeps (replace PID):
grep -E '^(Rss|Pss|Private_Dirty|Shared_Dirty):' /proc/$PID/smaps_rollup
perf stat -e page-faults,minor-faults,major-faults /tmp/cow-lab

The lab’s intended output is qualitative: after fork, each process’s PSS reflects shared resident pages; after the child writes half the range, the child’s private dirty memory rises and its minor-fault count jumps by roughly the number of written pages.

Expected result

You should see the child’s “after fork” report before its write, then a report with more private dirty memory after the write loop. The parent’s report should still describe its original view. Exact values vary because the process also has code, libraries, allocator metadata, and kernel accounting overhead.

Revealing variations

Change bytes / 2 to bytes and compare minor faults and PSS: writing the whole range materializes nearly the entire private copy. Then replace MAP_PRIVATE with MAP_SHARED; the child’s writes no longer need COW, and the mapping’s sharing/dirty behavior changes.

Environment result: this run is on macOS. The compiler can build the sample, but macOS has no Linux /proc/self/smaps_rollup or Linux perf interface, so the measurement lab was not executed successfully here. Re-run it on a Linux VM or container. On macOS, use vmmap <pid> as a rough fallback for virtual-memory inspection, but do not treat its categories as equivalent to Linux PSS/private-dirty accounting.

Production example: prefork application servers

A prefork web server often loads configuration, code, and caches once, then forks workers. Read-mostly pages remain physically shared, reducing startup memory. The design works well when workers mostly read inherited state and allocate their own request data.

It becomes expensive when a child mutates a large inherited cache, when a garbage collector dirties many pages, or when transparent huge pages make a small write trigger a large COW event. Production diagnosis should correlate fork rate, minor faults, memory bandwidth, RSS/PSS, THP events, and cgroup memory pressure. “RSS doubled after fork” is not automatically a leak; “PSS and minor faults spike after a write-heavy phase” is stronger evidence of COW materialization.

Primary sources

• Linux fork(2) manual
• Linux mmap(2) manual
• Linux source: process and address-space duplication
• Linux source: page faults and write-protection/COW handling
• Linux source: mm_struct, VMA, and memory types
• Kernel documentation: transparent huge pages
• Kernel documentation: /proc process memory information
• Kernel documentation: overcommit accounting

Remember: fork() makes the mappings look independent, then lets the MMU find the first write that would violate that illusion. Copy only that page, at that moment.