Linux networking / TCP connection admission

Linux TCP listen backlog: two queues, three limits, one surprising failure

A listening socket can be “up” while new clients still time out. The reason is that TCP connection admission has two different queues, three different capacity controls, and a failure mode that often looks like a network problem.

Linux deep dive · TCP connection management

The concrete problem

Imagine a web server whose accept loop pauses for 200 ms during a stop-the-world garbage collection, a CPU stall, or a brief overloaded event loop. During that pause, clients continue sending TCP SYNs. Some connections succeed immediately; others take a long time, retry, or fail. Increasing the application’s listen(fd, backlog) argument may help—but it may also do nothing, because that argument does not control the queue most people think it does.

TCP has to remember a connection before the application has accepted it. A new connection therefore passes through two conceptual stages:

The first queue protects the handshake state. The second queue holds completed connections waiting for user space. They have different pressure signals, different kernel code, and different tuning knobs.

Three limits, not one

The values are not independent memory reservations. The request queue is dynamically sized and subject to timers, memory pressure, protocol behavior, and version-specific implementation details. Treat these as admission controls, not as a promise that exactly that many clients will always be retained.

What listen() actually does

At the socket API level, a server creates a stream socket, binds an address, and calls listen(). In the Linux IPv4 path, the protocol operation eventually enters the inet connection-socket listen code. The requested backlog is clamped to the system’s somaxconn limit and stored in the socket’s sk_max_ack_backlog. The socket state becomes TCP_LISTEN.

int fd = socket(AF_INET, SOCK_STREAM, 0);
setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, ...);
bind(fd, ...);
listen(fd, 4096);                 /* request; not a guarantee */

for (;;) {
    int client = accept4(fd, NULL, NULL, SOCK_CLOEXEC | SOCK_NONBLOCK);
    /* read, hand off, or close client */
}

The completed accept queue is represented inside struct inet_connection_sock as icsk_accept_queue, a struct request_sock_queue. The name is historical: it carries both the half-open request bookkeeping and the linked list of established children awaiting accept(). The listener also has sk_ack_backlog, the current completed-queue count, and sk_max_ack_backlog, its configured ceiling.

Handshake data flow in the kernel

For an IPv4 SYN, the path begins in tcp_v4_conn_request(), which delegates common work to tcp_conn_request(). The kernel creates a compact struct request_sock rather than a full established socket. It records enough state to validate the final ACK and retransmit the SYN-ACK.

The “hashdance” is the transition from a request tracked in the listener’s request machinery to an established child socket tracked in the normal connection hash. On the successful path, inet_csk_reqsk_queue_add() links the request object, now pointing at the child, onto rskq_accept_head/rskq_accept_tail and calls sk_acceptq_added(). The listener’s readiness state becomes readable because an accept() can now return a child.

The surprising failure: the handshake can finish, then the server still loses

When the accept queue is full, Linux reaches the listen-overflow path in tcp_check_req(). With the default net.ipv4.tcp_abort_on_overflow=0, the kernel generally does not immediately send a reset. It marks the request as acknowledged and lets the client retransmit its final ACK; a later retry can succeed if the application drains the queue. To the client, this often appears as extra connection latency rather than a clean refusal.

With tcp_abort_on_overflow=1, the kernel aborts instead, producing a faster but harsher failure. This can be appropriate for some fail-fast services, but it turns a transient accept stall into an explicit connection error. The choice is operational policy, not a magic performance switch.

The kernel increments listen-overflow and listen-dropped statistics in this area. A packet capture can therefore show a client that sent SYN, received SYN-ACK, sent ACK, and still did not receive application service promptly. The wire-level handshake is not proof that the application successfully accepted the socket.

SYN cookies change the meaning of “full”

When the half-open request queue is under attack or a burst exceeds its capacity, Linux can use SYN cookies if net.ipv4.tcp_syncookies permits them. Instead of retaining all per-client handshake state, the server encodes enough information into the SYN-ACK sequence number to reconstruct a request when the final ACK comes back.

Cookies are a last-resort defense against state exhaustion, not a replacement for capacity planning. They reduce the server’s memory exposure, but they necessarily constrain which TCP options can be represented and can interact with timestamps, window scaling, and newer negotiation features. They also do not make the accept queue infinite: a fully established child still needs somewhere to wait for accept().

Important structures and functions

Names and line numbers move across kernel releases. The stable relationship to look for is: request allocation and hashing on the SYN path; child completion on the ACK path; queue insertion only after the handshake is validated; queue removal in the accept path.

Tradeoffs and failure modes

Backlog is a shock absorber, not throughput. A bigger queue can turn immediate refusal into higher tail latency and more memory retention. It is useful when bursts are brief and the consumer catches up; it is harmful when it hides a permanently undersized accept or worker path.

Security implications

Connection queues are denial-of-service surfaces. An attacker can consume half-open state with spoofed or deliberately slow handshakes, or consume completed accept slots by establishing connections that the application is slow to read or close. SYN cookies reduce one class of state exhaustion, but they do not eliminate bandwidth, CPU, file-descriptor, TLS, or application-level exhaustion.

Keep the listener’s backlog and cookie policy aligned with the threat model. Expose only the intended addresses, use a firewall or load balancer where appropriate, set per-process file-descriptor limits deliberately, and monitor kernel counters rather than relying on application “port is open” checks. Avoid treating tcp_abort_on_overflow as a security control by itself: it changes failure visibility, not attacker cost.

Debugging method

1. Measure the application first: time from readiness to accept4(), accepted connections per second, and the number of event-loop or worker stalls.
2. Inspect both queue states. A large SYN-RECV population points to the handshake side; a listener with many ESTAB children waiting to be accepted points to the application side.
3. Read kernel counters before and after a controlled burst. The names are in /proc/net/netstat; ListenOverflows and ListenDrops are particularly revealing.
4. Check the effective controls: the program’s backlog argument, somaxconn, tcp_max_syn_backlog, tcp_syncookies, and tcp_abort_on_overflow.
5. Use a packet capture to classify the failure: missing SYN-ACK, repeated SYN, final ACK followed by silence, or immediate RST. Those are different bugs.
6. Only then tune. If the accept loop is stalled for longer than the queue can absorb, increasing the queue merely moves the failure to a later point.

# Linux: inspect effective controls and listener state
sysctl net.core.somaxconn \
       net.ipv4.tcp_max_syn_backlog \
       net.ipv4.tcp_syncookies \
       net.ipv4.tcp_abort_on_overflow
ss -lntp
ss -ant state syn-recv
awk '/TcpExt:/ { if (!h) { for (i=2;i<=NF;i++) h[i]=$i; h=1; next }
                 for (i=2;i<=NF;i++) if ($i != 0 && h[i] ~ /Listen/) print h[i], $i }' \
    /proc/net/netstat

The exact ss display varies by iproute2 version and does not expose every internal queue counter directly. Use it as a view of socket states, then correlate with /proc/net/netstat, service metrics, and a trace or packet capture.

Compact reproducible lab

Goal

Make the accept queue visible by starting a listener that deliberately does not call accept(), then compare it with a listener that drains immediately.

Prerequisites

A Linux host or VM with Python 3, ss, sysctl, and a shell. Run this on an isolated test host; the commands change only a user-space test process unless you choose the optional sysctl variation.

Exact commands

cat > /tmp/backlog-lab.py <<'PY'
import socket, sys, time

port = int(sys.argv[1]) if len(sys.argv) > 1 else 23456
backlog = int(sys.argv[2]) if len(sys.argv) > 2 else 2
with socket.socket() as s:
    s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    s.bind(("127.0.0.1", port))
    s.listen(backlog)
    print(f"listening on 127.0.0.1:{port}, requested backlog={backlog}", flush=True)
    time.sleep(300)              # deliberately never call accept()
PY

python3 /tmp/backlog-lab.py 23456 2 &
SERVER_PID=$!
sleep 1
ss -lntp 'sport = :23456'

python3 - <<'PY'
import socket, time
port = 23456
for i in range(12):
    s = socket.socket()
    s.settimeout(0.3)
    try:
        s.connect(("127.0.0.1", port))
        print(i, "connected")
    except OSError as e:
        print(i, type(e).__name__, e)
    # Keep successful sockets open until the process exits.
    time.sleep(0.05)
PY

ss -ant 'sport = :23456 or dport = :23456'
kill "$SERVER_PID"

Observations and expected result

The listening socket reports the requested backlog, but only a small number of the twelve loopback connections should remain established while the process never accepts them. Later attempts may connect, time out, or be refused depending on the kernel version and the effective somaxconn. The exact count is intentionally not a portable assertion: Linux may allow transient queue behavior beyond the nominal value, and loopback timing is unusually favorable.

On a Linux host, compare the result with ss -lnt and the ListenOverflows/ListenDrops counters. The key observation is not “backlog 2 means exactly two sockets”; it is that a listener can accumulate completed children independently of the application’s ability to process them.

Revealing variations

First, run python3 /tmp/backlog-lab.py 23457 128 and repeat the client against port 23457. If somaxconn is below 128, the effective ceiling is still lower. Second, change the server to call accept() in a loop and watch the established waiting population disappear; this isolates the accept queue from the SYN queue.

Environment result: this scheduled run is on macOS, where the Linux ss, /proc/net/netstat, and Linux TCP sysctls are unavailable, so the live queue experiment was not executed here. The source inspection and HTML validation did run. To reproduce the kernel-specific observations, use a Linux VM or container with the listed tools; macOS users can still run the Python listener/client to observe generic socket behavior, but must not infer Linux backlog semantics from its counters.

Production example: a TLS service behind a load balancer

A service can show healthy CPU and a healthy listening port while its clients report intermittent connect latency. Suppose a deploy briefly pauses the acceptor, then TLS handshakes consume CPU faster than worker threads can drain them. A larger backlog may absorb the deploy burst, but if the worker pool remains saturated it simply stores more established sockets and increases tail latency.

The production fix is usually layered: keep the accept loop tiny and independent of expensive work, use a bounded handoff to workers, size backlog and file descriptors for the burst window, and alert on ListenOverflows/ListenDrops plus accept latency. If SYN cookies are active during normal traffic, investigate why; they may indicate a real attack, a capacity mismatch, or a misleadingly low half-open limit.

Primary sources

• Linux listen(2) manual
• Linux source: inet connection-socket queues and listen transitions
• Linux source: tcp_check_req() and listen-overflow handling
• Linux source: IPv4 SYN entry point
• Linux source: connection-socket queue helpers
• Kernel documentation: IPv4 TCP sysctls
• RFC 9293: Transmission Control Protocol

Remember: the SYN queue answers “can the kernel remember this handshake?” The accept queue answers “can the application take the completed connection yet?” Tune and debug them separately.