Reliability / performance engineering / production observability · 2026-10-14

Production profiling: read flamegraphs as evidence, not verdicts

Cloudflare announced on-demand CPU and memory profiling for deployed Workers and Durable Objects on October 9, 2026. The feature illustrates a broader lesson: production metrics tell you that a service is slow or memory-constrained; profiles can show which observed call paths consumed sampled CPU or allocated memory during a chosen window. They are powerful diagnostic evidence—but a profile is a sample, not a complete account of the program.

Metrics find symptoms; profiles locate work

Latency percentiles, CPU utilization, allocation rate, and out-of-memory counts narrow the search. A CPU profile samples active execution stacks over time. A heap/allocation profile records allocation activity during a capture, depending on the profiler's implementation. The resulting call stacks aggregate observations by function: a wide flamegraph frame means more sampled weight, not necessarily a single slow invocation or a proven root cause.

request → route → parse → business logic → serialize
                 ▲              ▲
       CPU samples accumulate   allocation samples accumulate

CPU and memory profiles answer different questions. A hot CPU frame may be doing repeated computation. A large allocation frame may create short-lived garbage, retained objects, or both; allocation alone does not prove a leak. For leak diagnosis, compare heap behavior across time and inspect object retention, rather than assuming the function that allocated an object still retains it.

Why the capture window matters

An on-demand profile sees what happened while it ran. If traffic is sparse, the capture may contain too few useful samples. If a memory spike occurs only at startup but the profile begins after warm-up, the capture can miss it entirely. One quiet profile is not proof of healthy code. Capture a representative period, select a version receiving real traffic, and correlate profile timestamps with request volume and error/latency graphs.

Sampling also changes interpretation: it estimates where observed work occurred, rather than tracing every operation. Short windows can be noisy; rare paths can disappear; long windows can mix workloads. Repeat a capture and compare like-for-like traffic before acting on a surprising frame.

Reproducible production capture

Cloudflare's documented CLI supports CPU and heap profiles. Install and authenticate the current Cloudflare CLI first, set the worker ID, and choose a version with steady traffic. This command captures five seconds of CPU activity and saves a profile locally:

export WORKER_ID="your-worker-id"
cf workers versions profile latest \
  --worker-id "$WORKER_ID" \
  --duration-ms 5000 \
  --profile-type cpu > worker-cpu.pprof

Use --profile-type heap for allocation profiling. Do not paste API tokens into shell history or public logs. The Cloudflare dashboard can display an interactive flamegraph; downloaded profiles can be inspected with compatible tooling such as pprof. Minified bundles need source maps for readable source frames; without them, the graph may identify generated bundle locations instead of original functions.

A disciplined investigation

Start from a concrete signal: a latency regression, elevated CPU, allocation growth, or memory-limit errors.
Record deployment version, traffic rate, capture interval, profile type, and any concurrent incidents.
Capture during representative load; repeat if the result is unexpected or the sample is small.
Inspect the widest relevant frames and their callers. Check whether cost comes from your code, dependencies, serialization, instrumentation, or garbage collection.
Form one testable change, then compare the same metrics and a fresh profile under similar traffic.
For suspected leaks, use multiple heap snapshots or time-separated captures and examine retained objects—not just allocation hotspots.

Common traps

Confusing width with wall-clock latency: frame weight represents sampled CPU or allocation, not request duration by itself.
Calling allocation a leak: frequent allocation can be reclaimed promptly; retention evidence is needed.
Profiling an idle version: ensure selected deployment version actually receives enough requests.
Ignoring startup: a post-start profile cannot explain work performed before capture.
Trusting unreadable symbols: configure source maps for minified code before drawing conclusions.
Optimizing one screenshot: validate changes with repeated, comparable captures and service-level outcomes.

Takeaway

Profiling narrows the distance between “the service is unhealthy” and “this call path deserves investigation.” It does not eliminate measurement bias or replace experiments. Treat a flamegraph as a workload-specific sample: establish context, read the frame type correctly, form a hypothesis, change one thing, and measure again.