chaowen. · Notes

AI systems / security engineering / observability · 2026-10-15

Security AI agents: make evidence and scope part of the system

A security alert is a lead, not a verdict. A recent Cloudflare engineering account of its Managed Defense agent harness describes an important design choice: collect and scope evidence with deterministic application code first, then ask models to interpret a fixed dossier. This is broadly useful anywhere an AI system may influence a consequential decision.

Why “just give the model the logs” fails

A single general-purpose agent given telemetry, policies, threat intelligence, and detector descriptions must infer which sources are authoritative, choose the relevant time range, retrieve records, and reach a conclusion. These are several different tasks hidden inside one prompt. The model can blur a detector’s hypothesis into a confirmed event, query the wrong scope, or describe a failed lookup as if it returned no matches.

These are not merely model-quality problems. They are system-design problems: the application has not made provenance, permissions, or incomplete retrieval explicit.

Separate collection from interpretation

Build a versioned, deterministic reconnaissance step. It should resolve identity and scope, run approved queries, and record each result with source, timestamp, query/version, and status. Then freeze that evidence snapshot for analysis. A model can summarize and compare records, but should not silently broaden the account, time window, or data sources.

Alert
  → deterministic scope + approved queries
  → evidence dossier (source, time, result status)
  → narrow specialist analyses
  → constrained synthesis + human review

Make missingness a first-class value. “No matching event” is not equivalent to “source unavailable,” “query timed out,” or “permission denied.” A typed result can preserve that distinction:

{
  "source": "edge-waf",
  "status": "unavailable",  // ok | empty | unavailable | denied
  "observed_at": "2026-10-15T08:00:00Z",
  "records": []
}

In production, validate this structure against a schema and include query/version metadata; the abbreviated example is illustrative.

Use specialists for bounded questions

When deeper reasoning is useful, split the work into narrowly framed analyses: traffic behavior, customer history, aggregate global signals, and admitted threat intelligence. Give each specialist only the data needed for its question. The final synthesizer should receive typed findings and their evidence references—not unrestricted access to fetch more data or invent a new classification. This makes disagreement visible and limits the blast radius of a mistaken inference.

Privacy boundaries deserve the same engineering treatment. Cross-customer context, if used, should be deliberately aggregated and designed not to expose another customer’s records or identity. A prompt saying “do not reveal private data” is weaker than an architecture that never provides that data.

Make the investigation replayable

Save the immutable input dossier, tool/query versions, model and prompt versions, specialist outputs, and final disposition. Replay the same dossier after a code or model change. That isolates interpretation differences from changing retrieval results. Evaluate not only classification accuracy, but also unsupported claims, correct abstention, scope violations, and whether the system distinguishes unavailable evidence from negative evidence.

for case in fixed_evaluation_cases:
    dossier = load_versioned_snapshot(case)
    result = run_analysis(dossier, pinned_config)
    assert every_claim_has_evidence(result)
    assert result.scope == dossier.scope
    score(result, expected_disposition=case.label)

This pseudocode is a test outline, not a vendor command. A useful lab is to create a small synthetic set of alerts with known outcomes, then deliberately inject a timeout, a permission denial, and an out-of-window record. Verify the agent abstains or flags missing data rather than treating each as a clean negative.

Operational checklist

  1. Enforce tenant, account, and time boundaries in code, not solely in instructions.
  2. Attach provenance and explicit retrieval status to every evidence item.
  3. Keep raw evidence distinct from model-generated interpretation.
  4. Constrain specialist inputs and allowed output labels.
  5. Require citations from each material claim to dossier evidence.
  6. Log enough versions to reproduce an investigation without exposing secrets.
  7. Keep a human accountable for high-impact response actions; recommendations are not authorization.

Takeaway

Reliability comes less from asking a model to “be careful” and more from designing a system in which scope, provenance, missing data, and authority are explicit. Deterministic retrieval creates a stable evidence boundary; narrow models reason within it; replayable evaluation shows whether the system improved.

Primary sources