Designing a Deep Research Agent: Plans, Parallel Reading, and Verified Citations

    20 min read
    AI agents
    research automation
    system design
    LLM applications
    citation verification

    When I design a deep research agent, I start from the moment the user stops waiting. They type a hard question, something like "What are the main approaches to grid-scale long-duration energy storage, how do their costs compare, and which ones are actually being deployed?", answer two clarifying questions, and then walk away. Ten minutes later they come back to a long report with a citation behind nearly every factual sentence. In between, the system planned sub-questions, ran dozens of searches, read a hundred pages and PDFs, threw most of them away, took notes, noticed two sources disagreed, and wrote the report section by section.

    A single-turn answer engine fights for every hundred milliseconds. A research job has minutes to spend, but also a budget of hundreds of thousands of tokens, a crowd of parallel workers, and intermediate state that must survive crashes and deploys. The hard parts are deciding what to read, reading it cheaply, refusing to write anything the sources do not support, and keeping a long job affordable and resumable.

    In this post I walk through how I would build it: scoping, requirements with capacity estimates, a high-level architecture, and four deep-dives:

    • Planning: turning a question into a research plan
    • Parallel search and reading at scale
    • Claim verification and citations
    • Long-running jobs: budgets, checkpoints, and streaming the report

    Scoping the Problem and Clarifying Assumptions

    I am building a research job service: a question goes in, a multi-minute agent job runs, and a long cited report comes out. I describe the techniques generically; several assistant products ship a feature like this, and I make no claims about how any of them works inside.

    What I'm including:

    • A short clarification step before the job starts
    • A research plan shaped as a tree of sub-questions, revised as evidence arrives
    • Many sub-agents searching and reading in parallel, sharing one evidence store
    • Reading long documents (web pages, PDFs, reports) by extracting notes with exact provenance
    • Claim-to-source verification before any sentence makes it into the report
    • Token, cost and wall-clock budgets per job, checkpointing, and live progress

    What I'm NOT building here:

    • A web index or crawler. I call external search APIs and fetch pages on demand.
    • The single-turn answer path (query rewriting, re-ranking, a grounded paragraph in a few seconds). I covered that in Designing an AI Answer Engine and reuse its retrieval pieces here.
    • The generic durable agent runtime (leases, fencing, event logs, tool sandboxes). That is Designing an AI Agent Platform. I build on it and only describe what research adds.

    Key assumptions:

    • Jobs run 3 to 15 minutes, with a hard ceiling of 30
    • Reports are 2,000 to 6,000 words, with 30 to 100 distinct cited sources
    • I have access to a large reasoning model (for planning and writing) and a small, cheap model (for extraction and verification), both behind an LLM gateway
    • Users tolerate waiting if they see real progress, but they do not tolerate a report that cites a page which does not say what the report claims

    How this differs from the classic system: the nearest classic system is a batch job scheduler, not a search engine. A research job is a small DAG whose shape a model decides at runtime, whose tasks call flaky external services, and whose quality depends on what gets thrown away.

    Functional and Non-Functional Requirements

    Functional Requirements

    1. Clarify: before starting, ask at most a few questions when the request is ambiguous (scope, time range, region, depth), then freeze a research brief.
    2. Plan: produce a research plan the user can see, and revise it as the job learns.
    3. Research: search, fetch and read sources in parallel; extract notes tied to exact passages.
    4. Verify: check every factual claim in the draft against the notes it cites before it is shown.
    5. Write: produce an outline, then sections, then a summary, with inline citations and a source list.
    6. Stream progress: show plan, sources read, and finished sections while the job runs.
    7. Control: let the user cancel, or stop early and get a report from the evidence gathered so far.
    8. Resume: survive worker crashes and deploys without starting over or double-billing.

    Non-Functional Requirements

    RequirementTargetWhy
    Time to first progress eventunder 3 s after submitThe user must see the job is alive
    Job durationp50 under 8 min, hard cap 30 minLong enough to read widely, short enough to wait for
    Citation precisionevery cited claim supported by its sourceThe whole product promise
    Cost per reportpredictable, within a per-tier budgetUnbounded agents spend unbounded money
    Durabilityno job lost to a crash, resumes within a minuteMinutes of paid work per job
    Politenessrespect robots rules and per-domain rate limitsHundreds of fetches per job hit real websites

    Citation precision gets its own line because, in a report someone forwards to their manager, one bad citation destroys trust in the whole product.

    Back-of-the-Envelope Capacity Estimation

    These are my planning assumptions, not measurements. The point is the method: work out the shape of one job, then multiply.

    Jobs:

    • I assume 50,000 research jobs per day
    • 50,000 / 86,400 s is about 0.6 jobs started per second, about 1.8 per second at peak (3x)
    • With an average duration of 6 minutes (360 s), Little's law gives about 1.8 × 360 ≈ 650 jobs running at once at peak

    One job, by assumption:

    • A plan of about 15 sub-questions, each issuing about 4 searches: 60 search calls
    • About 150 candidate URLs come back, about 80 survive dedup and get fetched, and about 60 are actually read
    • Each read page averages about 6,000 tokens of extracted text after boilerplate removal

    Tokens per job:

    extraction (small model):   60 pages x 6,000 tokens         = 360K input tokens
    verification (small model): 150 claims x 1,500 tokens       = 225K input tokens
    planning + re-planning:     10 calls x 8,000 tokens         =  80K input tokens (large)
    outline + section drafting: 9 calls x 18,000 tokens         = 160K input tokens (large)
    output (notes, drafts, report)                              ~  30K output tokens (large)
    total                                                       ~ 855K tokens per job
    

    Fleet-wide at peak: 650 jobs × 855K tokens / 360 s is about 1.5 million tokens per second, most of it input to the small model. That number tells me the extraction model, not the writing model, is my capacity problem.

    Fetch and search load at peak: 650 × 80 fetches / 360 s ≈ 145 page fetches per second, and 650 × 60 / 360 ≈ 110 search calls per second. Both are modest for infrastructure but large for a single search provider contract, so I plan for several providers and a cache.

    Cost per report, with assumed list prices of 0.20permillioninputtokensonthesmallmodel,0.20 per million input tokens on the small model, 3 per million input and 15permillionoutputonthelargemodel,and15 per million output on the large model, and 0.005 per search call:

    small model:  0.585M x $0.20  = $0.12
    large input:  0.24M  x $3.00  = $0.72
    large output: 0.03M  x $15.00 = $0.45
    search:       60     x $0.005 = $0.30
    total                         ~ $1.60 per report
    

    So I set a default budget of about $2.50 per job with headroom, and the planner has to work inside it. The breakdown also shows where to save money: large-model input is the biggest line, so what I feed the writer matters more than how many pages I read.

    Storage: about 60 snapshots at roughly 100 KB compressed is about 7 MB per job, or 350 GB per day; 30-day snapshot retention is around 10 TB of object storage, which is cheap.

    High-Level Architecture and Core Components

    The research job architecture: the client submits a question and receives progress through an API gateway; a job and planner service stores the job, plan and checkpoints in a SQL database and pushes ready plan nodes to a task queue; researcher sub-agents call search providers, a fetch and parse service and the LLM gateway, and write notes to a shared evidence store, which a verifier and writer service turns into a report saved in blob storage.

    Component Walkthrough

    API Gateway. Authenticates the user, accepts the job, and holds a Server-Sent Events connection for progress. It reads progress from a stream, so a reconnecting browser simply resubscribes from its last event id.

    Job and planner service. Owns the job record and the plan. It runs clarification, writes the plan as a DAG into SQL, pushes ready nodes to the queue, and calls the re-planner as nodes finish. It is the only writer of the plan, which keeps plan updates simple.

    Researcher sub-agents. Stateless workers that take one plan node ("find 2024 to 2026 cost estimates for iron-air batteries"), run a search, read and extract loop, and write notes. They run on the agent platform's runtime, so leases and retries come for free.

    Fetch and parse service. Fetches URLs politely, converts HTML and PDF to structured text (headings, tables, page numbers), and stores a snapshot. It is shared across jobs, so it hosts the URL and content cache.

    Evidence store. The shared memory of a job: sources (URL, title, publisher, date, content hash, snapshot pointer) and notes (a short statement, the exact quoted span, its character offsets, the source id, and the plan node that produced it). The writer reads only from here, which is what makes citations checkable.

    Verifier and writer. Outlines from the notes, drafts sections, verifies every claim against its cited notes, and assembles the report in blob storage.

    LLM gateway. Every model call goes through it, so token quotas, model routing (small for extraction, large for planning and writing) and cost metering live in one place.

    Request Flow

    1. The client sends POST /v1/research; the gateway returns a job id and opens the progress stream.
    2. If the question is ambiguous, the planner asks one to three clarifying questions; otherwise it plans straight away.
    3. The planner writes a research brief and a plan DAG of 10 to 20 sub-questions, streamed to the user.
    4. Sub-agents take ready nodes, search, dedup, fetch, read, and write notes.
    5. As nodes finish, the re-planner adds follow-ups, prunes dead ends, or declares the evidence enough.
    6. The writer outlines, drafts and verifies sections in parallel, streaming each finished one.
    7. A final pass writes the summary and source list, and the job completes with its cost recorded.

    Scaling It Out

    The scaled-out system: a regional gateway feeds a sharded job service, which splits work into an interactive research queue and a background queue for scheduled jobs; an autoscaled sub-agent pool calls models through an LLM gateway with token quotas and fetches pages through a shared fetch fleet with a URL and content cache behind egress proxies, while cost, progress and queue depth go to monitoring.

    • Shard by job id. A job's plan, checkpoints and evidence live together on one shard, so plan updates are local transactions.
    • Two queues. Interactive jobs (a user is watching) get priority. Scheduled or API batch jobs ("refresh this report weekly") fill idle capacity and are preempted first.
    • Autoscale on queue depth and in-flight model calls, not CPU. Sub-agents mostly wait on I/O, so one process runs many of them with async I/O.
    • The fetch fleet is shared and cached. Popular sources get fetched by many jobs, so a content cache keyed by normalised URL, with a freshness window per content type, saves time and politeness budget.
    • Egress through proxies with fixed addresses, which keeps fetchers away from internal networks.

    Deep-Dive: Planning: Turning a Question into a Research Plan

    The planning flow: the user question goes to a clarifier that may ask one to three questions, then a research brief feeds the planner, which writes a plan DAG to SQL; a ready-node scheduler fans out sub-questions to workers, whose findings per node trigger a re-planner that adds, prunes or stops nodes and records each plan revision.

    Clarify First, Once

    "Compare vector databases" could mean a feature table, benchmarks, pricing, or a recommendation for one workload. Guess wrong and I spend $2 and eight minutes on the wrong report. So a fast model call first classifies the question and, if ambiguous, generates at most three clarifying questions with clickable suggested answers.

    Rules I hold to:

    • Only ask what changes the plan. Scope, time range, region, audience and depth. Never "what format would you like?" unless it matters.
    • Ask once, at the start. Mid-job ambiguity is resolved by stating the assumption in the report, not by making the user babysit the job.
    • Time out to defaults. No answer within a few minutes means the job proceeds with stated assumptions.

    The output is a research brief: the question restated, the scope decisions, and what a good report must answer. Every later model call sees the brief, not the raw question, so the job does not drift.

    The Plan Is a DAG, Not a List

    A flat list misses two things: some questions depend on others ("which companies deploy iron-air batteries?" comes before "what did they report about cost?"), and some matter more than others. So the plan is a DAG of nodes:

    {
      "node_id": "n7",
      "question": "Reported capital cost per kWh for iron-air storage, 2024 to 2026",
      "parent": "n2",
      "depends_on": ["n4"],
      "purpose": "fills the cost comparison table in section 3",
      "priority": 0.8,
      "max_searches": 4,
      "status": "READY",
      "stop_when": "two independent sources with dated figures"
    }
    

    The fields that matter most are purpose and stop_when. A sub-agent without a stopping condition reads forever, and one that knows why it is searching judges relevance far better.

    The planner writes nodes two or three levels deep. Deeper levels come from the re-planner, once the job knows what is actually out there.

    Scheduling Ready Nodes

    A node is ready when its dependencies are done. The scheduler pushes ready nodes in priority order, with a per-job concurrency cap (say 8 sub-agents at once). The cap stops one job from monopolising the pool, and it gives the re-planner a chance to act on early findings before the whole budget is spent on the original plan.

    Re-Planning as Evidence Arrives

    Every time a node finishes, the re-planner sees a compact summary: the brief, the plan with statuses, and for each finished node the notes count, a two-line digest and any open questions the sub-agent reported. It can do four things:

    1. Add nodes for a new lead ("three sources mention a 2025 pilot in Minnesota")
    2. Prune nodes that are now redundant or that returned nothing useful after their search budget
    3. Re-prioritise nodes based on which report sections are still thin
    4. Stop research when coverage is enough, handing over to writing
    def on_node_done(job, node):
        job.plan.mark_done(node.id, digest=node.digest)
        if job.budget.research_fraction_used() > 0.75 or job.plan.coverage() >= 0.9:
            return job.start_writing()                     # enough, or out of research budget
    
        if job.plan.revisions_since(node.finished_at) == 0:  # debounce: one re-plan per wave
            change = replanner.call(job.brief, job.plan.summary(), max_new_nodes=5)
            job.plan.apply(change, revision=job.plan.revision + 1)   # validated, versioned
    
        for ready in job.plan.ready_nodes(limit=job.concurrency_free()):
            queue.push(ready, priority=ready.priority)
    

    Every plan change is a new revision with a diff, and the diff is what I stream to the user ("added: Minnesota pilot results; dropped: flow battery history"). It is also what makes the plan auditable afterwards.

    Coverage: Knowing When to Stop

    "Is this enough?" is the hardest question in the job, and I do not let the model answer it from vibes. The brief lists the questions a good report must answer, and coverage is the fraction with at least two notes from independent sources. Research stops at high coverage, at the research share of the budget (about 75%, keeping the rest for writing and verification), or at the wall-clock limit, whichever comes first.

    Failure Modes and What I Would Do First

    • Plan explosion. Re-planners love adding nodes. I cap new nodes per revision and total nodes per job, and every new node must name the report section it serves.
    • Topic drift. Following leads can wander far from the question. The re-planner always sees the brief, and nodes that do not map to a brief question are rejected.

    What I would build first: clarification, a two-level plan, and a fixed concurrency cap, with re-planning limited to "add follow-ups" and "stop". Pruning and re-prioritisation come once I have traces showing where budget is wasted.

    Deep-Dive: Parallel Search and Reading at Scale

    The read path for one sub-agent: it sends queries to search providers, result URLs pass through a seen-URL and content-hash dedup cache so only new sources reach the fetcher, which applies per-domain limits, fetches web pages and PDFs, stores snapshots, and hands text to a parse and chunk worker; relevant chunks go to a note extractor model, which writes notes with exact spans to the evidence store.

    Sub-Agents with a Shared Evidence Store

    Each sub-agent runs a short, bounded loop on one node: search, pick promising results, read, take notes, and check stop_when. The key decision is that sub-agents do not talk to each other. They coordinate only through the evidence store: before fetching a URL they check whether the job already has it, and they can see notes other nodes wrote on related topics.

    I chose a shared store over message passing because dedup, citation, checkpointing and the writer then all work against one table, and a crashed sub-agent leaves behind only notes it already committed.

    Dedup at Three Levels

    At this scale, the same content shows up again and again, so I dedup at three levels:

    1. URL. Normalise (lowercase host, strip tracking parameters, resolve redirects, prefer canonical links) and check the job's seen set. This catches most repeats before any fetch.
    2. Content. Syndicated news, mirrored PDFs and press releases appear under many URLs. After fetching, a hash of the normalised text catches exact copies, and a near-duplicate check (shingle-based MinHash is a standard choice) catches lightly edited ones. Duplicates are recorded as aliases of the first source.
    3. Claim. Ten articles repeating one press release are not ten sources. Notes carry a derived_from hint when the extractor sees attribution ("according to the company's announcement"), and the verifier uses it when counting independent support.

    The third level matters most: "widely reported" is misleading when every source quotes one blog post.

    Fetching Politely and Defensively

    Six hundred jobs fetching at once can look like an attack on a popular domain. The fetcher keeps a token bucket per domain across all jobs, honours robots rules and crawl delays, uses a clear user agent, and caches aggressively. Each fetch has a hard timeout (say 10 s for HTML, 30 s for large PDFs) and a size cap, so a slow site costs one source, never the job.

    Fetched content is untrusted input. Pages can carry instructions aimed at the model. The extractor treats page text as data and emits a fixed note schema rather than free text for the next prompt, and fetchers run in a network zone that cannot reach internal services.

    Reading Long Documents: Chunked Extraction into Notes

    Pasting whole pages into the writer's context means 360K tokens before writing a word, most of it irrelevant, and models use information buried in huge contexts poorly. So nobody reads raw pages except the extractor.

    The parser splits each document into structural chunks (sections, paragraphs, table rows, PDF pages), keeping headings and page numbers. A cheap relevance pass (embedding similarity against the node's question and purpose) picks the chunks worth reading; a 200-page PDF might yield 15. The small extractor model then reads those chunks with the node's question and writes notes:

    {
      "note_id": "src31-n7-004",
      "source_id": "src31",
      "node_id": "n7",
      "statement": "<illustrative> A utility filing estimated an installed cost of X dollars per kWh for a 100-hour iron-air project.",
      "quote": "<exact sentence copied from the source>",
      "locator": {"page": 14, "char_start": 2210, "char_end": 2262},
      "source_date": "2025-03-11",
      "derived_from": null,
      "confidence": "explicit"
    }
    

    Three properties make this work:

    • The quote is verbatim and located. After extraction, a plain string check confirms the quote really appears at those offsets in the snapshot. Notes whose quotes do not match are dropped. This one cheap check removes a whole class of hallucinated evidence.
    • The statement is atomic. One fact per note, so the verifier can match claims to notes one to one.
    • Provenance travels with the note. Source, page, date and which plan node asked for it. The writer never sees a fact without its origin.

    Notes are much smaller than pages, about 40 to 80 tokens each, so a job with 600 notes gives the writer roughly 40K tokens of dense, attributed evidence instead of 360K tokens of pages.

    Failure Modes and What I Would Do First

    • Search provider outages. At least two providers behind one interface.
    • Paywalls and blocked pages. The fetcher records them as unreadable sources so the planner can look elsewhere, rather than silently citing a search snippet.
    • SEO spam crowding out primary sources. Source quality scoring (primary documents, official filings, peer-reviewed papers and established publishers rank higher) feeds into which results a sub-agent reads first.

    What I would build first: URL dedup, per-domain limits, the content cache, and extraction with verbatim quote checks. Near-duplicate detection and derived_from tracking come next, once I can measure how often reports overcount sources.

    Deep-Dive: Claim Verification and Citations

    Verification of each section: the section drafter reads from the evidence store, a claim splitter breaks the draft into atomic claims, each claim is looked up against its cited notes, and an entailment judge sends supported claims to keep and cite, unsupported claims to revise or drop, and conflicting evidence to a contradiction note, all of which form the verified section.

    Outline First, Then Sections

    A 5,000-word report written in one call starts strong and fades, with citations getting vaguer as it goes. So the writer works in stages:

    1. Outline. The large model sees the brief and a digest of notes grouped by plan node, and writes an outline: sections, the key points each must make, and which note ids support each point.
    2. Section drafting, in parallel. Each section is drafted with only its assigned notes plus the outline for context. Every factual sentence must end with citation markers that reference note ids, not URLs.
    3. Verification per section, described below.
    4. Summary last. The executive summary is written from the verified sections, never from raw notes, so it cannot claim anything the body does not support.

    Citing note ids instead of URLs is the key decision. A URL says "somewhere on this page". A note id points at an exact quote, which is what makes automatic verification possible.

    Claims, Not Sentences

    A sentence like "Technology A is cheaper per kWh than technology B but less efficient" holds two claims. The claim splitter (a small model call) breaks each drafted paragraph into atomic claims and keeps the mapping back to the text span and the note ids cited.

    For each claim, the verifier:

    1. Loads the cited notes, plus the top few other notes from the job by similarity (to catch contradictions the drafter skipped)
    2. Asks an entailment judge: does this evidence support, contradict, or not address the claim?
    3. Checks numbers and dates with code, not the model: a figure in the claim must appear in a cited quote, within rounding
    def verify_section(section, store):
        verdicts = []
        for claim in split_claims(section.text):
            cited = store.notes(claim.note_ids)
            nearby = store.similar_notes(claim.text, k=5, exclude=claim.note_ids)
            verdict = judge(claim.text, cited, nearby)          # SUPPORTED / CONTRADICTED / NOT_ADDRESSED
            if verdict.label == "SUPPORTED" and not numbers_match(claim, cited):
                verdict = verdict.downgrade("number not in quote")
            verdicts.append((claim, verdict))
    
        for claim, v in verdicts:
            if v.label == "NOT_ADDRESSED":
                section.revise_or_drop(claim)                   # one rewrite attempt, then drop
            elif v.conflicting_notes:
                section.add_contradiction(claim, v.conflicting_notes)
        return section.with_audit(verdicts)
    

    A claim that fails gets one rewrite attempt with the judge's reason ("the source says 'about $20' for a single project, not an industry average"). If the rewrite still fails, the claim is dropped. I would rather publish a shorter report than a wrong one.

    Contradictions Are Content

    Disagreement between sources is often the most useful thing in a report. When the verifier finds contradicting notes, the model does not silently pick a winner. The section states who says what, when, and on what basis (a vendor estimate versus an independent study, a 2023 figure versus a 2026 one). Dates, publisher type and derived_from let the writer explain the conflict rather than average it.

    Citations the Reader Can Check

    The final report turns note ids into numbered citations carrying title, publisher, date, URL and the exact quote, shown on hover. Because I kept snapshots, a citation still shows what the page said when it was read, labelled with the fetch date.

    Each report stores a verification audit (claims supported, revised, dropped or contested). It is my core quality metric, alerted per model version and topic, with sampled human review as in LLM evaluation and observability.

    Cost of Verification

    Verification is about a quarter of the job's tokens in my estimate. The judge is the small model, since entailment against a short quote is a narrow task, escalating to the large model only when unsure, and sentences with no factual claim are skipped.

    Failure Modes and What I Would Do First

    • Citation laundering. The drafter cites a real note that is only loosely related. Entailment judging per claim, not per sentence, is the defence.
    • Over-pruning. A strict judge drops correct claims phrased differently from the quote. I tune thresholds on a labelled set and track the drop rate per section.
    • Unverifiable synthesis. Some valuable sentences are inferences across sources ("together these suggest costs fell by half"). I allow them only when marked as analysis and backed by the specific notes being combined.

    What I would build first: note-id citations, claim splitting, small-model entailment with the numbers check, and drop-on-fail. Contradiction surfacing comes second, because it needs the similar-notes lookup and careful writing prompts.

    Deep-Dive: Long-Running Jobs: Budgets, Checkpoints, and Streaming the Report

    The job lifecycle: a job runner reserves and charges spending against a budget meter, commits a checkpoint to SQL at every phase, and drives an outline writer, parallel section drafters and a report assembler that writes the final report to blob storage; the section drafters and report assembler publish section-ready and phase-done events to a progress stream that an SSE endpoint relays to the browser.

    Three Budgets per Job

    Every job carries three budgets, set by tier and visible in the job record:

    BudgetDefault (assumed)What happens when it runs out
    Cost$2.50Stop research, write from current evidence
    Tokens per model class700K small, 400K largeSame, and alert if hit often
    Wall clock15 min soft, 30 min hardSoft: start writing; hard: deliver what exists

    The budgets are split by phase: about 75% for research, the rest reserved for writing and verification. Without the reservation, a job can research until the money runs out and have nothing left to write with: the user paid and got nothing.

    Spending uses reserve then settle. Before a model call, the sub-agent reserves its expected cost (prompt tokens plus max_tokens) from the job's budget meter; after the call, it settles the actual amount. With eight sub-agents running at once, reservation stops them jointly overshooting. Per-tenant token limits in the LLM gateway sit on top.

    Graceful Degradation Instead of Failure

    When a budget is hit, the job does not fail. It moves on with what it has and the report says so: "Research stopped at the time limit; sections 4 and 5 rely on fewer sources than planned." The user's stop early button uses the same path, which works because notes are committed continuously.

    Checkpoints at Phase Boundaries

    The agent platform post covers the generic mechanics: leased steps, fencing epochs and an append-only event log so a crashed worker's step is retried elsewhere. Research jobs get a big advantage on top of that, because almost everything they produce is a durable, idempotent artifact:

    • Plan revisions are versioned rows
    • Sources are keyed by normalised URL and content hash
    • Notes are keyed by source, node and a sequence number
    • Sections are keyed by outline id and revision

    So the checkpoint is not a serialised agent state. It is the job's phase and plan revision, plus a pointer to these artifacts:

    CREATE TABLE research_jobs (
      job_id         UUID PRIMARY KEY,
      tenant_id      UUID NOT NULL,
      phase          TEXT NOT NULL,     -- CLARIFYING, RESEARCHING, OUTLINING, DRAFTING, VERIFYING, DONE
      plan_revision  INT  NOT NULL,
      outline_id     UUID,              -- set once OUTLINING commits
      budget_usd     NUMERIC(8,4) NOT NULL,
      reserved_usd   NUMERIC(8,4) NOT NULL DEFAULT 0,
      spent_usd      NUMERIC(8,4) NOT NULL DEFAULT 0,
      soft_deadline  TIMESTAMPTZ NOT NULL,
      hard_deadline  TIMESTAMPTZ NOT NULL,
      progress_seq   BIGINT NOT NULL DEFAULT 0
    );
    

    Resuming after a crash is a matter of reading the phase and asking what is missing. During research: which ready nodes have no committed "done" record? Re-queue them; their already-fetched sources hit the dedup set, so the retry is cheap. During drafting: which outline sections have no verified revision? Draft only those. Resume cost is bounded by the work in flight, never the whole job.

    A retried node may produce slightly different notes, and I accept that. Committed notes from the failed attempt are kept (they are real evidence with verified quotes), and the retry dedups against them.

    Streaming Progress, Then the Report

    Eight minutes of spinner looks broken. Progress events make the wait legible:

    id: 41
    event: plan
    data: {"revision": 3, "added": ["Minnesota pilot results"], "dropped": ["flow battery history"]}
    
    id: 57
    event: sources
    data: {"read": 38, "skipped_duplicates": 21, "unreadable": 4}
    
    id: 88
    event: section
    data: {"outline_id": "s3", "title": "Cost comparison", "status": "verified"}
    

    Events go to a per-job stream that the gateway relays over SSE, so a reconnect replays from the last event id. I stream verified sections, not raw drafts: text that may vanish a minute later teaches users not to trust early output.

    Users rarely keep the tab open, so completion sends a notification with a link, and the report loads from blob storage, not the job runtime.

    Cost per Report as the Unit Metric

    The number on my dashboard is cost per verified report, split by phase (planning, search, extraction, drafting, verification) and by tier. It moves with a few levers, roughly in order of impact:

    1. What the writer sees. Notes instead of pages keeps large-model input small.
    2. Extraction selectivity. The relevance pass decides how many chunks the small model reads.
    3. Dedup and cache hit rate. Every duplicate caught before fetch saves a fetch, a parse and an extraction.
    4. Plan discipline. Caps on nodes and searches, and stopping at coverage rather than at budget.
    5. Wasted jobs. Jobs abandoned after clarification, or cancelled late.

    I pair it with cost per supported claim, so a cheaper week that quietly drops more claims shows up as a regression.

    Failure Modes and What I Would Do First

    • Runaway jobs. A bug in the re-planner keeps adding nodes. Hard caps on nodes, budget and wall clock, enforced by the job runner rather than the model, contain it.
    • Thundering herd after an outage. Resumes enter the normal queue with jitter, and the per-job concurrency cap still applies.
    • Stuck clarification. Jobs waiting for a user answer hold no workers; they sit in the job table with a timer and proceed with defaults when it fires, using the durable timers from workflow orchestration.

    What I would build first: the three budgets with phase reservation, phase checkpoints over idempotent artifacts, and progress events for plan, sources and sections. Section streaming and notifications are cheap to add once those exist.

    Conclusion: Bringing It All Together

    A deep research agent looks like a chatbot that thinks for longer, but its design is closer to a small, model-driven batch system: a DAG decided at runtime, parallel and unreliable workers, rate-limited external services, and output that must be checkable line by line. Retrieval and grounded generation carry over from answer engines. What research adds is everything around time: plans that change, evidence that accumulates, budgets that run out, and crashes in the middle.

    The most important structural choice is the evidence store with verbatim, located notes. It is what sub-agents share instead of messages, what the writer reads instead of raw pages, what citations point at, what verification checks against, and what checkpoints are made of. Most of the other decisions fall out of it.

    If I had to summarise the design in four decisions:

    1. Clarify once, then plan as a DAG with purposes and stopping conditions. Re-plan in waves, and stop on coverage, not on budget.
    2. Fan out sub-agents that coordinate only through a shared evidence store, with dedup at the URL, content and claim levels and extraction into small, quoted notes.
    3. Cite note ids, split drafts into claims, and verify each one before it is shown. Drop what fails, and surface contradictions instead of hiding them.
    4. Give every job cost, token and time budgets with a reserved share for writing, checkpoint at phase boundaries over idempotent artifacts, and stream verified progress.

    Related reading: Designing an AI Answer Engine, Designing an AI Agent Platform, Designing an LLM Gateway, and Designing a Workflow Orchestration Engine.

    Structured data for LLMs, AI agents, and automated crawlers is available at/blog/system-design-deep-research-agent.md. Please reviewrobots.txt andllms.txt before crawling. All referenced data must be credited to roundz.ai with a link tohttps://roundz.ai