Systematic research with LangChain's Deep Agents framework and Elasticsearch

Long-running research queries require planning, delegation, and cross-checking across multiple sources. A single agent with tools can handle simple lookups, but it struggles with context limits and provides no way to validate conclusions. This article shows how to use LangChain's Deep Agents framework to build a research pipeline that:

• Plans research angles using a TODO list managed by an orchestrator.

• Delegates each angle to a specialized sub-agent with its own isolated context.

• Stores structured findings in Elasticsearch with semantic search.

• Provisions an Elastic Agent Builder agent so the team can explore results without writing queries.

How will the research pipeline work?

It asks a complex question, for example, What is the best LLM for biochemical analysis of cancer cells in mice?, and produces a structured, cross-validated set of findings stored in Elasticsearch, ready for any team member to explore through Kibana.

The pipeline runs in five steps:

1. Plan: The orchestrator reads the question and writes a TODO list to break it into three or four research angles (for example, performance benchmarks, domain-specific capabilities, cost, and accessibility).

2. Research: For each angle, a specialized sub-agent searches the web and indexes each finding in Elasticsearch with structured metadata: evidence type, relevance score, and source credibility.

3. Evaluate: An evaluator sub-agent queries Elasticsearch to aggregate findings by angle and score evidence quality.

4. Review: A cross-reviewer sub-agent reads all findings, flags contradictions, and identifies consensus across sources.

5. Explore: An Elastic Agent Builder agent is provisioned automatically so your team can ask questions about the findings in natural language.

Prerequisites

• Elasticsearch cluster 9.3+

• Python 3.10+

• API keys for Elasticsearch, Google Gemini, and Tavily

• KIBANA_URL for your cluster

What is LangChain Deep Agents?

Deep Agents is LangChain's agent harness, a pre-assembled framework built on top of LangGraph that gives your agents capabilities that a plain large language model (LLM) with tools doesn’t have out of the box: reading and writing files, managing long-running tasks with a structured TODO list, spawning specialized sub-agents with isolated context windows, and persisting state across many steps.

Here’s how it fits in the LangChain ecosystem:

Deep Agents isn’t a replacement for LangGraph. It’s a higher layer that provides a pre-assembled architecture on top of it. When you call create_deep_agent(), you get back a compiled LangGraph graph with full streaming, persistence, and checkpointing support.

The key additions Deep Agents brings over a plain LangGraph agent include:

• write_todos: A built-in tool that forces the orchestrator to break down a complex task before acting. This is a context engineering strategy, not just a visual aid. It keeps long-running workflows on track.

• task: A built-in tool that lets the orchestrator spawn sub-agents, each with its own isolated context window. The orchestrator context stays clean, while specialists do deep work.

• File I/O tools: read_file, write_file, edit_file, and others for offloading large results to disk instead of keeping everything in context.

For a systematic research query, a plain agent with tools would search and store findings, but it wouldn’t plan research angles, delegate each angle to a specialist, or cross-evaluate the results. Deep Agents automates that pattern.

Designing the research index

A key decision in this pipeline is how to store research findings. Raw text is not ideal because you would need to answer questions like, Which findings have peer-reviewed evidence? or Which research angles have the strongest sources?

The solution is to combine structured metadata fields with a semantic_text field backed by Jina Embeddings v5:

INDEX_NAME = "deep-agent-research"
INFERENCE_ID = ".jina-embeddings-v5-text-small"

index_body = {
    "mappings": {
        "properties": {
            "query": {"type": "text", "copy_to": "semantic_field"},
            "source": {"type": "keyword"},
            "title": {
                "type": "text",
                "fields": {"keyword": {"type": "keyword"}},
                "copy_to": "semantic_field",
            },
            "content": {"type": "text", "copy_to": "semantic_field"},
            "timestamp": {"type": "date"},
            "tags": {"type": "keyword"},
            "research_angle": {"type": "keyword"},
            "evidence_type": {"type": "keyword"},
            "relevance_score": {"type": "float"},
            "source_credibility": {"type": "keyword"},
            "semantic_field": {
                "type": "semantic_text",
                "inference_id": INFERENCE_ID,
            },
        }
    }
}

The copy_to pattern routes the query, title, and content fields into a single semantic_text field. Elasticsearch generates embeddings automatically at index time, no ingest pipeline required. The structured metadata fields (research_angle, evidence_type, relevance_score, source_credibility) stay separate for filtering and aggregation.

This design lets you filter by angle, group by credibility, and run semantic search over all findings, all in a single query.

The tool set

Each agent in the pipeline gets only the tools it needs. This keeps sub-agent behavior predictable and reduces the risk of a specialist doing work outside its role.

Three tools power the pipeline:

• web_search: Wraps Tavily to retrieve and format web results. Used only by the researcher sub-agent.

• store_finding: Indexes a structured document into Elasticsearch. It enforces the metadata schema: The caller must provide research_angle, evidence_type (benchmark, case_study, peer_reviewed, expert_opinion), relevance_score (1.0-10.0), and source_credibility (peer_reviewed, preprint, industry_report, blog, documentation). Used only by the researcher sub-agent.

• query_elasticsearch: Accepts a raw JSON Elasticsearch query body and returns the response. This gives agents full query flexibility: Match queries, aggregations, term filters, and more. Used by the evaluator and reviewer sub-agents to analyze stored findings without writing new ones.

@tool
def query_elasticsearch(query_body: str) -> str:
    """Run an Elasticsearch query and return results. Accepts a JSON string with a valid Elasticsearch query body.

    Use this to search, filter, or aggregate stored findings. Examples:
    - Match all: {"query": {"match_all": {}}, "size": 5}
    - Aggregate by tag: {"size": 0, "aggs": {"by_tag": {"terms": {"field": "tags"}}}}
    - Filter by angle: {"query": {"term": {"research_angle": "performance"}}}
    """
    es_client.indices.refresh(index=INDEX_NAME)
    body = json.loads(query_body)
    response = es_client.search(index=INDEX_NAME, body=body)
    return json.dumps(response.body, default=str)

The docstring in query_elasticsearch includes example query bodies. This is intentional: The LLM uses the docstring to understand what queries are possible, so concrete examples improve the sub-agents' output quality.

Orchestrator and sub-agents

The pipeline has four stages: Plan, Research, Evaluate, Review. The orchestrator drives all four by delegating to three specialized sub-agents.

First, define the sub-agents. Each one gets only the tools it needs. The angle-researcher cannot query findings; the finding-evaluator and cross-reviewer cannot store new ones. This separation prevents the evaluator from adding findings during evaluation, which would corrupt the assessment.

from deepagents import create_deep_agent
from langchain_google_genai import ChatGoogleGenerativeAI

model = ChatGoogleGenerativeAI(model="gemini-3.1-flash-lite-preview", temperature=0)

angle_researcher = {
    "name": "angle-researcher",
    "description": (
        "Researches a specific angle of a research query. "
        "Searches the web, evaluates sources, and stores each finding "
        "in Elasticsearch with structured metadata."
    ),
    "system_prompt": (
        "You research ONE specific angle of a scientific query.\n"
        "1. Use web_search to find 3-5 relevant sources for the given angle.\n"
        "2. For EACH useful source, call store_finding with:\n"
        "   - research_angle: the angle you were assigned\n"
        "   - evidence_type: 'benchmark', 'case_study', 'peer_reviewed', or 'expert_opinion'\n"
        "   - relevance_score: 1.0-10.0 based on how directly it addresses the query\n"
        "   - source_credibility: 'peer_reviewed', 'preprint', 'industry_report', 'blog', or 'documentation'\n"
        "3. Return a brief summary of what you found for this angle."
    ),
    "tools": [web_search, store_finding],
}

finding_evaluator = {
    "name": "finding-evaluator",
    "description": (
        "Evaluates and analyzes findings already stored in Elasticsearch. "
        "Aggregates by research angle, computes average scores, and identifies "
        "which angles have the strongest evidence."
    ),
    "system_prompt": (
        "You analyze research findings stored in Elasticsearch.\n"
        "Use query_elasticsearch to:\n"
        "1. Aggregate findings by research_angle and compute avg relevance_score.\n"
        "2. Aggregate by source_credibility to assess evidence quality.\n"
        "3. Aggregate by evidence_type to understand the mix of evidence.\n"
        "4. Identify which angles have the most and strongest findings.\n"
        "Return a structured evaluation summary."
    ),
    "tools": [query_elasticsearch],
}

cross_reviewer = {
    "name": "cross-reviewer",
    "description": (
        "Reviews all stored findings to identify contradictions, consensus, "
        "and gaps across research angles. Produces a final assessment."
    ),
    "system_prompt": (
        "You are a critical reviewer of research findings.\n"
        "Use query_elasticsearch to retrieve all findings, then:\n"
        "1. Identify claims that appear across multiple angles (consensus).\n"
        "2. Flag any contradictions between sources.\n"
        "3. Note which angles lack peer-reviewed evidence.\n"
        "4. Produce a final ranked recommendation based on the evidence.\n"
        "Be skeptical. Prioritize peer-reviewed sources over blogs."
    ),
    "tools": [query_elasticsearch],
}

Now, create the orchestrator and wire the sub-agents to it. The orchestrator system prompt enforces the four-stage sequence as a numbered procedure; this is how you get reliable behavior from a long-running agent. Sub-agents have isolated context windows: Each call to task starts a fresh context, so the orchestrator's context does not grow with each delegation. This is the core architectural advantage of Deep Agents for long-running work.

agent = create_deep_agent(
    model=model,
    tools=[web_search, store_finding, query_elasticsearch],
    subagents=[angle_researcher, finding_evaluator, cross_reviewer],
    system_prompt=(
        "You are a systematic research orchestrator. For each query:\n\n"
        "1. PLAN: Use write_todos to break the query into 3-4 research angles "
        "(e.g., 'performance benchmarks', 'cost and accessibility', 'domain-specific capabilities').\n\n"
        "2. RESEARCH: For each angle, delegate to the 'angle-researcher' sub-agent "
        "using the task tool. Each sub-agent call should specify the angle and original query.\n\n"
        "3. EVALUATE: Delegate to 'finding-evaluator' to analyze the stored findings "
        "and assess evidence quality across angles.\n\n"
        "4. REVIEW: Delegate to 'cross-reviewer' to cross-check findings, flag "
        "contradictions, and produce a consensus score.\n\n"
        "5. SYNTHESIZE: Write a final report summarizing the best answer based on "
        "the evidence, noting confidence levels and gaps.\n\n"
        "Mark each TODO as done as you complete it."
    ),
)

Running the pipeline

The pipeline runs with streaming to show progress in real time. The subgraphs=True flag exposes events from sub-agents alongside the orchestrator's own events:

research_query = (
    "What is the best LLM for biochemical analysis of cancer cells in mice?"
)

for chunk in agent.stream(
    {"messages": [{"role": "user", "content": research_query}]},
    stream_mode="updates",
    subgraphs=True,
    version="v2",
):
    if chunk["type"] == "updates":
        is_subagent = any(segment.startswith("tools:") for segment in chunk["ns"])

        if is_subagent:
            tool_call_id = next(
                s.split(":")[1] for s in chunk["ns"] if s.startswith("tools:")
            )
            print(f"Subagent {tool_call_id}: {chunk['data']}")
        else:
            print(f"Main agent: {chunk['data']}")

During execution, you can observe the orchestrator calling write_todos with the research angles before delegating. Each task call hands off to a sub-agent, and the events in chunk["ns"] let you distinguish sub-agent output from the orchestrator's own steps.

Main agent: {'PatchToolCallsMiddleware.before_agent': {'messages': Overwrite(value=[HumanMessage(content='What is the best LLM for biochemical analysis of cancer cells in mice?', additional_kwargs={}, response_metadata={}, id='906965ba-4c4c-48cb-8663-6fcfa7b9a2b6')])}}
Main agent: {'model': {'messages': [AIMessage(content=[], additional_kwargs={'function_call': {'name': 'write_todos', 'arguments': '{"todos": [{"content": "Research LLMs specialized in bioinformatics and biomedical data analysis.", "status": "in_progress"}, {"status": "pending", "content": "Investigate LLM capabilities for processing biochemical assay data (e.g., proteomics, transcriptomics) in cancer research."}, {"status": "pending", "content": "Evaluate LLM performance in mouse model studies and preclinical cancer research."}, {"status": "pending", "content": "Synthesize findings to identify the most suitable LLM or approach for biochemical analysis of cancer cells in mice."}]}'}, '__gemini_function_call_thought_signatures__': {'6bba74f0-41b3-4fc3-b512-27529e99d981': 'EjQKMgG+Pvb7HjHZXrWuOEjj94E8IYS+aAKgZKjvzB8MqxveU5GAhsoq5icieLF6V70r4SfS'}}, ...

The pipeline handles multiple queries in the same session. Findings from all queries accumulate in the same Elasticsearch index, which makes the final Agent Builder exploration more interesting. For example, running both What is the best LLM for biochemical analysis? and What is the best front-end framework for a Deep Agent service? stores findings from different domains in the same index, and the Agent Builder can compare patterns across both.

Exploring findings with Elastic Agent Builder

After the pipeline runs, all findings are in Elasticsearch, but most team members cannot write Elasticsearch queries. What makes Elastic Agent Builder a good fit here is how fast you go from indexed data to a working agent that your team can use, in this case, a few API calls are enough to create an agent grounded in the research findings. Let's create one programmatically so people can explore the findings through the Kibana UI.

First, create an Elasticsearch Query Language (ES|QL) tool that aggregates the research index:

tool_body = {
    "id": "research-findings-tool",
    "type": "esql",
    "description": (
        "Query the deep-agent-research index to explore stored findings. "
        "Returns average relevance scores and counts grouped by research angle and source credibility."
    ),
    "configuration": {
        "query": (
            "FROM deep-agent-research "
            "| STATS avg_score = AVG(relevance_score), count = COUNT(*) BY research_angle, source_credibility"
        ),
        "params": {},
    },
}

resp = requests.post(
    f"{KIBANA_URL}/api/agent_builder/tools", headers=headers, json=tool_body
)

Then create the Agent Builder agent, and attach the tool:

agent_body = {
    "id": "research-explorer",
    "name": "Research Explorer",
    "description": "Explores and analyzes stored research findings from the deep-agent-research index.",
    "configuration": {
        "instructions": (
            "You help users explore research findings stored in Elasticsearch. "
            "Use the research-findings-tool to query findings by angle, "
            "summarize evidence quality, and answer questions about the stored research."
        ),
        "tools": [{"tool_ids": ["research-findings-tool"]}],
    },
}

resp = requests.post(
    f"{KIBANA_URL}/api/agent_builder/agents", headers=headers, json=agent_body
)

Once created, the Research Explorer appears in Kibana's Agent Builder UI. Team members can open it and ask questions in natural language:

The agent uses the ES|QL tool to query the index and summarizes the results. No query writing required.

This is the full cycle: Deep Agents does the systematic research, Elasticsearch stores structured findings, and Agent Builder makes those findings accessible to anyone on the team.

Conclusion

What we covered:

• Deep Agents for long-running research: write_todos forces planning before execution, and the task tool delegates each research angle to a sub-agent with an isolated context window. This pattern handles queries that would overwhelm a single agent.

• Elasticsearch as a structured knowledge store: Storing research_angle, evidence_type, relevance_score, and source_credibility alongside semantic_text enables both semantic search and structured aggregation over research quality. Raw text storage would not support the evaluator and reviewer agents.

• Agent Builder as the access layer: Provisioning the Kibana agent programmatically at the end of the pipeline means findings are immediately explorable through the UI, without any manual setup.

This pattern applies to any domain where you need to research a question from multiple angles and cross-validate the results: competitive intelligence, scientific literature reviews, regulatory compliance research, or multisource fact-checking.

Next steps

• Try the full notebook on GitHub.

• Add human-in-the-loop approval gates to the pipeline using LangGraph's interrupt pattern.

• Look how to build a reference architecture for agentic applications using the Elastic Agent Builder and Model Context Protocol (MCP).

• Extend the ES|QL tool in Agent Builder to support parameterized queries for filtering by angle or date range.