Blog

Extract chart data standard OCR misses: Elastic Agent Builder and LlamaParse in one pipeline

Build an end-to-end pipeline that extracts structured data (including values from charts) out of complex PDFs and into Elasticsearch, ready for agent queries with ES|QL.

Jeffrey Rengifo
Agentic AIIntegrations

Enterprise documents are hard. PDFs with multi-column tables, scanned pages, mixed layouts, and embedded charts break naive text extraction. Standard processing pipelines miss structured data that lives inside those documents, which means your agents are reasoning over incomplete context. Most agent frameworks use basic OCR tools to handle PDFs, which get the job done for text-only documents, but they fall short for more complex scenarios.

To solve this, you need a system that accurately extracts structured data from those documents and automatically indexes it so an agent can use it. The combination of Elastic Agent Builder and the LlamaParse Extract API covers data processing and reasoning. LlamaParse Extract handles the document complexity: It applies a schema-defined extraction model against the raw PDF and returns structured JSON. Elastic Agent Builder handles orchestration: Using Elastic Workflows, it calls the LlamaParse Extract API, indexes the result into Elasticsearch, and gives an agent the ability to reason over that data using Elasticsearch Query Language (ES|QL) or semantic queries.

This article walks through the full implementation, from schema definition to a working agent. The complete code is available in the companion notebook.

When should you use LlamaParse Extract instead of LlamaParse?

Use LlamaParse Extract when you need specific typed fields returned as JSON. Use LlamaParse when you need full document content for a RAG pipeline.

LlamaCloud offers two document processing tools. LlamaParse converts documents into large language model–ready (LLM-ready) formats, like Markdown, preserving the full document layout. It's designed for retrieval augmented generation (RAG) pipelines where you want to feed entire document content into a vector store or LLM context window. LlamaParse Extract, on the other hand, takes a developer-defined schema and returns only the specific fields you asked for as validated JSON. It can also extract data from charts, figures, and visual elements inside the document, something standard text parsing misses entirely.

LlamaParse

LlamaParse Extract

Output format

Markdown (full document)

Structured JSON (schema-defined fields)

Best for

RAG pipelines, LLM context windows

Search indexes, databases, typed fields

Handles charts/figures

No

Yes

Input

PDF

PDF + developer-defined schema

or this use case, Extract is the better fit. We need specific numeric values (GDP share percentages, investment growth rates) and narrative summaries indexed as typed Elasticsearch fields. The schema tells the extraction model exactly what to look for, including values embedded in bar charts that only exist as visual elements in the PDF. If your use case is feeding full document content into a RAG pipeline, LlamaParse is the right tool. If you need structured records for a search index or database, especially when data lives in charts or tables, use Extract.

What does the LlamaParse Extract and Elastic Agent Builder pipeline do?

The pipeline works as follows: A user sends a question, along with a PDF URL. The agent invokes a workflow tool that uploads the PDF to LlamaParse, runs schema-driven extraction via the LlamaParse Extract API, and indexes the structured result into Elasticsearch.

The agent then queries that index with ES|QL to answer the question.

To demonstrate this, we use the World Bank Global Economic Prospects (January 2026) report as the source document. The agent will be able to answer questions about frontier markets, economic indicators, and policy recommendations directly from the PDF.

How to build an Elastic Workflows tool for LlamaParse Extract

The workflow tool runs inside Elastic Agent Builder as a YAML-defined automation. When triggered, it executes six steps. For a general walkthrough of connecting Agent Builder with Workflows, that post covers the setup in detail. Here we focus on the LlamaParse Extract–specific integration.

  1. Upload PDF: Uploads the PDF from the provided URL to LlamaCloud. The workflow accepts a pdf_url as input; your LlamaCloud API key, project ID, and configuration ID are configured as constants.

  2. Create extraction job: Starts the Extract v2 job against the uploaded file using the saved configuration.

  3. Wait: Pauses 15 seconds to give time to LlamaParse Extract to process the document.

  4. Poll until done: Polls the extraction status every 10 seconds in a while loop until the status is COMPLETED, up to 19 iterations (~3 minutes cap).

  5. Index: Writes all extracted fields from the PDF as a single Elasticsearch document.

  6. Verify: Runs a search to confirm the document was indexed successfully using the document_id along a query term.

Prerequisites

Implementing LlamaParse Extract with Elastic Agent Builder: step by step

The code below shows the key steps of the implementation. For the complete version with all details, refer to the companion notebook.

Configure the environment 

import os
import json
from dotenv import load_dotenv

load_dotenv()

ELASTICSEARCH_URL = os.getenv("ELASTICSEARCH_URL")
ELASTICSEARCH_API_KEY = os.getenv("ELASTICSEARCH_API_KEY")
LLAMA_CLOUD_API_KEY = os.getenv("LLAMA_CLOUD_API_KEY")
KIBANA_URL = os.getenv("KIBANA_URL")

Define the extraction schema

LlamaParse Extract is schema-driven. You define a Pydantic model that describes the fields you want to extract, and LlamaParse Extract uses it to guide extraction from the raw PDF. This is what makes it reliable for complex documents. Instead of hoping the LLM finds the right values, you tell it exactly what to look for.

As mentioned above, the agent needs to handle two question types:

  • Structured: "What is the frontier market GDP share in 2025?" This one requires numeric fields the agent can filter with ES|QL.

  • Exploratory: "What are the main risks for frontier markets?" Requires narrative text fields the agent can search semantically.

The schema captures only what's needed to support both. The description values aren’t documentation, they’re instructions to the extraction model, so being specific here directly improves extraction quality. Two of the fields (frontier_market_gdp_share_pct and frontier_market_investment_growth_2020s_pct) are extracted directly from bar charts in the report (Figures A and C), which standard text extraction tools would miss.

class EconomicReportSummary(BaseModel):
    report_title: str = Field(description="Full title of the report")
    publication_date: str = Field(
        description="Publication date in YYYY-MM format, e.g. '2026-01'"
    )
    frontier_market_gdp_share_pct: float = Field(
        description="Frontier markets' share of global GDP in 2025 as a percentage, "
                    "extracted from Figure ES.A bar chart"
    )
    frontier_market_investment_growth_2020s_pct: float = Field(
        description="Average annual per capita investment growth for frontier markets "
                    "in the early 2020s (2020-24), extracted from Figure ES.C bar chart, as a percentage"
    )
    executive_summary: str = Field(
        description="Concise summary of the report's main findings and conclusions"
    )
    key_vulnerabilities: str = Field(
        description="Main vulnerabilities and risks facing frontier markets, as a paragraph"
    )
    policy_recommendations: str = Field(
        description="Key policy recommendations for frontier market policymakers, as a paragraph"
    )

Create the Extract configuration

LlamaExtract v2 replaced extraction agents with saved configurations, a reusable parameter set that pairs your schema with the extraction tier. Fetch your project ID first, and then POST to create the configuration. Save both printed IDs, since you'll need them in the workflow YAML.

import requests

LLAMA_CLOUD_BASE_URL = "https://api.cloud.llamaindex.ai"
llama_headers = {
    "Authorization": f"Bearer {LLAMA_CLOUD_API_KEY}",
    "Content-Type": "application/json",
}

# Fetch the project ID (uses the first available project)
projects_response = requests.get(
    f"{LLAMA_CLOUD_BASE_URL}/api/v1/projects",
    headers=llama_headers,
)

projects_response.raise_for_status()
PROJECT_ID = projects_response.json()[0]["id"]
print(f"Project ID: {PROJECT_ID}")

# Create a saved Extract v2 configuration with our schema
config_response = requests.post(
    f"{LLAMA_CLOUD_BASE_URL}/api/v1/beta/configurations",
    headers=llama_headers,
    params={"project_id": PROJECT_ID},
    json={
        "name": "global-economic-extractor",
        "parameters": {
            "product_type": "extract_v2",
            "data_schema": EconomicReportSummary.model_json_schema(),
            "extraction_target": "per_doc",
            "tier": "agentic",
        },
    },
)

config_response.raise_for_status()
CONFIGURATION_ID = config_response.json()["id"]
print(f"Configuration ID: {CONFIGURATION_ID}")

Create the Elasticsearch index

Create the index with mappings that mirror the extraction schema. Numeric fields, like frontier_market_gdp_share_pct, use float for structured ES|QL filtering. The publication_date field uses date to enable date range queries. Narrative fields, like executive_summary and key_vulnerabilities, use text for full-text search. Identifiers like report_title use keyword. The full mapping definition is available in the notebook.

Build the Agent Builder workflow tool

Copy the YAML below and paste it in the Elastic UI at Elasticsearch > Workflows > Create a new Workflow. The Elastic Workflows documentation covers the full YAML schema and available step types.

The workflow uses the LlamaCloud REST API: the Files API (/api/v1/files/upload_from_url) to upload the PDF from a public URL; and the Extract v2 API (/api/v2/extract) to create the job and poll for the result (/api/v2/extract/{id}).

name: LlamaParse Extract Economic Report Processor
description: >
  Uploads a PDF from a URL to LlamaCloud, runs Extract v2,
  and indexes the structured results into Elasticsearch.
enabled: true

inputs:
  - name: pdf_url
    type: string
    description: Public URL of the PDF to process
    required: true

consts:
  indexName: economic-reports
  llamaBaseUrl: https://api.cloud.llamaindex.ai
  projectId: <YOUR_PROJECT_ID>
  configurationId: <YOUR_CONFIGURATION_ID>
  documentId: global-economic-prospects-jan-2026
  llamaCloudApiKey: llx-YOUR-API-KEY-HERE

triggers:
  - type: manual

steps:
  # Upload PDF from URL to LlamaCloud
  - name: upload_pdf
    type: http
    with:
      url: "{{ consts.llamaBaseUrl }}/api/v1/files/upload_from_url"
      method: PUT
      headers:
        Authorization: "Bearer {{ consts.llamaCloudApiKey }}"
        Content-Type: application/json
      body: |
        {
          "url": "{{ inputs.pdf_url }}"
        }

  # Create the Extract v2 job using our saved configuration
  - name: create_extraction_job
    type: http
    with:
      url: "{{ consts.llamaBaseUrl }}/api/v2/extract?project_id={{ consts.projectId }}"
      method: POST
      headers:
        Authorization: "Bearer {{ consts.llamaCloudApiKey }}"
        Content-Type: application/json
      body: |
        {
          "file_input": "{{ steps.upload_pdf.output.data.id }}",
          "configuration_id": "{{ consts.configurationId }}"
        }

  - name: wait_for_extraction
    type: wait
    with:
      duration: "15s"

  # Poll every 10s until status is COMPLETED (max ~3 min)
  - name: poll_until_done
    type: while
    condition: 'not steps.poll_get.output.data.status : "COMPLETED"'
    max-iterations:
      limit: 19
      on-limit: fail
    steps:
      - name: poll_wait
        type: wait
        with:
          duration: "10s"
      - name: poll_get
        type: http
        with:
          url: "{{ consts.llamaBaseUrl }}/api/v2/extract/{{ steps.create_extraction_job.output.data.id }}?project_id={{ consts.projectId }}"
          method: GET
          headers:
            Authorization: "Bearer {{ consts.llamaCloudApiKey }}"
            Accept: application/json

  - name: index_extracted_data
    type: elasticsearch.index
    with:
      index: "{{ consts.indexName }}"
      id: "{{ consts.documentId }}"
      document:
        report_title: "{{ steps.poll_get.output.data.extract_result.report_title }}"
        publication_date: "{{ steps.poll_get.output.data.extract_result.publication_date }}"
        frontier_market_gdp_share_pct: "{{ steps.poll_get.output.data.extract_result.frontier_market_gdp_share_pct }}"
        frontier_market_investment_growth_2020s_pct: "{{ steps.poll_get.output.data.extract_result.frontier_market_investment_growth_2020s_pct }}"
        executive_summary: "{{ steps.poll_get.output.data.extract_result.executive_summary }}"
        key_vulnerabilities: "{{ steps.poll_get.output.data.extract_result.key_vulnerabilities }}"
        policy_recommendations: "{{ steps.poll_get.output.data.extract_result.policy_recommendations }}"
      refresh: wait_for

  - name: verify_document
    type: elasticsearch.search
    with:
      index: "{{ consts.indexName }}"
      query:
        term:
          _id: "{{ consts.documentId }}"

Update the consts section with your own llamaCloudApiKey, projectId, and configurationId.

Connecting with Agent Builder

After saving the workflow, create two tools and one agent using the Agent Builder Kibana API or the UI.

import requests

headers = {
    "Authorization": f"ApiKey {ELASTICSEARCH_API_KEY}",
    "kbn-xsrf": "true",
    "Content-Type": "application/json",
}

WORKFLOW_ID = "workflow-abcabc-0073-4a08-98a8-werwer"  # Copy from UI after creating the workflow

# Create the workflow tool
workflow_tool_payload = {
    "id": "run_llamaextract_workflow",
    "type": "workflow",
    "description": (
        "Triggers the LlamaParse Extract extraction workflow. "
        "Use this tool to extract structured data from a PDF URL and index it into Elasticsearch. "
        "Requires only the public URL of the PDF to process."
    ),
    "tags": ["llama-extract", "workflow"],
    "configuration": {
        "workflow_id": WORKFLOW_ID,
    },
}

response = requests.post(
    f"{KIBANA_URL}/api/agent_builder/tools",
    headers=headers,
    json=workflow_tool_payload,
)

print(f"Workflow tool: {response.status_code}")

# Create the structured indicators query tool
structured_query_payload = {
    "id": "query_structured_indicators",
    "type": "esql",
    "description": (
        "Query structured economic indicators using exact filters on numeric fields. "
        "Use this for questions like 'Which reports show frontier market GDP share below 5%?' "
        "or 'Show investment growth for reports published after 2025-01'."
    ),
    "tags": ["economic-data", "llama-extract"],
    "configuration": {
        "query": (
            "FROM economic-reports "
            "| WHERE frontier_market_gdp_share_pct <= ?max_gdp_share "
            "| KEEP report_title, publication_date, "
            "frontier_market_gdp_share_pct, frontier_market_investment_growth_2020s_pct "
            "| SORT publication_date DESC "
            "| LIMIT 10"
        ),
        "params": {
            "max_gdp_share": {
                "type": "double",
                "description": "Maximum frontier market GDP share percentage to filter by",
            }
        },
    },
}

response = requests.post(
    f"{KIBANA_URL}/api/agent_builder/tools",
    headers=headers,
    json=structured_query_payload,
)

print(f"Structured query tool: {response.status_code}")

# Create the narrative text search tool
text_search_payload = {
    "id": "search_economic_narratives",
    "type": "esql",
    "description": (
        "Search narrative content in economic reports. "
        "Use this for open-ended questions like 'What are the main risks for frontier markets?' "
        "or 'What does the report recommend for policymakers?'."
    ),
    "tags": ["economic-data", "llama-extract"],
    "configuration": {
        "query": (
            "FROM economic-reports "
            "| WHERE MATCH(executive_summary, ?query) "
            "OR MATCH(key_vulnerabilities, ?query) "
            "OR MATCH(policy_recommendations, ?query) "
            "| KEEP report_title, executive_summary, key_vulnerabilities, policy_recommendations "
            "| LIMIT 5"
        ),
        "params": {
            "query": {
                "type": "keyword",
                "description": "The search query to find relevant narrative content",
            }
        },
    },
}

response = requests.post(
    f"{KIBANA_URL}/api/agent_builder/tools",
    headers=headers,
    json=text_search_payload,
)

print(f"Text search tool: {response.status_code}")

# Create the agent
agent_payload = {
    "id": "economic-report-analyst",
    "name": "Economic Report Analyst",
    "description": "Extracts and analyzes economic reports from PDFs using LlamaParse Extract and Elasticsearch.",
    "labels": ["economics", "llama-extract"],
    "configuration": {
        "instructions": (
            "You are an economic research assistant. You have three tools:\n"
            "1. run_llamaextract_workflow: Use this FIRST to extract and index data from a PDF.\n"
            "2. query_structured_indicators: Use this for structured questions that filter on "
            "numeric fields like GDP share or investment growth.\n"
            "3. search_economic_narratives: Use this for open-ended questions about risks, "
            "vulnerabilities, or policy recommendations.\n"
            "When presenting data, use clear formatting with bullet points or tables. "
            "Always cite the report title and publication date."
        ),
        "tools": [
            {
                "tool_ids": [
                    "run_llamaextract_workflow",
                    "query_structured_indicators",
                    "search_economic_narratives",
                ]
            }
        ],
    },
}

response = requests.post(
    f"{KIBANA_URL}/api/agent_builder/agents",
    headers=headers,
    json=agent_payload,
)

print(f"Agent: {response.status_code}")

The workflow ID is available on the URL: 

https://4622216ea8cd443ead5bef0a3de05135.us-central1.gcp.cloud.es.io/app/workflows/<WORKFLOW-ID>

Testing the workflow through Agent Builder

With the Economic Report Analyst agent configured, open the Agent Builder chat and send a message like this (substituting your actual IDs):

Process this PDF and extract its data: <your-pdf-url> 

Once extracted, answer: What are the main vulnerabilities facing frontier markets and what policy recommendations does the report suggest to address them?

The agent calls run_llamaextract_workflow first, waits for the workflow to complete, and then uses search_economic_reports to retrieve and summarize the extracted data.

Result

Let’s see what happens if we ask a question related to the information contained in the graphics in the PDF:

Conclusion: from raw PDF to agent-ready data

LlamaParse Extract solves the document understanding problem, extracting structured, schema-driven data from PDFs with charts and tables. Elastic Agent Builder solves the orchestration problem, chaining extraction, indexing, and querying into workflow and ES|QL tools an agent can invoke on demand. Together, they bridge the gap between raw enterprise documents and agent-ready data.

This pattern extends beyond economic reports. Any enterprise document with a predictable structure (contracts, specs, financial filings) can be modeled with a Pydantic schema and fed through the same pipeline.

Next steps

Related Content

Agentic AIElastic Cloud Serverless

Your AI agent doesn't need your API key: OAuth 2.1 for Elasticsearch MCP server authentication

Alex Chalkias
Agentic AI

Faster, cheaper support investigations with precomputed context

Abhimanyu Anand
AIAgentic AIES|QL

Building context in Elasticsearch: how AI Indices power smarter agents using fewer tokens

Kathleen DeRusso
Agentic AIOperations

Your agents have been keeping receipts: turning Elastic Agent Builder's built-in OTel traces into token cost dashboards in Kibana

Meghan Murphy

Ready to build state of the art search experiences?

Sufficiently advanced search isn’t achieved with the efforts of one. Elasticsearch is powered by data scientists, ML ops, engineers, and many more who are just as passionate about search as you are. Let’s connect and work together to build the magical search experience that will get you the results you want.

Try it yourself