56 scanned pages to a cited answer in 5 minutes: RAG pipeline for the documents search cannot read

Get hands-on with Elasticsearch: Dive into our sample notebooks in the Elasticsearch Labs repo, start a free cloud trial, or try Elastic on your local machine now.

Upload a scanned 56-page plant guide, and ask which trees tolerate shallow limestone soil. In less than five minutes, you’ll have an answer, along with the page that it came from and the applicable plants that you can buy. The retrieval augmented generation (RAG) pipeline behind it has two managed pieces. jina-ocr-v1 reads each scanned page and returns structured Markdown with its tables intact, and Elasticsearch’s semantic_text field chunks and embeds that Markdown at ingest. As a result, you don’t need chunking code or an embedding job. Hybrid retrieval fuses keyword and semantic rankings before Claude Haiku 4.5 writes the grounded answer, and all of it runs on Elastic Inference Service (EIS) inside one project.

This matters because most of the knowledge that organizations rely on never makes it into a database. It lives in PDFs or scanned forms, or in photographed documents that traditional search cannot read. Classic optical character recognition (OCR) pipelines got at it by chaining separate models for layout detection, text recognition, table parsing, and reading order, each with its own cost and failure modes. The new generation of vision-language document parsers does the same job in a single model pass. When paired with a vector-capable search engine, a PDF, or a photo of one, it becomes a queryable, citable knowledge source in minutes.

Combining document parsing with hybrid search and RAG lets teams:

• Make tables, handwriting, and complex layouts searchable, where keyword search and legacy OCR miss them.

• Ask natural language questions against documents and get grounded answers with page-level citations.

• Combine exact term matching (such as SKUs or botanical names) with semantic matching (meaning and intent) in a single query.

• Keep the entire pipeline, including parsing and embeddings, along with generation, inside one platform with one bill and a single security boundary.

In this post, we walk through adding a document-parsing feature to an ecommerce site for Texas native plants. (I love plants!) You upload the guide, watch jina-ocr-v1 parse every page, index the output into an Elasticsearch index named after the document, and query it through a RAG chatbot that cites sources by page number and recommends products from the catalog. The stack is as follows:

• Jina OCR (jina-ocr-v1): A document-parsing model that reads a page image and returns clean, structured Markdown in a single multimodal pass.

• EIS: GPU-accelerated inference included with Elastic Cloud Serverless. EIS hosts jina-ocr-v1 behind a preconfigured chat_completion endpoint, alongside embedding models and conversational large language models (LLMs) (in this build, Claude Haiku 4.5), so there’s no model deployment or GPU provisioning, and there’s no separate vendor account. EIS usage is metered per token and billed as a Serverless add-on. Current rates by model are on the Elastic Cloud pricing table.

• Elasticsearch: Stores the parsed documents in the semantic_text field type, which chunks and embeds text at ingest through an inference endpoint and serves hybrid retrieval that fuses lexical and semantic rankings with reciprocal rank fusion (RRF).

• React, TypeScript, and Vite with an Express back end: The storefront, extended with a document upload page that streams parsing progress over server-sent events, and a floating chat widget. The Express layer keeps all credentials server side.

How is jina-ocr-v1 different from classic OCR pipelines?

jina-ocr-v1 is a single model; that is, layout, text recognition, table reconstruction, and reading order happen in one pass. Traditional OCR pipelines did the same work by chaining separate models for layout detection, text recognition, table parsing, and reading order reconstruction. Its output is built for retrieval:

• Structured Markdown with reading order preserved and section headings kept as Markdown headings.

• Tables emitted as HTML with their values intact, so a 50-state license schedule or a plant care grid remains a table.

• Handwriting transcribed inline.

• Repeated page furniture (including running headers and footers, along with printed page numbers) dropped, so it never pollutes retrieval; page numbers come from the ingestion loop instead.

That clean structure is what makes chunking, embedding, and citation trustworthy downstream. And it reads one page image per pass (legibility holds up to about 2048 x 2048 pixels), so someone has to run the page loop.

Where the page loop runs: Jina Reader API, chat completions, or EIS

Who runs that loop is the architecture decision.

This post uses the EIS route; that is, one platform boundary, and the page loop is a few dozen lines in server/documents.js.

Why a scanned plant guide is a hard test for document parsing

A demo corpus is only convincing if it’s challenging. Our test document is Native and Adapted Landscape Plants: An Earthwise Guide for Central Texas, an educational publication codeveloped by the City of Austin and the Texas AgriLife Extension Service to help the public with water conservation and eco-friendly gardening. It’s the plant list behind the city's Grow Green program. Its 56 scanned pages are exactly the type of content that basic PDF text extraction gets wrong. Nearly every page is a dense multicolumn table packed with abbreviations (such as, ″VL″ for very low water, and ″B/E″ for Blackland Prairie and Edwards Plateau), check marks for wildlife value and deer resistance, botanical names in italics, and photo captions interleaved with data. A parser that reconstructs those tables with their values intact can handle your invoices.

The plant theme also gives document intelligence a home. The host application is a Texas native plant storefront with a seeded Elasticsearch catalog: 48 plants with semantic_text descriptions and a full-text versus hybrid search toggle. Add document intelligence, and a shopper can upload a regional planting guide and then ask Which trees tolerate shallow limestone soil? They’ll get an answer grounded in the guide, with pages cited and a list of products that fit.

Prerequisites: Elastic Cloud, an API key, and a scanned document

• An Elastic Cloud account (a free trial works). Create an Elastic Cloud Serverless project. The vector database project type fits this embedding-heavy workload.

• In your project, confirm that the EIS model list includes Jina OCR v1 (.jina-ocr-v1-chat_completion) and Claude Haiku 4.5 (.anthropic-claude-4.5-haiku-chat_completion). Both are preconfigured, so there’s nothing to deploy.

• An Elasticsearch API key with index read/write and inference privileges, and your project endpoint URL.

• Node.js 20 or later, npm, and git.

• A test document, such as the Native and Adapted Landscape Plants guide (linked above), or any scanned, table-heavy document of your own.

Set up the RAG pipeline for scanned PDFs

Step 1: Clone the repository, and install dependencies.

git clone https://github.com/elastic/elastic-labs.git
cd elastic-labs/supporting-blog-content/jina-ocr-v1-hybrid-rag-elasticsearch
npm install

Step 2: Configure credentials. Copy the template, and fill in your project values. The .env file is gitignored, so secrets never reach the repository. The OCR endpoint is preconfigured and the embedding model is the project default, so the chat model is the only model choice in the build, and it’s already made for you.

cp .env.example .env

ELASTICSEARCH_URL=https://<your-project>.es.<region>.elastic.cloud:443
ELASTICSEARCH_API_KEY=<your-api-key>
LLM_INFERENCE_ID=.anthropic-claude-4.5-haiku-chat_completion

Step 3: Seed the product catalog. The seed script creates the catalog index (long_description is a text field mirrored into semantic_text, so hybrid product search works immediately) and bulk loads 48 plants. The first run takes about a minute because Elasticsearch embeds every description at ingest.

npm run seed

Step 4: Start the back end and the front end in separate terminals, with the back end first.

In the first terminal, start the Express API. It listens on port 3001 (set by PORT in .env) and runs the OCR pipeline and chat.

npm run server

In the second terminal, start the Vite dev server for the React app. It proxies /api calls to the back end.

npm run dev

Vite prints the URL that it’s serving, normally http://localhost:5173. (It picks the next free port, if 5173 is taken.) Open that URL; don’t browse to port 3001 directly, because it only exposes the API.

How the OCR pipeline turns scanned pages into Markdown

The Documents page is where the parsing story unfolds on screen, rather than in a terminal. A user drops in a PDF or photo, and the Express server runs a four-stage pipeline while streaming every step back to the browser over server-sent events.

Stage 1: Render PDF pages to images

PDFs are rendered to PNG page images in memory at roughly 1224 x 1584 pixels, under the model's 2048-pixel legibility ceiling. A photographed document skips straight through as a single page.

Stage 2: Transcribe each page, and rebuild its tables

Each page image is sent to the EIS endpoint as a base64 data URL, exactly like a vision call to any chat model. Four pages run in parallel, and order is preserved on reassembly:

const resp = await fetch(
  `${ES_URL}/_inference/chat_completion/.jina-ocr-v1-chat_completion/_stream`,
  {
    method: 'POST',
    headers: { Authorization: `ApiKey ${ES_API_KEY}` },
    body: JSON.stringify({
      messages: [{
        role: 'user',
        content: [
          { type: 'image_url',
            image_url: { url: `data:image/png;base64,${png}` } },
          { type: 'text',
            text: 'Transcribe this page into clean Markdown, preserving reading order. Represent tables as HTML tables.' }
        ]
      }]
    })
  }
);

As each page completes, the browser logs it; for example, "page 12: 2,572 characters of Markdown". The moment that any page yields its first HTML table, the server emits it and the UI renders it fully styled. Watching a dense, scanned plant table reappear as a structured HTML table with every water code and height range intact is the most convincing moment of the demo.

Stage 3: Index and embed the document with semantic_text

Rather than piling everything into a generic documents index, each upload gets its own index named after the file: Native-and-Adaptive-Plants.pdf becomes native-and-adaptive-plants. (The index follows the file name, so yours will match whatever you upload.) The content field is indexed as lexical text and mirrored by copy_to into a semantic_text field, where Elasticsearch chunks the Markdown and generates embeddings at ingest through the project's default inference endpoint. As a result, you don’t need to write chunking code or build an embedding pipeline, and there’s no need for vector bookkeeping.

PUT native-and-adaptive-plants
{
  "mappings": {
    "properties": {
      "content":          { "type": "text", "copy_to": "content_semantic" },
      "content_semantic": { "type": "semantic_text" },
      "title":            { "type": "keyword" },
      "page_count":       { "type": "integer" }
    }
  }
}

Stage 4: Check the new index in Elastic

The document is indexed, and the UI tells you that parsing is complete and to check your Elastic indices. Open your project, and the index is sitting there, named after the document, chunked and embedded. A small registry index (doc-registry) tracks every parsed document so the chat can list them.

Asking questions: Hybrid retrieval plus grounded generation

The leaf-shaped chat button in the corner of every page opens the Plant advisor. By default, it answers from “everything” (the uploaded guide and the product catalog together). A user can narrow the scope to a single document or to the catalog alone, but the default is where the retail story lives. The server answers in three moves:

1. Hybrid retrieval: How lexical and semantic search combine 

Two hybrid searches run in parallel, one across the document indices and one across the catalog. Each uses an RRF retriever that fuses two rankings over the same text: a lexical match (which catches exact terms like "Quercus macrocarpa" or "VL" or a price), and a semantic query (which catches "trees for rocky alkaline soil", even when no sentence contains those words). On this corpus, hybrid consistently surfaced plants that keyword matching missed; the Compare view in the storefront shows exactly which ones for any query.

"retriever": {
  "rrf": {
    "retrievers": [
      { "standard": { "query": { "match":    { "content": question } } } },
      { "standard": { "query": { "semantic": { "field": "content_semantic",
                                               "query": question } } } }
    ]
  }
}

2. Source extraction and page citations

Highlighters do the work. The semantic highlighter returns the best matching embedded chunks, and the plain highlighter returns lexical fragments. The page numbers are recovered from the page markers that the ingestion loop embedded. Each source is labeled with the guide's name and page.

3. Generation: Grounded answers with Claude Haiku 4.5

The sources, the matching catalog products, and the question go to Claude Haiku 4.5 on EIS with strict grounding instructions: answer only from the sources, cite them by name and page, recommend only products that appear in the catalog, and say so if the answer is not present. The model also returns a hidden line listing the SKUs that it recommended, which the server strips from the stream; only those products are shown, so a question that the store cannot satisfy produces an honest answer with no product cards. The answer streams into the chat with an expandable sources panel and a collapsed "See recommendations" section underneath.

Queries to try against your indexed document

• Which trees tolerate shallow limestone soil? (Semantic retrieval does the lifting.)

• What does the guide say about Quercus macrocarpa? (Lexical retrieval identifies the exact botanical name.)

• Which invasive plants should I avoid, and what are the recommended alternatives? (Answers live in a table that jina-ocr-v1 reconstructed.)

• What do the water codes VL and M mean? (This tests the legend table from the guide's key page.)

From scanned PDF to cited answer

A year ago, this pipeline would have required a layout model, an OCR engine, a table parser, a chunking service, an embedding job, and a vector database, each separately deployed. Now two managed pieces carry the whole path from scanned page to cited answer: jina-ocr-v1 turns pixels into structured Markdown in one pass, and Elasticsearch's semantic_text field turns that Markdown into hybrid searchable knowledge at ingest.

The plant guide is a stand-in for whatever your hardest documents are. Swap in claim forms, lab reports, equipment manuals, or supplier catalogs, and then upload, parse, index per document, and ask. Start a free Elastic Cloud trial, and point the pipeline at the PDF that your team complains about most.