PromQL compiles to the same compute engine that runs ES|QL and inherits its planner and distributed execution, along with its release process. We didn’t build a second engine for this, and there’s no plugin to install.

This post is about how we built it.

Key takeaways:

• One engine: The implementation combines Elasticsearch’s mature distributed planning, storage, and testing infrastructure with its newer compute engine, which provides a columnar execution runtime. This lets PromQL reuse proven Elasticsearch capabilities while executing through a modern, native vectorized pipeline rather than introducing a separate runtime.

• One server: Elasticsearch implements the Prometheus remote write and query APIs directly, so Prometheus-compatible ingest and queries run without any additional plugins.

• Engineered for efficiency: Supporting PromQL required new engine primitives for range-aligned evaluation grids, backward-looking windows, dynamic label grouping, pipeline result reshaping, and compact wide aggregation keys. These primitives allow PromQL queries to execute efficiently end to end, with the relevant semantics implemented directly in the compute engine rather than through external post-processing.

• Compatibility measured in real use: In addition to Prometheus compliance tests, we built a differential-testing and quality-control pipeline over 2,000 PromQL queries collected from public repositories. 

Read more:

• Query Prometheus Metrics in Elasticsearch with PromQL

• Ship Prometheus Metrics to Elasticsearch with Remote Write

• Bringing Fire to Elasticsearch: Adding Native Prometheus APIs

Why run PromQL on Elasticsearch

Many teams already store logs and traces in Elasticsearch while running metrics in Prometheus or another dedicated metrics back end.

Prometheus and its ecosystem are strong and widely adopted, but large deployments can also bring operational sprawl and scaling challenges, along with limited retention. 

So we set out to combine Elastic’s highly optimized time-series database (TSDB) with a best-in-class metrics ecosystem. The result is a smaller observability stack, with fewer systems to operate and metrics storage that scales horizontally and supports long-term retention.

One engine: PromQL and ES|QL share the same compute engine

We made an early architectural decision not to run a separate PromQL engine next to Elasticsearch.

PromQL is instead another front end to the Elasticsearch compute engine.

This puts PromQL in the normal Elasticsearch development lifecycle. It uses the same planner, distributed execution engine, testing infrastructure, and release process as Elasticsearch itself.

To learn more about Elasticsearch’s query engine, check our blog.

Like ES|QL’s time series queries that use the TS source command, PromQL is translated into a highly optimized query plan and executed across the cluster. Nodes process columnar batches through vectorized operators, while partial results move through exchanges until the final result is assembled. 

This also means that PromQL and ES|QL operate over the same execution engine and time-series data. ES|QL can additionally extend a PromQL computation with post-processing that PromQL doesn’t support, such as lookup joins and inline aggregations.

For example, assume Prometheus request counters are stored in metrics-* and keyed by the service label. A lookup index, service_registry, maps each instance to its owning team and environment and to its service tier:

This architecture requires the execution engine to support PromQL semantics natively and efficiently rather than ES|QL syntax sugar. The following sections describe the changes and new execution primitives we introduced to achieve that.

One server: Prometheus remote write and HTTP API built into Elasticsearch

Query execution is only half of the story. The Prometheus ecosystem also expects familiar ingest and query APIs.

Prometheus protocols are the de facto standard for everything metrics in almost every team’s observability stack. So we built the HTTP API directly in the Elasticsearch server, which eliminated the need for a third component and tightened integration stability and performance.

On the ingest side, we added an endpoint for the Prometheus remote write protocol. It accepts Snappy-compressed Protocol Buffer messages, maps labels to TSDS dimensions, maps the metric name/value into metric fields, infers counter versus gauge mappings, and writes directly into TSDS. The built-in template is dynamic, so users don’t have to predeclare every Prometheus label or metric.

On the query side, Elasticsearch exposes Prometheus query APIs. A request enters through the Prometheus endpoint and is executed in the compute engine.

Running PromQL efficiently in a columnar engine

Sharing an execution engine doesn’t mean treating PromQL as syntax sugar over ES|QL. PromQL has different time, grouping, and response semantics, as well as workload characteristics that matter at scale. Supporting it efficiently required extending the compute engine rather than compensating in the API layer.

PromQL time grids: Aligning evaluation steps with TSTEP

Time-series query engines are optimized for grouping over time.

Elasticsearch normally groups timestamps with TBUCKET(...), which truncates each timestamp to a fixed interval boundary. Truncation is cheap and produces deterministic bucket boundaries. It also makes intermediate results easier to reuse.

Prometheus defines evaluation points differently. For a range query, timestamps are laid out as fixed steps anchored to the query range, rather than derived by truncating each sample timestamp. Two queries with the same step but different range boundaries can therefore produce different evaluation grids.

To preserve these semantics, we introduced TSTEP(...), which derives its grouping grid from the query range and step,  rather than truncating timestamps to globally aligned boundaries.

PromQL uses TSTEP(...) internally, preserving Prometheus timestamp semantics while still lowering the operation to a native Elasticsearch execution primitive.

Some might see this as a simple problem, but the nuances matter. 

Take, for example, a query that finds a 5m rolling average of a metric: 

At evaluation time T, the result represents the average over the preceding five-minute range: 

(T - 5m, T]

When the query is executed with a five-minute step, each output value is labeled with the upper end of its corresponding five-minute window.

Elasticsearch previously lacked this semantic and supported only forward-looking window aggregation functions:

We rewrote the window-evaluation path so that both ES|QL and PromQL use a common backward-looking windowing implementation:

Together, TSTEP(...) and backward-looking windows preserve the two time semantics that matter for PromQL range evaluation.

Dynamic label grouping: How PromQL without() resolves at runtime

Time grids determine when a PromQL expression is evaluated. Aggregation determines which input series are combined and which labels identify each output series.

For most analytical query engines, that identity is known when the query is planned. The planner can allocate grouping columns and choose an aggregation strategy. It also carries a fixed key through the execution pipeline.

That’s how ES|QL works:

The output series are grouped by an explicit key (cluster, namespace).

PromQL can express the same operation in the opposite direction:

Now we know which dimensions not to use. We don’t necessarily know the full grouping key until the query is executed. This is a small language difference with significant execution consequences. 

One possible implementation is to discover every label used by the metric, subtract instance and pod, and rewrite the expression into an ordinary by(...) aggregation. That adds a discovery phase before planning. It also becomes inefficient for high-dimensional metrics where only a subset of all possible dimensions may have useful value in a particular series. Most queries need only a small subset of the available dimensions, so carrying the entire dimension universe as an aggregation key wastes memory and adds bookkeeping overhead.

We instead extended the time-series execution path with dynamic grouping columns. The engine loads dimensions as the series are read and applies the exclusions per time series. This avoids making the grouping schema a prerequisite for planning and avoids carrying a large sparse set of grouping columns through aggregation.

Dimension packing: Keeping wide PromQL grouping keys cheap

The idiomatic way of writing PromQL aggregations involves heavy use of without(...) over by(...):

Excluding labels rather than explicitly listing them makes dashboards and alerts resilient to schema evolution. If a new label is added to the metric, the query continues to preserve it unless it’s explicitly excluded.

The consequence for the engine is that effective grouping keys can be wide. Many of those labels often have low cardinality, yet each still participates in every aggregation stage.

In a columnar engine, each grouping column is normally represented as a separate vector. Ten grouping labels therefore mean 10 vectors flowing through every aggregation operator:

Dimension fields are declared in the index mapping; the planner knows the schema up front, and grouping keys stay narrow and predictable.

In the columnar engine, each grouping label is carried as a separate vector or block. A key with 10 labels therefore requires 10 vectors to be read, hashed, compared, and retained by aggregation operators. As key width grows, so does the amount of data and bookkeeping that must move through the pipeline:

To avoid paying that per-column cost for every additional label, we introduced dimension packing. Before aggregation begins, the engine encodes the full grouping key into a single compact representation. Hash and comparison operations run on the packed key rather than on each block independently:

Packing lets hashing and comparison operate on a single compact key rather than an increasing number of grouping blocks, making aggregation overhead less sensitive to key width. Because the engine is shared, ES|QL time-series queries will benefit from this optimization as well.

Building the Prometheus HTTP API response inside the pipeline

Unlike Elasticsearch's ES|QL column-oriented response format, the Prometheus response is row-oriented.  The Prometheus API returns one result row per time series, with its samples represented as timestamp-value pairs. 

To support a compatible API layer, we had to regroup in the HTTP layer converting the columnar results into boxed row objects and accumulate them in map- and list-based structures until the complete Prometheus response could be produced. 

We replaced this with the TimeSeriesCollapse compute operator. It groups rows by series and aligns samples to the query’s fixed step grid. It emits the reshaped result as ordinary columnar pages containing one row per series, with aligned multi-valued timestamp and value blocks. And it preserves compact, vectorized block representation throughout the pipeline: 

The HTTP layer can now serialize those blocks directly, avoiding the maps, lists, boxed objects, and associated allocations required by the earlier implementation.

Testing PromQL compatibility against 2,000 real queries

Prometheus compliance tests were our starting point.

Even though they gave us a strong baseline, they didn’t tell us how frequently individual PromQL features appear in real workloads. To complement that baseline, we built a second test corpus from over 2,000 PromQL queries collected from public repositories. 

We then classified those queries by the language features and expression patterns they exercise:

For each compatible query shape, we run the same query against Elasticsearch and Prometheus and compare the results. 

In addition to that, we actively rely on fuzz testing, which catches issues that unit tests alone are unlikely to expose, including differences in timestamp alignment, label retention, aggregation behavior, range-vector evaluation, and response encoding.

Which PromQL functions and APIs are supported in 9.5

Since 9.4 (technical preview), PromQL support in Elasticsearch has expanded substantially. In Elasticsearch 9.5, both the PROMQL command in ES|QL and the Prometheus HTTP APIs are generally available (GA), with more than 80% of the PromQL workflows in our real-world corpus now running without modification:

The main additions since technical preview are:

Feature

Example

Status

Prometheus remote write ingest

POST /_prometheus/api/v1/write

GA in 9.5

Range queries

/api/v1/query_range

GA in 9.5

Instant queries

/api/v1/query

GA in 9.5

Metric metadata and build info

/api/v1/metadata, /api/v1/status/buildinfo

GA in 9.5

Native histogram functions

histogram_quantile, histogram_count, histogram_sum

GA in 9.5

Per-selector offset modifiers

[5m] offset 1h

GA in 9.5

Top-level or operator

rate(a[5m]) or rate(b[5m])

GA in 9.5, up to eight operands

Prometheus remote write ingest

Elasticsearch accepts Prometheus remote write (v1) messages directly:

Snappy-compressed Protocol Buffer messages are decoded, and labels are mapped to TSDS dimensions. Metric names and values are written into the time-series index. The built-in template is dynamic, so users don’t have to predeclare every Prometheus label or metric.

Range and instant queries through the Prometheus HTTP API

Both range and instant query endpoints are available:

Range queries return matrices evaluated over a time window, and instant queries return vectors evaluated at a single timestamp. These endpoints can be used by Kibana, Grafana, or Prometheus-compatible alerting tools, and custom dashboards.

Metric metadata and build info endpoints

Elasticsearch exposes metadata about available metrics and a build-info endpoint:

The metadata endpoint returns metric types and help text, and the build-info endpoint returns the Prometheus-compatible server version. Grafana and other tools use these endpoints for feature detection and UI behavior.

Native histogram functions: histogram_quantile, count, and sum

Elasticsearch supports the main PromQL operations over native histograms:

Native histograms adapt their bucket layout to the data, providing useful precision across a wide value range without requiring users to configure every bucket boundary in advance. Classic histograms continue to work alongside native histograms.

Per-selector offset modifiers in PromQL

Offset modifiers shift a selector’s time window backward:

This returns the request rate from one hour earlier. Per-selector offsets are commonly used to compare current traffic, latency, or resource usage with an earlier baseline, such as the same period one week ago.

Top-level or operator in PromQL

Elasticsearch supports the top-level PromQL or operator:

In PromQL, or isn’t a Boolean operation. It performs a union between two sets of time series. Results from the left side are retained; a series from the right side is added only when its label set doesn’t match a series already returned by the left side. This is useful during migrations where the same logical metric may exist under an old and a new name.

The implementation follows Prometheus’s left-side precedence rules and preserves the __name__ label. Top-level chains of up to eight operands are supported.

PromQL features not yet supported in Elasticsearch

GA doesn’t mean complete PromQL compatibility. Some less common and more complex parts of PromQL remain unsupported. These gaps now define the next phase of the work: 

Feature

Example

Status

Advanced vector matching

on(instance) group_left

Planned

Sorting and ranking

topk, bottomk, limitk, sort, sort_desc

Planned

Label manipulation

label_replace, label_join

Planned

Absolute time modifier

@ 1710000000

Planned

Mixed-offset compound expressions

rate(...) - rate(... offset 1h)

Planned

Alerting and target endpoints

/api/v1/alerts, /api/v1/targets

Out of scope

Advanced vector matching with on() and group_left

Some binary operations that require Prometheus vector matching aren’t yet part of GA.

For example, this query divides per-instance request rates by a per-instance capacity metric:

The on(instance) clause specifies which labels identify matching series. group_left permits many request-rate series to match a single per-instance capacity series, while retaining the labels from the higher-cardinality left-hand side.

These expressions are common when joining a detailed metric with metadata or a lower-cardinality capacity metric. Basic binary expressions are supported where applicable, while the remaining vector-matching forms are planned work.

Sorting and ranking: topk, bottomk, and sort

Prometheus sorting and ranking functions are also not yet part of GA:

This returns the 10 services with the highest request rate. Similar queries are widely used in “top offenders” dashboards for traffic, latency, errors, and resource consumption.

The remaining functions include:

Label manipulation with label_replace and label_join

PromQL can construct or rewrite labels during query evaluation. These functions are particularly useful when dashboard variables, naming conventions, or label schemas don’t match exactly:

This creates an environment label from the cluster label.

Another common example combines existing labels into a display-oriented label:

This produces a target label, such as payments/api-7f6d9. label_replace(...) and label_join(...) aren’t yet included in GA.

Advanced time modifiers: The @ modifier and mixed offsets

Several advanced time modifiers and expression forms remain outside the GA scope.

For example, an absolute @ modifier evaluates a selector at a fixed Unix timestamp rather than at the query’s normal evaluation time:

This is useful for comparisons against a fixed historical point.

PromQL also permits expressions in which the two sides use different offsets:

This compares current traffic with traffic one hour earlier. Per-selector offset is available in GA, but not every combination of offsets and compound expressions is part of GA yet.

Prometheus API endpoints not yet implemented

In addition, the Prometheus HTTP API surface isn’t yet fully complete. Notably:

Alerting metadata through:

used by tools that inspect active alert state.

Target discovery through:

used to inspect scrape targets, health, and labels.

These endpoints concern Prometheus server and scrape-target state rather than querying metrics stored in Elasticsearch.

For the full list of limitations, see the PromQL limitations page. 

Try PromQL in Elasticsearch 9.5

To query Prometheus metrics in Elasticsearch 9.5 or Serverless, see the PromQL documentation and the Prometheus HTTP API reference.