<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/">
    <channel>
        <title>Elastic Observability Labs - Articles by Costin Leau</title>
        <link>https://www.elastic.co/observability-labs</link>
        <description>Trusted security news &amp; research from the team at Elastic.</description>
        <lastBuildDate>Wed, 15 Apr 2026 16:40:31 GMT</lastBuildDate>
        <docs>https://validator.w3.org/feed/docs/rss2.html</docs>
        <generator>https://github.com/jpmonette/feed</generator>
        <image>
            <title>Elastic Observability Labs - Articles by Costin Leau</title>
            <url>https://www.elastic.co/observability-labs/assets/observability-labs-thumbnail.png</url>
            <link>https://www.elastic.co/observability-labs</link>
        </image>
        <copyright>© 2026. Elasticsearch B.V. All Rights Reserved</copyright>
        <item>
            <title><![CDATA[Query Prometheus Metrics in Elasticsearch with Native PromQL Support]]></title>
            <link>https://www.elastic.co/observability-labs/blog/elasticsearch-supports-promql</link>
            <guid isPermaLink="false">elasticsearch-supports-promql</guid>
            <pubDate>Wed, 15 Apr 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[Elasticsearch now supports PromQL natively as a first-class source command in ES|QL. Run familiar Prometheus queries on your time series data directly in Kibana.]]></description>
            <content:encoded><![CDATA[<p>Many teams already rely on PromQL in their day-to-day work.
We're making PromQL a first-class experience in Elasticsearch.</p>
<p>The new <code>PROMQL</code> command in ES|QL lets you query time series data in Elasticsearch with PromQL, whether it came from Prometheus Remote Write, OpenTelemetry, or another source.</p>
<p>Metrics, logs, and traces - all in one place, ready to explore in Kibana.</p>
<p><img src="https://www.elastic.co/observability-labs/assets/images/elasticsearch-supports-promql/image1.png" alt="" /></p>
<h2>The PROMQL source command</h2>
<p><code>PROMQL</code> is a source command in ES|QL, similar to <code>FROM</code> or <code>TS</code>.
It takes standard PromQL parameters and a PromQL expression, executes the query, and returns the results as regular ES|QL columns that you can continue to process with other commands.</p>
<p>Here is the general syntax:</p>
<pre><code class="language-esql">PROMQL [index=&lt;pattern&gt;] [step=&lt;duration&gt;] [start=&lt;timestamp&gt;] [end=&lt;timestamp&gt;]
  [&lt;value_column_name&gt;=](&lt;PromQL expression&gt;)
</code></pre>
<p>The parameters mirror the <a href="https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries">Prometheus HTTP API query parameters</a> (<code>step</code>, <code>start</code>, <code>end</code>), so they should feel familiar if you have used the Prometheus query API before.</p>
<h3>A basic range query</h3>
<p>This query calculates the per-second rate of HTTP requests over a sliding 5-minute window, grouped by instance:</p>
<pre><code class="language-esql">PROMQL index=metrics-*
  step=1m
  start=&quot;2026-04-01T00:00:00Z&quot;
  end=&quot;2026-04-01T01:00:00Z&quot;
  sum by (instance) (rate(http_requests_total[5m]))
</code></pre>
<p>The result contains three columns:</p>
<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>sum by (instance) (rate(http_requests_total[5m]))</code></td>
<td><code>double</code></td>
<td>The computed metric value</td>
</tr>
<tr>
<td><code>step</code></td>
<td><code>date</code></td>
<td>The timestamp for each evaluation step</td>
</tr>
<tr>
<td><code>instance</code></td>
<td><code>keyword</code></td>
<td>The grouping label from <code>by (instance)</code></td>
</tr>
</tbody>
</table>
<p>When the PromQL expression includes a cross-series aggregation like <code>sum by (instance)</code>, each grouping label becomes its own output column.
When there is no cross-series aggregation, all labels are returned in a single <code>_timeseries</code> column as a JSON string.</p>
<h3>Naming the value column</h3>
<p>By default, the value column name is the PromQL expression itself.
You can assign a custom name to make it easier to reference in downstream commands:</p>
<pre><code class="language-esql">PROMQL index=metrics-*
  step=1m
  start=&quot;2026-04-01T00:00:00Z&quot;
  end=&quot;2026-04-01T01:00:00Z&quot;
  http_rate=(sum by (instance) (rate(http_requests_total[5m])))
| SORT http_rate DESC
</code></pre>
<p>This works the same way as naming aggregations in <code>STATS</code>, for example <code>STATS avg_cpu = avg(system.cpu.usage)</code>.</p>
<h3>Index patterns</h3>
<p>The <code>index</code> parameter accepts the same patterns as <code>FROM</code> and <code>TS</code>, including wildcards and comma-separated lists.
If omitted, it defaults to <code>*</code>, which queries all indices configured with <a href="https://www.elastic.co/docs/manage-data/data-store/data-streams/time-series-data-stream-tsds"><code>index.mode: time_series</code></a>.
In production, specifying an explicit index pattern avoids scanning unrelated data.</p>
<h2>How it works under the hood</h2>
<p>The <code>PROMQL</code> command does not run a separate query engine.
Instead, <code>PROMQL</code> commands execute inside the ES|QL compute engine, using the same logic as time-series aggregations through the <a href="https://www.elastic.co/docs/reference/query-languages/esql/commands/ts"><code>TS</code></a> source command.</p>
<p>Consider this PromQL query:</p>
<pre><code class="language-esql">PROMQL index=metrics-*
  step=1m
  start=&quot;2026-04-01T00:00:00Z&quot;
  end=&quot;2026-04-01T01:00:00Z&quot;
  sum by (host.name) (rate(http_requests_total[5m]))
</code></pre>
<p>Internally, the <code>PROMQL</code> command translates this into an equivalent ES|QL query using the <code>TS</code> source:</p>
<pre><code class="language-esql">TS metrics-*
| WHERE TRANGE(&quot;2026-04-01T00:00:00Z&quot;, &quot;2026-04-01T01:00:00Z&quot;)
| STATS SUM(RATE(http_requests_total, 5m)) BY TBUCKET(1m), host.name
</code></pre>
<p>Both queries produce the same result.
The <code>PROMQL</code> command parses the PromQL syntax, resolves functions to their ES|QL equivalents (<code>rate</code> to <code>RATE</code>, <code>sum</code> to <code>SUM</code>, <code>avg_over_time</code> to <code>AVG_OVER_TIME</code>, and so on), and constructs a logical plan that the ES|QL engine executes.</p>
<p>This translation approach has a practical benefit: PromQL queries automatically benefit from all the optimizations in the ES|QL engine, including segment-level parallelism and time series-aware data access patterns.</p>
<p>There are currently 19 time series functions available, covering rates, deltas, derivatives, and various <code>*_over_time</code> aggregations.</p>
<h2>Smart defaults that simplify queries</h2>
<p>In Prometheus, a PromQL query requires explicit <code>start</code>, <code>end</code>, and <code>step</code> parameters.
In Kibana, those are usually determined by the date picker and panel size.
The <code>PROMQL</code> command has three features that make queries adapt automatically.</p>
<h3>Auto-step</h3>
<p>If you omit the <code>step</code> parameter, the command derives it automatically based on the time range and a target bucket count (default: 100).
You can also set the target explicitly with <code>buckets=&lt;n&gt;</code>.</p>
<pre><code class="language-esql">PROMQL index=metrics-*
  start=&quot;2026-04-01T00:00:00Z&quot;
  end=&quot;2026-04-01T01:00:00Z&quot;
  sum by (instance) (rate(http_requests_total[5m]))
</code></pre>
<p>With a 1-hour range and the default target of 100 buckets, the step would be 1m, resulting in 60 buckets.
This uses the same date-rounding logic as the ES|QL <a href="https://www.elastic.co/docs/reference/query-languages/esql/functions-operators/grouping-functions#esql-bucket"><code>BUCKET</code></a> function.</p>
<h3>Inferred start and end</h3>
<p>Kibana adds a time range filter to every ES|QL request via a Query DSL <code>range</code> filter on <code>@timestamp</code>.
The <code>PROMQL</code> command extracts those bounds and uses them as <code>start</code> and <code>end</code> when they are not specified in the query.
The command picks up the date picker range from the request context without any additional configuration.</p>
<h3>Implicit range selectors</h3>
<p>In standard PromQL, functions like <code>rate</code> require a range selector: <code>rate(http_requests_total[5m])</code>.
The <code>PROMQL</code> command allows omitting the range selector entirely:</p>
<pre><code class="language-esql">PROMQL sum by (instance) (rate(http_requests_total))
</code></pre>
<p>When the range selector is absent, the window is determined automatically as <code>max(step, scrape_interval)</code>.
The <code>scrape_interval</code> defaults to <code>1m</code> and can be overridden with the <code>scrape_interval</code> parameter if your data has a different collection interval, for example: <code>PROMQL scrape_interval=15s sum(rate(http_requests_total))</code>.</p>
<h3>The result</h3>
<p>Combining all three defaults, a fully adaptive query in Kibana looks like this:</p>
<pre><code class="language-esql">PROMQL sum(rate(http_requests_total))
</code></pre>
<p>This query responds to the date picker, adjusts the step size to the selected time range, and sizes the range selector window accordingly.
No manual tuning needed.</p>
<h2>Post-processing with ES|QL</h2>
<p>Because <code>PROMQL</code> is an ES|QL source command, its output flows into the rest of the ES|QL pipeline.
You can filter, sort, enrich, and transform PromQL results using any ES|QL command.</p>
<h3>Filter results</h3>
<pre><code class="language-esql">PROMQL index=metrics-*
  http_rate=(sum by (instance) (rate(http_requests_total[5m])))
| WHERE http_rate &gt; 100
</code></pre>
<h3>Sort and limit</h3>
<pre><code class="language-esql">PROMQL index=metrics-*
  http_rate=(sum by (instance) (rate(http_requests_total[5m])))
| SORT http_rate DESC
| LIMIT 10
</code></pre>
<h3>Enrich with a lookup</h3>
<pre><code class="language-esql">PROMQL index=metrics-*
  http_rate=(sum by (instance) (rate(http_requests_total[5m])))
| LOOKUP JOIN instance_metadata ON instance
</code></pre>
<p>This is something you cannot do in Prometheus.
PromQL results are self-contained; there is no way to join them with external data or apply arbitrary post-processing.
In Elasticsearch, the PromQL output is just the first stage of a query that can continue with any ES|QL operation.</p>
<h2>Current coverage and what's next</h2>
<p>In 9.4, the <code>PROMQL</code> command will be available as a tech preview with over 80% query coverage benchmarked against popular Grafana open source dashboards.</p>
<p>The most notable gaps in the current tech preview:</p>
<ul>
<li><strong>Group modifiers</strong> like <code>on(chip) group_left(chip_name)</code> are not yet supported.</li>
<li><strong>Binary set operators</strong> (<code>or</code>, <code>and</code>, <code>unless</code>) are not yet available.</li>
<li><strong>Some functions</strong> are still missing, including <code>histogram_quantile</code>, <code>predict_linear</code>, and <code>label_join</code>.</li>
</ul>
<p>These are all planned for upcoming releases.
The roadmap includes broader PromQL function and operator coverage, Prometheus-aligned step semantics, and support for native histograms.</p>
<h2>Try it</h2>
<p>PromQL support is available as a tech preview on Elasticsearch Serverless with no additional configuration.
For self-managed clusters, it is available starting with version 9.4.</p>
<p>To try it in Kibana:</p>
<ol>
<li>Go to <strong>Dashboards</strong>, create a new panel, and select <strong>ES|QL</strong> as the query type.</li>
<li>Enter a <code>PROMQL</code> query, for example: <code>PROMQL index=metrics-* sum by (host.name) (rate(http_requests_total))</code>.</li>
<li>The command automatically infers the time range from the Kibana date picker, so no additional parameters are needed.</li>
</ol>
<p>You can also run PromQL queries in the ES|QL mode of <strong>Discover</strong>, which shows results in a table and an XY chart.
Stay tuned for a full walkthrough of using PromQL in Kibana Dashboards, Discover, and Alerting in a dedicated Kibana blog post.</p>
<p>If you want to try it with a self-managed cluster, check out <a href="https://www.elastic.co/docs/deploy-manage/deploy/self-managed/local-development-installation-quickstart">start-local</a> to get up and running quickly.</p>
<p>If you run into issues or have feedback, open an issue on the <a href="https://github.com/elastic/elasticsearch">Elasticsearch repository</a>.</p>
]]></content:encoded>
            <category>observability-labs</category>
            <enclosure url="https://www.elastic.co/observability-labs/assets/images/elasticsearch-supports-promql/cover.svg" length="0" type="image/svg"/>
        </item>
    </channel>
</rss>