ES|QL TS_COLLAPSE command

TS_COLLAPSE is a processing command that collapses the expanded, per-step output of a PROMQL command into one row per time series. The metric column and the step column become multi-valued fields; dimension columns remain single-valued.

... | TS_COLLAPSE
		

None.

A PROMQL command produces one row per (series, step) pair, with dimension columns (for example host, cluster) repeated on every row. TS_COLLAPSE merges all rows that belong to the same series into a single row:

• The metric column (the named result column, for example rate) and the step column become multi-valued (MV) fields whose elements are ordered chronologically.
• Dimension columns keep a single value per row because they are identical for all steps of a given series.
• Missing or null samples are omitted from both MV fields, so the result is sparse rather than null-filled.

The MV fields are positionally aligned: step[i] corresponds to the metric value at index i. Applying MV functions independently to one of the two columns—for example MV_SORT—will break this alignment.

TS_COLLAPSE can only appear immediately after a PROMQL command. Other processing commands can follow TS_COLLAPSE.

A PROMQL query without TS_COLLAPSE returns one row per (series, step) pair. Adding TS_COLLAPSE produces one row per series, with the metric and step columns as positionally aligned MV arrays:

PROMQL index=k8s step=1m start="2024-05-10T00:11:00.000Z" end="2024-05-10T00:14:00.000Z" cost=(round(sum(avg_over_time(network.cost[1m])), 0.0001))
| TS_COLLAPSE
		

When the PROMQL query groups results by a dimension (for example cluster), TS_COLLAPSE returns one row per unique dimension value, with metric values and steps collapsed into MV arrays:

PROMQL index=k8s step=1m start="2024-05-10T00:02:00.000Z" end="2024-05-10T00:04:00.000Z" clients=(avg by (cluster) (last_over_time(network.eth0.currently_connected_clients[1m])))
| TS_COLLAPSE
| SORT cluster
		

Use EVAL with MV functions to compute per-series aggregates over the collapsed arrays, then sort or filter on the results:

PROMQL index=k8s step=1m start="2024-05-10T00:02:00.000Z" end="2024-05-10T00:04:00.000Z" clients=(avg by (cluster) (last_over_time(network.eth0.currently_connected_clients[1m])))
| TS_COLLAPSE
| EVAL avg_clients = MV_AVG(clients)
| SORT avg_clients DESC
| LIMIT 10