Category: Index Data

Articles tagged Index Data

Subscribe
Filters
One field, one copy: How Elasticsearch columnar storage drops the inverted index
Elasticsearch Labs

One field, one copy: How Elasticsearch columnar storage drops the inverted index

Storing each field once means no inverted index, so doc values now read in bulk and skippers let queries skip whole ranges of documents, while new mapping attributes control what each field is allowed to contain.

Martijn van Groningen
Backfill time series data in Elasticsearch: Load months of historical metrics through the bulk API
Elasticsearch Labs

Backfill time series data in Elasticsearch: Load months of historical metrics through the bulk API

Elasticsearch works out the time boundaries and creates the past backing indices as the documents land, so a historical data migration runs on your normal ingest path.

Mary Gouseti
ES95: Adaptive Compression for Elasticsearch Time-Series Metrics
Elasticsearch Labs

ES95: Adaptive Compression for Elasticsearch Time-Series Metrics

ES95 is Elasticsearch 9.5's new adaptive time series codec that cuts @timestamp storage by 92% and floating point fields by up to 74%, with zero configuration.

Salvatore Campagna
Two lines of JSON to replace your ILM policy: data stream lifecycle adds frozen tier support
Elasticsearch Labs

Two lines of JSON to replace your ILM policy: data stream lifecycle adds frozen tier support

In Elasticsearch 9.5, frozen_after in data stream lifecycle moves indices to searchable snapshots on object storage on their own, keeping them queryable alongside downsampling and retention.

Edward Lewis
One field, every modality: how Elasticsearch's semantic field indexes and searches images, audio, video and PDFs automatically
Elasticsearch Labs

One field, every modality: how Elasticsearch's semantic field indexes and searches images, audio, video and PDFs automatically

The semantic field turns images, audio, video, PDFs and text into multimodal embeddings at ingest time. Describe a scene and find the matching image or use a video frame to surface related clips, all from one Elasticsearch field.

Mike Pellegrini
Why your Elasticsearch cluster is hitting disk watermarks: 14 real-world causes explained
Elasticsearch Labs

Why your Elasticsearch cluster is hitting disk watermarks: 14 real-world causes explained

Learn how Elasticsearch disk watermarks work, why they trigger, and how to diagnose 14 of the most common scenarios Support encounters, from index bloat to ILM stalls.

Stef Nestor
How DocValuesSkippers in Lucene 10 make range queries faster without doubling your storage
Elasticsearch Labs

How DocValuesSkippers in Lucene 10 make range queries faster without doubling your storage

DocValuesSkippers add block-level skipping to Lucene DocValues fields, speeding up range queries on sorted or insert-ordered indexes with less than 0.1% storage overhead.

Alan Woodward
Elasticsearch reindex now relocates across nodes automatically: zero user intervention, no lost progress
Elasticsearch Labs

Elasticsearch reindex now relocates across nodes automatically: zero user intervention, no lost progress

Elasticsearch reindex now survives node shutdowns, uses Point in Time for more efficient source iteration, and ships with dedicated management APIs. Reindex-from-remote is GA in Serverless.

Pete Naylor
Elasticsearch downsampling methods: last-value vs. aggregate sampling
Elasticsearch Labs

Elasticsearch downsampling methods: last-value vs. aggregate sampling

Elasticsearch downsampling now gives you a choice: last-value sampling for maximum storage savings or aggregate sampling for precise rate calculations and counter resets, both fully queryable in ES|QL.

Mary Gouseti
When TSDS meets ILM: Designing time series data streams that don't reject late data
Elasticsearch Labs

When TSDS meets ILM: Designing time series data streams that don't reject late data

How TSDS time bounds interact with ILM phases; and how to design policies that tolerate late-arriving metrics.

Bret Wortman
Reducing Elasticsearch frozen tier costs with Deepfreeze S3 Glacier archival
Elasticsearch Labs

Reducing Elasticsearch frozen tier costs with Deepfreeze S3 Glacier archival

Learn how to leverage Deepfreeze in Elasticsearch to automate searchable snapshot repository rotation, retaining historical data and aging it into lower cost S3 Glacier tiers after index deletion.

Bret Wortman
Elastic Open Web Crawler as a code
Elasticsearch Labs

Elastic Open Web Crawler as a code

Learn how to use GitHub Actions to manage Elastic Open Crawler configurations, so every time we push changes to the repository, the changes are automatically applied to the deployed instance of the crawler.

Gustavo Llermaly