Oversized shards
One or more shards exceed the recommended size range (roughly 50 GB, or a very high document count). Oversized shards slow recovery, relocation, and search, and in extreme cases can become hard to move or write.
For a complete list of insights, refer to AutoOps insights.
| Field | Value |
|---|---|
| Component | Elasticsearch |
| Severity | Medium |
| Scope | Index |
| Domains | performance, index-management, shard-allocation |
You can customize these settings to adjust when AutoOps detects this event and presents the insight. Refer to AutoOps event settings for details.
The default customization settings are:
| Setting | Type | Default |
|---|---|---|
| Maximum optimal shard size in GB | Integer | 0.0004 |
| Exclude indices based on this pattern | List of strings |
The following is an example of what you might see when this insight is triggered. Real insights use live data and links from your deployment or cluster.
Total number of indices with large shards: 12
The index that contains the largest shard is: logs-prod-000045, and the shard size is: 42 GB.
See a sample list: logs-prod-000045, logs-prod-000046.
AutoOps shows different recommendations depending on how their conditions match your deployment or cluster.
ILM policy for large shards
Condition: Always shown for this insight.
If logs-prod-000045 is time-based, set or tune its ILM policy so rollover targets roughly 20–30 GB shards.
Reindex into smaller indices
Condition: Shown for non-system indexes.
Reindex into smaller indices to reduce shard sizes and improve search performance.
Split index into more shards
Condition: Shown when primary shard count is less than data node count and shard size warrants split.
Use the Split API to split logs-prod-000045 (current size 42 GB) into 2 primary shards with the action below. Run during low cluster load and confirm you have enough disk space. See the Split index API for prerequisites.
POST logs-prod-000045/_split/logs-prod-000045-0001
{
"settings": {
"index.number_of_shards": 2
}
}
Requires the manage cluster privilege. Requires Elasticsearch 8.0.0 or later. This action changes cluster or index configuration.
Index data is partitioned into shards when the index is created. Shard size grows with the data you write into that index.
Shards that grow well beyond about 50 GB, or that hold hundreds of millions of documents, take longer to recover and relocate. Searches that hit those shards can also run more slowly. In extreme cases a shard can approach hard limits (for example around 2 billion documents) and become difficult to write to or move.
Keeping shards in a moderate size range helps the cluster recover faster and keeps search and indexing more predictable as data volume grows.