Also see Breaking changes in 6.0.
Breaking changes
- Network
- Allow only a fixed-size receive predictor #26165 (issue: #23185)
- REST
- Standardize underscore requirements in parameters #27414 (issues: #26886, #27040)
/_nodes/settings?flat_settings returns "settings" key/value values as string and array values #26878 (issue: #27805)
- Scroll
- Fail queries with scroll that explicitely set request_cache #27342
- Search
- Add a limit to from + size in top_hits and inner hits. #26492 (issue: #11511)
Breaking Java changes
- Aggregations
- Moves deferring code into its own subclass #26421
- Core
- Unify Settings xcontent reading and writing #26739
- Settings
- Return List instead of an array from settings #26903
- Remove
Settings,put(Map<String,String>) #26785
Deprecations
- Aggregations
- Deprecate global_ordinals_hash and global_ordinals_low_cardinality #26173 (issue: #26014)
- Allocation
- Add deprecation warning for negative index.unassigned.node_left.delayed_timeout #26832 (issue: #26828)
- Analysis
- Geo
- [GEO] 6x Deprecate ShapeBuilders and decouple geojson parse logic #27345
- Mapping
- Deprecate the
index_options parameter for numeric fields #26672 (issue: #21475)
- Plugin Repository Azure
- Azure repository: Move to named configurations as we do for S3 repository and secure settings #23405 (issues: #22762, #22763)
- Search
- doc: deprecate _primary and _replica shard option #26792 (issue: #26335)
New features
- Aggregations
- Aggregations: bucket_sort pipeline aggregation #27152 (issue: #14928)
- Add composite aggregator #26800
- Analysis
- Added Bengali Analyzer to Elasticsearch with respect to the lucene update #26527
- Ingest
- Java High Level REST Client
- Added Delete Index support to high-level REST client #27019 (issue: #25847)
- Nested Docs
- Multi-level Nested Sort with Filters #26395
- Query DSL
- Add terms_set query #27145 (issue: #26915)
- Introduce sorted_after query for sorted index #26377
- Add support for auto_generate_synonyms_phrase_query in match_query, multi_match_query, query_string and simple_query_string #26097
- Search
- Similarities
- Add a scripted similarity. #25831
- Suggesters
- Expose duplicate removal in the completion suggester #26496 (issue: #23364)
- Support must and should for context query in context suggester #26407 (issues: #24421, #24565)
Enhancements
- Aggregations
- Allow aggregation sorting via nested aggregation #26683 (issue: #16838)
- Allocation
- Tie-break shard path decision based on total number of shards on path #27039 (issue: #26654)
- Balance shards for an index more evenly across multiple data paths #26654 (issue: #16763)
- Expand "NO" decision message in NodeVersionAllocationDecider #26542 (issue: #10403)
- _reroute’s retry_failed flag should reset failure counter #25888 (issue: #25291)
- Analysis
- Add configurable
max_token_length parameter to whitespace tokenizer #26749 (issue: #26643)
- CRUD
- Add wait_for_active_shards parameter to index open command #26682 (issue: #20937)
- Core
- Fix classes that can exit #27518
- Replace empty index block checks with global block checks in template delete/put actions #27050 (issue: #10530)
- Allow Uid#decodeId to decode from a byte array slice #26987 (issue: #26931)
- Use separate searchers for "search visibility" vs "move indexing buffer to disk #26972 (issues: #15768, #26802, #26912, #3593)
- Add ability to split shards #26931
- Make circuit breaker mutations debuggable #26067 (issue: #25891)
- Dates
- Discovery
- Stop responding to ping requests before master abdication #27329 (issue: #27328)
- Engine
- Ensure external refreshes will also refresh internal searcher to minimize segment creation #27253 (issue: #26972)
- Move IndexShard#getWritingBytes() under InternalEngine #27209 (issue: #26972)
- Refactor internal engine #27082
- Geo
- Add ignore_malformed to geo_shape fields #24654 (issue: #23747)
- Ingest
- add json-processor support for non-map json types #27335 (issue: #25972)
- Introduce templating support to timezone/locale in DateProcessor #27089 (issue: #24024)
- Add support for parsing inline script (#23824) #26846 (issue: #23824)
- Consolidate locale parsing. #26400
- Accept ingest simulate params as ints or strings #23885 (issue: #23823)
- Internal
- Avoid uid creation in ParsedDocument #27241
- Upgrade to Lucene 7.1.0 snapshot version #26864 (issue: #26527)
- Remove
_index fielddata hack if cluster alias is present #26082 (issue: #25885)
- Java High Level REST Client
- Adjust RestHighLevelClient method modifiers #27238
- Decouple BulkProcessor from ThreadPool #26727 (issue: #26028)
- Logging
- Add more information on failed_to_convert exception (#21946) #27034 (issue: #21946)
- Improve shard-failed log messages. #26866
- Mapping
- Allow ip_range to accept CIDR notation #27192 (issue: #26260)
- Deduplicate
_field_names. #26550 - Throw a better error message for empty field names #26543 (issue: #23348)
- Stricter validation for min/max values for whole numbers #26137
- Make FieldMapper.copyTo() always non-null. #25994
- Nested Docs
- Use the primary_term field to identify parent documents #27469 (issue: #24362)
- Prohibit using
nested_filter, nested_path and new nested Option at the same time in FieldSortBuilder #26490 (issue: #17286)
- Network
- Remove manual tracking of registered channels #27445 (issue: #27260)
- Remove tcp profile from low level nio channel #27441 (issue: #27260)
- Decouple
ChannelFactory from Tcp classes #27286 (issue: #27260)
- Percolator
- Use Lucene’s CoveringQuery to select percolate candidate matches #27271 (issues: #26081, #26307)
- Add support to percolate query to percolate multiple documents simultaneously #26418
- Hint what clauses are important in a conjunction query based on fields #26081
- Add support for selecting percolator query candidate matches containing range queries #25647 (issue: #21040)
- Plugin Discovery EC2
- update AWS SDK for ECS Task IAM support in discovery-ec2 #26479 (issue: #23039)
- Plugin Lang Painless
- Painless: Only allow Painless type names to be the same as the equivalent Java class. #27264
- Allow for the Painless Definition to have multiple instances for white-listing #27096
- Separate Painless Whitelist Loading from the Painless Definition #26540
- Remove Sort enum from Painless Definition #26179
- Plugin Repository Azure
- Add azure storage endpoint suffix #26432 #26568 (issue: #26432)
- Support for accessing Azure repositories through a proxy #23518 (issues: #23506, #23517)
- Plugin Repository S3
- Plugins
- Plugins: Add versionless alias to all security policy codebase properties #26756 (issue: #26521)
- Allow plugins to plug rescore implementations #26368 (issue: #26208)
- Query DSL
- Reindex API
- Update by Query is modified to accept short
script parameter. #26841 (issue: #24898) - reindex: automatically choose the number of slices #26030 (issues: #24547, #25582)
- Rollover
- Add size-based condition to the index rollover API #27160 (issue: #27004)
- Add size-based condition to the index rollover API #27115 (issue: #27004)
- Scripting
- Script: Convert script query to a dedicated script context #26003
- Search
- Make fields optional in multi_match query and rely on index.query.default_field by default #27380
- fix unnecessary logger creation #27349
ObjectParser : replace IllegalStateException with ParsingException #27302 (issue: #27147)- Uses norms for exists query if enabled #27237
- Cross-cluster search: make remote clusters optional #27182 (issues: #26118, #27161)
- Enhances exists queries to reduce need for
_field_names #26930 (issue: #26770) - Change ParentFieldSubFetchPhase to create doc values iterator once per segment #26815
- Change VersionFetchSubPhase to create doc values iterator once per segment #26809
- Change ScriptFieldsFetchSubPhase to create search scripts once per segment #26808 (issue: #26775)
- Make sure SortBuilders rewrite inner nested sorts #26532
- Extend testing of build method in ScriptSortBuilder #26520 (issues: #17286, #26490)
- Accept an array of field names and boosts in the index.query.default_field setting #26320 (issue: #25946)
- Reject IPv6-mapped IPv4 addresses when using the CIDR notation. #26254 (issue: #26078)
- Rewrite range queries with open bounds to exists query #26160 (issue: #22640)
- Sequence IDs
- Only fsync global checkpoint if needed #27652
- Log primary-replica resync failures #27421 (issues: #24841, #27418)
- Lazy initialize checkpoint tracker bit sets #27179 (issue: #10708)
- Returns the current primary_term for Get/MultiGet requests #27177 (issue: #26493)
- Settings
- Allow affix settings to specify dependencies #27161
- Represent lists as actual lists inside Settings #26878 (issue: #26723)
- Remove Settings#getAsMap() #26845
- Replace group map settings with affix setting #26819
- Throw exception if setting isn’t recognized #26569 (issue: #25607)
- Settings: Move keystore creation to plugin installation #26329 (issue: #26309)
- Snapshot/Restore
- Remove XContentType auto detection in BlobStoreRepository #27480
- Snapshot: Migrate TransportRequestHandler to TransportMasterNodeAction #27165 (issue: #27151)
- Fix toString of class SnapshotStatus (#26851) #26852 (issue: #26851)
- Stats
- Adds average document size to DocsStats #27117 (issue: #27004)
- Stats to record how often the ClusterState diff mechanism is used successfully #27107 (issue: #26973)
- Expose adaptive replica selection stats in /_nodes/stats API #27090
- Add cgroup memory usage/limit to OS stats on Linux #26166
- Add segment attributes to the
_segments API. #26157 (issue: #26130)
- Suggesters
- Improve error message for parse failures of completion fields #27297
- Support AND operation for context query in context suggester #24565 (issue: #24421)
Bug fixes
- Aggregations
- Disable the "low cardinality" optimization of terms aggregations. #27545 (issue: #27543)
- scripted_metric _agg parameter disappears if params are provided #27159 (issues: #19768, #19863)
- Cluster
- Properly format IndexGraveyard deletion date as date #27362
- Remove optimisations to reuse objects when applying a new
ClusterState #27317
- Core
- Do not set data paths on no local storage required #27587 (issue: #27572)
- Ensure threadcontext is preserved when refresh listeners are invoked #27565
- Ensure logging is configured for CLI commands #27523 (issue: #27521)
- Protect shard splitting from illegal target shards #27468 (issue: #26931)
- Avoid NPE when getting build information #27442
- Fix
ShardSplittingQuery to respect nested documents. #27398 (issue: #27378) - When building Settings do not set SecureSettings if empty #26988 (issue: #316)
- Engine
- Reset LiveVersionMap on sync commit #27534 (issue: #27516)
- Carry over version map size to prevent excessive resizing #27516 (issue: #20498)
- Geo
- Correct two equality checks on incomparable types #27688
- [GEO] fix pointsOnly bug for MULTIPOINT #27415
- Index Templates
- Prevent constructing an index template without index patterns #27662
- Ingest
- Add pipeline support for REST API bulk upsert #27075 (issue: #25601)
- Fixing Grok pattern for Apache 2.4 #26635
- Inner Hits
- Return an empty _source for nested inner hit when filtering on a field that doesn’t exist #27531
- Internal
- When checking if key exists in ThreadContextStruct:putHeaders() method，should put requestHeaders in map first #26068
- Adding a refresh listener to a recovering shard should be a noop #26055
- Java High Level REST Client
- Register ip_range aggregation with the high level client #26383
- add top hits as a parsed aggregation to the rest high level client #26370
- Mapping
- Fix dynamic mapping update generation. #27467
- Fix merging of _meta field #27352 (issue: #27323)
- Fixed rounding of bounds in scaled float comparison #27207 (issue: #27189)
- Nested Docs
- Ensure nested documents have consistent version and seq_ids #27455
- Prevent duplicate fields when mixing parent and root nested includes #27072 (issue: #26990)
- Network
- Throw UOE from compressible bytes stream reset #27564 (issue: #24927)
- Bubble exceptions when closing compressible streams #27542 (issue: #27540)
- Do not set SO_LINGER on server channels #26997
- Do not set SO_LINGER to 0 when not shutting down #26871 (issue: #26764)
- Close TcpTransport on RST in some Spots to Prevent Leaking TIME_WAIT Sockets #26764 (issue: #26701)
- Packaging
- Removes minimum master nodes default number #26803
- setgid on /etc/elasticearch on package install #26412 (issue: #26410)
- Percolator
- Avoid TooManyClauses exception if number of terms / ranges is exactly equal to 1024 #27519 (issue: #1)
- Plugin Analysis ICU
- Catch InvalidPathException in IcuCollationTokenFilterFactory #27202
- Plugin Lang Painless
- Painless: Fix variable scoping issue in lambdas #27571 (issue: #26760)
- Painless: Fix errors allowing void to be assigned to def. #27460 (issue: #27210)
- Plugin Repository GCS
- Create new handlers for every new request in GoogleCloudStorageService #27339 (issue: #27092)
- Recovery
- Flush old indices on primary promotion and relocation #27580 (issue: #27536)
- Reindex API
- Reindex: Fix headers in reindex action #26937 (issue: #22976)
- Scroll
- Fix scroll query with a sort that is a prefix of the index sort #27498
- Search
- Fix profiling naming issues #27133
- Fix max score tracking with field collapsing #27122 (issue: #23840)
- Apply missing request options to the expand phase #27118 (issues: #26649, #27079)
- Calculate and cache result when advanceExact is called #26920 (issue: #26817)
- Filter unsupported relation for RangeQueryBuilder #26620 (issue: #26575)
- Handle leniency for phrase query on a field indexed without positions #26388
- Sequence IDs
- Obey translog durability in global checkpoint sync #27641
- Fix resync request serialization #27418 (issue: #24841)
- Settings
- Allow index settings to be reset by wildcards #27671 (issue: #27537)
- Snapshot/Restore
- Do not swallow exception in ChecksumBlobStoreFormat.writeAtomic() #27597
- Delete shard store files before restoring a snapshot #27476 (issues: #20220, #26865)
- Fix snapshot getting stuck in INIT state #27214 (issue: #27180)
- Fix default value of ignore_unavailable for snapshot REST API (#25359) #27056 (issue: #25359)
- Do not create directory on readonly repository (#21495) #26909 (issue: #21495)
- Stats
- Include internal refreshes in refresh stats #27615
- Make Segment statistics aware of segments hold by internal readers #27558
- Ensure
doc_stats are changing even if refresh is disabled #27505
Upgrades
- Core
- Upgrade to Jackson 2.8.10 #27230
- Upgrade to Lucene 7.1 #27225
- Plugin Discovery EC2
- Plugin Discovery GCE
- Plugin Lang Painless
- Upgrade Painless from ANTLR 4.5.1-1 to ANTLR 4.5.3. #27153