Breaking changes

• Aliases:
  • Aliases: Throw exception if index is null or missing when creating an alias #8240 (issues: #7863, #7976)
• Benchmark:
  • Benchmark api: removed leftovers #10180
• Core:
  
  • Resiliency: Throw exception if the JVM will corrupt data #7580
• Engine:
  • [ENGINE] Remove full flush / FlushType.NEW_WRITER #9559
• Index APIs:
  • Change behaviour of indices segments api to allow no indices #9219 (issue: #5856)
• Plugins:
  • Plugins: Don't overwrite plugin configuration when removing/upgrading plugins #7890 (issue: #5064)
• Recovery:
  • Recovery: RecoveryState clean up #9811 (issue: #6644)
• Scripting:
  • Scripting: cleanup ScriptService & friends in preparation for #6418 #9992 (issue: #6418)
  • Disable dynamic Groovy scripting by marking Groovy as not sandboxed [ISSUE] #9655
  • Scripting: Script with _score: remove dependency of DocLookup and scorer #7819 (issues: #7043, #7487)

Deprecations

• CRUD:
  • Deprecate async replication #10161 (issue: #10114)
• Core:
  • Core: deprecate index.fail_on_merge_failure #10084
• Mapping:
  • Mappings: Deprecate _analyzer and per query analyzers #9383 (issue: #9279)
• More Like This:
  • Deprecation: MLT Field Query #8253
• Network:
  • Deprecated the thrift and memcached transports #10167 (issue: #10166)
• Search:
  • Core: add deprecation messages for delete-by-query #10082

New features

• Aggregations:
  • New aggregations feature - “PercentageScore" heuristic for significant_terms #9747 (issue: #9720)
  • significant terms: add scriptable significance heuristic #7850
• CAT API:
  • Cat API: show open and closed indices in _cat/indices #7936 (issue: #7907)
• Circuit Breakers:
  • Circuit Breakers: Add NoopCircuitBreaker used in NoneCircuitBreakerService #8063
• Engine:
  • Shadow replicas on shared filesystems #9727 (issue: #8976)
• More Like This:
  • MLT Query: Support for artificial documents #7725
• Query DSL:
  • Add time_zone setting for query_string #8164 (issue: #7880)
  • Search: add format support for date range filter and queries #7821 (issue: #7189)
  • Add min_score parameter to function score query to only match docs above this threshold #7814 (issue: #6952)
• Search:
  • Add inner hits to nested and parent/child queries #8153 (issues: #3022, #3152)
• Store:
  • Add index.data_path setting #9033 (issues: #8819, #8976)
• Term Vectors:
  • Term Vectors/MLT Query: support for different analyzers than default at the field #7801

Enhancements

• Aggregations:
  • Aggregations: deprecate pre_zone and <code>post_zone in date_histogram #9722 (issue: #9062)
  • Aggregations: Add 'offset' option to date_histogram #9614 (issue: #9062)
  • Aggregations: Numeric metric aggregations are now formattable #9032 (issue: #6812)
  • Aggregations: Adds methods to get to/from as Strings for Range Aggs #9026 (issue: #9003)
  • Made the nested, <code>reverse_nested and children aggs ignore unmapped nested fields or unmapped child / parent types. #8808 (issue: #8760)
  • Fixed #6061 size parsing consistent for strings #8645
  • Aggregations: Do not take deleted documents into account in aggregations filters. #8540
  • Aggs - support for arrays of numeric values in include/exclude clauses #7727 (issue: #7714)
  • Aggregations: Adds ability to sort on multiple criteria #7662 (issues: #6917, #7588)
  • Aggregations: Properly support top_hits aggregation in a nested and reverse_nested aggregations. #7164 (issue: #3022)
• Allocation:
  • Allocation: Speed-up disk-threshold decider #8803 (issue: #6372)
  • Allocation: DiskThresholdDecider#remain(...) should take shards relocating away into account #8659 (issue: #8538)
  • Allocation: Take percentage watermarks into account for reroute listener #8382 (issues: #8367, #8368)
  • Allocation: Reroute shards automatically when high disk watermark is exceeded #8270 (issue: #8146)
  • Add option to take currently relocating shards' sizes into account #7785 (issues: #6168, #7753)
• CAT API:
  • Mark shadow replicas with 's' in _cat/shards output #10023 (issue: #9772)
  • Admin: add total index memory in _cat/indices #7824 (issue: #7008)
  • Add file descriptor details to cat/nodes #7655 (issue: #7652)
• Circuit Breakers:
  • Circuit Breakers: Log if CircuitBreaker is tripping #8050
• Core:
  • Core: don't rethrow already handled merge exceptions #10083
  • [ENV] NodeEnv should lock all shards for an index #9799
  • [CORE] Allow primary promotion on shadow replica without failing the shard #9786
  • [INDICES] Retry if shard deletes fail due to IOExceptions #9784
  • Core: only do a single listAll from FileSwitchDir #9666 (issue: #6636)
  • Core: Remove ability to run optimize and upgrade async #9640 (issue: #9638)
  • Update joda-time to v2.7 #9610
  • [CORE] Consolidate index / shard deletion in IndicesService #9605
  • Minor hygiene, Removed Redundant inheritance #9427
  • Core: don't throttle recovery indexing operations #9396 (issue: #9394)
  • Core: increase default xlog flush size from 200mb to 512 mb #9341 (issue: #9265)
  • Pass through all exceptions in IndicesLifecycleListeners #9330
  • Pass index settings to IndicesLifecycle#beforeIndexCreated and #afterIndexShardClosed #9245
  • Core: Fix Store.checkIntegrity() for lucene 3.x index files. #9234
  • Core: Don't verify adler32 for lucene 3.x terms dict/terms index. #9142
  • [CORE] Delete shard content under lock #9083 (issues: #8608, #9009)
  • [SEARCH] close active contexts on SearchService#close() #8947 (issue: #8940)
  • Core: use Lucene's defaults for compound file format #8934 (issue: #8919)
  • [CORE] Remove explicit .cleanUp() on cache clear #8924
  • Core: clarify index removal log message #8641
  • [CORE] Ensure shards are deleted under lock on close #8579
  • [INDEX] Add before/after indexDeleted callbacks to IndicesLifecycle #8569 (issue: #8551)
  • [CORE] Free pending search contexts if index is closed #8551
  • [CORE] Ban all usage of Future#cancel(true) #8494
  • Core: set bloom default to false even when Directory doesn't have a codecService #8442
  • [CORE] Introduce shard level locks to prevent concurrent shard modifications #8436
  • Discovery: Don't wait joinThread when stopping #8359 (issue: #8327)
  • [STATE] Observe cluster state on health request #8350
  • [UTILITIES] Introduce a RefCounted interface and basic impl #8210
  • Use 1 instead of 0 as filler version value for nested docs #8145
  • [CORE] Add cluster and index state checksums #8010 (issue: #7586)
  • Resiliency: Perform write consistency check just before writing on the primary shard #7873
  • [CORE] Add ThreadPool.terminate to streamline shutdown #7868
  • [CORE] Add ActionRunnable support to ThreadPool to simplify async operation on bounded threadpools #7765
  • Internal: Add missing cluster blocks handling for master operations #7763 (issue: #7740)
• Discovery:
  • [Discovery] Prevent stale master nodes from sharing dated cluster states to nodes that have moved to a different master node #9632
  • Discovery: publishing timeout to log at WARN and indicate pending nodes #9551
  • Discovery: Concurrent node failures can cause unneeded cluster state publishing #8933 (issue: #8804)
  • [CLIENT] Only fetch the node info during node sampling #8685
  • Discovery: don't accept a dynamic update to min_master_nodes which s larger then current master node count #8321
  • Discovery: Improve handling of multicast binding exceptions #8243 (issue: #8225)
  • Discovery: Simplify discovery node initialization if version is unknown #8055 (issue: #8051)
  • Discovery: Close ping handler's executor service properly #7903
• Engine:
  • [CORE] move InternalEngine.segmentStats() into abstract Engine #9728 (issue: #9727)
  • [ENGINE] Move more methods into abstract Engine #9717
  • [CORE] Move as much as possible into abstract Engine #9678
  • [ENGINE] Factor out settings updates from Engine #9625
  • [ENGINE] Close Engine immediately if a tragic event strikes. #9616 (issue: #9517)
  • [CORE] Refactor InternalEngine into abstract Engine and classes #9585
  • [ENGINE] Remove FlushType and make resources final in InternalEngine #9565
  • [ENGINE] Simplify Engine construction and ref counting #9211
  • [CORE] Fold engine into IndexShard #9181
  • Core: Don't acquire Engine's readLock in segmentsStats #8910 (issue: #8905)
  • [ENGINE] Remove engine related command classes #8900
  • Internal: allow InternalEngine to be stopped and started #8784 (issue: #8720)
  • [ENGINE] Flush IndexWriter to disk on close and shutdown #7563
• Geo:
  • Revert "[GEO] Update GeoPolygonFilter to handle ambiguous polygons" #9463 (issues: #5968, #9304, #9339, #9462)
  • Geo: Update GeoPolygonFilter to handle polygons crossing the dateline #9339 (issues: #5968, #8672, #9304)
  • Geo: GeoPolygonFilter not properly handling dateline and pole crossing #9171 (issue: #5968)
  • Geo: Removing unnecessary orientation enumerators #9036 (issues: #8978, #9035)
  • Geo: Add optional left/right parameter to GeoJSON #8978 (issue: #8764)
  • Geo: Feature/Fix for OGC compliant polygons failing with ambiguity #8762 (issue: #8672)
  • Geo: Fixes BoundingBox across complete longitudinal range #7340 (issue: #5218)
• Index Templates:
  • Provide template usage information on index creation #8646 (issue: #7421)
• Internal:
  • Core: fix typo when primary is not available to index a document (UnavailableShardsException) #10140
  • [SHARD] make assert less strict to ensure local node is not null #10076
  • Use provided cluster state for indices service validations #10014
  • some more simple fs cleanups. #9827
  • Removed 'Master' from names.txt #9698 (issue: #9687)
  • Internal: Introduce TimedPrioritizedRunnable base class to all commands that go into InternalClusterService.updateTasksExecutor #9671 (issues: #8077, #9354)
  • Update Groovy dependency to 2.4.0 #9520
  • Add beforeIndexAddedToCluster callback #9514
  • [CACHE] Use a smaller expected size when serializing query results #9485
  • [CACHE] Use correct number of bytes in query cache accounting #9479
  • Internal: add AliasesRequest interface to mark requests that manage aliases #9460
  • Internal: ClusterInfoService should wipe local cache upon unknown exceptions #9449
  • Internal: fix shard state tranport action names #9440 (issue: #7105)
  • Mappings: Remove includeExisting flag from adding ObjectMapper and FieldMapper listeners #9184
  • Internal: assert that we do not call blocking code from transport threads #9164
  • [EXEC] Remove reduced stack size #9158 (issue: #9135)
  • Search: parse terms filters on a single term as a term filter. #9014
  • Internal: remove IndexCloseListener & Store.OnCloseListener #9009 (issues: #8436, #8608)
  • Core: add ignore_idle_threads (default: true) to hot threads #8985 (issue: #8908)
  • Remove circular dependency between IndicesService and IndicesStore #8918
  • Remove some Internal* abstractions #8904
  • [HEALTH] Don't mark health as timed out if desired state is reached #8683
  • Internal: Use a 1024 byte minimum weight for filter cache entries #8304 (issues: #8249, #8268)
  • Immediately remove filter cache entries on cache clear #8289 (issue: #8285)
  • Internal: Make indexQueryParserService available from ParseContext #8252 (issue: #8248)
  • Internal: Allow to configure custom thread pools #8247
  • Internal: Expose concurrency_level setting on all caches #8112 (issue: #7836)
  • Resiliency: Be more conservative if index.version.created is not set #8018
  • Internal: split internal fetch request used within scroll and search #7870 (issues: #6933, #7319, #7856)
  • Internal: split internal free context request used after scroll and search #7856 (issues: #6933, #7319)
  • Internal: Clarify when a shard search request gets created to be only used locally #7855
  • Internal: Add a listener thread pool #7837
  • Internal: remove ForceSyncDirectory #7804
  • Internal: Check if from + size don't cause overflow and fail with a better error #7778 (issue: #7774)
  • Internal: make sure that internally generated percolate request re-uses the original headers and request context #7767
  • Internal: make sure that update internal requests share the same original headers and request context #7766
  • Internal: Nest original exception while creating NoShardAvailableActionException #7757 (issue: #7756)
  • Internal: make sure that all delete mapping internal requests share the same original headers and context #7736
  • Internal: make sure that original headers are used when executing search as part of put warmer #7711
• Java API:
  • Java API: package private getters to become public if there have corresponding public setters #9273
  • java: QueryBuilders cleanup (add and deprecate) #8667
  • Added utility method #8594
  • Java API: Enabled overriding the request headers in the clients #8258
  • Internal: add indices setter to IndicesRequest interface #7734
• Logging:
  • Store: renaming temp files should log errors of delete in trace #9933
  • Fix logging a RoutingNode object, log an object with a good .toString instead #9863
  • Logging: improve logging messages added in #9562 #9603
  • Logging: add logging around gateway shard allocation #9562
  • Logging: Log byte and doc size for slow merges #8855 (issue: #8853)
  • Core: separately log file deletions #8662 (issue: #8603)
  • Logging: Add log4j-extras dependency #8464 (issue: #7927)
  • Core: log how long IW.rollback took, and when MockFSDir starts its check index #8388
  • Logs: Change log level for mpercolate #8306
  • Logging: suppress long mapping logging during mapping updates (unless in TRACE) #7949
  • Add log4j-extras dependency #7947 (issue: #7927)
  • Core: Add EnhancedPatternLayout to logging.yml options #4991
• Mapping:
  • Mappings: Add enabled flag for _field_names to replace disabling through index=no #9893
  • [Mapper] Add ignore_missing option to <code>timestamp #9104 (issues: #8882, #9049)
  • Mapping: Include currentFieldName into ObjectMapper errors #9020
  • Mappings: Make lookup structures immutable. #7486
• More Like This:
  • MLT Query: Support for when all fields are deprecated #8067
  • MLT Query: use minimum should match more extensive syntax #7898
• NOT CLASSIFIED:
  • [LIFECYCLE] Add before/afterIndexShardDelete callback #10173
• Network:
  • Netty HTTP Transport: Change access modifiers to protected #9724
  • Transport: added a simple request tracer, logging incoming and outgoing requests #9286
  • Netty Transport: Add profiles to transport infos #9134
  • Netty: Support binding on multiple host/port pair #8098
  • Internal: Chunk direct buffer usage by networking layer #7811
  • Netty: Make sure channel closing never happens on i/o thread #7726
• Packaging:
  • Windows: makes elasticsearch.bat more friendly to automated processes #9160 (issue: #8913)
  • Shutdown: Add support for Ctrl-Close event on Windows platforms to grace... #8993
  • Packaging: Add java7/8 java-package paths to init script #8815 (issue: #7383)
  • Packaging: Check if proc file exists before calling sysctl #8793 (issue: #4978)
  • deb: add systemd service config for upcoming Jessie #8765 (issue: #8493)
  • bin/elasticsearch: add help, fix endless loop #8729 (issues: #2168, #7104)
  • Packaging: Introduce elasticsearch.in.bat (i.e. es.in for Windows) #8244 (issue: #8237)
  • add more checks to build_release.py #7913
  • Prevent init script from returning when the service isn't actually started #6909
  • Boostrap: Log startup exception to console if needed and to file as error #6581
  • Packaging: Use the new command line syntax in the init script #5033
  • Startup: Add ES_HOME to ES_INCLUDE search path #4958
• Parent/Child:
  • Parent/child: Reduce memory usage in top children query #8165
• Percolator:
  • Introduce index option named 'index.percolator.map_unmapped_fields_as_string' #9054 (issues: #9025, #9053)
• Plugins:
  • [PLUGINS] Always send current ES version when downloading plugins #10131
• Query DSL:
  • Add support for minimum_should_match to <code>simple_query_string #9864 (issue: #6449)
  • inner hits: Nested parent field should be resolved based on the parent inner hit definition #9251 (issue: #8153)
  • Raise an exception on an array of values being sent as the factor for a ... #9246 (issue: #7408)
  • inner_hits: Added another more compact syntax for inner hits. #8770
  • function_score: use query and filter together #8675 (issue: #8638)
  • Query: add option for analyzing wildcard/prefix to simple_query_strinq #8422 (issue: #787)
  • Query DSL: Expose max_determinized_states in regexp query, filter #8384 (issue: #8357)
  • FunctionScore: RandomScoreFunction now accepts long, as well a strings. #8311 (issue: #8267)
  • Be stricter parsing ids for ids query #7945 (issue: #7686)
  • Adding "min" score mode to parent-child queries #7771 (issue: #7603)
  • Query DSL: Throw parsing exception if terms filter or query has more than one field #5137 (issue: #5014)
  • Query DSL: Expose dist/pre/post options for SpanNotQuery #4452
• REST:
  • [HTTP] add option to only return simple exception messages #10117
  • Rest: expose master_timeout flag on <code>GET _template & HEAD _template #9688
  • Rest: Adding support of multi-index query parameters for _cluster/state #9295 (issue: #5229)
  • Adds parameters to API endpoint cluster put settings specification #8769
  • Change IndexPrimaryShardNotAllocatedException from 409 to 500 #7987 (issue: #7632)
  • Percolator: Added missing percolate API parameters to the rest spec #7173
• Recovery:
  • Recovery: add throttle stats #10097
  • Engine: update index buffer size during recovery and allow configuring version map size #10046 (issues: #6363, #6667)
  • Recovery: unify RecoveryState management to IndexShard and clean up semantics #9902 (issue: #9503)
  • [RECOVERY] Only iterate the files that we recovered from the commit #9761
  • Recovery: add a timeout to local mapping change check #9575
  • Recovery: flush immediately after a remote recovery finishes (unless there are ongoing ones) #9439
  • [RECOVERY] Release store lock before blocking on mapping updates #9102
  • [RECOVERY] Ensure shards are identical after recovery #8723
  • Recovery: be more resilient to partial network partitions #8720
  • [RECOVERY] Throw IndexShardClosedException if shard is closed #8648
  • Backport ShardRecoveryHandlerrefactorings to 1.x #8570
  • Recovery: refactor RecoveryTarget state management #8092 (issues: #7315, #7893)
• Scripting:
  • Scripting: Add String to the default whitelisted receivers #9837 (issue: #8866)
  • Scripting: Make script.groovy.sandbox.method_blacklist_patch truly append-only #9473
  • Scripting: Make groovy sandbox method blacklist dynamically additive #9470
  • Scripting: Add explainable script again #8665 (issues: #7245, #8561)
  • Scripting: Created a parameter parser to standardise script options #7977
  • Provide more context variables in update scripts #5724 (issues: #1607, #2230, #2231)
• Search:
  • Added nested scope to parse context that keeps track the current nested level during search request parsing #9692 (issue: #9305)
  • Search: Reduce memory usage during fetch source sub phase #8138
  • Internal: Don't let took be negative. #7968
  • Core: switch to Lucene QueryRescorer #7707 (issue: #6232)
• Settings:
  • Add 'http.publish_port' setting to the HTTP module #8807 (issue: #8137)
  • Settings: Validates bool values in yaml for node settings #8186 (issue: #8097)
• Snapshot/Restore:
  • [RESTORE] Refactor how restore cleans up files after snapshot was restored #9770
  • Snapshot/Restore: add support for changing index settings during restore... #9285 (issue: #7887)
  • Snapshot/Restore: Override write(byte[] b, int off, int len) in FilterOutputStream for better performance #8749 (issue: #8748)
  • Snapshot/Restore: Allow custom metadata to specify whether or not it sho... #7901 (issue: #7900)
• Stats:
  • Recovery: add total operations to the _recovery API #10042 (issue: #9368)
  • API: add pending tasks count to cluster health #9877
  • Stats : Add time in index throttle to stats. #7896 (issue: #7861)
• Store:
  • [INDICES] Schedule pending delete if index store delete fails #9856
  • [STORE] Improve safety when deleting files from the store #9801
  • [STORE] Add simple cache for StoreStats #9709 (issue: #9683)
  • [STORE] use Directory#fileLength() less during calculating checksums #9689
  • [STORE] Cache fileLength for fully written files #9683
  • [STORE] Synchronize operations that modify file mappings on DistributorDirectory #8408
  • [STORE] Cut over MetaDataStateFormat to NIO Path API #8297
  • [STORE] Remove special file handling from DistributorDirecotry #8276
  • [STORE] Fold two hashFile implemenation into one #7720
• Term Vectors:
  • Term Vectors: requests are now timed #9583
  • Term Vectors: support for realtime #7846
• Translog:
  • [CORE] Handle truncated translog gracefully #9797 (issue: #9699)
  • Internal: Write translog opSize twice #7735
• Upgrade:
  • Core: upgrade to Lucene 4.10.4 bugfix release #9960
  • Internal: Log when upgrade starts and stops #9229 (issue: #9227)
  • Dependencies: Upgrade netty to 3.10.0.Final #9132
  • upgrade to lucene 4.10.3 release #9100
  • Internal: Upgrade to Jackson 2.4.2 #7934 (issue: #7932)
  • Internal: Upgrade to Lucene 4.10.1 snapshot #7844 (issue: #7905)

Bug fixes

• Aggregations:
  • Aggregations: Be lenient when converting local to utc time in time zone roundings #10031 (issue: #10025)
  • Aggs: Fix rounding issue using date_histogram with <code>pre_zone_adjust_large_interval #9828 (issue: #8209)
  • Aggs: Fix rounding issues when using date_histogram and time zones #9790 (issues: #7673, #9491)
  • Aggregations: Prevent negative intervals in date_histogram #9690 (issue: #9634)
  • Aggs: Remove limitation on field access within aggs to the types provided in the search #9487
  • Aggs: nested agg needs to reset root doc between segments. #9441 (issues: #9436, #9437)
  • Aggs: Fix handling of multiple buckets being emitted for the same parent doc id in nested aggregation #9346 (issues: #8454, #9317)
  • Aggs: In reverse nested aggregation, fix handling of the same child doc id being processed multiple times. #9345 (issues: #9263, #9346)
  • Aggs: The parent filter of the nested aggregator isn't resolved correctly all the time #9335 (issue: #9280)
  • Aggs: post collection the children agg should also invoke that phase on its wrapped child aggs. #9291 (issue: #9271)
  • Aggs: Validate the aggregation order on unmapped terms in terms agg. #8952 (issue: #8946)
  • Aggregations: Fix date_histogram issues during a timezone DST switch #8655 (issue: #8339)
  • Fielddata: Fix iterator over global ordinals. #8627 (issue: #8580)
  • Aggregations: Fix geohash grid doc counts computation on multi-valued fields #8574 (issue: #8512)
  • Aggregations: Parser throws NullPointerException when Filter aggregation clause is empty #8527 (issue: #8438)
  • Aggregations: Fix geohash grid aggregation on multi-valued fields. #8513 (issue: #8507)
  • Aggregations: Change nested agg to execute in doc id order #8454
  • Aggregations: The children agg didn't take deleted document into account #8180
  • Aggregations: Fixes scripted metrics aggregation when used as a sub aggregation #8037 (issue: #8036)
  • Aggregations: Makes script params consistent with other APIs in scripted_metric #7969
  • Aggregations: Significant terms can throw error on index with deleted docs. #7960 (issue: #7951)
  • Aggregations: size property parsing inconsistent [ISSUE] #6061
• Aliases:
  • Aliases: Take into account _default_ mapping in index template when parsing alias filter in index templates. #8577 (issue: #8473)
• Allocation:
  • Allocation: Weight deltas must be absolute deltas #9149 (issue: #9023)
  • Allocation: Fix handling of dangling_timeout set to 0 and <code>auto_import_dangled true #8257
  • Allocation: Enable ClusterInfoService by default #8206
• Bulk:
  • Bulk API: Handle failed request when auto create index is disabled #8163 (issue: #8125)
  • Bulk API: Bulk operation can create duplicates on primary relocation #7729
• CAT API:
  • Stats: Fix NPE in /_cat/nodes #7815 (issue: #6297)
• Circuit Breakers:
  • Circuit Breaker: Only set breaker when stats are retrieved #7721
• Core:
  • Core: also throttle delete by query when merges fall behind #9986
  • Internal: promptly cleanup updateTask timeout handler #9621
  • Core: Disable auto gen id optimization #9468 (issue: #8788)
  • Core: Verify the index state of concrete indices after alias resolution #9057
  • Core: ignore_unavailable shouldn't ignore closed indices #9047 (issue: #7153)
  • Core: Terms filter lookup caching should cache values, not filters. #9027 (issues: #1, #2)
  • Core: Don't forcefully release IW lock (this can cause corruption) #8892 (issue: #8588)
  • Core: Hard wire utf-8 encoding, so unicode filenames work #8847
  • Gateway: GatewayService should register cluster state listener before checking for current state #8789
  • Core: Remove unnecessary index removal on index creation #8639
  • [GATEWAY] Remove _state directory if index has been deleted #8610
  • disable bloom filters #8572 (issues: #8564, #8571)
  • Core: Change default eager loading behaviour for nested fields and parent/child in bitset cache #8440
  • Core: Don't eagerly load NestedDocsFilter in bitset filter cache, because it is never used. #8414 (issue: #8394)
  • Internal: Extend refresh-mapping logic to the _default_ type #8413 (issue: #4760)
  • Internal: Translog leaks filehandles if it's corrupted or truncated #8372
  • Return 0 instead of -1 for unknown/non-exposed ramBytesUsed() #8291 (issue: #8239)
  • Resiliency: Don't catch FNF/NSF exception when reading metadata #8207
  • Internal: Don't handle FNF exceptions when reading snapshot #8086
• Dates:
  • DateMath: Use time zone when rounding. #9885 (issue: #9814)
  • Query DSL: Resolve now in date ranges in percolator and alias filters at search time instead of parse time #8534 (issue: #8474)
• Discovery:
  • Zen: Node receiving a cluster state with a wrong master node should reject and throw an error #9963
  • Discovery: check index uuid when merging incoming cluster state into the local one #9541 (issue: #9489)
  • Discovery: only retry join when other node is not (yet) a master #8979 (issue: #8972)
  • Discovery: Removed unnecessary DiscoveryService reference from LocalDiscovery #8415 (issue: #8539)
  • Discovery: Improve the lifecycle management of the join control thread in zen discovery. #8327
• Engine:
  • [ENGINE] Upgrade 3.x segments on engine startup #9899
  • Core: SerialMergeScheduler never triggers index throttling #8405
  • Internal: Add current translog ID to commit meta before closing #8245
• Geo:
  • [GEO] Fix validate_* merge policy for GeoPointFieldMapper [OPEN] #10165 (issue: #10164)
  • Geo: Correct bounding box logic for GeometryCollection type #9550 (issue: #9360)
  • Geo: Throw helpful exception for Polygons with holes outside of shell #9105 (issue: #9071)
  • Geo: GIS envelope validation #9091 (issues: #2544, #8672, #9067, #9079, #9080)
  • Geo: Fix for NPE enclosed in SearchParseException for a "geo_shape" filter or query #8785 (issue: #8432)
  • Geo: Fix for geohash neighbors when geohash length is even. #8529 (issue: #8526)
  • Geo: Fix for ArithmeticException[/ by zero] when parsing a polygon #8475 (issue: #8433)
  • geo sort: remove unnecessary code from geo distance builder #8338
  • Geo: Fix IndexedGeoBoundingBoxFilter to not modify the bits of other filters. #8325
• Index APIs:
  • Indices API: Fix to make GET Index API consistent with docs #9178 (issue: #9148)
  • Indices API: Fixed backward compatibility issue #8387 (issue: #8364)
  • Internal: Fix optimize behavior with 'force' and 'flush' flags. #7920 (issues: #7886, #7904)
  • Indices API: fixes GET Alias API backwards compatibility #7892 (issue: #7793)
• Indexed Scripts/Templates:
  • Scripting: ScriptService can deadlock entire nodes if script index is recovering #8901
  • Indexed Scripts/Templates: GetIndexedScript call can deadlock #8266
  • Make template params take arrays #8255
  • Indexed Scripts/Templates: Cleaned up various issues #7787 (issues: #7559, #7560, #7567, #7568, #7647)
• Internal:
  • Internal: Fix PageCacheRecycler's max page size computation. #10087 (issue: #10077)
  • Ensure that we don't pass negative timeInQueue to writeVLong #9662 (issue: #8077)
  • Snapshot status api: make sure headers are handed over to inner nodes request #9409
  • fix equality check of timevalue after serialization #9218
  • Internal: AdapterActionFuture should not set currentThread().interrupt() #9141 (issue: #9001)
  • Internal: PlainTransportFuture should not set currentThread().interrupt() #9001
  • Internal: IndexService - synchronize close to prevent race condition with shard creation #8557
  • Index API: BWC layer for GetIndex should not block in a listener #8496
  • Internal: Don't pass acceptdocs down to nonNestedDocsFilter in ParentsFilter #8463 (issue: #8461)
  • Indices API: Fix GET index API always running all features #8452
  • Internal: When corruption strikes, don't create exceptions with circular references #8331
  • Internal: Fixed bitset filter cache leftover in nested filter #8303
  • Internal: Cleanup non nested filter to not flip the FixedBitSet returned by the wrapped filter. #8232 (issue: #8227)
  • Internal: Dynamic changes to max_merge_count are now picked up by index throttling #8136 (issue: #8132)
  • Internal: Only schedule another refresh if refresh_interval is positive #8087 (issue: #8085)
  • Internal: Fix serialization of PendingClusterTask.timeInQueue. #8077
  • Internal: dangling indices import ignores aliases #8059
  • Fix serialization of short[] arays. #8025 (issue: #7845)
  • Internal: Make close() synchronized during node shutdown #7885
• Java API:
  • Mappings: Fix Get field mapping api with pretty flag #8806 (issue: #6552)
  • Settings: Ensure fields are overriden and not merged when using arrays #8381 (issue: #6887)
  • Fixing SearchRequestBuilder aggregations call to facets #8121 (issue: #8120)
  • Fixing copy/paste mistake in SearchRequest.extraSource's exception message #8118 (issue: #8117)
  • Aggregations: Added missing module registration in TransportClient for Significant Terms #7852 (issue: #7840)
• Logging:
  • [STORE] Add filename to corruption message #10110 (issue: #10062)
  • Transport: fix potential NPE in new tracer log if request timeout #9994 (issue: #9286)
  • Fix example in logging daily rotate configuration #8550 (issue: #8464)
  • Internal: Fix location information for loggers #8052 (issue: #5130)
• Mapping:
  • Fixes ignore_malformed behaviour for ip fields #10112
  • Mapping: Throw StrictDynamicMappingException exception #9445 (issue: #9444)
  • Mapping: Using default=null for _timestamp field creates a index loss on restart #9233 (issues: #9104, #9223)
  • Mapping: Reencode transformed result with same xcontent #8974 (issue: #8959)
  • Mapping: serialize doc values settings for _timestamp #8967 (issue: #8893)
  • mappings: update cluster state with type mapping also for failed indexin... #8692 (issue: #8650)
  • Mapping: Fix conflict when updating mapping with _all disabled #8426 (issues: #7377, #8423)
  • Mappings: Generate dynamic mappings for empty strings. #8329 (issue: #8198)
  • Mapping: Throw exception if null_value is set to null #7978 (issue: #7273)
  • Mapping: Posting a mapping with default analyzer fails #7902 (issue: #2716)
• More Like This:
  • MLT Query: Fix exclude with artificial documents #8679
• Network:
  • Netty: Add HTTP pipelining support #8299 (issue: #2665)
• Packaging:
  • Packaging: Add antlr and asm dependencies #9696
  • Packaging: Added quotes to allow spaces in installation path #8428 (issue: #8441)
  • Packaging: Move forbidden api signature files to dev-tools. #7921 (issue: #7917)
  • Packaging: Export JAVA_HOME in RPM init script #5434
• Parent/Child:
  • Parent/child: Fix concurrency issues of the _parent field data. #9030 (issue: #8396)
  • Parent/child: Fixed p/c filters not being able to be used in alias filters. #8649 (issues: #5916, #8628)
  • Bulk API: Missing parent routing causes NullPointerException #8506 (issue: #8365)
  • Parent/child: has_parent filter must take parent filter into account when executing the inner query/filter #8020 (issue: #7362)
• Percolator:
  • Percolator: Support encoded body as query string param consistently #9628
  • Percolator: Take filters from aliases into account #9590 (issue: #6241)
  • Percolator: Fixed bug when using multi percolate api with routing #9161 (issue: #6214)
• Plugins:
  • Plugins: Installation failed when directories are on different file systems #9011 (issue: #8999)
  • Plugins: NPE when plugins dir is inaccessible #8839 (issue: #8837)
  • Plugins: Support usage of ES_JAVA_OPTS in plugin commands #8288
  • PluginManager: Fix config path extraction from plugin handle #7935
• Query DSL:
  • Query DSL: Fix wrong error messages in MultiMatchQueryParser #8597
  • DateMath: Fix semantics of rounding with inclusive/exclusive ranges. #8556 (issue: #8424)
  • Query DSL: Fix NumberFormatException in Simple Query String Query #7876 (issue: #7875)
• REST:
  • REST: Add fielddata_fields to the REST spec #9399 (issues: #4492, #9398)
  • Search: Passing fieddata_fields as a non array causes OOM #8203
• Recovery:
  • Gateway: improve assertion at the end of shard recovery #10028
  • Recovery: synchronize RecoveryState.timer methods #9943
  • [RECOVERY] Don't recover from buggy version #9925 (issues: #7210, #9922)
  • Engine: close snapshots before recovery counter #9760 (issue: #9439)
  • [ENGINE] Fix deadlock problems when API flush and finish recovery happens concurrently #9648
  • [RECOVERY] Handle corruptions during recovery finalization #9619
  • Core: Mapping update task back references already closed index shard #9607
  • Recovery: update access time of ongoing recoveries #9506 (issue: #8720)
  • Recovery: cleaner interrupt handling during cancellation #9000
  • Internal: harden recovery for old segments #8399
  • Gateway: Prefer recovering the state file that uses the latest format. #8343
  • Recovery: change check for finished to a ref count check #8271 (issue: #8092)
  • Recovery: RecoveriesCollection.findRecoveryByShard should call recoveryStatus.tryIncRef before accessing fields #8231 (issue: #8092)
• Scripting:
  • Scripting: File scripts cache key to include language and prevent conflicts #10033
  • script_file does not work with _update [ISSUE] #10007
  • Internal: Avoid unnecessary utf8 conversion when creating ScriptDocValues for a string field. #9557 (issue: #6908)
  • Scripting: Disallow method pointer expressions in Groovy scripting #9509
  • Scripting: Make _score in groovy scripts comparable #9094 (issue: #8828)
  • Search: Function score and optional weight : avg score is wrong #9004 (issue: #8992)
  • Scripting: Return new lists on calls to getValues. #8591 (issue: #8576)
  • Scripting: Add score() back to AbstractSearchScript #8417 (issues: #8377, #8416)
  • Scripting: Clear the GroovyClassLoader cache before compiling #8062 (issues: #7658, #8073)
  • Scripting: Fix NPE in ScriptService when script file with no extension is deleted #7953 (issue: #7689)
• Search:
  • inner hits: Don't fail if an object is specified as a nested value instead of an array. #9743 (issue: #9723)
  • Query cache: Remove query-cache serialization optimization. #9500 (issue: #9294)
  • inner_hits: Make sure inner hits on has_parent query resolve hits properly #9384
  • Query cache: Queries are never cached when date math expressions are used (including exact dates) #9269 (issue: #9225)
  • Search: Make sure that named filters/ queries defined in a wrapped query/filters aren't lost #9166 (issue: #6871)
  • Search: Fix paging on strings sorted in ascending order. #9157 (issue: #9136)
  • Refactor term analysis for simple_query_string prefix queries #8435
• Settings:
  • Settings: Reset TieredMP settings only if the value actually changed #9497 (issue: #8890)
  • Settings: cluster.routing.allocation.disk.threshold_enabled accepts wrong values [ISSUE] #9309
  • Settings: Tab characters in YAML should throw an exception #8355 (issue: #8259)
  • Settings: LogConfigurator resolveConfig also reads .rpmnew or .bak files #7457
• Snapshot/Restore:
  • Snapshot/Restore: delete operation should ignore finalizing shards on no... #9981 (issue: #9924)
  • Snapshot/Restore: Allow deletion of snapshots with corrupted snapshot files #9569 (issue: #9534)
  • Snapshot/Restore: better handling of index deletion during snapshot #9418 (issue: #9024)
  • Snapshot/Restore: add validation of restored persistent settings #9051 (issue: #8830)
  • Snapshot/Restore: restore with wait_for_completion=true should wait for succesfully restored shards to get started #8545 (issue: #8340)
  • Snapshot/Restore: keep the last legacy checksums file at the end of rest... #8358 (issue: #8119)
  • Snapshot/Restore: restore of indices that are only partially available i... #8341 (issue: #8224)
  • Snapshot/Restore: Fix snapshotting of a single closed index #8047 (issue: #8046)
  • Snapshot/Restore: Make it possible to delete snapshots with missing metadata file #7981 (issue: #7980)
  • Snapshot/Restore: Make sure indices cannot be renamed into restored aliases #7918 (issue: #7915)
• Stats:
  • Translog: make sure stats's op count and size are in sync #10041
  • Nodes Stats: Fix open file descriptors count on Windows #9397 (issue: #1563)
  • Stats: Relax restrictions on filesystem size reporting in DiskUsage #9283 (issues: #9249, #9260)
  • Indices API: Fix wrong search stats groups #8950 (issue: #7644)
  • Stats: Fix Bloom filter ram usage calculation #8584 (issue: #8564)
  • Stats: _status with #shards >> queue capacity failing with BroadcastShardOperationFailedException [ISSUE] #7916
• Store:
  • Internal: only ack index store deletion on data nodes #9672 (issue: #9605)
  • [Store] Only fail recovery if files are inconsistent #8779
  • Store: Use Lucene checksums if segment version is >= 4.9.0 #8599 (issue: #8587)
  • Store: Calculate Alder32 Checksums for legacy files in Store#checkIntegrity #8407
• Suggesters:
  • return an HTTP error code when a suggest request failed instead of 200 #10104
  • Completion Suggester: Fix CompletionFieldMapper to correctly parse weight #8197 (issue: #8090)
• Translog:
  • Translog: stats fail to serialize size #10105
  • [GATEWAY] copy translog file if rename fails after retries. #9980
  • Translog: Never delete translog file from a snapshot #8917
• Tribe Node:
  • Tribe node: system properties and configuration settings must not be forwarded to tribe clients #9721 (issue: #9576)
  • Tribe node: remove closed indices from cluster state #9334
• Upgrade:
  • Upgrade: Fix version check in bytes to upgrade that spans major versions #9340

Regression

• Mapping:
  • Mapping: Mapping With a null Default Timestamp Causes NullPointerException on Merge #9205 (issue: #9204)

Docs

• Geo:
  • [GEO] fix docs for geo_point "validate" option [OPEN] #10168 (issue: #9539)
• Java API:
  • A getHits() was forgotten! #9320
  • [Doc] Java API: add search templates #8716 (issue: #7321)
  • Docs: Add java documentation for aggregations #8694
• NOT CLASSIFIED:
  • Need additional line in config file, to work tutorial's example. [ISSUE] #10074
  • fix typo #10045
  • Fix some docs n typos #10029
  • Documented default value for node_initial_primaries_recoveries #9955
  • [DOCS] added a note for the default shard_size value #9873
  • [DOCS] Default units for Geo Distance Agg is meters not km [ISSUE] #9812
  • [doc] Link mapper-attachment type documentation to its repo #9756 (issue: #101)
  • Clarify no master block [ISSUE] #9739
  • Add warning to settings documentation because setting number_of_replicas on a closed can lead to index beeing not openable again #9591 (issue: #9566)
  • Docs: Use the new experimental annotation. #9563
  • Testing classes don't support Parameterized tests? [ISSUE] #9423
  • Updates the command to add the repo #9261
  • Fix typo in sample json of keyword marker tokenfilter #9253
  • [TEST] Add missing docs and tests for '_cat/segments' #9179 (issue: #5856)
  • [Doc] Java API: add information on JBoss EAP #8766 (issue: #3445)
  • [DOCS] Document ActionNames class and related tests #8759
  • Fix error in documentation #8605
  • Update DiskThresholdDecider javadoc #8585 (issue: #6201)
  • Docs: Fix a typo in a javadoc comment #8575
  • Missing quote in the example #8565
  • randomizedtesting jar versions mismatch [ISSUE] #8450
  • Docs: note about confusing disk threshold settings #8437
  • Docs: document action.replication_type setting #8290
  • MLT Query: use ParseField#withAllDeprecated for percent_terms_to_match #8241
  • Docs: Clarify that index.routing.allocation.total_shards_per_node applies to replica and primary #8002
  • Docs: rolling upgrade process seems incorrect [ISSUE] #7973
  • null_value and dynamic_templates not working correctly [ISSUE] #7874
  • Docs : Filtering based on exact values not working [ISSUE] #7807
  • Docs: Document the most important changes to zen discovery. #7746
  • Inconsistent ceiling round up in range search for "lt" (less than) [ISSUE] #7203
• Packaging:
  • Packaging: Added a command to start elasticsearch on bootup. #8600

Tests

• Aggregations:
  • Adding tests for TimezoneRounding in 1.x #9796
• NOT CLASSIFIED:
  • [TEST] Add tests that triggers truncated files on recovery #10155
  • [TESTS] Make sure test end with ..Tests #9947 (issue: #9945)
  • [TEST] Work around URI encode limitations in RestClient #9946 (issue: #9769)
  • Tests: Add back tests.cluster #9916
  • [TEST] Fix NPE in ElasticsearchIntegrationTest if no indexer is provided #9909 (issue: #9907)
  • Tests: check node ports availability when using unicast discovery #9886
  • Tests: Use all found index files instead of static list for static bwc tests #9854
  • Tests: Remove thread leak filter #9843
  • [TEST] Don't reuse single node tests node instance across tests #9813
  • Kernel32LibraryTests creates JNA proxy classes [ISSUE] #9802
  • Tests: Remove global shared cluster #9781 (issue: #8854)
  • Test: Remove use of globalTempDir() and forbid it from future use. #9777
  • Testing: Added REST tests to check if expression scripting works #9750
  • Testing: Ensure RepeatOnException rule is available in test-jar #9675
  • [TEST] Introduce basic validation of our REST spec #9629
  • Tests: Add static indexes to fill out 0.20.x coverage. #9537
  • [TEST] support stashed values within property names in our REST tests #9533
  • Testing: Add test rule to repeat tests on binding exceptions #9527 (issue: #9010)
  • Tests: Add type-unrestricted version of field mapper getter to SearchContext #9490
  • Test: use local gateway by default #9476
  • Tests: Rename tests with the same name #9370
  • Internal: Change snapshot state for unreleased versions and add validation tests for constants #9228
  • Tests: Add upgrade step to static bwc tests #9207
  • Add 0.20.x index. #9146
  • Don't enable custom data_path for backwards compatibility tests #9119
  • Fix TransportNodesListShardStoreMetaData for custom data paths #9101
  • [TEST] Make sure we restart the suite cluster after each test failure #9015
  • [TEST] upgrade randomized runner to 2.1.11 #8930
  • [TEST] LoggingListener to restore the initial logger levels after any modification #8845 (issue: #8820)
  • Test: fix InternalEngineTest.testIndexWriterIFDInfoStream to pass in IntelliJ #8822
  • [TEST] Pass class level test logging to external nodes #8820 (issue: #8552)
  • [TEST] Add unit tests for DiskThresholdDecider settings #8816
  • [TEST] guarantee REST tests execution order #8745
  • [TEST] make sure rest tests info is printed for any @Rest annotated test #8680 (issue: #7795)
  • [TEST] split base settings in ClusterDiscoveryConfiguration between node and transport client #8653
  • [TEST] Extend unicast ports generation to support more concurrent clusters #8634
  • [TEST] Register data.path for all nodes on close in InternalTestCluster #8519
  • Tests: Skip bwc analysis tests if using turkish or azuri locale. #8492
  • Allow -SNAPSHOT versions to be parsed by Version.fromString #8379
  • [TEST] move ClusterDiscoveryConfiguration to org.elasticsearch.test.discovery #8337
  • Tests: Pass through locale and timezone to test runner, and print in repro command line. #8315
  • [TEST] Remove redundant call to setTemplateType() #8295
  • [TEST] initialize SUITE | GLOBAL scope cluster in a private random context #8250
  • [TEST] Scope.SUITE is not reproducible due to late cluster initialization #8220
  • [TEST] Add simple BWC tests that start from pre-build indices [ISSUE] #8065
  • Tests: dump all thread stacks on failure #8033
  • [TEST] fix CurrentTestFailedMarker to reset its state after each test #8008
  • [TEST] create client nodes using node.client: true #7911
  • [TEST] Use Shutdown API only if nodes are on 1.3.3 or newer to prevent shutdown problems #7910 (issue: #7885)
  • [TEST] add regular scroll REST test #7860
  • Internal: MulticastChannel should wait on receiver thread to stop during shutdown #7835
  • Tests: Add a more restrictive thread leaks filter #7825 (issue: #7833)
  • [TEST] move REST tests to their own test group #7795
  • [TEST] Only reset test cluster if a test actually failed #7775
  • [TEST] Improve HTTP support in TestClusters #7738
  • [TEST] Update REST client before each test in our REST tests #7737 (issues: #6734, #7723)
  • Test: always run CheckIndex after a test, and fail the test if it detects corruption #7730
  • Tests: always run CheckIndex on all shards created by the test [ISSUE] #7724
  • [TEST] Minor REST tests infra cleanup #7723
  • [TEST] Expose ability to provide http headers when sending requests in our REST tests #7710
  • Tools: Add script to grab ES version for BWC tests. #7653
  • Geo: [TEST] Adds tests for GeoUtils #7510
  • Benchmark: BenchmarkIntegrationTest failure fixed [ISSUE] #6094
  • Test: Integration tests for benchmark API. #6023 (issue: #6003)
  • Test: Start nodes using node.bench true to enable benchmark api #5920 (issue: #5910)
• Recovery:
  • [RECOVERY] Make recovery delay configurable #8739
• Snapshot/Restore:
  • Tests: Make sure snapshots created with old version of elasticsearch can... #8968

Not classified

• NOT CLASSIFIED:
  • Core: check only call rate limiter every minPauseCheckBytes during recovery [OPEN] #10185 (issue: #6018)
  • Build: Check that all packages contain the same jar files #9757
  • Rename Maven repository id #9733
  • [BUILD] Add marvel and license to the release script #9697
  • ignore intellij project/workspace files #9044
  • Switch to forbidden-apis 1.7 #8620
• Recovery:
  • Recovery: node shut down during the last phase of recovery needlessly fails shard [ISSUE] #9496