Breaking changes

• Benchmark:
  • Benchmark api: removed leftovers #10180
• Highlighting:
  • Wildcard field names in highlighting should only return fields that can be highlighted #11364 (issue: #9881)
• Network:
  • Remove unsafe options #10360
• Snapshot/Restore:
  • Fix FSRepository location configuration #11284 (issue: #11068)

Deprecations

• CRUD:
  • Deprecate async replication #10161 (issue: #10114)
• Java API:
  • Query DSL: deprecate BytesFilterBuilder in favour of WrapperFilterBuilder #11112 (issue: #10919)
  • Deprecate async replication #10642 (issues: #10114, #10171)
• More Like This:
  • Deprecate the More-Like-This API in favour of the MLT query #10982 (issue: #10736)
• Rivers:
  • Deprecate rivers #10503
  • Warning in documentation for deprecation of rivers #10423 (issue: #10345)
• Network:
  • Deprecated the thrift and memcached transports #10167 (issue: #10166)
• Parent/Child:
  • Deprecate the top_children query #11022 (issue: #6511)
• Plugins:
  • Plugins: deprecate addQuery methods that are going to be removed in 2.0 #11532 (issue: #11481)
• Scripting:
  • Deprecate Groovy sandbox and related settings #10478 (issue: #10156)
• Search:
  • Deprecate delete-by-query in client/transport/action APIs too #10239 (issue: #10082)
• Suggesters:
  • Deprecate filter option in PhraseSuggester collate #11445 (issue: #11195)

New features

• Logging:
  • Add ability to specify a SizeBasedTriggeringPolicy for log configuration #10373 (issue: #10371)
• Mapping:
  • Bring back numeric_resolution #10420 (issue: #10072)
• REST:
  • API: Add response filtering with filter_path parameter #10980 (issue: #7401)
• Recovery:
  • Synced flush backport #11417
  • Move index sealing terminology to synced flush #11336 (issues: #10032, #11179, #11251)
  • Seal indices for faster recovery #11179 (issue: #10032)
• Scripting:
  • Add support for fine-grained settings #10116 (issues: #10274, #6418)
• Search:
  • Validate API: provide more verbose explanation #10147 (issues: #1412, #88247)
• Settings:
  • Add ability to prompt for selected settings on startup #10918 (issue: #10838)
  • bootstrap.mlockall for Windows (VirtualLock) #10887 (issues: #8480, #9186, #9923)
• Shadow Replicas:
  • Allow shards on shared filesystems to be recovered on any node #10960 (issue: #10932)
• Stats:
  • Add field stats api #10523

Enhancements

• Allocation:
  • Async fetch of shard started and store during allocation #11262 (issues: #11101, #9502)
  • When using recover_on_any_node on shared filesystem, respect Deciders #11168
  • Early terminate if the cluster can't be rebalanced #9162
• Analysis:
  • Document and test custom analyzer position_offset_gap #10934 (issue: #1812)
• CAT API:
  • Add node id to _cat apis #10246 (issue: #10226)
• Cluster:
  • Make routing_nodes an independent metric option in cluster state api #10412 (issue: #10352)
  • Add METADATA_READ and METADATA_WRITE blocks #9203 (issues: #10521, #10522, #2833, #3703, #5855, #5876, #8102)
• Core:
  • Add node setting to send SegmentInfos debug output to System.out #11546
  • Reduce shard inactivity timeout to 5m #11479 (issues: #11179, #11336)
  • Fail shard if search execution uncovers corruption #11440 (issue: #11419)
  • Acquire IndexWriter's write.lock lock before shard deletion #11127 (issue: #11097)
  • Refactor TransportShardReplicationOperationAction #10749 (issue: #10032)
  • Make getFileStore a bit more defensive #10696
  • Ref count write operations on IndexShard #10610
  • Refresh if many deletes in a row use up too much version map RAM #10312 (issue: #7052)
• Discovery:
  • Prevent over allocation for multicast ping request #10896
  • Unicast Ping should close temporary connections after returning ping results #10849
• Engine:
  • Remove reflection call to waitForMerges #10102
• Geo:
  • Add merge conflicts to GeoShapeFieldMapper #10533 (issues: #10513, #10514)
  • Coordinates can contain more than two elements (x,y) in GeoJSON parser. #9542 (issue: #9540)
• Index APIs:
  • Add check to MetaData#concreteIndices to prevent NPE #10342 (issue: #10339)
• Indexed Scripts/Templates:
  • Indexed scripts/templates: return response body when script is not found #10396 (issue: #7325)
  • Indexed Scripts/Templates: Return error message on 404 #7335 (issue: #7325)
• Internal:
  • Removed generic types from ContextAndHeaderHolder and HasHeaders#putHeader() #11231
  • Wait forever (or one day) for indices to close #10833 (issue: #10680)
  • Don't create a new BigArrays instance for every call of withCircuitBreaking #10800 (issue: #10798)
  • Change BigArrays to not extend AbstractComponent #10798
  • CommitStats doesn't need to allow for null values in commit user data #10774 (issue: #10687)
  • Add fairness option to KeyedLock. #10703
  • Fix string comparisons #10204
  • Remove unsafe field in BytesStreamInput #10157
  • Fix errors reported by error-prone #9817
  • Reduce the size of the search thread pool. #9165 (issue: #9135)
• Java API:
  • Unify SearchResponse and BroadcastOperationResponse code around shards header #11064
  • Remove duplicated consistency level and replication type setters #10188
• Logging:
  • Display low disk watermark to be consistent with documentation #11313 (issue: #10588)
  • Add index name to log statements when settings update fails #11124
  • Add logging of slow cluster state tasks #10907 (issue: #10874)
  • Log sending translog operation batches to nodes #10544
  • Log only a summary line of filesystem detail for all path.data on node startup #10527 (issue: #10502)
  • Add INFO logging saying whether each path.data is on an SSD #10502
  • Use static logger name in Engine.java #10497
  • Miscellaneous additional logging and cleanups #10376
  • Change logging to warning to match pattern #9593
  • Reduce apache (cloud-aws) logging when rootLogger is DEBUG #8856
• Network:
  • Default value for socket reuse should not be null #11255
  • Make Netty exceptionCaught method protected #10464
  • Remove content thread safe from REST layer #10429
  • Add getter for channel in NettyTransportChannel #10319
  • Schedule transport ping interval #10189
  • Return useful error message on potential HTTP connect to Transport port #10108 (issue: #2139)
• Packaging:
  • Export hostname as environment variable for plugin manager #11399 (issues: #10902, #9474)
  • Exclude jackson-databind dependency #10924
  • Add common SystemD file for RPM/DEB package #10725
  • Standardization of packages structure and install #10595 (issue: #10330)
  • Add properties files to configure startup and installation scripts #10330
  • service.bat file should explicitly use the Windows find command. #9532
  • Export the hostname as environment variable #9474 (issue: #8470)
• Plugins:
  • Use of default CONF_DIR/CONF_FILE in plugin install #10721 (issues: #10673, #7946)
  • Always send current ES version when downloading plugins #10131
• Query DSL:
  • Function score: Add default to field_value_factor #10845 (issue: #10841)
• REST:
  • Unify query_string parameters parsing #11057
  • HttpServer: Support relative plugin paths #10975 (issue: #10958)
  • Remove global source parameter from individual APIs in REST spec #10863
  • Better detection of CBOR #10026 (issue: #7640)
  • Remove indices_boost URL param #9244 (issue: #6281)
• Recovery:
  • Reduce cluster update reroutes with async fetch #11421
  • Check if the index can be opened and is not corrupted on state listing #11269 (issue: #11226)
  • Allow to recover into a folder containing a corrupted shard #10558
  • Only cancel recovery when primary completed relocation #10218
• Scripting:
  • Allow plugins to define custom operations that they use scripts for #10419 (issue: #10347)
• Search:
  • Single value numeric queries shouldn't be handled by NumericRangeQuery #10648 (issue: #10646)
• Settings:
  • Make prompt placeholders consistent with existing placeholders #11514 (issues: #10918, #11455)
• Shadow Replicas:
  • Implement retries for ShadowEngine creation #10688 (issue: #10637)
  • Allow rebalancing primary shards on shared filesystems #10585 (issue: #10469)
• Snapshot/Restore:
  • Add support for applying setting filters when displaying repository settings (Backport to 1.6) #11431 (issue: #11265)
  • Check that reading indices is allowed before creating their snapshots #11133
  • Don't throw an exception if repositories are unregistered with * #11113
  • Improve the error message when attempting to snapshot a closed index #10608 (issue: #10579)
  • AbstractBlobContainer.deleteByPrefix() should not list all blobs #10366 (issue: #10344)
  • Batching of snapshot state updates #10295
  • Expand wildcards in snapshot #9903 (issue: #6097)
• Stats:
  • Add CommitStats to supply information about the current commit point #10687
• Store:
  • Consolidate directory lock obtain code #11390
• Suggesters:
  • Phrase Suggester Collate Enhancements #10710 (issue: #9377)
• Translog:
  • Use buffered translog type also when sync is set to 0 #10993
  • Remove useless random translog directory selection #10589
• Upgrade:
  • Upgrade Jackson to 2.5.3 #11307
  • Upgrade to Netty 3.10.3 #11304
  • Update forbiddenapis to version 1.8 #10555
  • Upgrade to Jackson 2.5.1 #10210
  • Upgrade to Jackson 2.5.1 #10134
• Upgrade API:
  • Refactor upgrade API to use transport and write minimum compatible version that the index was upgraded to #11333 (issues: #11072, #11095)
  • Add upgrade_only_ancient_segments option to upgrade API #10540 (issue: #10213)

Bug fixes

• Aggregations:
  • Allow aggregations_binary to build and parse #11473 (issue: #11457)
  • Fix geo bounds aggregation when longitude is 0 #11090 (issue: #11085)
  • Fixes Infinite values return from geo_bounds with non-zero bucket-ordinals #10917 (issue: #10804)
  • Fix _as_string output to only show when format specified #10571 (issue: #10284)
• Allocation:
  • GatewayAllocator: reset rerouting flag after error #11519 (issue: #11421)
• Analysis:
  • Custom analyzer names and aliases must not start with _ #11303 (issue: #9596)
  • Fix tokenizer settings in SynonymTokenFilterFactory #10489
• Bulk:
  • Allow null values in the bulk action/metadata line parameters #11459 (issue: #11458)
  • Throw exception if unrecognized parameter in bulk action/metadata line #11331 (issue: #10977)
  • _default_ mapping should be picked up from index template during auto create index from bulk API #10762 (issue: #10609)
  • Removed duplicate timeout param #10205
• CAT API:
  • _cat/nodes: Thread null handling through stats and info #9938 (issue: #6297)
• CRUD:
  • detect_noop now understands null as a valid value #11210 (issue: #11208)
  • The parent option on update request should be used for upsert only. #9612 (issue: #4538)
• Cache:
  • Don't use bitset cache for children filters. #10663 (issues: #10629, #10662)
• Cluster:
  • ClusterHealth shouldn't fail with "unexpected failure" if master steps down while waiting for events #11493
• Core:
  • Improve exception message when shard has a partial commit (segments_N file) due to prior disk full #11539 (issue: #11249)
  • Use System.nanoTime for elapsed time #11058
  • Lucene merges should run on the target shard during recovery #10463 (issue: #9226)
  • Increase default rate limiting for snapshot, restore and recovery to 40 MB/sec #10185 (issue: #6018)
• Dates:
  • Query string time zone not working #10883 (issue: #10477)
• Engine:
  • Ignore 3x segment upgrade if unneeded #11383 (issue: #11361)
  • Sync translog before closing engine #10484
• Geo:
  • Fix OOM for high precision exotic shapes #10652 (issues: #10583, #2361, #9860)
  • Correct ShapeBuilder coordinate parser to ignore values in 3rd+ dimension #10539 (issue: #10510)
  • Fix hole intersection at tangential coordinate #10332 (issue: #9511)
• Highlighting:
  • Use analyzer set on the field #8943 (issue: #8757)
• Index APIs:
  • Remove expansion of empty index arguments in RoutingTable #10148 (issue: #9081)
• Indexed Scripts/Templates:
  • Fix updating templates. #10526 (issue: #10397)
• Inner Hits:
  • Fix multi level parent/child bug #11199
  • Make sure size=0 works on the inner_hits level. #10388 (issue: #10358)
  • Make sure inner hits also works for nested fields defined in object field #10353 (issue: #10334)
  • Fix bug where parse error is thrown if a inner filter is used in a nested filter/query. #10309 (issue: #10308)
  • Fix nested stored field support. #10235 (issue: #9766)
• Internal:
  • Make JNA optional for tests and move classes to bootstrap package #11378 (issue: #11360)
  • Fix CompressedString.equals. #11233
  • ThreadPool: make sure no leaking threads are left behind in case of initialization failure #11061 (issue: #9107)
  • Propagate headers & contexts to sub-requests #11060 (issue: #10979)
  • Fix NPE in PendingDelete#toString #11032
  • Search: FielddataTermsFilter equality is based on hash codes  #10728
  • Ensure that explanation descriptions are not null on serialization #10689 (issue: #10399)
  • Fix possible NPE in InternalClusterService$NotifyTimeout, the future field is set from a different thread #10630 (issue: #3)
  • Allow ActionListener to be called on the network thread #10573 (issue: #10402)
  • Add missing hashCode method to RecoveryState#File #10501
  • Don't try to send a mapping refresh if there is no master #10311 (issue: #10283)
  • Close all resources if doStart fails #9898
• Java API:
  • Add missing rewrite parameter to FuzzyQueryBuilder #11139 (issue: #11130)
  • Ensure netty I/O thread is not blocked in TransportClient #10644
  • toString for SearchRequestBuilder and CountRequestBuilder #9944 (issues: #5555, #5576)
• Logging:
  • Add filename to corruption message #10110 (issue: #10062)
  • Fix potential NPE in new tracer log if request timeout #9994 (issue: #9286)
• Mapping:
  • numeric_resolution should only apply to dates provided as numbers. #11002 (issue: #10995)
  • Unneccesary mapping refreshes caused by unordered fielddata settings #10370 (issue: #10318)
  • Fixed an equality check in StringFieldMapper. #10359 (issue: #10357)
  • Fix _field_names to be disabled on pre 1.3.0 indexes #10268 (issue: #9893)
  • Fix ignore_malformed behaviour for ip fields #10112
  • Update dynamic fields in mapping on master even if parsing fails for the rest of the doc #9874 (issue: #9851)
• Network:
  • Transport: fix racing condition in timeout handling #10220 (issue: #10187)
• Parent/Child:
  • Fix 2 bugs in children agg #10263 (issues: #10158, #9544, #9958)
• Percolator:
  • Prevent PercolateResponse from serializing negative VLong #11138
  • Fix wrong use of currentFieldName outside of a parsing loop #10307
• Plugins:
  • Only load a plugin once from the classpath #11301
  • HTTP: Ensure url path expansion only works inside of plugins #10815
• Query DSL:
  • Score value is 0 in _explanation with random_score query  #10742
  • Avoid NPE during query parsing #10333
  • Function score: Apply min_score to sub query score if no function provided #10326 (issue: #10253)
  • function_score: undo "Remove explanation of query score from functions" #9826
• REST:
  • fielddata_fields query string parameter was ignored. #11368 (issue: #11025)
  • REST spec for termvector missing paths on 1.x branch #11087 (issue: #11086)
• Recovery:
  • Don't write recoveryType twice #11347 (issues: #11179, #11335)
  • Decrement reference even if IndexShard#postRecovery barfs #11201
  • Fail recovery if retry recovery if resetRecovery fails #11149
  • Refactor state format to use incremental state IDs #10316
  • RecoveryState.File.toXContent reports file length as recovered bytes #10310
  • Fail shard when index service/mappings fails to instantiate #10283
• Scripting:
  • Allow script language to be null when parsing #10976 (issue: #10926)
• Search:
  • Release search contexts after failed dfs or query phase for dfs queries #11434 (issue: #11400)
  • Don't truncate TopDocs after rescoring #11342 (issues: #11277, #7707)
• Search Templates:
  • Fix JSON encoding for Mustache templates. #10820 (issue: #5473)
• Settings:
  • Prevent changing the number of replicas on a closed index #11410 (issue: #9566)
  • Read configuration file with .yaml suffix #10909 (issue: #9706)
  • Validate number_of_shards/_replicas without index setting prefix #10701 (issue: #10693)
  • Fix handling of IndicesOptions in update settings REST API #10030
• Snapshot/Restore:
  • Sync up snapshot shard status on a master restart #11450 (issue: #11314)
  • Fix check for locations in a repository path #11426
  • Don't reuse source index UUID on restore #10367
  • Separate repository registration #10354 (issue: #10351)
  • Automatically add "index." prefix to the settings are changed on restore if the prefix is missing #10269 (issue: #10133)
• Store:
  • Fix stream version check in ShardActiveRequest #11407
  • Ensure we mark store as corrupted if we fail to read the segments info #11230 (issue: #11226)
  • Fix NPE when checking for active shards before deletion #11110 (issue: #10172)
  • Shard not deleted after relocation if relocated shard is still in post recovery #10172 (issue: #10018)
• Suggesters:
  • Ensure empty string completion inputs are not indexed #11158 (issue: #10987)
  • Ensure collate option in PhraseSuggester only collates on local shard #11156 (issue: #9377)
  • Make GeoContext mapping idempotent #10602 (issues: #10581, #8937)
  • Return an HTTP error code when a suggest request failed instead of 200 #10104
• Term Vectors:
  • ShardTermVectorsService calls docFreq() on unpositioned TermsEnum #10660
• Translog:
  • Acquire index writer lock before renaming translog file #11396
  • Fail #snapshot if translog is closed #10809 (issue: #10807)
• Upgrade API:
  • Use the smallest version rather than the default version #11475 (issue: #11474)