Elasticsearch 2.0.0-beta1

Release Notes

IMPORTANT: This is a beta release and is intended for testing purposes only. There is no guarantee that Elasticsearch 2.0.0-beta1 will be compatible with Elasticsearch 2.0.0 GA.

DO NOT DEPLOY IN PRODUCTION

This changes list is divided into two:

Changes first released in 2.0.0-beta1

Breaking changes

  • Aggregations:
    • Removed unused factor parameter in DateHistogramBuilder #12850 (issue: #6490)
    • Aggregation: Removed Old Script Java API from metrics aggregations #12236
    • Change the default min_doc_count to 0 on histograms. #10904
    • Speed up include/exclude in terms aggregations with regexps, using Lucene regular expressions #10418 (issues: #7526, #9848)
    • Clean up time zone options for date_histogram #9637 (issue: #9062)
    • Add offset to date_histogram, replacing pre_offsest and post_offset #9597 (issue: #9062)
    • Facets: Removal from master. #7337
  • Allocation:
    • Remove old 0.90 shard allocator #10889
  • Analysis:
    • Add multi-valued text support to the analyzer API #10847 (issue: #3023)
    • Term positions in analyze API should start at 1, not 0 #10771
  • Bulk:
  • CAT API:
    • Filtered out non data-nodes in relevant cat api #9287
    • Default _cat APIs to verbose #8927 (issue: #8922)
  • CRUD:
    • Remove core delete-by-query implementation #10859 (issue: #10067)
    • Remove async replication #10171 (issue: #10114)
    • Remove async replication from the docs and REST spec #10162 (issue: #10114)
    • Delete api: remove broadcast delete if routing is missing when required #10136 (issue: #9123)
  • Cluster:
  • Core:
    • Remove MergeScheduler pluggability #11585
    • Don't allow indices containing too-old segments to be opened #11072 (issues: #10215, #11095)
    • Remove Restart API and remove Node#stop() #9921 (issue: #9841)
    • Remove component settings from AbstractComponent #9919
    • Refactor settings filtering, adding regex support #9748 (issue: #6295)
    • Cut over to Path API for file deletion #8366
    • Switch to murmurhash3 to route documents to shards. #7954
  • Discovery:
    • Default to unicast discovery, with default host list of 127.0.0.1, [::1] #12999 (issue: #12993)
  • Exceptions:
    • Generify Index and Shard exceptions #12023
    • Ban java serialization #11910
  • Highlighting:
    • Remove XPostingsHighlighter in favour of Lucene's PostingsHighlighter #11077 (issue: #10625)
    • require_field_match now defaults to true #11067 (issue: #10627)
  • Index Templates:
    • Remove file based index templates #11052 (issue: #10870)
    • GET templates doesn't honor the flat_settings parameter. #6672 (issue: #6671)
  • Internal:
    • Flatten SearchService and clean up build-in registration #12807 (issue: #12783)
    • Consolidate shard level abstractions #11847
    • Bake in TieredMergePolicy #11588
    • Remove Translog interface #10988
    • Remove InternalNode class and use Node directly #9844
    • Remove OperationRouting abstraction #9085
    • Drop support for state written pre 0.90 #8850
    • Remove some more bwc code #8778
    • Remove optional original indices #8777 (issue: #7406)
    • Remove runtime version checks #8768
    • Remove NoneGateway, NoneGatewayAllocator, & NoneGatewayModule #8537
  • Java API:
    • Java api: remove execution from TermsQueryBuilder as it has no effect #12884
    • Enhancement/terms lookup fixes #12870
    • Centralize admin implementations and action execution #10955
    • Automatically thread client based action listeners #10940
    • Remove redundant BytesQueryBuilder in favour of using WrapperQueryBuilder internally #10919
    • Aggregations: Clean up response API for Aggregations #9221
  • Logging:
    • Truncate log messages at 10,000 characters by default #11050
  • Mapping:
    • Disallow type names to start with dots for new indices except for .percolator #12561 (issue: #12560)
    • Remove ability to configure _index #12356 (issues: #12027, #12329)
    • Enforce field names do not contain dot #12068
    • Restrict fields with the same name in different types to have the same core settings #11812 (issue: #8871)
    • Lockdown _timestamp field #11794 (issues: #6730, #8143)
    • Remove the compress/compress_threshold options of the BinaryFieldMapper #11280
    • Remove ability to set the value of meta fields inside _source #11074 (issue: #11051)
    • Remove file based default mappings #10870 (issue: #10620)
    • Validate dynamic mappings updates on the master node. #10634 (issues: #8650, #8688)
    • Remove delete mapping API #10231 (issue: #8877)
    • Lock down _ttl field #9914 (issue: #8143)
    • Lock down _size field #9913 (issue: #8143)
    • Lock down _field_names field #9912 (issue: #8143)
    • Remove _boost field #9897 (issue: #8875)
    • Lock down _routing field #9895 (issue: #8143)
    • Lock down _index field #9870 (issue: #8143)
    • Lock down _type field #9869 (issue: #8143)
    • Lock down _id field #9842 (issue: #8143)
    • Lock down _uid field #9836 (issue: #8143)
    • Remove the ability to have custom per-field postings and doc values formats. #9741 (issue: #8746)
    • Remove support for new indexes using path setting in object/nested fields or index_name in any field #9570 (issue: #6677)
    • Remove type prefix support from field names in queries #9492 (issue: #8872)
    • Remove index_analyzer setting to simplify analyzer logic #9451 (issue: #9371)
    • Remove type level default analyzers #9430 (issues: #8874, #9365)
    • Remove _analyzer field #9381 (issue: #9279)
    • Remove fieldSearchAnalyzer and fieldSearchQuoteAnalyzer from MapperService. #9262
    • Remove allow_type_wrapper setting #9185
    • Add doc values support to boolean fields. #7961 (issues: #4678, #7851)
  • More Like This:
  • Network:
  • Packaging:
    • Remove Environment.homeFile() #12351
    • Startup: Remove getopt parsing in shell script, use java CLITool #12165
  • Parent/Child:
  • Plugins:
    • Simplify Plugin API for constructing modules #12952
    • Use 'name' from plugin descriptor file to determine plugin name #12775 (issue: #12715)
    • make java.version mandatory for jvm plugins #12424
    • Adapt pluginmanager to the new world #12408
    • Refactor pluginservice #12367 (issue: #11917)
    • CLITool: Port PluginManager to use CLITool #12290
    • One single (global) way to register custom query parsers #11481
  • Query DSL:
    • Don't allow fuzziness specified as a % and require edits [0,2] instead #12229 (issue: #10638)
    • Remove filter parsers. #10985
    • Deprecate the limit filter. #10532
    • Remove fuzzy_like_this query #10391
    • Remove numeric_range filter #7242 (issues: #4034, #7108)
  • REST:
    • Cluster state: return routing_nodes only when requested through specific flag #10486 (issue: #10412)
    • Remove jsonp support and associated tests #9242 (issue: #9108)
    • Remove status code from main action / #8865
    • Add all meta fields to the top level json document in search response #8131
    • Removed Index Status API #6062 (issue: #4854)
  • Recovery:
    • Decouple recoveries from engine flush #10624
  • Rivers:
  • Scripting:
    • Remove deprecated script APIs #11619
    • Add script type and script name to error messages #11449 (issue: #6653)
    • Added a new script construct #10649
    • Remove deprecated methods from ScriptService #10476
    • Remove support for script.disable_dynamic setting #10286 (issue: #10116)
    • Removed deprecated script parameter names #9815
  • Search:
    • Cut over to the Lucene filter cache #10897
    • Remove terms filter lookup cache. #9056
    • Fix script fields to be returned as a multivalued field when they produce a list #8592 (issue: #3015)
    • Remove partial fields #8133
  • Settings:
    • Do not permit multiple settings files #13043 (issue: #13042)
    • change CORS allow origin default to allow no origins #11890 (issue: #11169)
    • Require units for time and byte-sized settings, take 2 #11437 (issues: #10888, #7616, #7633)
    • Remove mapping.date.round_ceil setting for date math parsing #8889 (issues: #8556, #8598)
  • Shadow Replicas:
    • Remove the node.enable_custom_paths setting #12837 (issue: #12776)
    • Add path.shared_data, change index.data_path to be relative #11065
  • Snapshot/Restore:
  • Stats:
  • Store:
    • Simplify IndexStore and friends #10773
    • Remove memory/ ram store #8536
  • Term Vectors:
    • More consistent naming for term vector[s] #8484

Deprecations

  • Fielddata:
    • Remove non-default fielddata formats. #11669
  • Settings:
    • Remove index.fail_on_merge_failure #10088 (issue: #10084)
    • Remove dangling indices settings, always import it #10016

New features

  • Aggregations:
    • Add HDRHistogram as an option in percentiles and percentile_ranks aggregations #12362 (issue: #8324)
    • Aggregations: Adds other bucket to filters aggregation #11948 (issue: #11289)
    • Aggregations: Pipeline Aggregation to filter buckets based on a script #11941
    • Adds cumulative sum aggregation #11825
    • Allow users to perform simple arithmetic operations on histogram aggregations #11601 (issue: #11029)
    • Aggregations: add serial differencing pipeline aggregation #11196 (issue: #10190)
    • Add Holt-Winters to moving_avg aggregation #11043
    • Make it possible to configure missing values. #11042 (issue: #5324)
    • Adding Sum Bucket Aggregation #11013 (issue: #11007)
    • Adding Average Bucket Aggregation #11010 (issue: #11006)
    • min_bucket aggregation #10900 (issue: #9999)
    • Pipeline aggregations: Ability to perform computations on aggregations #10568 (issues: #10000, #10002, #9293, #9876)
    • Sampler aggregation #10221 (issue: #8108)
  • Allocation:
    • Cancel replica recovery on another sync option copy found #12421
  • CAT API:
  • Core:
    • Expose auto-IO-throttle from Lucene's ConcurrentMergeScheduler #9243 (issue: #9133)
  • Dates:
    • Added epoch date formats to configure parsing of unix dates #11453 (issues: #10971, #5328)
  • Index APIs:
  • Logging:
  • More Like This:
    • Add an unlike parameter #8674
  • Plugin Delete By Query:
    • Add delete-by-query plugin #11516
  • Plugins:
    • add list parse methods to XContentParser #10455
    • Migration advisory plugin #10214
  • Query DSL:
    • Query DSL: Add filter clauses to bool queries. #11142
    • Add span within/containing queries. #10913
  • REST:
    • Render REST errors in a structural way #10643 (issue: #3303)
  • Scripting:
    • Add Multi-Valued Field Methods to Expressions #11105
    • Add Groovy as a scripting language, switching default from Mvel -> Groovy #6106
  • Search Templates:
    • Search Templates: Adds API endpoint to render search templates as a response #11570 (issue: #6821)
  • Settings:
    • Add checksum option for index.shard.check_on_startup #9183
  • Stats:
    • Add script compilation stats #12733 (issue: #12673)
    • Add OS name to _nodes and _cluster/nodes #11807
    • Add an API to locate unrecovered shards and their state #11545 (issue: #10952)
    • Cluster Health: Add wait time for pending task and recovery percentage #11393 (issue: #10805)
  • Store:
    • Add best_compression option for indices #8863
  • Term Vectors:
    • Return term vectors as part of the search response #10729 (issue: #10823)
    • Support terms filtering #9561

Enhancements

  • Aggregations:
    • Make ValueParser.DateMath aware of timezone setting #12886 (issue: #12278)
    • Fix setting timezone on default DateTime formatter #12581 (issue: #12531)
    • Aggregations: Add better validation of moving_avg model settings #12280
    • Aggregations: Adds a new GapPolicy - NONE #11951
    • Aggregations: Makes ValueFormat and ValueFormatter never null #11943 (issue: #10594)
    • Add cost minimizer to tune moving_avg parameters #11881
    • Aggregations: moving_avg model parser should accept any numeric #11778 (issue: #11487)
    • Renaming reducers to Pipeline Aggregators #11275
    • Improve include/exclude clause list speed and scalability #11188 (issue: #11176)
    • Remove pointless term frequency lookups. #11094 (issue: #11093)
    • Rename Moving Average models to their "common" names #10964
    • Derivative Aggregation x-axis units normalisation #10898
    • Added context for significant_terms scoring #10633 (issue: #10613)
    • Removed aggregations from ReduceContext #10509
    • Format bucket key_as_string in date_histogram according to time_zone #9744 (issue: #9710)
    • Refactor aggregations to use lucene5-style collectors. #9544 (issues: #6477, #9098)
    • Add offset option to histogram aggregation #9505 (issue: #9417)
    • Unify histogram implementations #9446
    • Internal simplifications. #9097
    • Do not sort histogram buckets on shards. #8797
    • Added getProperty method to Aggregations #8421
    • Meta data support with each aggregation request/response #8279 (issue: #6465)
    • Buckets can now be serialized outside of an Aggregation #8113 (issue: #8110)
  • Aliases:
    • Don't require fields in alias filters to exist in the mapping #12150
    • Parse aliases at search time and never cache parsed alias filters #11930 (issue: #10485)
  • Allocation:
    • Add expectedShardSize to ShardRouting and use it in path.data allocation #12947 (issue: #11271)
    • Make RoutingNodes read-only by default #12690
    • Avoid extra reroutes of delayed shards in RoutingService #12678 (issues: #12456, #12515, #12532)
    • Reroute shards when a node goes under disk watermarks #12452 (issue: #12422)
    • No need to find replica copy when index is created #12435
    • Adapt IndicesClusterStateService to use allocation ids #12397 (issues: #12242, #12387)
    • Simplify handling of ignored unassigned shards #12339
    • Initial Refactor Gateway Allocator #12335
    • Use recently added allocation ids for shard started/failed #12299 (issue: #12242)
    • Unique allocation id #12242
    • Allow shards to be allocated if leftover shard from different index exists #12237 (issue: #10677)
    • Simplify assignToNode to only do initializing #12235
    • Async fetch of shard started and store during allocation #11101 (issue: #9502)
    • Verify shards index UUID when fetching started shards #10200
    • Add rebalance enabled allocation decider #8190 (issue: #7288)
  • Bulk:
    • Add support for retrieving fields in bulk updates #12114 (issue: #11527)
  • CAT API:
    • Add scroll stats to cat API #12331 (issue: #12330)
    • Add option to _cat/indices to return index creation date #11524 #11688 (issue: #11524)
  • CRUD:
    • Remove backward compatibility layer introduced in #6149 #6229 (issue: #6149)
  • Cache:
    • Left over from the query_cache to request_cache rename #12478
    • Rename caches. #11893 (issue: #11569)
    • Give the filter cache a smaller maximum number of cached filters. #11833
    • Remove the query parser cache. #10856
    • Don't use the fixed bitset filter cache for child nested level filters, but the regular filter cache instead #9740 (issue: #8810)
  • Circuit Breakers:
    • Add support for registering custom circuit breaker #8795
  • Cluster:
    • Remove double call to elect primaries #12147
    • Rename MetaData.uuid -> MetaData.clusterUUID and IndexMetaData.uuid-> IndexMetaData.indexUUID #11914 (issue: #11831)
    • Add MetaData.uuid to ClusterState.toXContent #11832
    • Remove deprecated METADATA cluster block level #10779 (issue: #9203)
  • Core:
    • Improve jvmcheck error failure #12696
    • Use explict flag if index should be created on engine creation #12671
    • Move Streams.copyTo(String|Bytes)FromClasspath() into StreamsUtils #12598
    • Improve toString on EsThreadPoolExecutor #12535 (issue: #9732)
    • Carry over shard exception failure to master node #12263
    • Allow IBM J9 2.8+ in version check #11850
    • Use System.nanoTime for ThreadPool's estimated time, since it's less likely to go backwards #11626
    • Cleanup MergeScheduler infrastrucutre #11602
    • Ban PathUtils.get (for now, until we fix the two remaining issues) #11069 (issues: #11065, #11068)
    • Refactor SSD/FileStore logic out of NodeEnvironment #10755 (issue: #10717)
    • Rename START phase into VERIFY_INDEX #10570
    • Move GatewayShardsState logic into IndexShard #10093
    • Remove IndexEngine #8955
    • Remove Gateway abstraction #8954
    • Cleanup LocalGatewayShardsState #8852
    • Let Lucene kick off merges normally #8643
    • Cut over MetaDataStateFormat to Path API in Gateway #8609
    • Remove usage of Directory#fileExists #8233
    • Change the default cache filter impl from FixedBitSet to WAH8DocIdSet #7577 (issues: #6280, #7037)
  • Dates:
    • Default date formats to use underscores via PUT #12509 (issue: #12429)
    • Allow for backwards compatibility for unix timestamp in pre 2.x indices #11515 (issue: #10971)
    • Allow for negative unix timestamps #11482 (issues: #11478, #11692)
    • More strict parsing of ISO dates #6227 (issue: #6158)
  • Discovery:
    • Wait on incoming joins before electing local node as master #12161
    • Don't join master nodes or accept join requests of old and too new nodes #11972 (issue: #11924)
  • Engine:
    • Pre sync flush cleanups #11252
    • Remove flushNeeded in favor of IW#hasUncommittedChanges() #11225
    • Remove the ability to flush without flushing the translog #11193
    • Make SearchFactory static class in InternalEngine #11154
    • Move back to single EngineConfig #10586 (issue: #10584)
    • Always fail engine on corruption #10092
    • Remove dirty flag and force boolean for refresh #9484
  • Exceptions:
    • Improve startup exceptions (especially file permissions etc) #13050
    • Fix formatting of startup/configuration errors #13029
    • Add serialization support for InterruptedException #12981
    • Include stacktrace in rendered exceptions #12260 (issue: #12239)
    • Render structured exceptions in mget / mpercolate #12240
    • Add index name to the upgrade exception #12213
    • Promote headers to first class citizens on exceptions #12006
    • Parameterized exception messages #11981
    • Carry on rest status if exceptions are not serializable #11973
    • Render strucutred exception in multi search #11849
    • Remove ElasticsearchIAE and ElasticsearchISE #10862 (issue: #10794)
    • Improve exception handling in transport local execution #10554
  • Fielddata:
    • Consult field info before fetching field data #12403
    • Remove the dependecy on IndexFielddataService from MapperService. #12371
    • Enable doc values by default, when appropriate #10209 (issue: #8312)
    • Change threshold value of fielddata.filter.frequency.max/min #9522 (issue: #9327)
  • Geo:
    • Update ShapeBuilder and GeoPolygonQueryParser to accept non-closed GeoJSON #11161 (issue: #11131)
    • Remove local Lucene Spatial package #10966
  • Highlighting:
    • Highlighting: keep track of the original query only in HighlighterContext #11317
  • Index APIs:
    • Show human readable Elasticsearch version that created index and date when index was created #11509 (issue: #11484)
  • Internal:
    • Cleanup bootstrap package. #13053 (issue: #13044)
    • Drop commons-lang dependency #12972
    • Flatten IndicesModule and add tests #12921 (issue: #12783)
    • Flatten ClusterModule and add more tests #12916 (issue: #12783)
    • Allow a plugin to supply its own query cache implementation #12881
    • Remove Environment.resolveConfig #12872
    • Remove ClassLoader from Settings #12868
    • Transport: allow to de-serialize arbitrary objects given their name #12571
    • Add RealtimeRequest marker interface to group realtime operations together #12537
    • Forbid Files.isHidden. #12484 (issue: #12465)
    • Remove unused QueryParseContext argument in MappedFieldType#rangeQuery() #12417
    • Simplify Replica Allocator #12401
    • Replace primaryPostAllocated flag and use UnassignedInfo #12374
    • Add the ability to wrap an index searcher. #12364
    • Cleanup TransportSingleShardAction and TransportInstanceSingleOperationAction #12361
    • Remove TransportSingleCustomOperationAction in favour of TransportSingleShardAction #12350
    • updated the elasticsearch versioning format #12210
    • Cleanup the data structures used in MetaData class for alias and index lookups #12202
    • Make 2.0.0.beta1-SNAPSHOT the current version. #12151 (issue: #12148)
    • Remove mapper references from Engines #12130
    • Cleanup ShardRoutingState uses and hide implementation details of ClusterInfo #12126
    • Consolidate ShardRouting construction #12125
    • Change JarHell to operate on Path instead of URL #12109
    • Refactor MetaData to split off the concrete index name logic to a dedicated service #12058
    • really ban exitVM with security policy #11982
    • Cut over to writeable for TransportAddress #11949
    • Internal: make sure ParseField is always used in combination with parse flags #11859
    • Remove XContentParser.map[Ordered]AndClose(). #11846
    • Use abstract runnable in scheduled ping #11795
    • Add DateTime ctors without timezone to forbidden APIs #11743
    • Fold ShardGetService creation away from Guice into IndexShard #11606
    • Create ShardSuggestService/Metrics manually outside of guice #11605
    • Minimize the usage of guava classes in interfaces, return types, arguments #11501
    • Make CompressedXContent.equals fast again. #11428 (issue: #11247)
    • Consolidate shard level modules without logic into IndexShardModule #11416
    • Serialization: Remove old version checks #11397
    • Catch UnsatisfiedLinkError on JNA load #11385
    • Deduplicate field names returned by simpleMatchToFullName & simpleMatchToIndexNames in FieldMappersLookup #11377 (issue: #10916)
    • Rename TransportShardReplicationOperationAction to TransportReplicationAction #11332
    • Absorb ImmutableSettings into Settings #11321 (issue: #7633)
    • Make some sec mgr / bootup classes package private and final. #11312
    • Tighten up our compression framework. #11279
    • Uid#createTypeUids to accept a collection of ids rather than a list #11263
    • Remove need for generics in ContextAndHeaderHolder #11222
    • Remove dependency on hppc:esoteric. #11144
    • Improve path mgmt on init, better error messages, symlink support #11106
    • Ensure JNA is fully loaded when its available, but don't fail its not #10989
    • Transport: read/writeGenericValue to support BytesRef #10878
    • Remove Preconditions class #10873
    • Remove index/indices replication infra code #10861
    • Reduce code duplication in TransportIndexAction/TransportShardBulkAction. #10819
    • Use Tuple only as return value in Bootstrap #10784
    • Prevent injection of unannotated dynamic settings #10763 (issue: #10614)
    • Refactor and cleanup transport request handling #10730
    • Cleanup local code transport execution #10582
    • Make APIs work per-segment like Lucene's Collector. #10389
    • Transport: shortcut local execution #10350 (issue: #10247)
    • Add support for cluster state diffs #10212
    • Fix Java 8 _ variable warning #10013
    • Stop passing default cluster name to cluster state read operations #9888
    • Add missing @Override annotations. #9832
    • Remove redundant fs metadata ops. #9807
    • Remove XCollector. #9677
    • Search: Reuse Lucene's MultiCollector. #9549
    • Remove HandlesStreamInput/Output #9486
    • Clean up memory reuse a bit. #9272
    • Remove action names bwc layer #8758 (issue: #7105)
    • Add File.java to forbidden APIs #8666
    • Inverse DocIdSets' heuristic to find out fast DocIdSets. #8380
    • Temporarily ban buggy IOUtils methods with forbidden #8375
    • Refactor shard recovery from anonymous class to ShardRecoveryHandler #8363
    • Remove DocSetCache. #7582
    • Remove CacheRecycler. #7366
  • Java API:
    • PrefixQueryParser takes a String as value like its Builder #12204 (issue: #12032)
    • Fix FuzzyQuery to properly handle Object, number, dates or String. #12020 (issue: #11865)
    • Treat path object as a simple value instead of Iterable in XContentBuilder #11903 (issue: #11771)
    • IdsQueryBuilder: Allow to add a list in addition to array #11409 (issue: #5089)
    • Fix typed parameters in IndexRequestBuilder and CreateIndexRequestBuilder #11382 (issue: #10825)
    • Remove duplicated buildAsBytes and corresponding toString methods #11063
    • Add internal liveness action to transport client #8763
    • Adding setters or making them public in ActionRequests #8123 (issue: #8122)
  • Logging:
    • Adds a setting to control source output in indexing slowlog #12806 (issue: #4485)
    • Add more debugging information to the Awareness Decider #12490 (issue: #12431)
    • Add shadow indicator when using shadow replicas #12399
    • Log warn message if leftover shard is detected #11826
    • ClusterStateObserver should log on trace on timeout #11722
    • Better error messages when mlockall fails #11433
    • Add logging for failed TTL purges #11302 (issue: #11019)
    • Logging: add the ability to specify an alternate logging configuration location #10852 (issues: #2044, #7395)
  • Mapping:
    • Move the _size mapper to a plugin. #12582
    • Remove index name from mapping parser #12352
    • Remove AbstractFieldMapper #12089
    • Completely move doc values and fielddata settings to field types #12014
    • Move short name access out of field type #11977
    • Rename "root" mappers to "metadata" mappers #11962
    • Mappings: Remove close() from Mapper #11863
    • Move merge simulation of fieldtype settings to fieldtype method #11783 (issue: #8871)
    • Hide more fieldType access and cleanup null_value merging #11770 (issue: #11764)
    • Replace fieldType access in mappers with getter #11764
    • Remove SmartNameObjectMapper #11686
    • Add equals/hashcode to fieldtypes #11644
    • Shortcut exists and missing queries when no types/docs exist #11586
    • Remove leftover sugar methods from FieldMapper #11565
    • Make index level mapping apis use MappedFieldType #11559
    • Move null value handling into MappedFieldType #11544
    • Refactor core index/query time properties into FieldType #11422 (issue: #8871)
    • Remove generics from FieldMapper #11292
    • Cleanup field name handling #11272
    • Remove document parse listener #11243
    • Remove SmartNameFieldMappers #11216
    • Make DocumentMapper.refreshSource() private. #11209
    • Make mapping updates atomic wrt document parsing. #11205
    • Remove the ignore_conflicts option. #11203
    • Add back support for enabled/includes/excludes in _source field #11171 (issue: #11116)
    • Make FieldNameAnalyzer less lenient. #11141
    • Remove mapper listeners #11045
    • Remove traverse functions from Mapper #11027
    • Consolidate document parsing logic #10802
    • Wait for required mappings to be available on the replica before indexing. #10786
    • Join MergeResults with MergeContext since they are almost the same #10765
    • Restrict murmur3 field type to sane options #10738 (issue: #10465)
    • Simplify dynamic mappings updates. #10720 (issue: #10593)
    • Simplified mapper lookups #10705 (issue: #8871)
    • Remove dead code after previous refactorings #10666 (issue: #8877)
    • Same code path for dynamic mappings updates and updates coming from the API. #10593 (issues: #8688, #9364, #9851)
    • Fix field mappers to always pass through index settings #9780
    • Store _timestamp by default. #8139
    • Better validation of mapping JSON #7534 (issue: #7205)
  • More Like This:
    • Renamed ignore_like to unlike #11117
    • Lenient default parameters #9412
    • Remove MLT Field Query #8238
    • Add versatile 'like' parameter #8039
    • Replaced exclude with include to avoid double negation #6248
  • Nested Docs:
    • Add min score mode to nested query #11909
  • Network:
    • Don't print lots of noise on IPv4 only hosts. #13026
    • Remove support for address resolving in InetSocketTransportAddress #13020 (issue: #13014)
    • Log network configuration at debug level #12979
    • Use preferIPv6Addresses for sort order, not preferIPv4Stack #12951
    • Make sure messages are fully read even in case of EOS markers. #11768 (issue: #11748)
    • Add profile name to TransportChannel #11261 (issue: #10483)
  • Packaging:
    • Bats testing: Remove useless systemctl check #12724 (issue: #12682)
    • improve sanity of securitymanager file permissions #12609
    • Remove Core Lib directory #12485 (issue: #12010)
    • Do not kill process on service shutdown #12298 (issue: #11248)
    • Improve JVM Arch Detection #12274 (issue: #12256)
    • fail plugins on version mismatch #12221
    • Allow use of bouncycastle #12102
    • Give a better exception when a jar contains same classfile twice. #12093
    • Don't jarhell check system jars #11979
    • detect jar hell before installing a plugin #11963 (issue: #11946)
    • jar hell check should fail, if jars require higher java version #11936
    • Fail startup (and tests) on jar hell #11932 (issue: #11926)
    • Load plugins into classpath in bootstrap #11918 (issue: #11917)
    • steps to remove dangerous security permissions #11898
    • Packaging: Add LICENSE and NOTICE files for all core dependencies #11705 (issues: #10684, #2794)
    • Use our provided JNA library, versus one installed on the system #11163
    • Remove unnecessary permissions. #11132
    • Tighten up script security more #10999
    • Add pid file to Environment #10986
    • Bail if ES is run as root #10970
    • Remove exitVM permissions #10963
    • Remove JNI permissions, improve JNI testing. #10962
    • Remove shutdownHooks permission #10953
    • Simplify securitymanager init #10936
    • Remove reflection permission for sun.management. #10848 (issue: #10553)
    • Security manager cleanups #10844
    • Enable securitymanager #10717
    • Remove working directory #10672
    • Use direct mapping call in Kernel32Library #9923 (issue: #9802)
    • CliTool: Add command to warn on permission/owner change #9508
    • Factor out PID file creation and add tests #8775 (issue: #8771)
    • Allow configuration of the GC log file via an environment variable #8479 (issues: #8471, #8479)
  • Parent/Child:
    • Enforce _parent field resolution to be strict #9521 (issue: #9461)
  • Percolator:
    • Don't cache percolator query on loading percolators #12862
    • Change percolator.getTime -> percolator.time #11954
    • The query parse context should be fetched from the IndexQueryParseService #11929
    • Remove index.percolator.allow_unmapped_fields setting. #8439
  • Plugin Cloud AWS:
    • Don't show access_key and filter_key in S3 repository settings #12845 (issues: #11265, #184)
    • Add support for base_path in elasticsearch.yml #12761
    • Adding retry when checking s3 snapshot repository #12498 (issue: #12462)
    • Upgrade AWS dependency to 1.10.0 #11659
  • Plugin Cloud GCE:
    • Update to GCE API v1-rev71-1.20.0 #12835
  • Plugins:
    • Lucene SPI support for plugins. #13051
    • Ensure additionalSettings() do not conflict #12967
    • Validate checksums for plugins if available #12888 (issue: #12750)
    • Expose zen ElectMasterService as a Discovery extension point #12828
    • Introduce a formal ExtensionPoint class to stream line extensions #12826
    • Flatten Allocation modules and add back ability to plugin ShardsAllocators #12818 (issue: #12781)
    • Apply additional plugin settings only if settings are not explicit #12796
    • PluginManager: Do not try other URLs if specific URL was passed #12766
    • PluginManager: Fix elastic.co download URLs, add snapshot ones #12641 (issue: #12632)
    • Fix plugin script to allow spaces in ES_HOME #12610 (issue: #12504)
    • PluginManager: Add Support for basic auth #12445
    • Ensure logging configuration is loaded in plugin manager #12081 (issue: #12064)
    • Simplify Plugin Manager for official plugins #11805
    • Allow security rule for advanced SSL configutation #11751
    • FileSystemUtils: Only create backup copies if files differ #9592
  • Query DSL:
    • Remove attemped (not working) support for array in not query parser #12890
    • simple query string: remove (not working) support for alternate formats #12798 (issue: #12794)
    • RegexpQueryParser takes a String as value like its Builder #12200 (issue: #11896)
    • Expose Lucene's new TopTermsBlendedFreqScoringRewrite. #12129
    • Default fuzzy transpositions to true #12090 (issue: #9278)
    • Special case the _index field in queries #12027 (issue: #3316)
    • Add support for query boost to SimpleQueryStringBuilder. #11696 (issue: #11274)
    • Change geo filters into queries #11137
    • Make the script filter a query. #11126
    • Return positions of parse errors found in JSON #10837 (issue: #3303)
    • Enable Lucene ranking behaviour for numeric term queries #10790 (issue: #10628)
  • REST:
    • Suppress rest exceptions by default and log them instead #12991
    • Add more utilities for source/body handling in RestAction #10724
    • Support JSON request body in scroll, clear scroll, and analyze APIs #9076 (issue: #5866)
    • Added _shards header to all write responses. #7994
  • Recovery:
    • No need to send mappings to the master node on phase 2. #11207
    • Integrate translog recovery into Engine / InternalEngine #10452
    • Wipe shard state before switching recovered files live #10179 (issue: #10053)
    • Allow to cancel recovery sources when shards are closed #8555
  • Scripting:
    • Allow scripts to expose whether they use the _score. #12695
    • Add path.scripts directory #12668
    • Simplify CacheKey used for scripts #12092
    • Allow executable expression scripts for aggregations #11689 (issue: #11596)
    • Unify script and template requests across codebase #11164 (issues: #10113, #10810, #11091)
    • Minor TimeZone Fix #10994
    • Run groovy scripts with no permissions #10969
    • Add Field Methods #10890
    • Remove groovy sandbox #10480 (issue: #10156)
    • Add explicit error message when script_score script returns NaN #8750 (issue: #2426)
    • Use groovy-x.y.z-indy jar for better scripting performance #8183 (issue: #8182)
  • Search:
    • Split SearchModule.configure() into separate methods #12827
    • Only compute scores when necessary with FiltersFunctionScoreQuery. #12707
    • Speed up the function_score query when scores are not needed. #12693
    • Make fetch sub phases pluggable #12400 (issue: #10729)
    • Add _replica and _replica_first as search preference. #12244 (issue: #12222)
    • Add global search timeout setting #12211 (issue: #12149)
    • Term Query: Be more strict during parsing #12195 (issue: #12184)
    • Clean up handling of missing values when merging shard results on the coordinating node. #12127 (issue: #9155)
    • Always return metadata in get/search APIs. #11816 (issue: #8068)
    • Removing top-level filter parameter from search API. #11600 (issue: #8862)
    • Do not specialize TermQuery vs. TermsQuery. #11308
    • Minor refactor of MultiValueMode removing apply and reduce #11290
    • Make FilteredQuery a forbidden API. #11224 (issue: #6511)
    • Count api to become a shortcut to the search api #11198 (issues: #9110, #9117)
    • Make SCAN faster. #11180
    • Remove (dfs_)query_and_fetch from the REST API #10864 (issue: #9606)
    • Cut over to IndexSearcher.count. #10674
    • Query scoring change for single-value queries on numeric fields #10631 (issue: #10628)
    • Remove unused normsField from MatchAllQuery #10592
    • Replace deprecated filters with equivalent queries. #10531 (issue: #8960)
    • Avoid calling DocIdSets.toSafeBits. #9546
    • Merge search_type=count and size=0. #9296 (issue: #7630)
    • Surgically removed slow scroll #8780
    • Filter cache: add a _cache: auto option and make it the default. #8573 (issue: #8449)
    • Do not force the post-filter to be loaded into a BitSet. #8488
  • Settings:
    • Throw Exception for missing settings file #12833 (issue: #11510)
    • Remove lenient store type parsing #12735
    • Add path.shared_data #12729 (issues: #11065, #12714)
    • ResourceWatcher: Rename settings to prevent watcher clash #11359 (issues: #11033, #11175)
    • Die cwd die #10923 (issue: #10877)
    • Remove cluster.routing.allocation.balance.primary #9159
    • Trimmed the main elasticsearch.yml configuration file #5861
  • Snapshot/Restore:
    • Add support for bulk delete operation in snapshot repository #12587 (issue: #12533)
    • Create a directory during repository verification #12323 (issue: #11611)
    • Add checksum to snapshot metadata files #12002 (issue: #11589)
    • Snapshot info should contain version of elasticsearch that created the snapshot #11985 (issue: #11980)
    • Extract all shard-level snapshot operation into dedicated SnapshotShardsService #11756
    • Change metadata file format #11507
    • Add support for applying setting filters when displaying repository settings #11270 (issue: #11265)
    • Add ability to retrieve currently running snapshots #9400 (issues: #7859, #8782, #8887)
  • Stats:
    • Refactor, remove _node/network and _node/stats/network. #12922 (issue: #12889)
    • Expose ClassloadingMXBean in Node Stats #12764 (issue: #12738)
    • Count scans in search stats and add metrics for scrolls #12069 (issue: #9109)
    • Field stats: added index_constraint option #11259 (issue: #11187)
    • Hot threads should include timestamp and params #9773
    • Added verbose option to segments api, with full ram tree as first additional element per segment #9111
    • Add more fine grained memory stats from Lucene segment reader to index stats #8832
  • Store:
    • Fall back to reading SegmentInfos from Store if reading from commit fails #11403 (issue: #11361)
    • Read segment info from latest commit whenever possible #11361
    • Move to one data.path per shard #10461 (issue: #9498)
    • Populate metadata.writtenBy for pre 1.3 index files. #9152
    • Expose ShardId via LeafReader rather than Directory API #8812
    • Drop pre 0.90 compression BWC #8385
    • Use DistributorDirectory only if there are more than one data direcotry #8383
  • Suggesters:
    • Remove filter from PhraseSuggester collate #11195
  • Term Vectors:
    • Support for shard level caching of term vectors #8395
    • Add support for distributed frequencies #8144
    • Support for version and version_type #7480
  • Translog:
    • Make translog file name parsing strict #11875
    • Some smallish translog cleanups #11200
    • Add translog checkpoints to prevent translog corruption #11143 (issues: #10933, #11011)
    • Make modifying operations durable by default. #11011 (issue: #10933)
    • Don't rename recovery translog in gateway #9719
    • Cut over to Path API #8611

Bug fixes

  • Aggregations:
    • Aggregation: Fix AggregationPath.subPath() to not throw ArrayStoreException #13035
    • Throw error if cardinality aggregator has sub aggregations #12989 (issue: #12988)
    • Full path validation for pipeline aggregations #12595 (issue: #12360)
    • Upgrade HDRHistogram to version 2.1.6. #12554
    • Fixes serialization of HDRHistogram in percentiles aggregations #12505
    • Adds new script API to ValuesSourceMetricsAggregationBuilder #12152
    • Aggregations: Makes SKIP Gap Policy work correctly for Bucket Script aggregation #11970
    • moving_avg forecasts should not include current point #11641
    • Fix bug where moving_avg prediction keys are appended to previous prediction #11465 (issue: #11454)
    • Sibling Pipeline Aggregations can now be nested in SingleBucketAggregations #11380 (issue: #11379)
    • Fixed Moving Average prediction to calculate the correct keys #11375 (issue: #11369)
    • Queries with size:0 break aggregations that need scores #11358 (issue: #11119)
    • Sampler agg could not be used with Terms agg’s order. #10785 (issue: #10719)
    • Fix multi-level breadth-first aggregations. #10411 (issues: #9544, #9823)
    • Make the nested aggregation call sub aggregators with doc IDs in order #9548 (issue: #9547)
  • Allocation:
    • ThrottlingAllocationDecider should not counting relocating shards #12409
  • Bulk:
    • Fix: Use correct OpType on Failure in BulkItemResponse #12060 (issue: #9821)
  • Cache:
    • Store filter cache statistics at the shard level instead of index. #11886
  • Cluster:
    • Changes in unassigned info and version might not be transferred as part of cluster state diffs #12387
    • Rename cluster state uuid to updateId #11862 (issue: #11831)
    • Write state also on data nodes if not master eligible #9952 (issue: #8823)
  • Core:
    • Remove Settings.getAsClass #12744 (issues: #12643, #12656)
    • Close lock even if we fail to obtain #11412
    • Balance new shard allocations more evenly on multiple path.data #11185 (issue: #11122)
  • Dates:
    • Add millisecond parser for dynamic date fields mapped from "yyyy/MM/dd" #12977 (issue: #12873)
  • Discovery:
    • Make sure NodeJoinController.ElectionCallback is always called from the update cluster state thread #12372
    • Only retry join when other node is not (yet) a master #8972
  • Engine:
    • Fix NPE when streaming commit stats #11266
    • Fixes InternalIndexShard callback handling of failure #8644 (issue: #5945)
  • Exceptions:
    • Use StartupError to format all exceptions hitting the console #13041 (issue: #13040)
    • Improve console logging on startup exception #12976
    • Don't special-case on ElasticsearchWrapperException in toXContent #12015 (issue: #11994)
    • Implement toXContent on ShardOpertionFailureException #11155 (issue: #11017)
    • Remove double exception handling that causes false replica failures #10990
  • Fielddata:
    • Reclaim memory from fielddata synchronously when an explicit clear is performed. #11697 (issue: #11695)
    • Load fielddata on behalf of scripts. #10997
  • Index APIs:
    • Upsert does not use ttl value #8715 (issue: #3256)
    • Fix GET index API always running all features #8392
  • Inner Hits:
    • Reset the ShardTargetType after serializing inner hits. #12261
    • Properly support named queries for both nested and parent child inner hits #11880 (issues: #10661, #10694)
  • Internal:
    • Add plugin modules before (almost all) others #13061 (issue: #12783)
    • Workaround JDK bug 8034057 #12970
    • Fix ShardUtils#getElasticsearchDirectoryReader() #12594
    • don't represent site plugins with 'null' anymore #12577
    • IndicesStore shouldn't try to delete index after deleting a shard #12494 (issue: #12487)
    • ShardUtils#getElasticsearchLeafReader() should use FilterLeafReader#getDelegate() instead of FilterLeafReader#unwrap #12437
    • Fix serialization of IndexFormatTooNewException and IndexFormatTooOldException #12277
    • Decode URL.getPath before resolving a real file #11940
    • Transport: remove support for reading/writing list of strings, use arrays instead #11276 (issue: #11056)
  • Java API:
    • Add missing support for escape to QueryStringQueryBuilder #13016
    • Fix PrefixQueryBuilder to support an Object value #12124 (issue: #12032)
    • Properly fix the default regex flag to ALL for RegexpQueryParser and Builder #12067 (issue: #11896)
    • Java api: add missing support for boost to GeoShapeQueryBuilder and TermsQueryBuilder #11810 (issue: #11744)
  • Mapping:
    • Move the murmur3 field to a plugin and fix defaults. #12931 (issue: #12874)
    • Fix field type compatiblity check to work when only one previous type exists #12779 (issue: #12753)
    • Wait for mappings to be available on the primary before indexing. #10949
    • Fix _field_names to not have doc values #10893 (issue: #10892)
    • Explicitly disallow multi fields from using object or nested fields #10745
    • Fix doc values representation to always serliaze if explicitly set #10302 (issue: #10297)
  • More Like This:
    • Fixes many misbehaving user parameters #8028 (issue: #2914)
  • Network:
    • Deduplicate addresses from resolver. #12995
    • Remove usage of InetAddress#getLocalHost #12959
    • Fix network binding for ipv4/ipv6 #12942 (issues: #12906, #12915)
    • Transport: Do not make the buffer skip while a stream is open. #11988
  • Packaging:
    • Fix variable substitution for OS's using systemd #12909
    • Fix rpm -e removing /etc/elasticsearch #12785
    • Makes sure all POMs contain a description #12771 (issue: #12550)
    • use spaces liberally in integration tests and fix space handling #12710 (issue: #12709)
    • rpm and deb create scripts directory #12704 (issue: #12702)
    • Fix shaded jar packaging #12589
    • Fix Bootstrap to not call System.exit #12586
    • elasticsearch (again) adds CWD to classpath #12580
    • jsr166e was left out of shaded jar #12194 (issue: #12193)
    • Don't add CWD to classpath when ES_CLASSPATH isn't set. #12001 (issue: #12000)
    • Fix missing dependencies for RPM/DEB packages #11664 (issue: #11522)
    • Update project.name in bin/elasticsearch script #11348
  • Parent/Child:
    • Explicitly disabled the query cache #12955
    • Fix _parent.type validation #11436
  • Percolator:
    • Fix NPE when percolating a document that has a _parent field configured in its mapping #12214 (issue: #12192)
    • Load percolator queries before shard is marked POST_RECOVERY #11799 (issue: #10722)
  • Plugin Cloud Azure:
    • Correctly list blobs in Azure Storage to prevent snapshot corruption and do not unnecessarily duplicate Lucene segments in Azure Storage #12380 (issues: #51, #99)
  • Plugin Delete By Query:
    • Fix number of deleted/missing documents in Delete-By-Query #11745
  • Plugins:
    • Fix automatically generated URLs for official plugins in PluginManager #12885
    • Skip hidden files #12465 (issue: #12433)
    • strip elasticsearch- and es- from any plugin name #12160 (issues: #12143, #12158)
    • remove elasticsearch- from name of official plugins #12158 (issues: #11805, #12143)
    • Fix pluginmanager permissions for bin/ scripts #12157 (issue: #12142)
    • Let HTTPS work correctly #10983
    • Plugins failed to load since #8666 #8756
  • Query DSL:
    • Add support for disable_coord param to terms query #12756 (issue: #12755)
    • Rewrite set twice in WildcardQueryParser #12207
    • Better exception if array passed to term query. #11384 (issue: #11246)
    • Query DSL: Fix bool parsing. #11120 (issue: #10985)
    • Minor fixes to the match query #8352 (issue: #6932)
  • Query Refactoring:
    • Query DSL: don't cache type filter in DocumentMapper #12447
  • REST:
    • Return 408 REQUEST_TIMEOUT if _cluster/health times out #12780
    • Update RestRequest.java #11305
    • Render non-elasticsearch exception as root cause #10850 (issue: #10836)
  • Recovery:
    • Endless recovery loop with indices.recovery.file_chunk_size=0Bytes #12919
    • Check for incompatible mappings while upgrading old indices #12406 (issue: #11857)
    • Fix MapperException detection during translog ops replay #11583 (issue: #11363)
    • Fix recovered translog ops stat counting when retrying a batch #11536 (issue: #11363)
    • Restart recovery upon mapping changes during translog replay #11363 (issue: #11281)
    • Add engine failure on recovery finalization corruption back #11241
  • Scripting:
    • Execute Scripting Engine before searching for inner templates in template query #11512
  • Search:
    • Never cache match_all queries. #13032
    • _all: Stop NPE querying _all when it doesn't exist #12495 (issue: #12439)
    • Matched queries: Remove redundant and broken code #10694 (issue: #10661)
  • Settings:
    • Do not swallow exceptions thrown while parsing settings #13039 (issue: #13028)
    • Medium Interval time for ResourceWatcher should be 30 seconds #12423
    • Replace references to ImmutableSettings with Settings #11843
    • Always normalize root paths during resolution of paths #11446 (issue: #11426)
  • Snapshot/Restore:
    • Blob store shouldn't try deleting the write.lock file at the end of the restore process #11517
    • Move in-progress snapshot and restore information from custom metadata to custom cluster state part #11486 (issue: #8102)
    • Fix cluster state task name for update snapshot task #11197
    • Improve snapshot creation and deletion performance on repositories with large number of snapshots #8969 (issue: #8958)
    • Switch to write once mode for snapshot metadata files #8782 (issue: #8696)
  • Stats:
    • Use time with nanosecond resolution calculated at the executing node #12346 (issue: #12345)
    • Failure during the fetch phase of scan should invoke the failed fetch… #12087 (issue: #12086)
  • Term Vectors:
    • Make sure filter is correctly parsed for multi-term vectors #12312 (issue: #12311)
  • Top Hits:
    • Protected against size and offset larger than total number of document in a shard #12518 (issue: #12510)
  • Translog:
    • Ignore EngineClosedException during translog fysnc #12384
    • Don't convert possibly corrupted bytes to UTF-8 #11911
    • Mark translog as upgraded in the engine even if a legacy generation exists #11860 (issue: #11858)

Regressions

  • Network:
    • Only resolve host if explicitly allowed. #12986

Upgrades

  • Core:
    • Upgrade to Lucene 5.2.1. #11662
    • Upgrade to Lucene 5.2 #11534
    • Upgrade to lucene-5.2.0-snapshot-1681024 #11296
    • Upgrade to lucene-5.2.0-snapshot-1680200. #11218
    • Upgrade to lucene-5.2.0-snapshot-1678978. #11125
    • Upgrade to HPPC 0.7.1 #11035
    • Upgrade to lucene-5.2-snapshot-1675363. #10727 (issue: #10728)
    • Upgrade to Lucene 5.2 r1675100 #10699
    • Upgrade to Lucene-5.2-snapshot-1674183. #10641
    • Upgrade to Lucene 5.2 r1673726 #10612
    • Upgrade to lucene-5.2.0-snapshot-1673124. #10562
    • Upgrade to lucene-5.1.0-snapshot-1671894. #10468
    • Update to Lucene 5.1 snapshot r1671277 #10435
    • Upgrade to lucene r1662607 #9915 (issue: #1)
    • Upgrade to Lucene r1660560 #9746
    • Upgrade to Lucene snapshot r1656366 #9524 (issue: #8870)
    • Upgrade to lucene r1654549 snapshot #9402
    • Upgrade to lucene-5.1.0-snapshot-1652032. #9318
    • Upgrade to lucene 5 r1650327 #9206
    • Upgrade to current Lucene 5.0.0 snapshot #8588
    • Upgrade master to lucene 5.0 snapshot #8347
    • Move master to Java 1.7 #5268 (issue: #5267)
  • Network:
    • Update Netty to version 3.10.3.Final #12310

Changes first released in a previous version

Breaking changes

  • Aggregations:
    • Changed the respnose structure of the percentiles aggregation #6079 (issue: #5870)
  • Aliases:
    • Throw exception if index is null or missing when creating an alias #8240 (issues: #7863, #7976)
  • Allocation:
    • Default delayed allocation timeout to 1m from 0 #12166
    • Enable DiskThresholdDecider by default, change default limits to 85/90% #6204 (issues: #6200, #6201)
  • Analysis:
    • Improvements to StemmerTokenFilter #6452
  • CRUD:
    • Version types EXTERNAL & EXTERNAL_GTE test for version equality in read operation & disallow them in the Update API #5929 (issues: #5661, #5663)
  • Cache:
  • Circuit Breakers:
    • Add HierarchyCircuitBreakerService #6739 (issue: #6129)
  • Cluster:
    • Cluster State API: Remove index template filtering #6020 (issue: #4954)
  • Core:
    • Resiliency: Throw exception if the JVM will corrupt data. #7580
    • Remove deprecated gateways #5520 (issue: #5422)
  • Engine:
    • Remove full flush / FlushType.NEW_WRITER #9559
  • Exceptions:
    • Streamline use of IndexClosedException introduced with #6475 #6990 (issues: #6475, #6988)
    • Thread pool rejection status code should be 429 #6629 (issue: #6627)
  • Fielddata:
    • Fielddata: Remove soft/resident caches. #7443
  • Highlighting:
    • Wildcard field names in highlighting should only return fields that can be highlighted #11364 (issue: #9881)
  • Index APIs:
    • Change behaviour of indices segments api to allow no indices #9219 (issue: #5856)
    • REST API: Removed support for aliases as part of index settings #5545
  • Internal:
    • Simplify reading / writing from and to BlobContainer #7551
    • Refactor guice startup #7289
    • Fixed filters execution order and fix potential concurrency issue in filter chains #7023 (issues: #7019, #7021)
    • Make transport action name available in TransportAction base class #6860
    • Cleanup Rest Response #5612
    • Remove Releasable in favor of Closeable#5423
  • Java API:
    • QueryBuilders cleanup (add and deprecate) #8667 (issue: #8721)
    • Remove operationThreaded setter from ExplainRequestBuilder #7186
    • Remove unnecessary intermediate interfaces #6517 (issue: #4355)
    • Unify IndicesOptions constants #6068 (issue: #6059)
    • Remove operation threading from broadcast actions #6044
    • Remove search operation threading option #6042
    • Make Create/Update/Delete classes less mutable #5939 (issue: #5917)
  • Mapping:
    • Remove unsupported postings_format / doc_values_format #7604 (issues: #7238, #7566)
    • Mappings: Update mapping on master in async manner #6648
    • The binary field shouldn't be stored by default, because it is already available in the _source #4957
  • Network:
  • Plugins:
    • Don't overwrite plugin configuration when removing/upgrading plugins #7890 (issue: #5064)
  • Query DSL:
    • Function Score: Refactor RandomScoreFunction to be consistent, and return values in range [0.0, 1.0] #7446 (issue: #6907)
    • Remove custom_score and custom_boost_factor queries #5076
  • REST:
    • Security: Disable JSONP by default #6795
  • Recovery:
  • Scripting:
    • Cleanup ScriptService & friends in preparation for #6418 #9992 (issue: #6418)
    • Disable dynamic Groovy scripting by marking Groovy as not sandboxed #9655
    • Created a parameter parser to standardise script options #7977
    • Script with _score: remove dependency of DocLookup and scorer #7819 (issues: #7043, #7487)
    • Remove MVEL as a built-in scripting language #6610
    • Switch to Groovy as the default scripting language #6571 (issue: #6233)
    • Disable dynamic scripting by default #5943 (issue: #5853)
  • Search:
    • Only return aggregations on the first page with scroll and forbidden with scan #7497 (issue: #1642)
  • Settings:
    • Add a index.query.parse.allow_unmapped_fields setting that fails if queries refer to unmapped fields. #6928 (issue: #6664)
    • Change default filter cache to 10% and circuit breaker to 60% #5990
  • Snapshot/Restore:
    • Url repository should respect repo.path for file urls #11687
    • Automatic verification of all files that are being snapshotted with Snapshot/Restore #7159 (issue: #5593)
    • Add Partial snapshot state #5793 (issue: #5792)
  • Stats:
    • Add human readable start_time and refresh_interval #5544 (issue: #5280)
    • Migrating NodesInfo API to use plugins instead of singular plugin #5072
  • Store:
    • Remove the in memory buffer Lucene store/directory #4994

Deprecations

  • Geo:
  • Rivers:
    • Deprecate rivers #10503
    • Warning in documentation for deprecation of rivers #10423 (issue: #10345)

New features

  • Aggregations:
    • PercentageScore heuristic for significant_terms #9747 (issue: #9720)
    • Return the sum of the doc counts of other buckets in terms aggregations. #8213
    • Significant terms: add scriptable significance heuristic #7850
    • Scriptable Metrics Aggregation #7075 (issue: #5923)
    • Added pre and post offset to histogram aggregation #6980 (issue: #6605)
    • Added Filters aggregation #6974 (issues: #6118, #6119)
    • Add children aggregation #6936
    • Significant Terms: Add google normalized distance and chi square #6858
    • Infrastructure for changing easily the significance terms heuristic #6561
    • Added percentile rank aggregation #6432 (issue: #6386)
    • Deferred aggregations prevent combinatorial explosion #6128
    • Support bounding box aggregation on geo_shape/geo_point data types. #5634
    • Add reverse nested aggregation #5485
    • Cardinality aggregation #5426
    • Percentiles aggregation #5323
    • Significant_terms aggregation #5146
    • Add preserve original token option to ASCIIFolding #5115 (issue: #4931)
    • Add script support to value_count aggregations. #5007 (issue: #5001)
  • Allocation:
    • Optional Delayed Allocation on Node leave #11712
  • Analysis:
    • Add keep_types for filtering by token type #7120
    • Add uppercase token filter #5539
  • CAT API:
    • Show open and closed indices in _cat/indices #7936 (issue: #7907)
    • Add /_cat/fielddata to display fielddata usage #6086 (issue: #4593)
    • Add _cat/plugins endpoint #4824
    • Add _cat/segments #4711
  • CRUD:
    • Update API: Add support for scripted upserts. #7144
    • Update API - allow scripted upserts #7143
    • Update API: Detect noop updates when using doc #6862 (issue: #6822)
    • Introducing VersionType.FORCE & VersionType.EXTERNAL_GTE #4993 (issues: #2946, #4213)
  • Cache:
    • Query Cache: Support shard level query response caching #7161
  • Circuit Breakers:
    • Allow setting individual breakers to "noop" breakers #8135
    • Add NoopCircuitBreaker used in NoneCircuitBreakerService #8063
  • Core:
    • Switch auto-generated IDs to Flake IDs from random UUIDs #7531 (issues: #5941, #6004)
  • Index APIs:
  • Index Templates:
    • Made template filtering generic and extensible #7454 (issue: #7459)
    • Added support for aliases to index templates #5180 (issues: #1825, #2739)
  • Indexed Scripts/Templates:
    • Allow search templates stored in an index to be retrieved and used at search time #5921 (issues: #5484, #5637)
  • Internal:
    • Added an option to add arbitrary headers to the client requests #7127
  • Java API:
  • Logging:
    • Add ability to specify a SizeBasedTriggeringPolicy for log configuration #10373 (issue: #10371)
  • Mapping:
    • Bring back numeric_resolution #10420 (issue: #10072)
    • Add new default option for timestamp field #7036 (issue: #4718)
    • Add transform to document before index. #6599 (issue: #6566)
    • Add doc values for binary field #5669
  • More Like This:
    • Support for artificial documents in MLT query #7725
  • Percolator:
    • Enable percolation of nested documents #5082
  • Query DSL:
    • Add time_zone setting for query_string #8164 (issue: #7880)
    • Add format support for date range filter and queries #7821 (issue: #7189)
    • Add min_score parameter to function score query to only match docs above this threshold #7814 (issue: #6952)
    • Function score multi values #5940 (issue: #3960)
    • Add the field_value_factor function to the function_score query #5519
    • Added cross_fields type to multi_match query #5005 (issue: #2959)
    • Allow for executing queries based on pre-defined templates #4879
  • REST:
    • API: Add response filtering with filter_path parameter #10980 (issue: #7401)
    • Add CBOR data format support #5509 (issue: #4860)
  • Recovery:
  • Scripting:
    • Add support for fine-grained settings #10116 (issues: #10274, #6418)
    • Add script engine for Lucene expressions #6819 (issue: #6818)
    • Add Groovy as a scripting language, add groovy sandboxing #6233
  • Search:
    • Validate API: provide more verbose explanation #10147 (issues: #1412, #88247)
    • Add inner hits to nested and parent/child queries #8153 (issues: #3022, #3152)
    • Sorting: Allow _geo_distance to handle many to many geo point distance #7097 (issue: #3926)
    • Add search-exists API to check if any matching documents exist for a given query #7026 (issue: #6995)
    • Add an option to early terminate document collection when searching/counting #6885 (issue: #6876)
    • Sequential rescores #4748
  • Settings:
  • Shadow Replicas:
    • Allow shards on shared filesystems to be recovered on any node #10960 (issue: #10932)
    • Shadow replicas on shared filesystems #9727 (issue: #8976)
  • Stats:
  • Store:
  • Suggesters:
    • Phrase Suggester: Add option to filter out phrase suggestions #6773 (issue: #3482)
    • ContextSuggester #4044 (issue: #3959)
  • Top Hits:
    • Add top_hits aggregation #6124
  • Upgrade API:
    • Add API to upgrade old Lucene indices to the latest version #7922 (issue: #7884)

Enhancements

  • Aggregations:
    • Numeric metric aggregations are now formattable #9032 (issue: #6812)
    • Adds methods to get to/from as Strings for Range Aggs #9026 (issue: #9003)
    • Made the nested, reverse_nested and children aggs ignore unmapped nested fields or unmapped child / parent types. #8808 (issue: #8760)
    • Make size property parsing inconsistent #8645 (issue: #6061)
    • Do not take deleted documents into account in aggregations filters. #8540
    • Support for arrays of numeric values in include/exclude clauses #7727 (issue: #7714)
    • Add ability to sort on multiple criteria #7662 (issues: #6917, #7588)
    • Encapsulate AggregationBuilder name and make getter public #7425
    • Merge LongTermsAggregator and DoubleTermsAggregator. #7279
    • Remove the logic to optionally sort/dedup values on the fly. #7276
    • Make the list of buckets for terms and histogram returned as a java.util.List. #7275
    • Stops direct subclassing of InternalNumericMetricsAggregation #7058
    • Better JSON output scoping #6985 (issue: #7004)
    • Better heuristic for setting default shard_size in terms aggregation #6960 (issue: #6857)
    • Added an option to show the upper bound of the error for the terms aggregation #6778 (issue: #6696)
    • Extend allowed characters in aggregation name #6702
    • Moved BucketsAggregator#docCounts field to IntArray #6529
    • GlobalOrdinalsStringTermsAggregator is inefficient for high-cardinality fields #6518
    • Remove ordinals execution hint. #6499
    • Delegation of nextReader calls #6477
    • Add shard_min_doc_count parameter to terms aggregation #6143 (issues: #5998, #6041)
    • Add shard_min_doc_count parameter for significant terms similar to shard_size #6041
    • Add include/exclude support to global ordinals based terms and significant terms aggregations #6000
    • Lower the initial sizing of sub aggregations. #5994
    • Improve the way sub-aggregations are collected. #5975
    • Add global ordinal based implementations for significant terms aggregation #5970
    • Use collectExistingBucket() if a bucket already exists #5955
    • Significant_terms agg: added option for a backgroundFilter #5944
    • Improve terms aggregation to perform the segment ordinal to global ordinal lookup post segment collection #5895
    • Remove abstraction in the percentiles aggregation. #5859
    • Instantiate facets/aggregations during the QUERY phase. #5821
    • Aggregation cleanup #5699
    • Aggregations cleanup #5614
    • Refactor common code for unmapped aggregators into NonCollectingAggregator. #5528
    • Significant_terms agg only creates term frequency cache when necessary #5459 (issue: #5450)
    • Added extended_bounds support for date_/histogram aggs #5444 (issue: #5224)
    • Make aggregations CacheRecycler-free. #5419 (issue: #4929)
    • Added support for sorting buckets based on sub aggregation down the current hierarchy #5340 (issue: #5253)
    • Terms aggs: only use ordinals on low-cardinality fields by default. #5304 (issue: #5303)
    • Rest API needs to be consistent across all multi-bucket aggs #4926
    • cleanup of aggregations api #4894 (issue: #4922)
  • Allocation:
    • Simplify ShardRouting and centralize move to unassigned #11634
    • When using recover_on_any_node on shared filesystem, respect Deciders #11168
    • Early terminate if the cluster can't be rebalanced #9162
    • Speed-up disk-threshold decider #8803 (issue: #6372)
    • DiskThresholdDecider#remain(...) should take shards relocating away into account #8659 (issue: #8538)
    • Take percentage watermarks into account for reroute listener #8382 (issues: #8367, #8368)
    • Reroute shards automatically when high disk watermark is exceeded #8270 (issue: #8146)
    • Add option to take currently relocating shards' sizes into account #7785 (issues: #6168, #7753)
    • Allow primaries that have never been allocated to be allocated if under the low watermark #6209 (issue: #6196)
    • Add explanations for all AllocationDeciders #4934 (issues: #2483, #4380)
    • Make shard balancing deterministic if weights are identical #4866
  • Analysis:
    • Document and test custom analyzer position_offset_gap #10934 (issue: #1812)
    • Expose Latvian analyzer #7542
    • Improve Hunspell error messages #6850
    • Share numeric date analyzer instances between mappings #6843
    • Add missing pre built analysis components #6770
    • PatternAnalyzer should use PatternTokenFilter instead #6717
    • More resource efficient analysis wrapping usage #6714
    • Add additional Analyzers, Tokenizers, and TokenFilters from Lucene #6693 (issue: #5935)
    • Use non analyzed token stream optimization everywhere #6001
    • Add support for char filters in the analyze API #5148
  • CAT API:
    • Add node id to _cat apis #10246 (issue: #10226)
    • Mark shadow replicas with 's' in _cat/shards output #10023 (issue: #9772)
    • Add total index memory in _cat/indices #7824 (issue: #7008)
    • Add file descriptor details to cat/nodes #7655 (issue: #7652)
    • Add configured thread pool sizes to _cat/thread_pool #5366
    • Added thread pool cat api #4906 (issue: #4907)
    • RestTable.renderValue() doesn't know about tera and peta #4871
    • Clean up cat headers #4853 (issue: #4852)
  • CRUD:
    • Allow GET access to _all field (return value was always null before) #6924
    • Provide more context variables in update scripts #5724 (issues: #1607, #2230, #2231)
  • Cache:
    • Use a smaller expected size when serializing query results #9485
    • Use correct number of bytes in query cache accounting #9479
    • Use a 1024 byte minimum weight for filter cache entries #8304 (issues: #8249, #8268)
    • Immediately remove filter cache entries on cache clear #8289 (issue: #8285)
    • Add hit and miss count to Query Cache #7355
    • Warmer (search) to support query cache #7326
    • Add a request level flag to control Query Cache #7167
    • Add a periodic cleanup thread for IndexFieldCache caches #7015 (issue: #7010)
  • Circuit Breakers:
    • Circuit Breakers: Log if CircuitBreaker is tripping #8050
    • Include name of the field that caused a circuit break in the log and exception message #5841 (issue: #5718)
    • Increase RamAccountingTermsEnum flush size from 1mb to 5mb #5335
    • Add circuit breaker for parent/child id cache #5325 (issue: #5325)
    • Add circuit breaker functionality to parent/child id field data cache #5276
  • Cluster:
    • Remove scheduled routing #11776
    • Reset registeredNextDelaySetting on reroute #11759
    • Add Unassigned meta data #11653
    • Make routing_nodes an independent metric option in cluster state api #10412 (issue: #10352)
    • Add METADATA_READ and METADATA_WRITE blocks #9203 (issues: #10521, #10522, #2833, #3703, #5855, #5876, #8102)
    • Don't mark cluster health as timed out if desired state is reached #8683
    • Add missing cluster blocks handling for master operations #7763 (issue: #7740)
    • Master election should demotes nodes which try to join the cluster for the first time #7558 (issue: #7493)
    • Do not use a background thread to disconnect node which are removed from the ClusterState #7543
    • Refactored ClusterStateUpdateTask protection against execution on a non master #7511 (issue: #7493)
    • Remove unneeded cluster state serialization during cluster join #6949
    • Resend failed shard messages when receiving a cluster state still referring to the failed shards #6881
    • Send shard exists requests if shard exists locally but is not allocated to the node #6870
    • Don't attempt to start or fail shard if no master node can be found #6841
    • Improve handling of failed primary replica handling #6825 (issue: #6808)
    • Add local node to cluster state #6811
    • During relocation, process pending mapping update in phase 2 #6762 (issue: #6648)
    • Improve pending api to include current executing class #6744
    • Start Master|Node fault detection pinging immediately during discovery #6706 (issue: #6480)
    • Clean shard bulk mapping update to only use type name #6695
    • Ensure index.version.created is consistent #6660
    • Refactored AckedClusterStateUpdateTask & co. to remove code repetitions in subclasses #6559
    • Wait till node is part of cluster state for join process #6480
    • Do not use versions to optimize cluster state copying for a first update from a new master #6466
    • Improve cluster update settings api #6244
    • When sending shard start/failed message due to a cluster state change, use the master indicated in the new state rather than current #6189
    • Raise node disconnected even if the transport is stopped #5918
    • Moved the updateMappingOnMaster logic into a single place. #5850 (issue: #5798)
    • A new ClusterStateStatus to indicate cluster state life cycles #5741
    • Optimize multiple cluster state processing on receiving nodes #5139
    • Introduced a new IMMEDIATE priority - higher than URGENT #5098 (issue: #5062)
    • Bulk process of shard started/failed should not execute on already processed events #5061
  • Core:
    • Reduce shard inactivity timeout to 5m #11479 (issues: #11179, #11336)
    • Fail shard if search execution uncovers corruption #11440 (issue: #11419)
    • Acquire IndexWriter's write.lock lock before shard deletion #11127 (issue: #11097)
    • Refactor TransportShardReplicationOperationAction #10749 (issue: #10032)
    • Make getFileStore a bit more defensive #10696
    • Ref count write operations on IndexShard #10610
    • Refresh if many deletes in a row use up too much version map RAM #10312 (issue: #7052)
    • Add before/afterIndexShardDelete callbacks to index lifecycle #10173
    • Don't rethrow already handled merge exceptions #10083
    • NodeEnv should lock all shards for an index #9799
    • Retry if shard deletes fail due to IOExceptions #9784
    • Only do a single listAll from FileSwitchDir #9666 (issue: #6636)
    • Consolidate index / shard deletion in IndicesService #9605
    • Increase default xlog flush size from 200mb to 512 mb #9341 (issue: #9265)
    • Pass through all exceptions in IndicesLifecycleListeners #9330
    • Pass index settings to IndicesLifecycle#beforeIndexCreated and #afterIndexShardClosed #9245
    • Delete shard content under lock #9083 (issues: #8608, #9009)
    • Use Lucene's defaults for compound file format #8934 (issue: #8919)
    • Remove explicit .cleanUp() on cache clear #8924
    • Ensure shards are deleted under lock on close #8579
    • Add before/after indexDeleted callbacks to IndicesLifecycle #8569 (issue: #8551)
    • Free pending search contexts if index is closed #8551
    • Ban all usage of Future#cancel(true) #8494
    • Set bloom default to false even when Directory doesn't have a codecService #8442
    • Introduce shard level locks to prevent concurrent shard modifications #8436
    • Observe cluster state on health request #8350
    • Introduce a RefCounted interface and basic impl #8210
    • Use 1 instead of 0 as filler version value for nested docs #8145
    • Add cluster and index state checksums #8010 (issue: #7586)
    • Resiliency: Perform write consistency check just before writing on the primary shard #7873
    • Add ActionRunnable support to ThreadPool to simplify async operation on bounded threadpools #7765
    • Verify checksums on merge #7360
    • Change numeric data types to use SORTED_NUMERIC docvalues type #6967
    • Disable loading of bloom filters by default #6959 (issues: #6298, #6349)
    • Don't close/reopen IndexWriter when changing RAM buffer size #6856
    • Don't acquire dirtyLock on autoid for create #6584
    • Reuse Lucene's TermsEnum for faster _uid/version lookup during indexing #6298 (issue: #6212)
    • Entirely cut over to TopDocs#merge for merging shard docs in the reduce phase #6197
    • Don't use AllTokenStream if no fields were boosted #6187
    • Remove SerialMergeScheduler #6120
    • Throttling incoming indexing when Lucene merges fall behind #6066
    • Use Lucene built-in checksumming #5924
    • Don't lookup version for auto generated id and create #5917
    • Use Lucene's defaults for CMS settings #5912 (issue: #5882)
    • Change default merge throttling to 50MB / sec #5902
    • Don't lookup version for auto generated id and create #5785
    • Prevent fsync from creating 0-byte files #5746
    • Move to use serial merge schedule by default #5447
    • Force merges to not happen when indexing a doc / flush #5319
    • Reuse pages more agressively in BigArrays. #5300 (issue: #5299)
  • Discovery:
    • Prevent over allocation for multicast ping request #10896
    • Unicast Ping should close temporary connections after returning ping results #10849
    • Prevent stale master nodes from sharing dated cluster states to nodes that have moved to a different master node #9632
    • Publishing timeout to log at WARN and indicate pending nodes #9551
    • Concurrent node failures can cause unneeded cluster state publishing #8933 (issue: #8804)
    • Client: Only fetch the node info during node sampling #8685
    • Don't wait joinThread when stopping #8359 (issue: #8327)
    • Improve handling of multicast binding exceptions #8243 (issue: #8225)
    • Simplify discovery node initialization if version is unknown #8055 (issue: #8051)
    • Remove MasterFaultDetection.Listener.notListedOnMaster #7995
    • Only accept unicast pings once started #7950
    • Add a finalize round to multicast pinging #7924
    • During discovery, master fault detection should fall back to cluster state thread upon error #7908 (issue: #7834)
    • Close ping handler's executor service properly #7903
    • NodesFD: simplify concurrency control to fully rely on a single map #7889
    • During discovery, remove any local state and use clusterService.state instead #7834
    • Update ZenDiscovery fields via the cluster service update task. #7790
    • Give a unique id to each ping response #7769
    • UnicastZenPing don't rename configure host name #7747 (issue: #7719)
    • Node join requests should be handled at lower priority than master election #7733
    • Not all master election related cluster state update task use Priority.IMMEDIATE #7718
    • Accumulated improvements to ZenDiscovery #7493 (issue: #2488)
    • UnicastZenPing should also ping last known discoNodes #7336
    • With unicast discovery, only disconnect from temporary connected nodes #6966
    • During discovery, verify connect when sending a rejoin cluster request #6779
    • Have a dedicated join timeout that is higher than ping.timeout for join #6342
    • Unicast discovery enhancement #5508
  • Engine:
    • Remove reflection call to waitForMerges #10102
    • Move InternalEngine.segmentStats() into abstract Engine #9728 (issue: #9727)
    • Move more methods into abstract Engine #9717
    • Move as much as possible into abstract Engine #9678
    • Factor out settings updates from Engine #9625
    • Close Engine immediately if a tragic event strikes. #9616 (issue: #9517)
    • Refactor InternalEngine into abstract Engine and classes #9585
    • Remove FlushType and make resources final in InternalEngine #9565
    • Simplify Engine construction and ref counting #9211
    • Fold engine into IndexShard #9181
    • Don't acquire Engine's readLock in segmentsStats #8910 (issue: #8905)
    • Remove engine related command classes #8900
    • Allow InternalEngine to be stopped and started #8784 (issue: #8720)
    • Flush IndexWriter to disk on close and shutdown #7563
    • Ensure close is called under lock in the case of an engine failure #5800
    • Fail the engine/shard when refresh failed #5633
  • Exceptions:
    • Reduce the size of the XContent parsing exception #11642
    • Fix typo when primary is not available to index a document (UnavailableShardsException) #10140
    • Change IndexPrimaryShardNotAllocatedException from 409 to 500 #7987 (issue: #7632)
    • Nest original exception while creating NoShardAvailableActionException #7757 (issue: #7756)
    • Improve exception from Store.failIfCorrupted #7695 (issue: #7596)
    • Introduced a new elasticsearch exception family that can hold headers #7269
    • Better message for invalid internal transport message format #6916
    • Function Score: Add missing whitespace in error message when throwing exception #6155
  • Fielddata:
    • Fielddata: Remove custom comparators and use Lucene's instead #6981 (issue: #5980)
    • Switch fielddata to use Lucene doc values APIs. #6908
    • Make BytesValues.WithOrdinals more similar to Lucene's SortedSetDocValues #6524
    • Don't expose hashes in Fielddata anymore. #6500
    • Add a dedicated field data type for the _index field mapper. #6073 (issue: #5848)
    • Provide meaningful error message if field has no fielddata type #5979 (issue: #5930)
    • Use segment ordinals as global ordinals if possible #5873 (issue: #5854)
    • Make use of global ordinals in parent/child queries #5846
    • Added a AppendingDeltaPackedLongBuffer-based storage format to single value field data #5706
    • Remove AtomicFieldData.isValuesOrdered. #5688
    • Add global ordinals #5672
    • Moved the decision to load p/c fielddata eagerly to a better place. #5569
  • Geo:
    • Add merge conflicts to GeoShapeFieldMapper #10533 (issues: #10513, #10514)
    • Coordinates can contain more than two elements (x,y) in GeoJSON parser. #9542 (issue: #9540)
    • Revert "[GEO] Update GeoPolygonFilter to handle ambiguous polygons" #9463 (issues: #5968, #9304, #9339, #9462)
    • Update GeoPolygonFilter to handle polygons crossing the dateline #9339 (issues: #5968, #8672, #9304)
    • GeoPolygonFilter not properly handling dateline and pole crossing #9171 (issue: #5968)
    • Removing unnecessary orientation enumerators #9036 (issues: #8978, #9035)
    • Add optional left/right parameter to GeoJSON #8978 (issue: #8764)
    • Feature/Fix for OGC compliant polygons failing with ambiguity #8762 (issue: #8672)
    • Fixes BoundingBox across complete longitudinal range #7340 (issue: #5218)
    • Adds support for GeoJSON GeometryCollection #7123 (issue: #2796)
    • Added caching support to geohash_filter #6478
    • Allow to parse lat/lon as strings and coerce them #5626
    • Improve error detection in geo_filter parsing #5371 (issue: #5370)
    • Improve geo distance accuracy #5192
    • Add support for distances in nautical miles #5088 (issue: #5085)
  • Highlighting:
    • Make the HighlightQuery class public #6446
    • Highlight fields in request order #6178 (issue: #4649)
  • Index APIs:
    • Add check to MetaData#concreteIndices to prevent NPE #10342 (issue: #10339)
    • Set maximum index name length to 255 bytes #8158 (issue: #8079)
    • Add wait_if_ongoing option to _flush requests #6996
    • Unified MetaData#concreteIndices methods into a single method that accepts indices (or aliases) and indices options #6169
    • Fix detection of unsupported fields with validate API #5782 (issue: #5685)
  • Index Templates:
    • Provide template usage information on index creation #8646 (issue: #7421)
    • Unified PUT/POST behaviour in relation to create parameter #6429
  • Indexed Scripts/Templates:
    • Indexed scripts/templates: return response body when script is not found #10396 (issue: #7325)
    • Make sure headers are handed over to internal requests and streamline versioning support #7569
    • Use preference("_local") on get calls. #7477
    • Indexed Scripts/Templates: Return error message on 404 #7335 (issue: #7325)
  • Inner Hits:
    • Nested parent field should be resolved based on the parent inner hit definition #9251 (issue: #8153)
    • Added another more compact syntax for inner hits. #8770
  • Internal:
    • Remove reroute with no reassign #11804
    • Mark store as corrupted instead of deleting state file on engine failure #11769
    • Wait forever (or one day) for indices to close #10833 (issue: #10680)
    • Don't create a new BigArrays instance for every call of withCircuitBreaking #10800 (issue: #10798)
    • Change BigArrays to not extend AbstractComponent #10798
    • CommitStats doesn't need to allow for null values in commit user data #10774 (issue: #10687)
    • Add fairness option to KeyedLock. #10703
    • Fix string comparisons #10204
    • Remove unsafe field in BytesStreamInput #10157
    • Make assert less strict to ensure local node is not null #10076
    • Use provided cluster state for indices service validations #10014
    • Some more simple fs cleanups. #9827
    • Fix errors reported by error-prone #9817
    • Removed 'Master' from names.txt #9698 (issue: #9687)
    • Introduce TimedPrioritizedRunnable base class to all commands that go into InternalClusterService.updateTasksExecutor #9671 (issues: #8077, #9354)
    • Add beforeIndexAddedToCluster callback #9514
    • Add AliasesRequest interface to mark requests that manage aliases #9460
    • ClusterInfoService should wipe local cache upon unknown exceptions #9449
    • Fix shard state tranport action names #9440 (issue: #7105)
    • Minor hygiene, Removed Redundant inheritance #9427
    • Remove includeExisting flag from adding ObjectMapper and FieldMapper listeners #9184
    • Reduce the size of the search thread pool. #9165 (issue: #9135)
    • Assert that we do not call blocking code from transport threads #9164
    • Remove reduced stack size and use JVM default instead #9158 (issue: #9135)
    • Remove IndexCloseListener & Store.OnCloseListener #9009 (issues: #8436, #8608)
    • Remove circular dependency between IndicesService and IndicesStore #8918
    • Remove some Internal* abstractions #8904
    • Make indexQueryParserService available from ParseContext #8252 (issue: #8248)
    • Allow to configure custom thread pools #8247
    • Expose concurrency_level setting on all caches #8112 (issue: #7836)
    • Resiliency: Be more conservative if index.version.created is not set #8018
    • Split internal fetch request used within scroll and search #7870 (issues: #6933, #7319, #7856)
    • Never send requests after transport service is stopped #7862
    • Split internal free context request used after scroll and search #7856 (issues: #6933, #7319)
    • Clarify when a shard search request gets created to be only used locally #7855
    • Add a listener thread pool #7837
    • Remove unused ForceSyncDirectory #7804
    • Force execution of delete index requests #7799
    • Check if from + size don't cause overflow and fail with a better error #7778 (issue: #7774)
    • Make sure that internally generated percolate request re-uses the original headers and request context #7767
    • Make sure that update internal requests share the same original headers and request context #7766
    • Make sure that all delete mapping internal requests share the same original headers and context #7736
    • Added scrollId/s setters to the different scroll requests/responses #7722
    • Make sure that original headers are used when executing search as part of put warmer #7711
    • Refactor copy headers mechanism to not require a client factory #7675 (issue: #7594)
    • In thread pools, use DirectExecutor instead of deprecated API #7636
    • Change LZFCompressedStreamOutput to use buffer recycler when allocating encoder #7613
    • Introduced a transient context to the rest request #7610
    • Refactor copy headers mechanism in REST API #7594 (issue: #6513)
    • Deduplicate useful headers that get copied from REST to transport layer #7590
    • Extract a common base class for (Master|Nodes)FaultDetection #7512 (issue: #7493)
    • Removing useless methods and method parameters from ObjectMapper.java and TypeParsers.java #7474 (issue: #7271)
    • Extended ActionFilter to also enable filtering the response side #7465
    • Removed ConcurrentHashMapV8 #7392 (issues: #6400, #7296)
    • Move index templates api back to indices category and make put template and create index implement IndicesRequest #7378
    • Make sure that multi_search request hands over its context and headers to its corresponding search requests #7374
    • Make sure that multi_percolate request hands over its context and headers to its corresponding shard requests #7371
    • Clarify XContentParser/Builder interface for binary vs. utf8 values #7367
    • Get request while percolating existing documents to keep around headers and context of the original percolate request #7333
    • Auto create index to keep around headers and context of the request that caused it #7331
    • Switch to fixed thread pool by default for management threads #7320 (issue: #7318)
    • Make sure that all shard level requests hold the original indices #7319
    • Refactored TransportMessage context #7303
    • Made it possible to disable the main transport handler in TransportShardSingleOperationAction #7285
    • Adjusted BroadcastShardOperationResponse subclasses visibility #7255
    • Add some @Nullable annotations and fix related compilation warnings. #7251
    • Adjusted visibility for BroadcastShardOperationRequest subclasses and their constructors #7235
    • Changed every single index operation to not replace the index within the original request #7223
    • Adjusted TermVectorRequest serialization to not serialize and de-serialize the index twice #7221
    • Refactored TransportSingleCustomOperationAction, subclasses and requests #7214
    • Removed needless serialization code from TransportIndexReplicationAction and corresponding request object #7211
    • Added transient header support for TransportMessage #7187
    • Check for null references that may be returned due to concurrent changes or inconsistent cluster state #7181
    • Better categorization for transport actions #7105
    • Added a cli infrastructure #7094
    • Introduced the notion of a FixedBitSetFilter that guarantees to produce a FixedBitSet #7037 (issue: #7031)
    • Remove use of recycled set in filters eviction #7012
    • Refactor TransportActions #6989
    • Expose the indices names in every action relates to if applicable #6933
    • Rename FieldMapper.termsFilter to fieldDataTermsFilter. #6888
    • Make XContentBuilder Releasable #6869
    • Remove (mostly) unused 'failure' member from ShardSearchFailure. #6861 (issue: #6837)
    • Use KeyedLock in IndexFieldDataService #6855
    • Cleanup of the transport request/response messages #6834
    • Don't replace indices within ActionRequest and check blocks against concrete indices #6777 (issues: #1, #2)
    • Separate parsing implementation from setter in SearchParseElement #6758 (issue: #3602)
    • Remove intern calls on FieldMapper#Names for better performance #6747
    • Disable explicit GC by default #6637
    • Make sure we don't reuse arrays when sending an error back #6631
    • Wrap RateLimiter rather than copy RateLimitedIndexOutput #6625
    • Re-shade MVEL as a dependency #6570
    • Copy the headers from REST requests to the corresponding TransportRequest(s) #6513 (issue: #6464)
    • Better default size for global index -> alias map #6504
    • Use ConcurrentHashMapV8 for lower memory overhead #6400
    • Made base64 decode parsing to detect more errors #6348 (issue: #6334)
    • Change the default type of the page recycler to CONCURRENT instead of SOFT_CONCURRENT. #6320
    • Some minor cleanups #6210
    • Remove SoftReferences from StreamInput/StreamOutput #6208
    • Use t-digest as a dependency. #6142
    • Add support for Byte and BytesRef to the XContentBuilder #6127
    • Remove unused dump infra #6060
    • Made it mandatory to specify IndicesOptions when calling MetaData#concreteIndices #6059
    • Limit the number of bytes that can be allocated to process requests. #6050
    • Fix code typo in FieldSortBuilder.java #5937
    • Improved bloom filter hashing #5901
    • Field data diet. #5874
    • Cleanup FileSystemUtils #5806
    • Make writePrimitive*() and readPrimitive*() methods public. #5710
    • LongHash add/key not consistent #5693
    • Releasable bytes output + use in transport / translog #5691
    • Make Releasable extend AutoCloseable. #5689
    • Replaces usage of StringBuffer with StringBuilder #5606 (issue: #5605)
    • Internally manipulate the terms execution hint as an enum instead of a String. #5530
    • Let ByteArray/BigByteArray.get() indicate whether a byte[] was materialized. #5529
    • BytesReference.Helper should never materialize a byte[] array. #5517
    • Clean the query parse context after usage #5475
    • BytesReference usage to properly work when hasArray is not available #5455
    • MulticastChannel returned wrong channel in shared mode #5441
    • New class PagedBytesReference: BytesReference over pages #5427 (issue: #5420)
    • Rewrite BytesStreamOutput on top of BigArrays/ByteArray. #5331 (issue: #5159)
    • Add tracking of allocated arrays. #5264
    • Remove thread local recycler #5254
    • Recycler: better lifecycle control for pooled instances #5217 (issue: #5214)
    • Remove useless URL instantiation #5206
    • Variable renamings to reduce unnecessary variable naming diversity #5075
    • Add RamUsageEstimator#sizeOf(Object) to forbidden APIs #4975
    • Remove redundant version checks in transport serialisation #4731
  • Java API:
    • Unify SearchResponse and BroadcastOperationResponse code around shards header #11064
    • Remove duplicated consistency level and replication type setters #10188
    • Package private getters to become public if there have corresponding public setters #9273
    • Added utility method #8594
    • Enabled overriding the request headers in the clients #8258
    • Add indices setter to IndicesRequest interface #7734
    • Mark transport client as such when instantiating #7552
    • Allow nullable queryBuilder in FilteredQueryBuilder to match rest api #7398 (issue: #7365)
    • Some PercolateRequest "setters" allow for method chaining, some don't #7294
    • Throw IllegalStateException if you try to .addMapping for same type more than once #7243 (issue: #7231)
    • XContentBuilder.map(Map) method modified to use a wildcard for value's type. #7212
    • Add suggestRequest to Requests and fix broken javadocs in client #7207 (issue: #7206)
    • Add index, type and id to ExplainResponse #7201
    • Add a blocking variant of close() method to BulkProcessor #6586 (issues: #4158, #6314)
    • Client intermediate interfaces removal follow-up #6563 (issue: #6517)
    • TransportClient: Improve logging, fix minor issue #6376
    • flush method for BulkProcessor class. #5575 (issue: #5570)
    • Add BoolFilterBuilder#hasClauses to be consitent with BoolQueryBuilder #5476 (issue: #5472)
    • Allow iteration over MultiGetRequest#Item instances #5470 (issue: #3061)
    • Java API does not have a way to set global highlighting settings #5281
  • Logging:
    • Add -XX:+PrintGCDateStamps when using GC Logs #11735 (issue: #11733)
    • Display low disk watermark to be consistent with documentation #11313 (issue: #10588)
    • Add index name to log statements when settings update fails #11124
    • Add logging of slow cluster state tasks #10907 (issue: #10874)
    • Log sending translog operation batches to nodes #10544
    • Log only a summary line of filesystem detail for all path.data on node startup #10527 (issue: #10502)
    • Add INFO logging saying whether each path.data is on an SSD #10502
    • Use static logger name in Engine.java #10497
    • Miscellaneous additional logging and cleanups #10376
    • Fix logging a RoutingNode object, log an object with a good .toString instead #9863
    • Logging: improve logging messages added in #9562 #9603 (issue: #9562)
    • Change logging to warning to match pattern #9593
    • Add logging around gateway shard allocation #9562
    • Added a simple request tracer, logging incoming and outgoing Transport requests #9286
    • Reduce apache (cloud-aws) logging when rootLogger is DEBUG #8856
    • Log byte and doc size for slow merges #8855 (issue: #8853)
    • Separately log file deletions #8662 (issue: #8603)
    • Clarify index removal log message #8641
    • Add log4j-extras dependency #8464 (issue: #7927)
    • Log how long IW.rollback took, and when MockFSDir starts its check index #8388
    • Change log level for mpercolate #8306
    • Suppress long mapping logging during mapping updates (unless in TRACE) #7949
    • Add log4j-extras dependency #7947 (issue: #7927)
    • Boostrap: Log startup exception to console if needed and to file as error #6581
    • Log script change/add and removal at INFO level #6104
    • Include thread name when logging IndexWriter's infoStream messages #5973
    • Tie in IndexWriter's infoStream output to "lucene.iw" logger with level=TRACE #5934 (issue: #5891)
    • Be less verbose logging ClusterInfoUpdateJob failures #5222
    • Togglable stacktrace display #5187 (issues: #5102, #5103)
    • Add EnhancedPatternLayout to logging.yml options #4991
  • Mapping:
    • Add enabled flag for _field_names to replace disabling through index:no #9893
    • Add ignore_missing option to timestamp #9104 (issues: #8882, #9049)
    • Include currentFieldName into ObjectMapper errors #9020
    • Make lookup structures immutable. #7486
    • Report conflict when merging _all field mapping and throw exception when doc_values specified #7377 (issue: #777)
    • Enforce non-null settings. #7032
    • Control whether MapperService docMapper iterator should contain DEFAULT_MAPPING #6793
    • Call callback on actual mapping processed #6748
    • Improve performance for many new fields introduction in mapping #6707
    • Better logic on sending mapping update new type introduction #6669
    • Wait for mapping updates during local recovery #6666
    • Check if root mapping is actually valid #6093 (issues: #4483, #5864)
    • Support empty properties array in mappings #6006 (issue: #5887)
    • Update default precision step, modulo tests #5908 (issue: #5905)
    • Add lucene LMSimilarity #5701 (issue: #5697)
    • Support externalValue() in mappers #4986
    • Norms disabling on existing fields #4813
  • More Like This:
    • Support for when all fields are deprecated #8067
    • Replace percent_terms_to_match with minimum_should_match #7898
    • Default to all possible fields for items #7382
    • Switch to using the multi-termvectors API #7014
    • Fetch text from all possible fields if none are specified #6740
    • Ensure selection of best terms is indeed O(n) #6657
    • Create only one MLT query per field for all queried items #6404
    • Add the ability to specify the analyzer used for each Field #6329
    • Added syntax for single item specification. #6311
    • Values of a multi-value fields are compared at the same level #6310
    • Allow for both like_text and docs/ids to be specified. #6246
    • Added the ability to include the queried document for More Like This API. #6067
    • Fix behavior on default boost factor for More Like This. #6021
    • Added searching for multiple similar documents #5857 (issue: #4075)
    • Upgrade to Jackson 2.4.1.1 #6789
    • Upgrade to mvel 2.2.0.Final #5877
  • Nested Docs:
    • Added nested scope to parse context that keeps track the current nested level during search request parsing #9692 (issue: #9305)
    • Fix support for named filters/queries inside nested filters. #6293
  • Network:
    • Default value for socket reuse should not be null #11255
    • Make Netty exceptionCaught method protected #10464
    • Remove content thread safe from REST layer #10429
    • Add getter for channel in NettyTransportChannel #10319
    • Schedule transport ping interval #10189
    • Return useful error message on potential HTTP connect to Transport port #10108 (issue: #2139)
    • Change access modifiers to protected in Netty HTTP Transport #9724
    • Add profiles to Netty transport infos #9134
    • Support binding on multiple host/port pairs #8098
    • Chunk direct buffer usage by networking layer #7811
    • Make sure channel closing never happens on i/o thread #7726
    • Support "default" for tcpNoDelay and tcpKeepAlive #7136 (issue: #7115)
    • Refactoring to make MessageChannelHandler extensible #6915 (issue: #6889)
    • Refactoring to make Netty MessageChannelHandler extensible #6889
    • Improve large bytes request handling by detecting content composite buffer #6756
    • Use loopback when localhost is not resolved #5719
  • Packaging:
    • Export hostname as environment variable for plugin manager #11399 (issues: #10902, #9474)
    • Exclude jackson-databind dependency #10924
    • Add common SystemD file for RPM/DEB package #10725
    • Standardization of packages structure and install #10595 (issue: #10330)
    • Add properties files to configure startup and installation scripts #10330
    • service.bat file should explicitly use the Windows find command. #9532
    • Export the hostname as environment variable #9474 (issue: #8470)
    • Windows: makes elasticsearch.bat more friendly to automated processes #9160 (issue: #8913)
    • Shutdown: Add support for Ctrl-Close event on Windows platforms to grace... #8993
    • Packaging: Add java7/8 java-package paths to init script #8815 (issue: #7383)
    • Check if proc file exists before calling sysctl #8793 (issue: #4978)
    • deb: add systemd service config for upcoming Jessie #8765 (issue: #8493)
    • bin/elasticsearch: add help, fix endless loop #8729 (issues: #2168, #7104)
    • Introduce elasticsearch.in.bat (i.e. es.in for Windows) #8244 (issue: #8237)
    • Make .zip and .tar.gz release artifacts contain same files #7578 (issue: #2793)
    • Add default oracle jdk 7 (x64) path to JDK_DIRS #7132
    • Prevent init script from returning when the service isn't actually started #6909
    • Windows: Modify command window title (windows) #6752 (issue: #6336)
    • Remove java-6 directories from debian init script #6350
    • Reset locale to C in bin/elasticsearch #6047
    • Remove spaces from commented config lines in elasticsearch.yml and logging.yml #5842
    • Use the new command line syntax in the init script #5033
    • Startup: Add ES_HOME to ES_INCLUDE search path #4958
    • Mark 'lucene-expression' as 'provided' in pom.xml #4861 (issues: #4858, #4859)
    • Move systemd files from /etc to /usr/lib #4029
  • Parent/Child:
    • Reduce memory usage in top children query #8165
    • Adding min score mode to parent-child queries #7771 (issue: #7603)
    • Support min_children and max_children on has_child query/filter #6019
    • Fix P/C assertions for rewrite reader #5731
    • Migrated p/c queries from id cache to field data. #4878 (issue: #4930)
  • Percolator:
    • Introduce index option named 'index.percolator.map_unmapped_fields_as_string' #9054 (issues: #9025, #9053)
    • Percolator should cache index field data instances. #7081 (issue: #6806)
    • Reuse IndexFieldData instances between percolator queries #6845 (issue: #6806)
    • Add MemoryIndex reuse when percolating doc with nested type #5332
  • Plugins:
    • Use of default CONF_DIR/CONF_FILE in plugin install #10721 (issues: #10673, #7946)
    • Always send current ES version when downloading plugins #10131
    • Add executable flag to every file in bin/ after install #7177
    • Lucene version checker should use Lucene.parseVersionLenient #7056
    • Introduced pluggable filter chain to be able to filter transport actions execution #6921
    • Enables plugins to define default logging configuration for their needs. #6805 (issue: #6802)
    • bin/plugin tests for missing plugin name when passing --url #6013 (issues: #5976, #5977)
    • Check plugin Lucene version #4984
    • Serving _site plugins do not pick up on index.html for sub directories #4850 (issue: #4845)
    • Add version to plugins #4378 (issues: #2668, #2784)
  • Query DSL:
    • Function score: Add default to field_value_factor #10845 (issue: #10841)
    • Add support for minimum_should_match to simple_query_string #9864 (issue: #6449)
    • Raise an exception on an array of values being sent as the factor for a field_value_factor query #9246 (issue: #7408)
    • function_score: use query and filter together #8675 (issue: #8638)
    • Add option for analyzing wildcard/prefix to simple_query_strinq #8422 (issue: #787)
    • Expose max_determinized_states in regexp query, filter #8384 (issue: #8357)
    • FunctionScore: RandomScoreFunction now accepts long, as well a strings. #8311 (issue: #8267)
    • Be stricter parsing ids for ids query #7945 (issue: #7686)
    • Function Score: Add optional weight parameter per function #7137 (issue: #6955)
    • Add time zone setting for relative date math in range filter/query #7113 (issue: #3729)
    • Add support for the _name parameter to the simple_query_string query #6979
    • Function score parser should throw exception if both functions:[] and single function given #5995
    • Refactor SimpleQueryParser settings into separate Settings class, add "lenient" option #5208 (issue: #5011)
    • Throw parsing exception if terms filter or query has more than one field #5137 (issue: #5014)
    • Add "locale" parameter to query_string and simple_query_string #5131 (issue: #5128)
    • Add support for lowercase_expanded_terms flag to simple_query_string #5126 (issue: #5008)
    • Add fuzzy/slop support to simple_query_string #4985
    • Range filter no cache behaviour for now with rounding #4955 (issues: #4846, #4947)
    • Expose dist/pre/post options for SpanNotQuery #4452
  • REST:
    • Create Snapshot: remove _create from POST path to match PUT #11928 (issue: #11897)
    • Add rewrite query parameter to the indices.validate_query API spec #11580 (issue: #10147)
    • Unify query_string parameters parsing #11057
    • HttpServer: Support relative plugin paths #10975 (issue: #10958)
    • Remove global source parameter from individual APIs in REST spec #10863
    • Add option to only return simple exception messages #10117
    • Better detection of CBOR #10026 (issue: #7640)
    • Expose master_timeout flag on GET _template & HEAD _template #9688
    • Add support for multi-index query parameters for _cluster/state #9295 (issue: #5229)
    • Remove indices_boost URL param #9244 (issue: #6281)
    • Adds parameters to API endpoint cluster put settings specification #8769
    • Changed the root rest endpoint ('/') to use cluster service #7933 (issue: #7899)
    • Add the cluster name to the "/" endpoint #7524
    • A content decompressor that throws a human readable message when #7241
    • Added missing percolate API parameters to the rest spec #7173
    • Add REST API spec for /_search_shards endpoint #5907
    • Rest layer refactoring phase 2 + recycling in http layer #5708
    • Add explain flag support to the reroute API #5027 (issues: #2483, #5169)
    • Throw exception if an additional field was placed inside the "query" body #4913 (issue: #4895)
    • REST API: Consistent get field mapping response #4822 (issue: #4738)
    • Rest API: Ensure 503 signals == retry on another node #4066
  • Recovery:
    • Reduce cluster update reroutes with async fetch #11421
    • Check if the index can be opened and is not corrupted on state listing #11269 (issue: #11226)
    • Allow to recover into a folder containing a corrupted shard #10558
    • Only cancel recovery when primary completed relocation #10218
    • Engine: update index buffer size during recovery and allow configuring version map size #10046 (issues: #6363, #6667)
    • Unify RecoveryState management to IndexShard and clean up semantics #9902 (issue: #9503)
    • Only iterate the files that we recovered from the commit #9761
    • Add a timeout to local mapping change check #9575
    • Node shut down during the last phase of recovery needlessly fails shard #9496
    • Flush immediately after a remote recovery finishes (unless there are ongoing ones) #9439
    • Don't throttle recovery indexing operations #9396 (issue: #9394)
    • Release store lock before blocking on mapping updates #9102
    • Ensure shards are identical after recovery #8723
    • Be more resilient to partial network partitions #8720
    • Throw IndexShardClosedException if shard is closed #8648
    • Refactor RecoveryTarget state management #8092 (issues: #7315, #7893)
    • During recovery, mark last file chunk to fail fast if payload is truncated #7830
    • Remove unneeded waits on recovery cancellation #7717
    • Set a default of 5m to recover_after_time when any of the expected*Nodes is set #6742
    • Add a best effort waiting for ongoing recoveries to cancel on close #6741
    • Cancel recovery if shard on the target node closes during recovery operation #6645
    • RecoveryID should not be a per JVM but per Node #6207
    • Before deleting a local unused shard copy, verify we're connected to the node it's supposed to be on #6191
    • Change default recovery throttling to 50MB / sec #5913
    • Fail replica shards locally upon failures #5847 (issue: #5800)
  • Scripting:
    • Allow plugins to define custom operations that they use scripts for #10419 (issue: #10347)
    • Add String to the default whitelisted receivers #9837 (issue: #8866)
    • Make script.groovy.sandbox.method_blacklist_patch truly append-only #9473
    • Make groovy sandbox method blacklist dynamically additive #9470
    • Add explainable script again #8665 (issues: #7245, #8561)
    • Add GroovyCollections to the sandbox whitelist #7250 (issues: #7088, #7089)
    • Make ScoreAccessor utility class publicly available for other script engines #6898 (issue: #6864)
    • Remove setNextScore in SearchScript. #6864
    • Add a transformer to translate constant BigDecimal to double #6609
    • Add Groovy sandboxing for GString-based method invocation #6596
    • Fix optional default script loading #6582
    • Exposed _uid, _id and _type fields as stored fields (_fields notation) #6406
  • Search:
    • Search preference based on node specification #11464 (issue: #5925)
    • Single value numeric queries shouldn't be handled by NumericRangeQuery #10648 (issue: #10646)
    • Parse terms filters on a single term as a term filter. #9014
    • Close active search contexts on SearchService#close() #8947 (issue: #8940)
    • Reduce memory usage during fetch source sub phase #8138
    • Don't let took be negative. #7968
    • Switch to Lucene QueryRescorer #7707 (issue: #6232)
    • Use FixedBitSetFilterCache for delete-by-query #7581 (issue: #7037)
    • Speed up string sort with custom missing value #7005
    • Wrap filter only once in ApplyAcceptedDocsFilter #6873
    • Remove Queries#optimizeQuery - already handled in BooleanQuery #6743
    • Improve 404 on missing scroll id #6063 (issue: #6040)
    • Return missing (404) if a scroll_id is cleared that no longer exists. #5865 (issue: #5730)
    • Speed up exists and missing filters on high-cardinality fields #5659
    • Freq terms enum #5597
    • Capture and set start time in Delete By Query operations #5540
    • Add dedicated /_search/template endpoint for query templates #5353
    • Add failures reason to delete by query response #5095 (issue: #5093)
    • Use patched version of ReferenceManager to prevent infinite loop in ReferenceManager#acquire() #5043
    • Improve scroll search by using IndexSearcher#searchAfter(...) #4968 (issue: #4940)
  • Settings:
    • Add node setting to send SegmentInfos debug output to System.out #11546
    • Add http.publish_port setting to the HTTP module #8807 (issue: #8137)
    • Don't accept a dynamic update to min_master_nodes which is larger then current master node count #8321
    • Validates bool values in yaml for node settings #8186 (issue: #8097)
    • Security: Disable CORS by default #7642 (issue: #7151)
    • Store index creation time in index metadata #7218 (issue: #7119)
    • Make cluster.routing.allocation.allow_rebalance a dynamic setting #7095 (issue: #7092)
    • Security: Allow to configure CORS allow-credentials header to work via SSL #7059 (issue: #6380)
    • Allow index.merge.scheduler.max_thread_count to be dynamically changed #6925 (issue: #6882)
    • Security: Support regular expressions for CORS allow-origin to match against #6923 (issues: #5601, #6891)
    • Added three frequency levels for resource watching #6896
    • Added more utility methods to Settings #6840
    • Improve Settings#get lookup for camel case support #6765
    • Security: Make JSONP responses optional. #6164
    • Allow to change concurrent merge scheduling setting dynamically #6098
    • Throw error when incorrect setting applied to auto_expand_replicas#5752
    • Add getAsRatio to Settings class, allow DiskThresholdDecider to take percentages #5690
    • Corrected issue with throttle type setting not respected upon updates #5392
    • Made possible to dynamically update discovery.zen.publish_timeout cluster setting #5068 (issue: #5063)
  • Shadow Replicas:
    • Implement retries for ShadowEngine creation #10688 (issue: #10637)
    • Allow relocating primary shards on shared filesystems without failing the shard #10469
    • Allow primary promotion on shadow replica without failing the shard #9786
  • Snapshot/Restore:
    • Add validation of snapshot FileInfo during parsing #12108
    • Add snapshot name validation logic to all snapshot operations #11617
    • Check that reading indices is allowed before creating their snapshots #11133
    • Don't throw an exception if repositories are unregistered with * #11113
    • Improve the error message when attempting to snapshot a closed index #10608 (issue: #10579)
    • AbstractBlobContainer.deleteByPrefix() should not list all blobs #10366 (issue: #10344)
    • Batching of snapshot state updates #10295
    • Expand wildcards in snapshot #9903 (issue: #6097)
    • Refactor how restore cleans up files after snapshot was restored #9770
    • Add support for changing index settings during restore process #9285 (issue: #7887)
    • Override write(byte[] b, int off, int len) in FilterOutputStream for better performance #8749 (issue: #8748)
    • Allow custom metadata to specify whether or not it should be in a snapshot #7901 (issue: #7900)
    • Add repository validation #7680 (issue: #7096)
    • Write Snapshots directly to the blobstore stream #7637
    • It should be possible to restore an index without restoring its aliases #6457
    • Snapshot/Restore: Add ability to restore partial snapshots #6368 (issue: #5742)
    • Switch to shared thread pool for all snapshot repositories #6182 (issue: #6181)
    • Improve speed of running snapshot cancelation #5244 (issue: #5242)
    • Add ability to get snapshot status for running snapshots #5123 (issue: #4946)
    • Add throttling to snaphost and restore operations #4891 (issue: #4855)
  • Stats:
    • Add CommitStats to supply information about the current commit point #10687
    • Add throttle stats to index stats and recovery stats #10097
    • Recovery: add total operations to the _recovery API #10042 (issue: #9368)
    • Add pending tasks count to cluster health #9877
    • Add ignore_idle_threads (default: true) to hot threads #8985 (issue: #8908)
    • Add time in index throttle to index stats. #7896 (issue: #7861)
    • Add segments.index_writer_max_memory to stats #7440 (issues: #6483, #7438)
    • Track the number of times the CircuitBreaker has been tripped #6134 (issue: #6130)
    • Remove field names in stats url #6054 (issue: #5671)
    • Add suggest stats #5563 (issue: #4032)
    • Disable RAM usage estimation on Lucene 3.x segments. #5202 (issue: #5201)
  • Store:
    • Consolidate directory lock obtain code #11390
    • Schedule pending delete if index store delete fails #9856
    • Improve safety when deleting files from the store #9801
    • Add simple cache for StoreStats #9709 (issue: #9683)
    • Use Directory#fileLength() less during calculating checksums #9689
    • Cache fileLength for fully written files #9683
    • Synchronize operations that modify file mappings on DistributorDirectory #8408
    • Cut over MetaDataStateFormat to NIO Path API #8297
    • Remove special file handling from DistributorDirectory #8276
    • Try to increment store before searcher is acquired #7792
    • Fold two hashFile implemenation into one #7720
    • Before deleting shard verify that another node holds an active shard instance #6692
    • Make a hybrid directory default using mmapfs / niofs #6636
  • Suggesters:
    • Phrase Suggester Collate Enhancements #10710 (issue: #9377)
    • Collate option in PhraseSuggester should allow returning phrases with no matching docs #6929 (issue: #6927)
    • Speed up phrase suggestion scoring #5396 (issue: #5395)
  • Term Vectors:
    • Only load term statistics if required #11737
    • Requests are now timed #9583
    • Add support for realtime term vectors #7846
    • Support for custom analyzers in term vectors and MLT query #7801
    • Support for artificial documents #7530
    • Return found: false for docs requested between index and refresh #7124 (issue: #7121)
    • Adds support for wildcards in selected fields #7061
    • Compute term vectors on the fly if not stored in index #6567 (issue: #5184)
  • Top Hits:
    • In top_hits agg, when parent doc is found stop iterating over segments and continue with the next top matching child doc #6895 (issue: #1843)
    • Added pagination support to top_hits aggregation by adding from option #6312 (issue: #6299)
  • Translog:
    • Use buffered translog type also when sync is set to 0 #10993
    • Remove useless random translog directory selection #10589
    • Handle truncated translog gracefully #9797 (issue: #9699)
    • Refactor the Translog.read(Location) method #7780
    • Write translog opSize twice #7735
    • Remove unused stream #7683
    • Clean up translog interface #7564
    • Add translog checksums #7232 (issue: #6554)
    • Set default translog flush_threshold_ops to unlimited, to flush by byte size by default and not penalize tiny documents #6783 (issue: #6443)
    • Use unlimited flush_threshold_ops for translog (again) #6726
    • Raise proper failure if not fully reading translog entry #6562
    • Use unlimited flush_threshold_ops for translog #5900
    • Fix visibility in buffered translog #5609
    • Use BytesReference to write to translog files #5463
    • Don't throttle the translog stage of recovery #4890
  • Tribe Node:
    • Index level blocks, index conflict settings #5501
  • Upgrade API:
    • Refactor upgrade API to use transport and write minimum compatible version that the index was upgraded to #11333 (issues: #11072, #11095)
    • Add upgrade_only_ancient_segments option to upgrade API #10540 (issue: #10213)
    • Remove ability to run optimize and upgrade async #9640 (issue: #9638)
    • Log when upgrade API starts and stops #9229 (issue: #9227)
  • Bug fixes

    • Aggregations:
      • Fix cidr mask conversion issue for 0.0.0.0/0 and add tests #12005 #12430 (issue: #12005)
      • Allow aggregations_binary to build and parse #11473 (issue: #11457)
      • Fix geo bounds aggregation when longitude is 0 #11090 (issue: #11085)
      • Fixes Infinite values return from geo_bounds with non-zero bucket-ordinals