Elasticsearch 1.4.0.Beta2
Release Notes
Breaking changes
- Aggregations:
- Only returned on the first page with scroll and forbidden with scan #7497 (issue: #1642)
- Bulk UDP:
- Circuit Breaker:
- Add HierarchyCircuitBreakerService #6739 (issue: #6129)
- Fielddata:
- Remove soft/resident caches. #7443
- Internal:
- Simplify reading / writing from and to BlobContainer #7551
- Refactor guice startup #7289
- Fixed filters execution order and fix potential concurrency issue in filter chains #7023 (issues: #7019, #7021)
- streamline use of IndexClosedException introduced with #6475 #6990 (issues: #6475, #6988)
- Java API:
- Remove operationThreaded setter from ExplainRequestBuilder #7186
- Mapping:
- Remove unsupported postings_format / doc_values_format #7604 (issues: #7238, #7566)
- REST API:
- Removed support for aliases as part of index settings #5545
- Resiliency:
- Forbid index names over 100 characters in length #7252 (issue: #4417)
- Automatic verification of all files that are being snapshotted with Snapshot/Restore #7159 (issue: #5593)
- Scripting:
- Script with _score: remove dependency of DocLookup and scorer #7819 (issues: #7043, #7487)
- Remove MVEL as a built-in scripting language #6610
- Switch to Groovy as the default scripting language #6571 (issue: #6233)
- Settings:
- Add a
index.query.parse.allow_unmapped_fields
setting that fails if queries refer to unmapped fields. #6928 (issue: #6664)
New features
- Aggregations:
- Scriptable Metrics Aggregation #7075 (issue: #5923)
- Added pre and post offset to histogram aggregation #6980 (issue: #6605)
- Added Filters aggregation #6974 (issues: #6118, #6119)
- Add children aggregation #6936
- Analysis:
- Add keep_types for filtering by token type #7120
- Index templates:
- Made template filtering generic and extensible #7454 (issue: #7459)
- Indexing:
- Switch auto-ids to Flake IDs from random UUIDs #7531 (issues: #5941, #6004)
- Indices API:
- Internal:
- Added an option to add arbitrary headers to the client requests #7127
- Java API:
- Mapping:
- Add new
default
option for timestamp field #7036 (issue: #4718)
- More Like This Query:
- Adds the ability to specify the analyzer used for each Field #6329
- Query Cache:
- Support shard level query response caching #7161
- Resiliency:
- Add basic upgrade test script #7074
- Search:
- Add Exists API to check if any matching documents exist for a given query #7026 (issue: #6995)
- Add an option to early terminate document collection when searching/counting #6885 (issue: #6876)
- Significant Terms:
- Add google normalized distance and chi square #6858
- Sorting:
- Allow _geo_distance to handle many to many geo point distance #7097 (issue: #3926)
- Term Vectors:
- Support for artificial documents #7530
- Computes term vectors on the fly if not stored in index #6567 (issue: #5184)
- Test:
- Add ElasticsearchSingleNodeTest. #6874
- Update API:
- Add support for scripted upserts. #7144 (issue: #7143)
- Detect noop updates when using doc #6862 (issue: #6822)
Enhancements
- Aggregations:
- Encapsulate AggregationBuilder name and make getter public #7425
- Merge LongTermsAggregator and DoubleTermsAggregator. #7279
- Remove the logic to optionally sort/dedup values on the fly. #7276
- Make the list of buckets for terms and histogram returned as a java.util.List. #7275
- Stops direct subclassing of InternalNumericMetricsAggregation #7058
- Better JSON output scoping #6985 (issue: #7004)
- Change to default shard_size in terms aggregation #6960 (issue: #6857)
- Added an option to show the upper bound of the error for the terms aggregation #6778 (issue: #6696)
- Analysis:
- Expose Latvian analyzer #7542
- Discovery:
- Close ping handler's executor service properly #7903
- Give a unique id to each ping response #7769
- UnicastZenPing don't rename configure host name #7747 (issue: #7719)
- node join requests should be handled at lower priority than master election #7733
- not all master election related cluster state update task use Priority.IMMEDIATE #7718
- back port #7558 to 1.x and add bwc protections of the new ping on master gone introduced in #7493 #7694 (issue: #7558)
- Fielddata:
- Remove custom comparators and use Lucene's instead #6981 (issue: #5980)
- Flush:
- Add
wait_if_ongoing
option to _flush requests #6996
- Function Score:
- Add optional weight parameter per function #7137 (issue: #6955)
- Add missing whitespace in error message when throwing exception #6155
- Geo:
- Adds support for GeoJSON GeometryCollection #7123 (issue: #2796)
- Get API:
- Allow GET access to _all field (return value was always null before) #6924
- Indexed Scripts/Templates:
- Make sure headers are handed over to internal requests and streamline versioning support #7569
- Use preference("_local") on get calls. #7477
- Indexing:
- Internal:
- upgrade to Lucene 4.10.1 release #7905
- split internal fetch request used within scroll and search #7870 (issues: #6933, #7319, #7856)
- Never send requests after transport service is stopped #7862
- split internal free context request used after scroll and search #7856 (issues: #6933, #7319)
- Upgrade to Lucene 4.10.1 snapshot #7844
- Add a listener thread pool #7837
- Use internal time estimation for time limited search collector #7833
- Chunk direct buffer usage by networking layer #7811
- remove ForceSyncDirectory #7804
- Try to increment store before searcher is acquired #7792
- Check if from + size don't cause overflow and fail with a better error #7778 (issue: #7774)
- make sure that internally generated percolate request re-uses the original headers and request context #7767
- make sure that update internal requests share the same original headers and request context #7766
- Add missing cluster blocks handling for master operations #7763 (issue: #7740)
- make sure that all delete mapping internal requests share the same original headers and context #7736
- Write translog opSize twice #7735
- add indices setter to IndicesRequest interface #7734
- Added scrollId/s setters to the different scroll requests/responses #7722
- make sure that original headers are used when executing search as part of put warmer #7711
- Improve exception from Store.failIfCorrupted #7695 (issue: #7596)
- refactor copy headers mechanism to not require a client factory #7675 (issue: #7594)
- In thread pools, use DirectExecutor instead of deprecated API #7636
- Introduced a transient context to the rest request #7610
- Refactor copy headers mechanism in REST API #7594 (issue: #6513)
- Upgrade Guava to 18.0. #7593 (issue: #6268)
- Deduplicate useful headers that get copied from REST to transport layer #7590
- Upgrade to Lucene 4.10 #7584
- Use FixedBitSetFilterCache for delete-by-query #7581 (issue: #7037)
- Extract a common base class for (Master|Nodes)FaultDetection #7512 (issue: #7493)
- Removing useless methods and method parameters from ObjectMapper.java and TypeParsers.java #7474 (issue: #7271)
- Extended ActionFilter to also enable filtering the response side #7465
- Removed ConcurrentHashMapV8 #7392 (issues: #6400, #7296)
- move index templates api back to indices category and make put template and create index implement IndicesRequest #7378
- Make sure that multi_search request hands over its context and headers to its corresponding search requests #7374
- Make sure that multi_percolate request hands over its context and headers to its corresponding shard requests #7371
- Clarify XContentParser/Builder interface for binary vs. utf8 values #7367
- Upgrade to Apache HttpComponents client 4.3.5 #7346 (issue: #7342)
- Get request while percolating existing documents to keep around headers and context of the original percolate request #7333
- Auto create index to keep around headers and context of the request that caused it #7331
- Upgrade to Netty 3.9.3.Final #7328
- DistributorDirectory shouldn't search for directory when reading existing file #7323 (issue: #7306)
- Switch to fixed thread pool by default for management threads #7320 (issue: #7318)
- Make sure that all shard level requests hold the original indices #7319
- Refactored TransportMessage context #7303
- Made it possible to disable the main transport handler in TransportShardSingleOperationAction #7285
- Introduced a new elasticsearch exception family that can hold headers #7269
- Adjusted BroadcastShardOperationResponse subclasses visibility #7255
- Add some @Nullable annotations and fix related compilation warnings. #7251
- Adjusted visibility for BroadcastShardOperationRequest subclasses and their constructors #7235
- changed every single index operation to not replace the index within the original request #7223
- Adjusted TermVectorRequest serialization to not serialize and de-serialize the index twice #7221
- Store index creation time in index metadata #7218 (issue: #7119)
- refactored TransportSingleCustomOperationAction, subclasses and requests #7214
- Removed needless serialization code from TransportIndexReplicationAction and corresponding request object #7211
- Added transient header support for TransportMessage #7187
- Check for null references that may be returned due to concurrent changes or inconsistent cluster state #7181
- Version bump HPPC to 0.6.0 #7139
- Randomized testing version bump #7138
- Support "default" for tcpNoDelay and tcpKeepAlive #7136 (issue: #7115)
- Better categorization for transport actions #7105
- Added a cli infrastructure #7094
- Introduced the notion of a FixedBitSetFilter that guarantees to produce a FixedBitSet #7037 (issue: #7031)
- Add a periodic cleanup thread for IndexFieldCache caches #7015 (issue: #7010)
- Remove use of recycled set in filters eviction #7012
- Refactor TransportActions #6989
- Change numeric data types to use SORTED_NUMERIC docvalues type #6967
- With unicast discovery, only disconnect from temporary connected nodes #6966
- Remove unneeded cluster state serialization during cluster join #6949
- expose the indices names every action relates to if applicable #6933
- Better message for invalid internal transport message format #6916
- Refactoring to make MessageChannelHandler extensible #6915 (issue: #6889)
- Switch fielddata to use Lucene doc values APIs. #6908
- Added three frequency levels for resource watching #6896
- Refactoring to make Netty MessageChannelHandler extensible #6889
- Rename FieldMapper.termsFilter to fieldDataTermsFilter. #6888
- Remove (mostly) unused 'failure' member from ShardSearchFailure. #6861 (issue: #6837)
- Use KeyedLock in IndexFieldDataService #6855
- Separate parsing implementation from setter in SearchParseElement #6758 (issue: #3602)
- Add a best effort waiting for ongoing recoveries to cancel on close #6741
- Java API:
- allow nullable queryBuilder in FilteredQueryBuilder to match rest api #7398 (issue: #7365)
- Some PercolateRequest "setters" allow for method chaining, some don't #7294
- Throw IllegalStateException if you try to .addMapping for same type more than once #7243 (issue: #7231)
- XContentBuilder.map(Map) method modified to use a wildcard for value's type. #7212
- Add index, type and id to ExplainResponse #7201
- Add a blocking variant of close() method to BulkProcessor #6586 (issues: #4158, #6314)
- Mapping:
- Report conflict when merging
_all
field mapping and throw exception when doc_values specified #7377 (issue: #777) - Enforce non-null settings. #7032
- More Like This Query:
- Default to all possible fields for items #7382
- Switch to using the multi-termvectors API #7014
- Fetch text from all possible fields if none are specified #6740
- Added syntax for single item specification. #6311
- Nested:
- Fix support for named filters/queries inside nested filters. #6293
- Netty:
- Make sure channel closing never happens on i/o thread #7726
- NodesFD:
- simplify concurrency control to fully rely on a single map #7889
- Packaging:
- Make .zip and .tar.gz release artifacts contain same files #7578 (issue: #2793)
- Add default oracle jdk 7 (x64) path to JDK_DIRS #7132
- Parent/Child:
- When parent doc is found stop iterating over segments and continue with the next top matching child doc #6895 (issue: #1843)
- Percolator:
- Percolator should cache index field data instances. #7081 (issue: #6806)
- Reuse IndexFieldData instances between percolator queries #6845 (issue: #6806)
- Plugins:
- Add executable flag to every file in bin/ after install #7177
- Lucene version checker should use
Lucene.parseVersionLenient
#7056 - Introduced pluggable filter chain to be able to filter transport actions execution #6921
- Query Cache:
- Add hit and miss count #7355
- Warmer (search) to support query cache #7326
- Add a request level flag to control query cache #7167
- Query DSL:
- Add time zone setting for relative date math in range filter/query #7113 (issue: #3729)
- Add support for the
_name
parameter to the simple_query_string
query #6979
- REST API:
- Changed the root rest endpoint ('/') to use cluster service #7933 (issue: #7899)
- Add the cluster name to the "/" endpoint #7524
- A content decompressor that throws a human readable message when #7241
- Recovery:
- remove unneeded waits on recovery cancellation #7717
- Resiliency:
- During discovery, master fault detection should fall back to cluster state thread upon error #7908 (issue: #7834)
- During discovery, remove any local state and use clusterService.state instead #7834
- During recovery, mark last file chunk to fail fast if payload is truncated #7830
- Update ZenDiscovery fields via the cluster service update task. #7790
- Change LZFCompressedStreamOutput to use buffer recycler when allocating encoder #7613
- Clean up translog interface #7564
- Master election should demotes nodes which try to join the cluster for the first time #7558 (issue: #7493)
- Do not use a background thread to disconnect node which are removed from the ClusterState #7543
- Refactored ClusterStateUpdateTask protection against execution on a non master #7511 (issue: #7493)
- Accumulated improvements to ZenDiscovery #7493 (issue: #2488)
- Verify checksums on merge #7360
- Discovery - UnicastZenPing should also ping last known discoNodes #7336
- Add translog checksums #7232 (issue: #6554)
- Do not use versions to optimize cluster state copying for a first update from a new master #6466
- Scripting:
- Add GroovyCollections to the sandbox whitelist #7250 (issues: #7088, #7089)
- Make ScoreAccessor utility class plublicly available for other script engines #6898 (issue: #6864)
- Remove setNextScore in SearchScript. #6864
- Security:
- Disable CORS by default #7642 (issue: #7151)
- Allow to configure CORS allow-credentials header to work via SSL #7059 (issue: #6380)
- Support regular expressions for CORS allow-origin to match against #6923 (issues: #5601, #6891)
- Settings:
- Make "cluster.routing.allocation.allow_rebalance" a dynamic setting #7095 (issue: #7092)
- Allow
index.merge.scheduler.max_thread_count
to be dynamically changed #6925 (issue: #6882)
- Snapshot/Restore:
- Allow custom metadata to specify whether or not it sho... #7901 (issue: #7900)
- Write Snapshots directly to the blobstore stream #7637
- Add ability to restore partial snapshots #6368 (issue: #5742)
- Sorting:
- Speed up string sort with custom missing value #7005
- Stats:
- Suggesters:
- add suggestRequest to Requests and fix broken javadocs in client #7207 (issue: #7206)
- Collate option in PhraseSuggester should allow returning phrases with no matching docs #6929 (issue: #6927)
- Term Vectors:
- Return found: false for docs requested between index and refresh #7124 (issue: #7121)
- Term Vectors API:
- adds support for wildcards in selected fields #7061
- Test:
- use local random instance rather than thread local version #7665
- Ensure BWC test run even if node.mode=local is set #7660
- Allow SingleNodeTest to reset the node if really needed after test. #7608 (issue: #7550)
- Randomly enable AggressiveOpts in builds #7533
- Unify the randomization logic for number of shards and replicas #7522
- Allow global test cluster to have configurable settings source #7482
- Improve test coverage #7428
- Use a default host name if localAddress is not available #7409
- Use a dedicated port range per test JVM #7396
- Only reset clients on nightly tests #7329
- move plugin dir to plugins dir #7253
- Allow CliTool to write out stacktraces #7222
- Improved CLI testing infrastructure #7198
- Added custom transport client settings to test infra #7129
- Pick random version from backwards directory #7076
- Remove HttpClient to only use one HTTP client implementation #7003
- Maven reproductions should always include 'clean' target #6975
- Add simple sort assertions for bwc tests #6968 (issue: #6967)
- Randomize translog implementation #6658
- Randomization script refactoring #5914
- Translog:
- remove unused stream #7683
- TransportClient:
- Mark transport client as such when instantiating #7552
Bug fixes
- Aggregations:
- Significant Terms Heuristics now registered correctly #7853 (issue: #7840)
- Inconsistent sorting of top_hits fixed #7697
- Fixes resize bug in Geo bounds Aggregator #7565 (issue: #7556)
- The nested aggregator should iterate over the child doc ids in ascending order. #7514 (issue: #7505)
- Fixes pre and post offset serialisation for histogram aggs #7313 (issue: #7312)
- key_as_string only shown when format specified in terms agg #7160 (issue: #7125)
- fixed value count so it can be used in terms order #7051 (issue: #7050)
- The
nested
aggregator should also resolve and use the parentFilter of the closest reverse_nested
aggregator. #7048 (issue: #6994) - Fix infinite loop in the histogram reduce logic. #7022 (issue: #6965)
- Make
_source
parsing in top_hits
aggregation and search api consistent #6997 - More lenient type parsing in histo/cardinality aggs #6948 (issue: #6893)
- Track scores should be applied properly for
top_hits
aggregation. #6934 - Fixed Histogram key_as_string bug #6830 (issue: #6655)
- Bulk API:
- Bulk operation can create duplicates on primary relocation #7729
- Cluster block with auto create index bulk action can cause bulk execution to not return #7109 (issue: #7086)
- Do not fail whole request on closed index #6790 (issue: #6410)
- Circuit Breaker:
- Only set
breaker
when stats are retrieved #7721
- Discovery:
- UnicastZenPing - use temporary node ids if can't resolve node by it's address #7719
- Function Score:
- Refactor RandomScoreFunction to be consistent, and return values in range [0.0, 1.0] #7446 (issue: #6907)
- Fix explain distance string #7248
- Remove explanation of query score from functions #7245
- GET API:
- Add parameter to GET for checking if generated fields can be retrieved #6973 (issue: #6676)
- Geo:
- Fix circle radius calculation #7338 (issue: #7301)
- Improved error handling in geo_distance #7272 (issue: #7260)
- fixes computation of geohash neighbours #7247 (issue: #7226)
- Better error for invalid multipolygon #7190 (issue: #7126)
- Fix geo_shapes which intersect dateline #7188 (issue: #7016)
- Fixes parse error with complex shapes #6976 (issue: #5773)
- Indexed Scripts/Templates:
- Indexing:
- Don't throw DAEE on replica for create operation; use IW.updateDocument/s instead #7146 (issue: #7142)
- Indices API:
- fixes GET Alias API backwards compatibility #7892 (issue: #7793)
- Internal:
- Fix optimize behavior with 'force' and 'flush' flags. #7920 (issues: #7886, #7904)
- Make close() synchronized during node shutdown #7885
- ClusterHealthAPI does not respect waitForEvents when local flag is set #7731
- Ensure GroupShardsIterator is consistent across requests, to ensure consistent sorting #7698
- Turn unexpected exceptions when reading segments into CorruptedIndexException #7693
- Fix ordering of Regex.simpleMatch() parameters #7661 (issue: #7651)
- Unify search context cleanup #7643 (issue: #7625)
- Use
SEARCH
threadpool for potentially blocking operations #7624 (issue: #7623) - Validate create index requests' number of primary/replica shards #7496 (issue: #7495)
- Make network interface iteration order consistent #7494
- Add all unsafe variants of LZF compress library functions to forbidden APIs. #7468
- Wait until engine is started up when acquiring searcher #7456 (issue: #7455)
- Fix discovery.id.seed #7439 (issue: #7437)
- Indexes unuseable after upgrade from 0.2 to 1.3 and cluster restart #7430
- Use node's cluster name as a default for an incoming cluster state who misses it #7414 (issue: #7386)
- made original indices optional for broadcast delete and delete by query shard requests #7406
- Force optimize was not passed to shard request #7405 (issue: #7404)
- Ignore segments.gen on metadata snapshots #7379
- Upgrade to Jackson Smile 2.4.1.1 #7327 (issue: #18)
- Fixed a request headers bug in transport client #7302
- Parsing command line args multiple times throws
AlreadySelectedException
#7282 - Fix explanation streaming #7257
- VerboseProgress(PrintWriter) does not set the writer #7254
- Fix a very rare case of corruption in compression used for internal cluster communication. #7210
- Fix BytesStreamInput(BytesReference) ctor with nonzero offset #7197
- Fix serialization bug in reroute API #7135
- Support parsing lucene minor version strings #7055
- Use the provided cluster state instead of fetching a new cluster state from cluster service. #7013
- Fix connect concurrency, can cause connection nodes to close #6964
- Drop UnsafeUtils #6963 (issue: #6962)
- When serializing HttpInfo, return null info if service is not started #6906
- Java API:
- get indexed script shouldn't allow to set the index #7553
- Add back string op type to IndexRequest #7387
- MLT Query:
- fix percent_terms_to_match #7754
- Mapping:
- Add explicit error when PUT mapping API is given an empty request body. #7618 (issue: #7536)
- Enable merging of properties in the
_timestamp
field #7614 (issues: #5772, #6958, #777) - Fix
index
setting in _boost
field #7557 - Keep parameters in mapping for
_timestamp
and _size
even if disabled #7475 - Report conflict when trying to disable
_ttl
#7316 (issues: #7293, #777) - Make sure that multi fields are serialized in a consistent order. #7220 (issue: #7215)
- Fix dynamic mapping of geo_point fields #7175 (issue: #6939)
- Improve IP address validation #7141 (issue: #7131)
- Fix
copy_to
behavior on nested documents. #7079 (issue: #6701) - Add multi_field support for Mapper externalValue (plugins) #6867 (issue: #5402)
- More Like This API:
- remove unused search_query_hint parameter #7691
- MultiGet:
- Fail when using no routing on an alias to an index that requires routing #7145
- Nested:
- If the
_type
field isn't indexed nested inner docs must be filtered out. #7410
- Packaging:
- Move forbidden api signature files to dev-tools. #7921 (issue: #7917)
- Don't remove ancestors on deb removal #7078 (issue: #5770)
- Parent/Child:
- has_child or other p/c query wrapped in a query filter may emit wrong results #7685
- Add support for the field data loading option to the
_parent
field. #7402 (issue: #7394) - If _parent field points to a non existing parent type, then skip the has_parent query/filter #7362 (issue: #7349)
- Percolator:
- Pass down the types from the delete mapping request to the delete by query request #7091 (issue: #7087)
- PluginManager:
- Fix config path extraction from plugin handle #7935
- Plugins:
bin
and config
only plugins do not install correctly #7154 (issue: #7152)- Properly quote $JAVA in bin/plugin #5765
- Query DSL:
- Fix NumberFormatException in Simple Query String Query #7876 (issue: #7875)
- Empty bool {} should return match_all #7347 (issue: #7240)
- Cache range filter on date field by default #7122 (issue: #7114)
- REST:
- Reroute API response didn't filter metadata #7523 (issue: #7520)
- REST API:
- Allows all options for expand_wildcards parameter #7290 (issue: #7258)
- Added support for empty field arrays in mappings #7271 (issue: #6133)
- Recovery:
- mapping check during phase2 should be done in cluster state update task #7744
- Resiliency:
- Don't update indexShard if it has been removed before #7509
- Add BWC layer to .si / segments_N hashing to identify segments accurately #7436 (issues: #7351, #7434)
- Improve recovery / snapshot restoring file identity handling #7351
- Do not execute cluster state changes if current node is no longer master #6230
- Scan:
- Use ConcurrentHashMap in scan search to keep track of the reader states. #7499 (issue: #7478)
- Snapshot/Restore:
- Allow to get metadata from arbitrary commit points #7376
- Fix NPE in SnapshotsService on node shutdown #7322 (issue: #6506)
- Sort:
- Make
ignore_unmapped
work for cross-index queries. #7039 (issue: #2255)
- Stats:
- _status with #shards >> queue capacity failing with BroadcastShardOperationFailedException #7916
- Fix NPE in /_cat/nodes #7815 (issue: #6297)
- Update action returns before updating stats for
NONE
operations #7639 - NPE in ShardStats when routing entry is not set yet on IndexShard #7358 (issue: #7356)
- Suggesters:
- infinite loop in GeolocationContextMapping #7433
- Bugs with encoding multiple levels of geo precision #7369 (issue: #7368)
- Completion mapping type throws a misleading error on null value #6926 (issue: #6399)
- Test:
- Fix circuit breaker tests for remote clusters and bwc #7062
- Transport Client:
- fixed the node retry mechanism which could fail without trying all the connected nodes #6829
- Transport client:
Regression
- Discovery:
- restore preference to latest unicast pings describing the same node #7702 (issues: #5413, #7558)