This Week in Elasticsearch and Apache Lucene - September 29 2015
Welcome to This Week in Elasticsearch and Apache Lucene! With this weekly series, we're bringing you an update on all things Elasticsearch and Apache Lucene at Elastic, including the latest on commits, releases and other learning resources.
Top News
Now OnDemand: #Elasticsearch @ Scale: Reference Architecture & Common Pitfalls” ft. lessons from @pickypg & @alanhrdy http://t.co/EOtMJkC9F8
— elastic (@elastic) September 28, 2015
Elasticsearch Core
- Internal: Factor expressions scripts out to
lang-expression
plugin (#13726, 3.0.0) - Internal: Reduced the number of
ClusterStateUpdateTask
variants (#13735, 3.0.0) - Internal: More helpful error message on parameter order (#13737, 3.0.0, 2.1.0, 2.0.0)
- Internal: Remove
ExpressionScriptCompilationException/ExpressionScriptExecutionException
(#13742, 3.0.0) - Internal: Pending operations in the translog prevent shard from being marked as inactive (#13759, 3.0.0, 2.1.0, 2.0.0, 1.7.3)
- Internal: block process execution with
seccomp
on linux/amd64 (#13753, 3.0.0, 2.1.0) - Packaging: Give vagrant boxes 2gb of ram (#13765, 3.0.0, 2.1.0, 2.0.0)
- Internal: Move
ShardPercolateService
creation into IndexShard (#13777, 3.0.0) - Internal: Move
ShardTermVectorService
to be on indices level as TermVectorService (#13786, 3.0.0) - Internal: Remove ES internal deletion policies in favour of Lucenes implementations (#13794, 3.0.0)
- Settings:
ByteSizeValue.equals
should normalize units (#13784, 3.0.0, 2.1.0, 2.0.0) - Logging: Don't log multi-megabyte guice exceptions (#13782, 3.0.0, 2.1.0, 2.0.0)
- Exceptions: Remove reflection hacks from
ElasticsearchException
(#13796, 3.0.0) - Exceptions: Make
ElasticsearchException
ids for 2.0 consistent with master (#13805, 2.0.0) - Core: Removes and bans
ImmutableSet
(#13754, 3.0.0) - Packaging: Don't let ubuntu try to install its crazy jayatana agent (#13813, 3.0.0, 2.1.0, 2.0.0)
- Internal: An inactive shard is activated by triggered synced flush (#13802, 3.0.0, 2.1.0, 2.0.0, 1.7.3)
- Core:
LoggingRunnable.run
should catch and log all errors, not just Exception? (#13718, 3.0.0, 2.1.0, 2.0.0) - Translog: Remove
TranslogService
and fold it into synchronous IndexShard API (#13707, 3.0.0) - Geo: Fix
GeoPointFieldMapper
to index geohash at correct precision. (#13649, 3.0.0, 2.1.0, 2.0.0) - Network: Expose all addresses that the transports are bound to (#13586, 3.0.0, 2.1.0, 2.0.0)
- Core: Remove and ban
ImmutableMap#entrySet
(#13724, 3.0.0) - Internal: Fix ping timeout settings inconsistencies (#13701, 3.0.0)
- CRUD: Index name expressions should not be broken up (#13691, 3.0.0, 2.1.0, 2.0.0)
- Index APIs: Date math index names should work too when an index doesn't exist (#13570, 3.0.0, 2.1.0, 2.0.0)
Apache Lucene
- Lucene 5.3.1 bug fix release is out
- Index time sorting is soon moving to Lucene's core to make it easier to use
- Fix file handle leak in the new
RangeTreeWriter
OfflineSorter
no longer removes the output file up-front making it safer to useFiles.createTempFile
- Upgrade to Spatial4j 0.5 to fix test bugs uncovered by Lucene's randomized tests
- Sidestep an auto-prefix terms test bug
- Upgrade
RandomizedTesting
to 2.1.17 ComplexPhraseQuery.toString
doesn't include the slop BooleanQuery.equals
is unfortunately sensitive to clause order, but fixing it is controversial- Can we simplify Lucene's search APIs by merging
rewrite
andcreateWeig<wbr>ht
? - Disjunctions could advance lazily, speeding up cases when they are MUST'd with another query
GeoPointDistanceQuery
is buggy with large distances- We should use approximations when
minShouldMatch
is used with aBooleanQuery
to speed up cases when they are MUST'd with another query PatternTokenizer
should not hold heap equal to the largest document its ever processed- Should we add a read-only interface to
BitSet
? - Maybe
FrenchAnalyzer
should strip accents, but proving that consistently helps relevance is problematic - Is there a clever way to encode x, y, z earth surface coordinates into only 8 bytes with enough precision?
- Should
TermOrdValComparator.<wbr>value
be forced to make a deep clone of the returnedBytesRef
?
Watch This Space
Stay tuned to this blog, where we'll share more news on the whole ELK ecosystem including news, learning resources and cool use cases!