This Week in Elasticsearch - January 16, 2014
Welcome to This week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.
Elasticsearch core
- Elasticsearch 1.0.0.RC1 has been released! We are eager to get your feedback on it, so please try it out and let us know what you think. We also upgraded most of the official plugins already to support elasticsearch 1.0.
- Elasticsearch 0.90.10 has been released including an important bugfix for thoseusing multiple data paths as well as some geo performance improvements
- Added internal get index settings API, which is used by the GET /_settings API (#4620, master)
- Fixed open/close index API when using wildcard only (#4564, master)
- Add support for
flat_settings
flag to all REST APIs that output settings (#4140, master) - Create standard gc and memory_pool names for JVM stats (#4661, master and 0.90)
- Added Joda convert to dependencies and shading (#4660, master and 0.90)
RestRequest
is now an abstract class, and exposes local/remote address (commit, master and 0.90)RestRequest
now exposes headers as list (#4609, master and 0.90)ByteSizeUnit
now supports terabytes and petabytes (commit, master and 0.90)- Added extra REST endpoint for Get settings API (commit, master)
- Percolate API: Change the
sort
boolean option to the sort DSL available in search API (#4625, master) - Deprecated disable allocation decider (#4488, master)
- Added
action.destructive_requires_name
that controls whether wildcard expressions and_all
is allowed to be used for destructive operation (#4549, #4481, master) - Renamed
fuzziness/min_similarity
toedit_distance
(#4082, master) - Deprecated document boost (#4664, master)
PageCacheRecycler
better reuses memory across threads (#4647, master)- Added host name and host address to
DiscoveryNode
(commit, master and 0.90) - Fixed sorting by sub-aggregation in terms agg (#4643, master)
- Disabling
allow_explicit_index
breaks bulk (#4668, master and 0.90) - Multi data path config can cause a shard to be perceived as corrupted (#4674, master and 0.90)
#4681, master and 0.90) - Fixed NPE in bool filter, when an empty should filter clause returns a
DocIdSet
, but null as iterator (#4685, master and 0.90) - REST Update Settings API should not take
timeout
andmaster_timeout
as index parameters (#4692, master and 0.90) - Huge geo cleanup, using meters as default unit of measurement, the geo bounding box filter can take arbitrary corners as arguments (#4515, #4084, master)
- Replaced the multi-field type in favour for the multi fields option that can be set on any core field (#4521, master)
- Added new option
min_doc_count
to terms and histogram aggregations (#4662, master) - Default stopwords list should be
_none_
for all but language-specific analyzers (#4699, master) - Renamed `RobinEngine` to
LuceneEngine
orDefaultEngine
(#4633, master) - Removed
engine.robin.refresh_interval
setting (#4706, master) - Cat API: Make cat/indices primary shard columns enabled with an HTTP parameter (#4689, master)
- Make
length
parameters consistent (#4083, master) - Make type wrapping optional for PUT Mapping API request (#4483, master)
- Added a new
coerce
flag for numeric field mappings which defaults totrue
(#4117, master) - Tribe Node: The tribes feature allows to create a tribe node that can act as a federated client across multiple clusters (#4708, master)
- Removed
ElasticsearchInterruptedException
and handle interrupt state correctly (#4712, master) - Don't accept type wrapper in index request by default (#4484, master)
- Improved support for partial snapshots (#4701, master)
- Cat API: Use seconds instead of millis for timestamps in cat API (#4696, master)
- Cat API: Fix cat/allocation diskRatio calculation (#4670, master)
- Rest APIs: PUT, DELETE and GET consistent for
_mapping
,_alias
,_setting
and_warmer
(#4071, master ) - Source filtering: Excluding all fields of an object should not remove parent (#4715, #4047, master)
Elasticsearch Ecosystem
Here's some more information about what is happening in the ecosystem we are maintaining around Elasticsearch, including plugin and driver releases, as well as news about Logstash and Kibana.
- A new version of the elasticsearch-ruby client has been released, featuring improved test task and allowing for starting your own test cluster for running tests. Also, all of the official clients now use the REST specification of the Elasticsearch core repository
- Greg Brown at Automattic published two blog posts, one on Managing Elasticsearch Clusters Restart Time and Scaling Elasticsearch. He'll be doing a follow up piece to the Scaling Elasticsearch article, so stay tuned to his blog for more.
- Jake Farrell wrote up a how to on Creating an Elasticsearch Plugin.
- Ashutosh Sharma authored a tutorial on Getting Started with Elasticsearch.
- Chris Simpson shared his Elasticsearch Snapshot Restore Overview.
- Jon Tai published his thoughts on Advanced Scoring in Elasticsearch.
- Kevin Stone authored a how to on Using Elasticsearch with ElasticUtils.
Slides & Videos
Loggly at AWS re:Invent 2013 - Infrastructure at Scale: Apache Kafka, Twitter Storm & Elasticsearch
- Boaz Leskes shared the slides from his presentation at 010Dev in Rotterdam: From A to JSON - an overview of Elasticsearch.
- David Pilato shared the slides from his presentation at Human Talks in Angers: Make Sense of Your (BIG) Data!
Mastering Elasticsearch with Ruby
Where to find Us
Belgium
Czech Republic
Honza Kral will give two presentations at DevConf.cz: Design for Cloud with Elasticsearch and Centralized Logging with Logstash. Honza's presentations take place on Friday, February 7th, and the conference runs from the 7th through the 9th.
France
- Vladislav Pernin will present on using Elasticsearch, Logstash and Kibana in his talk Centralizing Large Volumes of Logs at the Lyon JUG. The event takes place on January 21st and doors open at 7 PM.
- David Pilato will be speaking on Elasticsearch: Make sense of your (BIG) data on Azure! at Microsoft Tech Days in Paris. David's presentation is scheduled for February 13th at 4:30 PM.
Germany
- Alexander Reelsen will talk about Elasticsearch at the E-Commerce Hacktable in Hamburg on January 22nd. The meetup will also feature a talk from Sebastian Betz of Antevorte on their use of Elasticsearch. Doors open at 7 PM.
- Elasticsearch will have a booth at the OOP Konferenz in Munich Feb. 4-6th. There will also be a workshop on February 5th, which will cover an Introduction to Elasticsearch, Logstash and Kibana.
- Alexander Reelsen will discuss Using Elasticsearch, Logstash & Kibana to Create Real-Time Dashboards at the Lightweight Java Users Group Munich meeting on February 11th. Doors open at 7 PM.
Japan
Thanks to Jun Ohtani, the 3rd Elasticsearch Meetup will be held in Tokyo on February 7th starting at 7 PM. Please remember to register for the meetup.
United Kingdom
Elasticsearch will have two sessions at QCon London, which takes place March 3-7th. You can join Mark Harwood for a tutorial on Using Elasticsearch for Anomaly Detection plus see Shay Banon and Graham Tackley co-present on How Elasticsearch Powers The Guardian's Newsroom. Make sure to stop by our booth to say hello!
United States
- The Elasticsearch Los Angeles Meetup will convene for their first meeting tonight at 7 PM. Jordan Sissel will be speaking on All Things Logstash.
- The January Elasticsearch San Francisco Meetup will be held on January 21st. Doors open at 6:30 PM.
- The second Silicon Valley Elasticsearch Meetup is slated for January 23rd. Netflix will discuss their use of Elasticsearch. Doors open at 6:30 PM.
- Shay Banon will hold an open format Q&A session at the Elasticsearch Boston Meetup on February 6th. Doors open at 6 PM.
- The Elasticsearch New York Meetup will be held on February 3rd. In addition to hearing from our host, The Ladders, about their use of Elasticsearch, Shay Banon will be on hand for Q&A on all things Elasticsearch.
- The Elasticsearch DC Meetup will get together on February 5th at 6:30 PM. Attendees will be treated to an in depth look at how NGP VAN uses Elasticsearch.
- The next Elasticsearch Boston Meetup will take place on February 6th. Attendees will be treated to an open Q&A from Elasticsearch creator Shay Banon, plus a series of lightning talks. Doors open at 6 PM.
- The Search and Machine Learning Seattle Meetup group will welcome Steve Mayzak on February 6th. Steve will present on Elasticsearch, Logstash and Kibana. Doors open at 7 PM.
- Several folks from Elasticsearch will be attending the Strata Conference from February 11-13th in Santa Clara, California. Stop by our booth to say hello!
- The Elasticsearch Denver Meetup group will be getting together on February 24th to hear about Automattic's use of Elasticsearch. Doors open at 6:30 PM and Elasticsearch core developer Lee Hinman will also be on hand for Q&A.
- Leslie Hawthorn, Kevin Kluge and Jordan Sissel will all be presenting at the 12th Annual Southern California Linux Expo, a.k.a. SCALE 12x. If you can't catch the talks, make sure to stop by our table to say hello.
- We're working on setting dates for our first ever meetup in Portland, Oregon. Sign up for the Portlandia Meetup Group to get regular updates.
Where to Find You
Our Community Manager, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, take a moment to let her know.
Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note. We care more about your skill set and passion for Elasticearch, Kibana and Logstash than where you rest your head.
Trainings
If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:
- Stockholm - February 5, 2014
- Paris - February 11, 2014
- Boulder - February 24, 2014
- London - February 25, 2014
- San Francisco - February 27, 2014