This Week in Elasticsearch - January 22, 2014
Welcome to This week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.
- The Elasticsearch 1.0 branch has been cut and will be used for the upcoming 1.0 release and related enhancements. The 1.x branch has been added as well for future development, still related the 1.x major versions. The next major version of Elasticsearch (2.0) will be developed on master. No changes have been made to the 0.90 branch.
- Added ability to setup custom
_allfields (#4520, master, 1.x and 1.0)
- Added the ability to retrieve fields from the field data cache (#4492, master, 1.x and 1.0)
- Added local flag support to all the cluster state read operations that get executed on the master node (#3345, master, 1.x and 1.0)
- Tribe nodes automatically execute cluster state read operations locally (commit, master, 1.x and 1.0)
- Optimized histogram aggregations through a finer-grained values rounding (#4800, master and 1.x)
- Terms aggregations returns now all terms by setting its
sizeto <code>0 (#4837, master and 1.x)
DateHistogramBuildersupports now setting a value for <code>min_doc_count (#4848, master, 1.x and 1.0)
partial_fieldsin favor of source filtering (#4118, master, 1.x and 1.0)
- Added ability to configure circuit breaker with a percentage (#4616, master, 1.x and 1.0)
- Many commits relate to the REST spec and tests: the test coverage for the REST layer has been remarkably improved, and those same test suites are used to test all our official clients too (e.g. commit 1 and commit 2)
- Improved detection of hot threads (#4775, master, 1.x, 1.0 and 0.90)
omit_term_freq_and_positionsfor new indices, deprecated since 0.20 (#4722, master, 1.x and 1.0)
- Fixed timestamp column regression in cat api (#4782, master, 1.x and 1.0)
- Made sure that filters that wrap parent/child filters are never cached (#4757, master, 1.x, 1.0 and 0.90)
- Made sure that filters that use the
nowdate math expression are never cached (#4846, master, 1.x, 1.0 and 0.90)
- Fixed a bug that caused a scroll search to return no hits on 2nd or consecutive requests when using has_child or has_parent filters (#4703, master, 1.x, 1.0 and 0.90)
IndexOutOfBoundsExceptionwhile indexing empty JSON document (#4771, master, 1.x, 1.0 and 0.90)
master_timeoutparameter is now taken into account when calling the cluster pending tasks api through the REST layer (#4806, master, 1.x and 1.0)
- Made sure that the same boolean values are accepted by all
RestRequestmethods (#4808, master, 1.x, 1.0 and 0.90)
- Fixed logging on immediate exit on start (#4805, master, 1.x, 1.0 and 0.90)
- No results are now returned instead of an error when executing an empty query_string (#3952, master, 1.x, 1.0 and 0.90)
- Made sure a merge mapping conflict gets returned when trying to enable or disable norms (#4761, master, 1.x and 1.0)
- Added page tracking to
MockPageCacheRecycler(#4814, master, 1.x and 1.0)
- Fixed inconsistency between how the
omit_normsoption is parsed and serialized (#4760, master, 1.x and 1.0)
- Fixed a bug that prevented rivers from getting started when using index templates that match the
_riverindex (#4577, master, 1.x, 1.0 and 0.90)
paramAsBooleanOptionalmethods (#4817, master, 1.x and 1.0)
search_typerandomization to <code>ElasticsearchIntegrationTest, to make sure
DFS_QUERY_THEN_FETCHis tested too (#4793, master, 1.x and 1.0)
- Upgraded to guava 16.0 (commit, master, 1.x, 1.0 and 0.90)
- Improved error handling when trying to allocate a shard on a non-data node (#4833, master, 1.x, 1.0 and 0.90)
Here's some more information about what is happening in the ecosystem we are maintaining around Elasticsearch, including plugin and driver releases, and news about Logstash and Kibana.
- Big news in new Elasticsearch 1.0 features over the past week; read more from Drew Raines on the new cat API and Igor Motov on Snapshot & Restore.
- Aaron Mildenstein and Honza Kral released Curator this week, a tool for tending your time-series indices that is based upon expire-logs.
- Richard Pijnenburg released v 0.2.1 of the Elasticsearch Puppet Module and v 0.4.0 of the Logstash Puppet Module.
- You can now find modules for Elasticsearch and Logstash on the Puppet Forge.
- A preview release of the integration for Ruby's ActiveModel-based models has been published as
elasticsearch-model, with ActiveRecord and Mongoid working out-of-the-box. The semantics are very similar to the widely-used (Re)Tire gem, making the migration easy. You can generate a fully working Ruby on Rails application with a template
- Dan Radez authored an article on using Elasticsearch, Logstash and Kibana with OpenStack.
- Michael Lussier wrote up An Introduction to Elasticsearch Aggregations.
- Brian O'Neill shared a post on using Elasticsearch with Angular-JS.
- Shawn Tan wrote up an article on using Elasticsearch for Naive Bayes Categorisation.
- Pablo Casado authored an article on Revisiting charts: NodeJS, Twitter and Elasticsearch.
- Dian Dwi Nugraha wrote a post on Log Analytics Using Elasticsearch, Logstash and Kibana.
- Lalit Jha penned a tutorial on Talend Elasticsearch Indexing.
- Roy Russo shared some interesting statistics on usage of Elasticsearch.
- BusHorn wrote up their story on using Elasticsearch, Logstash and Kibana to do flow activity monitoring of their camel integration.
- Bearstech authored an article on creating a highly available MQTT service using Elasticsearch, Logstash and Kibana.
- Ryan Robitale penned an post, Connecting Tableau to Elasticsearch, on querying Elasticsearch with Hive SQL and Hadoop.
- You may enjoy this article on Rexster, Titan and Elasticsearch Indexes.
- Diana Kupfer wrote about the Elasticsearch 1.0 release for JAXenter. (auf Deutsch)
Slides & Videos
Jordan Sissel's slides from the Inaugural Elasticsearch Los Angeles Meetup
- Boaz Leskes presented 1/10th of a Version, 10x the Punch: coming features in 1.0 last week at the first ever Elasticsearch Atlanta Meetup.
- The folks at Intelligrape shared a nice overview presentation for those who are just getting started, Intro to Elasticsearch.
Zachary Tong's presentation on Query Optimization from the Inaugural Elasticsearch Atlanta Meetup
Where to find Us
We'd love to feature all the great Elasticsearch presentations happening worldwide in this section. If you're speaking on Elasticsearch, let our Community Manager know!
Honza Kral will give two presentations at DevConf.cz: Design for Cloud with Elasticsearch and Centralized Logging with Logstash. Honza's presentations take place on Friday, February 7th, and the conference runs from the 7th through the 9th.
- David Pilato will be speaking on Elasticsearch: Make sense of your (BIG) data on Azure! at Microsoft Tech Days in Paris. David's presentation is scheduled for February 13th at 4:30 PM.
- Alexander Reelsen will talk about Elasticsearch at the E-Commerce Hacktable tonight in Hamburg. The meetup will also feature a talk from Sebastian Betz of Antevorte on their use of Elasticsearch. Doors open at 7 PM.
- Elasticsearch will have a booth at the OOP Konferenz in Munich Feb. 4-6th. There will also be a workshop on February 5th, which will cover an Introduction to Elasticsearch, Logstash and Kibana.
- Britta Weber will present on Customizing Your Scoring Using function_score at the Munich Search Meetup on February 5th. Sebastian Paetzold from ysura will speak on Permission Management with Elasticsearch. Doors open at 7 PM.
- Alexander Reelsen will discuss Using Elasticsearch, Logstash & Kibana to Create Real-Time Dashboards at the Lightweight Java Users Group Munich meeting on February 11th. Doors open at 7 PM.
Thanks to Jun Ohtani, the 3rd Elasticsearch Meetup will be held in Tokyo on February 7th starting at 7 PM. Please remember to register for the meetup.
Elasticsearch will have two sessions at QCon London, which takes place March 3-7th. You can join Mark Harwood for a tutorial on Using Elasticsearch for Anomaly Detection plus see Shay Banon and Graham Tackley co-present on How Elasticsearch Powers The Guardian's Newsroom. Make sure to stop by our booth to say hello!
- The January Elasticsearch San Francisco Meetup will be held on January 21st. Doors open at 6:30 PM.
- The second Silicon Valley Elasticsearch Meetup is slated for January 23rd. Netflix will discuss their use of Elasticsearch. Doors open at 6:30 PM.
- Elasticsearch is sponsoring the NWA TechFest 2014 in Fayetteville, Arkansas on January 31st. Our friends at partner firm StackSearch will be presenting an Elasticsearch Primer, by Sloan Ahrens, and Site Search Business Best Practices for the Business User, by Mark Brandon. You can stop by the StackSearch booth to learn more about Elasticsearch. The event is open to the public with a donation of $2 or 2 cans of food. All proceeds will benefit the local Food Bank.
- The Elasticsearch New York Meetup will be held on February 3rd. In addition to hearing from our host, The Ladders, about their use of Elasticsearch, Shay Banon will be on hand for Q&A on all things Elasticsearch.
- The Elasticsearch DC Meetup will get together on February 5th at 6:30 PM. Attendees will be treated to an in depth look at how NGP VAN uses Elasticsearch.
- The next Elasticsearch Boston Meetup will take place on February 6th. Attendees will be treated to an open Q&A from Elasticsearch creator Shay Banon, plus a series of lightning talks. Doors open at 6 PM.
- The Search and Machine Learning Seattle Meetup group will welcome Steve Mayzak on February 6th. Steve will present on Elasticsearch, Logstash and Kibana. Doors open at 7 PM.
- Several folks from Elasticsearch will be attending the Strata Conference from February 11-13th in Santa Clara, California. Stop by our booth to say hello!
- The Boston Azure User Group will convene on February 20th at 6 PM. Chris Morley will present on Elasticsearch Running on Linux in Azure.
- Leslie Hawthorn, Kevin Kluge and Jordan Sissel will all be presenting at the 12th Annual Southern California Linux Expo, a.k.a. SCALE 12x. If you can't catch the talks, make sure to stop by our table to say hello.
- The Elasticsearch Denver Meetup group will be getting together on February 24th to hear about Automattic's use of Elasticsearch. Doors open at 6:30 PM and Elasticsearch core developer Lee Hinman will also be on hand for Q&A.
- We're working on setting dates for our first ever meetup in Portland, Oregon. Sign up for the Portlandia Meetup Group to get regular updates.
Where to Find You
Our Community Manager, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, take a moment to let her know.
Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note. We care more about your skill set and passion for Elasticearch, Kibana and Logstash than where you rest your head.
If you are interested in Elasticsearch training we have courses taught by our core developers coming up in: