This Week in Elasticsearch - January 29, 2014
Welcome to This week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.
- Upgrade to Lucene 4.6.1 (#4897, master, 1.x, 1.0 and 0.90)
- Ensure rivers
_metadocuments are retrieved via get with preference
_primary(#4864, master, 1.x, 1.0 and 0.90)
- Reducing pulled in dependencies by marking unused
provided(#4859, master, 1.x, 1.0 and 0.90)
- Support for multiple
rescoreoperations has been added (#4748, master, 1.x and 1.0)
- When executing sub aggregations executing on a single shard, the reduce call was not propagated properly down the agg hierarchy (#4843, master, 1.x and 1.0)
_siteplugins did not pick up on index.html in sub directories (#4845, master, 1.x, 1.0 and 0.90)
- Removed pre 1.0 transport serialization logic (commit, master, 1.x and 1.0)
- cat API: Normalize headers, support for pb and tb (#4852, #4871, master, 1.x and 1.0)
- Dont throttle the translog during recovery (commit, master, 1.x, 1.0 and 0.90)
- REST mget API: Support for
sourceparameter (#4892, master, 1.x, 1.0 and 0.90)
- Bulk API: Do not fail the whole request in case the JSON cannot be parsed and special mapping is configured (#4745, master, 1.x, 1.0 and 0.90)
- REST GET Cluster State: Filtering did not make use of the recently changed parameters correctly (#4885, master, 1.x and 1.0)
- Percolator response should always return the
matcheskey (#4881, master, 1.x and 1.0)
- Get Field Mapping API: Behave consistent to other changes when a single field mapping is not found (#4738, master, 1.x and 1.0)
- REST API: Support for
sourceparameter in percolate mpercolate, msearch and mtermvector APIs (#4900, #4901, #4902, #4903, master, 1.x and 1.0)
- Cleanup of the Aggregations API (#4922, master, 1.x and 1.0)
- Hot Threads API: Prevent rare
ArrayIndexOutOfBoundsException(#4927, master, 1.x, 1.0 and 0.90)
- Aggregations: Made REST output consistent over all aggregation types (#4926, master, 1.x and 1.0)
Here's some more information about what is happening in the ecosystem we are maintaining around Elasticsearch, including plugin and driver releases, and news about Logstash and Kibana.
- We released Elasticsearch Marvel yesterday, Elasticsearch's first product for management and monitoring. Marvel is free for development use. You can get the full story on Marvel from Boaz Leskes in his post Why We Built Marvel.
- A new version of Google Compute Engine plugin has been released due to a breaking change in the GCE API resulting in failing discovery
- Petr Jasek released a new version of Eve-Elastic, an Elasticsearch data layer for Eve REST framework.
- John Hamelink talked about his experiences with various configuration management tools. His awesomely titled article, Server nirvana: my journey towards infrastructual mindfulness, includes an Ansible playbook for installing, configuring and securing Logstash, Elasticsearch and Kibana.
- Barnaby Gray treated us to a very interesting look at Utilizing Elasticsearch and Kibana to bet on results of a TV show.
- MapCentia shared their experiences moving from PostGIS to Elasticsearch in two posts, part 1 - Indexing PostGIS data and part 2 - Make a map
- Rishav Rohit wrote an article about using Elasticsearch, Logstash and Kibana for clickstream weblog integration
- Dario Blanco wrote up a how to on Seamless Upgrades with Elasticsearch
- Bogdan Dumitrescu authored a tutorial on Using Logstash, Elasticsearch and Kibana to Monitor Your Video Card.
- Sloan Ahrens shared a tutorial on Multi-field Partial Word Autocomplete in Elasticsearch Using nGrams.
- Vincent Teyssier penned a post on Elasticsearch Index Migration.
- Pierre André Ménard published a research paper on Indexing and Multilingual Searches with Elasticsearch. (en français)
- Sebastien Vanotti shared his insights on using Talend Big Data together with Hadoop and Elasticsearch. (en français)
- Maxime Sanglan shared a write up of David Pilato's recent brown bag lunch talk on Elasticsearch. Maxime's post includes David's slides, which are en français.
- For our friends in Japan, Jun Ohtani wrote a blog post on Elasticsearch Marvel. You may also enjoy this extensive Elasticsearch tutorial, which includes many examples.
- Luiz Augusto Amelotti authored an article on Using Redis, Logstash, Elasticsearch and Kibana for managing logs. (em português brasileiro)
Slides & Videos
Netflix in Elasticsearch: January 2014 Silicon Valley Meetup
- If you enjoyed the video from the Netflix Elasticsearch Meetup, you may also want to take a look at Danny Yuan's slides.
- Alexander Reelsen published his slides about Elasticsearch in Ecommerce, from the Ecommerce Hacktable Meetup in Hamburg last week.
- Boaz Leskes talked about utilizing time-based indices with Elasticsearch at the Search Engines Amsterdam seminar last week.
- Valentino Gagliardi shared his slides on Open Source Log Analysis with Elasticsearch, Logstash and Kibana. (in italiano)
en français: Gérald Croes and Julien Salleyron's Recent Talk from Forum PHP 2013
Where to find Us
We'd love to feature all the great Elasticsearch, Logstash and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Community Manager, Leslie Hawthorn, know!
- Honza Kral will be speaking on What's New in Elasticsearch 1.0 with Aggregations at the Benelux Elasticsearch Meetup in Brussels on Friday, January 31st. Doors open at 6 PM.
- Leslie Hawthorn and Honza Kral will be attending FOSDEM 2014 on February 1st and 2nd. Stop by the Elasticsearch table to say hello! We'll be in Building K on the 2nd Floor.
- Leslie Hawthorn will be speaking at the Infrastructure.Next Conference on DevOps: For Happier, More Productive People. Infrastructure.Next takes place on February 5th in Ghent, following Config Management Camp.
Honza Kral will give two presentations at DevConf.cz: Design for Cloud with Elasticsearch and Centralized Logging with Logstash. Honza's presentations take place on Friday, February 7th, and the conference runs from the 7th through the 9th.
- David Pilato will present Elasticsearch: Make sense of your (BIG) data at the Paris JUG on February 11th. Doors open at 7:15 PM.
- David will speak on Elasticsearch: Make sense of your (BIG) data on Azure! at Microsoft Tech Days in Paris. David's presentation is scheduled for February 13th at 4:30 PM.
- Elasticsearch will have a booth at the OOP Konferenz in Munich Feb. 4-6th. There will also be a workshop on February 5th, which will cover an Introduction to Elasticsearch, Logstash and Kibana.
- Britta Weber will present on Customizing Your Scoring Using function_score at the Munich Search Meetup on February 5th. Sebastian Paetzold from ysura will speak on Permission Management with Elasticsearch. Doors open at 7 PM.
- Alexander Reelsen will discuss Using Elasticsearch, Logstash & Kibana to Create Real-Time Dashboards at the Lightweight Java Users Group Munich meeting on February 11th. Doors open at 7 PM.
Thanks to Jun Ohtani, the 3rd Elasticsearch Meetup will be held in Tokyo on February 7th starting at 7 PM. Please remember to register for the meetup.
- Elasticsearch will have two sessions at QCon London, which takes place March 3-7th. You can join Mark Harwood for a tutorial on Using Elasticsearch for Anomaly Detection plus see Shay Banon and Graham Tackley co-present on How Elasticsearch Powers The Guardian's Newsroom. Make sure to stop by our booth to say hello!
- Elasticsearch is sponsoring the NWA TechFest 2014 in Fayetteville, Arkansas on January 31st. Our friends at partner firm StackSearch will be presenting an Elasticsearch Primer, by Sloan Ahrens, and Site Search Business Best Practices for the Business User, by Mark Brandon. You can stop by the StackSearch booth to learn more about Elasticsearch. The event is open to the public with a donation of $2 or 2 cans of food. All proceeds will benefit the local Food Bank.
- The Elasticsearch New York Meetup will be held on February 3rd. In addition to hearing from our host, The Ladders, about their use of Elasticsearch, Shay Banon will be on hand for Q&A on all things Elasticsearch.
- The Elasticsearch DC Meetup will get together on February 5th at 6:30 PM. Attendees will be treated to an in depth look at how NGP VAN uses Elasticsearch.
- The next Elasticsearch Boston Meetup will take place on February 6th. Attendees will be treated to an open Q&A from Elasticsearch creator Shay Banon, plus a series of lightning talks. Doors open at 6 PM.
- The Search and Machine Learning Seattle Meetup group will welcome Steve Mayzak on February 6th. Steve will present on Elasticsearch, Logstash and Kibana. Doors open at 7 PM.
- Several folks from Elasticsearch will be attending the Strata Conference from February 11-13th in Santa Clara, California. Stop by our booth to say hello!
- The Boston Azure User Group will convene on February 20th at 6 PM. Chris Morley will present on Elasticsearch Running on Linux in Azure.
- Leslie Hawthorn, Kevin Kluge and Jordan Sissel will all be presenting at the 12th Annual Southern California Linux Expo, a.k.a. SCALE 12x. If you can't catch the talks, make sure to stop by our table to say hello. SCALE runs from February 21st-23rd.
- The Elasticsearch Denver Meetup group will be getting together on February 24th to hear about Automattic's use of Elasticsearch. Doors open at 6:30 PM and Elasticsearch core developer Lee Hinman will also be on hand for Q&A.
- We're working on setting dates for our first ever meetup in Portland, Oregon. Sign up for the Portlandia Meetup Group to get regular updates.
Where to Find You
Our Community Manager, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, take a moment to let her know.
Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note. We care more about your skill set and passion for Elasticearch, Kibana and Logstash than where you rest your head.
If you are interested in Elasticsearch training we have courses taught by our core developers coming up in: