This Week in Elasticsearch - January 28, 2015
Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.
- Core: don't throttle recovery indexing operations (#9396, 2.0.0, 1.5.0, 1.4.3)
- Snapshot status API: Ensure headers are handed over to inner nodes request (#9409, 2.0.0, 1.5.0, 1.4.3)
- Nested aggregator: Fix handling of multiple buckets being emitted for the same parent doc id (#9346, 2.0.0, 1.5.0, 1.4.3)
- Mappings: Remove
- Mappings: Deprecate
_analyzerand per query analyzers (#9383, 1.5.0)
- Mapping: Explicit
_timestampdefaulting to <code>null is set to
- Core: Removed Redundant inheritance (#9427, 2.0.0, 1.5.0)
- Internal: fix shard state tranport action names (#9440, 2.0.0, 1.5.0)
- Aggs: nested agg needs to reset root doc between segments (#9441, 2.0.0, 1.5.0, 1.4.3)
- Mapping: Throw
StrictDynamicMappingExceptionexception (#9445, 2.0.0, 1.5.0, 1.4.3, 1.3.8)
ClusterInfoServiceshould wipe local cache upon unknown exceptions (#9449, 2.0.0, 1.5.0, 1.4.3)
- Snapshot/Restore: Better handling of index deletion during snapshot (#9418, 2.0.0, 1.5.0)
- Recovery: Flush immediately after a remote recovery finishes (unless there are ongoing ones) (#9439, 2.0.0, 1.5.0, 1.4.3)
- Snapshot/Restore: Add support for changing index settings during restore process (#9285, 2.0.0, 1.5.0)
- Rest: Adding support of multi-index query parameters for
/_cluster/state(#9295, 2.0.0, 1.5.0)
- Aggs: The parent filter of the nested aggregator was not resolved correctly all the time (#9335, 2.0.0, 1.5.0, 1.4.3, 1.3.8)
- Disk based allocation: Relax restrictions on filesystem size reporting in
DiskUsage(#9283, 2.0.0, 1.5.0, 1.4.3, 1.3.8)
- Reverse nested aggregation: fix handling of the same child doc id being processed multiple times. (#9345, 2.0.0, 1.5.0, 1.4.3, 1.3.8)
- Aggregations: Clean up response API for Aggregations (#9221, 2.0.0)
- Geo: Update
GeoPolygonFilterto handle polygons crossing the dateline (#9339, 2.0.0, 1.5.0, 1.4.3)
In Apache Lucene this Past Week
- Fixed an int overflow corruption case in skip data that can strike on very large segments (720 GB in this case) that also contain high-frequency terms (e.g. stopwords).
- Committed speedups to how Lucene resolves deleted terms to documents.
- Improved performance for bulk merging of term vectors.
- Improved performance of Boolean queries involving low frequency clauses.
- Work continues on issue 6196, adding a geo3d spatial package to Lucene. Many thanks to the Nokia engineering team for their contributions!
. @coolacid presenting at Montreal @elasticsearch meetup! pic.twitter.com/mGQ75Tlw6V
— Colin Surprenant (@colinsurprenant) January 27, 2015
Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.
- We celebrated the first GA release of Shield, our security plugin for Elasticsearch, yesterday. Check out the full details in the release blog post from Uri Boness, including a deep dive on Shield's inner workings and instructions for evaluating Shield for free in your development environment.
- This very morning, we released NEST & Elasticsearch .NET 1.4. Martijn Laarman has all the details about this latest release.
- Zachary Tong authored the second article in his series on Aggregations, this time deep diving into sub-aggregations. If you missed his first article, an Introduction to Aggregations, its well worth a read before checking out the second post in the series.
- The fine folks at TrackJS authored an in-depth guest blog post on their Elasticsearch use case, which is of particular interest given their resource constraints. Check out the article for more information, and details on how to register for our upcoming webcast with the TrackJS team.
- Bez Hermoso shared the story of why the ActiveLAMP developer team chose Elasticsearch to power their latest project, including integrations with AngularJS, Drupal, and Symfony.
- Cédrick Lunven and Julien Kirch shared a post on efficient data flow monitoring. Their recommended solution? Elasticsearch and Kibana, of course! (en français)
- The team at Enigmo shared details on their use case for Elasticsearch, Fluentd and Kibana 4, all running on AWS. (日本語で)
Slides & Videos
Taylor Lovett shares how Elasticsearch can make WordPress search sing
Colin Goodheart-Smithe introduces the ELK stack
Pour nos amis francophone par David Pilato
Britta Weber speaks about #Elasticsearch, #Logstash and #Kibana at #Fyber. pic.twitter.com/za3bfxlQZ2
— Fyber (@Fyber) January 27, 2015
Where to Find Us
We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Director of Developer Relations, Leslie Hawthorn, know!
- If you're headed out to FOSDEM (Europe's largest gathering of open source developers!) in Brussels at the end of January, be sure to squeeze time into your schedule for the BeLux (Belgium / Luxembourg) Elasticsearch user group meetup on Friday, January 30.
- Can't make it the meetup? Catch us at the conference at the Elasticsearch booth, or visit us in the Open Source Search Dev Room.
- Right after FOSDEM, we're heading to Ghent for Configuration Management Camp. Leslie Hawthorn & Robyn Bergeron will see you in the hallway track between talks, plus don't miss Robyn's panel on Provisioning Infrastructure as Code.
- The Search Meetup Karlsruhe group will be gathering on January 29 for a presentation from Nico Heid on the ELK stack. Check out their meetup page for more information and to register for the event.
- Alexander Reelsen will talk about ELK stack use cases at the OOP Konferenz in Munich next Thursday, Jan 29. Ping him if you'd like to meet up!
- The Search Meetup Munich group will get together on Feb 5 for a special Elasticsearch edition: Oliver Eilhard on his Elasticsearch Go Client, Alexander Reelsen on the Percolator, plus Q&A with our CTO, Shay Banon. Register now to save your seat.
- The Ansible Berlin User Group is holding their inaugural meetup on Feb 11, and they'll be talking Elasticsearch and Kibana along with Jenkins, Rethinkdb and, of course, Ansible. Join them and help celebrate their first ever meetup!
- The Berlin Elasticsearch User Group will be holding their monthly meeting on February 24. Register now to save your seat, and even better volunteer to speak!
The Tel Aviv-Yafo ELK meetup group will be talking How to Use ELK to Analyze Logs from a Large Production AWS Environment on February 24. These folks are looking for space to meet, so you can host get in touch with the organizers. You can register for the meetup now while they're finding a location.
Registration is already full for the next Elasticsearch Japan Study Session in Tokyo, but you can add yourself to the waitlist. The user group will get together on February 13 at 7:30 PM.
Join the NDC Meetup in Oslo on Feb 18 and hear @astensby talk about Data Exploration with #Elasticsearch http://t.co/iKRSnUuJX8
— Found (@foundsays) January 28, 2015
Just announced: #Elasticsearch Stockolm Meetup for @jfokus on 2 Feb. Many thanks to @spotify for hosting & speaking! http://t.co/iLuhfDM7De
— Leslie Hawthorn (@lhawthorn) January 26, 2015
- The Taipei Python Users Group will get together tomorrow, Thursday, January 29 to talk all about Elasticsearch and Docker. Register now to attend the meetup.
- The Agile Code Camp team are convening a developer and designer hack day on February 23, and attendees will be developing with Elasticsearch. Register now to attend the full day event.
- Heading to Monki Gras before FOSDEM? So are we! Alan Hardy will be attending talks and would love to talk all things Elasticsearch & the ELK stack with you in the hallway track. May sure to say hello!
- The January London Elasticsearch Meetup is already full up, but you can still join the waitlist. The group convenes on January 29 to talk the ELK stack on AWS, Elasticsearch for Lead Generation and Building UIs with Search Templates.
- If you're a star in the Ansible Galaxy, you're no doubt attending AnsibleFest London on Feb 5. Stop by and say hello to Alan Hardy and Samir Bennacer at our table in the exhibits area! Our Developer Advocate, Robyn Bergeron, will also be hanging out in the hallway track if you'd like to say hello!
- The South Shore .NET user group in Plymouth, Massachusetts, is getting together on January 29 for a presentation on Elasticsearch for data mining. This meetup will be covering Elasticsearch basics and a few other areas, and have you ready to add Elasticsearch to your data analysis toolkit. Sign up to save your spot!
- Heading to Strata in San Jose? Join Costin Leau, creator of Elasticsearch for Apache Hadoop, at the conference. He'll present on Search Evolved: Unraveling Your Data on Friday, Feb 20 at 11:30 AM. Plus, we're working on scheduling an Elasticsearch Silicon Valley Meetup to coincide with Costin's visit to the Bay Area. Stay tuned!
- Robyn Bergeron will be speaking at SCALE 13x on DevOps + Open Source == BFF Practices! Join her to learn more about DevOps (practice, theory, and otherwise!), shared habits of successful open source communities and DevOps practitioners, and tips for how you or your organization can start applying these habits today.
- The Chicago MySQL Users Group will be getting together on Feb 19 to talk MySQL Audit Logging and the ELK stack. Register now to save your seat.
Where to Find You
PSST! If you're a regular reader of This Week in Elasticsearch, a.k.a TWIES, you're thinking of skipping this section. You may even be thinking to yourself, yes of course I will drop a note on Twitter when I am giving a talk on all things ELK. That's awesome, because we'd like to showcase every meetup, conference presentation and workshop on Elasticsearch, Logstash, and Kibana happening worldwide. And now, we've made it even easier for you to get support for your meetup!
Head on over to our meetups page! (And we'll still totally send you swag if you're giving a talk on anything ELKy at a conference.)
Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note. We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.
If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:
- London - January 29, 2015 (Core Elasticsearch Training)
- Pune - February 2, 2015 (Hands on Workshop)
- Pune - February 3, 2015 (Core Elasticsearch Training)
- San Francisco - February 5, 2015 (Core Elasticsearch Training)
- Melbourne - February 9, 2015 (Core Elasticsearch Training)
- New York - February 11, 2015 (Hands on Workshop)
- New York - February 12, 2015 (Core Elasticsearch Training)
- Sydney - February 16, 2015 (Hands on Workshop)
- Sydney - February 17, 2015 (Core Elasticsearch Training)
- Amsterdam - February 18, 2015 (Hands on Workshop)
- Amsterdam - February 19, 2015 (Core Elasticsearch Training)
- Los Angeles - February 19, 2015 (Hands on Workshop)
- London - February 25, 2015 (Hands on Workshop)
- Bangalore - February 26, 2015 (Hands on Workshop)
- Bangalore - February 26, 2015 (Core Elasticsearch Training)
- Munich - March 24, 2015 (Core Elasticsearch Training)
- Munich - March 26, 2015 (Hands on Workshop)
- Stockholm - March 25, 2015 (Core Elasticsearch Training)
- Los Angeles - March 25, 2015 (Core Elasticsearch Training)
- Paris - March 25, 2015 (Core Elasticsearch Training)
- Paris - March 26, 2015 (Core Elasticsearch Training)