21 May 2014

This Week in Elasticsearch - May 21, 2014

By Alexander Reelsen

Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.

Elasticsearch Core

  • Cluster state: When sending shard start/failed message due to a cluster state change, use the master indicated in the new state rather than current (#6189, master and 1.x)
  • Dependencies: Shade mustach dependency (#6192, master, 1.x and 1.1)
  • Testing: Randomize MergeScheduler settings (#6194, master and 1.x)
  • Testing: Recovery ID now is per node and not per JVM (#6207, master and 1.x)
  • Plugins: Allow to require a certain lucene version (#4984, master and 1.x)
  • Metadata: Unified MetaData#concreteIndices methods into a single method that accepts indices (or aliases) and indices options (#6169, master and 1.x)
  • Versioning: Allow 0 as a valid external version (#5662, master and 1.x)
  • Snapshot/Restore API: Add Partial snapshot state (#5792, master and 1.x)
  • Snapshot/Restore API: Switch to shared thread pool for all snapshot repositories (#6181, master and 1.x)
  • More Like This Query: Added searching for multiple items (#5857, master and 1.x)
  • Phrase suggester: Tie-break by term (#5978, master and 1.x)
  • Query API: Handle multi match query for numeric only fields more gracefully (#6215, master, 1.x and 1.1)
  • Lucene: Don't report terms as live if all its docs are filtered out (#6221, master and 1.x)
  • Query API: limit filter returns wrong results if deleted document are present (#6234, master, 1.x and 1.1)
  • Benchmark API: list() should return empty result if no bench nodes are available (#6146, master and 1.x)
  • Lucene: Upgrade to 4.8.1 (commit, master and 1.x)
  • Mapping: CharArraySet doesn't know how to lookup the original string
    in an ImmutableList for stem exclusion (#6237, master, 1.x and 1.1)
  • Recovery: Honor time delay when retrying (#6226, master and 1.x)
  • Recovery: Before deleting a local unused shard copy, verify we're connected to the node it's supposed to be on (#6191, master and 1.x)
  • Snapshot/Restore API: Add ability to snapshot replicating primary shards (#5531, master and 1.x)
  • Benchmark API: Benchmark abort accepts wildcard patterns (#6185, master and 1.x)
  • Indexing: Don't use AllTokenStream when no fields were boosted (#6187, master and 1.x)
  • Cluster Update Settings API: Do not execute reroute if minimum_master_nodes is not set (#6244, master and 1.x)
  • Aggregations: Fail queries that have two aggregations with the same name (#6255, master and 1.x)
  • Query: XFilteredQuery default strategy prefers query first in the deleted docs case (#6247, master and 1.x)

Elasticsearch Ecosystem

Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.


We <3 you, too, Grant!

Slides & Videos


Honza Kral's slides from last week's DjangoCon EU conference


Alexander Reelsen on the ELK Stack, with an emphasis on Logstash


Jeff Sogolov will be at this Thursday's Chicago Meetup if you want to hear more about how "The ELK stack really saves companies millions of dollars"


Nik Everett treats his fellow Wikimedians to a crash course on Elasticsearch and how it is used for Wikipedia


If you're heading to Gluecon 2014, say hello to Daniel & Jordan!

Where to Find Us

We'd love to feature all the great Elasticsearch, Logstash and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Community Manager, Leslie Hawthorn, know!


Clinton Gormley showing off how the Elasticsearch Query DSL is not just for wizards

Austria

The Elasticsearch Vienna Meetup Group has scheduled their first meetup for Thursday, June 12th. Please join us at 7 PM to hear from Alexander Reelsen on What's New in Elasticsearch.

Belgium

  • You can get together with the fine folks from the BigBoards Garage user group in Aarschot on Thursday, May 22nd for a hackfest. People will be getting together at 7:30 PM, and looks like Logstash and Kibana will be amongst the tools to hack on! They're getting together again on June 3rd, too.
  • The WAJUG in Liège will get together on Tuesday, May 27th to hear from Christoph Evers on Elasticsearch. Doors open at 7:00 PM.

Bulgaria

Honza Kral will discuss how to Explore Your Data using Elasticsearch at the Bulgarian Web Summit 2014. The conference takes place on May 31st in Sofia.

Canada

The fine folks at the Polyglot Unconference in Vancouver, BC are hosting their annual event on May 23-25th. The conference kicks off with Ganesh Swami presenting a half day workshop on Getting Started with Elasticsearch.

France

  • David Pilato will run a workshop on Elasticsearch and Kibana at the Breizhcamp 2014. The event runs from May 21-23rd in Rennes.
  • The Seventh Elasticsearch France User Group meeting is coming up on June 10th in Paris. Doors open at 7:30 PM.

Germany

  • The Elasticsearch Stuttgart meetup will convene their second meeting on May 26th at 7 PM. Topics will include running Elasticsearch on AWS and Google Cloud Engine.
  • The Berlin .Net Users Group will talk Elasticsearch on Monday, May 26th at 7 PM.
  • The Elasticsearch team will be at Berlin Buzzwords. (When we say the team, we mean most of our folks in the EU and several of our employees from the US. :)) We have many talks on the program and look forward to hosting you in the developer chill area, as well. Even better, the Berlin Elasticsearch User Group will convene a hackathon on Wednesday, May 28th. Please join us!

The Netherlands

We've got not one but two awesome conferences going on in Amsterdam the week of June 16th:

  • DevOps Days Amsterdam: Join us June 19-21st for an Elasticsearch workshop, a little on DevOps for Happier Humans and all the news you can use at the Elasticsearch table. Swing by to say hello to Leslie Hawthorn and Kurt Hurtado! Lee Hinmann will also join us for the workshop. Yay Lee!
  • GOTO Amsterdam: Our CTO, Shay Banon, will treat the crowd to a deep dive on Elasticsearch's APIs, plus we'll have lots of knowledge and swag on offer at our booth. See you there!

Norway

Martijn Laarman will be speaking on from text to full-text search at the NDC Oslo 2014 conference. The show runs from June 2-6th.

Poland

The Torun JUG will get together on May 28th at 6 PM to talk about Lucene and all of her friends, with a spotlight on Elasticsearch.

Romania

Costin Leau will speak at Topconf Bucharest 2014 on Big data real time search and analytics. Topconf Bucharest runs from June 10-13th and Costin will speak at 3:20 PM on June 12th.

Russia

Honza Kral will speak at PyCon Ru. The schedule is still being finalized, but mark your calendars for June 2nd and 3rd. If you're heading to PyCon Ru, make sure to say hello to Honza!

Switzerland

  • Alexander Reelsen will be speaking at Mimacom Days Zurich on June 4th. He will cover Elasticsearch - Beyond Full Text Search at 9:45 AM, directly after the conference welcome remarks.
  • Alexander Reelsen and Britta Weber will be speaking at the Zurich Elasticsearch Meetup on June 7th. Alex will discuss What's new in Elasticsearch and Britta will cover the Significant Terms Aggregation. Doors open at 7 PM.

United Kingdom

United States

  • New meetup: The Elasticsearch New York City Meetup group will get together on June 4th at 6:30 PM. Agenda is still being finalized, but you'll hear from our core developers during the meetup. Many thanks to ThoughtWorks for hosting us!
  • New meetup: The Bay Area Search Meetup will welcome our CTO, Shay Banon, on June 3rd. We're still sorting the final details, but register now. All of our Bay Area meetups fill lightning fast!
  • New meetup: The San Francisco Elasticsearch Meetup will focus on all things logging and DevOps goodness. Join Kurt Hurtado and Gaurav Gupta to learn all about how to get more love from your logs.
  • The Elasticsearch Silicon Valley Meetup group will convene tonight at 7 PM, and there's still space to register now. You'll hear from Kurt Hurtado on the Logstash team on using the ELK Stack in a DevOps Environment, plus engineers at LinkedIn will talk about how they use the ELK stack. Many thanks to LinkedIn for hosting us!
  • Elasticsearch will have a table and some tasty treats at GOTO Chicago. The conference runs from May 20-21st. Make sure to stop by and say hello!
  • The Elasticsearch Chicago Meetup group will have their fifth meeting on May 22nd. If you're staying in town an extra few days for GOTO Chicago, please join us! (And if you're in town anyway, you should also totally join us!) Doors open at 6:00 PM.
  • The DevOps Columbus folks will welcome Stephen Smith to talk about his real world use case for the Logstash and the ELK stack. Doors open at 6:30 PM. That's Columbus, Ohio, US for anyone wondering which Columbus. :)
  • Jordan Sissel will be speaking at Gluecon 2014! Make sure to catch his talk and visit the Elasticsearch booth. The conference runs from May 21-22nd in Bloomfield, Colorado.
  • The Washington DC Elasticsearch Meetup group will get together on May 28th at 6:30 PM. You'll hear from engineers at AOL about Moloch, their open source network forensics tool built on top of Elasticsearch.
  • Costin Leau will speak at Hadoop Summit North America on Real-time Analytics and Anomalies Detection using Elasticsearch, Hadoop and Storm. The conference runs from June 3-5th in San Jose, California. Costin will take the stage at 4:35 PM on June 3rd.
  • The Philadelphia Elasticsearch Meetup group will be getting together for their second meetup on June 26th. Details are still in the works, but sign up now so you know where to go for ELK treats on the 26th.

Where to Find You

Our Community Manager, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, take a moment to let her know.

Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note.  We care more about your skill set and passion for Elasticearch, Kibana and Logstash than where you rest your head.

Trainings

If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:

  • New York - June 4, 2014 (core Elasticsearch training)
  • Zurich - June 5, 2014 (core Elasticsearch training)
  • San Francisco - June 6, 2014 (ELK workshop)
  • Amsterdam - June 27, 2014 (ELK workshop)
  • Seoul - July 10, 2014 (core Elasticsearch training)
  • Tokyo - July 14, 2014 (core Elasticsearch training)