This Week in Elasticsearch - May 21, 2014
Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.
- Cluster state: When sending shard start/failed message due to a cluster state change, use the master indicated in the new state rather than current (#6189, master and 1.x)
- Dependencies: Shade mustach dependency (#6192, master, 1.x and 1.1)
- Testing: Randomize MergeScheduler settings (#6194, master and 1.x)
- Testing: Recovery ID now is per node and not per JVM (#6207, master and 1.x)
- Plugins: Allow to require a certain lucene version (#4984, master and 1.x)
- Metadata: Unified
MetaData#concreteIndicesmethods into a single method that accepts indices (or aliases) and indices options (#6169, master and 1.x)
- Versioning: Allow 0 as a valid external version (#5662, master and 1.x)
- Snapshot/Restore API: Add Partial snapshot state (#5792, master and 1.x)
- Snapshot/Restore API: Switch to shared thread pool for all snapshot repositories (#6181, master and 1.x)
- More Like This Query: Added searching for multiple items (#5857, master and 1.x)
- Phrase suggester: Tie-break by term (#5978, master and 1.x)
- Query API: Handle multi match query for numeric only fields more gracefully
(#6215, master, 1.x and 1.1)
- Lucene: Don't report terms as live if all its docs are filtered out (#6221, master and 1.x)
- Query API:
limitfilter returns wrong results if deleted document are present (#6234, master, 1.x and 1.1)
- Benchmark API:
list()should return empty result if no bench nodes are available (#6146, master and 1.x)
- Lucene: Upgrade to 4.8.1 (commit, master and 1.x)
CharArraySetdoesn't know how to lookup the original string
ImmutableListfor stem exclusion (#6237, master, 1.x and 1.1)
- Recovery: Honor time delay when retrying (#6226, master and 1.x)
- Recovery: Before deleting a local unused shard copy, verify we're connected to the node it's supposed to be on (#6191, master and 1.x)
- Snapshot/Restore API: Add ability to snapshot replicating primary shards (#5531, master and 1.x)
- Benchmark API: Benchmark abort accepts wildcard patterns (#6185, master and 1.x)
- Indexing: Don't use
AllTokenStreamwhen no fields were boosted (#6187, master and 1.x)
- Cluster Update Settings API: Do not execute reroute if
minimum_master_nodesis not set (#6244, master and 1.x)
- Aggregations: Fail queries that have two aggregations with the same name (#6255, master and 1.x)
XFilteredQuerydefault strategy prefers query first in the deleted docs case (#6247, master and 1.x)
NOC Display with switching Tabs, powered by #elasticsearch, #kibana and some jQuery to parse #Icinga JSON Output pic.twitter.com/ohRUpFh72N
— Andri Steiner (@andristeiner) May 21, 2014
Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.
- In case you missed it, we released Kibana 3.1 last week.
- Zachary Tong shared a great how to post on building a negative connotation filter for Elasticsearch. Zach has a few more of these "bag of tricks" posts in the works, so stay tuned to this blog for more ways to make Elasticsearch sing.
- Levi Notik released a new Elasticsearch scala client named Sprastic.
- If you love Django and Elasticsearch, take a look at this Django debug toolbar panel for Elasticsearch. Grab the code from GitHub, thanks to Benoss.
- Alexander Beletsky shared some tips on migrating to Elasticsearch from MongoDB.
- Itamar Syn-Hershko authored a how to on using Elasticsearch as storage for your Git repos.
- Mark Paluch has some tasty tips for all you JBoss users who'd like a little Logstash integration love.
- The fine folks at Microsoft Research had a look at Logstash. If you're the kind of person who loves academic papers and logging, download the research findings as a PDF.
- Jettro Coenradie would like to teach you all about how to scripting advanced Kibana dashboards. The post starts with a lovely introduction to Kibana if you're less familiar with it.
- Njal Karevoll shared a how to on managing Elasticsearch fields when searching, a nifty discussion of ways to make sure your cluster performance isn't slowed down by how you use fields.
- Andrea Longhi shows how easy it is to add search support to your application with the new elasticsearch-model gem.
- Michael Pollmeier shared his first experiences using Elasticsearch for analytics. Love at first sight, indeed. We think you're pretty nifty, too, Michael.
- And from the community love department, Tom Anderson shared his experiences contributing to NEST, the official Elasticsearch .Net client. It has been a good week!
.@vodori is a happy #elasticsearch customer RT @vinjmarino speaking about @elasticsearch at #gotochgo. pic.twitter.com/pKoF0pcAAy
— Grant Gochnauer (@GrantGochnauer) May 20, 2014
We <3 you, too, Grant!
Slides & Videos
Honza Kral's slides from last week's DjangoCon EU conference
Alexander Reelsen on the ELK Stack, with an emphasis on Logstash
Jeff Sogolov will be at this Thursday's Chicago Meetup if you want to hear more about how "The ELK stack really saves companies millions of dollars"
Nik Everett treats his fellow Wikimedians to a crash course on Elasticsearch and how it is used for Wikipedia
See you soon @gluecon! @jordansissel and I are ready to get our @Elasticsearch #ELKstack on! pic.twitter.com/t55TcpRkKM
— Daniel Palay (@danielpalay) May 20, 2014
If you're heading to Gluecon 2014, say hello to Daniel & Jordan!
Where to Find Us
We'd love to feature all the great Elasticsearch, Logstash and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Community Manager, Leslie Hawthorn, know!
Great #bcnonrails beer & pizza! @xingdatascience: @elasticsearch at @xing_bcn with @clintongormley pic.twitter.com/mrLCiaNDLD
— XING Barcelona (@xing_bcn) May 16, 2014
Clinton Gormley showing off how the Elasticsearch Query DSL is not just for wizards
- You can get together with the fine folks from the BigBoards Garage user group in Aarschot on Thursday, May 22nd for a hackfest. People will be getting together at 7:30 PM, and looks like Logstash and Kibana will be amongst the tools to hack on! They're getting together again on June 3rd, too.
- The WAJUG in Liège will get together on Tuesday, May 27th to hear from Christoph Evers on Elasticsearch. Doors open at 7:00 PM.
The fine folks at the Polyglot Unconference in Vancouver, BC are hosting their annual event on May 23-25th. The conference kicks off with Ganesh Swami presenting a half day workshop on Getting Started with Elasticsearch.
- David Pilato will run a workshop on Elasticsearch and Kibana at the Breizhcamp 2014. The event runs from May 21-23rd in Rennes.
- The Seventh Elasticsearch France User Group meeting is coming up on June 10th in Paris. Doors open at 7:30 PM.
- The Elasticsearch Stuttgart meetup will convene their second meeting on May 26th at 7 PM. Topics will include running Elasticsearch on AWS and Google Cloud Engine.
- The Berlin .Net Users Group will talk Elasticsearch on Monday, May 26th at 7 PM.
- The Elasticsearch team will be at Berlin Buzzwords. (When we say the team, we mean most of our folks in the EU and several of our employees from the US. :)) We have many talks on the program and look forward to hosting you in the developer chill area, as well. Even better, the Berlin Elasticsearch User Group will convene a hackathon on Wednesday, May 28th. Please join us!
We've got not one but two awesome conferences going on in Amsterdam the week of June 16th:
- DevOps Days Amsterdam: Join us June 19-21st for an Elasticsearch workshop, a little on DevOps for Happier Humans and all the news you can use at the Elasticsearch table. Swing by to say hello to Leslie Hawthorn and Kurt Hurtado! Lee Hinmann will also join us for the workshop. Yay Lee!
- GOTO Amsterdam: Our CTO, Shay Banon, will treat the crowd to a deep dive on Elasticsearch's APIs, plus we'll have lots of knowledge and swag on offer at our booth. See you there!
The Torun JUG will get together on May 28th at 6 PM to talk about Lucene and all of her friends, with a spotlight on Elasticsearch.
- Alexander Reelsen will be speaking at Mimacom Days Zurich on June 4th. He will cover Elasticsearch - Beyond Full Text Search at 9:45 AM, directly after the conference welcome remarks.
- Alexander Reelsen and Britta Weber will be speaking at the Zurich Elasticsearch Meetup on June 7th. Alex will discuss What's new in Elasticsearch and Britta will cover the Significant Terms Aggregation. Doors open at 7 PM.
- The London Elasticsearch User Group has already announced that there next meeting will take place on June 4th. RSVP now to save your place - this one fills up very quickly!
- David Pilato will take the stage at DevoxxUK to discuss Advanced Search for Your Legacy Application DevoxxUK takes place June 12-13th in London, and David will speak on June 12th at 4 PM in Room 1.
- New meetup: The Elasticsearch New York City Meetup group will get together on June 4th at 6:30 PM. Agenda is still being finalized, but you'll hear from our core developers during the meetup. Many thanks to ThoughtWorks for hosting us!
- New meetup: The Bay Area Search Meetup will welcome our CTO, Shay Banon, on June 3rd. We're still sorting the final details, but register now. All of our Bay Area meetups fill lightning fast!
- New meetup: The San Francisco Elasticsearch Meetup will focus on all things logging and DevOps goodness. Join Kurt Hurtado and Gaurav Gupta to learn all about how to get more love from your logs.
- The Elasticsearch Silicon Valley Meetup group will convene tonight at 7 PM, and there's still space to register now. You'll hear from Kurt Hurtado on the Logstash team on using the ELK Stack in a DevOps Environment, plus engineers at LinkedIn will talk about how they use the ELK stack. Many thanks to LinkedIn for hosting us!
- Elasticsearch will have a table and some tasty treats at GOTO Chicago. The conference runs from May 20-21st. Make sure to stop by and say hello!
- The Elasticsearch Chicago Meetup group will have their fifth meeting on May 22nd. If you're staying in town an extra few days for GOTO Chicago, please join us! (And if you're in town anyway, you should also totally join us!) Doors open at 6:00 PM.
- The DevOps Columbus folks will welcome Stephen Smith to talk about his real world use case for the Logstash and the ELK stack. Doors open at 6:30 PM. That's Columbus, Ohio, US for anyone wondering which Columbus. :)
- Jordan Sissel will be speaking at Gluecon 2014! Make sure to catch his talk and visit the Elasticsearch booth. The conference runs from May 21-22nd in Bloomfield, Colorado.
- The Washington DC Elasticsearch Meetup group will get together on May 28th at 6:30 PM. You'll hear from engineers at AOL about Moloch, their open source network forensics tool built on top of Elasticsearch.
- Costin Leau will speak at Hadoop Summit North America on Real-time Analytics and Anomalies Detection using Elasticsearch, Hadoop and Storm. The conference runs from June 3-5th in San Jose, California. Costin will take the stage at 4:35 PM on June 3rd.
- The Philadelphia Elasticsearch Meetup group will be getting together for their second meetup on June 26th. Details are still in the works, but sign up now so you know where to go for ELK treats on the 26th.
Where to Find You
Our Community Manager, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, take a moment to let her know.
Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note. We care more about your skill set and passion for Elasticearch, Kibana and Logstash than where you rest your head.
トーキョー / Core elasticsearch on Jul 14 in Tokyo http://t.co/rMSiMUyqnA
— Jun Ohtani (@johtani) May 21, 2014
If you are interested in Elasticsearch training we have courses taught by our core developers coming up in: