This Week in Elasticsearch - October 29, 2014
Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.
- Parent/child: Check if there is a search context, otherwise throw a query parse exception (#8177, master, 1.x, 1.4 and 1.3)
- Core: Add warmer listener only when index service is set, in order to prevent possible NPE (#8168, master and 1.x)
- Aggregations: the
childrenagg didn't take deleted document into account (#8180, master, 1.x and 1.4)
- Query API: Make
simple_query_stringleniency more fine-grained (#7967, master and 1.x)
- Core: Don't handle FNF exceptions when reading snapshot (#8086, master, 1.x and 1.4)
- Stats : Add time in index throttle to stats (#7861, master, 1.x and 1.4)
- Startup: Add
ES_INCLUDEsearch path (#4958, master and 1.x)
- Core: Enable
ClusterInfoServiceby default (#8206, master and 1.x)
- Term Vectors: support for distributed frequencies (#8144, master)
- Search API: Passing
fieddata_fieldsas a non array causes OOM (#8203, master, 1.x, 1.4 and 1.3)
- Allocation: Add rebalance enabled allocation decider (#8190, master and 1.x)
- Recovery: refactor
RecoveryTargetstate management (#8092, master and 1.x)
- Utilities: Introduce a
RefCountedinterface and basic impl (#8210, master and 1.x)
- Caching: Expose
concurrency_levelon all caches (#8112, master, 1.x and 1.4)
- Store: Don't catch FNF/NSF exception when reading metadata (#8207, master, 1.x, 1.4 and 1.3)
- ParseField: Support for when all fields are deprecated (#8067, master, 1.x and 1.4)
- Bulk: Fix 8125 hanged request when auto create index is off (#8125, master, 1.x and 1.4)
- MLT Query: versatile
likeparameter (#8039, master)
recoveryStatus.tryIncRefbefore accessing fields (#8231, master and 1.x)
- Core: Validate bool values in yaml for node settings (#8186, master, 1.x and 1.4)
- Aggregations: Return the sum of the doc counts of other buckets (#8213, master, 1.x and 1.4)
- Core: Cleanup non nested filter to not flip the bits in the
FixedBitSetreturned by the wrapped filter (#8232, master, 1.x, 1.4 and 1.3)
- Core: remove usage of deprecated
- Aliases: Throw exception if index is null or missing when creating an alias (#7863, master and 1.x)
- Core: Make
indexQueryParserServiceavailable (#8248, master, 1.x and 1.4)
- Packaging: Introduce
elasticsearch.in.bat(i.e. es.in for Windows) (#8237, master, 1.x, 1.4 and 1.3)
- Discovery: Improve handling of multicast binding exceptions (#8225, master, 1.x, 1.4 and 1.3)
- Packaging: Use the simpler command line syntax in the Debian init script (#5033, master and 1.x)
- Engine: Add current translog ID to commit meta before closing (#8245, master, 1.x, 1.4 and 1.3)
- Completion Suggester: Fix
CompletionFieldMapperto correctly parse weight (#8090, master, 1.x, 1.4 and 1.3)
Revealing the Uncommonly Common with Elasticsearch @bigdata @elasticmark http://t.co/8frP2nK9lI #StrataHadoop Webcast Oct 30 9am PT/ 12pm ET
— O'Reilly Strata (@strataconf) October 29, 2014
Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.
- A new version of the Ruby client has been released, adding performant HTTP client for JRuby (Manticore) and support for new Elasticsearch APIs. See the change log.
- Want to hear all about what's new in Elasticsearch, and how the project came into being? Have a listen to Binpress' podcast with our CTO, Shay Banon.
- Elasticsearch + Apache Spark + Python = data scientist heaven. Read on to hear from Sloan Ahrens all about how to get up and running.
- Steven Merill authored an excellent how to: Simplify Your Logstash Configuration. Curious about the mentioned drupal.org ELK stack upgrade? You can learn more from Steven's recent presentation at DrupalCon Amsterdam.
- Writing integration tests against Elasticsearch in Scala? Want to learn how? Orr Sella shared all the news you can use.
- Nell Alex shared a tutorial on using Elasticsearch for E-Commerce. It's the first post in a two part series, so stay tuned here for part 2!
- Dennis Schulte from our partner firm, codecentric, penned a tutorial on log management with the ELK stack for Spring Boot applications. (auf Deutsch)
- The Qiita engineering team recently migrated to Elasticsearch. You can read all about their new and improved user experience on their engineering blog. (日本語で)
.@rashidkpc talking about Kibana 3, 4 & #logstash at @elasticsearch Tempe meetup. All things ELK-y. pic.twitter.com/qznpXVZR5E
— robyn bergeron (@robynbergeron) October 24, 2014
Slides & Videos
Thanks SO much all who saw the #ElasticSearch #Boston meetup on #Geo replication tonight! Slides here (see comments): http://t.co/lNuERvca60
— sunnygleason (@sunnygleason) October 14, 2014
And from the oldies but goodies files, you can now watch Kevin Kluge and Steve Mayzak's talk from OSCON 2014, where they show you how to monitor data from a drone using ELK. You'll also learn how to literally crash your demo, plus fun stuff for all fans of the Internet of Things. Check it out!
Where to Find Us
We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Director of Developer Relations, Leslie Hawthorn, know!
David Pilato will host a Hands on Elasticsearch and Kibana workshop at Devoxx Belgium 2014. David's workshop kicks off on Monday, November 10th at 9:30 AM, and the conference runs from November 10-14th in Antwerp.
I'll be talking about #Logstash, #Elasticsearch and #Kibana at @BSidesTO -- ELK != Plain 'Ol Syslog - More info: http://t.co/iCaXCvDQbY
— Jason Kendall (@coolacid) October 29, 2014
- Honza Kral will present Analyzing and Searching Data using Elasticsearch at the Findwise Elasticsearch Breakfast Seminar on November 5th in Copenhagen. Registration is free of charge, but you'll need to sign up to attend.
- Simon Willnauer will keynote the DrivingIT Conference in Copenhagen. Stay tuned for more schedule details, but Simon will cover Randomized Testing during his presentation, a.k.a. "evil tests." The conference takes place on November 14, 2014.
- The 10th Elasticsearch Paris Meetup is on for November 13th. RSVPs won't open until two weeks before the meetup, so stay tuned to this space so you can get registered.
- You can also hear from David Pilato at the Drupagora conference on all matters Elasticsearch & Drupal. Join David on November 14th at 2:00 PM for his talk; the conference runs the full day in Paris.
For folks in Berlin, we'll have a booth at the GOTO Berlin conference, plus two awesome talks from Elasticsearchers:
- Britta Weber: Make sense of your logs
- Leslie Hawthorn: The Human Element in Development: What Your Tools Say about Your Company Culture
Join us at the Kosmos Event Center on November 6th and 7th for GOTO Berlin 2014!
And, we have a few of other great events coming up in Germany:
- Open Source Monitoring Conference, Nuremburg: Alexander Reelsen on Using the ELK stack in Systems Administration.
- data 2 day, Karlsruhe: Patrick Peschlow on Elasticsearch successfully at scale.
- Mimacom Days, Stuttgart: Dmitri Marx on Making Sense of your data to give you actionable insights.
Isabel Drost-Fromm will be giving two talks at ApacheCon Europe:
- Patches Welcome: Contributing to Apache Projects in a Nutshell - Nov. 17th at 2:40 PM
- Your search doesn't work: how to find out whether the search box you offer your users is helping - Nov. 19th at 10:40 AM
ApacheCon Europe is on November 17-21, 2014 in Budapest.
Our CEO, Steven Schuurman, will be speaking at the Web Summit conference. You can hear from Steven on Friday, November 6th on raising capital outside of Silicon Valley. Web Summit is on in Dublin November 4-6th.
Tomorrow, October 30th, we'll be working with Zimbra & Friends - the official Zimbra event in Italy (organized by Seacom) - to help show the ELK stack as a Zimbra log analyzer. You may also with to join us that evening at 5:30 PM for the second Elasticsearch Italy meetup, including talks and networking.
- The Japan Java Users Group Community Conference is coming up on November 15th in Tokyo. You can hear from our Developer Advocate, Jun Ohtani, on all things Elasticsearch and Kibana. Elasticsearch is proud to sponsor this great community event!
- The next Elasticsearch Study Session has been scheduled for November 18th at 7:30 PM. Register now to save your place.
ING on IT tour. Proud to host Meetup Events. Elastic Search next. Mon 17/11 press center AMP Amsterdam #werkenbijING http://t.co/wSaz46jpim
— Rob van Elburg (@RAVanElburg) October 29, 2014
Honza Kral will speak at PolyConf 2014 on Elasticsearch's language clients, Bridging the gap: one API, 5 languages. Honza takes the stage at 5:30 PM tomorrow night, October 30th, and the conference runs the 30th-31st in Poznan.
Honza Kral will be speaking at the Øredev conference in Malmö on Analyzing & Search Data Using Elastisearch. The conference runs Nov. 4-7th, and Honza takes the stage on Nov. 6th at 5:40 PM.
While Honza is visiting Sweden, we figured it would be a great time to host a meetup in Malmö. Please join us Friday, November 7th to hear from Honza on How to Make Sense of Your Big Data using Elasticsearch. Register now to save your place, and, even better, let us know if you're interested in sharing your story at the meetup!
The Elasticsearch team will be at the Big Data Analytics 2014 conference in London on November 13th and we are honored to have the UK's own Daily Mail present how Elasticsearch is helping to put newspapers ahead of the digital fold.
- New meetup: The Elasticsearch Seattle Meetup will convene on November 6th and you can hear from our Director of Solutions Architecture, Steve Mayzak, on Elasticsearch in the Real World. Register now to save your place.
- The Elasticsearch LAX meetup group will convene tonight, Oct. 29th to talk Multilingual text search + Elasticsearch from the bottom up. Register now to save your place.
- The next Elasticsearch San Francisco meetup is on for October 30th. Register now to save your place.
- Kurt Hurtado will speak at the San Francisco Cloud Mafia meetup on November 5th, where he'll talk the ELK stack in a DevOps environment. This meetup is incredibly popular, so get registered now if you'd like to attend.
Where to Find You
Our Director of Developer Relations, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, let her know.
Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note. We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.
If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:
- London - October 30, 2014 (Getting started workshop)
- Berlin - November 11, 2014 (Core Elasticsearch Training)
- Paris - November 17, 2014 (Getting started workshop)
- Bangalore - November 17, 2014 (Getting started workshop)
- Paris - November 18, 2014 (Core Elasticsearch Training)
- Bangalore - November 18, 2014 (Core Elasticsearch Training)
- London - November 26, 2014 (Core Elasticsearch Training)