This Week in Elasticsearch - July 23, 2014

Par
Alexander Reelsen

Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.

Elasticsearch Core

  • Aggregations: Fixed Histogram key_as_string bug (#6655, master 1.4 and 1.3)
  • Threadpool Info: Allow to serialize negative thread pool sizes properly (#6325, master, 1.4 and 1.3)
  • Fielddata: Use KeyedLock in <code>IndexFieldDataService (#6855, master and 1.4)
  • Search: Added support for named filters/queries inside nested filters and queries (#6293, master and 1.4)
  • Query: When parent doc is found stop iterating over segments and continue with the next top matching child do (#6895, master and 1.4)
  • Scripting: Remove setNextScore in <code>SearchScript (#6864, master and 1.4)
  • Scripts: Make ScoreAccessor utility available for other script engines (#6898, master and 1.3)
  • Plugins: bin/plugin may never try to remove itself (#6745, master, 1.3 and 1.2)
  • Refactoring: Rename FieldMapper.termsFilter to <code>fieldDataTermsFilter (#6888, master and 1.4)
  • Infra: Remove indicesLifecycle.Listener from <code>IndexingMemoryController (#6892, master and 1.3)
  • Store: Delete unallocated shards under a cluster state task (#6902, master and 1.3)
  • ResourceWatcher: Added three frequency levels for resource watching (#6896, master and 1.3)
  • Bulk API: Add blocking close method to <code>BulkProcessor (#6314, master and 1.3)
  • Serialization: Return null info if service is not started (#6906, master and 1.3)
  • Term Vectors API: Compute term vectors on the fly if not stored (#6567, master and 1.4)
  • Client: Unknown node version should be a lower bound (#6894, master and 1.4)
  • Serialization: Better message for invalid internal transport message format (#6916, master and 1.3)
  • Plugin Manager: Properly quote $JAVA in <code>bin/plugin (#5765, master, 1.4, 1.3 and 1.2)
  • Plugins: Introduced pluggable filter chain to be able to filter transport actions execution (#6921, master and 1.4)
  • Netty: Refactoring to make MessageChannelHandler extensible (#6915, master and 1.4)
  • Aggregations: Track scores should be applied properly for top_hits aggregation (#6934, master and 1.4)
  • Engine: index.fail_on_corruption is now dynamically updateable (#6941, master, 1.4 and 1.3)
  • Snapshot/Restore: Fail restore if snapshot is corrupted (#6938, master, 1.4 and 1.3)
  • Aggregations: More lenient type parsing in histo/cardinality aggs (6948#, master and 1.4)
  • Discovery: Remove unneeded cluster state serialization during cluster join process (#6949, master and 1.4)
  • Fielddata: Switch to Lucene DV APIs (#6908, master and 1.4)
  • Core: allow index.merge.scheduler.max_thread_count to be updated dynamically (#6925, master and 1.4)
  • Netty: Fix connect concurrency, can cause connection nodes to close (#6964, master and 1.4)
  • Discovery: Only disconnect from temporary connected nodes in unicast discovery (#6966, master and 1.4)
  • PhraseSuggester: Collate option should not allow returning phrases with no matching docs (#6927, master and 1.4)
  • Core: Drop UnsafeUtils (#6962, master and 1.4)
  • Get API: Make _all field accessible (#6924, master and 1.4)
  • Core: Don't load bloom filters by default (#6959, master and 1.4)

Elasticsearch Ecosystem

Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.

Slides & Videos

For all you Hadoop lovers out there: Holden Karau on Streamlining Search Indexing using Elasticsearch and Spark

And some more Elasticsearch & Hadoop love, this time from folks in the .gov space

How to use Elasticsearch to make your Scala sing

Where to Find Us

We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Community Manager, Leslie Hawthorn, know!

Germany

  • New conference: Elasticsearch is a proud sponsor of eurucamp 2014. If you'll be attending, make sure to say hello to Colin Surprenant and ask him about our open positions on the Logstash engineering team.
  • EuroPython comes to Berlin this year, with the conference running from July 21-27th at the Berlin Congress Center. Please stop by the Elasticsearch table to learn all about how the ELK stack can make life better for Pythonistas. Make sure to catch Honza Kral, Django and Elasticsearch core developer, on lessons learned from building the Python Elasticsearch client. Honza will take the stage at 2:00 PM on July 24th. Bonus: we're hosting a meetup during the conference on Thursday, July 28th from 6-8 PM. Mark your calendar now, and keep an eye out for registration details soon!
  • Following on from EuroPython, Honza Kral will be speaking at the PyData Conference on Making Sense of your big data with Elasticsearch. Honza presents on July 27th at 2:10 PM, and the PyData Conference will also be held at the Berlin Congress Center.

Israel

  • The Second Elasticsearch Tel Aviv Meetup will convene on July 28th, focusing on real world use cases. The agenda is now finalized and you can hear from three different companies on how they use Elasticsearch. You can register now.
  • The OpsTalk Meetup Group will host a workshop in Tel Aviv on August 4th from 6-9 PM. The organizers are looking to teach you all about the wonders of log analysis using the ELK stack. Register now to save your place.

Poland

The Warsaw Java Users Group is holding a Microservices Hackathon on August 9th from 10 AM to 10 PM. You can spend those twelve hours hacking on the ELK stack and a variety of other tools. Register now to attend.

United States

  • New Meetup: Join us for the inaugural Elasticsearch Meetup in Minneapolis! We're going casual style, so come talk shop with us over beers on July 31st. Register now so we order sufficient noms.
  • The annual O'Reilly Open Source Conference is back again in Portland, Oregon! The Elasticsearch team will be out in force, with two talks, an ELK BoF and a lovely table in the exhibit hall where we can answer all of your questions. Look out for more details in next Monday's edition of Where in the World is Elasticsearch. We hope to see many of our friends there! OSCON runs from July 20-24th at the Oregon Convention Center.
  • Living in Portland but not planning to attend OSCON? No worries! We have an Elasticsearch Meetup co-located with our ELK BoF TONIGHT Wednesday, July 23rd. Register now to join us for the festivities, including real-world use case story from Build Energy!
  • If you haven't gotten enough of the Pacific Northwest with all things OSCON, then head on over to CascadiaJS on July 31st - August 1st. Joe Fleming from the Kibana engineering team will be attending, so say hello to him in the hallways!

Where to Find You

Our Community Manager, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, take a moment to let her know.

Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note.  We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.

Trainings

If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:

  • New York - August 12, 2014 (core Elasticsearch training)
  • Sao Paulo - August 14, 2014 (core Elasticsearch training)
  • Rio de Janeiro - August 18, 2014 (core Elasticsearch training)
  • Amsterdam - August 20, 2014 (core Elasticsearch training)
  • Boston - August 27, 2014 (core Elasticsearch training)
  • New York - August 27, 2014 (ELK workshop)
  • San Francisco - August 28, 2014 (ELK workshop)