16 Oktober 2013

This Week in Elasticsearch - October 16, 2013

Von Alexander Reelsen

Welcome to This week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.

Elasticsearch core

  • Support the has_child, has_parent, top_children queries and filters in count, explain (return dummy explanation) and delete by query APIs (#3822, 0.90 & master)
  • QueryBuilders#fuzzyQuery method accepts value parameter as an object (#3616, 0.90 & master)
  • Added IndexRequest#source(Object...) input validation (commit, 0.90 & master)
  • DateFieldMapper uses Locale.ROOT as default locale and does not rely on the system locale (#3852, 0.90 & master)
  • Year units in date math expressions are now supported (#3828, 0.90 & master)
  • Source filtering parameters of the Get Source API were inconsistent (#3886, master only)
  • Integration test for PluginManager to ensure downloading plugins was added (#3894, 0.90 & master)
  • Support date math for `origin` decay function parsing (#3892, 0.90 & master)
  • Fix GeoShapeQueryBuilder#toXContent (#3878, 0.90 & master)
  • Query String query now supports multiple fields regexp queries (#3901, 0.90 & master)
  • Set queue sizes by default on bulk/index thread pools (#3888, 0.90 & master)
  • Fielddata can now go off-heap by storing it into Lucene DocValues on disk (configurable via an index option) (#3806, master)
  • Segments API: Support merge id on segments, which returns a merge_id element in each segment of the segments API, allowing to group segments that are being merged as part of a single merge and indicate which ones are being merged now (#3904, 0.90 & master)
  • The has_child query's inner query now is wrapped in a filtered query with the child type as filter, this prevents other children from being returned as hit. (#3818, 0.90 & master)
  • Add match query support for stacked tokens (#3881, 0.90 & master)
  • A small bug in the query string query creation, using a word delimeter, using more than one field and containing a special char in the query was fixed (#3898, 0.90 & master)
  • Naming in function score was fixed (#3872, 0.90 & master)
  • The new percolator exposes more statistics (#3883, master)
  • Add support for Lucene SuggestStopFilter (#3913, 0.90 & master)
  • A cluster state update ack mechanism has been added, which ensures that cluster states are properly ackknowledged. Support for warmer and delete APIs has been added already (#3786, #3831, #3833, 0.90 & master)
  • A generic count down mechanism has been added instead of having to write this kind of functionality on your own (#3910, 0.90 & master)
  • Index requests which specify a parent are rejected now, if no parent type is defined (#3848, 0.90 & master)
  • Adding a parent mapping at runtime fails now (#3849, 0.90 & master)

Elasticsearch Ecosystem

Here's some more information about what is happening in the ecosystem we are maintaining around Elasticsearch, including plugin and driver releases, as well as news about Logstash and Kibana.

  • Last week Elasticsearch Inc held its second company All Hands meeting, with tons of technical discussion around future features and enhancements to Elasticsearch, Kibana and Logstash. Shay Banon provides a great in-depth summary of the conversations in this blog post.
  • Check out these interesting statistics and numbers from the ElasticHQ plugin.
  • es-nozzle is a scalable open source framework for connecting source content repositories like file systems or mail servers to elasticsearch
  • Matthias Nehlsen wrote a nice summary of this tiny application BirdWatch, which uses Play, AngularJS and Elasticsearch to show tweets in real-time.
  • Shelly Cloud produced a tutorial for adding search to Rails apps using Elasticsearch, Searchkick and typeahaed.js.
  • Any Stacker did a great write up on Indexing Project Gutenberg with MongoDB and Elasticsearch.
  • Itamar Syn-Hershko blogged about searching Hebrew text using Apache Lucene and Elasticsearch.
  • Igor Motov shared a write up of the recent Hopper Elasticsearch hackathon and his experience as one of the event mentors.

Slides

Last week, the Core Developer Team presented several lightning talks on Elasticsearch to more than 65 people at the Elasticsearch Netherlands User Group Meeting. You can check out all the presentations on Speakerdeck and read Zach Tong's excellent write up for more details on the lightning talks.

You might also want to check out the slides from Explore your data with Elasticsearch, a presentation by Honza Kral at last week's RuPy conference.

Where to find Us

Estonia

Honza Kral will explore Beyond Search at Topconf on Thursday, November 7th at 3:30 PM. Topconf runs November 6th and 7th in Tallinn.

France

David Pilato will present Elastify your app at the Nantes Java User Group meeting on November 4th. The meeting begins at 7:00 PM.

Germany

Ireland

Adrien Grand will present What is in a Lucene index? at Lucene Revolution on Thursday, November 7th at 2:45 PM. Lucene Revolution takes place November 4th - 7th in Dublin.

Norway

Alex Brasetvik from Found will present Elasticsearch: much more than just search at the javaBin Trondheim User Group on October 22nd.

Poland

Honza Kral will present Explore Your Data at PyCon Poland on Friday, October 18th. The conference runs Thursday, October 17th through Sunday, October 20th in Szczyrk.

Switzerland

David Pilato will be in Geneva on October 24th and 25th for the SoftShake Conference. You can visit with David in the Elasticsearch booth, and make sure to check out his presentation on Make sense of your (BIG) data! on Thursday at 4:30 PM.

United Kingdom

Richard Pijnenburg will give a talk at JAX London about how to Level Up Your Logging on Wednesday, October 30th.

United States

  • You can learn all about how Atlassian uses Elasticsearch for Heavy lifting in HipChat at the San Francisco Bay Area meetup on Monday, November 18th. Atlassian will be hosting the meetup at their offices and doors open at 6:30 PM.
  • Uri Boness  & Zach Tong will be presenting together at the Washington DC meetup on a very interesting combination of subjects- Elasticsearch'­s New Aggregations feature and Genomic Sequencing with Elasticsearch. The meetup is scheduled for October 22nd.
  • Shay Banon will be speaking at the inaugural Silicon Valley Elasticsearch Meetup on Wednesday, October 23rd. Our meetup hosts, Facebook, will also share what they've learned using Elasticsearch in a fast moving environment over the last few years.
  • Steve Mayzak will be speaking on Getting Started with Elasticsearch at the second Chicago Elasticsearch meetup, scheduled for October 24th.
  • Ralph Meijer from Mailgun will be talking at the San Francisco meetup scheduled for October 25th on The Business End of the [Mail]gun.
  • Martijn van Groningen will cover Document Relations with Elasticsearch at the Basis Open Source Search Conference in Chantilly, Virginia on Wednesday, November 6th. Martijn's presentation starts at 10:35 AM.
  • Martijn will also present at a Lunch and Learn session on New Features in Elasticsearch 1.0 at Verizon's offices in Chantilly, Virginia on Thursday, November 7th. OUr hosts, Verizon, will also treat the audience to information on their use of Elasticsearch. Doors open at 11:30 AM; please arrive early to give yourself plenty of time to check in at reception.
  • Rashid Khan will be attending Desert Code Camp in Phoenix, Arizona on Saturday, November 9th. If you'll also be there, have a chat with him on all things Kibana.

Where to Find You

Are you hosting an Elasticsearch meetup or giving a talk about Elasticsearch? We would love to know so we can feature that information in future editions of This Week in Elasticsearch. Just let our Community Manager know what you're up to and we're happy to help promote your efforts.

Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note.  We care more about your skill set and passion for Elasticearch, Kibana and Logstash than where you rest your head.

Trainings

If you are interested in Elasticsearch training we have courses taught by our core developers coming up in: