This Week in Elasticsearch - April 30, 2014

Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.

Elasticsearch core

  • Field data: Improved circuit breaker error messages to include name of the field that caused a circuit break (#5718, master and 1.x)
  • Field data: Code cleanup, removed unused or almost unused methods (#5874, master and 1.x)
  • Field data: Use segment ordinals as global ordinals when possible (#5873)
  • Field data: Made ordinals start from 0 (#5871, master and 1.x)
  • Field data: Improved global ordinals on low cardinality fields (#5854, master and 1.x)
  • Field data: Provided better error message if field has no field data type (#5979, master and 1.x)
  • Lucene: Enabled turning on IndexWriter's InfoStream (#5891, master and 1.x)
  • Lucene: Upgraded to Lucene 4.8 (#5932, master and 1.x)
  • Document versioning: Versioned get operations tests for version equality in all version types (#5929, master and 1.x)
  • Document versioning: Throw a request validation error when calling the update API with the external & external_gte version type as their usage can lead to data loss (#5929, master and 1.x)
  • Internal: Made Create/Update/Delete classes less mutable (commit, master and 1.x)
  • Internal: Don't lookup version for create operations with auto-generated id (#5917)
  • Internal: Upgraded to Guava 17 (#5953, master and 1.x)
  • Internal: Upgraded to Jackson 2.3.3 (commit, master and 1.x)
  • Search: Fail a DFS_QUERY_THEN_FETCH search request if all shards failed the query phase (commit, master and 1.x)
  • Query DSL: Thrown exception if an additional field was placed inside the query (#4913, master and 1.x)
  • Query DSL: Added support for multiple field values in function score query (#5940, master and 1.x)
  • Query DSL: Disabled parent/child queries in delete by query api as they can cause index corruptions (#5916, master and 1.x)
  • Query DSL: Made use of global ordinals in parent-child queries (#5846, master and 1.x)
  • Query DSL: Range/Term query/filter on dates fail to handle numbers properly (#5969, master, 1.x and 1.1)
  • Query DSL: Fixed sorting on nested documents (#5986, master, 1.x, 1.1 and 1.0)
  • Search templates: size and fields parameters were ignored if placed after the query in a search template (#5933, master, 1.x and 1.1)
  • Scripting: Disabled dynamic scripting by default (#5853, master and 1.x)
  • Aggregations: Removed abstraction that allowed to plug in different percentiles estimators (#5859, master and 1.x)
  • Aggregations: Delayed creation of aggregations/facets to QUERY phase for better object recycling (#5821, master and 1.x)
  • Aggregations: Improved bucket creation if the bucket already exists (#5955, master and 1.x)
  • Aggregations: Improved parsing of aggregations, made it more strict (#5827, master and 1.x)
  • Aggregations: Added global ordinal based implementation for significant terms aggregation (#5970, master and 1.x)
  • Aggregations: Improved the way sub-aggregations are collected (#5975, master and 1.x)
  • Cat apis: Cat allocation returns now no value instead of -1 for disk.total when not available (#5948, master and 1.x)
  • Cat apis: Cat thread_pool returns now the threadpool type and the configured sizes (#5366, master and 1.x)
  • Testing: Increased upper bound for number of replicas used in our tests: up to the number of the data nodes - 1 instead of 1 (# #5896, master and 1.x)
  • Testing: Added test that verifies no duplicates are created (commit, master and 1.x)
  • Testing: Added support for compatibility mode to our test infrastructure so that specific test features (e.g. randomization of number of shards) can be disabled via system property (#5897, master and 1.x)
  • Testing: Randomly introduced a client node in test clusters (#5949, master and 1.x)
  • Testing: Used URI rather than URL to access files from classpath (#5950, master and 1.x)

Elasticsearch Ecosystem

Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.

Slides & Videos


Jordan Sissel's presentation on the ELK stack from the OSDC 2014 conference


Boaz Leskes' presentation from NoSQL Matters Cologne 2014


Luca Cavanna on new features in 1.1 and Boaz Leskes treats us to an Elasticsearch Marvel Demo

Where to find Us

We'd love to feature all the great Elasticsearch, Logstash and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Community Manager, Leslie Hawthorn, know!

Austria

The Elasticsearch Vienna Meetup Group just formed! Join the group now to get updates on their first meeting.

Brasil

  • Elasticsearch is very excited to send our first speaker from the company to Brasil! Leslie Hawthorn, our Community Manager, will be returning to FISL once again to speak on Community 2.0: Beyond Using Software Livre. Leslie will speak on May 7th and FISL 15 runs from May 7-10th in Porto Alegre. Her talk will be in English but most of the talks on the program are given in BR-PT. Leslie will also be on hand to answer questions you may have about Elasticsearch, Logstash and Kibana.
  • During her visit for FISL 15, Leslie will also speak at the first ever Rio de Janiero Elasticsearch Meetup on May 5th.
  • If you are attending FISL 15 or make your home in or near Porto Alegre, please join us for the first Porto Alegre Elasticsearch Meetup on May 8th..

Bulgaria

Honza Kral will discuss how to Explore Your Data using Elasticsearch at the Bulgarian Web Summit 2014. The conference takes place on May 31st in Sofia.

France

  • Honza Kral will be speaking at DjangoConEU on From __icontains to search. The conference takes place May 13-17th on the Île des Embiez in France.
  • The sixth Elasticsearch France Meetup will take place on May 14th at Nuxeo's offices in Paris. Doors open at 7:30 PM on May 14th.
  • David Pilato will be attending dotSCale on May 19th in Paris. Don't miss his dotScale workshop, Elasticsearch Overview, on May 17th!
  • David Pilato will host an Elasticsearch workshop at the Solutions Linux Conference on May 20th. The conference runs from May 20-21st in Paris. If you don't have time to attend David's workshop, make sure to stop by the Elasticsearch booth to say hello!
  • David Pilato will run a workshop on Elasticsearch and Kibana at the Breizhcamp 2014. The event runs from May 21-23rd in Rennes, and David's workshop takes place on May 22nd.

Germany

  • Alexander Reelsen will be speaking at the Search Meetup Karslruhe on May 6th. Doors open at 7:15 PM.
  • Our friends at XING have created the Hamburg Elasticsearch User Group and will welcome Alexander Reelsen at the first meeting on Wednesday, May 7th. Alex will cover Using Elasticsearch, Logstash & Kibana to create realtime dashboards. Doors open at 6:30 PM.
  • The Elasticsearch Stuttgart Meetup will host their second meeting on May 26th. Attendees can look forward to hearing about Elasticsearch on AWS or Google Compute Engine. Doors open at 7 PM.
  • The Elasticsearch team will be at Berlin Buzzwords from May 25-28th.. (When we say the team, we mean most of our folks in the EU and several of our employees from the US. :)) We have many talks on the program and look forward to hosting you in the developer chill area, as well. Even better, the Berlin Elasticsearch User Group will convene a hackathon on Wednesday, May 28th. Please join us!

Norway

Martijn Laarman will be speaking on from text to full-text search at the NDC Oslo 2014 conference. The show runs from June 2-6th.

Romania

Costin Leau will speak at Topconf Bucharest 2014 on Big data real time search and analytics. Topconf Bucharest runs from June 10-13th and Costin will speak at 3:20 PM on June 12th.

Spain

Clinton Gormley has been invited to speak at the Barcelona on Rails Meetup on May 15th. Join him for a presentation on Elasticsearch's Query DSL: Not just for wizards! Doors open at 7 PM, and thanks to the fine folks at XING for hosting us!

Tunisia

David Pilato will speak at the Esprit JUG Days in Ariana. Further details of the conference schedule are forthcoming, but mark your calendar for May 7th and 8th. In the meantime, you may want to visit the conference's Facebook page.

United Kingdom

United States

  • New:Elasticsearch and MIT Sloan Data Analytics Hackathon scheduled for May 10th! Anyone can attend, not just students!
  • Aaron Mildenstein will be at DevOps Days Austin on May 5th and 6th. Make sure to stop by the Elasticsearch booth to say hello!
  • Rashid Khan will be presenting at Monitorama. The conference runs from May 5-7, 2014 in Portland, Oregon.
  • The Elasticsearch Boston Meetup group will be getting together on May 6th at 6:00 PM. You'll hear from Igor Motov on Improving Elasticsearch Resiliency and from the Yieldbot team on their use of Elasticsearch.
  • The Miami JVM Group will convene on May 13th and discuss an Introduction to Elasticsearch. Doors open at 7 PM.
  • Jordan Sissel will be speaking at Gluecon 2014! Make sure to catch his talk and visit the Elasticsearch booth. The conference runs from May 21-22nd in Bloomfield, Colorado.
  • The next Chicago Elasticsearch Meetup will take place on May 22nd. Doors open at 6 PM.
  • Costin Leau will speak at Hadoop Summit North America on Real-time Analytics and Anomalies Detection using Elasticsearch, Hadoop and Storm. The conference runs from June 3-5th in San Jose, California. Costin will take the stage at 4:35 PM on June 3rd.

Where to Find You

Our Community Manager, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, take a moment to let her know.

Oh yeah, we're also  hiring. If you'd like us to find you for employment purposes, just drop us a note.  We care more about your skill set and passion for Elasticearch, Kibana and Logstash than where you rest your head.

Trainings

If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:

  • Copenhagen - May 13, 2014 (core Elasticsearch training)
  • Paris - May 15, 2014 (core Elasticsearch training)
  • Seattle - June 3, 2014 (core Elasticsearch training)
  • New York - June 4, 2014 (core Elasticsearch training)
  • London - June 4, 2014 (core Elasticsearch training)
  • Zurich - June 5, 2014 (core Elasticsearch training)
  • San Francisco - June 6, 2014 (ELK stack hands on workshop)
  • Amsterdam - June 27, 2014 (ELK stack hands on workshop)