This Week in Elasticsearch - April 09, 2014
Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.
- Lucene internal: Added a frequency caching terms enum, used by aggregations and the phrase suggester (#5597, master, 1.x and 1.1)
- Bulk API: Fix possible missing mapping creation (#5623, master, 1.x, 1.1 and 1.0)
- Scripting internal:
ScriptDocValues.EMPTYdoesn't implement <code>getValue (#5546, master, 1.x, 1.1 and 1.0)
- Suggest:Require precision in mapping for geo context suggestions (#5621, master, 1.x and 1.1)
- Bulk API: Added
BulkProcessor.flush()method (#5575, master and 1.x)
- Parent/Child: Moved the decision to load fielddata eagerly to a better place (#5569, master and 1.x)
- Internals: Fix possible
NullPointerExceptionin <code>PagedBytesReference (#5667, master and 1.x)
- Mapping: Support null values for
geo_pointtype (#5680, master, 1.x and 1.1)
- Startup: Allow to configure to display stack traces during start up (#5102, master and 1.x)
- Settings API: Add
getAsRatioto <code>Settings class (#5690, master and 1.x)
- Internal: Use a new releasable bytes output, which can be recycled, used netty and the translog, where the recycling will help (#5691, master and 1.x)
- Fielddata: Added support for global ordinals (#5672, master and 1.x)
- Mapping: Use doc-values for binary fields (#5669, master and 1.x)
- Lucene: Added lucene
LMSimilaritysimilarity (#5697, master and 1.x)
LongHashadd/key not consistent (#5693, master and 1.x)
- Internal: Renamed
readPrimitive*Array()/writePrimitive*Array()methods in StreamInput/StreamOutput (#5710, master abd 1.x)
- REST API: Refactoring and code deduplication (#5708, master 1.x)
- Allocation: Ensure failed shards are not re-assigned to the same nodes if multiple replicas failed at once (#5725, master, 1.x, 1.1 and 1.0)
- Parent/Child: Fix assertions for rewrite reader (#5731, master and 1.x)
Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.
- Costin Leau has released Elasticsearch for Apache Hadoop 1.3 M3. You can read all about the new features. Take the code for a spin and let us know what you think!
- Richard Pijnenburg announced the release of our Elasticsearch Puppet modules v 0.3.0 on Monday. This blog post has more information, including information on Augeaus support, new distro support, and more.
- Adrien Grand shared a tutorial on the cardinality aggregation in Elasticsearch 1.1.
- elasticsearch-ruby: Check out the working draft of using Elasticsearch as a persistence layer for Ruby objects using the repository pattern. An example application built with Sinatra is available in the bundled examples. Live demo temporarily available here.
- Kevin Wang recently submitted a pull request to add doc values for binary fields to Elasticsearch, which was merged into trunk. This functionality will make Kevin's Image Plugin for Elasticsearch much more efficient and will give Elasticsearch plugin authors more power when doing complex scoring. Check out the demo of Kevin's work in action!
- Alfred, the Elasticsearch butler, has been released. Alfred is a tool to handle maintenance work on Elasticsearch.
- Matt Gregory shared a post on how Benefitfocus uses the ELK stack to minimize misery and maximize visibility within their organization.
- Josh Reichardt wrote up a great introduction to the ELK stack. Good piece all around, especially nice for sys admins.
- Anders Aarvik also shared a wonderful introduction to the ELK stack, great for the first time user.
- Jeremy Tinley shared an excellent how to on running the ELK stack in Docker. If you're using the ELK stack and love Docker, or vice versa, you may also be interested in gsogol's docker-elk Dockerfile or qnib's docker-elk.
- Laurent Doguin shared a how to on Monitoring Nuxeo Docker Container Logs with Logstash, Elasticsearch and Kibana.
- Alex Brasetvik shared a tutorial on Sizing Elasticsearch: Scaling Up and Scaling Out.
- Yannick Pereira-Reis shared an article on setting up Elasticsearch with Vagrant. (en français)
Attendees of the first Elasticsearch Korea User Group share a meal after the meeting.
Photo courtesy of JongMin Kim.
Slides & Videos
Graham Tackley, Director of Guardian News and Media, speaks on Democratizing Attention Data at The Guardian using Elasticsearch
Health Market Science tell us how they use Elasticsearch during the first ever Elasticsearch Philadelphia Meetup
JongMin Kim's presentation at the first Elasticsearch Korea User Group meeting
Full house at the April Elasticsearch Amsterdam Meetup, where Luca Cavanna talked about new features in 1.1
Where to Find Us
We'd love to feature all the great Elasticsearch, Logstash and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Community Manager, Leslie Hawthorn, know!
Elasticsearch is very excited to send our first speaker from the company to Brasil! Leslie Hawthorn, our Community Manager, will be returning to FISL once again to speak on Community 2.0: Beyond Using Software Livre. Leslie will speak on May 7th and FISL 15 runs from May 7-10th in Porto Alegre. Her talk will be in English but most of the talks on the program are given in BR-PT. Leslie will also be on hand to answer questions you may have about Elasticsearch, Logstash and Kibana.
- Honza Kral will be attending PyCon North America and the accompanying Django sprints. Make sure to stop by and hear more from him during his poster session Collect and Analyze your logs with Logstash, Elasticsearch and Kibana.
- The Montreal Elasticsearch User Group will welcome Honza Kral and Colin Suprenant for their second meeting on April 16th. Doors open at 6 PM.
- David Pilato and Adrien Grand will both be at Devoxx France 2014, where they will co-present the Hands on Elasticsearch workshop. Devoxx France runs April 16-18th in Paris.
- Honza Kral will be speaking at DjangoConEU on From __icontains to search. The conference takes place May 13-17th on the Île des Embiez in France.
- The sixth Elasticsearch France Meetup will take place on May 14th at Nuxeo's offices in Paris. Details on the talks are still forthcoming, but for now save the date for May 14th.
- Jordan Sissel will speak at the Open Source Data Center Conference in Berlin, discussing how to Find Happiness in Your Logs. The conference will run from April 8-10th. He will also be speaking at PuppetCamp Berlin, directly following the Open Source Data Center Conference in Berlin on April 11th. Please plan to stop by the Elasticsearch table at PuppetCamp Berlin to say hello to Jordan and Leslie!
- The first ever Frankfurt Elasticsearch Meetup will convene on April 14th. The first meeting will be an informal gathering to discuss future talks, meeting times and to enjoy each others' company. Plan to join the meetup at 7 PM.
- The Search Meetup Munich group will get together on April 15th. Our own Alexander Reelsen is the group's organizer, and will likely treat you to some delightful bits on Elasticsearch. Doors open at 7 PM.
- Boaz Leskes will be speak at NoSQL Matters Cologne on Elasticsearch: Deep dive into analytics using Aggregations. The conference runs from April 29-30th.
- Our friends at XING have created the Hamburg Elasticsearch User Group and will welcome Alexander Reelsen at the first meeting on Wednesday, May 7th. Alex will cover Using ElasticSearch, Logstash & Kibana to create realtime dashboards. Doors open at 6:30 PM.
David Pilato will present Make sense of your (BIG) data! at the Codemotion Roma Conference The conference takes place in Rome on April 11-12th, and David will speak at 2:10 PM on the 12th.
The 4th Elasticsearch study session has been scheduled by Jun Ohtani. Please plan to join the meeting at 7 PM on April 21st.
Clinton Gormley has been invited to speak at the Barcelona on Rails Meetup on May 15th. Join him for a presentation on Elasticsearch's Query DSL: Not just for wizards! Doors open at 7 PM, and thanks to the fine folks at XING for hosting us!
Clinton Gormley will take the stage at ScaleConf to talk Scaling real time search and analytics with Elasticsearch. The conference runs April 10th and 11th in Cape Town.
David Pilato will speak at the Esprit JUG Days in Ariana. Further details of the conference schedule are forthcoming, but mark your calendar for May 7th and 8th. In the meantime, you may want to visit the conference's Facebook page.
- Mark Harwood will be sharing his analysis of using Elasticsearch to find the "uncommonly common" results in searches - think fraud or anomaly detection - at the Financial Engineers and & Quants London Meetup. Join Mark on Thursday, April 24th at 6:30 PM at SkillsMatter. The organizers ask that you kindly register on both meetup.com and SkillsMatter website. A link to the registration form for SkillsMatter is include on the meetup.com invitation to the event. Attendance is free of charge.
- David Pilato will take the stage at DevoxxUK to discuss Advanced Search for Your Legacy Application DevoxxUK takes place June 12-13th in London, and David will speak on June 12th at 4 PM in Room 1.
- You can still join us for tonight's Elasticsearch Austin Meetup. Attendees will hear from from Zachary Tong on Elasticsearch & Query Optimization, and from Aaron Mildenstein on Logstash. Doors open at 6:30 PM.
- For folks in Washington, DC, the Elasticsearch Meetup group will convene on April 10th at LivingSocial's offices. Their engineering team will present their Elasticsearch use case.
- Cameron Lane will be talk Elasticsearch and Python clients at the Python Atlanta Meetup on April 10th.
- Kevin Kluge will present ELK: How Elasticsearch, Logstash, and Kibana provide real-time data enlightenment at the DevNation Conference. Kevin's talk will take place on Thursday, April 17th at 9:45 AM. DevNation, co-located with Red Hat Summit, runs April 13-17 2014 at Moscone Center in San Francisco. Elasticsearch will be participating in the Partner Pavilion at Red Hat Summit, so if you are attending either conference we hope you will stop by our booth to say hello!
- The Miami JVM Group will convene on May 13th and discuss an Introduction to Elasticsearch. Doors open at 7 PM.
Where to Find You
Our Community Manager, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, take a moment to let her know.
Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note. We care more about your skill set and passion for Elasticearch, Kibana and Logstash than where you rest your head.
If you are interested in Elasticsearch training we have courses taught by our core developers coming up in: