Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.
- Core: Ensure
index.version.createdis consistent (#6660, master and 1.3)
- REST GET Template API: Executiong did not honor the
flat_settingsparameter (#6671, master and 1.3)
- Search Java API: Fix source excludes setting if no includes were provided (#6632, master, 1.3 and 1.2)
IndexingMemoryControllershould only update buffer settings of fully recovered shards (#6667, master and 1.3)
- Bulk API: Fix return of wrong request type on failed updates (#6630, master, 1.3 and 1.2)
- Discovery: Handle ConnectionTransportException during a Master/Node fault detection ping (#6686, master and 1.3)
- Mapping: Better logic on sending mapping update new type introduction (#6669, master and 1.3)
- Cleanup: Shard bulk mapping update to only use type (#6695, master and 1.3)
- Lucene: Add factories for some Tokenizers/TokenFilters from Lucene (#5935, master and 1.3)
- Core: More resource efficient analysis wrapping usage (#6714, master and 1.3)
- Query API:
QueryParsercan return null from a query (#6722, master, 1.3, 1.2 and 1.1)
- Core: The query string cache should return cloned Query instances (#6733, master, 1.3 and 1.2)
- Query API: Remove
Queries#optimizeQuery(...), is it is done internally already (#6743, master and 1.3)
- Mapping: Improve performance for many new fields introduction in mapping (#6707, master and 1.3)
- Pending tasks API: Include current executing tasks (#6744, master and 1.3)
- Mapping: Call callback only if actual mapping was processed (#6748, master and 1.3)
- Dependencies: Upgrade Jackson to 2.4.1 (#6757, master and 1.3)
- PluginManager: tests for missing plugin name when passing
--url(#6013, master and 1.3)
- PluginManager: Removing plugin does not fail when plugin dir is read only (#6546, master, 1.3 and 1.2)
- Allocation: Disk-aware allocation decider allows initial primary allocation if under the high watermark (#6196, master and 1.3)
- Query DSL: Improved explanation for
match_phrase_prefix(#2449, master and 1.3)
- Relocation: Process pending mapping update in phase 2 (#6762, master and 1.3)
- Core: don't replace indices within the
ActionRequestwith the concrete ones, and make sure check blocks is executed on concrete indices (#6777, master and 1.3)
- Discovery verify connect when sending a rejoin cluster request(#6779, master and 1.3)
- Aggregations: Aggregation names can now include dot (#6702, master and 1.3)
- Core: Trigger refresh when the live version map is using too much RAM (#, master and 1.3)
- Translog: Change
flush_threshold_opsto unlimited by default, so we only flush by byte size by default (#6783, master and 1.3)
- Analysis: Add missing pre built analysis components (#6770, master and 1.3)
- Mapping: Remove intern calls on
FieldMapper#Names(#6747, master and 1.3)
- Engine: Prevent NPE if engine is closed while version map is checked (#6786, master and 1.3)
Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.
For all you Pythonistas out there ...
I just released new version of #elasticsearch #python client (1.1.0) - https://t.co/p2WKV3jibc
— Honza Král (@HonzaKral) July 2, 2014
- What an amazing way to start the week! Take a look at Gergo Horanyi's post from Monday on how CERN uses the ELK stack to power the Large Hadron Collider.
- We released Elasticsearch 1.2.2 earlier today.
- Christophe Vandeplas kicked off an article series on using the ELK stack for incident handling and forensics. The first installment focuses on BlueCoat Proxy log search and analytics.
- In case you missed it last week, Jordan Sissel hosted a webinar Introducing the ELK Stack. You can check out the recording any time. (Registration required.)
- Florian Hopf shared his first post in a series on Elasticsearch use cases. First up: document store.
- Interested in writing Elasticsearch plugins? Check out Njal Karevoll's overview of the different plugin types and scopes made available in Elasticsearch.
- If you're running Elasticsearch on Azure, Itamar Syn-Hershko's definitive guide on this very subject will make your day even better. Check it out!
- And, if Itamar's work inspired you, head over to Kostas Pardalis' blog for even more Elasticsearch on Azure goodness. Kostas tells you all about how to build a highly available minimum viable product.
- From the for cool apps department: ElasticUI - a nifty little library to help you build AngluarJS apps with an Elasticsearch backend.
- We're bringing you more tips and tricks on making your life with Elasticsearch better. This week, we have Zachary Tong on regex bucket filters and Lee Hinman on sane Scripting and Security.
- Patrick Peschlow from our partner firm Codecentric shared two great blog posts recently, one on various Elasticsearch Management & Monitoring Plugins, and the other a fine Indexing Performance Cheatsheet. Great resources as you're scaling up your clusters!
- Valentin Zacharias authored a recent article for Jaxenter on What's New in Elasticsearch 1.2. (auf Deutsch)
@dadoonet Bonjour, powered by ES ;) http://t.co/xIdZbAcISF
— Jean-Yves Stervinou (@jy) July 3, 2014
The Magic Zapper team uses Elasticsearch to help find relevant TV content for you, all from your mobile phone "remote control."
Slides & Videos
Heading to our EU hometown for DrupalCon Amsterdam? Check out David Pilato on Elasticsearch and Drupal before you arrive!
From the oldies but goodies files: Igor Motov on Ongoing Resiliency Improvements in Elasticsearch
Love Scala and Elasticsearch? Hear from the folks at Sports195 about their custom Scala libraries that make Elasticsearch sing.
Learn all about how PSA Peugeot Citroën uses the ELK Stack (en français)
Where to find Us
We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Community Manager, Leslie Hawthorn, know!
"Search: A Journey of Delivery on a Budget" from @ThatPandaDev & @s_mcleod - July 10 @MelbourneSearch @elasticsearch http://t.co/oU22fhbH4e
— ⓜⓡ ⓜⓞⓝⓓⓞ (@s_mcleod) June 5, 2014
- New event: The lovely souls from the Elasticsearch Berlin User Group will be hosting an ELK Stack Hackathon on Monday, July 14th. If you're in town for Tech Open Air Berlin or call this fair city home, please join the community for a full day of ELK stack hacking goodness. Many thanks to Florian Gilcher for providing technical mentorship and to ThoughtWorks for hosting! (Register now on Meetup.com)
- If you're in or around Cologne on July 9th, please join Alexander Reelsen and Britta Weber for the first ever Elasticsearch Meetup in Cologne! Alex will cover What's New in Elasticsearch, and Britta will present on the Significant Terms Aggregation. Doors open at 6:30 PM, and registration is open.
- Our brand new employee in Germany, Dmitri Marx, will be attending Java Forum Stuttgart along with Livia Froelicher. Make sure to say hello to them between sessions! The conference takes place on July 17th at the Culture & Convention Centre Liederhalle.
- EuroPython comes to Berlin this year, with the conference running from July 21-27th at the Berlin Congress Center. Please stop by the Elasticsearch table to learn all about how the ELK stack can make life better for Pythonistas. Make sure to catch Honza Kral, Django and Elasticsearch core developer, on lessons learned from building the Python Elasticsearch client. Honza will take the stage at 2:00 PM on July 24th. Bonus: we're hosting a meetup during the conference on Thursday, July 28th from 6-8 PM. Mark your calendar now, and keep an eye out for registration details soon!
- Following on from EuroPython, Honza Kral will be speaking at the PyData Conference on Making Sense of your big data with Elasticsearch. Honza presents on July 27th at 2:10 PM, and the PyData Conference will also be held at the Berlin Congress Center.
The Second Elasticsearch Tel Aviv Meetup will convene on July 28th, focusing on real world use cases. The agenda is now finalized and you can hear from three different companies on how they use Elasticsearch. You can register now.
The very first Elasticsearch Italy Meetup has been scheduled for July 16th! Please join us to hear from Luca Cavanna on What's New in Elasticsearch. You'll also be treated to a case study talk on how Lutech uses Elasticsearch in its Threat Management System for Breach Detection, Intelligence & Response. Doors open at 6:00 PM, and registration is now open.
The Elasticsearch Study Session will convene on July 14th at 6:30 PM. Register now to get a chance to hear from our core developers Honza Kral and Igor Motov. Even cooler, our CTO Shay Banon will be attending this meetup, so register now to save your place!
The Auckland JVM Users Group will rebooting their meetup series, with their newest offering focused on Elasticsearch. You can join them on Tuesday, July 22nd at 6:00 PM, and registration is open.
Clinton Gormley will be speaking on Scaling Real-Time Search and Analytics with Elasticsearch at Mimacom Days Barcelona on July 10th. Clint takes the stage at 9:15 AM.
- If you're in or around Bath on July 14th, the South-West Elasticsearch Community will convene to talk about how you're using Elasticsearch. You can register for the inaugural meeting, which will kick off at 7 PM.
- The London VoiP User Group will get together on July 22nd at 6:30 PM. These folks are in search of a venue, so if you're excited to hear about Eye-candy from CDRs with the ELK stack, register now and help these good folks find a place to meetup.
Meeting up in Portlandia, OSCON style
Join us @OSCON for an #elasticsearch focused BOF session, July 23rd at 7pm at OSCON - details are here http://t.co/yh0yEqHowW
— stevemayzak (@smayzak) July 7, 2014
- The annual O'Reilly Open Source Conference is back again in Portland, Oregon! The Elasticsearch team will be out in force, with two talks, an ELK BoF and a lovely table in the exhibit hall where we can answer all of your questions. More details to follow, but we hope to see many of our friends there! OSCON runs from July 20-24th at the Oregon Convention Center.
- The TechLife Cincinnati Meetup Group will get together on July 16th at 6 PM to talk the ELK stack and other open source monitoring tools. The hosts are welcoming the Cincinnati Blacks in Technology group to present during this meetup.
Where to Find You
Our Community Manager, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, take a moment to let her know.
Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note. We care more about your skill set and passion for Elasticearch, Kibana, and Logstash than where you rest your head.
The price of @elasticsearch support is dirt cheap compared to the value that their stack represents so I'll be a happy (paying) customer ...
— Henrik Johansen (@HenrikJohansen) July 7, 2014
We think you're pretty nifty too, Henrik! If you're interested, here's where to learn all about our support offerings.
If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:
- Seoul - July 10, 2014 (core Elasticsearch training)
- Tokyo - July 14, 2014 (core Elasticsearch training)
- Berlin - July 15, 2014 (ELK workshop)
- Milan - July 16, 2014 (core Elasticsearch training)
- Tel Aviv - July 23, 2014 (core Elasticsearch training)
- Sao Paulo - August 14, 2014 (core Elasticsearch training)
- Rio de Janeiro - August 18, 2014 (core Elasticsearch training)
- San Francisco - August 28, 2014 (ELK workshop)