<p>Welcome to <em>This Week in Elastic</em>! In this weekly round up post, we bring you news on all aspects of the ELK Stack, including updates on Elasticsearch, Logstash and Kibana development, tips and tricks and upcoming events.</p><blockquote class="twitter-tweet" lang="en">Getting started w/Watcher -alerting for <a href="https://twitter.com/hashtag/Elasticsearch?src=hash">#Elasticsearch</a>? Join <a href="https://twitter.com/skearns64">@skearns64</a> on Jul 21 for live demos, real use cases+more <a href="http://t.co/8HUondQF08">http://t.co/8HUondQF08</a><br />— elastic (@elastic) <a href="https://twitter.com/elastic/status/621770650869346304">July 16, 2015</a></blockquote><h2>Elastic News</h2><p>Useful blog posts, how tos, tutorials&nbsp;and articles&nbsp;from the wide world of all the ELK stack ecosystem.</p><ul><li>We just released Elasticsearch 1.7.0 and 1.6.1. These releases contain a security fix and we advise all users to upgrade immediately. Check out the&nbsp;<a href="https://www.elastic.co/blog/elasticsearch-1-7-0-and-1-6-1-released">release blog post</a>&nbsp;for full details.</li><li>We're bringing you better query execution in Elasticsearch 2.0!&nbsp;Read on to hear more about&nbsp;<a href="https://www.elastic.co/blog/better-query-execution-coming-elasticsearch-2-0">how we've optimized querying</a> for this forthcoming release.</li><li>Check out our weekly deep dive into the latest developments in <a href="https://www.elastic.co/blog/this-week-in-elasticsearch-and-apache-lucene-2015-07-15">Elasticsearch and Apache Lucene</a>, plus a great walk through from Mike McCandless on how the&nbsp;Lucene&nbsp;team&nbsp;has helped <a href="https://www.elastic.co/blog/lucene-jvm-bugs">squash JVM bugs</a>.</li><li>Always wished that Logstash would receive data via HTTP requests? Wait no longer! Our new plugin, logstash-input-http, is now available and will ship as a default plugin in Logstash 1.5.2 and beyond. You can get the full details in the <a href="https://www.elastic.co/blog/introducing-logstash-input-http-plugin">release blog post</a>.&nbsp;</li><li>Shelby Switzer shares how Elasticsearch is helping Code for America solve the local data problem in <a href="https://www.elastic.co/blog/solving-the-local-data-problem-elasticsearch-docker-compose">her guest blog</a> post, which includes your daily dose of civic hacking love, Elasticsearch and Docker.&nbsp;</li><li>The forloop engineering team brings you detailed walk through on <a href="http://forloop.co.uk/blog/optimistic-concurrency-with-elasticsearch-and-nest#sthash.QIfBRnNs.uSQdd7Pt.dpbs">Optimistic Concurrency with Elasticsearch and NEST</a>, complete with code snippets and some lovely architectural diagrams.&nbsp;(And, if you enjoy this post, you may also like their how to&nbsp;on <a href="http://forloop.co.uk/blog/geospatial-search-with-elasticsearch-and-nest#sthash.qa1MVHW7.dpbs">Geospatial search with Elasticsearch and NEST</a>.)</li><li>Qing Zou's recent article in the Code4Lib journal offers a deep dive into using the ELK stack to <a href="http://journal.code4lib.org/articles/10589">monitor EZproxy logs in real-time</a> - another wonderful walk through full of code samples and architecture diagrams.</li><li>Fatih Cetinkaya treats you to a look at&nbsp;<a href="https://www.yieldbot.com/blog/servers-as-microservices-with-node-js-at-yieldbot/">Servers as Microservices at Yieldbot</a>, all glued together using the ELK stack for the logging and monitoring backend.</li><li>Enjoy our news round up? We have even more cool (and different!) how tos on offer in our weekly developer newsletter, <em>Elastic's Thing a Week</em>. Subscribers also receive the invitation to attend our <a href="https://www.elastic.co/community/devhangouts">Developer Hangouts</a> live. If you'd like to subscribe - and it's all we'll send you unless you ask for more - just fill out this <a href="https://www.elastic.co/community/newsletter">handy form</a>.</li></ul><blockquote class="twitter-tweet" lang="en">We just released beta2 of our NEST extension that adds support for <a href="https://twitter.com/hashtag/elasticsearch?src=hash">#elasticsearch</a> <a href="https://twitter.com/hashtag/watcher?src=hash">#watcher</a> GA! <a href="https://t.co/v8MWuwtw0m">https://t.co/v8MWuwtw0m</a><br />— Martijn Laarman (@Mpdreamz) <a href="https://twitter.com/Mpdreamz/status/621331578950656001">July 15, 2015</a></blockquote><blockquote class="twitter-tweet" lang="en">Support for MongoDB, multiple Elasticsearch nodes and other goodies in the new Packetbeat 1.0.0-beta2 <a href="https://twitter.com/hashtag/elasticbeats?src=hash">#elasticbeats</a> <a href="http://t.co/bdhcm5uk1J">http://t.co/bdhcm5uk1J</a><br />— Packetbeat (@packetbeat) <a href="https://twitter.com/packetbeat/status/620645782648172544">July 13, 2015</a></blockquote><h2>Slides and Videos</h2><p>If you're a PHP developer,&nbsp;<a href="http://twitter.com/shaunak">Shaunak Kashyap</a>&nbsp;just gave a presentation on&nbsp;<a href="http://ycombinator.github.io/presentations/2015-07-15-meetup-php-south-bay/index.html#/">how to make the best use of our Elasticsearch PHP client</a>. You can check out the accompanying demo application&nbsp;<a href="https://github.com/elastic/demo/tree/master/recipe_search">on GitHub</a>.</p><p><a href="http://twitter.com/dadoonet"><strong>David Pilato</strong></a><strong> helps you make sense of your (BIG) data at the recent Web2Day Nantes Conference</strong></p><div class="video embed-container" style="height: 319.725px;"><iframe src="//www.youtube.com/embed/QMEwvPv5New" class="video-iframe" style="width: 580px;height: 326.25px;"></iframe></div><p><a href="https://www.uslide.io/presentations/2GK0iplSTvjx6XGopRsleg/embed" target="_blank">View Full Presentation</a></p><h2>Where to find Us</h2><p>We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking about the ELK stack&nbsp;or hosting a meetup, <a href="mailto:stories@elastic.co">let us know!</a></p><h3>Australia</h3><ul><li>Our team will be in the exhibits hall at <a href="http://www.devopsdays.org/events/2015-melbourne/">DevOps Days Melbourne</a>, on July 16 &amp; 17. We're also proposing an open space on the ELK Stack, so perhaps we'll see you in the open space zone, too.</li><li><a href="http://twitter.com/honzakral">Honza Kral</a> is heading Down Under for <a href="http://2015.pycon-au.org/">PyCon Australia</a>, which is co-located this year with <a href="http://www.djangocon.com.au/">DjangoCon AU</a>. The conferences and sprints run July 31 - August 4 in Brisbane. See you there!</li><li>The Elasticsearch Sydney Meetup will get together on August 6th to discuss a comparison of Elasticsearch to SQL, plus more speaker to be announced. <a href="http://www.meetup.com/Sydney-Elastic-Fantastics/events/223544187/">Register now</a> to save your seat.</li></ul><h3>France</h3><p>The Elastic Paris Meetup will get together on July 29 to talk Fuzzy Name Matching in Elasticsearch. <a href="http://www.meetup.com/ElasticFR/events/223330719/">Register now</a> to save your seat.</p><h3>Germany</h3><ul><li>The Search Technology Meetup Hamburg will convene on July 22 to talk Elasticsearch: Speed is Key. <a href="http://www.meetup.com/Search-Technology-Meetup-Hamburg/events/223758134/">Register now</a> to save your seat.</li><li>The Search Meetup Munich will get together on July 28 to talk about GuteFrage's use case. <a href="http://www.meetup.com/Search-Meetup-Munich/events/223572356/">Register now</a> to save your seat.</li><li>The Logstash team will be out in force this year at the co-located <a href="http://2015.jrubyconf.eu/">JRubyConf</a>&nbsp;and <a href="http://2015.eurucamp.org/">Eurucamp</a> conferences, coming up in Berlin July 31 - August 2. See you in the hallway track!</li></ul><h3>Hong Kong</h3><p><a href="http://twitter.com/kimchy">Shay Banon</a> will be speaking at the <a href="https://riseconf.com/">RISE Conference</a>, coming up July 31 - August 2 in Hong Kong. Stay tuned for more details on his talk!</p><h3>India</h3><p>The Big Data Analytics on Cloud Meetup group of Hyderabad will get together on August 1 for a Hands on ELK Workshop. <a href="http://www.meetup.com/abctalks/events/223791624/">Register now</a> to attend.&nbsp;</p><h3>Israel</h3><p>For our second Tel Aviv meetup in July, you can hear about Elasticsearch and Fuzzy Matching, Fighting Crime with Elasticsearch and Unstructured Data Analytics and more. <a href="http://www.meetup.com/Tel-Aviv-Elastic-Fantastics/events/223040687/">Register now</a> to attend this meetup on July 22.</p><h3>Japan</h3><blockquote class="twitter-tweet" lang="en">タイトル決めました（中身はまだだけど）/ 【東京】【女性限定】Javaでデータ解析　基本のキ！ - Java女子部 | Doorkeeper - <a href="https://t.co/mXf6NYt9Ka">https://t.co/mXf6NYt9Ka</a><br />— Jun Ohtani (@johtani) <a href="https://twitter.com/johtani/status/616097338373738497">July 1, 2015</a></blockquote><blockquote class="twitter-tweet" lang="en">今回は、CTOでありElasticsearchの生みの親であるShayが来日してトークします。 / 第11回elasticsearch勉強会 <a href="https://twitter.com/hashtag/elasticsearch?src=hash">#elasticsearch</a> <a href="https://twitter.com/hashtag/elasticsearchjp?src=hash">#elasticsearchjp</a> - <a href="https://t.co/faUQGokmGq">https://t.co/faUQGokmGq</a><br />— Jun Ohtani (@johtani) <a href="https://twitter.com/johtani/status/621486294602919937">July 16, 2015</a></blockquote><h3>Korea</h3><blockquote class="twitter-tweet" lang="en">[Elastic Seoul Meetup - 엘라스틱 서울 밋업] 모임을 온오프믹스에 개설하였습니다. 많이 참석해주세요! <a href="http://t.co/fc3G7bDuGd">http://t.co/fc3G7bDuGd</a><br />— JongMin Kim 김종민 드미트리 (@kimjmin) <a href="https://twitter.com/kimjmin/status/621190692975411200">July 15, 2015</a></blockquote><h3>South Africa</h3><p>The Elasticsearch South Africa group will convene in Jo'Burg on July 29 for a Gentle Introduction to Elasticsearch. <a href="http://www.meetup.com/Elasticsearch-South-Africa/events/223731279/">Register now</a> to save your seat.</p><h3>Spain</h3><p>EuroPython 2015 heads to Bilbao this year, featuring two talk from <a href="http://twitter.com/honzakral">Honza Kral</a>: <em>Beyond the Basics with Elasticsearch</em> and <em>Don't be afraid to search</em>. You can learn more about Honza's talks in the <a href="https://ep2015.europython.eu/conference/p/honza-kral">conference program</a>; EuroPython runs July 20-26.</p><h3>United Kingdom</h3><ul><li>The London Elastic Meetup will get together on July 21 for two talks, including <a href="http://twitter.com/elasticmark">Mark Harwood</a> on using Kibana 4 to analyze data from the Ministry of Transportation. <a href="http://www.meetup.com/London-ElasticSearch-User-Group/events/222015564/">Register now</a> to save your seat - we're hosting at the Elastic London office!</li><li>The Bristol Java Meetup will convene on August 4 to talk Elasticsearch Internals, Spring&nbsp;Boot and Microservices. <a href="http://www.meetup.com/Bristol-Java-Meetup/events/221730916/">Register now</a> to save your place.</li></ul><h3>United States - East</h3><ul><li>The Elastic New York Meetup will get together on July 22 to hear all about how Pelias uses Elasticsearch. <a href="http://www.meetup.com/New-York-City-Elastic-Fantastics/events/223917206/">Register now</a> to save your seat.</li><li>The Tampa JUG will convene on July 23 to talk Centralized Logging with Logstash. <a href="http://www.meetup.com/Tampa-JUG/events/222242519/">Register now</a> to join the festivities.</li><li>Chris Earle will be speaking at GR8Conf US on <a>Using Groovy Extension Modules to make Elasticsearch's Java Client Groovy</a>. Stay tuned for further schedule details to hear from Chris, but you can join him July 29 - 31 in Minneapolis.</li><li>Shaunak Kashyap will give an Introduction to Elasticsearch at That Conference, coming up August 10-12 in the Wisconsin Dells. You can catch&nbsp;<a href="https://www.thatconference.com/Sessions/Session/6768">Shaunak's presentation</a> at 10:30 AM on August 10.</li><li>The RubyJAX Meetup of Jacksonville-Beach, Florida, will get together on August 13 to talk about Data Store Alternatives, including Elasticsearch. <a href="http://www.meetup.com/RubyJax/events/222505065/">Register now</a> to save your seat.</li></ul><h3>United States - West</h3><ul><li>The Elastic team will be out in force at <a href="http://www.oscon.com/open-source-2015/">OSCON 2015</a>. The conference runs from July 20-24 in Portland, Oregon. Join us in the exhibits hall for an awesome demo, getting all your questions answered and some nifty swag! We'll also have a few folks from our Developer Advocate team at the co-located <a href="http://www.communityleadershipsummit.com/">Community Leadership Summit</a> &amp; <a href="http://flosscommunitymetrics.org/">FLOSS Metrics Summit</a> the weekend before, so if you're heading to these events, we'll see you there!</li><li>The Seattle Elasticsearch Meetup will get together on July 21 to review Elasticsearch on Azure: Guidance, Tips and&nbsp;Best Practices. <a href="http://www.meetup.com/Seattle-ElasticSearch-Meetup/events/223801588/">Register now</a> to attend.</li><li>The Cloud Austin Meetup will welcome speakers from Elastic on July 21 to talk Elasticsearch and Docker. <a href="http://www.meetup.com/CloudAustin/events/222891441/">Register now</a> to save your seat.</li><li>The Dallas Elasticsearch User Group will convene on July 23 for an Introduction to Elasticsearch plus live demo of analyzing Twitter data. <a href="http://www.meetup.com/Elasticsearch-User-Group-Dallas/events/222051704/">Register now</a> to save your seat.</li><li>The Advanced Apache Spark Meetup of San Francisco will convene on July 28 to talk Realtime Advanced Analytics, including Elasticsearch. <a href="http://www.meetup.com/Advanced-Apache-Spark-Meetup/events/223763502/">Register now</a> to save your place.</li><li>The Elastic San Diego Meetup will get together on July 29 for an Introduction to the ELK stack. <a href="http://www.meetup.com/San-Diego-Elastic-Fantastics/events/223607998/">Register now</a> to save your seat.</li><li>Heading to Blackhat in Las Vegas August 1-6? Be sure to catch Quentin Long's&nbsp;<a href="https://www.blackhat.com/us-15/arsenal.html#elastalert">talk on ElastAlert</a>.</li></ul><h3>Uruguay</h3><p>The Montevideo Big Data and Data Science Meetup will get together on August 6 for talks on Elasticsearch. <a href="http://www.meetup.com/Montevideo-BigData-DataScience-Meetup/events/223833509/">Register now</a> to save your seat.<br /></p><h2>Where to Find You</h2><p><strong>PSST!</strong>&nbsp;If you're a regular reader, you're thinking of skipping this section. You may even be thinking to yourself, yes of course I will drop a note on Twitter when I am giving a talk on all things ELK. That's awesome, because we'd like to showcase every meetup, conference presentation and workshop on Elasticsearch, Logstash, and Kibana happening worldwide. And now, we've made it even easier for you to get support for your meetup!</p><p>Head on over to&nbsp; <a href="https://www.elastic.co/community/meetups">our meetups page</a>! (And we'll still totally send you swag if you're giving a talk on anything ELKy at a conference.)</p><p>Oh yeah, we're also&nbsp; <a href="https://www.elastic.co/about/careers">hiring</a>. If you'd like us to find you for employment purposes, just&nbsp;<a href="mailto:jobs@elastic.co">drop us a note</a>. We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.</p><h2>Trainings</h2><p>If you are interested in&nbsp;training on Elastic's product line,&nbsp;we have courses coming up in:</p><ul><li><a href="http://purchases.elastic.co/class/Elastic/Workshop/SpringPeople/2015-07-27">Bangalore</a> - July 27, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Elastic/Core/SpringPeople/2015-07-28">Bangalore</a> - July 28, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/London/Core/July2015">London</a> - July 29, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/How/NY/2015-08-3">New York</a> - August 3, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Dev/NY/2015-08-11">New York</a> - August 4, 2015 (Core Elasticsearch: Developer)</li><li><a href="http://purchases.elastic.co/class/Ops/NY/2015-08-06">New York</a> - August 6, 2015 (Core Elasticsearch: Operations)</li><li><a href="http://purchases.elastic.co/class/CoreES/Montreal/2015-08-11">Montreal</a> - August 11, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Dev/Seattle/2015-08-11">Kirkland</a> - August 11, 2015 (Core Elasticsearch: Developer)</li><li><a href="http://purchases.elastic.co/class/Ops/Seattle/2015-08-13">Kirkland</a> - August 13, 2015 (Core Elasticsearch: Operations)</li><li><a href="http://purchases.elastic.co/class/How/SF/2015-08-17">San Francisco</a> - August 17, 2015 (Hands-On Workhop)</li><li><a href="http://purchases.elastic.co/class/Dev/SF/2015-08-18">San Francisco</a> - August 18, 2015 (Core Elasticsearch: Developer)</li><li><a href="http://purchases.elastic.co/class/Ops/SF/2015-08-20">San Francisco</a> - August 20, 2015 (Core Elasticsearch: Operations)</li><li><a href="http://purchases.elastic.co/class/How/Amsterdam/2015-08-20">Amsterdam</a> - August 20. 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Kibana/Amsterdam/2015-08-21">Amsterdam</a> - August 21, 2015 (Kibana 4 Workshop)</li><li><a href="http://purchases.elastic.co/class/Dev/Amsterdam/2015-08-20">Amsterdam</a> - August 24, 2015 (Core Elasticsearch: Developer)</li><li><a href="http://purchases.elastic.co/class/Elastic/Workshop/SpringPeople/2015-08-24">Bangalore</a> - August 24, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/How/Austin/2015-08-25">Austin</a> - August 25, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Elastic/Core/SpringPeople/2015-08-25">Bangalore</a> - August 25, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Dev/Austin/2015-08-26">Austin</a> - August 26, 2015 (Core Elasticsearch: Developer)</li><li><a href="http://purchases.elastic.co/class/Ops/Amsterdam/2015-08-26">Amsterdam</a> - August 26, 2015 (Core Elasticsearch: Operations</li><li><a href="http://purchases.elastic.co/class/How/Cologne/2015-08-26">Cologne</a> - August 26, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Dev/Cologne/2015-08-27">Cologne</a> - August 27. 2015 (Core Elasticsearch: Developer)</li></ul>

This Week in Elastic - Better query execution in Elasticsearch 2.0

<p>Welcome to <em>This Week in Elastic</em>! In this weekly round up post, we bring you news on all aspects of the ELK Stack, including updates on Elasticsearch, Logstash and Kibana development, tips and tricks and upcoming events.</p><blockquote class="twitter-tweet" lang="en"><a href="https://twitter.com/hashtag/Found?src=hash">#Found</a> - hosted <a href="https://twitter.com/hashtag/elasticsearch?src=hash">#elasticsearch</a> - is here. Join us on July 15 for a live webinar, demo, &amp; Q&amp;A. <a href="http://t.co/enEkIwFF8p">http://t.co/enEkIwFF8p</a> <a href="http://t.co/3wYW2LhYPO">pic.twitter.com/3wYW2LhYPO</a><br />— elastic (@elastic) <a href="https://twitter.com/elastic/status/618831938850197504">July 8, 2015</a></blockquote><h2>Elastic News</h2><p>Useful blog posts, how tos, tutorials&nbsp;and articles&nbsp;from the wide world of all the ELK stack ecosystem.</p><ul><li>Check out our weekly deep dive into the latest developments in <a href="https://www.elastic.co/blog/this-week-in-elasticsearch-and-apache-lucene-2015-07-07">Elasticsearch and Apache Lucene</a>.&nbsp;</li><li>We've also got the <a href="https://www.elastic.co/blog/logstash-lines-2015-07-02">The Logstash Lines</a>, our weekly Logstash update. This edition includes some great blog posts from the Logstash community on using Logstash Forwarder with Docker.</li><li>Curator 3.2.0 was released last week, including support for synced flushes and more! Get the full details in the <a href="https://www.elastic.co/blog/curator-3-2-0-released">release blog post</a>.</li><li><a href="https://twitter.com/jpountz">Adrien Grand</a>&nbsp;shares a deep dive into store compression in Elasticsearch and Lucene, which includes discussion of several improvements to Lucene 5 that will benefit users of Elasticsearch 2.0. <a href="https://www.elastic.co/blog/store-compression-in-lucene-and-elasticsearch">Check it out</a>!</li><li><a href="https://twitter.com/colings86es">Colin Goodheart-Smithe's</a>&nbsp;latest post on aggregations offers an overview of all the pipeline aggs forthcoming in Elasticsearch 2.0. Learn all about these new types of aggregations in <a href="https://www.elastic.co/blog/out-of-this-world-aggregations">Colin's walk through</a>.</li><li>Automattic, the folks who bring you WordPress, bid farewell to MySQL and have replaced it with Elasticsearch. Learn how the switch yielded a 200x improvement for some of their customers in <a href="https://vip.wordpress.com/2015/07/08/scaling-wp-queries-with-elasticsearch/">this blog post</a>, which also includes a presentation video and accompanying slide deck.</li><li>James Thomas <a href="http://jamesthom.as/blog/2015/07/08/making-logs-awesome-with-elasticsearch-and-docker/">offers this lovely tutorial</a> on making your logging process more awesome using Elasticsearch and Docker in the cloud, with a little bonus material on managing&nbsp;IBM containers - complete with a ton of code snippets. Enjoy!</li><li>The Hatena engineering team shares an <a href="http://blog.shibayu36.org/entry/2015/07/08/095838">Introduction to Elasticsearch</a>. (日本語で)</li><li>Enjoy our news round up? We have even more cool (and different!) how tos on offer in our weekly developer newsletter, <em>Elastic's Thing a Week</em>. Subscribers also receive the invitation to attend our <a href="https://www.elastic.co/community/devhangouts">Developer Hangouts</a> live. If you'd like to subscribe - and it's all we'll send you unless you ask for more - just fill out this <a href="https://www.elastic.co/community/newsletter">handy form</a>.</li></ul><blockquote class="twitter-tweet" lang="en">Gearing up 4 the <a href="https://twitter.com/elastic">@elastic</a> Tel Aviv meetup.View from the Gigya offices are amazing.Thx for hosting us. <a href="https://twitter.com/hashtag/elasticsearch?src=hash">#elasticsearch</a> <a href="http://t.co/LC5UOb2EfT">pic.twitter.com/LC5UOb2EfT</a><br />— Boaz Leskes (@bleskes) <a href="https://twitter.com/bleskes/status/618810053357744128">July 8, 2015</a></blockquote><h2>Slides and Videos</h2><p><a href="http://twitter.com/bleskes"><strong>Boaz Leskes</strong></a><strong> on Resiliency in Elasticsearch</strong></p><iframe width="560" height="315" src="https://www.youtube.com/embed/vQFk1OZg8TU" frameborder="0" allowfullscreen=""></iframe><p><br /></p><p>And you can enjoy <a href="http://twitter.com/dadoonet">David Pilato</a> on <em>Donnez du sens à vos données Elasticsearch</em> from last week's JDEV Conference <a href="https://webcast.in2p3.fr/videos-jdev2015_donnez_du_sens_a_vos_donnees_elasticsearch">here</a>.</p><h2>Where to find Us</h2><p>We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking about the ELK stack&nbsp;or hosting a meetup, <a href="mailto:stories@elastic.co">let us know!</a></p><h3>Australia</h3><ul><li>Our team will be in the exhibits hall at <a href="http://www.devopsdays.org/events/2015-melbourne/">DevOps Days Melbourne</a>, on July 16 &amp; 17. We're also proposing an open space on the ELK Stack, so perhaps we'll see you in the open space zone, too.</li><li><a href="http://twitter.com/honzakral">Honza Kral</a> is heading Down Under for <a href="http://2015.pycon-au.org/">PyCon Australia</a>, which is co-located this year with <a href="http://www.djangocon.com.au/">DjangoCon AU</a>. The conferences and sprints run July 31 - August 4 in Brisbane. See you there!</li></ul><h3>France</h3><p>The Elastic Paris Meetup will get together on July 29 to talk Fuzzy Name Matching in Elasticsearch. <a href="http://www.meetup.com/ElasticFR/events/223330719/">Register now</a> to save your seat.</p><h3>Germany</h3><ul><li>Join our team in the exhibits hall at <a href="http://www.java-forum-stuttgart.de/de/Home.html">Java Forum Stuttgart</a> on July 9. We'll be there to answer all your questions and give you some lovely Elastic goodies.</li><li>The Search Technology Meetup Hamburg will convene on July 22 to talk Elasticsearch: Speed is Key. <a href="http://www.meetup.com/Search-Technology-Meetup-Hamburg/events/223758134/">Register now</a> to save your seat.</li><li>The Search Meetup Munich will get together on July 28 to talk about GuteFrage's use case. <a href="http://www.meetup.com/Search-Meetup-Munich/events/223572356/">Register now</a> to save your seat.</li><li>The Logstash team will be out in force this year at the co-located <a href="http://2015.jrubyconf.eu/">JRubyConf</a>e and <a href="http://2015.eurucamp.org/">Eurucamp</a> conferences, coming up in Berlin July 31 - August 2. See you in the hallway track!</li></ul><h3>Hong Kong</h3><p><a href="http://twitter.com/kimchy">Shay Banon</a> will be speaking at the <a href="https://riseconf.com/">RISE Conference</a>, coming up July 31 - August 2 in Hong Kong. Stay tuned for more details on his talk!</p><h3>Israel</h3><p>For our second Tel Aviv meetup in July, you can hear about Elasticsearch and Fuzzy Matching, Fighting Crime with Elasticsearch and Unstructured Data Analytics and more. <a href="http://www.meetup.com/Tel-Aviv-Elastic-Fantastics/events/223040687/">Register now</a> to attend this meetup on July 22.</p><h3>Japan</h3><blockquote class="twitter-tweet" lang="en">大阪でもやります！スピーカー募集中です！/ Elasticsearch勉強会 in 大阪 - elasticsearch勉強会 | Doorkeeper - <a href="https://t.co/48ZpSJ00jQ">https://t.co/48ZpSJ00jQ</a><br />— Jun Ohtani (@johtani) <a href="https://twitter.com/johtani/status/613909701072257024">June 25, 2015</a></blockquote><blockquote class="twitter-tweet" lang="en">出張Elasticsearch勉強会第2弾！ / Elasticsearch勉強会 in 京都 - elasticsearch勉強会 | Doorkeeper - <a href="https://t.co/pMuNSFHAow">https://t.co/pMuNSFHAow</a><br />— Jun Ohtani (@johtani) <a href="https://twitter.com/johtani/status/613905700238987264">June 25, 2015</a></blockquote><h3>South Africa</h3><p>We are a proud swag sponsor of the <a href="http://www.jsinsa.com/jsinsa/2015/">JavaScript in South Africa Conference</a>. If you happen to be in Johannesburg on July 11, pop in for knowledge and some cool Elastic goodies.</p><h3>Spain</h3><p>EuroPython 2015 heads to Bilbao this year, featuring two talk from <a href="http://twitter.com/honzakral">Honza Kral</a>: <em>Beyond the Basics with Elasticsearch</em> and <em>Don't be afraid to search</em>. You can learn more about Honza's talks in the <a href="https://ep2015.europython.eu/conference/p/honza-kral">conference program</a>; EuroPython runs July 20-26.</p><h3>Switzerland</h3><p>The Elasticsearch Switzerland Meetup group will get together on July 14 to talk Elasticsearch Under the Hood with <a href="http://twitter.com/spinscale">Alex Reelsen</a>. <a href="http://www.meetup.com/elasticsearch-switzerland/events/223462482/">Register now</a> to save your seat.</p><h3>United Kingdom</h3><p>The London Elastic Meetup will get together on July 21 for two talks, including <a href="http://twitter.com/elasticmark">Mark Harwood</a> on using Kibana 4 to analyze data from the Ministry of Transportation. <a href="http://www.meetup.com/London-ElasticSearch-User-Group/events/222015564/">Register now</a> to save your seat - we're hosting at the Elastic London office!</p><h3>United States - East</h3><ul><li>The RubyJax Meetup in Jacksonville, Florida, will get together on July 9 to talk Elasticsearch. <a href="http://www.meetup.com/RubyJax/events/222505065/">Register now</a> to enjoy this subject matter and a few other talks.</li><li>Heading to the <a href="http://www.codestock.org/">Codestock 2015</a> Conference? Join our Developer Advocate, <a href="http://twitter.com/shaunak">Shaunak Kashyap</a> in the hallway track. Codestock runs July 10-11 in Knoxville.</li><li>The Tampa JUG will convene on July 23 to talk Centralized Logging with Logstash. <a href="http://www.meetup.com/Tampa-JUG/events/222242519/">Register now</a> to join the festivities.</li><li>Chris Earle will be speaking at GR8Conf US on <a>Using Groovy Extension Modules to make Elasticsearch's Java Client Groovy</a>. Stay tuned for further schedule details to hear from Chris, but you can join him July 29 - 31 in Minneapolis.</li></ul><h3>United States - West</h3><ul><li><strong>On tonight</strong>:The Ansible Los Angeles User Group will convene on July 8 to talk about GumGum's use of Ansible to manage EC2 instances running Elasticsearch and many other technologies. <a href="http://www.meetup.com/Ansible-Los-Angeles/events/223335826/">Register now</a> to save your seat.</li><li><strong>On tonight</strong>: The Elastic Silicon Valley User Group will convene on July 8 to hear about Palantir's use case. <a href="http://www.meetup.com/Silicon-Valley-Elastic-Fantastics/events/223582024/">Register now</a> to save your seat while we're finalizing the agenda.</li><li>The Denver Elastic User Group will get together on July 14 for a casual meetup over drinks and snacks. <a href="http://www.meetup.com/Denver-Elastic-Fantastics/events/223628309/">Register now</a> to attend and talk shop.</li><li>The Salt Lake City Elastic Meetup will get together on July for a casual meetup over drinks and snacks. <a href="http://www.meetup.com/Salt-Lake-City-Elastic-Fantastics/events/223502179/">RSVP</a> to let us know you're coming!</li><li>The Seattle University Web Developer Meetup will convene on July 14 to talk KnockoutJS and Elasticsearch. <a href="http://www.meetup.com/Seattle-University-Web-Developers-Meetup/events/220459694/">Register now</a> to save your place.</li><li>The OpenLate @ OpenDNS Meetup will get together in San Francisco on July 14 to talk Elasticsearch for SQL Users. Our awesome Developer Advocate, <a href="http://twitter.com/shaunak">Shaunak Kashyap</a>, will be the invited speaker. <a href="http://www.meetup.com/OpenLate/events/222654757/">Register now</a> to save your seat.</li><li>The Salt Lake City Elastic Meetup will get together for a casual meetup over pizza on July 14. <a href="http://www.meetup.com/Salt-Lake-City-Elastic-Fantastics/events/223502179/">Register now</a> to save your seat.</li><li>The South Bay PHP User Group will join the Elastic Silicon Valley Meetup on July 15. Talk details are still in the works, but <a href="http://www.meetup.com/Silicon-Valley-Elastic-Fantastics/events/223355147/">register now</a> to save your seat.</li><li>The Elastic team will be out in force at <a href="http://www.oscon.com/open-source-2015/">OSCON 2015</a>. The conference runs from July 20-24 in Portland, Oregon. Join us in the exhibits hall for an awesome demo, getting all your questions answered and some nifty swag! We'll also have a few folks from our Developer Advocate team at the co-located <a href="http://www.communityleadershipsummit.com/">Community Leadership Summit</a> &amp; <a href="http://flosscommunitymetrics.org/">FLOSS Metrics Summit</a> the weekend before, so if you're heading to these events, we'll see you there!</li><li>The Dallas Elasticsearch User Group will convene on July 23 for an Introduction to Elasticsearch plus live demo of analyzing Twitter data. <a href="http://www.meetup.com/Elasticsearch-User-Group-Dallas/events/222051704/">Register now</a> to save your seat.</li><li>The Elastic San Diego Meetup will get together on July 29 for an Introduction to the ELK stack. <a href="http://www.meetup.com/San-Diego-Elastic-Fantastics/events/223607998/">Register now</a> to save your seat.</li></ul><h2>Where to Find You</h2><p><strong>PSST!</strong>&nbsp;If you're a regular reader, you're thinking of skipping this section. You may even be thinking to yourself, yes of course I will drop a note on Twitter when I am giving a talk on all things ELK. That's awesome, because we'd like to showcase every meetup, conference presentation and workshop on Elasticsearch, Logstash, and Kibana happening worldwide. And now, we've made it even easier for you to get support for your meetup!</p><p>Head on over to&nbsp; <a href="https://www.elastic.co/community/meetups">our meetups page</a>! (And we'll still totally send you swag if you're giving a talk on anything ELKy at a conference.)</p><p>Oh yeah, we're also&nbsp; <a href="https://www.elastic.co/about/careers">hiring</a>. If you'd like us to find you for employment purposes, just&nbsp;<a href="mailto:jobs@elastic.co">drop us a note</a>. We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.</p><h2>Trainings</h2><p>If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:</p><ul><li><a href="http://purchases.elastic.co/class/Zurich/Core/July2015">Zurich</a> - July 14, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Dallas/Core/July2015">Dallas</a> - July 14, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Stockholm/July2015">Stockholm</a> - July 14, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Dallas/Core/July2015">Dallas</a> - July 14, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/LosAngeles/July2015">Los Angeles</a> - July 16, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/MtView/How/July2015">Mountain View</a> - July 20, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Portland/July2015">Portland, OR</a> - July 23, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Elastic/Workshop/SpringPeople/2015-07-27">Bangalore</a> - July 27, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/London/How/July2015">London</a> - July 28, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Elastic/Core/SpringPeople/2015-07-28">Bangalore</a> - July 28, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/London/Core/July2015">London</a> - July 29, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/How/NY/2015-08-3">New York</a> - August 3, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Dev/NY/2015-08-11">New York</a> - August 4, 2015 (Core Elasticsearch: Developer)</li><li><a href="http://purchases.elastic.co/class/Ops/NY/2015-08-06">New York</a> - August 6, 2015 (Core Elasticsearch: Operations)</li><li><a href="http://purchases.elastic.co/class/CoreES/Montreal/2015-08-11">Montreal</a> - August 11, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Dev/Seattle/2015-08-11">Seattle</a> - August 13, 2015 (Core Elasticsearch: Developer)</li><li><a href="http://purchases.elastic.co/class/Ops/Seattle/2015-08-13">Seattle</a> - August 13, 2015 (Core Elasticsearch: Operations)</li><li><a href="http://purchases.elastic.co/class/How/SF/2015-08-17">San Francisco</a> - August 17, 2015 (Hands-On Workhop)</li><li><a href="http://purchases.elastic.co/class/Dev/SF/2015-08-18">San Francisco</a> - August 18, 2015 (Core Elasticsearch: Developer)</li><li><a href="http://purchases.elastic.co/class/Ops/SF/2015-08-20">San Francisco</a> - August 20, 2015 (Core Elasticsearch: Operations)</li><li><a href="http://purchases.elastic.co/class/How/Amsterdam/2015-08-20">Amsterdam</a> - August 20. 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Kibana/Amsterdam/2015-08-21">Amsterdam</a> - August 21, 2015 (Kibana 4 Workshop)</li><li><a href="http://purchases.elastic.co/class/Dev/Amsterdam/2015-08-20">Amsterdam</a> - August 24, 2015 (Core Elasticsearch: Developer)</li><li><a href="http://purchases.elastic.co/class/Elastic/Workshop/SpringPeople/2015-08-24">Bangalore</a> - August 24, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/How/Austin/2015-08-25">Austin</a> - August 25, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Elastic/Core/SpringPeople/2015-08-25">Bangalore</a> - August 25, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Dev/Austin/2015-08-26">Austin</a> - August 26, 2015 (Core Elasticsearch: Developer)</li><li><a href="http://purchases.elastic.co/class/Ops/Amsterdam/2015-08-26">Amsterdam</a> - August 26, 2015 (Core Elasticsearch: Operations</li><li><a href="http://purchases.elastic.co/class/How/Cologne/2015-08-26">Cologne</a> - August 26, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Dev/Cologne/2015-08-27">Cologne</a> - August 27. 2015 (Core Elasticsearch: Developer)</li></ul>

This Week in Elastic - July 08, 2015

This Week in Elastic - July 01, 2015

This Week in Elastic - June 24, 2015

Learn how analytics company Opentracker.net helps thousands of clients and how their experiences with Elasticsearch led to the creation of LeadBoxer.

<html><head></head><body><p><em>Cralan Deutsch and Wart Fransen are co-founders of Netherlands-based LeadBoxer and Opentracker. LeadBoxer is an actionable tool for B2B&#xA0;sales agents which delivers qualified leads via real-time signals, based on proprietary customisable lead scoring technology. Opentracker specialises in web tracking, data analytics and statistics innovation, while its&#xA0;hallmarks are&#xA0;simple, intuitive, and easy-to-read reporting interfaces, combined with an enterprise-class API.</em></p><hr><p><a href="http://www.opentracker.net/">Opentracker.net</a> is an analytics company focusing on website and mobile app user behavior. Today, our 1000s of clients from webmasters to corporate, financial and governmental institutions have become addicted to seeing exactly who, and how, people are using their online services in real-time, letting them track users, improve client experience, and increase conversion rates. Read on to learn how Opentracker.net and our experiences with Elasticsearch led us to create <a href="http://leadboxer.com">LeadBoxer</a>, a sophisticated&#xA0;web and mobile based lead capture and qualification system.</p><p><img src="https://api.contentstack.io/v2/assets/575e4c7658208ba076e27c75/download?uid=blt45a0865696d69071?uid=blt45a0865696d69071" data-sys-asset-uid="blt2ddf3a970ddaca85" style="width: 437px; height: 761.398773006135px;"></p><p>In our early stages, like a lot of other companies founded during this period,&#xA0;Opentracker&#x2019;s technology was built around MySQL. While providing a great starting point, MySQL suffered from inherent bottlenecks relative to scaling and structuring big data. As NoSQL solutions began appearing, we explored them and eventually settled on Cassandra, which resolved many of our data maintenance, redundancy, storage, and scaling issues. The one thing it did not provide us with, however, was real-time search.
</p><p>Real-time searches are important because every client has specific insights they want to extract from their data, and they need the flexibility to find exactly what they need to improve their business. Some clients are interested in users from certain locations, so that they can focus their on-boarding efforts by sales region, while others are interested in finding users on conversion pages that pre-qualify the visitors with interest in a specific product or service. Others are looking for users that match a specific lead qualification criteria in order to increase the chance of making sales, allowing sales agents to target potentials more effectively, and make warmer calls.
</p><p>In order to provide this functionality, we experimented with Apache Solr and Elasticsearch solely for the purpose of running real-time searches on both web and mobile app user data. Elasticsearch worked smoothly and presented an easy learning curve, while its schemaless structure allowed our clients to search through high volumes of unstructured data with flexibility, stability, and good response times.
</p><p>After implementing Elasticsearch, we started getting the same request from our B2B users: &quot;Show me the companies who are visiting, and help me get in touch with them.&quot; An opportunity presented itself: to develop a new service that could provide intelligence in the process of generating and capturing leads. We branched the technology that targeted the identification of companies, and <em>LeadBoxer was born</em>.
</p><p><img src="https://api.contentstack.io/v2/assets/575e4c759e7a83165490bd49/download?uid=bltf8f0ac5dc2ce8c3b?uid=bltf8f0ac5dc2ce8c3b" data-sys-asset-uid="blt4cc55e28d3bb9329">
</p><p>LeadBoxer&#x2019;s value proposition? Identify sales leads from online activity, then qualify the leads based on metrics the client finds important. (You can check out the service at <a href="https://www.leadboxer.com/">leadboxer.com</a> by setting up a trial account.)
</p><p>After implementing Elasticsearch in Opentracker, we realised we could use it to power LeadBoxer&#x2019;s basic functionality by manipulating the scoring and ordering algorithms. Our guess was that we could utilize the <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html">function_score</a> and <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-boosting-query.html">boosting queries</a> to help sort and rank leads by importance.
</p><p>At the time, documentation on the subject was limited&#xA0;and distributed through many different community channels, so we started to develop our own API to query Elasticsearch in order to create a lead scoring system that could be customized and generated by our customers.
</p><p>We were taking a risk. We thought that&#xA0;search engines&#xA0;were designed for searching through docs, not qualifying sales leads! The big question was would it work?
</p><p>The short answer, is <strong>yes</strong>.
</p><p>We now use Elasticsearch to deliver useful, qualified, and beautifully-designed lead results to sales and marketing teams. Our customers can adjust sliding weights to influence a lead score preference, and the preferences are converted into percentile values. The values are stored and converted to a formula in the API call, which in turn translates the formula to an Elasticsearch query. The Elasticsearch query returns the leads, ranked and prioritised based on the formula. Changing the weighted preferences gives different leads, allowing sales teams to focus on leads that they want to target.
</p><p><img src="https://api.contentstack.io/v2/assets/575e4c759e7a83165490bd40/download?uid=bltb67be06c94c1eaee?uid=bltb67be06c94c1eaee" data-sys-asset-uid="blt18f487c141189fc8" style="width: 631px; height: 780.197289156627px;">
</p><p><br>
</p><p>So, how do the lead score settings in the LeadBoxer UI translate to values? By using the AngularJS framework, we built a settings page where our users can set their scoring preferences with sliders. The values that are set by these sliders can range from 0 to 20.
</p><p><br>
</p><p><img src="https://api.contentstack.io/v2/assets/575e4c7512798d8e381fd269/download?uid=blt968d38f8f8ba04b1?uid=blt968d38f8f8ba04b1" data-sys-asset-uid="blt8ea284c2ec6464cd" style="width: 868px; height: 298.865037194474px;"><br>
</p><p>We decided that our lead score value should be a number between 1-100, 100 being the perfect lead. Because we keep on adding sliders and ways to influence the lead score, we need to make these absolute numbers percentiles. We achieve this by use of a formula:
</p><p><em>Formula value = ( value in the slider x maximum_total_ie_100 ) / total_of_all_slider_values ) * 100</em>
</p><p>We use boosting/scoring in different ways to calculate match, criteria and range.&#xA0;We defined three different property types that could be used to influence the lead score:
</p><ul>
	<li><strong>contains</strong>: when the enriched visitor (lead) contains any value for field X, add 10 points to the lead score</li>
	<li><strong>exact match</strong>: when the lead contains a value X for field Y, add 20 points to the leadscore</li>
	<li><strong>range</strong>: when the number of pageviews of a lead is between 5-10, add 5 points to the lead score</li>
</ul><p>These values are then translated into a formula by storing the UI-generated percentiles as URL parameters, e.g. <code>?criteriaMatch=industry|Insurance|10</code> which designates 10 leadscore percentile points to any lead identified via company profile as belonging to the Insurance sector.
</p><p>These API calls are then translated to an Elasticsearch query by processing them via a Java execution code path. For the Elasticsearch client, we are using a transport client. (And you can read more about the <a href="http://www.elastic.co/guide/en/elasticsearch/client/java-api/current/query-dsl-queries.html">Elasticsearch Java DSL</a> here.)
</p><p>We store and index our data by collecting and sending user, session, and event data to our cluster of &#x2018;log&#x2019; servers. In turn, the data is processed and enriched before it is stored to both the index of the Elasticsearch cluster and a Cassandra node.
</p><p>We learned a lot when using all this technology. The information is documented, but we couldn&#x2019;t find any case studies or examples, leading us at the time to conclude that this powerful technology is underutilised. 20+ million downloads later, though, means lots of people have figured out how useful Elasticsearch can be for their businesses!
</p><p>The best part about Elasticsearch is that it&apos;s a search engine that lets you manipulate the ranking/returns, meaning that you can be your own search engine master, and determine the outcomes with which the results are ranked and therefore presented to the user. Bottom line: using Elasticsearch in this way, you can let the user customise the results, meaning the results from any system-wide search, i.e. the order of results, will be based on custom criteria.
</p><p>For those of you who are fans of the nitty gritty details of our installation, our cluster consists of 4 master/data nodes with a monitoring node. The cluster is using version 0.90.7 and JVM 1.6 update 25. We&apos;re planning an upgrade to Elasticsearch 1.6, so stay tuned for more updates!</p><p>		So, what&#x2019;s next? We anticipate that with our current setup and development trajectory, utilising Elasticsearch, we are positioned to evolve the way B2B, or even B2C, sales processes are shaped. Similar to an IoT future where only relevant ads and personal offers are displayed, we can present our clients with ready-to-be-picked fruit for sales deals, whereby customers have directly or indirectly indicated they want or need a product or service. Combining all external online (big) data with internal client information and user-behaviour is the key to a perfect sales deal.
</p></body></html>

Elasticsearch: Powering Real-Time Mobile and Web Analytics for LeadBoxer and Opentracker

This Week in Elastic: Elasticsearch for Apache Hadoop 2.1 RC1 released

This Week in Elastic: First dev hangout &  Logstash 1.4.3 released

This Week in Elastic: New discussion forums are live!

<p>Welcome to <em>This Week in Elastic</em>! In this weekly round up post, we bring you news on all aspects of the ELK Stack, including updates on Elasticsearch, Logstash and Kibana development, tips and tricks and upcoming events.</p><h2>Elastic News</h2><blockquote class="twitter-tweet" lang="en">Community + one a-mazing data scientist concur: you want to hear each others real world stories. We're here to help! <a href="https://t.co/Hyh23wp17a">https://t.co/Hyh23wp17a</a><br />— Leslie Hawthorn (@lhawthorn) <a href="https://twitter.com/lhawthorn/status/603277176394375169">May 26, 2015</a></blockquote><p>Useful blog posts, how tos, tutorials&nbsp;and articles&nbsp;from the wide world of all the ELK stack ecosystem.</p><ul><li>We just welcomed Packetbeat into the Elastic family!<a href="https://www.elastic.co/blog/welcome-packetbeat-tudor-monica">Read on</a> to learn more about the future of all things Beats. Like the blog post? <a href="http://demo.elastic.co/packetbeat/#/dashboard/Packetbeat-Dashboard">Try the demo</a>!</li><li>Check out our weekly deep dive into the latest developments in <a href="https://www.elastic.co/blog/this-week-in-elasticsearch-and-apache-lucene-2015-05-26">Elasticsearch and Apache Lucene</a>. There's some cool news in this post about solving a 10+ year old bug in Lucene!&nbsp;</li><li>We've also got the <a href="https://www.elastic.co/blog/logstash-lines-2015-05-22">The Logstash Lines</a>, our weekly Logstash update. Find out what's coming up in 1.5.1 and 1.6!</li><li>For all the latest developments in Kibana, check out our<em> Kurrently in</em> series. This <a href="https://www.elastic.co/blog/kurrently-kibana-2015-05-22">past edition</a> contains kittens. There are no photos of the kittens included - epic fail! - but there's news&nbsp;about&nbsp;a ton of fixes and new features in progress, so we're still calling that a win.&nbsp;</li><li>The Leap Second coming up on June 30th already ruining your day? We've got you covered with <a href="https://www.elastic.co/blog/elasticsearch-2015-leap-second">our post</a> on how to ensure your Elasticsearch deployments stay happy and healthy, courtesy of <a href="https://twitter.com/nellicus">Antonio Bonuccelli</a>.</li><li>Smart meters are cool, but advising an entire utility company about how to help their customers save energy is even cooler. Especially when it's powered by Elasticsearch. Read on to learn all about <a href="https://www.elastic.co/blog/opowering-energy-efficiency-with-elastic">Opower's use case</a>, plus you can also&nbsp;watch&nbsp;the interviews with their&nbsp;engineering team.&nbsp;</li><li>Justin Lintz from the Chartbeat Engineering team <a href="http://engineering.chartbeat.com/2015/05/26/logstash-deployment-and-scaling-tips/">shares details on their ELK cluster architecture</a>, all the tools they use to make it sing and some Logstash scaling tips. Thanks for the excellent walk through, Justin!</li><li>Shaker Islam from the Belly Engineering team has a ton of useful advice for folks using Elasticsearch and Ruby. <a href="https://tech.bellycard.com/blog/ruby-meets-elasticsearch/">Read on</a> for his great tutorial!</li></ul><blockquote class="twitter-tweet" lang="en"><a href="https://twitter.com/amy725">@amy725</a> <a href="https://twitter.com/dadoonet">@dadoonet</a> <a href="https://twitter.com/elastic">@elastic</a> Thought you might like my mention of ELK here: <a href="http://t.co/zcw4DV0mHh">http://t.co/zcw4DV0mHh</a> :)<br />— Samuel Scott (@samueljscott) <a href="https://twitter.com/samueljscott/status/603167312414384131">May 26, 2015</a></blockquote><h2>Slides and Videos</h2><p><strong>From last week's Barcelona Elastic Meetup</strong></p><div class="video embed-container" style="height: 319.725px;"><iframe src="//speakerdeck.com/player/817adcaadb684040b0208312a9d52371" class="video-iframe" style="width: 580px;height: 326.25px;"></iframe></div><p><strong>From last night's France Elastic Meetup</strong></p><div class="video embed-container" style="height: 319.725px;"><iframe src="//speakerdeck.com/player/9a0de62bb7b0435a855797c003487076" class="video-iframe" style="width: 580px;height: 326.25px;"></iframe></div><p><strong>From the recent DevOps London Exchange Meetup</strong></p><div class="video embed-container" style="height: 319.725px;"><iframe src="//www.slideshare.net/slideshow/embed_code/key/wSQuWGlLS9Xc31" class="video-iframe" style="width: 580px;height: 326.25px;"></iframe></div><h2>Where to find Us</h2><p><strong>Featured Meetup: On Tonight!</strong> (Don't let that tomorrow bit fool you. ;)</p><blockquote class="twitter-tweet" lang="en">TOMORROW! <a href="https://twitter.com/elastic">@elastic</a> is coming to talk about elasticsearch, logstash and kibana. Come get your <a href="https://twitter.com/hashtag/ELK?src=hash">#ELK</a> learn on! <a href="http://t.co/QWsLu1hyZV">http://t.co/QWsLu1hyZV</a><br />— San Antonio DevOps (@SADevOps) <a href="https://twitter.com/SADevOps/status/603209018388119552">May 26, 2015</a></blockquote><p>We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking about the ELK stack&nbsp;or hosting a meetup, <a href="mailto:stories@elastic.co">let us know!</a></p><h3>Austria</h3><p>The .NET Community Austria Meetup will get together on June 11 to talk all about the ELK stack. <a href="http://www.meetup.com/dotnet-austria/events/222279540/">RSVP now</a> to save your seat.</p><h3>France</h3><ul><li><a href="http://twitter.com/dadoonet">David Pilato</a> will also give two talks at the <a href="http://web2day.co/en/">web2day conference</a>. Join him in Nantes from June 3-5.</li><li>Our team will be hanging out in the exhibits hall at <a href="http://www.dotscale.io/">dotScale</a>, which takes place in Paris June 8. Come say hello!</li></ul><blockquote class="twitter-tweet" lang="en">During <a href="https://twitter.com/RivieraDEV">@RivieraDEV</a> 2015, <a href="https://twitter.com/dadoonet">@dadoonet</a> will show us how to add <a href="https://twitter.com/Elasticsearch">@ElasticSearch</a> to legacy applications in a blink ! <a href="https://t.co/SEQK4M9ZVe">https://t.co/SEQK4M9ZVe</a><br />— RivieraDEV (@RivieraDEV) <a href="https://twitter.com/RivieraDEV/status/595545106373029888">May 5, 2015</a></blockquote><blockquote class="twitter-tweet" lang="en">Université Kibana et Elasticsearch <a href="https://twitter.com/breizhcamp">@breizhcamp</a> : <a href="http://t.co/ZRAomhneKC">http://t.co/ZRAomhneKC</a>. Analyse de données temps réel avec la suite <a href="https://twitter.com/elastic">@elastic</a> et Adelean.<br />— Lucian Precup (@lucianprecup) <a href="https://twitter.com/lucianprecup/status/595532355265245184">May 5, 2015</a></blockquote><h3>Germany</h3><p>The Elastic team will be out in force at Berlin Buzzwords, including 3 talks, a booth on the show floor and a special conference meetup. You can <a href="http://berlinbuzzwords.com/">register now</a> for the conference and RSVP for <a href=" http://de.amiando.com/elastic_techtalk.html">our special meetup on June 3</a>. Buzzwords runs May 31 - June 3. Bonus: meet the newest members of our team, <a href="https://twitter.com/monicasarbu">Monica Sarbu</a> and <a href="https://twitter.com/tudor_g">Tudor Golubenco</a> of Packetbeat fame!</p><h3>Ireland</h3><p>Join Mark Harwood for an expert level session at NoSQL Matters Dublin on <a href="https://2015.nosql-matters.org/dub/speakers/#2156601127178">Building Entity Centric Indexes</a>. The conference takes place on June 4.</p><h3>Japan</h3><p>Please join all members of Elastic's team in Japan at <a href="http://events.linuxfoundation.jp/events/linuxcon-japan">LinuxCon Japan</a> in Tokyo on June 3-5. We will have a booth in the exhibits hall and will be very happy to answer any questions you may have.</p><h3>The Netherlands</h3><p><strong>On tomorrow</strong></p><blockquote class="twitter-tweet" lang="en">Technical Expert Aurélien Foucret of <a href="https://twitter.com/SmileNederland">@SmileNederland</a> talks about ElasticSearch at <a href="https://twitter.com/hashtag/MM15NL?src=hash">#MM15NL</a>, more info: <a href="http://t.co/yTD0XQ5X4E">http://t.co/yTD0XQ5X4E</a><br />— Meet Magento NL (@mm15nl) <a href="https://twitter.com/mm15nl/status/601002253059792896">May 20, 2015</a></blockquote><p><strong>GOTO Amsterdam Special Edition Meetup</strong></p><blockquote class="twitter-tweet" lang="en">Reg now: <a href="https://twitter.com/elastic">@elastic</a> Netherlands Meetup will get together 17 June w/ talks from <a href="https://twitter.com/rashidkpc">@rashidkpc</a> &amp; <a href="https://twitter.com/Wikimedia">@Wikimedia</a> dev Nik Everett <a href="http://t.co/Sh1R9u1sxR">http://t.co/Sh1R9u1sxR</a><br />— Leslie Hawthorn (@lhawthorn) <a href="https://twitter.com/lhawthorn/status/603516502197837825">May 27, 2015</a></blockquote><ul><li>But wait, there's more! Join us at <a href="http://gotocon.com/amsterdam-2015/schedule/index.jsp">GOTO Amsterdam 2015</a> for a full track on Elasticsearch, featuring five talks on topics from Scaling your ELK Cluster to Securing it with Shield. The conference runs June 18-19, with the Elasticsearch track running all day on June 18.</li><li>And even more! <a href="http://www.devopsdays.org/events/2015-amsterdam/">DevOps Days Amsterdam</a> returns to the Pakhuis de Zwijger June 24-26. You can visit us at our booth or hear from <a href="http://twitter.com/lhawthorn">Leslie Hawthorn</a> on her latest DevOps and Culture topic, <em>Fear of Failing Fast: How to Avoid Sabotaging Your Success</em>. Rumor has it that the amazing BBQ will be reprised this year, so <strong>do not</strong> miss this one!</li></ul><h3>Poland</h3><p>David Pilato and <a href="https://twitter.com/jsvd">João Duarte</a> will attend Devoxx Poland 2015 taking place June 22-25 in Krakow. Between the two of them, they'll deliver three talks:</p><ul><li>June 23 at 11:50 AM: David on <a href="http://cfp.devoxx.pl/2015/talk/NPG-0853/Make_sense_of_your_(BIG)_data">Make Sense of Your Big Data</a></li><li>June 24 at 3:10 PM: João on <a href="http://cfp.devoxx.pl/2015/talk/WEH-2600/Why_I_love_Logstash_and_you_should_too">Why I Love Logstash and You Should Too</a></li><li>June 25 at 1:30 PM: David and João host the <a href="http://cfp.devoxx.pl/2015/talk/NES-3891/Hands_On_Logstash">Hands on Logstash Workshop</a></li></ul><h3>United Kingdom</h3><ul><li>For folks in Cardiff, Wales, your welcome to attend our Elasticsearch Meetup hosted at DjangoCon EU. Make sure to pop by the conference to say hello to Honza Kral in the hallway track. <a href="http://2015.djangocon.eu/">DjangoCon EU 2015</a> runs May 31 - June 5. The nifty meetup featured below takes place on June 4.</li><li>Join us at Big Data Analytics London on June 16 for a seminar on <a href="http://www.whitehallmedia.co.uk/bda/seminars/">How To Extract Value From Your Data and Create Dazzling Dashboards</a>. The event takes place at the Hotel Russell.</li></ul><blockquote class="twitter-tweet" lang="en">South Wales Elasticsearchers: a meetup with a talk by Honza Král <a href="https://twitter.com/HonzaKral">@HonzaKral</a>, Elasticsearch developer: <a href="http://t.co/ZwUe3ln5VL">http://t.co/ZwUe3ln5VL</a> All welcome.<br />— DjangoCon Europe (@DjangoConEurope) <a href="https://twitter.com/DjangoConEurope/status/597880392717377536">May 11, 2015</a></blockquote><h3>United States - East</h3><ul><li>The Detroit Elastic User Group will hold its inaugural meeting tonight, May 27, where <a href="http://twitter.com/peterkimnyc">Peter Kim</a> will talk about using the ELK stack to get visibility into your datap. <a href="http://www.meetup.com/Elastic-User-Group-Detroit-MI/events/222155971/">Registration is closed</a> but if you still want to attend ping the organizers via meetup.com and they'll see what they can do.</li><li>The St. Louis Machine Learning and Data Science Meetup will get together on June 17 to talk Python and Elasticsearch. <a href="http://www.meetup.com/Elastic-User-Group-St-Louis-Missouri/events/222042377/">Register now</a> to save your place.</li><li>The Boston Elasticsearch Meetup will convene on June 18 to talk Elasticsearch &amp; Fuzzy Matching. <a href="http://www.meetup.com/Elasticsearch-Boston/events/222682277/">Register now</a> to save your seat.</li><li>Our team will be on the show floor to answer any questions you may have at <a href="http://www.redhat.com/summit/">Red Hat Summit 2015</a> slated for June 23-26 in Boston. Co-located with the event, check out Red Hat's Developer Conference <a href="http://www.devnation.org/">DevNation</a>, on June 21-15. Amongst the many great talks on offer at DevNation, our very own <a href="http://twitter.com/robynbergeron">Robyn Bergeron</a> will address <a href="http://www.devnation.org/#57c9a8c050ede2807fb0bfe462c4e6ec">The Democratization of DevOps</a>.</li></ul><h3>United States - West</h3><ul><li>The San Antonio DevOps Meetup will welcome 3 speakers from Elastic tonight, May 27. <a href="http://www.meetup.com/SanAntonioDevOps/events/222233563/">Register now</a> to hear all about the ELK stack for DevOps, including how we eat our own ELK-food from Elastic's infrastructure team lead.</li><li>The San Francisco MySQL Users Group will get together tonight, May 27, to hear all about operational visibility, powered by the ELK stack. <a href="http://www.meetup.com/sf-mysql/events/222051652/">Register now</a> to save your place.</li><li>On May 28, the Elasticsearch Dallas Fort Worth Meetup will convene to talk going into Production with Elasticsearch. <a href="http://www.meetup.com/Elasticsearch-User-Group-Dallas/events/221571448/">Register now</a> to save your place.</li><li>The Kansas City Big Data Meetup will get together for a lunch and learn on Elasticsearch &amp; Data Analysis on May 28. <a href="http://www.meetup.com/Kansas-City-Big-Data-Projects-Group/events/222325851/">Register now</a> to save your place. Lunch will be provided for attendees.</li><li>Our team will be on the show floor at <a href="http://2015.hadoopsummit.org/san-jose/">Hadoop Summit</a>, which runs June 9-11 in San Jose, California. Stop by, say hello and learn all about our solutions to make Hadoop sing by using Elasticsearch.</li><li>Join us&nbsp;at <a href="http://monitorama.com/">Monitorama 2015</a> in Portland, Oregon and&nbsp;catch&nbsp;a lightning demo&nbsp;from our very own <a href="http://twitter.com/suyograo">Suyog Rao</a>. Monitorama runs June 15-17.</li></ul><blockquote class="twitter-tweet" lang="en">On stage now: Rob Tice, Director at <a href="https://twitter.com/knowint">@knowint</a> talking about complex modelling of rich text data in <a href="https://twitter.com/hashtag/elasticsearch?src=hash">#elasticsearch</a> <a href="http://t.co/RqIho6753I">pic.twitter.com/RqIho6753I</a><br />— elastic London (@elastic_london) <a href="https://twitter.com/elastic_london/status/603269569097146368">May 26, 2015</a></blockquote><h2>Where to Find You</h2><p><strong>PSST!</strong>&nbsp;If you're a regular reader, you're thinking of skipping this section. You may even be thinking to yourself, yes of course I will drop a note on Twitter when I am giving a talk on all things ELK. That's awesome, because we'd like to showcase every meetup, conference presentation and workshop on Elasticsearch, Logstash, and Kibana happening worldwide. And now, we've made it even easier for you to get support for your meetup!</p><p>Head on over to&nbsp; <a href="https://www.elastic.co/community/meetups">our meetups page</a>! (And we'll still totally send you swag if you're giving a talk on anything ELKy at a conference.)</p><p>Oh yeah, we're also&nbsp; <a href="https://www.elastic.co/about/careers">hiring</a>. If you'd like us to find you for employment purposes, just&nbsp;<a href="mailto:jobs@elastic.co">drop us a note</a>. We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.</p><h2>Trainings</h2><p>If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:</p><ul><li><a href="http://purchases.elastic.co/class/Elastic/Paris/2015-05-27">Paris</a> - May 27, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Elastic/Miami/2015-05-27">Miami</a>&nbsp;-&nbsp;May 27, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/NewYork/2015-06-16">New York</a> - June 16, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/SanFracisco/2015-06-16">San Francisco</a> - June 16, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/NewYork/2015-06-17">New York</a> - June 17, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/SanFrancisco/2015-06-17">San Francisco</a>- June 17, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/SanFrancisco/2015-06-17">San Francisco</a> - June 17, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Elastic/Workshop/SpringPeople/2015-06-22">Bangalore</a> - June 22, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Elastic/Core/SpringPeople/2015-06-23">Bangalore</a> - June 23, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Amsterdam/2015-06-23">Amsterdam</a> - June 23, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Manchester/2015-06-23">Manchester</a>&nbsp;-&nbsp;June 23, 2015&nbsp;(Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Toronto/2015-06-23">Toronto</a> -&nbsp;June 23, 2015&nbsp;(Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Amsterdam/2015-06-24">Amsterdam</a> - June 24, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Manchester/2015-06-24">Manchester</a> - June 24, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Toronto/2015-06-24">Toronto</a> - June 24, 2015&nbsp;(Core Elasticsearch)</li></ul>

This Week in Elastic: Welcome Packetbeat!

<p>Welcome to <em>This Week in Elastic</em>! In this weekly round up post, we bring you news on all aspects of the ELK Stack, including updates on Elasticsearch, Logstash and Kibana development, tips and tricks and upcoming events.</p><p><strong>We think you're pretty darn nifty, too, Nick!</strong></p><blockquote class="twitter-tweet" lang="en">Proud to have my second PR into Elasticsearch. Minor changes, but the <a href="https://twitter.com/elastic">@elastic</a> community has been great and makes me want to contribute more<br />— Nick Canzoneri (@nick_canz) <a href="https://twitter.com/nick_canz/status/601016624137805824">May 20, 2015</a></blockquote><h2>Elastic News</h2><p>Useful blog posts, how tos, tutorials&nbsp;and articles&nbsp;from the wide world of all the ELK stack ecosystem.</p><ul><li>Check out our weekly deep dive into the latest developments in <a href="https://www.elastic.co/blog/this-week-in-elasticsearch-and-apache-lucene-2015-05-19">Elasticsearch and Apache Lucene</a>. Includes a pointer to Mike McCandless' latest work on bug hunting in Lucene - a great read!</li><li>We've also got the <a href="https://www.elastic.co/blog/logstash-lines-2015-05-15">The Logstash Lines</a>, our weekly Logstash update. All about 1.5 GA!</li><li>Aaron Mildenstein teaches you all about how to check Logstash's pulse, a.k.a. using our brand new <a href="https://www.elastic.co/blog/how-to-check-logstashs-pulse">Logstash heartbeat plugin</a>.</li><li>Christoffer Vig has shared the first article in a tutorial series, <em>Analyzing Weblogs with Elasticsearch in the Cloud</em>. All about <a href="https://www.found.no/foundation/analyzing-weblogs-with-elasticsearch/">using Logstash and Kibana on the Found by Elastic</a> service.</li><li>Our friends over at Predikto have penned a post on using Elasticsearch, Apache Spark and es-hadoop to <a href="https://www.elastic.co/blog/elasticsearch-at-predikto">gain insights into the world of IoT data</a>. Excellent overview, with insights into keeping costs low when running on AWS.</li><li>Fan of consul.io? Turns out, consul will&nbsp;be of great help in herding your ELK clusters. Read more on the <a href="http://doics.co/2015/05/18/increase-your-elk-herd-with-consul-io/">DevOps is Common Sense blog</a>.</li><li>Ash Powell from thetrainline.com shares how the ELK stack powers their <a href="http://engineering.thetrainline.com/2015/05/20/what-the-elk-log-aggregation/">centralized logging infrastructure</a>.&nbsp;&nbsp;Stay tuned to hear how they head to the cloud, scale up and out, and more!</li></ul><blockquote class="twitter-tweet" lang="en">Just announced: <a href="https://twitter.com/hashtag/elasticwatcher?src=hash">#elasticwatcher</a> Beta (alerting for <a href="https://twitter.com/hashtag/elasticsearch?src=hash">#elasticsearch</a>) goes public. Try it today! <a href="http://t.co/tNmbKe3KC8">http://t.co/tNmbKe3KC8</a> <a href="http://t.co/V7gRYQKXDl">pic.twitter.com/V7gRYQKXDl</a><br />— elastic (@elastic) <a href="https://twitter.com/elastic/status/601048142453317633">May 20, 2015</a></blockquote><h2>Slides and Videos</h2><p><a href="http://twitter.com/spinscale"><strong>Alexander Reelsen's</strong></a><strong> recent presentation from Craft Conference 2015 in Budapest</strong></p><div class="video embed-container" style="height: 319.725px;"><iframe src="//www.ustream.tv/embed/recorded/61443343?v=3&wmode=direct" class="video-iframe" style="width: 580px;height: 326.25px;"></iframe></div><blockquote class="twitter-tweet" lang="en">Slides from my talk at <a href="https://twitter.com/MelbourneSearch">@MelbourneSearch</a> on Migrating from Hibernate Search to Elasticsearch <a href="https://t.co/0JnoXG7jAv">https://t.co/0JnoXG7jAv</a><br />— Christian Strzadala (@cstrzadala) <a href="https://twitter.com/cstrzadala/status/600812453551738880">May 19, 2015</a></blockquote><blockquote class="twitter-tweet" lang="en">Slides from my <a href="https://twitter.com/elastic">@elastic</a> <a href="https://twitter.com/hashtag/Elasticsearch?src=hash">#Elasticsearch</a>, Logstash &amp; Kibana talk at <a href="https://twitter.com/rittmanmead">@rittmanmead</a> <a href="https://twitter.com/hashtag/BIForum2015?src=hash">#BIForum2015</a> last week: <a href="http://t.co/fxLt5zXKSM">http://t.co/fxLt5zXKSM</a><br />— Robin Moffatt (@rmoff) <a href="https://twitter.com/rmoff/status/597758553349365760">May 11, 2015</a></blockquote><h2>Where to find Us</h2><p><strong>Featured Meetup: On Tonight!</strong></p><blockquote class="twitter-tweet" lang="en">On tonight at 7 PM: <a href="https://twitter.com/elastic">@elastic</a> LAX meetup (Venice, CA) on logging AWS Cloudwatch &amp; Cloudtrail data. RSVP now: <a href="http://t.co/apHaXVLEAc">http://t.co/apHaXVLEAc</a><br />— Leslie Hawthorn (@lhawthorn) <a href="https://twitter.com/lhawthorn/status/601032286696513536">May 20, 2015</a></blockquote><p>We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking about the ELK stack&nbsp;or hosting a meetup, <a href="mailto:stories@elastic.co">let us know!</a></p><h3>Algeria</h3><p><a href="https://twitter.com/BennacerSamir">Samir Bennacer</a> will take the stage at Voxxed Days Algiers to present An Introduction to the ELK Stack. This first ever conference takes place May 23, and Samir's talk is on for 2:00 PM.</p><h3>Austria</h3><p>The .NET Community Austria Meetup will get together on June 11 to talk all about the ELK stack. <a href="http://www.meetup.com/dotnet-austria/events/222279540/">RSVP now</a> to save your seat.</p><h3>Canada</h3><ul><li>If you're attending the upcoming Open Stack Summit in Vancouver, make sure to catch the <a href="https://openstacksummitmay2015vancouver.sched.org/mobile/#session:3f98df0b3864b58ce30bbac3dc5d7b2f">In Search of OpenStack Logs: A Hands-On Workshop</a> session. You can also join <a href="http://twitter.com/robynbergeron">Robyn Bergeron</a>, our Operations Advocate, at our <a href="http://openstacksummitmay2015vancouver.sched.org/event/a5f64162b41103c6d6f6d01f405e525e#.VVIT2tOqqko">ELK Crossing BoF session</a>, on this very afternoon. OpenStack Summit runs May 18-22.</li><li>Two engineers from Elastic, <a href="https://twitter.com/areekzillur">Areek Zillur</a> and <a href="https://twitter.com/suyograo">Suyog Rao</a>, will be speaking at tonight's Polyglot Vancouver meetup. There's still more than 50 seats left, so let's get this meetup up to 200 cool humans! <a href="http://www.meetup.com/PolyglotVancouver/events/222325728/">RSVP here</a> to attend.</li><li>Rumor has it that Suyog will also be attending this week's <a href="http://www.polyglotconf.com/">Polyglot Unconference</a> in Vancouver, which takes place May 22-23. Join him and propose an unconference session, or just enjoy the workshops.</li></ul><h3>France</h3><ul><li><a href="http://twitter.com/dadoonet">David Pilato</a> will visit the Toulouse JUG tomorrow to talk about all things Elasticsearch. You can still <a href="http://www.toulousejug.org/2015/05/12/Elastic.html">register now</a> to attend his talk on the evening of May 21.</li><li>The Elasticsearch France Meetup returns to Paris on May 26. <a href="http://www.meetup.com/elasticsearchfr/events/221413422/">Watch this event page</a> to see if space opens on the waitlist!</li><li>David will also give two talks at the <a href="http://web2day.co/en/">web2day conference</a>. Join him in Nantes from June 3-5.</li><li>Our team will be hanging out in the exhibits hall at <a href="http://www.dotscale.io/">dotScale</a>, which takes place in Paris June 8. Come say hello!</li></ul><blockquote class="twitter-tweet" lang="en">During <a href="https://twitter.com/RivieraDEV">@RivieraDEV</a> 2015, <a href="https://twitter.com/dadoonet">@dadoonet</a> will show us how to add <a href="https://twitter.com/Elasticsearch">@ElasticSearch</a> to legacy applications in a blink ! <a href="https://t.co/SEQK4M9ZVe">https://t.co/SEQK4M9ZVe</a><br />— RivieraDEV (@RivieraDEV) <a href="https://twitter.com/RivieraDEV/status/595545106373029888">May 5, 2015</a></blockquote><h3>Germany</h3><ul><li>Check out <a href="http://twitter.com/purbon">Pere Urbon's</a> talk tonight&nbsp;at the May Berlin DevOps Meetup, where he'll talk <em>Scaling Logstash: A Collection of War Stories</em>. <a href="http://www.meetup.com/blndevops/events/221953734/">Register now</a> to save your seat.</li><li>The Stuttgart Elasticsearch Meetup will get together on May 21 to talk Shield: Securing Elasticsearch. <a href="http://www.meetup.com/Elasticsearch-Stuttgart/events/221992283/">Register now</a> to save your seat.</li><li>The Elastic team will be out in force at Berlin Buzzwords, including 3 talks, a booth on the show floor and a special conference meetup. You can <a href="http://berlinbuzzwords.com/">register now</a> for the conference and RSVP for <a href=" http://de.amiando.com/elastic_techtalk.html">our special meetup on June 3</a>. Buzzwords runs May 31 - June 3.</li></ul><h3>Ireland</h3><p>Join Mark Harwood for an expert level session at NoSQL Matters Dublin on <a href="https://2015.nosql-matters.org/dub/speakers/#2156601127178">Building Entity Centric Indexes</a>. The conference takes place on June 4.</p><h3>Israel</h3><p>We've got dueling meetups in Tel-Aviv Yafo on May 26:</p><ul><li>Israel AWS User Group: How to Use ELK to Monitor Large-Scale AWS Environments - <a href="http://www.meetup.com/AWS-IL/events/221495902/">RSVP</a></li><li>Big things are happening hear meetup: Geo Data analytics &amp; Real-world use cases for Elasticsearch - <a href="http://www.meetup.com/Big-things-are-happening-here/events/221195508/">RSVP</a></li></ul><h3>Japan</h3><p>Please join all members of Elastic's team in Japan at <a href="http://events.linuxfoundation.jp/events/linuxcon-japan">LinuxCon Japan</a> in Tokyo on June 3-5. We will have a booth in the exhibits hall and will be very happy to answer any questions you may have.</p><h3>The Netherlands</h3><blockquote class="twitter-tweet" lang="en">Technical Expert Aurélien Foucret of <a href="https://twitter.com/SmileNederland">@SmileNederland</a> talks about ElasticSearch at <a href="https://twitter.com/hashtag/MM15NL?src=hash">#MM15NL</a>, more info: <a href="http://t.co/yTD0XQ5X4E">http://t.co/yTD0XQ5X4E</a><br />— Meet Magento NL (@mm15nl) <a href="https://twitter.com/mm15nl/status/601002253059792896">May 20, 2015</a></blockquote><h3>Poland</h3><p><a href="http://twitter.com/honzakral">Honza Kral</a> will take the stage at 6:30 PM on May 26 to talk Beyond the Basics with Elasticsearch at the <a href="http://summit.pywaw.org/">PyWaw Summit</a>. The event runs May 25-26 in Warsaw. (PS - We've also heard a rumor that the organizers will be offering Schema-Free beer on Elastic. Raise a glass with Honza if we're right!)</p><h3>Spain</h3><p>Join <a href="https://twitter.com/nellicus">Antonio Bonucelli</a> for the next Elasticsearch Barcelona Meetup on May 21, where you'll learn all about Moving to Production with Elasticsearch: Best Practices. <a href="http://www.meetup.com/ElasticSearch-Barcelona/events/222153460/">Register now</a> to save your seat.</p><h3>Sweden</h3><p>The Stockholm Elasticsearch Meetup will get together on May 27 to hear about two use cases. <a href="http://www.meetup.com/Stockholm-Elastic-Fantastics/events/222346404">RSVP now</a> to save your seat.</p><h3>Switzerland</h3><p>The Elasticsearch Zurich Meetup group will convene on May 27 to hear about Elastic's security product, Shield. <a href="http://www.meetup.com/elasticsearch-switzerland/events/222349897/">RSVP here</a> to attend.</p><h3>United Kingdom</h3><ul><li>We are full up for the May London Elastic Meetup on May 26, but you can <a href="http://www.meetup.com/London-ElasticSearch-User-Group/events/219614566/">join the waitlist</a>. We're hoping some space frees up! (Even if that doesn't work out, we'll have another London meetup coming soon.)</li><li>For folks in Cardiff, Wales, your welcome to attend our Elasticsearch Meetup hosted at DjangoCon EU. Make sure to pop by the conference to say hello to Honza Kral in the hallway track. <a href="http://2015.djangocon.eu/">DjangoCon EU 2015</a> runs May 31 - June 5. The nifty meetup featured below takes place on June 4.</li></ul><blockquote class="twitter-tweet" lang="en">South Wales Elasticsearchers: a meetup with a talk by Honza Král <a href="https://twitter.com/HonzaKral">@HonzaKral</a>, Elasticsearch developer: <a href="http://t.co/ZwUe3ln5VL">http://t.co/ZwUe3ln5VL</a> All welcome.<br />— DjangoCon Europe (@DjangoConEurope) <a href="https://twitter.com/DjangoConEurope/status/597880392717377536">May 11, 2015</a></blockquote><h3>United States - East</h3><ul><li>The Detroit Elastic User Group will hold its inaugural meeting on May 27, where Peter Kim will reprise his talk from the DevOps and Automation New Jersey Meetup. <a href="http://www.meetup.com/Elastic-User-Group-Detroit-MI/events/222155971/">Register now</a> to save your seat.</li></ul><h3>United States - West</h3><ul><li>Heading to <a href="http://gluecon.com/">Gluecon</a>? Great, so are we! You can visit us in the exhibits hall to get questions answered throughout the conference, and make sure to check our <a href="http://twitter.com/theuntergeek">Aaron Mildenstein's</a> talk: <em>An Introduction to the ELK stack</em>. Gluecon runs May 20-21 in Broomfield, Colorado.</li><li>The San Antonio DevOps Meetup will welcome 3 speakers from Elastic on May 27. <a href="http://www.meetup.com/SanAntonioDevOps/events/222233563/">Register now</a> to hear all about the ELK stack for DevOps, including how we eat our own ELK-food from Elastic's infrastructure team lead.</li><li>The San Francisco MySQL Users Group will get together on May 27 to hear all about operational visibility, powered by the ELK stack. <a href="http://www.meetup.com/sf-mysql/events/222051652/">Register now</a> to save your place.</li><li>On May 28, the Elasticsearch Dallas Fort Worth Meetup will convene to talk going into Production with Elasticsearch. <a href="http://www.meetup.com/Elasticsearch-User-Group-Dallas/events/221571448/">Register now</a> to save your place.</li><li>The Kansas City Big Data Meetup will get together for a lunch and learn on Elasticsearch &amp; Data Analysis on May 28. <a href="http://www.meetup.com/Kansas-City-Big-Data-Projects-Group/events/222325851/">Register now</a> to save your place. Lunch will be provided for attendees.</li><li>Our team will be on the show floor at <a href="http://2015.hadoopsummit.org/san-jose/">Hadoop Summit</a>, which runs June 9-11 in San Jose, California. Stop by, say hello and learn all about our solutions to make Hadoop sing by using Elasticsearch.</li></ul><blockquote class="twitter-tweet" lang="en">The Elastic family: Big data search, discovery and analytics (p 20-22) - by Christoffer Vig <a href="https://twitter.com/Babadofar">@babadofar</a> <a href="http://t.co/9As91C9SUQ">http://t.co/9As91C9SUQ</a> <a href="https://twitter.com/hashtag/Elasticsearch?src=hash">#Elasticsearch</a><br />— Found by Elastic (@foundsays) <a href="https://twitter.com/foundsays/status/600275036113633281">May 18, 2015</a></blockquote><h2>Where to Find You</h2><p><strong>PSST!</strong>&nbsp;If you're a regular reader, you're thinking of skipping this section. You may even be thinking to yourself, yes of course I will drop a note on Twitter when I am giving a talk on all things ELK. That's awesome, because we'd like to showcase every meetup, conference presentation and workshop on Elasticsearch, Logstash, and Kibana happening worldwide. And now, we've made it even easier for you to get support for your meetup!</p><p>Head on over to&nbsp; <a href="https://www.elastic.co/community/meetups">our meetups page</a>! (And we'll still totally send you swag if you're giving a talk on anything ELKy at a conference.)</p><p>Oh yeah, we're also&nbsp; <a href="https://www.elastic.co/about/careers">hiring</a>. If you'd like us to find you for employment purposes, just&nbsp;<a href="mailto:jobs@elastic.co">drop us a note</a>. We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.</p><h2>Trainings</h2><p>If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:</p><ul><li><a href="http://purchases.elastic.co/class/Elastic/Workshop/SpringPeople/2015-05-25">Bangalore</a> - May 25, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Elastic/Core/SpringPeople/2015-05-26">Bangalore</a> - May 26, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Elastic/Paris/2015-05-26">Paris</a>&nbsp;-&nbsp;May 26, 2015&nbsp;(Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Elastic/Paris/2015-05-27">Paris</a> - May 27, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Elastic/Miami/2015-05-27">Miami</a>&nbsp;-&nbsp;May 27, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/NewYork/2015-06-16">New York</a> - June 16, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/SanFracisco/2015-06-16">San Francisco</a> - June 16, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/NewYork/2015-06-17">New York</a> - June 17, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/SanFrancisco/2015-06-17">San Francisco</a>- June 17, 2015 (Core Elasticsearch)</li><li><a href="<a href=">New York</a>- June 17, 2015 (Core Elasticsearch Operations - <em>New Course</em>)</li><li><a href="http://purchases.elastic.co/class/SanFrancisco/2015-06-17">San Francisco</a> - June 17, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Elastic/Workshop/SpringPeople/2015-06-22">Bangalore</a> - June 22, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Elastic/Core/SpringPeople/2015-06-23">Bangalore</a> - June 23, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Amsterdam/2015-06-23">Amsterdam</a> - June 23, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Manchester/2015-06-23">Manchester</a>&nbsp;-&nbsp;June 23, 2015&nbsp;(Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Toronto/2015-06-23">Toronto</a> -&nbsp;June 23, 2015&nbsp;(Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Amsterdam/2015-06-24">Amsterdam</a> - June 24, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Manchester/2015-06-24">Manchester</a> - June 24, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Toronto/2015-06-24">Toronto</a> - June 24, 2015&nbsp;(Core Elasticsearch)</li></ul>

This Week in Elastic: Watcher Beta goes public

<p>Welcome to <em>This Week in Elastic</em>! In this weekly round up post, we bring you news on all aspects of the ELK Stack, including updates on Elasticsearch, Logstash and Kibana development, tips and tricks and upcoming events.</p><blockquote class="twitter-tweet" lang="en"><a href="https://twitter.com/hashtag/Watcher?src=hash">#Watcher</a>: You know, for Alerting (Coming Soon)...Get details &amp; sign up for the webinar now <a href="http://t.co/b8o7xw4G9N">http://t.co/b8o7xw4G9N</a> <a href="http://t.co/L1p8Hjfydn">pic.twitter.com/L1p8Hjfydn</a><br />— elastic (@elastic) <a href="https://twitter.com/elastic/status/597765780315443201">May 11, 2015</a></blockquote><h2>Elastic News</h2><p>Useful blog posts, how tos, tutorials&nbsp;and articles&nbsp;from the wide world of all the ELK stack ecosystem.</p><ul><li>Check out our weekly deep dive into the latest developments in <a href="https://www.elastic.co/blog/2015-05-12-this-week-in-elasticsearch-and-apache-lucene">Elasticsearch and Apache Lucene</a>.</li><li>We've also got the <a href="https://www.elastic.co/blog/2015-05-08-logstash-lines">The Logstash Lines</a>, our weekly Logstash update. All about 1.5 RC4, scaling Logstash and more!</li><li>The Kibana team made some major progress towards the 1.4 release over the past week, including pinned filters. Read on to learn about these and other updates in <a href="https://www.elastic.co/blog/kurrently-kibana-2015-05-08">Kurrently in Kibana</a>.</li><li>And for all you lovers of Kibana, here's the next video tutorial in our series: <a href="https://www.elastic.co/blog/kibana-4-video-tutorials-part-4">embedding K4 visualizations into your homepage</a>.</li><li>Looking for best practices for using Elasticsearch in virtualized environments? This <a href="https://www.elastic.co/blog/how-to-handle-elasticsearch-virtualization">awesome tutorial</a> is brought to you courtesy of <a href="https://twitter.com/gmoskovicz">Gabriel Moskovicz</a>, one of our awesome support engineers!</li><li><a href="http://twitter.com/mpdreamz">Martijn Laarman</a> and <a href="http://twitter.com/gregmarzouka">Greg Marzouka</a> released <a href="https://www.elastic.co/blog/nest-1-5-released">Elasticsearch .NET and NEST 1.5</a> last week. You should read the whole release blog post to learn about all the awesome changes, but if you just skip to the <a href="https://github.com/elastic/elasticsearch-net-example">amazing tutorial</a> they published along with the latest release, we will totally understand.</li><li>The ELK Stack: bringing you all the <a href="http://leandromoreira.com.br/2015/04/26/fifa-2014-world-cup-live-stream-architecture/">monitoring infrastructure</a> for the FIFA 2014 World Cup live stream architecture.</li><li>Last, but not the least, learn&nbsp;all&nbsp;about our newly-launched <a href="https://www.elastic.co/blog/join-the-conversation">discussion forums</a>!</li></ul><blockquote class="twitter-tweet" lang="en">New article from one of the Found founders: A Dive into the Elasticsearch Storage <a href="https://t.co/Bfm27oI4yb">https://t.co/Bfm27oI4yb</a> <a href="https://twitter.com/hashtag/Elasticsearch?src=hash">#Elasticsearch</a><br />— Found by Elastic (@foundsays) <a href="https://twitter.com/foundsays/status/596230810660904960">May 7, 2015</a></blockquote><h2>Slides and Videos</h2><div class="video embed-container" style="height: 319.725px;"><iframe src="//speakerdeck.com/player/ba016e8c03514e318ef17de2369ba302" class="video-iframe" style="width: 580px;height: 326.25px;"></iframe></div><div class="video embed-container" style="height: 319.725px;"><iframe src="//speakerdeck.com/player/17175e6a1f8e4a7498bef3e03ba0822a" class="video-iframe" style="width: 580px;height: 326.25px;"></iframe></div><div class="video embed-container" style="height: 319.725px;"><iframe src="//speakerdeck.com/player/14b4a923983d4fdd93a2cc218f0deee7" class="video-iframe" style="width: 580px;height: 326.25px;"></iframe></div><h2>Where to find Us</h2><p><strong>Featured Meetup</strong></p><blockquote class="twitter-tweet" lang="en">If you're in Denver &amp; interested in <a href="https://twitter.com/hashtag/Elasticsearch?src=hash">#Elasticsearch</a>, come hear me on things I wish I'd known as a newbie ES app dev: <a href="http://t.co/KNkatnr4m0">http://t.co/KNkatnr4m0</a><br />— Shaunak Kashyap (@shaunak) <a href="https://twitter.com/shaunak/status/598188068512763904">May 12, 2015</a></blockquote><p>We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking about the ELK stack&nbsp;or hosting a meetup, <a href="mailto:stories@elastic.co">let us know!</a></p><h3>Algeria</h3><p><a href="https://twitter.com/BennacerSamir">Samir Bennacer</a> will take the stage at Voxxed Days Algiers to present An Introduction to the ELK Stack. This first ever conference takes place May 23, and Samir's talk is on for 2:00 PM.</p><h3>Australia</h3><p>The Melbourne Search Users Group will meet up on May 19 for a talk from Movideo: Migrating from Hibernate Search to Elasticsearch. <a href="http://www.meetup.com/melbourne-search/events/222171087/">Register now</a> to save your seat. We're pretty sure our very own <a href="https://twitter.com/warkolm">Mark Walkom</a> will be joining you, too!</p><h3>Canada</h3><ul><li>If you're attending the upcoming Open Stack Summit in Vancouver, make sure to catch the <a href="https://openstacksummitmay2015vancouver.sched.org/mobile/#session:3f98df0b3864b58ce30bbac3dc5d7b2f">In Search of OpenStack Logs: A Hands-On Workshop</a> session. You can also join <a href="http://twitter.com/robynbergeron">Robyn Bergeron</a>, our Operations Advocate, at our <a href="http://openstacksummitmay2015vancouver.sched.org/event/a5f64162b41103c6d6f6d01f405e525e#.VVIT2tOqqko">ELK Crossing BoF session</a> on Wednesday afternoon.&nbsp;OpenStack Summit runs May 18-22.</li><li>The Elasticsearch Toronto Meetup group will get together on May 19 to talk Centralized Logs and Analytics with ELK. <a href="http://www.meetup.com/Elasticsearch-Toronto/events/222202947/">Register now</a> to save your place.</li></ul><h3>Czech Republic</h3><p>Our very own <a href="http://twitter.com/karmiq">Karel Minařík</a> will be on hand for Q&amp;A on Elasticsearch in Prague on May 7. You can RSVP for this talk <a href="https://www.facebook.com/events/1578142242444167/">on Facebook</a>.</p><h3>France</h3><p>The Elasticsearch France Meetup returns to Paris on May 26. <a href="http://www.meetup.com/elasticsearchfr/events/221413422/">Watch this space</a> so you know when we open up RSVPs!</p><blockquote class="twitter-tweet" lang="en">During <a href="https://twitter.com/RivieraDEV">@RivieraDEV</a> 2015, <a href="https://twitter.com/dadoonet">@dadoonet</a> will show us how to add <a href="https://twitter.com/Elasticsearch">@ElasticSearch</a> to legacy applications in a blink ! <a href="https://t.co/SEQK4M9ZVe">https://t.co/SEQK4M9ZVe</a><br />— RivieraDEV (@RivieraDEV) <a href="https://twitter.com/RivieraDEV/status/595545106373029888">May 5, 2015</a></blockquote><h3>Germany</h3><ul><li>Our partner firm, Trifork, has organized a Berlin GOTO Night with our very own <a href="http://twitter.com/purbon">Pere Urbon</a> addressing Making Sense of Your Data. The GOTO Night takes place on May 13 and <a href="https://secure.trifork.com/berlin-2015/freeevent/index.jsp?eventOID=6986">registration</a> is free of charge.</li><li>And because we know you can't get enough of Pere, check out his talk at the May Berlin DevOps Meetup on May 20, where he'll talk <em>Scaling Logstash: A Collection of War Stories</em>. <a href="http://www.meetup.com/blndevops/events/221953734/">Register now</a> to save your seat.</li><li>The Stuttgart Elasticsearch Meetup will get together on May 21 to talk Shield: Securing Elasticsearch. <a href="http://www.meetup.com/Elasticsearch-Stuttgart/events/221992283/">Register now</a> to save your seat.</li><li>The Elastic team will be out in force at Berlin Buzzwords, including 3 talks, a booth on the show floor and a special conference meetup. Stay tuned for more details, but you can <a href="http://berlinbuzzwords.com/">register now</a> for the conference. The event runs May 31 - June 3.</li></ul><h3>Israel</h3><p>We've got dueling meetups in Tel-Aviv Yafo on May 26:</p><ul><li>Israel AWS User Group: How to Use ELK to Monitor Large-Scale AWS Environments - <a href="http://www.meetup.com/AWS-IL/events/221495902/">RSVP</a></li><li>Big things are happening hear meetup: Geo Data analytics &amp; Real-world use cases for Elasticsearch - <a href="http://www.meetup.com/Big-things-are-happening-here/events/221195508/">RSVP</a></li></ul><h3>The Netherlands</h3><p>Johan Guldmyr will host a talk on dCache and the ELK stack at the <a href="https://indico.desy.de/conferenceTimeTable.py?confId=11773#20150519">9th Annual dCache Workshop</a> at 2:20 PM on May 19. The Workshop runs May 18-20 in Amsterdam.</p><h3>Poland</h3><p><a href="http://twitter.com/honzakral">Honza Kral</a> will take the stage at 6:30 PM on May 26 to talk Beyond the Basics with Elasticsearch at the <a href="http://summit.pywaw.org/">PyWaw Summit</a>. The event runs May 25-26 in Warsaw.</p><h3>Portugal</h3><p>The Merge Lisbon Meetup Group will get together on May 14 to talk about the ELK stack. <a href="http://www.meetup.com/Merge-Lisbon/events/221655010/">Register now</a> to hear from our very own Logstash engineering team member <a href="https://twitter.com/jsvd">João Duarte</a>.</p><h3>Spain</h3><p>Join <a href="https://twitter.com/nellicus">Antonio Bonucelli</a> for the next Elasticsearch Barcelona Meetup on May 21, where you'll learn all about Moving to Production with Elasticsearch: Best Practices. <a href="http://www.meetup.com/ElasticSearch-Barcelona/events/222153460/">Register now</a> to save your seat.</p><h3>Turkey</h3><p>Voxxed Days Istanbul is coming up on May 9. Join <a href="http://twitter.com/dadoonet">David Pilato</a> to learn all about Advanced Search for Your Legacy Application Using Elasticsearch. David <a href="https://voxxeddaysistanbul2015.sched.org/event/4ee52a07030088d7f97fbca7c72a97a1?iframe=no&w=&sidebar=no&bg=no#.VS4xAxOUdTU">takes the stage</a> at 11:30 AM.</p><h3>United Kingdom</h3><p>For folks in Cardiff, Wales, your welcome to attend our Elasticsearch Meetup hosted at DjangoCon EU. Make sure to pop by the conference to say hello to Honza Kral in the hallway track. <a href="http://2015.djangocon.eu/">DjangoCon EU 2015</a> runs May 31 - June 5.</p><blockquote class="twitter-tweet" lang="en">South Wales Elasticsearchers: a meetup with a talk by Honza Král <a href="https://twitter.com/HonzaKral">@HonzaKral</a>, Elasticsearch developer: <a href="http://t.co/ZwUe3ln5VL">http://t.co/ZwUe3ln5VL</a> All welcome.<br />— DjangoCon Europe (@DjangoConEurope) <a href="https://twitter.com/DjangoConEurope/status/597880392717377536">May 11, 2015</a></blockquote><h3>United States - East</h3><ul><li>The Elastic Chicago Meetup will convene tonight, May 13, to discuss Performance Tuning for Elasticsearch. <a href="http://www.meetup.com/Chicago-Elastic-Fantastics/events/221994604/">Register now</a> to save your place. For folks in Chicago interested in all things .NET, you may also want to <a href="http://www.meetup.com/Chicago-Elastic-Fantastics/events/222042578/">join the CNUG User Group meeting</a> on May 20 for an Introduction to the ELK stack.</li><li>The Detroit Elastic User Group will hold its inaugural meeting on May 27, where Peter Kim will reprise his talk from the DevOps and Automation New Jersey Meetup. <a href="http://www.meetup.com/Elastic-User-Group-Detroit-MI/events/222155971/">Register now</a> to save your seat.</li></ul><h3>United States - West</h3><ul><li>Does your whale need an ELK buddy? Join the Seattle User Group on May 18 to hear all about using Docker together with the ELK stack. <a href="http://www.meetup.com/Seattle-ElasticSearch-Meetup/events/220885729/">Register now</a> to save your seat.</li><li>Heading to <a href="http://gluecon.com/">Gluecon</a>? Great, so are we! You can visit us in the exhibits hall to get questions answered throughout the conference, and make sure to check our <a href="http://twitter.com/theuntergeek">Aaron Mildenstein's</a> talk: <em>An Introduction to the ELK stack</em>. Gluecon runs May 20-21 in Broomfield, Colorado.</li><li>If you're not heading to Gluecon but find yourself in Denver, you can still hear from our awesome Gluecon speaker, our newest Developer Advocate, <a href="http://twitter.com/shaunak">Shaunak Kashyap</a>, and Bitly's engineering team on their Elasticsearch user case. <a href="http://www.meetup.com/Elasticsearch-Denver/events/222066510/">Register now</a> to save your place for the Denver Meetup coming up on May 18!</li><li>The San Antonio DevOps Meetup will welcome 3 speakers from Elastic on May 27. <a href="http://www.meetup.com/SanAntonioDevOps/events/222233563/">Register now</a> to hear all about the ELK stack for DevOps, including how we eat our own ELK-food from Elastic's infrastructure team lead.</li><li>On May 28, the Elasticsearch Dallas Fort Worth Meetup will convene to talk going into Production with Elasticsearch. <a href="http://www.meetup.com/Elasticsearch-User-Group-Dallas/events/221571448/">Register now</a> to save your place.</li><li>The Kansas City Big Data Meetup will get together for a lunch and learn on Elasticsearch &amp; Data Analysis on May 28. <a href="http://www.meetup.com/Kansas-City-Big-Data-Projects-Group/events/222325851/">Register now</a> to save your place. Lunch will be provided for attendees.</li></ul><h2>Where to Find You</h2><p><strong>PSST!</strong>&nbsp;If you're a regular reader, you're thinking of skipping this section. You may even be thinking to yourself, yes of course I will drop a note on Twitter when I am giving a talk on all things ELK. That's awesome, because we'd like to showcase every meetup, conference presentation and workshop on Elasticsearch, Logstash, and Kibana happening worldwide. And now, we've made it even easier for you to get support for your meetup!</p><p>Head on over to&nbsp; <a href="https://www.elastic.co/community/meetups">our meetups page</a>! (And we'll still totally send you swag if you're giving a talk on anything ELKy at a conference.)</p><p>Oh yeah, we're also&nbsp; <a href="https://www.elastic.co/about/careers">hiring</a>. If you'd like us to find you for employment purposes, just&nbsp;<a href="mailto:jobs@elastic.co">drop us a note</a>. We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.</p><h2>Trainings</h2><p>If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:</p><ul><li><a href="http://purchases.elastic.co/class/MtView/2015-05-12">Mountain View</a>&nbsp;- May 12, 2015&nbsp;(Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Elastic/Paris/2015-05-27">Paris</a> - May 8, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Ops/2015-05-15">Mountain View</a> - May 15, 2015 (Elasticsearch Operations - <strong>new course</strong>)</li><li><a href="http://purchases.elastic.co/class/Elastic/Denver/2015-05-18">Denver</a> - May 18, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Virginia/2015-05-19">Southwest Washington DC</a>&nbsp;-&nbsp;May 19, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Elastic/Vancouver/2015-05-20">Vancouver</a>&nbsp;-&nbsp;May 20, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Elastic/Workshop/SpringPeople/2015-05-25">Bangalore</a> - May 25, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Elastic/Paris/2015-05-26">Paris</a>&nbsp;-&nbsp;May 26, 2015&nbsp;(Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Elastic/Core/SpringPeople/2015-05-26">Bangalore</a> - May 26, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Elastic/Miami/2015-05-27">Miami</a>&nbsp;-&nbsp;May 27, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Elastic/Paris/2015-05-27">Paris</a> - May 27, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Brazil/2015-06-02">Sao Paulo</a>&nbsp;- June 2, 2015&nbsp;(Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Brazil/2015-06-03">Sao Paulo</a>&nbsp;-&nbsp;June 3, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/NewYork/2015-06-16">New York</a> - June 16, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/SanFracisco/2015-06-16">San Francisco</a> - June 16, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/NewYork/2015-06-17">New York</a> - June 17, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/SanFrancisco/2015-06-17">San Francisco</a>- June 17, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Elastic/Workshop/SpringPeople/2015-06-22">Bangalore</a> - June 22, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Amsterdam/2015-06-23">Amsterdam</a> - June 23, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Elastic/Core/SpringPeople/2015-06-23">Bangalore</a> - June 23, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Manchester/2015-06-23">Manchester</a>&nbsp;-&nbsp;June 23, 2015&nbsp;(Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Toronto/2015-06-23">Toronto</a> -&nbsp;June 23, 2015&nbsp;(Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Amsterdam/2015-06-24">Amsterdam</a> - June 24, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Manchester/2015-06-24">Manchester</a> - June 24, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Toronto/2015-06-24">Toronto</a> - June 24, 2015&nbsp;(Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Chicago/2015-06-24">Chicago</a> - June 24, 2015&nbsp;(Core Elasticsearch)</li></ul>

This Week in Elastic: Watcher, alerting for Elasticsearch, is coming soon

At Elastic, we love to solve users' problems. This solution hopefully works well for all of our community: our discussion forum, https://discuss.elastic.co

<html><head></head><body><p>As we&#x2019;ve begun to scale up development on three different open source projects, we&#x2019;ve found mailing lists&#xA0;to be a difficult solution for dealing with all of our needs for community support. We&#x2019;ve had&#xA0;multiple mailing lists going,&#xA0;one dedicated to Elasticsearch and another to Logstash, plus numerous lists for speakers of languages other than English. We&apos;ve found these many vectors of entry can be confusing for new folks trying to figure out where to go to ask a question.</p><p>We&#x2019;ve also found our lists are becoming noisy in the &#x201C;good problem to have&#x201D; kind of way. As we&#x2019;ve seen more user adoption, and across such a wide variety of use cases, we&#x2019;re getting widely different types of questions asked. It can be hard to separate signal from noise on a general list. Plus, not all of our users are comfortable with mailing lists.</p><p>At Elastic, we love to solve our users&apos; problems. We&apos;ve found a solution that we hope will work well for all of our community members, so head on over to our discussion forum at <a href="https://discuss.elastic.co">https://discuss.elastic.co</a>. Join the conversation and let us know what you think of the new communication tool.
</p><p>Lover of mailing lists, but not a fan of web forums? No problem! Create a user profile on the forum - you can also do this simply by using your GitHub, Facebook, Twitter or Google Apps credentials, plus plain old sign in with email address - and do a quick one time set up of your preferences for email notifications. Once that&apos;s done,&#xA0;you can interact with the forums solely via email <a href="https://discuss.elastic.co/t/email-only-interaction-with-the-forums/106">using these handy instructions</a>.</p><p>Use, share and enjoy. And, of course, <a href="https://discuss.elastic.co/c/meta">let us know what you think</a> and how we can improve your experience!</p></body></html>

david-pilato-elasticon-ask-me-anything-discuss-announcement-full-bleed.jpg

david-pilato-elasticon-ask-me-anything-discuss-announcement-thumbnail.jpeg

Join the Conversation: Discuss.Elastic.Co

This Week in Elastic: Shield versions 1.2.1 and 1.1.1 released

This Week in Elastic: ICYMI: Elasticsearch 1.5.2 and 1.4.5 were released

This Week in Elastic: The True Story Behind Elasticsearch Storage Requirements

This Week in Elastic: Costin Leau discusses Elasticsearch in the Apache Hadoop ecosystem

<p>Welcome to <em>This Week in Elastic</em>! In this weekly round up post, we bring you news on all aspects of the ELK Stack, including updates on Elasticsearch, Logstash and Kibana development, tips and tricks and upcoming events.</p><blockquote class="twitter-tweet" lang="en">Kibana 4 talk by <a href="https://twitter.com/zagnut">@zagnut</a> at Elasticsearch Seoul meetup. <a href="http://t.co/PJujTjwQSb">pic.twitter.com/PJujTjwQSb</a><br />— Igor Motov (@imotov) <a href="https://twitter.com/imotov/status/585391037633298434">April 7, 2015</a></blockquote><h2>Elastic News</h2><p>Useful blog posts, how tos, tutorials&nbsp;and articles&nbsp;from the wide world of all the ELK stack ecosystem.</p><ul><li>Your <a href="https://www.elastic.co/blog/2015-04-07-this-week-in-elasticsearch-and-apache-lucene">weekly deep dive</a> into our latest commits to Apache Lucene and Elasticsearch is here! Don't miss the details on our latest plugin releases, or anything else.&nbsp;</li><li>Want to catch up on the progress of Logstash 1.5 development? We've got the details in this week's <a href="https://www.elastic.co/blog/2015-04-03-this-week-in-logstash">update from the Logstash team</a> -- which also shares details on viewing Logstash-related videos from Elastic{ON}.</li><li>They're baaaaaack! If you've been enjoying <a href="https://www.elastic.co/blog/kibana-4-video-tutorials-part-1">our</a> <a href="https://www.elastic.co/blog/kibana-4-video-tutorials-part-2">previous</a>&nbsp;Kibana 4 video tutorials, you'll be happy to hear that a <a href="https://www.elastic.co/blog/kibana-4-video-tutorials-part-3">third installment</a>, covering the utilization of tile maps, is now ready for viewing.&nbsp;</li><li>If you've ever heard about codecs in Apache Lucene and wondered what they were all about, Michael&nbsp;McCandless brings you information about how codecs&nbsp;read and write index files, and much much more, in <a href="https://www.elastic.co/blog/what-is-an-apache-lucene-codec">this informative&nbsp;blog post</a>.&nbsp;</li><li>Looking for a great resource bridging the science and practice of search relevance? Check out the details on the upcoming book, <a href="https://www.elastic.co/blog/introducing-taming-search">Taming Search</a>, written by Doug Turnbull and John Berryman, in their guest blog post for Elastic.</li><li>Simon Dittlmann&nbsp;has shared not only&nbsp;his tips on setting up a centralized logging system on CoreOS with rsyslog, Elasticsearch, and Kibana, but also his code! Read all about it in <a href="http://www.itnotes.de/coreos/deployment/tools/infrastructure/aws/docker/fleet/2015/04/03/how-to-setup-rsyslog-elasticsearch-kibana-on-coreos/">his blog post</a>. &nbsp;</li></ul><blockquote class="twitter-tweet" lang="en">3 new Official Images added to <a href="https://twitter.com/docker">@docker</a> Hub in March: <a href="https://twitter.com/alpinelinux">@alpinelinux</a> <a href="https://twitter.com/logstash">@logstash</a> <a href="https://twitter.com/JuliaLanguage">@JuliaLanguage</a><br />— Docker (@docker) <a href="https://twitter.com/docker/status/583674073538043905">April 2, 2015</a></blockquote><h2>Slides and Videos</h2><p><strong>Advanced search for your legacy application, using Elasticsearch</strong></p><p><strong>Three months? How about 30 minutes...&nbsp;&nbsp;(</strong><span style="font-size: 14px;"><strong>en</strong></span><strong>&nbsp;</strong><span style="font-size: 14px;"><strong>français</strong></span><strong>)</strong></p><p><strong>Security Event Dashboard for OpenAM using ELK Stack</strong></p><p><br /></p><iframe width="560" height="315" src="//www.youtube.com/embed/TRMnXZfxhdA" frameborder="0" allowfullscreen=""></iframe><p><b>How and why I've been leveraging Kibana for DevOps&nbsp;(in Japanese)</b></p><h2>Where to find Us</h2><p>We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking about the ELK stack&nbsp;or hosting a meetup, <a href="mailto:stories@elastic.co">let us know!</a></p><blockquote class="twitter-tweet" lang="en">First Elasticsearch Meetup in Thailand on Wednesday, April 22, 2015 by Thai Programmer Network Register here ... <a href="http://t.co/OohMMA36n6">http://t.co/OohMMA36n6</a><br />— UP1 (@somkiat) <a href="https://twitter.com/somkiat/status/586108032779685888">April 9, 2015</a></blockquote><h3>Australia</h3><p>Join us for the April Sydney ELK Meetup on April 22 for some pretty amazing talks on Docker and ELK plus Elasticsearch + Cassandra + Kafka + Spark in a Windows Environment. <a href="http://www.meetup.com/Sydney-ELK-Meetup/events/220890298/">Sign up now</a> to save your seat.&nbsp;</p><h3>Belgium</h3><p>The EU edition of <a href="http://2015.hadoopsummit.org/brussels/agenda/">Hadoop Summit</a> will be held this year in Brussels April 15-16. You can join <a href="http://twitter.com/costinl">Costin Leau</a>, lead developer of Elasticsearch for Apache Hadoop, for his presentation Real Insights, Real-time: Bridging Batch and Real-time Systems for Anomaly Detection, on <a href="http://2015.hadoopsummit.org/brussels/agenda/">Wednesday at 2:30 PM in Silver Hall</a>.&nbsp;</p><h3>Canada</h3><p>Join the Elasticsearch Toronto User Group on April 21 to talk about Zero Downtime for Elasticsearch in Production. <a href="http://www.meetup.com/Elasticsearch-Toronto/events/220384588/">Register now</a> to save your seat.</p><h3>Denmark</h3><p>The Aarhus Ruby User Group will get together on April 13 to discuss log aggregation on AWS using the ELK stack. They're also on the look out for additional speakers. You can RSVP and get in touch with the organizers to volunteer to present on their <a href="http://www.meetup.com/aarhusrb/events/221207127/">meetup.com event&nbsp;page</a>.</p><h3>Germany</h3><ul><li><a href="http://twitter.com/spinscale">Alexander Reelsen</a>&nbsp;will be one of the stalwart folks organizing the Developer Camp Munich unconference, coming up on April 18-19. Make sure to say hello to Alex during the sessions, and check out the <a href="https://devcampmuc.de/">Developer Camp website</a> for information on registration and volunteer opportunities for folks who'd like to help organize.</li><li>If you will be attending <a href="https://jax.de/2015/">JAX Mainz</a>, you can get a demo of ELK from our Solutions Architect team. They'll be hanging out with our partner, Innovex, at their booth. Jax Mainz runs April 20-24.</li><li>The <a href="https://www.netways.de/en/events_trainings/osdc2015/overview/">Open Source Data Center Conference</a> returns to Berlin this year from April 21-23. Please join <a href="http://twitter.com/purbon">Pere Urbon</a>, Logstash developer, for his presentation <a href="https://www.netways.de/index.php?id=2567&L=1">Scaling Logstash: A Collection of War Stories</a>.&nbsp;</li></ul><h3>Hungary</h3><p>The Craft Conference will be held April 22-24 in Budapest. If you're heading to the conference, make sure to catch <a href="http://twitter.com/spinscale">Alexander Reelsen's</a> talk&nbsp;<a href="http://craft-conf.com/2015/#speakers/AlexanderReelsen">Speed is key: Elasticsearch Under the Hood</a>.</p><h3>Japan</h3><ul><li>Jun Ohtani will speak at JJUG CCC on April 15 on <a href="http://www.java-users.jp/?page_id=1661#F-6">Visualizing your data with Kibana 4</a>. You can find full details on the event, including registration information, on the <a href="http://www.java-users.jp/?page_id=1647">JJUG CC event site</a>.</li><li>The next Elasticsearch Study Session in Tokyo will take place on April 15 and will feature <a href="http://twitter.com/imotov">Igor Motov</a> from Elastic on Resiliency in Elasticsearch and <a href="https://twitter.com/zagnut">Nathan Zamecnik</a> on Kibana 4. <a href="https://elasticsearch.doorkeeper.jp/events/23012">Register here</a> to attend.&nbsp;</li></ul><h3>Israel</h3><p>The Fullstack Developers User Group of Tel Aviv-Yafo will meet on April 13 to for a workshop on Elasticsearch Basics and Dev Tools. Doors open at 4 PM. <a href="http://www.meetup.com/full-stack-developer-il/events/221413918/">Register now</a> to attend.</p><h3>Poland</h3><blockquote class="twitter-tweet" lang="en">Join the enterprise search meetup in Warsaw tuesday next week (14th Apr) w/ <a href="https://twitter.com/karmiq">@karmiq</a> &amp; me about all things <a href="https://twitter.com/elastic">@elastic</a> <a href="http://t.co/eru3g2oUUk">http://t.co/eru3g2oUUk</a><br />— Alexander Reelsen (@spinscale) <a href="https://twitter.com/spinscale/status/585434544813252609">April 7, 2015</a></blockquote><p>The Enterprise Search Warsaw Meetup will welcome <a href="http://twitter.com/spinscale">Alexander Reelsen</a> and <a href="http://twitter.com/karmiq">Karel Minarik</a>&nbsp;at their April 14 meeting. Please join them to hear about speed in Elasticsearch and Elasticsearch's Rails integration. <a href="http://www.meetup.com/es-warsaw/events/221545576/">Register now</a> to save your seat.</p><h3>Spain</h3><ul><li>The next&nbsp;Elasticsearch Barcelona Meetup will take place on April 13, covering <em>Elasticsearch Applied for Music Monitoring</em> and <em>Elasticsearch: More than Just Search for&nbsp;your applications</em>. <a href="http://www.meetup.com/ElasticSearch-Barcelona/events/220598910/">Register now</a> to save your seat, as this meetup is filling up fast!</li><li>If you're heading to the Spring I/O Conference in Barcelona, make sure to catch <a href="http://twitter.com/clintongormley">Clinton Gormley's</a>&nbsp;talk&nbsp;<a href="http://www.springio.net/scaling-real-time-search-and-analytics-with-elasticsearch/">Scaling Real-Time Search and Analytics with Elasticsearch</a>. The conference runs April 29-30.</li></ul><h3>United Kingdom</h3><ul><li>Heading to <a href="https://puppetlabs.com/events/puppet-camp-london-3">Puppet Camp London</a> on April 13? Great! Elastic's very own Puppet expert, <a href="https://twitter.com/richardp82">Richard Pijnenburg</a>, will be attending, also. Make sure to say hello to him in the hallway track and let him know if you're a fan of his Puppet modules for Elasticsearch and Logstash!</li><li>The London Elasticsearch Meetup will next convene on April 29 for casual drinks and Q&amp;A with the Elastic developer team. <a href="http://www.meetup.com/London-ElasticSearch-User-Group/events/221391083/">Register now</a> to let us know you're coming!</li></ul><h3>United States</h3><ul><li><strong>Featured conference</strong>: We're proud sponsors of the <a href="http://dotnetfringe.org/">.NET Fringe conference</a>, a first time event for members of the .NET open source community. Please join us on Sunday, April 12 for an afternoon workshop from <a href="https://twitter.com/gregmarzouka">Greg Marzouka</a> and <a href="http://twitter.com/mpdreamz">Martijn Laarman</a> on <em>Elasticsearch and the ELK Stack for .NET Developers</em>.&nbsp;&nbsp;The conference runs from April 12-14 in Portland, Oregon. Tickets are still available, but get a move on before they sell out!</li><li><strong>Featured meetup</strong>: On Monday, April 13, Greg and Martijn will speak at the Elasticsearch Portlandia Meetup on Elasticsearch for .NET developers. You can <a href="http://www.meetup.com/The-Portlandia-ElasticSearch-Meetup-Group/events/221442669/">register to attend live here</a> or, if you call Silicon Valley home instead of Portland, you can <a href="http://www.meetup.com/Elasticsearch-Silicon-Valley/events/221575260/">join us in our Mountain View office</a>&nbsp;for the webcast.&nbsp;</li><li>The Elasticsearch Oklahoma User Group will get together on Saturday, April 11 to talk tips and tricks for Elasticsearch. <a href="http://www.meetup.com/ElasticSearch-Oklahoma-City/events/221295912/">Register now</a> to save your seat.</li><li>If you're planning to be at <a href="http://events.linuxfoundation.org/events/apachecon-north-america/program/schedule">ApacheCon North America</a> in Austin, April 13-16, be sure to say hello to Apache Lucene developer <a href="https://twitter.com/areekzillur">Areek Zillur</a> in the hallway track!</li><li>The Austin Puppet User&nbsp;Group will get together on April 14 to discuss using Elasticsearch's Puppet Modules, including a live demo of installing the ELK stack using Puppet. <a href="http://www.meetup.com/Austin-Puppet-User-Group/events/221087871/">Register now</a> to save your seat.</li><li>In Silicon Valley? Please join us for an office warming party and meetup at Elastic's new offices on April 15. In addition to fun, food and beverage, we'll hear from Reltio's Chief Architect on <em>Illuminating Enterprise Data-driven Applications with Elasticsearch</em>. <a href="http://www.meetup.com/Elasticsearch-Silicon-Valley/events/221527752/">Register now</a> to save your seat, and please do so early. This meetup fills up lightning fast!</li><li>For our fans in New York City, our two .NET developers, Greg Marzouka and&nbsp;Martijn Laarman will be visiting on April 15 and will be presenting <em>Elasticsearch for .NET Developers</em> at the Elasticsearch New York City Meetup. <a href="http://www.meetup.com/Elasticsearch-Silicon-Valley/events/221575260/">Register now</a> to save your seat - we're expecting a long waiting list!</li><li>The Minneapolis DevOps Meetup will host a series of lightning talks at their April 20 meeting. Amongst several talks, you will hear from Vern Burton on <em>The ELK Stack: How? When? Why?</em>&nbsp;<a href="http://www.meetup.com/DevOps-Minneapolis/events/221149690/">Register now</a> to join the festivities, and don't forget they're still looking for more lightning talk submissions!</li><li>The Seattle Elasticsearch User Group will meet on April 21 to talk Docker and the ELK stack. This meetup always fills up quickly, so <a href="http://www.meetup.com/Seattle-ElasticSearch-Meetup/events/220885729/">register now</a> to save your seat.</li><li>The Human Language Technology Conference is coming up in Herndon, Virginia on April 28. If you're attending the conference, make sure to see <a href="https://twitter.com/skearns64">Steve Kearns'</a>&nbsp;presentation <a href="http://www.basistech.com/events/hltcon/hltcon-abstracts/#elasticsearch">Elasticsearch &amp; Kibana - Interactive Analysis of Multisource Data</a>.</li><li>Join the folks from LinkedIn for a meetup on their use of the ELK stack on April 29, which they are kindly hosting in their Mountain View, California office. Tal Levy of Elastic will also be speaking about Logstash and Apache Kafka. More details are forthcoming, but you should&nbsp;<a href="http://www.meetup.com/Mountain-View-ElasticSearch-Meetup/events/221528703/">register now</a> to save your space and stay tuned!</li></ul><blockquote class="twitter-tweet" lang="en">With <a href="https://twitter.com/imotov">@imotov</a> talking <a href="https://twitter.com/hashtag/elasticsearch?src=hash">#elasticsearch</a> in Seoul, Korea! <a href="https://twitter.com/elastic">@elastic</a> <a href="http://t.co/lkuPcumUxR">pic.twitter.com/lkuPcumUxR</a><br /><br />— Nathan Zamecnik (@zagnut) <a href="https://twitter.com/zagnut/status/585395813624356864">April 7, 2015</a></blockquote><h2>Where to Find You</h2><p><strong>PSST!</strong>&nbsp;If you're a regular reader, you're thinking of skipping this section. You may even be thinking to yourself, yes of course I will drop a note on Twitter when I am giving a talk on all things ELK. That's awesome, because we'd like to showcase every meetup, conference presentation and workshop on Elasticsearch, Logstash, and Kibana happening worldwide. And now, we've made it even easier for you to get support for your meetup!</p><p>Head on over to&nbsp; <a href="https://www.elastic.co/community/meetups">our meetups page</a>! (And we'll still totally send you swag if you're giving a talk on anything ELKy at a conference.)</p><p>Oh yeah, we're also&nbsp; <a href="https://www.elastic.co/about/careers">hiring</a>. If you'd like us to find you for employment purposes, just&nbsp;<a href="mailto:jobs@elastic.co">drop us a note</a>. We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.</p><h2>Trainings</h2><p>If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:</p><ul><li><a href="https://purchases.elastic.co/class/Tokyo/2015-04-13">Tokyo</a>&nbsp;– April 13, 2015 (Core Elasticsearch Training)</li><li><a href="http://purchases.elastic.co/class/Elastic/Workshop/Vienna/2015-05-12">Vienna</a> - May 12, 2015 (Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/MtView/2015-05-12">Mountain View</a>&nbsp;- May 12, 2015&nbsp;(Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/MtView/2015-05-13">Mountain View</a> - May 13, 2015 (Core Elasticsearch)</li><li><a href="http://purchases.elastic.co/class/Elastic/Denver/2015-05-18">Denver</a> - May 18, 2015 (Core Elasticsearch&nbsp;Training)</li><li><a href="http://purchases.elastic.co/class/Elastic/Hamburg/2015-05-19">Hamburg</a>&nbsp;-&nbsp;May 19, 2015 (Core Elasticsearch&nbsp;Training)</li><li><a href="http://purchases.elastic.co/class/Virginia/2015-05-19">Herndon</a>&nbsp;-&nbsp;May 19, 2015 (Core Elasticsearch&nbsp;Training)</li><li><a href="http://purchases.elastic.co/class/Elastic/Vancouver/2015-05-20">Vancouver</a>&nbsp;-&nbsp;May 20, 2015 (Core Elasticsearch&nbsp;Training)</li><li><a href="http://purchases.elastic.co/class/Elastic/Paris/2015-05-26">Paris</a>&nbsp;-&nbsp;May 26, 2015&nbsp;(Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Elastic/Miami/2015-05-27">Miami</a>&nbsp;-&nbsp;May 27, 2015 (Core Elasticsearch&nbsp;Training)</li><li><a href="http://purchases.elastic.co/class/Brazil/2015-06-02">Sao Paulo</a>&nbsp;- June 2, 2015&nbsp;(Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Brazil/2015-06-03">Sao Paulo</a>&nbsp;-&nbsp;June 3, 2015 (Core Elasticsearch&nbsp;Training)</li><li><a href="http://purchases.elastic.co/class/Manchester/2015-06-23">Manchester</a>&nbsp;-&nbsp;June 23, 2015&nbsp;(Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Toronto/2015-06-23">Toronto</a> -&nbsp;June 23, 2015&nbsp;(Hands-On Workshop)</li><li><a href="http://purchases.elastic.co/class/Manchester/2015-06-24">Manchester</a> - June 24, 2015 (Core Elasticsearch&nbsp;Training)</li><li><a href="http://purchases.elastic.co/class/Toronto/2015-06-24">Toronto</a> - June 24, 2015&nbsp;(Core Elasticsearch&nbsp;Training)</li><li><a href="http://purchases.elastic.co/class/Chicago/2015-06-24">Chicago</a> - June 24, 2015&nbsp;(Core Elasticsearch&nbsp;Training)</li></ul>

This Week in Elastic: Kibana 4 video tutorials are back

This Week in Elastic: Release of Kibana 4.0.2 & Deprecating Rivers

<html><head></head><body><p>
	Welcome to 
	<em>This Week in Elastic</em>! In this weekly round up post, we bring you news on all aspects of the ELK Stack, including updates on Elasticsearch, Logstash and Kibana development, tips and tricks and upcoming events.<br>
</p>
<h2>What Do You Think?</h2>
<p>
	We&apos;re experimenting with a new format for our blog, publishing more granular updates on each project weekly. We&apos;ll then share a wider overview of the full ELK ecosystem each Wednesday. We&apos;d love to hear what you think, and even better if you have a story to suggest that we share with the community.
</p>
<p>
	We welcome your feedback and submissions, so 
	<a href="mailto:stories@elastic.co">get in touch!</a>
</p>
<h2>Elastic News</h2>
<p>
	Useful blog posts, how tos, tutorials&#xA0;and articles&#xA0;from the wide world of all the ELK stack ecosystem.
</p>
<blockquote class="twitter-tweet" lang="en">
	<a href="https://twitter.com/hashtag/elasticsearch?src=hash">#elasticsearch</a> <a href="https://twitter.com/hashtag/puppet?src=hash">#puppet</a> module 0.9.3 released! Get your Puppetizing fix at <a href="https://t.co/WCyNxzaKlC">https://t.co/WCyNxzaKlC</a><br>
	&#x2014; Richard pijnenburg (@Richardp82) 
	<a href="https://twitter.com/Richardp82/status/580371604217180160">March 24, 2015</a>
</blockquote>
<script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script>

<ul>
	<li>We had a number of Elasticsearch-related releases over the past week, including <a href="https://www.elastic.co/blog/elasticsearch-1-5-0-released">Elasticsearch 1.5</a>, <a href="/blog/curator-3-0-released">Curator 3.0</a>, and the <a href="/blog/shield-1-1-and-1-2-released">latest release of Shield</a>. You can get a complete overview of our <a href="/blog/25-mar-2015-this-week-in-elasticsearch-and-apache-lucene">latest commits to Elasticsearch &amp; Apache Lucene</a>, plus plugin releases, in our recent update. </li>
	<li>Check out our <a href="/blog/2015-03-23-this-week-in-logstash">latest updates on Logstash</a>, including a summary of our recent engineering all hands meeting and thoughts on future features.</li>
	<li>Designing an <a href="https://www.found.no/foundation/multi-tenancy/">Elasticsearch-based multi-tenant application</a>? <a href="https://twitter.com/beiske">Konrad Beiske</a>&#xA0;shares tips and tricks to help you get started and avoid pitfalls as you scale up and out.</li>
	<li>A delightful read for any Site Reliability Engineer you know and love from the LinkedIn SRE team: Stephen Bisordi recently <a href="http://engineering.linkedin.com/metrics/scaling-collection-self-service-metrics">shared updates on Autometrics</a>, their Elasticsearch-based self-service metrics collection tool. There&apos;s some impressive numbers cataloging the growth of their data set over the past 3.5 years, plus updates on their planned future improvements to the platform. </li>
	<li>Craig Schotke from Spot Trading shared their story of moving to the ELK stack for monitoring their <a href="http://www.spottradingllc.com/snmp-monitoring-part-1-of-a-series/">hardware infrastructure</a> and <a href="http://www.spottradingllc.com/mining-business-insights-and-operational-data-via-a-unified-open-source-monitoring-platform-part-2-of-a-series/">logs</a>. Excellent two part series with lots of details on their use case and cluster specs. </li>
	<li>Using FAST and considering giving Elasticsearch a try? Seb Muller from our partner firm, Comperio Search, <a href="http://blog.comperiosearch.com/blog/2015/03/20/elasticsearch-at-posten/">shared the details</a> of migrating Norway&apos;s postal service to Elasticsearch. </li>
	<li>If you&apos;re loving Kibana 4 but would like a deeper dive on its many features, Tim and Anna Roes have written an excellent tutorial series. Check out their <a href="https://www.timroes.de/2015/02/07/kibana-4-tutorial-part-1-introduction/">Introduction</a>, which also includes links to their K4 tutorials on data discovery, building visualizations and building dashboards from several indexes. &#xA0;</li>
	<li>Stay tuned to this blog tomorrow for This Week in Kibana. This post will also be a regular weekly feature.</li>
</ul>
<blockquote class="twitter-tweet" lang="en">
	Elasticsearch &#x3092;&#x4F7F;&#x3063;&#x305F;&#x4F4D;&#x7F6E;&#x60C5;&#x5831;&#x691C;&#x7D22; 
	<a href="http://t.co/KW5bE1wV1R">http://t.co/KW5bE1wV1R</a><br>
	&#x2014; &#x30AF;&#x30C3;&#x30AF;&#x30D1;&#x30C3;&#x30C9;&#x958B;&#x767A;&#x8005;&#x30D6;&#x30ED;&#x30B0; (@cookpad_tech) 
	<a href="https://twitter.com/cookpad_tech/status/575801269337505792">March 11, 2015</a>
</blockquote>
<p>
	<br>
	<br>
</p>
<h2>Slides and Videos</h2>
<p>
	This week, we&apos;re delighted to bring you all things Elastic{ON}, our first ever user conference. You can check out all of the slides and videos we currently have online in our 
	<a href="/elasticon/2015/sf/videos">Elastic{ON} gallery</a>, but here are a few of our favorites.
</p>
<ul style="margin-bottom:30px;">
	<li style="float:left; margin-bottom:20px; list-style-type:none;"><a href="/elasticon/2015/sf/unlocking-interplanetary-datasets-with-real-time-search"><img src="https://api.contentstack.io/v2/assets/575e4d9dd8edd48f76938c37/download?uid=bltcbd57789961960b9?uid=bltcbd57789961960b9" style="float:left; padding:0 20px 0 0;"></a>Elasticsearch on Mars. Well, not actually on Mars, but analyzing all the data from the Curosity Rover.<br>
	<a href="/elasticon/2015/sf/unlocking-interplanetary-datasets-with-real-time-search">Watch Now &#xBB;</a></li>
	<li style="float:left; margin-bottom:20px; list-style-type:none;"><a href="/elasticon/2015/sf/videos/search-for-cancer-causes-and-cures"><img src="https://api.contentstack.io/v2/assets/575e4d9c0342dfd7382660d0/download?uid=bltfd0d357b96b32c1c?uid=bltfd0d357b96b32c1c" style="float:left; padding:0 20px 0 0;"></a>Learn how the Yale Department of Laboratory Medicine Uses Elasticsearch to identify novel causes of cancer, potential therapeutic targets, and determine if patients are eligible for clinical trials.<br>
	<a href="/elasticon/2015/sf/videos/search-for-cancer-causes-and-cures">Watch Now &#xBB;</a>
	</li>
	<li style="float:left; margin-bottom:20px; list-style-type:none;"><a href="/elasticon/2015/sf/scaling-elasticsearch-for-production-at-verizon"><img src="https://api.contentstack.io/v2/assets/575e4d9d3dc542cb38c0b17c/download?uid=bltd263ddfccc5d6843?uid=bltd263ddfccc5d6843" style="float:left; padding:0 20px 0 0;"></a>How Verizon Business Scaled their Elasticsearch Cluster to More than 500 Billion Documents<br>
	<a href="/elasticon/2015/sf/scaling-elasticsearch-for-production-at-verizon">Watch Now &#xBB;</a>
	</li>
</ul>
<h2>Where to find Us</h2>
<p>
	We&apos;d love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you&apos;re speaking about the ELK stack&#xA0;or hosting a meetup, 
	<a href="mailto:stories@elastic.co">let us know!</a>
</p>
<h3>Australia</h3>
<p>
	<a href="https://twitter.com/warkolm">Mark Walkolm</a>&#xA0;will visit the Australian Computer Society (ACS) meeting on April 8 to discussing Corralling Logs with ELK. You can&#xA0;<a href="http://www.acs.org.au/branches/new-south-wales/events/upcoming-events/event-details?eveID=60316591862792" target="_blank">register now</a>&#xA0;for this session, which is free of charges for ACS Members.
</p>
<h3>Belgium</h3>
<p>
	The EU edition of 
	<a href="http://2015.hadoopsummit.org/brussels/agenda/">Hadoop Summit</a> will be held this year in Brussels April 15-16. You can join <a href="http://twitter.com/costinl">Costin Leau</a>, lead developer of Elasticsearch for Apache Hadoop, for his presentation Real Insights, Real-time: Bridging Batch and Real-time Systems for Anomaly Detection, on <a href="http://2015.hadoopsummit.org/brussels/agenda/">Wednesday at 2:30 PM in Silver Hall</a>.&#xA0;
</p>
<h3>Canada</h3>
<p>
	The Elastic team is proud to sponsor 
	<a href="https://us.pycon.org/2015/">PyCon North America 2015</a>, and <a href="https://twitter.com/alexbrasetvik">Alex Brasetvik</a>, <a href="https://twitter.com/honzakral">Honza Kral</a>, <a href="https://twitter.com/katysuesays">Katy Wendland</a>,&#xA0;<a href="https://twitter.com/peterkimnyc">Peter Kim</a> and&#xA0;<a href="https://twitter.com/pierhugues">Pier-Hugues (PH)&#xA0;Pellerin</a>&#xA0;look forward to seeing you at our booth on the show floor. We&apos;ll be sharing all things ELK Stack for Pythonistas, plus telling you all about how <a href="http://found.no">Found by Elastic</a>&#xA0;can make life better for all you folks looking to integrate Elasticsearch as a Service in your Python applications.&#xA0;
</p>
<p>
	Even better, Honza, PH and Peter&#xA0;will bring you a 1.5 hour tutorial on 
	<a href="https://us.pycon.org/2015/schedule/presentation/475/">Getting Started with&#xA0;Elasticsearch</a>, including an introduction to our Python language clients. The tutorial takes place on April 8 from 9 - 10:30 AM, and PyCon runs April 8-16, including conference sessions and sprints, in Montreal.
</p>
<h3>Denmark</h3>
<p>
	The Aarhus Ruby User Group will get together on April 13 to discuss log aggregation on AWS using the ELK stack. They&apos;re also on the look out for additional speakers. You can RSVP and get in touch with the organizers to volunteer to present on their 
	<a href="http://www.meetup.com/aarhusrb/events/221207127/">meetup.com event&#xA0;page</a>.
</p>
<h3>France</h3>
<ul>
	<li>The Elasticsearch France Meetup returns to Paris on March 26.&#xA0;<a href="http://www.meetup.com/elasticsearchfr/events/220092327/" target="_blank">Registration</a>&#xA0;is currently open - sign up today!</li>
	<li>The Rennes DevOps User Group is holding their second DevOps Bootcamp this Saturday, March 28. Among the many topics on offer, they&apos;ll explore using Logstash. <a href="http://www.meetup.com/rennes-devops/events/220809907/">Registration is still open</a>&#xA0;for this all day event.</li>
	<li><a href="http://twitter.com/dadoonet">David Pilato</a>&#xA0;will speak at NoSQL Matters Paris on March 27 at 12 noon. He&apos;ll treat you to&#xA0;<a href="http://2015.nosql-matters.org/par/#agenda">Advanced Search for your Legacy Application</a>.</li>
	<li>Attending&#xA0;<a href="http://www.devoxx.fr/" target="_blank">Devoxx</a>&#xA0;France April 8-10 in Paris?&#xA0;<a href="https://twitter.com/colinsurprenant">Colin Surprenant</a>&#xA0;and David Pilato will be speaking in several sessions, including:
	<ul>
		<li><a href="http://cfp.devoxx.fr/2015/talk/OMZ-7895" target="_blank">Elles ressemblent &#xE0; quoi mes donn&#xE9;es?</a>&#xA0;- Wednesday at 5:55 PM</li>
		<li><a href="http://cfp.devoxx.fr/2015/talk/HEX-3717" target="_blank">Birds of a Feather</a>: Elasticsearch, Logstash and Kibana - Thursday at 7:30 PM</li>
		<li><a href="http://cfp.devoxx.fr/2015/talk/GBV-9698">Logstash Workshop</a>&#xA0;- Friday at 2:25 PM</li>
	</ul>
	</li>
</ul>
<h3>Germany</h3>
<ul>
	<li>The Elasticsearch Berlin User Group will get together on March 31. As usual, there will be separate track with&#xA0;an introduction to Elasticsearch for beginners, plus a look at using Elasticsearch in Production over two years.&#xA0;&#xA0;RSVP for the meetup&#xA0;<a href="http://www.meetup.com/Elasticsearch-UG-Berlin/events/220098003/" target="_blank">here</a>.</li>
	<li><a href="http://twitter.com/spinscale">Alexander Reelsen</a>&#xA0;will be one of the stalwart folks organizing the Developer Camp Munich unconference, coming up on April 18-19. Make sure to say hello to Alex during the sessions, and check out the <a href="https://devcampmuc.de/">Developer Camp website</a> for information on registration and volunteer opportunities for folks who&apos;d like to help organize.</li>
</ul>
<h3>Japan</h3>
<ul>
	<li>The first ever Elasticsearch Study Session in Nagoya will take place on April 4, including a general introduction to Elasticsearch &amp; the ELK stack from <a href="http://twitter.com/johtani">Jun Ohtani</a>. The Study Session will also include two&#xA0;use case presentations. <a href="https://elasticsearch.doorkeeper.jp/events/21984">Register here</a> to attend the event.</li>
	<li>Jun Ohtani will speak at JJUG CCC on April 15 on <a href="http://www.java-users.jp/?page_id=1661#F-6">Visualizing your data with Kibana 4</a>. You can find full details on the event, including registration information, on the <a href="http://www.java-users.jp/?page_id=1647">JJUG CC event site</a>.</li>
</ul>
<h3>Korea</h3>
<p>
	The Seoul community will hold their next&#xA0;Study Session on April 7, with presentations from 
	<a href="http://twitter.com/imotov">Igor Motov</a> and <a href="http://twitter.com/zagnut">Nathan Zamecnik</a> from Elastic. Check out the <a href="http://okky.kr/article/274542">full details</a> on the event from Jong Min Kim, the Seoul Study Session organizer. Jong Min&apos;s post includes details on registering for the Study Session.
</p>
<h3>India</h3>
<p>
	The TechNxt user group in Pune will meet on April 4 to talk Application Containers and Measuring Systems Performance. You guessed it, the systems performance talk will feature details on the ELK stack. 
	<a href="http://www.meetup.com/TechNxt/events/220726607/">Register now</a> to attend.
</p>
<h3>Italy</h3>
<p>
	<a href="http://twitter.com/dadoonet">David Pilato</a>&#xA0;will take the stage at Code Motion Rome to discuss&#xA0;<a href="http://rome2015.codemotionworld.com/conference/23/" target="_blank">Advanced Search for Your Legacy Application</a>. David speaks at 3 PM on March 28, and the conference runs the 25-28.
</p>
<h3>Spain</h3>
<p>
	The next&#xA0;Elasticsearch Barcelona Meetup will take place on April 13, covering Elasticsearch Applied for Music Monitoring and Elasticsearch: More than Just Search for&#xA0;your applications. 
	<a href="http://www.meetup.com/ElasticSearch-Barcelona/events/220598910/">Register now</a> to save your seat, as this meetup is filling up fast!
</p>
<h3>Sweden</h3>
<p>
	The first ever Elastic Meetup in G&#xF6;teborg will convene on April 9 at 5 PM to discuss getting started with Elasticsearch and the ELK Stack.&#xA0;
	<a href="http://www.meetup.com/Elastic-Goteborg/events/221207423/" target="_blank">Register now</a>&#xA0;to save your seat.
</p>
<h3>United States</h3>
<ul>
	<li>The Los Angeles Elasticsearch User Group will be meeting <strong>tonight</strong>&#xA0;to chat all about what&apos;s new with Elastic and the ELK Stack. Rashid Khan, creator of Kibana, and Kurt Hurtado, Logstash developer, will be doing informal Q&amp;A with the audience on March 25.&#xA0;<a href="http://www.meetup.com/Elasticsearch-Los-Angeles/events/221196010/" target="_blank">Register now</a>&#xA0;to save your seat.</li>
	<li>The South Shore .NET Users Group in Plymouth, MA will convene on March 26, covering a recap from the Elastic{ON} conference.&#xA0;<a href="http://www.meetup.com/South-Shore-NET-Users-Group/events/219496914/">Register now</a>&#xA0;to save your seat.</li>
	<li>We&apos;re hosting a casual networking meetup in Atlanta on March 31 at Front Page News on Crescent. <a href="http://www.meetup.com/Elasticsearch-User-Group-Atlanta/events/220895003/">Register now</a> to let us know you&apos;re coming!</li>
	<li>The League of Professional Systems Administrators in New Jersey will get together on April 2 in Lawrence Township to talk Systems Log Aggregation with the ELK stack. <a href="http://www.meetup.com/LOPSA-NJ/events/220979009/">Register now</a> to attend.</li>
	<li>Three of Elastic&apos;s developers who focus on Apache Lucene will be attending <a href="http://events.linuxfoundation.org/events/apachecon-north-america/program/schedule">ApacheCon North America</a> on Austin, April 13-16. Make sure to say hello to <a href="https://twitter.com/areekzillur">Areek Zillur</a>, <a href="https://twitter.com/rcmuir">Robert Muir</a> and <a href="https://twitter.com/ryanjernst">Ryan Ernst</a> in the hallway track!</li>
	<li>The Austin Puppet User&#xA0;Group will get together on April 14 to discuss using Elasticsearch&apos;s Puppet Modules, including a live demo of installing the ELK stack using Puppet. <a href="http://www.meetup.com/Austin-Puppet-User-Group/events/221087871/">Register now</a> to save your seat.</li>
</ul>
<blockquote class="twitter-tweet" lang="en">
	Ever have a need to search, analyze, or visualize your data? Join Rick Haffey to learn how 
	<a href="https://twitter.com/hashtag/Elasticsearch?src=hash">#Elasticsearch</a> can help. <a href="https://t.co/LosJuVvXTa">https://t.co/LosJuVvXTa</a><br>
	&#x2014; LCNUG (@LCNUG) 
	<a href="https://twitter.com/LCNUG/status/580519300626292736">March 24, 2015</a>
</blockquote>
<p>
	<br>
	<br>
</p>
<h2>Where to Find You</h2>
<p>
	<strong>PSST!</strong>&#xA0;If you&apos;re a regular reader, you&apos;re thinking of skipping this section. You may even be thinking to yourself, yes of course I will drop a note on Twitter when I am giving a talk on all things ELK. That&apos;s awesome, because we&apos;d like to showcase every meetup, conference presentation and workshop on Elasticsearch, Logstash, and Kibana happening worldwide. And now, we&apos;ve made it even easier for you to get support for your meetup!
</p>
<p>
	Head on over to&#xA0;
	<a href="https://www.elastic.co/community/meetups">our meetups page</a>! (And we&apos;ll still totally send you swag if you&apos;re giving a talk on anything ELKy at a conference.)
</p>
<p>
	Oh yeah, we&apos;re also&#xA0;
	<a href="https://www.elastic.co/about/careers">hiring</a>. If you&apos;d like us to find you for employment purposes, just&#xA0;<a href="mailto:jobs@elastic.co">drop us a note</a>. We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.
</p>
<h2>Trainings</h2>
<p>
	If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:
</p>
<ul>
	<li><a href="https://purchases.elastic.co/class/Paris/20150325">Paris</a>&#xA0;&#x2013; March 25, 2015 (Hands on Workshop)</li>
	<li><a href="http://purchases.elastic.co/class/elasticsearch/workshop/springpeople/2015-03-25">Bangalore</a>&#xA0;- March 25, 2015 (Hands on Workshop)</li>
	<li><a href="http://purchases.elastic.co/class/elasticsearch/core/springpeople/2015-03-26">Bangalore</a>&#xA0;- March 26, 2015 (Core Elasticsearch Training)</li>
	<li><a href="https://purchases.elastic.co/class/Paris/20150326">Paris</a>&#xA0;&#x2013; March 26, 2015 (Core Elasticsearch Training)</li>
	<li><a href="https://purchases.elastic.co/class/SanFrancisco/2015-04-07">San Francisco</a>&#xA0;&#x2013; April 7, 2015 (Core Elasticsearch Training)</li>
	<li><a href="https://purchases.elastic.co/class/Amsterdam/2015-04-14">Amsterdam</a>&#xA0;&#x2013; April 7, 2015 (Hands on Workshop)</li>
	<li><a href="https://purchases.elastic.co/class/Amsterdam/2015-04-15">Amsterdam</a>&#xA0;&#x2013; April 8, 2015 (Core Elasticsearch Training)</li>
	<li><a href="https://purchases.elastic.co/class/Seoul/2015-04-08">Seoul</a>&#xA0;&#x2013; April 8, 2015 (Core Elasticsearch Training)</li>
	<li><a href="https://purchases.elastic.co/class/Tokyo/2015-04-13">Tokyo</a>&#xA0;&#x2013; April 13, 2015 (Core Elasticsearch Training)</li>
	<li><a href="https://purchases.elastic.co/class/Austin/2015-04-23">Austin</a>&#xA0;&#x2013; April 23, 2015 (Core Elasticsearch Training)</li>
	<li><a href="https://purchases.elastic.co/class/NewYork/2015-04-28">New York</a>&#xA0;&#x2013; April 28, 2015 (Hands on Workshop)</li>
	<li><a href="https://purchases.elastic.co/class/NewYork/2015-04-29">New York</a>&#xA0;&#x2013; April 29, 2015 (Core Elasticsearch Training)</li>
	<li><a href="https://purchases.elastic.co/class/London/2015-04-29">London</a>&#xA0;&#x2013; April 29, 2015 (Hands on Workshop)</li>
</ul></body></html>

This Week in Elastic: A new format for our blog

<html><head></head><body><p>Over the last few weeks, <a href="http://twitter.com/MaineC" target="_blank">Isabel Drost-Fromm</a> published a series of blog posts about how we go about testing and quality assurance (QA) for Elasticsearch. From continuous integration to the various levels of testing to performing randomized test runs, the importance of testing and QA for Elasticsearch cannot be understated. In addition to these great articles, Isabel has also authored a <a href="/pdf/elastic-white-paper-testing-automation-elasticsearch.pdf">white paper</a> for those who&apos;d like to go even more in-depth.
</p>
<p>Get <a href="/pdf/elastic-white-paper-testing-automation-elasticsearch.pdf">reading</a>. Get testing. And enjoy!
</p></body></html>

White Paper: Testing Automation for Distributed Applications

<html><head></head><body><p>Today we&#x2019;re bringing you another story of Elasticsearch&#x2019;s use in the field: Vodori and Pepper, their home grown enterprise content management system based on Alfresco and Elasticsearch. <span style="text-decoration: underline;"><a href="http://www.vodori.com/index.html">Vodori</a></span> is a full service digital agency that blends Strategy, Design, Technology, and Product to deliver high-velocity digital global marketing solutions. They are based in Chicago, Illinois, US and they&#x2019;re the gracious hosts of the <span style="text-decoration: underline;"><a href="http://www.meetup.com/Elasticsearch-User-Group-Chicago/">Chicago Elasticsearch Meetup</a></span>. Grant Gochnauer Vice President and Co-Founder of Vodori was kind enough to share their story with us.</p>
<p>Two and a half years ago, Vodori set out to redesign its enterprise content management system, Pepper, to delivery high-velocity digital marketing to our customers. With a global scale and complex content management and distribution requirements, Pepper had to be flexible, scalable, and of course provide rich search capabilities. In 2010, we evaluated technologies that would enable us to meet our product goals and decided on two core technologies: Alfresco and Elasticsearch.</p>
<h2>Why Elasticsearch?</h2>
<p>Before diving into our decision to use Elasticsearch, it helps to understand a bit more about some of our requirements. Content management systems base their data storage on an <a href="http://alistapart.com/article/content-modelling-a-master-skill">&#x201C;object model&#x201D; or &#x201C;content model&#x201D;</a> allowing developers to specify how content types are defined and relate to one another in a system. For example, we might have an object called &#x201C;Document&#x201D; that contains properties such as &#x201C;title&#x201D; and &#x201C;filename&#x201D;. We can then extend &#x201C;Document&#x201D; with a child object in our object model &#x2013; a &#x201C;WebDocument&#x201D;. It has additional properties such as &#x201C;seoDescription&#x201D; and &#x201C;url&#x201D;. &#x201C;WebDocument&#x201D; inherits the properties defined on &#x201C;Document&#x201D;.</p>
<p>Pepper provides an object model as part of the core product but allows (and encourages) implementation teams to extend this model depending on the business requirements for the solution being delivered. Alfresco provides its own &#x201C;object model&#x201D; out of box and we extend this concept significantly for Pepper in our searching paradigm. Because project implementations can contain widely varied content types and properties, we have to support the myriad ways our customers may want to search and filter content.</p>
<p>While Alfresco implements CMIS, the industry standard for interacting with content data (creating, updating, searching), the performance did not meet Pepper&#x2019;s specific needs.  Our challenge: how to get content out of Alfresco (and by extension Pepper) in such a way that:</p>
<ul>
<li>Implementation teams could write complex queries based on client-specific business requirements</li>
<li>The Pepper team could support more robust control on our searches including sorting, faceting, and paging.</li>
<li>We could loosely couple Pepper&#x2019;s user interface and business logic to the underlying data stores (Alfresco and other relational stores), enabling more elegant evolution of each tier.</li>
</ul>
<p>Enter Elasticsearch. After evaluating a number of technology options, we selected Elasticsearch because it met our requirements and also because of the trust we had in the people and product gained through the successful use of Shay Banon&#x2019;s previous technology, Compass. By introducing this facade layer on top of Alfresco, we not only introduced a high-performance and scalable way to interact with our data, it also gave us complete control of how we search content.</p>
<h2>Let&#x2019;s look at an example: Pepper Library</h2>
<p>One of the many ways users can find and interact with content is through the Pepper Library.</p>
<p><img src="https://api.contentstack.io/v2/assets/575e4c923dc542cb38c0836a/download?uid=bltf525438b27a8acc5?uid=bltf525438b27a8acc5" data-sys-asset-uid="bltc71083056410dff0" width="421" style="max-width:100%;"></p>
<p>We allow customers to toggle a number of parametric filters as the results appear through an endless scroll list. Elasticsearch provides everything we need to implement this user experience out of the box.</p>
<p>When users toggle filters along the left panel, we simply construct the appropriate Elasticsearch query using the TermsFilter combined with a BoolFilter and Elasticsearch immediately provides the data we need.</p>
<p>We also provide for free-text search by using the search box in the upper right corner of the Library. Thankfully this is also extremely easy to do with Elasticsearch &#x2013; even when searching with the left panel filters applied. If a user types in a search query, we simply append a Match Query to the existing BoolFilter in our previous query. Elasticsearch happily returns what we asked for. We can further customize the way free-text search works by implementing custom Index and Search analyzers for each property.</p>
<h2>Modelling Content in Elasticsearch</h2>
<p>One of the challenges we faced when building our architecture was making it easy for product engineering to create content models without worrying about the details of Elasticsearch. Similarly, we wanted to provide a layer of abstraction so that our implementation teams could focus on the client solution and not on the intricacies of Elasticsearch mapping and index creation.</p>
<p>We built a framework that allows us to annotate our Java content object model in such a way that generates Elasticsearch mappings that abstract the complexities from our engineers. Our framework provides the contract between our product API&#x2019;s and Elasticsearch.</p>
<p><img src="https://api.contentstack.io/v2/assets/575e4c929e7a83165490c239/download?uid=blta7cb49036a039db1?uid=blta7cb49036a039db1" data-sys-asset-uid="bltb6f2fb1af45872e9" width="608" style="max-width:100%;"></p>
<p>In this example, &#x201C;logicalName&#x201D; represents the filename of a document exposed to the end-user which is different than the system generated physical filename we store in Alfresco. With our @ElasticSearchProperty annotation, we can specify the following:</p>
<ul>
<li>type: This is the field type that Elasticsearch uses to define how this value is indexed</li>
<li>filterable: Whether we want to be able to use Elasticsearch Filters on this value. When set to true, we don&#x2019;t want to analyze the values.</li>
<li>freeTextSearchable: Whether we want to index the analyzed value for free text searching.</li>
<li>searchAnalyzer: The custom analyzer we use when searching.</li>
<li>indexAnalyzer: The custom analyzer we use when indexing the document.</li>
</ul>
<p>When using a combination of &#x201C;filterable&#x201D; and &#x201C;freeTextSearchable&#x201D; we index the value in a &#x201C;multi_field&#x201D; type in Elasticsearch so that we can use either the analyzed or non-analyzed value. Our framework is smart enough to know when to use each.</p>
<h2>Parent child relationships</h2>
<p>The other Elasticsearch modelling feature we use heavily is Parent/Child relationships. We needed to support the ability to manage and distribute a single document in many different distribution channels (Website, iPad, social media, etc). By leveraging the Parent/Child relationships, we can index a single source document which represents the canonical binary asset and an &#x201C;index card&#x201D; that represents a pointer back to the source document with additional descriptor metadata. We can then create additional index card objects in each channel of distribution, all pointing back to the same source document. The Index card becomes the parent document with the source document becoming the child document.</p>
<p>Our framework also extends this idea with another annotation on the child source document:</p>
<p><img src="https://api.contentstack.io/v2/assets/575e4c92d8edd48f76935e90/download?uid=bltf5bfb53420b0db54?uid=bltf5bfb53420b0db54" data-sys-asset-uid="bltace37fa150ccf40c" width="780" style="max-width:100%;"></p>
<p>So why would we want to use a parent/child relationship in this way? Among other things, it allows us to write really interesting queries to fetch all content for a distribution channel while also filtering on values in the source document. Let&#x2019;s look at two examples.</p>
<p>Our customers store product marketing and catalog data within Pepper. As we mentioned earlier, this content is leveraged across many different distribution channels. As a global marketing manager, I may need to search across the entire content ecosystem and find where a particular set of documents are being used. With Elasticsearch and our Parent/Child relationships, we can execute queries that answer this question by leveraging the <i>has_child</i> filter. We simply search for child documents with a specific attribute value (e.g. productCatalogId) and Elasticsearch will return all the parent documents. Parent documents represent our &#x201C;index card&#x201D; which is specific to a distribution channel and contains specific usage metadata for that channel.</p>
<pre class="prettyprint noescape lang=json">{
   &quot;from&quot;:0,
   &quot;size&quot;:2,
   &quot;filter&quot;:{
      &quot;has_child&quot;:{
         &quot;type&quot;:&quot;com.vodori.pepper.content.model.document.PepperWebpage&quot;,
         &quot;query&quot;:{
            &quot;term&quot;:{
               &quot;productCatalogId&quot;:&quot;CRX1234A&quot;
            }
         }
      }
   }
}</pre>
<p>We also need to be able to serve and query content by distribution channel. For example, when our iPad application, Pepper Mobile, needs to fetch all the content approved for consumption from within the iOS application, we can perform this search using the <i>has_parent </i>filter. In this case, we want to find child &#x201C;source documents&#x201D; where the parent object has a specific attribute value such as a distributionChannelId value of 5.</p>
<pre class="prettyprint noescape lang=json">{
   &quot;from&quot;:0,
   &quot;size&quot;:2,
   &quot;filter&quot;:{
      &quot;has_parent&quot;:{
         &quot;type&quot;:&quot;com.vodori.pepper.content.model.descriptor.PostDescriptor&quot;,
         &quot;query&quot;:{
            &quot;term&quot;:{
               &quot;distributionChannel&quot;:&quot;5&quot;
            }
         }
      }
   }
}</pre>
<h2>What&#x2019;s next for Pepper&#x2019;s usage in Elasticsearch?</h2>
<p>We have a number of new features on our roadmap that will be based on Elasticsearch. A few examples include:</p>
<ul>
<li><span style="text-decoration: underline;">Index Aliasing</span>: This will allow us to rebuild our Elasticsearch Index while the existing index remains in service. Once complete, we can simply update the alias so that Pepper points to the newly rebuilt index. Zero downtime.</li>
<li><span style="text-decoration: underline;">Updating Mappings API</span>: Elasticsearch provides capabilities to update its mappings on the fly and so when we release new versions of Pepper, it will be smart enough to inspect the current mappings and apply an update to the index based on changes in the Product version requirements.</li>
<li><span style="text-decoration: underline;">Distributed Percolator:</span> We are very excited about this 1.0 feature because it will allow us to build a mechanism for our customers to subscribe to content updates in the system and be notified of any kind of relevant changes or compliance issues that they specify. It allows our products to be proactive about what&#x2019;s going on in the system based on arbitrary business criteria defined by our customers. Very exciting.</li>
</ul>
<h2>Wrapping up</h2>
<p>It has been really exciting to see the Elasticsearch product and team grow over the last two years. The Elasticsearch community has been fantastic and really reinforces the fact that we made the right decision on our choice to use Elasticsearch. We think that this technology will continue its growth into new industries and applications as more companies realize the power and flexibility of Elasticsearch. We are excited to be a part of that growth.</p>
<p><em>Many thanks to Grant Gochnauer for sharing their experiences with us. We are always excited to share stories from the community about how they are using Elasticsearch, so please <a href="http://twitter.com/lhawthorn">reach out to me</a> any time you&#x2019;d like to share you story.</em></p>
<p>&#xA0;</p></body></html>

Building Enterprise Content Management Systems on Elasticsearch

<p><em>We're pleased to welcome our first guest blogger, <a href="https://twitter.com/ralphm">Ralph Meijer</a>, Software Developer at Rackspace working on Mailgun. Ralph graciously offered to send us a write up of Mailgun's use of Elasticsearch after we met up at <a href="http://monitorama.eu/">Monitorama EU</a>. He has been an enthusiastic member of the Elasticsearch community for some time now and frequently attends our <a href="http://www.meetup.com/ElasticSearch-NL/">Elasticsearch Netherlands Meetup</a>.</em>
</p><p><img src="https://api.contentstack.io/v2/assets/575e4c75d8edd48f76935964/download?uid=bltbe28c350d98aaf02?uid=bltbe28c350d98aaf02" data-sys-asset-uid="bltb2405b77b1c447cb">
</p><p><img title="Needle in the Haystack" alt="Needle in the Haystack" src="https://api.contentstack.io/v2/assets/57e3ec127363b6f8741c104c/download?uid=blt3ec885c92f0314f1" data-sys-asset-uid="blt3ec885c92f0314f1" style="float:right;padding:0 0 1em 1em" height="204" width="275"><br>
	<a href="http://www.mailgun.com" target="_blank">Mailgun</a> sends and receives a lot of emails and we track and store every event that happens to every email. It adds up to billions of events per month that we need to make available to our customers, along with the ability easily parse through this data with full text search. Below is a detailed account of this challenge and how we solved it with the help of Elasticsearch and Logstash (note: we were delighted to hear about <a href="http://elasticsearch.com/blog/welcome-jordan-logstash/" target="_blank">Logstash joining Elasticsearch</a> shortly after completing this project).
</p><div id="events">
	<h2>Events</h2>
	<p>When thinking of events in Mailgun, consider the following: an incoming message is accepted or rejected, an outgoing message is delivered or rejected (spam? bounce?), the message is opened or a link in the message is clicked on, the recipient wants to unsubscribe. Beside the kind of event, each of these events can have meta data that helps our customers figure out what happened to their messages, why, and <em>when</em> it happened. This meta data includes message sender, recipient address, the message-id, SMTP error codes, link URLs, geographic information, etc.
	</p>
	<p>Every event basically comes down to a timestamp and a number of fields with their values. A typical representation of an event is an associative array, also known as dictionary or hash table.
	</p>
</div><div id="giving-access-to-events">
	<h2>Giving Access To Events</h2>
	<p>Assuming we have all these events, we need a way for our customer to use them in some way. In the Mailgun control panel, there is a logs tab, that shows log events in reverse chronological order and allows one to filter the view by domain and severity. An example log line is this:
	</p>
	<p style="text-align: center;"><img class="alignnone" title="Mailgun Logs UI" alt="Mailgun Logs UI" src="https://api.contentstack.io/v2/assets/57e3ec220ca44bba50267cf0/download?uid=bltbfbb29bc9220581c" data-sys-asset-uid="bltbfbb29bc9220581c" style="max-width:100%;">
	</p>
	<p>In this case, the event has severity 'warn' as the SMTP code signifies a temporary problem and we will reattempt delivery. There are two fields here, a timestamp and a pre-formatted, unstructured text message. In addition, we color code the event based on severity.
	</p>
	<p>Besides this web interface, we also have an API for retrieving logs programmatically, and way to set up webhooks for getting notifications when events happen. The latter notifications are already structured documents with many meta-data fields formatted as JSON. For example, the SMTP code is in its own field, as are the sender and recipient address and the message subject.
	</p>
	<p>Unfortunately, the old logs API was limited. It only returned events for message delivery (no tracking) and the contents were a pre-formatted string as shown in the control panel. There was no way to get or search in individual fields (like those in the webhook notifications) or do any kind of full text search. Similarly, the control panel lacked full text search.
	</p>
</div><div id="elasticsearch-for-storage-and-querying">
	<h2>Elasticsearch For Storage And Querying</h2>
	<p>To provide an API and access via the control panel, we wanted a new backend to improve on these shortfalls, along with several additional requirements:
	</p>
	<ul>
		<li>Allow filtering on most properties.</li>
		<li>Allow full text search.</li>
		<li>Capable of storing events for at least 30 days, but with limited retention.</li>
		<li>Easily extendable by adding nodes.</li>
		<li>Resilient to failing nodes.</li>
	</ul>
	<p>Enter Elasticsearch, a search engine that can index new documents in near real-time and make them immediately available for querying. Elasticsearch is based on Apache Lucene and allows for setting up clusters of nodes that store any number of indices in a distributed, fault-tolerant way. If a node disappears, the cluster will rebalance the (shards of) indices over the remaining nodes. You can configure how many shards make up each index and how many replicas of these shards there should be. If a master shard goes offline, one of the replicas is promoted to master and used to repopulate another node.
	</p>
	<p>Elasticsearch is document-oriented and optionally schema-free. This means that you can throw any JSON document at it and have it indexed as individual fields. This is a perfect match for our events.
	</p>
	<p>Elasticsearch also has a powerful querying/filtering interface for search in particular fields or do full text searches.
	</p>
</div><div id="getting-events-into-elasticsearch">
	<h2>Getting Events Into Elasticsearch</h2>
	<p>There are quite a bit of projects and services out there focussed on logging events. We ultimately picked <a href="/logstash">Logstash</a>, a tool for collecting, parsing, mangling and passing on logs.
	</p>
	<p>Internally, the events pushed out via our webhooks are also used in other parts of our system. We currently use Redis for this. Logstash has a Redis input plugin that retrieves log events from a Redis list. After some minor filtering, the events can then be sent out via an output plugin. A very commonly used output plugin is the Elasticsearch plugin.
	</p>
	<p>A great way to use Elasticsearch's very rich API is by setting up Kibana, a tool to "make sense of a mountain of logs". The new incarnation, <a href="http://three.kibana.org/" target="_blank">Kibana 3</a>, is fully client side JavaScript, and will be the default interface for Logstash. Unlike previous versions, it no longer depends on a Logstash-like schema, but is now usable for any Elasticsearch index.
	</p>
	<p><img title="Mailgun Kibana Dashboard" alt="Mailgun Kibana Dashboard" src="https://api.contentstack.io/v2/assets/57e3ec380ca44bba50267d14/download?uid=blt8e9a2c56f726cf68" data-sys-asset-uid="blt8e9a2c56f726cf68" style="max-width:100%;">
	</p>
</div><div id="authorization">
	<h2>Authorization</h2>
	<p>With this setup we have a full solution for putting our events in a central place and a rich API to dive into these logs. However, we don't want to expose all the events to just everybody, so we need a way to add authorization to the equation. Currently Elasticsearch nor Kibana offer authentication and authorization, so directly exposing the Elasticsearch API is not an option.
	</p>
	<p>We opted for building a two-staged proxy setup. One proxy for authorization and rate-limiting and one for translating requests to our new Events API to Elasticsearch requests. We have made the former proxy available under the Apache 2.0 license as <a href="https://github.com/mailgun/vulcan" target="_blank">vulcan</a> on Github. We also ported our old, now deprecated, Logs API to use Elasticsearch instead of the old system.
	</p>
</div><div id="index-design">
	<h2>Index Design</h2>
	<p>There are various ways to select how you organize your indexes, based on the amount of documents (per time period) and querying patterns.
	</p>
	<p>Logstash, by default, creates a new index per day, which then contains all events received from its inputs. You can configure it to change the rotation period (if any) or make separate indices per user or other property (like the type of event).
	</p>
	<p>In our case we have about 1500 events per second and we wanted retention periods configurable per account. The options:
	</p>
	<ul>
		<li>One big index.</li>
		<li>One index per day.</li>
		<li>One index per user account.</li>
	</ul>
	<p>Of course each these could be split up further by sharding on, for example, the event type, if needed.
	</p>
	<p>One way to manage retention in Elasticsearch is by setting <a href="/guide/reference/mapping/ttl-field/" target="_blank">TTLs</a> on each document. Elasticsearch will then periodically delete expired documents in bulk. This makes it easy to do custom retention per account. However, it turns out to be a rather expensive (repetitive) operation.
	</p>
	<p>A much lighter weight operation is dropping entire indices. This is why Logstash defaults to daily indices. When an index falls outside your retention period, you can simply drop the index from a cron job.
	</p>
	<p>However, this doesn't give us custom retentions. Looking at our numbers, it was more efficient for us to keep all entries and limit queries to older indices from the API layer. We have many user accounts, and keeping an index per day <em>and</em> per user would become impractical. Of course, keeping all logs for all accounts in daily indices means that we are storing all of that on disk. However, if an account has a two day retention, it does limit the amount of data kept in caches and the processing power for queries that span multiple days. On top of that, for our own internal uses, it is very helpful for debugging to still have older logs and access them through Kibana.
	</p>
</div><div id="mappings">
	<h2>Mappings</h2>
	<p>To define how a (fields in a) document should be massaged, indexed and stored in an index, Elasticsearch keeps a so-called <a href="/guide/reference/mapping/" target="_blank">mapping</a>. For each field for a it defines the type, how values are analyzed and tokenized for indexing and querying, whether the value needs to be stored and some other settings. By default, the mapping is dynamic, i.e. Elasticsearch will try to determine the type of the field from the first value it gets and applies settings to it.
	</p>
	<p>If you have homogeneous data, this works quite well. However, if you have data coming from different sources, or different types of logs, in our case, it might be that the value of a field with the same name represents something else and even has a different type. As Elasticsearch will reject an attempt to index a document with a type mismatch, we need to define a mapping.
	</p>
	<p>I defined a single mapping for the types of log events exposed via our <a href="http://documentation.mailgun.com/api-events.html" target="_blank">Events API</a>. Not all types events have all fields, but a field with a particular name always means the same.
	</p>
</div><div id="analysis">
	<h2>Analysis</h2>
	<p>By default, fields are set up with the <a href="/guide/reference/index-modules/analysis/standard-analyzer/" target="_blank">standard analyzer</a> as part of the mapping. In general, this means that a string value is put in lower case and split up in words. These <em>tokens</em> are then put in the index to point to this document for this field.
	</p>
	<p>In some cases, you want something different because it has undesirable effects. For example identifiers like account IDs, e-mail addresses or URLs, the default tokenizer will split on dashes and doesn't consider the domain name as a single token (besides its constituent domain labels). If you then facet on the domain name field, you get (parts of) the individual domain labels as a facet.
	</p>
	<p>To resolve this, can set the <span class="highlighted">index</span> attribute for such fields to <span class="highlighted">not_analyzed</span>. This causes the value to be indexed as is, without any case mapping or tokenization. When applying this to the <span class="highlighted">domain.name</span> field, for example, each domain will be a facet.
	</p>
	<p>If you still want to be able to have the document to be found by parts in such fields, you can use the so-called <a href="/guide/en/elasticsearch/reference/current/mapping-core-types.html#_multi_fields_3" target="_blank">multi-field type</a>. It allows you to map the same value to different core types an/or different mapping attributes and make them available under different names. We used this technique for IP addresses, where the <em>default</em> field (e.g. <span class="highlighted">sending-ip</span>) has the type <span class="highlighted">ip</span>, and a <em>non-default</em> field (e.g. <span class="highlighted">sending-ip.untouched</span>) is configured to be <span class="highlighted">not_analyzed</span> and of type <span class="highlighted">string</span>. You can then use the default field for doing range queries on IP addresses, and the <span class="highlighted">.untouched</span> one for faceting.
	</p>
	<p>For most fields, though, we haven't done this (yet). We are looking into applying the <a href="/guide/en/elasticsearch/reference/current/analysis-pattern-capture-tokenfilter.html" target="_blank">pattern capture tokenfilter</a> to some of these fields (like e-mail addresses) in the near future, in combination with the <span class="highlighted">multi-field</span> type trick.
	</p>
</div><div id="monitoring">
	<h2>Monitoring</h2>
	<p>To see how your cluster is holding up, you really want to set up monitoring for it. Elasticsearch has a great API for retrieving the <a href="/guide/en/elasticsearch/reference/current/cluster-state.html" target="_blank">cluster state</a> and <a href="/guide/en/elasticsearch/reference/current/cluster-nodes-stats.html" target="_blank">node statistics</a>. We have set up <a href="http://graphite.wikidot.com/" target="_blank">Graphite</a> for storing these metrics and made comprehensive dashboards out of them, below is one of the panels.
	</p>
	<p style="text-align: center;"><img class="alignnone no-shadow no-border" title="Mailgun Graphite Monitoring Dashboard" alt="Mailgun Graphite Monitoring Dashboard" src="https://api.contentstack.io/v2/assets/57e3ec4dccfdfe6162a655df/download?uid=blt885060276ad65188" data-sys-asset-uid="blt885060276ad65188" style="max-width:100%;">
	</p>
	<p>To poll these metrics and pass them to Graphite I created <a href="http://github.com/mochi/vor" target="_blank">Vör</a>, which was made available under the MIT/X11 License by <a href="http://www.mochimedia.com/" target="_blank">Mochi Media</a>. An additional poller for keeping track of the size of Redis lists is also in the works.
	</p>
	<p>On top of that, we count things like messages received, clicks, opens, API requests and time such requests via <a href="https://github.com/etsy/statsd" target="_blank">StatsD</a> and have added these metrics to our Graphite dashboards, too.
	</p>
	<p style="text-align: center;"><img class="alignnone no-shadow no-border" title="Mailgun Graphite Event Dashboard" alt="Mailgun Graphite Event Dashboard" src="https://api.contentstack.io/v2/assets/57e3ec60189bd8fd612df82d/download?uid=bltb4adabc6b18fdd64" data-sys-asset-uid="bltb4adabc6b18fdd64" style="max-width:100%;">
	</p>
	<p>This is a great way to see what is happening. Graphite has a large number of functions you can apply to (series of) metrics before graphing or returning the result as a JSON document. For example, it is really easy to create a graph that shows the number of API requests against server load or indexing speed.
	</p>
</div><div id="current-setup">
	<h2>Current Setup</h2>
	<p>Some statistics on our set up:
	</p>
	<ul>
		<li>Between 40 to 60 million log events per day.</li>
		<li>30 days log retention.</li>
		<li>30 indexes</li>
		<li>5 shards per index</li>
		<li>1 replica for each shard</li>
		<li>2x 50 to 80 GB per index (times two because of the replica shards).</li>
	</ul>
	<p>For this we have set currently set up a total of 9 Rackspace cloud servers, in various configurations, based on our findings with the dashboards above:
	</p>
	<ul>
		<li>6x 30GB RAM, 8 vCPUs, 1TB disk: Elasticsearch data nodes.</li>
		<li>2x 8GB RAM, 4 vCPUs: Elasticsearch proxy node, Logstash, Graphite and StatsD.</li>
		<li>2x 4GB RAM, 2 core: Elasticsearch proxy node, Vulcan and API server</li>
	</ul>
	<p>Most of these will eventually move to dedicated boxes, while retaining the ability to create additional cloud servers to scale up.
	</p>
	<p>The Elasticsearch data nodes are configured to take and lock 16GB of RAM for the JVM heap. The rest of the configuration is pretty standard. We set the maximum fieldcache size to 40% of the heap to make sure the cluster does not break down on expensive and ill-advised faceting and sorting on fields with many different values. We also increased a few <a href="/guide/en/elasticsearch/reference/current/cluster-update-settings.html" target="_blank">cluster wide settings</a> for speeding up recovery and rebalancing. Especially the <span class="highlighted">indices.recovery.max_bytes_per_sec</span> is set very low by default, for the amount of documents we store.
	</p>
</div><div id="conclusion">
	<h2>Conclusion</h2>
	<p>We are very happy with Elasticsearch for keeping our log events and have been getting very positive feedback from our users about the new Events API and the improvements in the Logs tab of the Control panel. Digging through our logs by searching in almost any field is a significant improvement for them and ElasticSearch makes this fast and painless. Also, this tool chain of Logstash, Elasticsearch and Kibana is also ideal for internal application logging.
	</p>
	<p>If you want to know more about how we use Elasticsearch or questions about the Events API, be sure to drop us a line!  You can <a href="http://blog.mailgun.com/post/new-events-api-detailed-email-tracking-and-search/" target="_blank">read more details about the Events API itself</a> on the Mailgun blog.
	</p>
	<p>Happy logging, er, emailing!
	</p>
</div><p><em>Many thanks to Ralph for sharing his experiences with the wider community. If you're interested in being a guest blogger or have other Elasticsearch related write ups you'd like us to help you reshare, <a href="https://twitter.com/lhawthorn">let me know</a>.</em>
</p>

Using ElasticSearch and Logstash to Serve Billions of Searchable Events for Customers

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

Leslie Hawthorn

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_self">Terms of Service</a>, and you agree to receive the <a href="/legal/privacy-statement#how-we-use-the-information" target="_self">selected communications</a> at the contact details you provided above. See <a href="/legal/privacy-statement/" target="_self">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Communication Preferences - ONLY for /communication-preferences

<p>En confirmant votre envoi, vous reconnaissez que vous avez lu et que vous acceptez nos <a href="/fr/legal/elastic-cloud-account-terms" target="_self">Conditions générales d'utilisation</a>, et vous acceptez de recevoir les <a href="/fr/legal/privacy-statement#how-we-use-the-information" target="_self">types de notification sélectionnés</a> à l'adresse que vous avez fournie plus haut. Pour en savoir plus ou pour revenir sur votre décision (possible à tout moment), consultez la <a href="/fr/legal/privacy-statement/" target="_self">Déclaration de confidentialité d'Elastic</a>.</p>

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p><p>By submitting this form, I agree to Elastic sharing my contact information with Google Cloud. I also agree to Elastic giving my information to the vendor for the purpose of shipping my food package.</p>

GDPR Text + Google Cloud

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/serverless-preview-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Serverless GDPR Text

<p>En confirmant votre envoi, vous reconnaissez que vous avez lu et que vous acceptez nos <a href="/fr/legal/serverless-preview-terms" target="_self">Conditions générales d'utilisation</a>, et vous permettez à Elastic de <a href="/fr/legal/privacy-statement#how-we-use-the-information" target="_self">vous contacter</a> à propos de produits et services similaires en utilisant les informations que vous avez fournies plus haut. Pour en savoir plus ou pour revenir sur votre décision (possible à tout moment), consultez la <a href="/fr/legal/privacy-statement/" target="_self">Déclaration de confidentialité d'Elastic</a>.</p>

<p>By submitting you agree to <a href="/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud Terms of Service</a>. Your personal data will be processed in accordance with <a href="/legal/privacy-statement">Elastic's Privacy Statement</a>.</p>

Cloud Trial GDPR Text

<p>En envoyant votre demande d'inscription, vous acceptez les <a href="/fr/agreements/global/cloud-services-monthly" rel="nofollow">conditions générales d'utilisation d'Elastic Cloud</a>. Vos données personnelles seront traitées conformément à la <a href="/fr/legal/privacy-statement">déclaration de confidentialité d'Elastic</a>.
</p>

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/terms-of-use" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Newsletter GDPR Text

<p>En confirmant votre envoi, vous reconnaissez que vous avez lu et que vous acceptez nos <a href="/fr/legal/terms-of-use" target="_self">Conditions générales d'utilisation</a>, et vous permettez à Elastic de <a href="/fr/legal/privacy-statement#how-we-use-the-information" target="_self">vous contacter</a> à propos de produits et services similaires en utilisant les informations que vous avez fournies plus haut. Pour en savoir plus ou pour revenir sur votre décision (possible à tout moment), consultez la <a href="/fr/legal/privacy-statement/" target="_self">Déclaration de confidentialité d'Elastic</a>.</p>

Explore similar demos

Overview

You'll also receive an email with related content.

You'll also receive an email with related content

Nous vous enverrons plus de contenu susceptible de vous intéresser.

Speakers

Close

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

The content on this page is not available in the selected language.

Le contenu de cette page n'est pas disponible dans la langue sélectionnée. Chez Elastic, nous mettons tout en œuvre pour vous proposer du contenu dans différentes langues. En attendant, nous vous remercions de votre patience !

See more insights

Articles by

Author

Learn more

Watch now (no PT)

Watch now

See all top stories

All (no PT translation)

Contact information

Press Release

Share on Reddit

Share this story

Share by Email

Thank you for your interest!

Contact Info

Follow us on Youtube

Load More

Share on LinkedIn

Follow us on LinkedIn

Search Integrations

All Solutions

Video for

Follow us on Twitter

See when this webinar starts in my time zone

Voir à quelle heure débute le webinar dans mon fuseau horaire

Register to Watch

Register now

See All Posts

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

Si vous ne pouvez pas vous libérer le jour J, inscrivez-vous et nous vous enverrons l'enregistrement. Vous recevrez aussi un e-mail contenant des ressources connexes.

Small image for

On-demand webinar

Featured webinar

Register to Attend

Share on Facebook

Reset all

Upcoming webinar

View more posts

Print

Hosted by

Sign In to Attend

View next

Follow us on Facebook

Share

Highlights

Read less

Thank you for registering. We will send you a confirmation email soon.

Nous vous remercions de votre inscription. Vous allez recevoir un e-mail de confirmation.

Author

Articles by Leslie Hawthorn

This Week in Elastic - Better query execution in Elasticsearch 2.0

This Week in Elastic - July 08, 2015

This Week in Elastic - July 01, 2015

This Week in Elastic - June 24, 2015

Elasticsearch: Powering Real-Time Mobile and Web Analytics for LeadBoxer and Opentracker

This Week in Elastic: Elasticsearch for Apache Hadoop 2.1 RC1 released

This Week in Elastic: First dev hangout & Logstash 1.4.3 released

This Week in Elastic: New discussion forums are live!

This Week in Elastic: Welcome Packetbeat!

Sign up for Elastic Cloud free trial