26 December 2017 Engineering

The Elastic Advent Calendar 2017, Week 4

By Aaron AldrichMichelle CarrollMark Walkom

As we mentioned in our Week One post, the Engineering team here at Elastic wanted to celebrate the end of the 2017 Calendar via our own tech-advent series. We took a lot of inspiration from both the Elastic Stack calendar Qiita (fully in Japanese) and SysAdvent (in English) and we’d like to thank them for providing the awesome quality we have aspired to maintain.

We have summarised weeks one, two and three in previous blog posts, and this post covers the last and final (all be it short) week and also provides a summary of all the topics that were posted in the series.

Here’s all 25 topics;

Dec 1: [EN] [Elastic Stack] Upgrading to 6.0.0! by Mark Walkom

The first and best recommendation we have is to upgrade to the latest 5.6.X release, which is 5.6.4 as of writing. Once you are on 5.6.X, you can do rolling upgrades to 6.X, which is just as amazing as it sounds. 😀

Dec 2: [JP][Elasticsearch] Elasticsearch Ingest Plugin by Jun Ohtani

Ingest Nodeの機能は5で追加されました。Elasticsearchにデータを送信し、インデックスに登録する前処理として、色々な処理が可能です。例えば、Grok pluginを利用すれば、正規表現を利用して文字列を構造化したりできます。また、GeoIP pluginを利用すれば、IPアドレスを元に、緯度経度などの地理情報を追加するといったことも前処理で可能です。

Dec 3: [EN][Elastic Stack] Fighting Fires and Intrinsic Exploration—Getting Started with the Elastic Stack by Tyler Hannan

Often, if learning is driven by trying to solve a specific problem, the joy of exploration is lost in the need to put out the fire. Alternatively, if learning the new tech is driven by intrinsic interest without a focused goal, the process never feels satisfactorily “done.” We felt these pains, and examined how people were successfully learning new tech.

Dec 4: [FR][Elasticsearch] Advanced Rest Java Client by David Pilato

Aujourd’hui, nous allons vous montrer en quelques lignes comment utiliser le client Java REST d’Elasticsearch.

Dec 5: [EN][Elasticsearch] Using Painless to prepare for the Holiday Season Deals by Tal Levy

Let’s build a toy search engine for this holiday season.

Dec 6: [KR][Elasticsearch] Cross Cluster Search로 할 수 있는 것들 by Jongmin Kim

Cross Cluster Search 는 여러 클러스터의 상태를 직접 관리하지 않고 원격 클러스터 데이터를 검색하는 기능만을 제공하기 때문에 쉽고 안전하게 사용이 가능합니다.

Dec 7: [EN][Elasticsearch] Holiday Proof Your Cluster by Christopher Wurm

Before you take off for your well-deserved holidays, here are some tips on how to avoid being rudely pulled back by a pager alert.

Dec 8: [CN][Elasticsearch] Migrate your data to 6.x with reindex API and tips by Medcl Zeng

Elasticsearch 从 6.0 开始,不支持多个 type 了,所以如果你的旧索引有多个 type 的话,这样直接 reindex 会失败的。
怎么解决呢?这里有个小技巧就是可以分别按 type 来进行导入,reindex 支持查询条件来过滤要导入的数据,也就是一个 type 一个新索引,同样记得先创建好索引并进行相应的 setting 和 mapping 设置。

Dec 9: [EN][Kibana] A Competitive Edge in Sports by Jordan Sissel

I play adult league ice hockey here in California. I don’t make time to practice, so my only ice time is during games. Given this, is there something I can do between games to give my team an edge? Yes! GRAPHS! Everybody loves graphs, right?
Our hockey league posts the score sheets online, and after putting these sheets into Elasticsearch, I can build a pre-game report in Kibana

Dec 10 [DE][ElasticStack] Centralised application logs with the elastic stack by Philipp Krenn

Sobald man mehrere Server, Applikationen oder Container im Einsatz hat, besteht früher oder später der Bedarf an einer zentralen Logging-Lösung. Dabei bietet sich der Elastic Stack als flexible und gleichzeitig leistungsfähige Lösung an. Doch wie speichert man seine Applikations-Logs am einfachsten? Dafür wollen wir uns drei verschiedene Ansätze jeweils mit einer möglichen Implementierung sowie ihren jeweiligen Vor- und Nachteilen ansehen.

Dec 11: [PT-BR][Elasticsearch] Must I index parent before child? by Thiago Souza

Em um mapeamento Parent/Child, não é necessário que o documento pai tenha sido indexado primeiro para indexar o documento filho. Sendo assim é possível indexar o documento filho primeiro, antes do pai ser indexado. O único requisito é que, ao indexar o documento filho, é preciso saber o ID do documento pai de antemão.

Dec 12: [IT][ElasticStack] Tracking cryptocurrenceis: Elastic Stack FTW by Atonio Bonuccelli

Bitcoin e le “crypto-valute” in generale continuano a riscuotere molto interesse per gli addetti ai lavori ma non solo.
Oggi vedremo come poter monitorare le performance di queste valute con Elastic Stack e CoinMarketCap API
Vogliamo anche far uso di una nuova fantastica feature di Logstash che ci consente di modificare le nostre pipelines direttamente da Kibana UI senza dover riavviare Logstash dopo ogni modifica

Dec 13: [EN][Beats] Moving to the Beat: Filebeat config updates in 6.x by Aaron Aldrich

I ran into the Filebeat modules when I was re-familiarizing myself with 5.X (ironically just weeks before the 6.0 release) and loved how simple it was to set up shipping of some common logs to Elasticsearch and load basic dashboards to get started on a new project quickly.

Dec 14: [JP][Elasticsearch] How to use _analyze API for understanding text analysis by Jun Ohtani


Dec 15: [EN][Elasticsearch][Using the Reindex API to migrate from multiple _types in Elasticsearch 6.0] by Abdon Pijpelink

Every document in Elasticsearch has a type. It has long been the recommendation of Elastic to use only one document type per index, but with the release of version 6.0 this has become more than just advice. For new indices, Elasticsearch now only accepts one document type per index, as a first step to the complete removal of document types in future versions of Elasticsearch.

Dec 16: [FR][Elasticsearch] Tests de performance pour votre plugin Elasticsearch by David Pilato

Imaginons que nous ayons développé un nouveau plugin Ingest pour Elasticsearch.
Se pose la question de l’impact éventuel sur les performances lors de l’insertion de nouveaux documents.
Aujourd’hui, nous allons voir comment utiliser Rally2 pour comparer les performances d’ingestion entre un pipeline vide et un pipeline utilisant notre plugin.

Dec 17: [EN][ElasticCloudEnterprise] ECE from the trenches by Mat Schaffer

The Elastic Cloud service runs Elastic Cloud Enterprise (ECE) every day. And over time we’ve developed quite a bit of monitoring around it.
If you’re running ECE in your own environment, here are a few things that you’ll want to keep an eye on based on our own experiences.

Dec 18: [KR][Elasticsearch] Sparse Doc Value가 부리는 마법 by Jongmin Kim

6.0 에 추가된 기능 중에 Sparse Doc Value 가 있습니다. 간단하게 설명드리자면, Elasticsearch가 저장하는 Doc Value 는 기존에는 중간중간 비어있는 값이 있어도 값에 대한 저장 공간을 확보 하고 있었습니다.

Dec 19: [EN][Kibana] A Brief Tour of Kubernetes Monitoring with the Elastic Stack by Tyler Langlois

If you’re a kubernaut (and I’ll assume you are from here on out), running these DaemonSets3 on your cluster is sufficient to start collecting data. You’ll need the rest of the stack running somewhere to store documents for each Beat and use Kibana, but the rest is pretty hands-off. The aforementioned DaemonSets encapsulate the necessary settings to let Filebeat and Metricbeat pull in the requisite data.

Dec 20: [EN][Elasticsearch] Build your own Google with Elasticsearch and Gopa by Medcl Zeng

The goal here is to build a vertical search engine, which is used to search all the elasticsearch related stuff. This is a “Google” focus on Elastic topics, and we hope to use it find Elastic articles, posts or Discuss posts in one place and so we are going to index the websites elastic.co and discuss.elastic.co.

Dec 21: [EN][Elasticsearch] Knobs to turn for better indexing performance by Sherry Ger

You have a high volume logging use case and have followed the existing best practises.
What other knobs can you turn to improve indexing performance?

Dec 22: [EN][X-Pack] I lost/forgot the “elastic” user password! Am I locked out forever? by Thiago Souza

You are using X-Pack Security and you lost/forgot the “elastic” superuser password.

Don’t panic!

Dec 23: [EN][Kibana] Kibana tips for new users by Bhavya Mandya

Today I am hoping to give some tips to new users of Kibana. I started working at Elastic roughly an year ago as a QA engineer in the Kibana team. I don’t have a search background, so, it’s been a fun, adventurous and educational ride with Kibana and the rest of the Elastic Stack.

Kibana has 4 distinct parts to explore and learn for a new user — Management, Discover, Visualize, and Dashboard.

Dec 24: [DE][Elastic Stack] Daten-Exploration mit Elasticsearch und Kibana by Philipp Krenn

Welche Informationen verbergen sich eigentlich hinter einer API? In der reinen Textausgabe sieht man oft den Wald vor lauter Bäumen nicht — eine sinnvolle Visualisierung muss her. Dazu kann man entweder selbst ein Programm schreiben, um die Daten einzusammeln, aufzubereiten und anschließend grafisch darzustellen, oder … man verwendet einfach Elasticsearch und Kibana.

Dec 25: [EN][Elastic Stack] Stocking Stuffers - AKA Protips from the Source! by Mark Walkom

These Elastic Discuss forums are a fantastic source of help if you get stuck, but they also a treasure trove of hints that sometimes get lost in the volume. These are a few awesome posts we’ve been collecting for a time just like this one, so we hope you like them as the closing topic for our 2017 Advent Series!

Thank You!

We will be keeping all the of the topics available on the Elastic Discuss Forums so you can refer back to them at any time. And, as these are Discuss topics, you can also continue the conversation with the authors!

Thanks for following on through this series, we hope it’s provided some useful inspiration for your use of the Elastic Stack. If you’d like us to repeat this, or if you have ideas for next year, please let us know via Twitter (@elastic) or feel free to create a topic in our Meta category with your comments.

We hope 2017 has been a great year and we look forward to 2018 being even better!