<img src="https://api.contentstack.io/v2/uploads/58789eed6c36611650aaa093/download?uid=blt23e25287504f727a" data-sys-asset-uid="blt23e25287504f727a" alt="mapp logo.jpg" width="305" height="88" style="float: right;width: 305;height: 88;margin-top: 0px;margin-right: 0px;margin-bottom: 0px;margin-left: 0px;text-align: right;"/>We are a small <a href="https://www.elastic.co/what-is/root-cause-analysis" target="_self">Root Cause Analysis (RCA)</a> team at <a href="https://mapp.com">MAPP</a>, and we are using Elasticsearch, Logstash and Kibana (Elastic stack) for logfile analysis. Logfiles have always been our friends, but at some point the standard Unix command line tools that we used before didn't suffice anymore, which is when&nbsp;we realized that we need a more powerful tool. We built a cluster and pumped lots of logs into it. Today, our biggest cluster holds 8.5 TB of data, giving visibility across applications and locations. The Elastic stack is changing the way we work, and it's spreading to ever more applications and teams.<h2>The vitality of the Elastic technology stack</h2>Different modules of our software produce their own logs, in different formats. Previously, we did all those dirty grep, sed, awk moves to extract certain patterns out of the raw logs and then do analysis on this. It was absolutely difficult, and it was impossible to see any trends based on different parameters. When we started using the power of the Elastic Stack, the whole log analysis became faster, more proactive, meaningful, and accurate. With Elasticsearch we can now do full text search in near real time. Using Logstash and its powerful plugins, we can deal with different types of logs from different software modules and extract valuable fields out of logs. Kibana gives us the power of creating and using dashboards as well as extracting the analysis results based on the logs. Also, passing the information from team to team became easy by&nbsp;sharing dashboards. Plugins such as Kopf, Curator and Marvel help us to manage Elasticsearch and its indices.<img src="https://api.contentstack.io/v2/assets/575e4d209985d589763782d5/download?uid=bltb489d6adac242038?uid=bltb489d6adac242038" dstyle="max-width: 100%;" style="max-width: 100%;"/>&nbsp;<h2>The Elastic technology stack from head to toe</h2>When we say that we use the Elastic Stack, this means that we are covering the full logfile analysis story, content and technology. Let's start with the application that produces the logs. We know it and the people behind it because of our <a href="https://www.elastic.co/what-is/root-cause-analysis" target="_self">Root Cause Analysis (RCA)</a> role. So we can give recommendations on the content of the logfile, and also on the structure. Next we make this visible in the Elastic stack, either by feeding it into an existing cluster or by creating a new stack. In the latter case, it's us who set up Elasticsearch, Logstash and Kibana. Then we go back to the users to ask how this works for them. Working with the Kibana dashboards, they will start to think differently and come up with new use cases. We'll keep developing the Elastic Stack, configure Logstash, do the sizing, optimize and upgrade all components. The Elastic Stack users are Third Level Support, DevOps, Developers and Product Management. And us. This is very important, there's no better way to understand your user than being one yourself.<h2>A powerful combination: Elastic Stack and RCA</h2>In this post, I'll argue that this is an amazingly powerful combination: RCA, the Elastic Stack and a full team.&nbsp;For one, as RCA, we've always been bridging teams and departments. We talk to everyone who is affected by incidents, and to everyone who knows about their causes and helps to prevent them - in short, to everyone.&nbsp;Second, we have a&nbsp;vital interest in the Elastic Stack, using it a lot, while at the same time the&nbsp;Elastic Stack&nbsp;has a&nbsp;great "Power to the User" built in philosophy that&nbsp;makes it possible to answer questions from many perspectives.&nbsp;Third - let me take one step back. Jez Humbel wrote in <a href="https://www.thoughtworks.com/de/insights/blog/there-no-such-thing-devops-team" target="_blank">this article</a>&nbsp;that "Bad behavior arises when you abstract people away from the consequences of their actions." In this spirit, we don't tell people "Don't worry, we take care of this", but aim at bringing people back in touch with the consequences of their actions. The Elastic stack is a great tool for this.&nbsp;Fourth, the Elastic Stack is flexible, so we are, too. Every component, Elasticsearch, Logstash and Kibana, adapts very well to many situations. It processes all kinds of input and lets you combine this with other data sources. For example, we extract information on human readable names from a database, build a dictionary, then use the Logstash translate plugin to&nbsp;add this information to our events.<h2>Attack of the Buzzwords</h2>Now, what happens when you have a team with the mission to identify and analyse incidents and the tool and the knowledge to spread this across the company? I'd like to show how this relates to Agile, Microservices, DevOps and Lean - sorry about this avalanche of buzzwords, but I promise that I have a reason to use them. The Elastic Stack is a natural microservice. It is small, it serves one purpose, and it is deployable independently. We are so lucky. Enjoy it, use it, and build a team that can handle it. Some members who are close to the content and interpretation, like experienced RCA people, and one or two DevOps. This team is a cross-functional group of people that have everything, and everyone, necessary to get your Logstash analysis flying - and is an agile team by this definition. There will be requests for more Elastic Stacks&nbsp;springing up all over the place. You have one in production, you need one for&nbsp;smoke tests. You have one for this&nbsp;applications, you need one for&nbsp;another one, too. You have one in EMEA, you need one in the U.S.. And so on.This in turn means that your team will be setting up Elastic Stacks in all kinds of environments. If server setups differ, they will be feeling it. If the configuration is not the same, this will show in the dashboards. If automation isn't simple, they'll be slowed down. They say that you can recognize the vanguard by the arrows sticking out of their breast - let's have a look, yes, there's a number. But this is what a vanguard is for. We push for smoother deployment processes and automation, and we cross department borders. We aren't the only ones, of course, but we have a good chance to succeed because our service is so small. When we do, this paves the road for a more agile culture.We've already dealt with most those buzzwords, except for "Lean". This comes in when you look at the Build-Measure-Learn Circle:<img src="https://api.contentstack.io/v2/assets/575e4d0512798d8e381fea0b/download?uid=blt9fe95f4ae864aa0c?uid=blt9fe95f4ae864aa0c" data-sys-asset-uid="bltbb6643e0c36f1940" alt="build_measure_learn_circle_teradata.png" style="max-width: 100%;"/>Full presentation of Eric Ries can be viewed <a href="http://www.slideshare.net/startuplessonslearned/2012-05-15-eric-ries-the-lean-startup-pwc-canada" target="_blank">here</a>. Focus on the lower left: From "Measure" to "Learn". There tend to be gaps. The Elastic Stack / RCA cannot fill all of them, but help some. The Elastic stack is good for "Measure Faster", and RCA is good for "Learn Faster”. And this is something worth working for.<h2>Outlook</h2>We are constantly working on our Elastic Stacks, presently implementing Shield and Watcher. Shield is important to enforce multitenancy, while retaining all the data in one cluster to facilitate analysis across customers. Watcher will help us to give special attention to issues, new workflows or jumpy customers. There are more adventures waiting for us with the Elastic stack.<hr/><img src="https://api.contentstack.io/v2/assets/575e4d1e58208ba076e298e9/download?uid=blte449d5ffb89ea94e?uid=blte449d5ffb89ea94e" data-sys-asset-uid="bltd7d31dd0108e9ed6" alt="Headshot_Suny.png" width="106" height="142" style="width: 106;height: 142;float: left;margin-top: 0px;margin-right: 0px;margin-bottom: 0px;margin-left: 0px;text-align: left;"/>Sun-Tsung Kim started working with Elasticsearch end of 2012 at Autoscout24 and has been working with it ever since. She has a background in statistics, programming, OS and database administration, various data stores like meetups or coursera. Additionally Sun-Tsung is interested in new ways to work and organise.

thumbnail-teradata-dashboard.png

Elastic stack for Root Cause Analysis at Mapp

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

Sun-Tsung Kim

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

The content on this page is not available in the selected language.

El contenido de esta página no está disponible en el idioma seleccionado. Elastic está trabajando para garantizar que el contenido esté disponible en varios idiomas. Gracias por tu paciencia mientras trabajamos.

By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_self">Terms of Service</a>, and you agree to receive the <a href="/legal/privacy-statement#how-we-use-the-information" target="_self">selected communications</a> at the contact details you provided above. See <a href="/legal/privacy-statement/" target="_self">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Communication Preferences - ONLY for /communication-preferences

Al enviar, confirma que ha leído y acepta nuestras <a href="/es/legal/elastic-cloud-account-terms" target="_self">Condiciones de servicio</a>, y que acepta recibir las <a href="/es/legal/privacy-statement#how-we-use-the-information" target="_self">comunicaciones seleccionadas</a> en los detalles de contacto que proporcionó anteriormente. Consulte la <a href="/es/legal/privacy-statement/" target="_self">Declaración de privacidad de Elastic</a> para obtener más información o para cancelar su suscripción en cualquier momento.

By submitting you acknowledge that you've read and agree to our <a href="/legal/serverless-preview-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Serverless GDPR Text

Al enviar, confirma que ha leído y acepta nuestras <a href="/es/legal/serverless-preview-terms" target="_self">Condiciones de servicio</a>, y que Elastic puede <a href="/es/legal/privacy-statement#how-we-use-the-information" target="_self">ponerse en contacto con usted</a> acerca de nuestros productos y servicios relacionados. Consulte la <a href="/es/legal/privacy-statement/" target="_self">Declaración de privacidad de Elastic</a> para obtener más información o para cancelar su suscripción en cualquier momento.

By submitting you agree to <a href="/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud Terms of Service</a>. Your personal data will be processed in accordance with <a href="/legal/privacy-statement">Elastic's Privacy Statement</a>.

Cloud Trial GDPR Text

Al enviar esta información, aceptas los <a href="/es/agreements/global/cloud-services-monthly" rel="nofollow">Términos de servicio de Elastic</a>. Los datos personales se procesarán de conformidad con la <a href="/es/legal/privacy-statement">declaración de privacidad de Elastic</a>.

By submitting you acknowledge that you've read and agree to our <a href="/legal/terms-of-use" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Newsletter GDPR Text

Al enviar, confirma que ha leído y acepta nuestras <a href="/es/legal/terms-of-use" target="_self">Condiciones de servicio</a>, y que Elastic puede <a href="/es/legal/privacy-statement#how-we-use-the-information" target="_self">ponerse en contacto con usted</a> acerca de nuestros productos y servicios relacionados. Consulte la <a href="/es/legal/privacy-statement/" target="_self">Declaración de privacidad de Elastic</a> para obtener más información o para cancelar su suscripción en cualquier momento.

Hosted by

All Solutions

Follow us on Youtube

Watch now (no PT)

Articles by

Share by Email

Filters

All (no PT translation)

Contact information

See when this webinar starts in my time zone

Ver cuándo comienza este webinar en mi zona horaria

Thank you for your interest!

Register to Attend

Small image for

Register to Watch

Continue reading

Featured webinar

What to explore next...

Reset all

Load more press releases

Sign In to Attend

Share

Upcoming webinar

Learn more

Search Integrations

Share on Reddit

Share on Twitter

Share on Facebook

Load more news

View more posts

On-demand webinar

Follow us on Facebook

Author

Press Release

You'll also receive an email with related content.

You'll also receive an email with related content

Además, te enviaremos contenidos relevantes

Print

Contact Info

Overview

Video for

Read less

Register now

Share this story

See All Posts

See all top stories

View next

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

¿No puedes asistir? Regístrate y te enviaremos la grabación. También recibirás un correo electrónico con contenido relacionado.

Follow us on LinkedIn

Highlights

Explore similar demos

Follow us on Twitter

Share on LinkedIn

Load More

See more insights

Speakers

Close

Thank you for registering. We will send you a confirmation email soon.

Gracias por registrarte. Te enviaremos un correo electrónico de confirmación pronto.

Author

Articles by Sun-Tsung Kim

Elastic stack for Root Cause Analysis at Mapp

Sign up for Elastic Cloud free trial