The Elastic Stack can be easily leveraged to monitor Go applications. It allows to do things like analyzing memory usage (memory leaks anyone?), performing long-term monitoring, tuning and capturing diagnostics. Beats in particular, the lightweight data shippers in the Stack, are designed to sit alongside the applications and are a natural fit for this kind of monitoring.Metricbeat is a Beat specialized in shipping service and/or server metrics, but also happens to be written in Go. It ships in a relatively small package (only about 10MB), and does not bring any additional dependencies with it. While its CPU overhead and memory footprint are also very light, it ships with modules for a variety of services such as:<ul><li>Apache</li><li>Couchbase</li><li>Docker</li><li>HAProxy</li><li>Kafka</li><li>MongoDB</li><li>MySQL</li><li>Nginx</li><li>PostgreSQL</li><li>Prometheus</li><li>Redis</li><li>System</li><li>ZooKeeper</li></ul>If the service you’re looking for is not listed, don’t worry: Metricbeat is extensible and <a href="https://www.elastic.co/guide/en/beats/metricbeat/current/creating-metricsets.html">you can easily implement a module</a> (and this post is proof of that!). We’d like to introduce you to the Golang Module for Metricbeat. It has merged into the master branch of <a href="https://github.com/elastic/beats/tree/master/metricbeat/module/golang">elastic/beats</a>, and is expected to be released in version 6.0.<h2>Sneak preview</h2>Here’s a Kibana Dashboard that captures the potential of the Golang Module for Metricbeat:<img asset_uid="blt4e97dc508fe57d70" src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt4e97dc508fe57d70/5bbf10620e6edaf014d240f0/kibana-dashboard-golang.jpg" alt="kibana-dashboard-golang.jpg" height="auto"/>The top panel shows a summary of the heap usage, which gives us a general idea of Go's memory usage and GC status:<ul><li>System Total Memory: the total number of bytes obtained from the system</li><li>Bytes allocated: overall bytes allocated, including memory that has since been freed</li><li>GC cycles: the number of garbage collections (GC) that occurred</li><li>GC limit: when heap memory allocation reaches this limit, the garbage collector is started. This can be different in each GC</li></ul>The middle panel has three charts in it with a breakdown of:<ul><li>heap memory</li><li>system memory</li><li>object statistics</li></ul>Heap Allocated represents both memory in use and not yet reclaimed, while Heap Inuse is obviously the size of objects that are active. Heap Idle accounts for objects that have been allocated but can be reclaimed as free memory.The two charts in the bottom panel are the GC time and GC count statistics. CPU Fraction represents the percentage of CPU time spent on GC. The greater the value, the more frequently GC occurs, in other words more time wasted on GC. The trend seems upward and pretty steep, but the range of values is between 0.41% and 0.52% so not too worrisome. Normally the GC ratio warns an optimization in the code when it goes into the single digits.<h3>Memory leaks</h3>With this information we will be able to know in much detail about Go's memory usage, distribution and GC implementation. For instance if we wanted to analyze whether there is a memory leak, we could check if the memory usage and heap memory allocation are somewhat stable. If for example GC Limit and Byte Allocation are clearly rising, it could be due to a memory leak.Historical information gives us great granularity in analyzing memory usage and GC patterns across different versions, or even commits!<h2>Great, now how do I get it?</h2><h3>expvar</h3>First things first, we need to enable Go's <a href="https://golang.org/pkg/expvar">expvar service</a>. expvar is a package in Go's standard library that exposes internal variables or statistics. Its usage is very simple, it's basically just a matter of importing the package in the application. It will automatically detect and register to an existing HTTP server:<pre class="prettyprint prettyprinted"></pre>If no HTTP server is available, the code below allows us to start one on port 6060:<pre class="prettyprint prettyprinted"></pre>The path registered by default is /debug/vars, we can access it at http://localhost:6060/debug/vars. It will expose data in JSON format, by default provides Go's <a href="https://golang.org/pkg/runtime/#MemStats">runtime.Memstats</a> but of course we can also register our own variables.<h3>Go Metricbeat!</h3>Now that we have an application with expvar, we can use Metricbeat to get this information into Elasticsearch. The installation of Metricbeat is very simple, it's just a matter of <a href="https://www.elastic.co/downloads/beats/metricbeat">downloading a package</a>. Before starting Metricbeat we just need modify the configuration file (metricbeat.yml) to enable the golang module:<pre class="prettyprint prettyprinted"></pre>The above configuration enables the Go monitoring module to poll for data every 10 seconds from heap.path. The other info that matters in the configuration file is the output:<pre class="prettyprint prettyprinted"></pre>Now assuming Elasticsearch is already running, we can finally start Metricbeat:<pre class="prettyprint prettyprinted"></pre>Now we are in business! Elasticsearch should have data, we can now start Kibana and customize the visualization for our needs. For this type of analysis <a href="https://www.elastic.co/guide/en/kibana/current/timelion.html">Timelion</a> is a particulary good fit, and to get started quickly we can leverage the existing sample <a href="https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-sample-dashboards.html">Kibana Dashobards</a> for Metricbeat.<h3>More than memory</h3>In addition to monitoring the existing memory information, through expvar we can expose some additional internal information. For example we could do something like:<pre class="prettyprint prettyprinted"></pre>It's also possible to expose Metricbeat's internal stats, so it can basically can monitor itself. It can be done via the -httpprof option:<pre class="prettyprint prettyprinted"></pre>Now we can navigate to http://127.0.0.1:6060/debug/vars and see statistics about the Elasticsearch output such as output.elasticsearch.events.acked, which represents the message sent to Elasticsearch for which Metricbeat received an ACK:<pre class="prettyprint prettyprinted"></pre>We Metricbeat exposing its own metrics, we can modify its configuration to use both sets of metrics. We can do so by adding a new expvar type:<pre class="prettyprint prettyprinted"></pre>As you can see we also used the namespace parameter and set it to metricbeat. We can now restart Metricbeat and we should start seeing the new metric.<h4>Timelion</h4>We can take the output.elasticsearch.events.acked and output.elasticsearch.events.not_acked fields and use a simple Timelion expression to plot successes and failures in messages from Metricbeat to Elasticsearch:<pre class="prettyprint prettyprinted"></pre>Here's the result in Kibana:<img asset_uid="bltb97507142fea6f86" src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltb97507142fea6f86/5bbf106541d9d6bb1426368c/graph-golang.jpg" alt="graph-golang.jpg" height="auto"/>From the chart the channel between Metricbeat and Elasticsearch appears to be stable and no messages were lost.Finally, we can also compare the Metricbeat memory stats around the same time on the dashboard:<img asset_uid="bltbd45fb9460b34183" src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltbd45fb9460b34183/5bbf106a0e6edaf014d240f6/graph-golang2.jpg" alt="graph-golang2.jpg" height="auto"/><h2>Coming up in Beats 6</h2>This module will be released with Beats 6.0, but you can start using it right now by cloning (or forking ;) the <a href="https://github.com/elastic/beats">Beats repo</a> on GitHub and building the binary yourself.Banner image credit: <a href="https://golang.org/doc/gopher">golang.org</a>

banner-five-years-animals-pig-brain-machine.jpg

thumb-five-years-animals-pig-brain-machine.jpg

Tuning Go Apps with Metricbeat

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

Giuseppe Valente

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

The content on this page is not available in the selected language.

O conteúdo desta página não está disponível no idioma selecionado. A Elastic está trabalhando para garantir que o conteúdo esteja disponível em vários idiomas. Agradecemos a compreensão.

By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_self">Terms of Service</a>, and you agree to receive the <a href="/legal/privacy-statement#how-we-use-the-information" target="_self">selected communications</a> at the contact details you provided above. See <a href="/legal/privacy-statement/" target="_self">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Communication Preferences - ONLY for /communication-preferences

Ao enviar a confirmação, você reconhece que leu e concorda com nossos <a href="/pt/legal/elastic-cloud-account-terms" target="_self">Termos de Serviço</a> e também concorda em receber <a href="/pt/legal/privacy-statement#how-we-use-the-information" target="_self">comunicações especiais</a> por meio dos dados de contato que forneceu acima. Consulte a <a href="/pt/legal/privacy-statement/" target="_self">Declaração de Privacidade da Elastic</a> para saber mais detalhes ou para cancelar a qualquer momento.

By submitting you acknowledge that you've read and agree to our <a href="/legal/serverless-preview-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Serverless GDPR Text

Ao enviar a confirmação, você reconhece que leu e concorda com nossos <a href="/pt/legal/serverless-preview-terms" target="_self">Termos de Serviço</a> e que a Elastic pode <a href="/pt/legal/privacy-statement#how-we-use-the-information" target="_self">entrar em contato com você</a> para divulgar nossos produtos e serviços relacionados usando os dados que você forneceu acima. Consulte a <a href="/pt/legal/privacy-statement/" target="_self">Declaração de Privacidade da Elastic</a> para saber mais detalhes ou para cancelar a qualquer momento.

By submitting you agree to <a href="/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud Terms of Service</a>. Your personal data will be processed in accordance with <a href="/legal/privacy-statement">Elastic's Privacy Statement</a>.

Cloud Trial GDPR Text

Ao enviar, você concorda com os <a href="/pt/agreements/global/cloud-services-monthly" rel="nofollow">Termos de Serviço do Elastic Cloud</a>. Seus dados pessoais serão tratados de acordo com a <a href="/pt/legal/privacy-statement">Declaração de Privacidade da Elastic</a>.

By submitting you acknowledge that you've read and agree to our <a href="/legal/terms-of-use" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Newsletter GDPR Text

Ao enviar a confirmação, você reconhece que leu e concorda com nossos <a href="/pt/legal/terms-of-use" target="_self">Termos de Serviço</a> e que a Elastic pode <a href="/pt/legal/privacy-statement#how-we-use-the-information" target="_self">entrar em contato com você</a> para divulgar nossos produtos e serviços relacionados usando os dados que você forneceu acima. Consulte a <a href="/pt/legal/privacy-statement/" target="_self">Declaração de Privacidade da Elastic</a> para saber mais detalhes ou para cancelar a qualquer momento.

Explore similar demos

Explore outras demonstrações semelhantes

Overview

Load More

Speakers

Close

See more insights

Author

Articles by

Learn more

Watch now (no PT)

Watch now

See all top stories

All (no PT translation)

Contact information

Press Release

Share on Reddit

Share this story

Share by Email

Thank you for your interest!

Contact Info

Follow us on Youtube

Share on LinkedIn

Follow us on LinkedIn

Search Integrations

All Solutions

Video for

Follow us on Twitter

See when this webinar starts in my time zone

Consultar a hora de início do webinar no meu fuso horário

Register to Watch

Register now

See All Posts

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

Não vai conseguir participar? Inscreva-se e nós lhe enviaremos a gravação. Você também receberá um email com conteúdo relacionado.

Author

Articles by Giuseppe Valente

Tuning Go Apps with Metricbeat

Sign up for Elastic Cloud free trial