We’re starting our road towards the 6.0 release of the Elastic stack, with the first alpha version being released today. In the Beats team, we’ve been baking a lot of new features that we’re really excited to put out there for the first time. This blog post will cover some of the highlights.
With this Beats release we’re turning the Elastic stack into a perfect monitoring companion for Kubernetes. Both Filebeat and Metricbeat now have integrations with Kubernetes to collect pod logs and metrics, as well as enhance them with Kubernetes specific metadata.
Metricbeat gets a Kubernetes module, which works by interrogating periodically the kubelets. It gives you details about the running containers pods, like the CPU usage, memory usage, bytes exchanged over the network, about filesystem, or about the logs capacity.
The provided sample Kibana dashboard shows you at a glance the monitoring status of your Kubernetes cluster.
Filebeat, and other Beats as well, is now able to automatically attach Kubernetes specific metadata to the log lines by using a new processor. By default, the Kubernetes processor adds the following metadata:
- pod name
- pod namespace
- container name and pod labels
Depending on the Beat, the Kubernetes processor uses slightly different logic to obtain this metadata. For example in Filebeat, it takes the path of the log file, extracts the container ID from it and uses it to retrieve metadata about the pod from which the log message originated. In Metricbeat, it uses the IP and port of the endpoints to resolve which containers are part of which pods.
The Kubernetes support in Beats is a wonderful example of open source collaboration. Vijay Samuel, from Ebay, worked closely with the Elastic engineers to contribute upstream a lot of the code they had for monitoring their Kubernetes cluster with Beats. Vijay’s experience was essential to shape these features, so we’d like to thank him and Ebay on behalf of the Elastic community.
More Metricbeat modules
Besides the Kubernetes module which we already mentioned, Metricbeat 6.0.0-alpha1 comes with a bunch of new modules: memcached, perfmon, dropwizard, HTTP, vSphere, Elasticsearch, and Kibana.
An interesting one is the HTTP module. We liked the idea of the Httpbeat, a community Beat created by Christian Galsterer. It offers a generic solution for collecting data periodically from multiple HTTP endpoints and indexing the responses in Elasticsearch. Christian contributed it as a module in Metricbeat to be used for monitoring application that export their metrics over HTTP. For example, if the HTTP endpoint returns the result in a JSON format, you can configure the json metricset from this module to index the contents of each JSON object into Elasticsearch.
Perfmonbeat is another interesting community Beat, created by Martin Scholz, that we adopted in Metricbeat as part of the Windows module. It comes with the perfmon metricset to collect performance counters from Windows. It uses the PDH functions to collect performance data.
Metricbeat - Report the top N processes
One of the most appreciated Metricbeat features is that it reports CPU and memory statistics about the individual processes, which gives you a lot of insight into what is taking up the resources. However, often hundreds of processes are running on each monitored system, and that tends to consume significant amounts of disk space on the Elasticsearch side.
From the first days of Metricbeat, it was possible to filter the process statistics by the process names or other fields, which is useful when you know which of the processes you want to monitor in advance. With the 6.0.0-alpha1 release, we’re adding the option to only report the top N processes by CPU and/or memory. We found that this option strikes a good balance between the need to limit the amount of consumed disk space and the need to have granular enough metrics, so we’ve made it the default in the Metricbeat configuration.
Dashboards using the Time Series Visual Builder
The system module of Metricbeat gets two new Kibana dashboards, built using the new Time Series Visual Builder that was released with Kibana 5.4. It is the perfect match for the Metricbeat data, so we plan to recreate many of the existing Metricbeat dashboards using the new time series capabilities in Kibana.
For more details, you can watch a demo of the time series visual builder during the Kibana visualizations deep dive.
Better upgrade experience
With this release, we’re changing the default format of the Elasticsearch index names to include the Beat version. For example, instead of
filebeat-2017.05.09 we are now using
filebeat-6.0.0-alpha1-2017.05.09 which allows us to apply a different mapping template depending on the Beat version. This will make rolling upgrades easier, where you have multiple Beats versions running in parallel. This change doesn’t affect the current Kibana dashboards as the
beatname-* index pattern matches the new template.
Another improvement is that the mapping template is now generated by the Beat on startup. This makes it easier to adjust and optimize the template depending on the Elasticsearch version. It also makes it easier to add custom fields to the template, by editing a human friendly
fields.yml file instead of a large JSON document.