<p><em>This post is part of the </em><a href="/elasticon/conf/2018/sf"><em>Elastic{ON} 2018</em></a><em> blog series where we recap specific demos and related deep-dive sessions from the conference. From machine learning forecasting to APM to security analytics with Mr. Robot — check out the full list below.</em></p><p>For any monitoring solution, it is critical to be able to dynamically adjust the configuration when something happens in your Kubernetes cluster. This past year Elastic embraced a new way of building applications — in particular we’ve done a lot when it comes to autodiscover for various services running in a common platform. Watch Monica Sarbu demo how Metricbeat and Filebeat can help dynamically <a href="https://www.elastic.co/observability/kubernetes-monitoring" target="_self">monitor your Kubernetes ecosystem</a>.</p><div class="video embed-container" style="height: 478px;margin-bottom: 15px;"><img style="width: 100%;margin: auto;display: block;" class="vidyard-player-embed" src="https://play.vidyard.com/Thpfs7Lo798nnwyzqA4Zr6.jpg" data-uuid="Thpfs7Lo798nnwyzqA4Zr6" data-v="4" data-type="inline" width="100%"/></div><p>Because you don't want to change your Beats configuration each time a new container is deployed, the autodiscover feature allows you to watch for Kubernetes/Docker changes plus the application inside the container and will dynamically adapt settings for them as they occur. This is especially useful when running your infrastructure on containers.</p><p>Beginning with the release&nbsp;of Beats&nbsp;6.2, for Kubernetes&nbsp;there are&nbsp;manifest files for easier deployment of Metricbeat and Filebeat. For example, in the Metricbeat manifest file, you can configure the autodiscover feature to listen to Kubernetes events — or other containers, such as Docker if that’s what you use.</p><p>To make monitoring easier, we are working to add a new app for Kibana called Infra-UI that can quickly provide the overview of running services as well as your overall infrastructure. Again, for this demo we show how this works for Kubernetes, yet it will also monitor containers in Docker. In Kibana, there will be a Kubernetes tab where you will see all of the different pods — and for each pod, you can see an overview dashboard. The dashboard will show metrics such as CPU usage, memory usage, network traffic, and more. Beats makes it easy to collect metrics and logs from your pods with Metricbeat and Filebeat.</p><p>In our demo, we show a deployment for Nginx for two pods running on different nodes, then we scale the deployment to five Nginx pods. You’ll see data in real time on the dashboard. Each time a new container for Nginx starts, then it enables the modules to&nbsp;deploy the Nginx service inside the new pod. Additionally, you can see the collected logs from all of the instances as well as the server and access logs, and in the Metricbeat overview, you can see aggregated metrics that are coming from Nginx instances, such as active connections, request rate, growth rate, and more.</p><p>At Elastic, we try to see beyond how logging and metrics are done currently. We look at the present state of technology and try to figure out how these things will be done in the very near future. During <a href="/elasticon/conf/2018/sf/whats-brewing-in-beats">What’s Brewing in Beats</a>, Monica — along with Tudor Golubenco — talk about what’s coming up for the platform. Also, watch <a href="/elasticon/conf/2018/sf/kubernetes-docker-and-containers-at-elastic-monitoring-logging-and-more">Kubernetes, Docker, and Containers at Elastic: Monitoring, Logging, and More</a> for more futuristic inspirations or check out a real-world use case with <a href="/elasticon/conf/2018/sf/monitoring-anything-and-everything-with-beats-at-ebay">Monitoring Anything and Everything with Beats at eBay</a>.</p><p><em>See what else we announced&nbsp;</em><a href="/elasticon/conf/2018/sf"><em>during the conference</em></a><em>&nbsp;in these recaps:&nbsp;</em></p><ul><li><a href="/blog/geo-roadmap-for-elasticsearch-and-kibana-layers-geojson-vega">Geo Roadmap for Elasticsearch and Kibana: Layers, GeoJSON, Vega</a></li><li><a href="/blog/elasticon-kibana-canvas-story-elasticoffee">But First, Coffee - An Elastic{ON} Canvas Story</a><a href="/blog/elasticon-kibana-canvas-story-elasticoffee"><br /></a></li><li><a href="/blog/how-mr-robot-technical-consultant-ryan-kazanciyan-used-kibana">How Mr. Robot's Technical Consultant Used Kibana in the Show</a></li><li><a href="/blog/make-operational-data-simple-with-the-elastic-stack">Operational Data Analysis with the Elastic Stack</a></li><li><a href="/blog/elasticon-kibana-canvas-story-ama">Here to Help - An Elastic{ON} Canvas Story</a></li><li><a href="/blog/data-rollups-in-elasticsearch-you-know-for-saving-space">Data Rollups in Elasticsearch</a></li><li><a href="/blog/swiftype-app-search-using-its-intuitive-interface-to-implement-elasticsearch">App Search with Elasticsearch</a></li><li><a href="/blog/preview-of-sql-in-canvas-with-rashid-creator-of-kibana">A Preview of SQL in Canvas with Rashid, Creator of Kibana</a></li><li><a href="/blog/using-kibana-and-beats-for-security-analytics">Using Kibana and Beats for Security Analytics</a></li><li><a href="/blog/recap-elasticsearch-machine-learning-forecasting-on-time-series-data">Machine Learning Forecasting on&nbsp;Elasticsearch Data</a></li><li><a href="/blog/getting-started-with-apm-for-the-elastic-stack-a-recap">APM for the Elastic Stack - A Recap</a></li><li><a href="/blog/whats-coming-with-machine-learning-logging-and-more-on-hosted-elasticsearch">Machine Learning, Logging, and More on Hosted Elasticsearch</a></li></ul><div style="display: flex;"><div style="flex: 0 0 46%;"><ul></ul></div></div><div></div>

containers-large.jpg

containers.jpg

Monitor Kubernetes with Beats Autodiscover Feature and Elasticsearch

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

Blog Footer

Charla Myers

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content.

You'll also receive an email with related content.

You'll also receive an email with related content

Load More

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Newsletter GDPR Text

Speakers

Close

See more insights

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

Elasticsearch Platform

ELK Stack

Elastic Cloud

Observability

Security

Search

By industry

By solution

Customer spotlight

Developers

Connect

Learn

Help

See what's happening at Elastic

Author

Articles by Charla Myers

Monitor Kubernetes with Beats Autodiscover Feature and Elasticsearch

Follow us

About us

Join us

Press

Partners

Trust & Security

Investor relations

EXCELLENCE AWARDS