Monitoring Anything and Everything with Beats at eBay

One year ago, Vijay Samuel and the monitoring team at eBay embarked on a journey to collect logs and metrics from any application deployed in Kubernetes. And thus Collectbeat was born. It's the little Beat that could:

  • Collect metrics from any Pod that exposes metrics that abide to all supported Metricbeat modules
  • Collect logs written to stdout or files inside the Docker container
  • Append Pod metadata on every log and metric collected
  • Allow Pods to push metrics through Graphite protocol and parse them uniquely
  • Stitch stack traces for application logs

Hear about the team's journey, current path, and vision for the future. Plus, get a glimpse at how eBay is planning to tackle even more complex problems like providing QoS to important data and applying sampling and throttling — all using Beats. They're on a mission to put Beats on every host inside of eBay and collect logs and metrics from any application being deployed (inside and outside of Kubernetes).

Vijay Samuel

Sr. Software Engineer, eBay

Vijay Samuel works with eBay Platform's logging and monitoring team and is an Open source enthusiast. He and his team work on the eBay solution to ship logs and metrics from all eBay applications to it's monitoring infrastructure. Vijay is also an active contributor to the Elastic Beats project.