Running Metricbeat on Kubernetes
editRunning Metricbeat on Kubernetes
editYou can use Metricbeat Docker images on Kubernetes to retrieve cluster metrics.
Kubernetes deploy manifests
editYou deploy Metricbeat in two different ways at the same time:
- As a DaemonSet to ensure that there’s a running instance on each node of the cluster. These instances are used to retrieve most metrics from the host, such as system metrics, Docker stats, and metrics from all the services running on top of Kubernetes.
- As a single Metricbeat instance created using a Deployment. This instance is used to retrieve metrics that are unique for the whole cluster, such as Kubernetes events or kube-state-metrics.
Everything is deployed under the kube-system
namespace by default. To change
the namespace, modify the manifest file.
To download the manifest file, run:
curl -L -O https://raw.githubusercontent.com/elastic/beats/7.6/deploy/kubernetes/metricbeat-kubernetes.yaml
If you are using Kubernetes 1.7 or earlier: Metricbeat uses a hostPath volume to persist internal data. It’s located
under /var/lib/metricbeat-data
. The manifest uses folder autocreation (DirectoryOrCreate
), which was introduced in
Kubernetes 1.8. You need to remove type: DirectoryOrCreate
from the manifest and create the host folder yourself.
Settings
editBy default, Metricbeat sends events to an existing Elasticsearch deployment, if present. To specify a different destination, change the following parameters in the manifest file:
- name: ELASTICSEARCH_HOST value: elasticsearch - name: ELASTICSEARCH_PORT value: "9200" - name: ELASTICSEARCH_USERNAME value: elastic - name: ELASTICSEARCH_PASSWORD value: changeme
Red Hat OpenShift configuration
editIf you are using Red Hat OpenShift, you need to specify additional settings in the manifest file and enable the container to run as privileged.
-
In the manifest file, edit the
metricbeat-daemonset-modules
ConfigMap, and specify the following settings underkubernetes.yml
in thedata
section:kubernetes.yml: |- - module: kubernetes metricsets: - node - system - pod - container - volume period: 10s host: ${NODE_NAME} hosts: ["https://${HOSTNAME}:10250"] bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token ssl.certificate_authorities: - /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt
-
Under the
metricbeat
ClusterRole, add the following resources:- nodes/metrics - nodes/stats
-
Grant the
metricbeat
service account access to the privileged SCC:oc adm policy add-scc-to-user privileged system:serviceaccount:kube-system:metricbeat
This command enables the container to be privileged as an administrator for OpenShift.
-
Override the default node selector for the
kube-system
namespace (or your custom namespace) to allow for scheduling on any node:oc patch namespace kube-system -p \ '{"metadata": {"annotations": {"openshift.io/node-selector": ""}}}'
This command sets the node selector for the project to an empty string. If you don’t run this command, the default node selector will skip master nodes.
Deploy
editMetricbeat gets some metrics from kube-state-metrics.
If kube-state-metrics
is not already running, deploy it now (see the
Kubernetes
deployment docs).
To deploy Metricbeat to Kubernetes, run:
kubectl create -f metricbeat-kubernetes.yaml
To check the status, run:
$ kubectl --namespace=kube-system get ds/metricbeat NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE-SELECTOR AGE metricbeat 32 32 0 32 0 <none> 1m $ kubectl --namespace=kube-system get deploy/metricbeat NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE metricbeat 1 1 1 1 1m
Metrics should start flowing to Elasticsearch.