Run Filebeat on Kubernetesedit

You can use Filebeat Docker images on Kubernetes to retrieve and ship container logs.

Running Elastic Cloud on Kubernetes? See Run Beats on ECK.

Kubernetes deploy manifestsedit

You deploy Filebeat as a DaemonSet to ensure there’s a running instance on each node of the cluster.

The Docker logs host folder (/var/lib/docker/containers) is mounted on the Filebeat container. Filebeat starts an input for the files and begins harvesting them as soon as they appear in the folder.

Everything is deployed under the kube-system namespace by default. To change the namespace, modify the manifest file.

To download the manifest file, run:

curl -L -O

If you are using Kubernetes 1.7 or earlier: Filebeat uses a hostPath volume to persist internal data. It’s located under /var/lib/filebeat-data. The manifest uses folder autocreation (DirectoryOrCreate), which was introduced in Kubernetes 1.8. You need to remove type: DirectoryOrCreate from the manifest and create the host folder yourself.


By default, Filebeat sends events to an existing Elasticsearch deployment, if present. To specify a different destination, change the following parameters in the manifest file:

  value: elasticsearch
  value: "9200"
  value: elastic
  value: changeme
Running Filebeat on master nodesedit

Kubernetes master nodes can use taints to limit the workloads that can run on them. To run Filebeat on master nodes you may need to update the Daemonset spec to include proper tolerations:

 - key:
   effect: NoSchedule
Red Hat OpenShift configurationedit

If you are using Red Hat OpenShift, you need to specify additional settings in the manifest file and enable the container to run as privileged. Filebeat needs to run as a privileged container to mount logs written on the node (hostPath) and read them.

  1. Modify the DaemonSet container spec in the manifest file:

        runAsUser: 0
        privileged: true
  2. Grant the filebeat service account access to the privileged SCC:

    oc adm policy add-scc-to-user privileged system:serviceaccount:kube-system:filebeat

    This command enables the container to be privileged as an administrator for OpenShift.

  3. Override the default node selector for the kube-system namespace (or your custom namespace) to allow for scheduling on any node:

    oc patch namespace kube-system -p \
    '{"metadata": {"annotations": {"": ""}}}'

    This command sets the node selector for the project to an empty string. If you don’t run this command, the default node selector will skip master nodes.


To deploy Filebeat to Kubernetes, run:

kubectl create -f filebeat-kubernetes.yaml

To check the status, run:

$ kubectl --namespace=kube-system get ds/filebeat

filebeat   32        32        0         32           0           <none>          1m

Log events should start flowing to Elasticsearch. The events are annotated with metadata added by the add_kubernetes_metadata processor.