Configuration examplesedit
This section contains manifests that illustrate common use cases, and can be your starting point in exploring Logstash deployed with ECK. These manifests are self-contained and work out-of-the-box on any non-secured Kubernetes cluster. They all contain a three-node Elasticsearch cluster and a single Kibana instance.
The examples in this section are for illustration purposes only. They should not be considered production-ready.
Some of these examples use the node.store.allow_mmap: false setting on Elasticsearch which has performance implications and should be tuned for production workloads, as described in Virtual memory.
Single pipeline defined in CRDedit
kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.12/config/recipes/logstash/logstash-eck.yaml
Deploys Logstash with a single pipeline defined in the CRD
Single Pipeline defined in Secretedit
kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.12/config/recipes/logstash/logstash-pipeline-as-secret.yaml
Deploys Logstash with a single pipeline defined in a secret, referenced by a pipelineRef
Pipeline configuration in mounted volumeedit
kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.12/config/recipes/logstash/logstash-pipeline-as-volume.yaml
Deploys Logstash with a single pipeline defined in a secret, mounted as a volume, and referenced by
path.config
Writing to a custom Elasticsearch indexedit
kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.12/config/recipes/logstash/logstash-es-role.yaml
Deploys Logstash and Elasticsearch, and creates an updated version of the eck_logstash_user_role to write to a user specified index.
Creating persistent volumes for PQ and DLQedit
kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.12/config/recipes/logstash/logstash-volumes.yaml
Deploys Logstash, Beats and Elasticsearch. Logstash is configured with two pipelines:
- a main pipeline for reading from the Beats instance, which will send to the DLQ if it is unable to write to Elasticsearch
-
a second pipeline, that will read from the DLQ.
In addition, persistent queues are set up.
This example shows how to configure persistent volumes outside of the default
logstash-datapersistent volume.
Elasticsearch and Kibana Stack Monitoringedit
kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.12/config/recipes/logstash/logstash-monitored.yaml
Deploys an Elasticsearch and Kibana monitoring cluster, and a Logstash that will send its monitoring information to this cluster. You can view the stack monitoring information in the monitoring cluster’s Kibana
Multiple pipelines/multiple Elasticsearch clustersedit
kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.12/config/recipes/logstash/logstash-multi.yaml
Deploys Elasticsearch in prod and qa configurations, running in separate namespaces. Logstash is configured with a multiple pipeline→pipeline configuration, with a source pipeline routing to prod and qa pipelines.