Learn how to orchestrate and operate the Elastic Stack using Red Hat OpenShift and Elastic Cloud on Kubernetes (ECK). And then learn how to use your deployment to monitor your OpenShift logs and metrics.

<p>Managing hundreds or thousands of containers has quickly become the standard for many organizations. With infrastructures growing more complex, we want every user to find value with Elastic (regardless of where or how they operate). We created <a href="https://www.elastic.co/elastic-cloud-kubernetes">Elastic Cloud on Kubernetes (ECK) </a>— the official Operator — to simplify setup, upgrades, scaling, and more for running Elasticsearch and Kibana on Kubernetes. When you use ECK on OpenShift, Red Hat's Kubernetes container platform, you can fully orchestrate and manage a stateful Elastic Stack deployment in minutes, as well as take advantage of ECK's built-in best practices well beyond day 1.
</p><p>Of course, with diverse infrastructures, monitoring becomes both more challenging and more critical. With <a href="https://www.elastic.co/observability">Elastic Observability</a>, you can store and analyze the logs and metrics from your OpenShift ecosystem alongside your other infrastructure monitoring data for a unified view. In this blog post we're going to see how easy it is to get the Elastic Stack up and running on OpenShift with ECK, as well as how to start monitoring OpenShift.
</p><p>You will learn to:
</p><ol>
	<li>Set up Elasticsearch and Kibana on OpenShift using ECK.</li>
	<li>Ship OpenShift logs and metrics to Elasticsearch using Beats (also on ECK).</li>
</ol><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt3b44b1229005fccf/5fb534c26f82405d9a4ad566/blog-openshift-monitoring-eck.jpg" data-sys-asset-uid="blt3b44b1229005fccf" alt="Monitoring OpenShift with Elastic Observability" style="display: block; margin: auto;">
</p> <strong><h2>Prerequisites</h2></strong><p>To run the following instructions, you must first:
</p><ol>
	<li>Be a <code>system:admin</code> user or a user with the privileges to create Projects, CRDs, and RBAC resources at the cluster level.</li>
	<li>Set virtual memory settings on the Kubernetes nodes (as described in Step 1 below).</li>
</ol> <strong><h2>Part 1: Set up an Elasticsearch deployment</h2></strong><p>In this first part, we're going to walk through deploying Elasticsearch and Kibana on OpenShift using ECK. By the end of this section, you'll have a fully functional Elastic Stack deployment up and running.
</p> <strong><h3>Step 1: Increase your virtual memory (recommended)</h3></strong><p>Elasticsearch uses a mmapfs directory by default to efficiently store its indices. The default operating system limits on mmap counts is likely to be too low, which may result in out of memory exceptions. For production workloads, it is strongly recommended to increase the kernel setting <code>vm.max_map_count</code> to <code>262144</code>. The kernel setting <code>vm.max_map_count=262144</code> can be set on the host either directly or by a dedicated init container, which must be privileged.
</p><pre class="prettyprint">echo 'vm.max_map_count=262144' &gt;&gt; /etc/sysctl.conf
</pre> <strong><h3>Step 2: Create new project</h3></strong><p>Create an <code>elastic</code> OpenShift project:
</p><pre class="prettyprint">oc new-project elastic
</pre> <strong><h3>Step 3: Install the ECK operator</h3></strong><pre class="prettyprint">oc apply -f &lt;a href="https://download.elastic.co/downloads/eck/1.3.0/all-in-one.yaml"&gt;https://download.elastic.co/downloads/eck/1.3.0/all-in-one.yaml&lt;/a&gt;
</pre> <strong><h3>Step 4: Deploy Elasticsearch</h3></strong><p>Create an Elasticsearch monitoring cluster with an OpenShift route.
</p><pre class="prettyprint">cat &lt;&lt;EOF | oc apply -n elastic -f - 
# This sample sets up an Elasticsearch cluster with an OpenShift route 
apiVersion: elasticsearch.k8s.elastic.co/v1 
kind: Elasticsearch 
metadata: 
  name: monitoring 
spec: 
  version: 7.10.0 
  nodeSets: 
  - name: default 
    count: 3 
    config: 
      node.store.allow_mmap: false 
--- 
apiVersion: route.openshift.io/v1 
kind: Route 
metadata: 
  name: monitoring 
spec: 
  #host: elasticsearch.example.com # override if you don't want to use the host that is automatically generated by OpenShift (&lt;route-name&gt;[-&lt;namespace&gt;].&lt;suffix&gt;) 
  tls: 
    termination: passthrough # Elasticsearch is the TLS endpoint 
    insecureEdgeTerminationPolicy: Redirect 
  to: 
    kind: Service 
    name: monitoring-es-http 
EOF
</pre> <strong><h3>Step 5: Deploy Kibana</h3></strong><pre class="prettyprint">cat &lt;&lt;EOF | oc apply -n elastic -f - 
apiVersion: kibana.k8s.elastic.co/v1 
kind: Kibana 
metadata: 
  name: kibana 
spec: 
  version: 7.10.0 
  count: 1 
  elasticsearchRef: 
    name: "monitoring" 
  podTemplate: 
    spec: 
      containers: 
      - name: kibana 
        resources: 
          limits: 
            memory: 1Gi 
            cpu: 1 
--- 
apiVersion: v1 
kind: Route 
metadata: 
  name: kibana 
spec: 
  #host: kibana.example.com # override if you don't want to use the host that is automatically generated by OpenShift (&lt;route-name&gt;[-&lt;namespace&gt;].&lt;suffix&gt;) 
  tls: 
    termination: passthrough # Kibana is the TLS endpoint 
    insecureEdgeTerminationPolicy: Redirect 
  to: 
    kind: Service 
    name: kibana-kb-http 
EOF
</pre> <strong><h3>Step 6: Collect ES admin password</h3></strong><p>To extract the Elasticsearch <code>elastic</code> admin user, run the following command:
</p><pre class="prettyprint">PW=$(oc get secret "monitoring-es-elastic-user" -o go-template='{{.data.elastic | base64decode }}')
</pre> <strong><h2>Part 2: Monitoring OpenShift using Metricbeat and Filebeat on ECK</h2></strong><p>Now that the Elastic Stack is deployed, we can start using it to monitor OpenShift. We'll deploy <a href="https://www.elastic.co/beats/metricbeat">Metricbeat</a> and <a href="https://www.elastic.co/beats/filebeat">Filebeat</a> on ECK to ingest observability data directly from OpenShift, and then we'll be able to easily monitor system health right from <a href="https://www.elastic.co/observability">Elastic Observability</a>.
</p> <strong><h3>Step 1: Setup Beats</h3></strong><p>Download the Beats manifests:
</p><pre class="prettyprint">curl -L -O https://raw.githubusercontent.com/liladler/eck-on-openshift/main/filebeat_openshift.yaml
</pre><pre class="prettyprint">curl -L -O https://raw.githubusercontent.com/liladler/eck-on-openshift/main/metricbeat_openshift.yaml
</pre><h3>Step 2: Add permissions to the Beats service accounts</h3><p>Now that the Beats are configured, they're going to need proper access to OpenShift.
</p><pre class="prettyprint">oc adm policy add-scc-to-user privileged system:serviceaccount:openshift-monitoring:metricbeat 
oc adm policy add-scc-to-user privileged system:serviceaccount:openshift-monitoring:filebeat
</pre> <strong><h3>Step 3: Patch node-selector</h3></strong><p>Override the default node selector for the openshift-monitoring namespace (or your custom namespace) to allow for scheduling on any node:
</p><pre class="prettyprint">oc patch namespace openshift-monitoring -p \ '{"metadata": {"annotations": {"openshift.io/node-selector": ""}}}'
</pre> <strong><h3>Step 4: Enable network between projects</h3></strong><pre class="prettyprint">oc adm pod-network join-projects --to=elastic openshift-monitoring
</pre> <strong><h3>Step 5: Deploy the Beats</h3></strong><pre class="prettyprint">oc apply -f filebeat_openshift.yaml
oc apply -f metricbeat_openshift.yaml
</pre> <strong><h2>Monitor OpenShift with Elastic Observability</h2></strong><p>Now that we have Filebeat and Metricbeat shipping OpenShift logs and metrics to Elasticsearch, we can open up Kibana and start monitoring system health from Elastic Observability.
</p><div class="video embed-container" style="height: 319.725px;">
	<img style="width: 100%; margin: auto; display: block;" class="vidyard-player-embed" src="https://play.vidyard.com/Xx5Ttsr5DpBKzu9vNfbR5L.jpg" data-uuid="Xx5Ttsr5DpBKzu9vNfbR5L" data-v="4" data-type="inline" data-autoplay="1" data-loop="1" data-disable_analytics="1" data-hidden_controls="1" data-muted="1" disable_analytics="1">
</div><p><strong></strong>
</p><h2><strong>Wrapping up</strong></h2><p>Congratulations! You’ve provisioned a monitored OpenShift environment with&nbsp; maintainable and reusable manifests! If you'd like to take your system observability even further, you can <a href="https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-apm-server.html">deploy Elastic APM Server on ECK</a> and <a href="https://www.elastic.co/blog/using-elastic-apm-java-agent-on-kubernetes-k8s">bundle your application with the Elastic APM Agent</a>.&nbsp;
</p><p>With logs, metrics, and APM you can easily correlate it all into a single view allowing you to observe and troubleshoot your production environment in a single tool. Try it yourself <a href="https://elastic.co/cloud">free for 14-days on Elastic Cloud</a>. And if you'd like to learn more about Elastic Observability, we'd recommend watching the <a href="https://www.elastic.co/webinars/elastic-observability-a-single-pane-of-glass-for-all-your-observability-data">Elastic Observability: A single pane of glass for all your observability data</a> webinar.
</p>

eck-blog-banner.png

eck-blog-thumb.png

How to deploy the Elastic Stack using ECK to monitor OpenShift

Lilach Adler

Articles By

Register to Attend

Sign In to Attend

Register to Watch

You'll also receive an email with related content

Watch Now

Learn More

Contact Info

See All Posts

What to explore next...

Load more news

Load more press releases

Load More

View more learning opportunities

More

Date

Location

Agenda

Hosted by

Related workshops

Translate Content

Articles by Lilach Adler

How to deploy the Elastic Stack using ECK to monitor OpenShift

Products & Solutions

Company

Resources

Featured topics

News

Articles by Lilach Adler

How to deploy the Elastic Stack using ECK to monitor OpenShift