Learn how to monitor Amazon ECS EC2 and ECS Fargate with Elastic Observability. With more teams migrating their applications and workloads to containers, the ability to monitor and track container health and usage is more critical than ever.

<p>With an increasing number of organizations migrating their applications and workloads to containers, the ability to monitor and track container health and usage is more critical than ever. Many teams are already using the Metricbeat <a href="https://www.elastic.co/guide/en/beats/metricbeat/7.11/metricbeat-module-docker.html">docker module</a> to collect&nbsp;Docker container monitoring data so it can be stored and analyzed in Elasticsearch for further analysis. But what happens when users are using Amazon Elastic Container Service (Amazon ECS)? Can Metricbeat still be used to monitor Amazon ECS? Yes!
</p>
<p>In this blog, we will lead you step by step through how to monitor Amazon ECS with <a href="https://www.elastic.co/observability">Elastic Observability</a>!
</p>
<h2>What is Amazon ECS?</h2>
<p>Amazon ECS is a highly scalable, fully managed service that makes it easy to orchestrate and provision Docker containers. It is backed by two different underlying cloud services called “launch types” that determine how the compute resources will be managed: EC2 and Fargate.
</p>
<p>When using the EC2 launch type, containers run on EC2 instances that users have to manage themselves. When using the alternate <a href="https://aws.amazon.com/fargate">Fargate</a> launch type instead, users only need to specify containers and tasks. Fargate will then take care of provisioning, configuring, and managing the backend infrastructure.
</p>
<p>Let’s look at how to monitor each launch type with <a href="https://www.elastic.co/beats/metricbeat">Metricbeat</a>!&nbsp;
</p>
<h2>How to monitor Amazon ECS EC2 containers with Elastic</h2>
<p>When deploying containers on EC2, users are responsible for deploying and managing their own Amazon ECS cluster. From a monitoring perspective, these EC2 instances are like any other normal VM, which means that users can connect to them and collect Docker statistics from the Docker daemon directly. Let’s see how we can monitor our Amazon ECS containers running on EC2 in a few simple steps:
</p>
<h4>Prerequisite: SSL key</h4>
<p>When creating your Amazon ECS cluster, make sure you attach a valid SSL key so the instances are accessible via SSH:
</p>
<p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltb7cd4e057faec42c/60108f1d0a80720ff10f39a0/blog-monitor-ecs-1.png" data-sys-asset-uid="bltb7cd4e057faec42c" alt="" style="display: block; margin: auto; width:350px;">
</p>
<h4>Step 1: Security group</h4>
<p>In the networking section make sure you select a <strong>Security group</strong> that will allow you to enable SSH access (port 22):
</p>
<p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt43de6adf82cabad4/60108f283e567f1011da3663/blog-monitor-ecs-2.png" data-sys-asset-uid="blt43de6adf82cabad4" alt="" style="display: block; margin: auto; width:350px;">
</p>
<h4>Step 2: Verify EC2 instance</h4>
<p>After you have properly launched the Amazon ECS cluster, you should be able to list your EC2 instances:
</p>
<p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltefc70c5ca53f3854/60108f4b82548c0f8284c5a6/blog-monitor-ecs-3.png" data-sys-asset-uid="bltefc70c5ca53f3854" alt="" style="display: block; margin: auto;">
</p>
<h4>Step 3: SSL connection</h4>
<p>You can connect to each one of them via SSH using the key pair you attached on step 1, for instance:
</p>
<p><pyou can="" connect="" to="" each="" one="" of="" them="" via="" ssh="" using="" the="" key="" pair="" you="" attached="" on="" step="" 0,="" for="" instance:<="" p=""></pyou>
</p>
<pre class="prettyprint">ssh -i "ec2-ecs-test.pem" ec2-user@3.125.104.129
</pre>
<h4>Step 4: Install Metricbeat</h4>
<p>Since you have access to all of the cluster’s instances, you can <a href="https://www.elastic.co/guide/en/beats/metricbeat/7.11/metricbeat-installation-configuration.html">install Metricbeat</a> on each instance and start collecting metrics from the Docker daemon:
</p>
<pre class="prettyprint">curl -L -O &lt;a href="https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-7.11.1-x86_64.rpm"&gt;https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-7.11.1-x86_64.rpm&lt;/a&gt; 
sudo rpm -vi metricbeat-7.10.1-x86_64.rpm 
sudo metricbeat modules disable system 
sudo metricbeat modules enable docker
</pre>
<p>We can now define the Elasticsearch endpoint to send metrics to — in this case an instance running on <a href="https://www.elastic.co/cloud">Elastic Cloud</a>. You can <a href="https://cloud.elastic.co/registration">spin up a free trial</a> if you'd like to follow along.
</p>
<p>Edit <code>metricbeat.yml</code> and add Elastic Cloud options:
</p>
<p>Usually you will find this file under <code>/etc/metricbeat/metricbeat.yml</code> and you can edit it with:
</p>
<pre class="prettyprint">sudo vi /etc/metricbeat/metricbeat.yml
</pre>
<pre class="prettyprint">cloud.id: "elastic-observability-deployment:ZXV…….k3MTI1ZTA1Ng==" 
cloud.auth: "elastic:2vXXXXXXXXXXXxxxxxxxxxxxV"
</pre>
<p>If you’re running a self-managed Elasticsearch cluster, you’ll need to configure the <a href="https://www.elastic.co/guide/en/beats/metricbeat/7.11/elasticsearch-output.html">Elasticsearch output</a>:
</p>
<pre class="prettyprint">output.elasticsearch: 
  hosts: ["https://myEShost:9200"] 
  username: "elastic" 
  password: "changeme"
</pre>
<p>Next, you can set up predefined dashboards, indexes, and mappings:
</p>
<pre class="prettyprint">sudo metricbeat setup -e
</pre>
<p>Start Metricbeat:
</p>
<pre class="prettyprint">./metricbeat -e -d & 
sudo service metricbeat start
</pre>
<p>You can also configure Metricbeat service to start automatically when the system boots:
</p>
<pre class="prettyprint">systemctl enable metricbeat
</pre>
<table style="background: #FFFFD2;">
<tbody>
<tr>
	<td>
		<strong>Note:</strong> This process must be performed on all EC2 instances of the cluster.
	</td>
</tr>
</tbody>
</table>
<h4>Step 5: Verify metrics collection</h4>
<p>Now that Metricbeat is running, we can verify that events are stored into our Elasticsearch cluster by looking at Docker-related metrics in Kibana’s Discover view:
</p>
<p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt3fea0d4d1db7c3a4/60108f875ad9610f6cb78f79/blog-monitor-ecs-4.png" data-sys-asset-uid="blt3fea0d4d1db7c3a4" alt="" style="display: block; margin: auto;">
</p>
<p>Now we can go on and deploy tasks on our Amazon ECS cluster, and Metricbeat will collect metrics from them. We can use these metrics to create visualizations and dashboards while leveraging additional metadata, such as <code>cloud_metadata</code> and <code>host_metadata</code>, which are added by default in order to correlate metrics per AWS accounts, regions, host identifiers, types, etc. For example, by combining <code>docker.info.containers.running</code> metric with <code>host.name</code> we can provide an overview of how many running containers we have per host over time:
</p>
<p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltc17fe409b1f6cc39/60108f983e567f1011da3667/blog-monitor-ecs-5.png" data-sys-asset-uid="bltc17fe409b1f6cc39" alt="" style="display: block; margin: auto;">
</p>
<h4>Step 6: View metrics in prebuilt Kibana dashboards</h4>
<p>Additionally, we have a nice overview of our container workloads in the default Docker dashboard bundled with Metricbeat:
</p>
<p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltb9ca733e2b873f47/60108fa90a80720ff10f39a4/blog-monitor-ecs-6.png" data-sys-asset-uid="bltb9ca733e2b873f47" alt="" style="display: block; margin: auto;">
</p>
<p>Monitoring Amazon ECS tasks only takes a few steps, and from there you can keep adding visibility into the EC2 cluster <a href="https://www.elastic.co/blog/monitoring-aws-ec2-using-metricbeat-and-the-elastic-stack">using Metricbeat</a>. Let’s now move forward and see how we can monitor our Amazon ECS tasks when running on Fargate.
</p>
<h2>How to monitor Fargate containers with Elastic</h2>
<p>Amazon ECS on Fargate exposes various metadata, network metrics, and <a href="https://docs.docker.com/engine/api/v1.30/#operation/ContainerStats">Docker stats</a> about running tasks and containers through the <a href="https://docs.aws.amazon.com/AmazonECS/latest/userguide/task-metadata-endpoint-v4-fargate.html">task metadata endpoint</a>. Metricbeat uses version 4 of the task metadata endpoint, which is available per container and enabled by default on AWS Fargate platform version 1.4.0 or later.
</p>
<p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt168e1b01bc535526/60108fbd5ad9610f6cb78f7d/blog-monitor-ecs-7.png" data-sys-asset-uid="blt168e1b01bc535526" alt="" style="display: block; margin: auto; width:650px;">
</p>
<p>To monitor a task running on Fargate you need to deploy a Metricbeat container onto the task itself by manually adding a new container definition. In the <strong>Standard</strong> section, you can give a <code>Container name</code> such as <code>metricbeat-container</code> and use the Metricbeat docker image from the Elastic Docker registry <code>docker.elastic.co/beats/metricbeat:7.11.0</code> for a specific version for Metricbeat image.
</p>
<p>Also we need to add a command for this container to set up and start Metricbeat with sending data to Elastic Cloud:
</p>
<pre class="prettyprint">./metricbeat setup -E cloud.id=$ELASTIC_CLOUD_ID -E cloud.auth=$ELASTIC_CLOUD_AUTH && ./metricbeat modules disable system && ./metricbeat modules enable awsfargate && ./metricbeat -e -E cloud.id=$ELASTIC_CLOUD_ID -E cloud.auth=$ELASTIC_CLOUD_AUTH
</pre>
<p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blta811eb2d9bbe96ea/60108fd10a80720ff10f39a8/blog-monitor-ecs-8.png" data-sys-asset-uid="blta811eb2d9bbe96ea" alt="" style="display: block; margin: auto; width:650px;">
</p>
<p>You need to add values for <code>$ELASTIC_CLOUD_ID</code> and <code>$ELASTIC_CLOUD_AUTH</code> in the <strong>Environment variables</strong> section with the actual credentials, possibly using AWS secret manager to avoid storing these values in plain text:
</p>
<p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt6c9d77e9aef54218/60108fe50cb0aa0ffcdd5b24/blog-monitor-ecs-9.png" data-sys-asset-uid="blt6c9d77e9aef54218" alt="" style="display: block; margin: auto; width:750px;">
</p>
<p>Once the new task definition revision is created, click <strong>run task</strong> under the Actions dropdown menu and Metricbeat will be added to the task when it restarts.
</p>
<p>If you have more than a few tasks to monitor, manually adding Metricbeat containers can be tedious and time-consuming. In this case we recommend using AWS CloudFormation, an AWS native way to manage resources by provisioning and updating them throughout their life cycle without manual actions. AWS CloudFormation lets you treat infrastructure as code and manage it through a template file that defines all the required resources and their dependencies.
</p>
<p>Here is a simplified example of how to add Metricbeat container into a task using CloudFormation:
</p>
<pre class="prettyprint"> TaskDefinition: 
   Type: AWS::ECS::TaskDefinition 
   Properties: 
     Family: !Ref TaskName 
     Cpu: 256 
     Memory: 512 
     NetworkMode: awsvpc 
     ExecutionRoleArn: !Ref ExecutionRole 
     ContainerDefinitions: 
       - Name: metricbeat-container 
         Image: docker.elastic.co/beats/metricbeat:7.11.0 
         Secrets: 
           - Name: ELASTIC_CLOUD_ID 
             ValueFrom: !Ref CloudIDArn 
           - Name: ELASTIC_CLOUD_AUTH 
             ValueFrom: !Ref CloudAuthArn 
                 EntryPoint: 
           - sh 
           - -c 
         Command: 
           - ./metricbeat setup -E cloud.id=$ELASTIC_CLOUD_ID -E cloud.auth=$ELASTIC_CLOUD_AUTH && ./metricbeat modules disable system && ./metricbeat modules enable awsfargate && ./metricbeat -e -E cloud.id=$ELASTIC_CLOUD_ID -E cloud.auth=$ELASTIC_CLOUD_AUTH
</pre>
<p>You can also find a <a href="https://github.com/elastic/beats/blob/master/x-pack/metricbeat/module/awsfargate/cloudformation.yml">full AWS CloudFormation template</a> with <a href="https://www.elastic.co/guide/en/beats/metricbeat/7.11/metricbeat-module-awsfargate.html">awsfargate module</a>, which creates a new cluster, a task definition with both Metricbeat container and stress test container, and starts the service. Please feel free to copy it into a <code>cloudformation.yml</code> file locally and give it a try.
</p>
<p>If you are using Elastic Cloud, it’s recommended to store cloud id and cloud auth into AWS secret manager — here is the AWS CLI example:
</p>
<p>Create secret <code>ELASTIC_CLOUD_AUTH</code>:
</p>
<pre class="prettyprint">aws --region us-east-1 secretsmanager create-secret --name ELASTIC_CLOUD_AUTH --secret-string XXX
</pre>
<p>Create secret <code>ELASTIC_CLOUD_ID</code>:
</p>
<pre class="prettyprint">aws --region us-east-1 secretsmanager create-secret --name ELASTIC_CLOUD_ID --secret-string YYY
</pre>
<p>Now with the secrets ready, let’s create an entire stack with one AWS CLI call using the CloudFormation template file <code>cloudformation.yml</code>.
</p>
<p>Create CloudFormation Stack:
</p>
<pre class="prettyprint">aws --region us-east-1 cloudformation create-stack --stack-name test-metricbeat-deployment --template-body file://./cloudformation.yml --capabilities CAPABILITY_NAMED_IAM --parameters ParameterKey=SubnetID,ParameterValue=subnet-5d89de27 ParameterKey=CloudAuthArn,ParameterValue=arn:aws:secretsmanager:us-east-1:xxx:secret:ELASTIC_CLOUD_AUTH-abc ParameterKey=CloudIDArn,ParameterValue=arn:aws:secretsmanager:us-east-1:xxx:secret:ELASTIC_CLOUD_ID-abc ParameterKey=ClusterName,ParameterValue=metricbeat-fargate ParameterKey=RoleName,ParameterValue=ecsFargateTaskExecutionRole 
ParameterKey=TaskName,ParameterValue=task-metricbeat ParameterKey=ServiceName,ParameterValue=metricbeat-service ParameterKey=LogGroupName,ParameterValue=metricbeat-fargate-log-group
</pre>
<p>Once CloudFormation shows <code>CREATE_COMPLETE</code> status for your stack, you should be able to see monitoring metrics sent into Elasticsearch every 10 seconds. <code>awsfargate</code> module collects runtime metrics from each container including CPU, memory, network, and diskIO information. There is also an out of the box dashboard you can use to display all monitoring data from <code>awsfargate</code> in Kibana:
</p>
<p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt191e80097164e839/601090110839e910126d76b1/blog-monitor-ecs-10.jpg" data-sys-asset-uid="blt191e80097164e839" alt="" style="display: block; margin: auto;">
</p>
<p>Once you are done experimenting, you can easily delete the whole stack using AWS CLI:
</p>
<pre class="prettyprint">aws --region us-east-1 cloudformation delete-stack --stack-name test-metricbeat-deployment
</pre>
<h2>What’s next</h2>Are you looking for a monitoring solution for your Amazon ECS container deployments? With Metricbeat <code>docker</code> and <code>awsfargate</code> modules, users can easily monitor Amazon ECS containers and ship metrics to <a href="https://www.elastic.co/elasticsearch/service">Elasticsearch Service on Elastic Cloud</a>. If you have any questions or feedback, please feel free to post on our <a href="https://discuss.elastic.co/c/elastic-stack/beats">Beats forum</a>!

blog-banner-container-barge.jpg

blog-thumb-container-barge.jpg

How to monitor Amazon ECS with Elastic Observability

In this blog post, we'll take a look at monitoring Istio and its microservices with Elastic Observability.

<p>Istio is an open source service mesh that can be used by developers and operators to successfully control, secure, and connect services together in the world of distributed microservices. While Istio is a powerful tool for teams, it's also important for administrators to have full visibility into its health. In this blog post, we'll take a look at monitoring Istio and its microservices with <a href="/observability">Elastic Observability</a>.
</p><p>As the <a href="https://istio.io/latest/docs/concepts/what-is-istio">Istio docs</a> mention:
</p><p style="padding-left: 15pt;"><em>Istio makes it easy to create a network of deployed services with load balancing, service-to-service authentication, monitoring, and more, with few or no code changes in service code. You add Istio support to services by deploying a special sidecar proxy throughout your environment that intercepts all network communication between microservices, then configure and manage Istio using its control plane functionality, which includes:</em>
</p><ul><em>
	<li>Automatic load balancing for HTTP, gRPC, WebSocket, and TCP traffic.</li>
	<li>Fine-grained control of traffic behavior with rich routing rules, retries, failovers, and fault injection.</li>
	<li>A pluggable policy layer and configuration API supporting access controls, rate limits and quotas.</li>
	<li>Automatic metrics, logs, and traces for all traffic within a cluster, including cluster ingress and egress.</li>
	<li>Secure service-to-service communication in a cluster with strong identity-based authentication and authorization.</li>
	</em>
</ul><p>Prior to version 1.5, Istio was built with a microservice architecture and its control plane and management components consisted of multiple microservices: <em>Pilot</em>, <em>Mixer</em>, <em>Galley</em>, and <em>Citadel</em>.
</p><p>Metricbeat had support for monitoring these microservices, but in version 1.5 Istio changed its architecture to a monolithic approach and now the control plane comes with a single application called <code>istiod</code>. Pilot, Galley, and Citadel are now part of istiod, while the functionality of Mixer, which was responsible for collecting traffic metrics from the Envoy proxies, is now provided by Istio proxies directly. The current architecture of Istio looks like this:
</p><p>
	<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltc2f32779c3a87a41/5ffc4487fc835846353c37c8/istio-arch.png" data-sys-asset-uid="bltc2f32779c3a87a41" alt="Istio Architecture" style="display: block; margin: auto; width:700px;">
</p><h2>Crafting a monitoring solution with Elastic</h2><p>While versions prior to 1.5 were already supported by the multiple metricsets of the Istio Metricbeat module, in this blog we will focus more on the support for the newer versions when it comes to Istio running on Kubernetes.
</p> <strong><h3>Control plane metrics</h3></strong><p>As shown in the Istio architecture illustration above, we only have one resource from which we can collect control plane metrics. Istiod provides a Prometheus exporter from which we can collect Prometheus metrics.
</p><p>In order to consume metrics from the Prometheus endpoint, we need a metricset to <strong>collect</strong> these metrics properly, <strong>filter</strong> them out, and <strong>store</strong> them accordingly. This can easily be achieved by creating a <a href="/blog/introducing-metricbeat-light-modules">lightweight metricset</a> based on the Prometheus <a href="/guide/en/beats/metricbeat/7.x/metricbeat-metricset-prometheus-collector.html">module</a> leveraging powerful options like <a href="/guide/en/beats/metricbeat/7.x/metricbeat-metricset-prometheus-collector.html#_filtering_metrics_2">metric filtering</a> and using <a href="/guide/en/beats/metricbeat/7.x/metricbeat-metricset-prometheus-collector.html#_histograms_and_types">histograms and types</a>.
</p><p>Let’s look at the definition of this new lightweight metricset:
</p><pre class="prettyprint">input: 
  module: prometheus 
  metricset: collector 
  defaults: 
    metrics_path: /metrics 
    metrics_filters: 
      include: ["citadel_*", "galley_*", "pilot_*"] 
      exclude: ["^up$"] 
    use_types: true 
    rate_counters: true
</pre><p>This defines the path from which the collector metricset will scrape metrics and filters out metrics that we don’t need, as well as enabling <a href="/guide/en/beats/metricbeat/7.x/metricbeat-metricset-prometheus-collector.html#_histograms_and_types">rates and types</a> so that the data will be properly stored in Elasticsearch, allowing us to make the most out of it.
</p><p>The way to configure this metricset when deploying metricbeat on a Kubernetes cluster looks like this:
</p><pre class="prettyprint">- module: istio 
  metricsets: ['istiod'] 
  period: 10s 
  hosts: ['istiod.istio-system:15014']
</pre><p>Where <code>istiod</code> is the name of the Kubernetes service exposing the Istiod Pod and <code>istio-system</code> is the namespace where it’s deployed.
</p><p>And that’s all! We already have the <a href="/guide/en/beats/metricbeat/7.x/metricbeat-metricset-istio-istiod.html">istiod metricset</a>&nbsp;to collect the metrics from istiod, which also comes with a pre-built dashboard to provide an overview of the service mesh’s control plane, complete with several visualizations that you can use in your own custom dashboards:
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt7ae22c92a5a3cf9f/5ffc451cda1b7963accaf01f/overview-dashboard.png" data-sys-asset-uid="blt7ae22c92a5a3cf9f" alt="Overview Dashboard" style="display: block; margin: auto;">
</p><strong><h3>Data plane metrics</h3></strong><p>Now that we are collecting metrics from the control plane with the <code>istiod</code> metricset, we can extend our monitoring by collecting metrics from the data plane. This will give us a powerful overview of the traffic between the services that are managed by Istio.
</p><p>As mentioned earlier, Mixer was the microservice responsible for collecting and providing these data plane metrics. But after version 1.5, these metrics are collected and exposed directly from Istio proxies using a Prometheus exporter.
</p><p>All we need to do is to specify another lightweight metricset, similar to what we did for <code>istiod</code>, to collect these additional metrics:
</p><pre class="prettyprint">input: 
  module: prometheus 
  metricset: collector 
  defaults: 
    metrics_path: /stats/prometheus 
    metrics_filters: 
      include: ["istio_*"] 
      exclude: ["^up$"] 
    use_types: true 
    rate_counters: true
</pre><p>Same as before, we set the <code>metrics_path</code>, and we only keep the metrics we want and store them using types.&nbsp;
</p><p>There’s one piece missing though: we don’t know how to reach these proxy containers as we don’t know their IP addresses. Even if we knew the IPs of these containers before deploying Metricbeat, we would not be able to collect data from services that would be deployed after Metricbeat was started. We need a way to automatically identify these containers and start collecting metrics as they start — the perfect job for the <a href="/guide/en/beats/metricbeat/7.x/configuration-autodiscover.html">autodiscover</a> feature of Metricbeat. This means that we define an autodiscover condition to identify these containers and whenever a new Istio-proxy sidecar container is spotted, Metricbeat will automatically enable the proxy metricset and start collecting data from it.
</p><p>Here is an example of this autodiscover configuration:
</p><pre class="prettyprint">metricbeat.autodiscover: 
  providers: 
    - type: kubernetes 
      node: ${NODE_NAME} 
      templates: 
        - condition: 
            contains: 
              kubernetes.annotations.prometheus.io/path: "/stats/prometheus" 
          config: 
            - module: istio 
              metricsets: ["proxy"] 
              hosts: "${data.host}:15090"
</pre><p>And there we are! We are collecting metrics from all of the Istio sidecar containers running on the cluster and we are able to identify any new one of them on the fly. This is the <a href="/guide/en/beats/metricbeat/7.x/metricbeat-metricset-istio-proxy.html">proxy metricset</a> of the Istio module, which also comes with a prebuilt dashboard:
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blteeb8fa571880d815/5ffc45331ae93152b725eb48/traffic-dashboard.png" data-sys-asset-uid="blteeb8fa571880d815" alt="Traffic Dashboard" style="display: block; margin: auto;">
</p><p>Additionally, we can leverage graph analytics in Kibana to explore correlations between our data and the services. For instance, with the graph below, we can see an overview of how our services are connected to each other and how strongly they are related with http status codes. A service with a strong relationship with a 500 status code would indicate an issue that we should investigate.
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt64d3c37236ed07c8/5ffc456922252d41f7285d63/service-mesh-graph.png" data-sys-asset-uid="blt64d3c37236ed07c8" alt="Service Mesh Graph" style="display: block; margin: auto;">
</p><strong><h2>Monitoring Istio today</h2></strong><p>If you want to start monitoring your Istio service mesh, <a href="https://www.elastic.co/downloads/beats/metricbeat">download Metricbeat 7.11</a> and get started exploring your metrics efficiently with Elasticsearch and Kibana. The fastest way to deploy your cluster is to spin up a <a href="/elasticsearch/service">free trial of Elasticsearch Service</a>. And&nbsp;if you have any questions, remember that we are always happy to help on the Discuss <a href="https://discuss.elastic.co/">forums</a>.
</p>

blog-banner-istio-observability.png

blog-thumb-istio-observability.png

Istio monitoring with Elastic Observability

Learn how to keep service and container credentials safe during autodiscovery with the new Metricbeat Kubernetes secrets keystore.

<p>In the world of containers and Kubernetes, <a href="/observability">observability</a> is crucial. Cluster administrators need visibility into the infrastructure and cluster operators need to know the status of their workloads at any given time. And in both cases, they need observability into moving objects.
</p>
<p>This is where <a href="/beats/metricbeat">Metricbeat</a> and its autodiscover feature do the hard part for you. Operators can set up fairly complex monitoring scenarios with a self-service approach following the principle of least privilege at the cluster level.
</p>
<p>The self-service approach is great, but what if the service I want to monitor is behind authentication? In this blog, we’ll take a look at how easy it is to configure observability for authenticated services with Elastic.
</p>
<h2>How autodiscover works</h2>
<p>Applications running on containers and pods become moving targets to the monitoring system. Autodiscover tracks changes at the container and pod level and dynamically adjusts the monitoring configuration using two strategies:
</p>
<ol>
	<li><strong>Template-based autodiscover:</strong> By defining configuration templates, the autodiscover subsystem can use them to monitor services as they start running.</li>
	<li><strong>Hints-based autodiscover:</strong> The <code>hints</code> system looks for hints in Kubernetes pod annotations or Docker labels which have the prefix <code>co.elastic.metrics</code>. As soon as the container starts, Metricbeat will check if it contains any hints and launch the proper configuration for it. Hints tell Metricbeat how to get metrics for the given container.</li>
</ol>
<p>Let’s look at an example with <em>template-based autodiscover</em>. Here is a configuration template for automatically detecting Redis pods and starting to collect metrics using the respective Redis Metricbeat module:
</p>
<pre class="prettyprint">metricbeat.autodiscover:
  providers:
    - type: kubernetes
      templates:
        - condition:
            contains:
              kubernetes.labels.app: "redis"
          config:
            - module: redis
              metricsets: ["info"]
              hosts: "${data.host}:6379"
</pre>
<p>Defining autodiscover templates at Metricbeat’s configuration level is a valid approach, but users must be granted the permission to globally configure Metricbeat. In addition, a restart of Metricbeat is required every time the autodiscover template is updated. 
With hints-based autodiscover, we let workload owners enable monitoring for their services on the fly, avoiding over-granted permissions without restarting the Beats. The following example configures the same Redis monitoring from above using hints:
</p>
<pre class="prettyprint">metricbeat.autodiscover:
  providers:
    - type: kubernetes
      hints.enabled: true
</pre>
<p>In this case, Redis pods need to be <a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/">annotated</a> accordingly:
</p>
<pre class="prettyprint">apiVersion: v1
kind: Pod
metadata:
  name: annotations-demo
  annotations:
    co.elastic.metrics/module: redis
    co.elastic.metrics/metricsets: info
    co.elastic.metrics/hosts: '${data.host}:6379'
</pre>
<p><strong></strong>
</p>
<h2><strong>Using passwords with autodiscover</strong></h2>
<p>In many cases, the target service requires password authentication in order to provide monitoring metrics. For instance, Redis can be configured to be password protected, meaning we have to provide a password to the Redis module in Metricbeat in order to collect metrics successfully. With template-based autodiscover, we would configure the template like this:
</p>
<pre class="prettyprint">metricbeat.autodiscover:
  providers:
    - type: kubernetes
      templates:
        - condition:
            contains:
              kubernetes.labels.app: "redis"
          config:
            - module: redis
              metricsets: ["info"]
              hosts: "${data.host}:6379"
              password: 'myred1sp@ss'
</pre>
<p>Similarly, with hints-based autodiscover, we would annotate the pod with a string containing the password:
</p>
<pre class="prettyprint">apiVersion: v1
kind: Pod
metadata:
  name: annotations-demo
annotations:
  co.elastic.metrics/module: redis
  co.elastic.metrics/metricsets: info
  co.elastic.metrics/hosts: '${data.host}:6379'
  co.elastic.metrics/password: 'myred1sp@ss'
</pre>
<p>While we can use these setups for development or testing purposes, storing passwords in plain text poses a serious risk and should be avoided. We could put the password in an environment variable and let Metricbeat interpolate the manifest, but this wouldn’t make our security posture any better. Let’s see a preferable alternative.
</p> <strong>
<h3>The Metricbeat keystore</h3></strong>
<p>Metricbeat provides a feature called <a href="https://www.elastic.co/guide/en/beats/metricbeat/7.8/keystore.html">keystore</a> that can be used to securely store sensitive information and reference it in the Metricbeat configuration as a secret.
Let’s see how we can leverage this feature in our autodiscover template configurations. First, we create the keystore and add the password for Redis under the name 
	<code>REDIS_PASSWORD</code>:
</p>
<pre class="prettyprint">metricbeat keystore create
metricbeat keystore add REDIS_PASSWORD
</pre>
<p>And provide “myred1sp@ss” when prompted.
</p>
<p>Then the configuration template will be able to reference the string <code>REDIS_PASSWORD</code> to access the actual password:
</p>
<pre class="prettyprint">metricbeat.autodiscover:
  providers:
    - type: kubernetes
      templates:
        - condition:
            contains:
              kubernetes.labels.app: "redis"
          config:
            - module: redis
              metricsets: ["info"]
              hosts: "${data.host}:6379"
              password: "${REDIS_PASSWORD}"
</pre>
<p>And that’s all! We managed to avoid using plain text passwords or defining ENV variables, and instead we leveraged the Metricbeat keystore to securely store the password for the target Redis pod.
</p>
<p>Unfortunately, we can't use this strategy with hints-based autodiscovery — it wouldn’t be safe to expose the keystore to the hints mechanism because it could be a target for an attacker trying to exploit the whole keystore just using hints annotations. This is why we introduced a completely new Keystore, the <a href="https://www.elastic.co/guide/en/beats/metricbeat/7.9/configuration-autodiscover.html#_kubernetes_secrets">Kubernetes secrets keystore</a>.
</p> <strong>
<h3>The Kubernetes Secrets Keystore</h3></strong>
<p>With the Kubernetes Secrets Keystore we allow users to consume <a href="https://kubernetes.io/docs/concepts/configuration/secret/">Kubernetes secrets</a> directly in autodiscover configuration. It can be used for both template-based and hints-based autodiscover.
</p>
<p>Users can refer a Kubernetes Secret with the following pattern:
</p>
<pre class="prettyprint">${kubernetes.&lt;namespace&gt;.&lt;secret_name&gt;.&lt;value&gt;}
</pre>
<p>For instance, <code>${kubernetes.default.redisPass.value}</code> where <code>kubernetes.default.redisPass.value</code> specifies a key stored as a Kubernetes secret with the following:
</p>
<ol>
	<li>Kubernetes Namespace: <code>default</code></li>
	<li>Secret Name: <code>redisPass</code></li>
	<li>Secret Data Key: <code>value</code></li>
</ol>
<p>Note that pods can only consume secrets that belong to the same Kubernetes namespace. For example if our Redis pod is running in the <code>staging</code> namespace, it cannot access a secret created in the <code>testing</code> namespace.
</p>
<p>Let's see how we could leverage the new Keystore backend. First we need to create the secret:
</p>
<pre class="prettyprint">cat &lt;&lt; EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: redisPass
type: Opaque
data:
  value: $(echo -n "myred1sp@ss" | base64)
EOF
</pre>
<p>Now we can consume the <code>redisPass</code> secret from within our hint based autodiscover configuration:
</p>
<pre class="prettyprint">apiVersion: v1
kind: Pod
metadata:
  name: annotations-demo
  app: redis
annotations:
  co.elastic.metrics/module: redis
  co.elastic.metrics/metricsets: info
  co.elastic.metrics/hosts: '${data.host}:6379'
  co.elastic.metrics/password: '${kubernetes.default.redisPass.value}'
</pre>
<p>Kubernetes secrets keystore also works with autodiscover templates and we can access Kuberentes secrets from our static configurations, too:
</p>
<pre class="prettyprint">metricbeat.autodiscover:
  providers:
    - type: kubernetes
      templates:
        - condition:
            contains:
              kubernetes.labels.app: "redis"
          config:
            - module: redis
              metricsets: ["info", "keyspace"]
              hosts: "${data.host}:6379"
              password: "${kubernetes.default.redisPass.value}"
</pre> <strong>
<h2>Conclusions</h2></strong>
<p>If you find the idea of securing sensitive configurations in your Kubernetes environment of utmost importance, then you will love Metricbeat keystore and Kubernetes secrets keystore — they allow you to securely connect without leaking plain text passwords.
</p>
<p><a href="/downloads">Download Metricbeat</a> and start storing your passwords securely for your autodiscovered workloads without worrying about leaking sensitive data. And make sure your data stays safe by using Elasticsearch Service on <a href="/cloud">Elastic Cloud</a>, where security comes standard. <a href="/elasticsearch/service">Start a free trial today</a>.
</p>If you have any questions, we are always happy to help on the Discuss <a href="https://discuss.elastic.co/">forums</a>.

blog-banner-release-observability.png

blog-thumb-kubernetes-observability.png

Securely manage credentials while monitoring Kubernetes workloads with autodiscovery

Prometheus plays a significant role in observability, but it has its limitations. Learn how to use the Elastic Stack with Prometheus to boost scalability and durability, as well as open the door for more use cases.

<p>Prometheus plays a significant role in the observability area. An increasing number of applications use Prometheus exporters to expose performance and monitoring data, which is later scraped by a Prometheus server.
</p>
<p>However, when it comes to storage, Prometheus faces some limitations in its scalability and durability since its local storage is limited by single nodes. In order to tackle this, users should either go with clustered storage in Prometheus itself, or use the Prometheus interfaces that allow integrating with remote storage systems. In this blog, we're going to look at how simple and effective it is to integrate Prometheus with Elasticsearch for long-term storage.
</p>
<p>Elasticsearch is a powerful, scalable, and efficient storage system that can store various types of data (text, numerics, geo, structured, unstructured, etc.) in the long term. The scaling capabilities and search performance of Elasticsearch make it a prime option when it comes to storing monitoring data. Plus, combining Elasticsearch with Kibana gives users dynamic insights into their data.
</p>
<h2>Integrating Prometheus with Elasticsearch for long-term storage</h2>
<p>Prometheus integrates with remote storage systems in two ways:
</p>
<ul>
	<li>Prometheus can write samples that it ingests to a remote URL in a standardized format</li>
	<li>Prometheus can read (back) sample data from a remote URL in a standardized format</li>
</ul>
<p>The read and write protocols both use a snappy-compressed protocol buffer encoding over HTTP. For details on the request and response messages, see the remote storage <a href="https://github.com/prometheus/prometheus/blob/master/prompb/remote.proto">protocol buffer definitions</a>.
</p>
<p>In order to use Elasticsearch as a remote storage system for Prometheus, the user can employ the official Metricbeat module and more specifically the <a href="https://www.elastic.co/guide/en/beats/metricbeat/7.7/metricbeat-metricset-prometheus-remote_write.html">remote_write metricset</a>. In this, the user can configure Metricbeat like:
</p>
<pre class="prettyprint"># Metrics sent by a Prometheus server using remote_write option 
- module: prometheus 
  metricsets: ["remote_write"] 
  host: "localhost" 
  port: "9201"
</pre>
<p>And Prometheus:
</p>
<pre class="prettyprint">remote_write: 
  - url: "http://localhost:9201/write"
</pre>
<p>In addition, the user can set up secure settings for the server using TLS/SSL.
</p>
<p>With this type of configuration, Metricbeat, on startup, will start an http server that will wait for metrics from a Prometheus server. When metrics are received they will be processed accordingly by Metricbeat and will be efficiently stored in Elasticsearch. Then the user can retrieve and analyze the metrics using Kibana's visualizations and dashboards.
</p>
<p>It’s also worth mentioning that Metricbeat, Elasticsearch, and Kibana can run out of the box in Kubernetes environments. This can be achieved by using <a href="https://www.elastic.co/guide/en/cloud-on-k8s/current/index.html">Elastic Cloud on Kubernetes (ECK)</a>, the official Elasticsearch and Kibana operator, along with the predefined Metricbeat <a href="https://github.com/elastic/cloud-on-k8s/tree/1.0/config/recipes/beats">manifests</a>. Putting all these together gives the user the opportunity to quickly set up an Elastic-based remote storage integration for their Prometheus servers that run on Kubernetes.
</p>
<p>As a result, users will benefit from doubling down on observability for their cloud-native applications and workloads. By creating a long-term storage solution for their data with Elasticsearch, they’ll also have the opportunity to efficiently query them back and further analyze them at any time. Moreover, users can leverage Kibana to build visualizations.
</p>
<p>Below is an example of how we can build a dashboard visualizing Prometheus metrics from node exporters and cAdvisor:
</p>
<p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt8e72ab55c29b5702/5ea36f2ddd109257057d68a6/long-term-prometheus-blog-stats.png" data-sys-asset-uid="blt8e72ab55c29b5702" alt="" style="display: block; margin: auto;">
</p><figcaption>Node statistics
</figcaption>
<p>And it goes without saying that we would like to monitor how remote storage statistics evolve by visualizing how many samples have been successfully sent to remote storage or how many have been retried, failed, or dropped. Another significant metric to monitor here is the number of shards, which shows the number of concurrency/parallelism with which Prometheus server sends metrics to the remote storage endpoint.
</p>
<p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt0871b8297a8451f0/5ea36f18ae23d30cd1dfb35c/long-term-prometheus-blog-metrics.png" data-sys-asset-uid="blt0871b8297a8451f0" alt="" style="display: block; margin: auto;">
</p><figcaption>Prometheus remote storage statistics
</figcaption>
<h2>Implement long-term storage for Prometheus today</h2>
<p>Are you already considering a long-term storage solution for your Prometheus metrics? <a href="https://www.elastic.co/downloads">Download</a> Metricbeat 7.7 and start storing your data in Elasticsearch without worrying about scalability and query efficiency. The fastest way to get your cluster deployed is to spin up a <a href="https://www.elastic.co/elasticsearch/service">free trial of Elasticsearch Service</a>. Of course, if you have any questions, remember that we are always happy to help on the Discuss <a href="https://discuss.elastic.co/">forums</a>.
</p>

observability-prometheus-swiss-army-knife-blog-banner.png

observability-prometheus-swiss-army-knife-blog-thumb.png

How to implement Prometheus long-term storage using Elasticsearch

Capture and analyze NATS logs and metrics in Elasticsearch using the new NATS modules, contributed, and battle-tested by real-world NATS users in production.

<p>In a world where stateless applications are optimized to run blazingly fast, message exchanging cannot be allowed to affect their performance. Having the ability to <a href="https://hackernoon.com/want-a-super-fast-messaging-system-c19c9f8cf560">process millions of messages per second</a>, NATS is the sprinter of messaging systems.
</p><p>While benchmarks are good indicators for choosing a tool, there is no way to confirm its value without monitoring its performance in production. As a team working together in the telecommunications sector, we're building NFV solutions on top of Kubernetes microservices, and we're using NATS as a messaging system to ensure the resiliency of the critical paths. These include the communication between the software components running on the clients’ data centers and our centralized platforms. It has allowed us to achieve high rates of message exchanging with great reliability. In this regard, we need to assure its health by providing actionable visibility in real-time. Given the fact that we use <a href="https://docs.fluentd.org/v/0.12/articles/kubernetes-fluentd">EFK</a> to monitor our <a href="https://github.com/fluent/fluentd-kubernetes-daemonset/blob/master/fluentd-daemonset-elasticsearch-rbac.yaml">Kubernetes</a> stack, monitoring of NATS had to be achieved by the exact same way.
</p><p>Everything started with a goal: Ship NATS monitoring data to Elasticsearch.&nbsp;
</p><p>What we achieved: Extending Beats, by providing NATS modules for both <a href="https://www.elastic.co/guide/en/beats/metricbeat/7.0/metricbeat-module-nats.html">Metricbeat</a> and <a href="https://www.elastic.co/guide/en/beats/filebeat/master/filebeat-module-nats.html">Filebeat</a> shippers. Get on board to find out how!
</p><h2>Motivated by an internal hackathon</h2><p>We wanted results and we wanted them fast, but our internal backlog was full of requirements forcing our time to be very limited. Luckily, our company was organizing its annual hackathon. Without hesitation, we grabbed the opportunity to work on creating our own NATS Beat by leveraging the <a href="https://github.com/nfvsap/natsbeat">Beats extension mechanism</a>. After two days of effort our NATS Beat succeeded into becoming part of the official list of <a href="https://www.elastic.co/guide/en/beats/libbeat/current/community-beats.html">community Beats</a>. And that was only the beginning. Intrigued by the idea, the Elastic Beats team encouraged us to put our code into the core Beats project. With their support, we managed to develop a Metricbeat module for metrics collection, and a Filebeat module, for log aggregation. Today, NATS monitoring is deeply integrated into the core of Beats upstream.
</p><h2>How to monitor NATS</h2><p>NATS is very helpful when it comes to providing <a href="https://nats.io/documentation/managing_the_server/monitoring/">monitoring data</a>. If requested, NATS server can serve such data in JSON format by exposing four different HTTP endpoints:
</p><ul>
	<li>/varz: reports general statistics such as CPU utilization, memory consumption, etc.
	</li>
	<li>/connz: reports detailed information on client connections
	</li>
	<li>/routez: reports information on routes between servers of the NATS cluster
	</li>
	<li>/subz: reports detailed information about current subscriptions and the routing data structure</li>
</ul><p>By querying these endpoints in a periodic way, user can understand a lot about the running state of his messaging queue. And so we did with the brand new NATS module of Metricbeat.
</p><p>But NATS can provide more. Another interesting place to find useful information regarding monitoring is the server’s logs. If requested, NATS server can provide <a href="https://nats.io/documentation/server/gnatsd-logging/">TRACE level logs</a> for every message received or transmitted. These log entries can be proven really valuable if handled properly. Our NATS module for Filebeat did exactly that. Every log is parsed and all its meaningful bits are squeezed out of it.
</p><h2>NATS as a Metricbeat module</h2><p>As it happens with any Metricbeat module, NATS needs to be activated and configured accordingly in order to fetch the data from the monitoring endpoints. Upon configured properly, it fetches data from the endpoints, curates them accordingly and ships them to Elasticsearch.
</p><h3>Configuration
</h3><p>The module can be configured as simple as this:
</p><pre class="prettyprint">metricbeat.modules:
- module: nats
  metricsets: ["connections", "routes", "stats", "subscriptions"]
  period: 10s
  hosts: ["localhost:8222"]
  stats.metrics_path: "/varz"
  connections.metrics_path: "/connz"
  routes.metrics_path: "/routez"
  subscriptions.metrics_path: "/subsz"
  enabled: true
</pre><p>The above configuration provides the location of the NATS server(s) we want to monitor, the query interval for metrics, which metrics to fetch and the URIs under which they can be accessed.
</p><h3>Let the metrics speak</h3><p>Fetching the data is one thing. Sending them to Elasticsearch is another. But what is the value of storing them if you cannot visualize them? With that in mind, this module ships with a pre-built dashboard which can be loaded automatically from Beats and give a good starting point to the user.
</p><p>Let’s take a sneak peak:
</p><p><img src="https://images.contentstack.io/v3/assets/bltf7afce26b89a5b33/blt28de268fcb47839e/5cabacdbef14588e0283421b/download" data-sys-asset-uid="blt28de268fcb47839e" alt="nats-let-metrics-speak.png">
</p><p>Users&nbsp;can detect slow consumers which may affect NATS performance, monitor how NATS cache is performing, or even realize the utilization of resources made by our NATS servers. Additionally, users can correlate the stored data according to their needs and extend the dashboard. It is in their&nbsp;hands!
</p><h2>NATS as a Filebeat module</h2><p>After making Metricbeat module a reality, a brilliant idea was put on the table: Is there something we could do for NATS server logs? The answer was standing in front of us… Filebeat!
</p><p>And so we did. Filebeat is an agent dedicated to fetching logs, parsing them with a wide range of text processors, and creating docs out of them in an Elasticsearch index. But to go far you need to start simple. Our very first version of NATS module was just extracting log timestamp, level, and message, and putting them into the datastore. The second version though is taking it to the next level. After investigating all log patterns we processed the log messages even more. The outcome is far more interesting, it can be visualized and it can offer a much deeper understanding of the NATS cluster’s state
</p><h3>Configuration</h3><p>Filebeat module can be configured as the Metricbeat module by adding the following lines to the configuration file:
</p><pre class="prettyprint">filebeat.modules:
- module: nats  
  log:
    enabled: true
    var.paths:
      - /var/log/nats/nats.log*
</pre><p>As it can be easily observed, apart from enabling the module, the only configurable field a user has to set is a list of log path regular expressions. Given the location of the logs, NATS module has everything it needs to start scraping data from them.
</p><h3>Logs as a Dashboard (LAAD)</h3><p>As the title implies, NATS module gives the opportunity to visualize the logs that the user's NATS cluster is producing. With the pre-built dashboard accompanying the module and the log trace option enabled to NATS server, user can come up with a dashboard as interesting as the following:
</p><p><img src="https://images.contentstack.io/v3/assets/bltf7afce26b89a5b33/blt3a5a663088e662c6/5cabacaa051f0ba57b53c4d5/download" data-sys-asset-uid="blt3a5a663088e662c6" alt="nats-logs-as-a-dashboard-filebeat-1.png">
</p><p>Filebeat dashboard can provide various forms of data and statistics. Users can observe the distribution of messages by its type, the heat received by the NATS subjects, and the type of logs produced in a timeline. Furthermore, users have a good indicator on messages’ traffic, both incoming and outgoing, as well as the IP range distribution of the clients. The most interesting part: All of the above can happen with a click of a button!
</p><h3>At the end of the day</h3><p>All in all, time invested on this effort was definitely worth it. Not only did we solve our problem of NATS monitoring, which could be achieved with only the community Beat, but we also gained a lot of experience on the way. We had the chance to work with excellent engineers from Elastic and we felt the joy of giving back to the community. So it goes without saying that we enjoyed that and we would suggest such kind of contributions to everyone who faces similar situations.
</p><p>Are you a NATS user?&nbsp; Download <a href="https://www.elastic.co/downloads/beats/metricbeat">Metricbeat</a> 7.0 and build <a href="https://www.elastic.co/downloads/beats/filebeat">Filebeat</a>&nbsp;master branch, enable the NATS modules, and start looking under the covers to see what you can discover about your NATS environment.&nbsp;
</p><hr><div>
	<p><img src="https://images.contentstack.io/v3/assets/bltf7afce26b89a5b33/bltdd3ec6d6a9397204/5cabb848ef14588e0283427f/download" data-sys-asset-uid="bltdd3ec6d6a9397204" alt="stamatis-katsaounis.png" width="100" style="width: 100px; float: left; margin: 10px;"><strong><em>Stamatis Katsaounis&nbsp;</em></strong><em>is a software engineer, focused mainly on the Cloud. He is an active member of the OPNFV community with contributions to various OPNFV projects. He experiments with OpenStack and Kubernetes deployments and is a Linux enthusiast. He speaks C, Bash and Python. Lately, he is a frequent visitor of Go playground</em>.
	</p>
</div><div>
	<p><img src="https://images.contentstack.io/v3/assets/bltf7afce26b89a5b33/bltb63f1699387ac50a/5cabb8978f70e39878691c08/download" data-sys-asset-uid="bltb63f1699387ac50a" alt="michael-katsoulis.jpg" width="100" style="width: 100px; float: left; margin: 10px;"><strong><em>Michael Katsoulis</em></strong><em> is a cloud software engineer, backed with working knowledge in telecommunications systems, currently focusing in SDN/NFV area. Leveraging Linux power and open source tools he always tries to combine work in internal projects with open source contributions.</em>
	</p>
</div><div>
	<p><br><strong><em>Christos Markou</em></strong><em> is a systems and software engineer that's passionate about distributed systems and cloud technologies. He has hands-on experience in web services, cloud computing and large system architecture, and is an open source enthusiast with various contributions in high class projects.</em>
	</p>
</div>

Articles by Christos Markou

How to monitor Amazon ECS with Elastic Observability

Istio monitoring with Elastic Observability

Securely manage credentials while monitoring Kubernetes workloads with autodiscovery

How to implement Prometheus long-term storage using Elasticsearch

Monitoring a NATS messaging system with Beats

Products & Solutions

Company

Resources

Featured topics

News

Articles by Christos Markou

How to monitor Amazon ECS with Elastic Observability

Istio monitoring with Elastic Observability

Securely manage credentials while monitoring Kubernetes workloads with autodiscovery

How to implement Prometheus long-term storage using Elasticsearch

Monitoring a NATS messaging system with Beats