In this short tutorial, learn how to deploy the Elastic Stack on Azure Kubernetes Service using the official Elastic Cloud on Kubernetes operator.

<p>It's safe to say that Kubernetes is the de facto standard for orchestrating containers and the applications running in them. As the standard, a variety of managed services and orchestration options are available to choose from. In this blog post, we're going to take a look at running the Elastic Stack on Azure Kubernetes Service (AKS) using Elastic Cloud on Kubernetes (ECK) as the operator.
</p><p><a href="/elastic-cloud-kubernetes">Elastic Cloud on Kubernetes</a>, is the official operator for running the Elastic Stack on Kubernetes. ECK helps to manage, scale, upgrade, and deploy the Elastic Stack securely. In the steps below, we will deploy ECK on AKS and then use that deployment to collect logs, metrics, security events from a virtual machine on Azure.
</p><p>Here's what we'll do:
</p><ol>
	<li>Create an Azure Kubernetes Service cluster</li>
	<li>Install Elastic Cloud on Kubernetes</li>
	<li>Create an Elasticsearch cluster</li>
	<li>Deploy Kibana</li>
	<li>Create an Azure VM for us to monitor</li>
	<li>Deploy Metric beat to collect VM metrics, and events</li>
</ol> <h2>Deploying AKS, ECK, Elasticsearch, and Kibana</h2><p><strong>Note:</strong> You need to have <a href="https://azure.microsoft.com/en-us/free/?WT.mc_id=elastic-blog-arsaha">Azure Account</a> and the <a href="https://docs.microsoft.com/en-us/cli/azure/install-azure-cli">Azure CLI</a> for Microsoft Azure installed to run some platform specific commands. This helps you to create your cluster using this Azure CLI command.
</p> <h3>Step 1: Create an AKS cluster</h3><pre class="prettyprint">az aks create --resource-group resourceGroupName --name clusterName --node-count 3 --generate-ssh-keys
</pre> <h3>Step 2: Connect to the AKS cluster</h3><pre class="prettyprint">az aks get-credentials --resource-group resourceGroupName --name clusterName
</pre> <h3>Step 3: Install the ECK operator</h3><pre class="prettyprint">kubectl apply -f https://download.elastic.co/downloads/eck/1.1.2/all-in-one.yaml
</pre><pre class="prettyprint">kubectl -n elastic-system logs -f statefulset.apps/elastic-operator
</pre> <h3>Step 4: Create an Elasticsearch cluster with a external IP</h3><p>We're using the default load balancer that is available with Azure Kubernetes Service.
</p><pre class="prettyprint">cat &lt;&lt;EOF | kubectl apply -f -apiVersion: elasticsearch.k8s.elastic.co/v1 
kind: Elasticsearch 
metadata: 
  name: quickstart 
spec: 
  version: 7.9.2 #Make sure you use the version of your choice 
  http: 
    service: 
      spec: 
        type: LoadBalancer #Adds a External IP 
  nodeSets: 
  - name: default 
    count: 1 
    config: 
      node.master: true 
      node.data: true 
      node.ingest: true 
      node.store.allow_mmap: false 
EOF
</pre> <h3>Step 5: Monitor the cluster creation</h3><pre class="prettyprint">kubectl get elasticsearch 
kubectl get pods -w
</pre> <h3>Step 6: Check the logs of the pod created</h3><pre class="prettyprint">kubectl logs -f quickstart-es-default-0 
kubectl get service quickstart-es-http
</pre> <h3>Step 7: Retrieve the password of Elasticsearch cluster</h3><pre class="prettyprint">PASSWORD=$(kubectl get secret quickstart-es-elastic-user -o=jsonpath='{.data.elastic}' | base64 --decode)
</pre><pre class="prettyprint">curl -u "elastic:$PASSWORD" -k "https://&lt;IP_ADDRESS&gt;:9200"
</pre><table style="background: #FFFFD2;">
<tbody>
<tr>
<td><strong>Note:</strong> The public IP address of Elasticsearch can be picked by running:
<pre class="prettyprint">kubectl get "kubectl get svc quickstart-es-http"
</pre>
</td>
</tr>
</tbody>
</table><h3>Step 8: Deploy Kibana</h3><pre class="prettyprint">cat &lt;&lt;EOF | kubectl apply -f - 
apiVersion: kibana.k8s.elastic.co/v1 
kind: Kibana 
metadata: 
  name: quickstart 
spec: 
  version: 7.9.2 #Make sure Kibana and Elasticsearch are on the same version. 
  http: 
    service: 
      spec: 
        type: LoadBalancer #Adds a External IP 
  count: 1 
  elasticsearchRef: 
    name: quickstart 
EOF
</pre> <h3>Step 9: Monitor the Kibana deployment</h3><pre class="prettyprint">kubectl get kibana
</pre> <h2>Ingesting and analyzing Azure metrics</h2><p>Now that we have created an Elasticsearch cluster with Kibana in AKS, let's go ahead and ingest some observability data from Azure Cloud itself. <a href="/beats/filebeat">Filebeat</a> and <a href="/beats/metricbeat">Metricbeat</a> make this easy by coming with out-of-the-box an Azure Module, helping to easily gather <a href="https://www.elastic.co/guide/en/beats/filebeat/7.9/filebeat-module-azure.html">logs (activity, sign in, audit)</a> and <a href="https://www.elastic.co/guide/en/beats/metricbeat/7.9/metricbeat-module-azure.html">metrics (vm, container registry, billing)</a> from Azure Cloud Platform.
</p><p>In this tutorial, we will install Metricbeat on an Azure VM and enable the Azure cloud module. Before that, we also need to have credentials to authenticate with Azure Monitor REST API which uses Azure Resource Manager authentication model.
</p><p>We need to have <code>client_id</code>, <code>client_secret</code>, <code>subscription_id</code>, <code>tenant_id</code> which can be obtained by creating an Azure Active Directory App. You can use <a href="https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal">this guide to Azure AD application and service principal that can access resources</a>.
</p><h3>Step 1: Create a Azure VM and SSH into the VM</h3><pre class="prettyprint">az vm create \ 
  --resource-group myResourceGroup \ 
  --name myVM \ 
  --image UbuntuLTS \ 
  --admin-username azureuser \ 
  --generate-ssh-keys 
</pre><pre class="prettyprint">ssh azureuser@&lt;IP_ADDRESS&gt;
</pre><h3>Step 2: Install Metricbeat</h3> <pre class="prettyprint">wget https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-7.9.2-amd64.deb
</pre><h3>Step 3: Configure Elasticsearch and Kibana credentials in Metricbeat</h3><p>This helps us to ship the data to Elasticsearch Cluster created on AKS as well as load dashboards in Kibana. </p><pre class="prettyprint">vim /etc/metricbeat/metricbeat.yml
</pre><pre class="prettyprint"> setup.kibana: 
   host: "https://&lt;public_ip_addr&gt;:5601"
</pre><table style="background: #FFFFD2;">
<tbody>
<tr>
<td><strong>Note:</strong> The public IP address of Kibana can be picked by running:
<pre class="prettyprint">kubectl get "kubectl get svc quickstart-kb-http"
</pre>
</td>
</tr>
</tbody>
</table><pre class="prettyprint">vim /etc/metricbeat/modules.d/azure.yml.disabled
</pre><p>Replace the <code>client_id</code>, <code>client_secret</code>, <code>subscription_id</code>, <code>tenant_id</code> for all the metricsets listed in the yml file. A sample might look like this.
</p><pre class="prettyprint">- module: azure 
  metricsets: 
  - monitor 
  enabled: true 
  period: 300s 
  client_id: '8dec1ab1-1691-48a6-af43-f87de68e971b' 
  client_secret: '~fwL-MhOcguaD2yK1e_.OWHhhqwdp-p974' 
  tenant_id: 'aa40685b-417d-4664-b4ec-8f7640719adb' 
  subscription_id: '70bd6e77-4b1e-4835-8896-db77b8eef364' 
  refresh_list_interval: 600s 
  resources: 
  - resource_query: "resourceType eq 'Microsoft.DocumentDb/databaseAccounts'" 
    metrics: 
    - name: ["DataUsage", "DocumentCount", "DocumentQuota"] 
      namespace: "Microsoft.DocumentDb/databaseAccounts"
</pre><h3>Step 4: Enable Azure Module and start Metricbeat</h3><pre class="prettyprint">cd /usr/bin/
./metricbeat modules enable azure
./metricbeat setup --dashboards
./metricbeat -e
</pre><h3>Step 5: Monitor Metrics of Azure in Kibana</h3><p>Log into Kibana and headover to Dashboards. Search for "Azure" to look at several preconfigured dashboards regarding storage, database, billing. Here is what a sample monitoring dashboard would look like:</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt2172bd516ecab195/5f8f24ca1252ff30caf63729/blog-eck-on-aks-metrics.jpg" data-sys-asset-uid="blt2172bd516ecab195" alt="Prebuilt Azure monitoring dashboard in Kibana" style="display: block; margin: auto;"></p><h2>Wrapping up</h2><p>And that's that! You have successfully built a secure Elastic Stack deployment on a managed Kubernetes service. You can also deploy other applications like <a href="/apm">Elastic APM</a> or <a href="/workplace-search">Elastic Workplace Search</a>. In addition to that, you can also enable <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.9/modules-cross-cluster-search.html">cross-cluster search</a> and <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.9/xpack-ccr.html">replication</a>, which enables you to deploy a Elastic Stack on Kubernetes cluster across regions to serve users.
</p><p>We encourage you to <a href="/downloads/elastic-cloud-kubernetes">try ECK for yourself</a> (on any service), and if you have further questions related to this blog post, you can always reach us out on our <a href="http://discuss.elastic.co">community discussion forums</a> or <a href="https://ela.st/slack">Slack workspace</a>.</p><p>If you'd like to learn more how Elastic can help with Kubernetes observability, check out our <a href="/webinars/best-of-elastic-observability">Best of Elastic Observability webinars</a> series</p>.

cloud-azure-blog-banner.png

blog-thumb-cloud-azure.png

Running Elastic Cloud on Kubernetes from Azure Kubernetes Service

In this quick tutorial, we will look at how to generate an SDK and create workflows using the Elastic Cloud API.

<p>We recently announced the <a href="/blog/elastic-cloud-elasticsearch-service-api-is-now-ga">general availability of our Elasticsearch Service API</a>. APIs help to automate tasks such as creating and scaling deployments, integrating with existing workflows, and testing.&nbsp;
</p><p>The Elasticsearch Service API supports the <a href="https://en.wikipedia.org/wiki/OpenAPI_Specification">Open API Specification</a>, which allows you to use tools like <a href="https://swagger.io/">Swagger</a> to generate software development kits (SDKs) in any programming language. You can import the <a href="https://cloud.elastic.co/api/v1/api-docs/spec.json">API spec</a> onto Postman and create a <a href="http://postman.com/collection/">Postman Collection</a> to create a test suite.
</p><p>In this blog post, we’ll generate an SDK using Swagger and demonstrate how you can perform operations using the Elasticsearch Service API.&nbsp;
</p><h2>Generating an SDK</h2><p><strong>Note:</strong> If you don’t want to generate an SDK, you can skip this step and use the <a href="https://github.com/aravindputrevu/ess-cloud-sdk-java">ESS Java Client SDK</a>.&nbsp;
</p><p>As mentioned above, the Elasticsearch Service API follows the Open API Specification. Here, we’ll be using Swagger Generator to generate respective language SDK.&nbsp;
</p><ol>
	<li>First, <a href="https://cloud.elastic.co/api/v1/api-docs/spec.json">copy the API specification from this link</a>.</li>
	<li>Open <a href="https://editor.swagger.io/">https://editor.swagger.io</a>.</li>
	<li>Paste the API spec in the above link and accept the prompt to convert to YAML.</li>
	<li>Go to <strong>Generate Client</strong> &gt; <strong>Java </strong>(we’ll select Java for this tutorial, but you can use the programming language of your choice).</li>
</ol><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt5edecb7d7e93365c/5f4f76430ab106265a187e7f/swagger-generator.png" data-sys-asset-uid="blt5edecb7d7e93365c" alt="Using Swagger Generator to generate the SDK" style="display: block; margin: auto;">
</p><p>We’ve generated a <a href="https://github.com/aravindputrevu/ess-cloud-sdk-java">Java Client SDK</a> and <a href="https://github.com/aravindputrevu/ess-cloud-sdk-java/releases/download/v0.1/cloud-sdk-java.zip">compiled a JAR</a> to use it further as a dependency in our sample application.&nbsp;
</p><h2>Creating an API key</h2><p>To perform any of the API calls, we need an API key to talk to Elasticsearch Service securely. This requires an Elastic Cloud account. <a href="http://ela.st/dev-dish">Visit this link to spin up an extended 30-day free trial</a> of Elastic Cloud.
</p><p>Now you’ll need to create an API key using the Elastic Cloud console. You can do this under the <strong>Account</strong> section:
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt7f92f076bff5a124/5f4f7690c0ed4228f9e5831f/image4.png" data-sys-asset-uid="blt7f92f076bff5a124" alt="Create a API key from the Elastic Cloud console" style="display: block; margin: auto;">
</p><p><strong>Note:</strong> API keys are sensitive and tied to your account, and they grant privileges to create, list, and destroy resources in Elasticsearch Service. Please keep your keys safe.
</p><h2>Performing operations using the Elasticsearch Service API and SDK</h2><p>The Elasticsearch Service API contains multiple operation APIs to manage your deployments.&nbsp;
</p><p>For example, we can use the SDK to automate essential workflows. I’m using Intellij and a sample Java project to use the SDK jar that we generated earlier. You can also use other editors to do the same thing.
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt220035ca734cfbd9/5f4f77524a7bbc279b84532c/image2.png" data-sys-asset-uid="blt220035ca734cfbd9" alt="Elastic Cloud API as a Postman Collection" style="display: block; margin: auto;">
</p><p>Here are some operations you can perform with the Elasticsearch Service API SDK:
</p><ul>
	<li>Create an Elasticsearch Service deployment</li>
</ul><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltf71d7865c70110d9/5f4f77cab5badb2465bb943c/image5.png" data-sys-asset-uid="bltf71d7865c70110d9" alt="Code for Creating a Elasticsearch Deployment on Elastic Cloud" style="display: block; margin: auto;"><br>
</p><ul>
	<li>List Elasticsearch Service deployments</li>
</ul><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltced3ff72b160039b/5f4f7826177a662511db2c64/image1.png" data-sys-asset-uid="bltced3ff72b160039b" alt="Code snippet to list Elastic Cloud deployments " style="display: block; margin: auto;">
</p><p>You can also see the API request equivalent to what you’ve configured in the user interface after selecting the required deployment specifications along with the cloud platform and region.
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltabe9e690a7392d5c/5f4f787a891266279cd6e1c4/image3.png" data-sys-asset-uid="bltabe9e690a7392d5c" alt="Equivalent REST API request for the selected Cloud Platform and Region in Elastic Cloud Console" style="display: block; margin: auto;">
</p><p>To see the list of available regions, refer to the <a href="https://www.elastic.co/guide/en/cloud/current/ec-regions-templates-instances.html">available regions page</a> in our documentation.&nbsp;
</p><h2>Further reading</h2><p>Thanks for following along! The code for this project is available in this <a href="https://github.com/aravindputrevu/cloud-sdk-java/">GitHub repository</a>.
</p>If you have any further questions, please do not hesitate to reach out on our community Slack channel #dev-cloud-api (<a href="https://ela.st/slack">request an invite here</a>) or drop a question on our <a href="https://discuss.elastic.co/">forums</a>.

blog-banner-cloud-patterned-blue.png

blog-thumb-cloud-patterned-blue.png

Automate Elastic Cloud workflows using an SDK and Elasticsearch Service API

A start-to-finish Elastic App Search tutorial for creating a search experience with a Python client and containerizing the app to deploy on a cloud platform.

<p>This post is a brief summary of a <a href="https://www.youtube.com/watch?v=gVtQ42Fp_v8">presentation</a> I gave recently where I deploy Elastic App Search, show off the ease of setup, data indexing, and relevance tuning, and take look at a few of the many refined APIs. It’s also <a href="https://aravind.dev/elastic-app-search-python/">written up in a codelab</a> with step-by-step instructions for building a movies search engine app using Python Flask. The app will work on desktop or mobile and is a fast, simple, and reliable way to query the information. Here’s a quick summary of what you’ll learn:
</p><ul>
	<li>Creating an App Search instance on Elastic Cloud</li>
	<li>Navigating to the admin console</li>
	<li>Ingesting data</li>
	<li>Creating a search experience with a Python client</li>
	<li>Containerizing the app and deploying it on a cloud platform</li>
</ul> <strong><h2><strong>A little insight on App Search</strong></h2></strong><p>Elastic App Search is a ready-to-use, complete search solution with user-friendly relevance tuning and analytics built in. It lets users — of almost any skill level — easily add a powerful and customizable search experience to any application, website, or mobile app using a refined set of APIs and management tools. You can deploy App Search instances from the Elastic Cloud dashboard, so you get all the tooling needed for a powerfully relevant search experience with the operational flexibility and scale of Elastic Cloud. It powers search for thousands of popular applications around the world and it's backed by the speed and relevance of <a href="https://github.com/elastic/elasticsearch">Elasticsearch</a>, no matter how large your deployment grows.
</p><p><img clas="shadow" src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltb189d60f5a105ae7/5db0fb016af83b6d7032cd04/illustrated-screenshot-hero-app-search.png" data-sys-asset-uid="bltb189d60f5a105ae7" alt="Elastic App Search UI visualization" style="display: block; margin: auto;"><br>
</p><strong><h2><strong>Starting the App Search tutorial</strong></h2></strong><p>The tutorial uses Elastic Cloud to deploy App Search. If you’re not a current Elastic Cloud user, you can <a href="https://cloud.elastic.co/registration">spin up a free 14-day trial</a>. Then you can jump to the codelab and/or the video presentation to follow along:
</p><ul>
	<li><a href="https://aravind.dev/elastic-app-search-python/">Codelab tutorial</a></li>
	<li><a href="https://www.youtube.com/watch?v=gVtQ42Fp_v8">YouTube video of this guide</a></li>
</ul><p><a href="https://aravind.dev/elastic-app-search-python/"><img class="shadow" src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt02ddbbd5599aa4a4/5f20592ec82cd309f62e4ad6/app-search-python-tutorial-blog.png" style="background-color: initial; display: block; margin: auto;" data-sys-asset-uid="blt02ddbbd5599aa4a4" alt="App Search codelab tutorial"></a>
</p><p>Thanks for following along! Looking for something to read next? Check out <a href="https://www.elastic.co/blog/how-to-build-great-react-search-experiences-quickly">How to build React search experiences quickly</a> and <a href="https://www.elastic.co/blog/what-your-elastic-app-search-analytics-are-telling-you">What your Elastic App Search analytics are telling you</a>.
</p>

app-search-blog-banner.png

app-search-blog-thumb.png

Building a Python web application with Elastic App Search

Our new workspace on Slack's got your back if you get off track with the Elastic Stack.

<p>A few weeks back, <a href="https://twitter.com/elastic/status/1202400283931734016">we announced our community Slack workspace</a>. While we already have great platforms (like <a href="http://discuss.elastic.co/">Discuss</a>&nbsp;and <a href="https://www.elastic.co/elasticon/">Elastic{ON} Tour</a>)&nbsp;and <a href="http://community.elastic.co/">community programs</a> for engaging with our users, this new <a href="https://elasticstack.slack.com/">Slack workspace</a> makes for a lively way for our community to interact with each other, as well as a great new way for users to stay updated with the latest happenings in Elastic Stack.
</p><p>So whether you're looking for help on a Beats module you've been developing, trying to find testers for a new Kibana plugin pet project, or even just trying to find the latest meetup happening in your city, the Slack workspace can be a great place for questions, answers, and community collaboration.
</p><p>Our Slack channels have users joining from all over the world to interact on various products and solutions like #elasticsearch, #app-search, and #logs. And we also have the #usergroup-organizers channel where you can engage with our global community organizers.
</p><p>So if you’re looking to be more active in the Elastic community — and you’ve got room in your channel list — please join the Slack group using this link: <a href="https://ela.st/slack">https://ela.st/slack</a>.
</p><p>For any questions or suggestions please reach out to our #slack-moderators channel.
</p><p><br></p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blta137a2f6e3b0cde7/5e2b6feee147ae4537d91fa6/elastic-slack-blog-welcome-body.jpg" data-sys-asset-uid="blta137a2f6e3b0cde7" alt="" style="display: block; margin: auto; width: 600px">
</p>

Articles by Aravind Putrevu

Running Elastic Cloud on Kubernetes from Azure Kubernetes Service

Automate Elastic Cloud workflows using an SDK and Elasticsearch Service API

Building a Python web application with Elastic App Search

Join our Elastic Stack workspace on Slack

Products & Solutions

Company

Resources

Featured topics

News

Articles by Aravind Putrevu

Running Elastic Cloud on Kubernetes from Azure Kubernetes Service

Automate Elastic Cloud workflows using an SDK and Elasticsearch Service API

Building a Python web application with Elastic App Search

Join our Elastic Stack workspace on Slack