Native Azure integrationedit

This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.

The Elastic Cloud native Azure integration allows you to deploy managed instances of the Elastic Stack directly in Azure, through the Azure integrated marketplace. The integration brings the following benefits:

  • Easy deployment for managed Elastic Stack instances

    Elastic Stack instances managed by Elastic can be deployed directly from the Azure console. This provides the complete Elastic Stack experience with all commercial features.

  • Integrated billing

    You are billed directly to your Azure account; no need to configure billing details in Elastic.

  • Easy consolidation of your Azure logs in Elastic

    Use a one-click setup to ingest logs from your Azure services into the Elastic Stack.

See the following sections to learn more:

Getting startededit

How do I get started with Elastic Cloud?

Elastic Cloud is available as an offering through the Azure console.

Prerequisites

Before setting up an Elastic Cloud deployment, make sure that these two requirements are met:

  • To deploy Elasticsearch, you first must be configured in Azure as a subscription Owner. See Assign a subscription administrator in the Azure documentation for steps to assign an Owner role.
  • When you create an Elasticsearch resource in Azure you cannot use an email address that already has an Elastic Cloud account. If your Azure account address is currently used for an Elastic Cloud account, either use a different Azure account to set up the Elasticsearch resource, or contact the Elastic Support Team for assistance.

Getting started

Once you’re configured in Azure as a subscription owner, you can access Elastic Cloud in two ways:

  1. Search for elastic in the Azure Search bar. On the results page, the Elasticsearch link under the Services menu will open a page that guides you through deploying the Elastic Stack.

    Search results showing Elasticsearch at the top of a list under Services
  2. Navigate to the Azure Marketplace page. There, you’ll see the Elastic native integration listed as Elasticsearch (Elastic Cloud) - Preview. Click the link and follow the on-screen steps to deploy the Elastic Stack.

When you create a deployment, an Elastic Stack cluster is created for you. The size of this deployment is 16GB of RAM and 480GB of storage, across two availability zones for redundancy. The size of the deployment, both RAM and storage, can be changed directly in the Elastic console. Usage charges are based on the size of your deployment so ensure that your instance is sized efficiently. The deployment defaults to the latest available version of the Elastic Stack. See our Version policy to learn more about when new versions are made available and old versions are removed from service.

What is the pricing for this offer?

Pricing is pay-as-you-go per hour for each Elastic Stack deployment that you have created. Note that there is no free trial period for the offering. Charges are applied to your Azure bill at the end of the month. You can use the Elastic Cloud Pricing Calculator to size a deployment and view the corresponding hourly rate.

Elastic charges include:

Which Azure regions are supported?
Here is the list of available Azure regions supported in Elastic Cloud.
Which Elastic Cloud subscription levels are available?
Elastic Cloud offers a number of different subscription levels. Your Elastic Cloud subscription will default to the Platinum subscription, giving you immediate access to advanced Elastic Stack features like machine learning, as well as premium support response time SLAs.
How can I change my Elastic Cloud subscription level?

You can modify your subscription level on the billing page in the Elastic console.

  1. Open the Elastic Stack list deployments page in Azure. From here, you can view any existing Elastic Stack deployments that you have created.
  2. Click on a deployment to open the deployment overview page.
  3. Click the Advanced Settings link to access your deployment in the Elastic Cloud console.

    The deployment overview page in Azure with the change my subscription link highlighted
  4. In the Elastic Cloud console, click your account avatar icon at the top of the page, and then select Account & Billing.
  5. Select the Billing tab and click Change my subscription.

    The Elastic Account Billing page with Advanced Settings highlighted
  6. Select the subscription level that you’d like.

    The Update Subscription page showing Standard
Is the Elasticsearch native integration connected with Azure user management?

No. Elastic is not currently integrated with Azure user management. Azure users who deploy Elasticsearch on Azure can view and manage their own cluster through the Cloud console, but other Azure users in the same tenant cannot access clusters through the Cloud console other than those that they themselves created.

In the Cloud console, when you try to access resources such as Elasticsearch, Kibana, Enterprise Search, or APM in a deployment that was created by another Azure user, the following error is shown:

Error message displayed in the Elastic Cloud console: To access the resource {resource-name}

However, you can share deployment resources directly with other Azure users by configuring Active Directory single sign-on with the Elasticsearch cluster.

I already have an Elastic Cloud account, can I use this integration?
Yes. If you already have an Elastic Cloud account with the same email address as your Azure account you may need to contact support@elastic.co.
Can I sign up for an Elastic Cloud trial account and then convert to the Azure Elasticsearch native integration?

Yes. You can start a free Elasticsearch Service trial and then convert your account over to Azure. There are a few requirements:

  • Make sure that when you create deployments in the trial account you specify Azure as the cloud provider.
  • To convert your trial to the Azure marketplace you need to create a deployment in the Azure console. You can just delete the new deployment if you don’t need it. After you create the new deployment your marketplace subscription will be ready.
  • Any deployments that you create during your trial won’t show up in the Azure console, since they weren’t created in Azure, but they are still accessible through the Elasticsearch Service Console and you will be billed for their usage.
Does Elasticsearch get deployed into my tenant in Azure?
No. Elasticsearch resources get deployed in an Azure tenant that is managed by Elastic. This tenant, and the management capabilities associated with it, are the same that are used to run Elastic’s managed service, which also allows users to deploy on Azure.
Can I deploy Elasticsearch using the Azure CLI, REST API, or an ARM Template?

Currently, Elasticsearch cannot be deployed on Azure using these methods. You can use any of the following methods instead:

Managing your Elastic Stack deploymentedit

What is included in my Elastic Stack deployment?

Each Elastic Stack deployment includes a number of Elastic Stack components:

  • An Elasticsearch cluster
  • A Kibana instance which provides data visualization and a front-end for the Elastic Stack
  • An APM server that allows you to easily collect application traces
  • An Enterprise Search instance that allows you to easily build a search experience with an intuitive interface
How can I access my Elastic Stack deployment?

You can access your Elastic Stack deployment in a few ways. First, you need to navigate to the deployment overview page in Azure:

  1. Open the Elastic Stack list deployments page in Azure. From here, you can view any existing Elastic Stack deployments that you have created.
  2. Click on a deployment to open the deployment overview page.

    The deployment overview page in Azure

    You now have a few options to access your deployment:

    • Elasticsearch endpoint - the URL for the Elasticsearch cluster itself
    • Kibana endpoint - the UI for the Elastic Stack, a great way for new users to get started
    • Elastic Cloud - Click the Advanced Settings link to access your deployment in the Elastic Cloud console, where you can make changes such as resizing or upgrading your deployment.
How can I modify my Elastic Stack deployment?

You can modify your Elastic Stack deployment in the Elastic Cloud console, which you can access from the Azure UI through the Advanced Settings link on the deployment overview page. In the Elastic Cloud console you can perform a number of actions against your deployment, including:

How can I delete my Elastic Stack deployment?

You can delete your deployment directly from the Azure console. The delete operation, when evoked from Azure, will perform clean-up activities in the Elastic console to ensure any running components are removed, so that no additional charges occur.

The deployment overview page in Azure with the Delete button highlighted

Configuring logs and metricsedit

How do I monitor my existing Azure services?

The Elastic Cloud native Azure integration greatly simplifies logging for Azure services with the Elastic Stack. This integration supports the easy setup of Azure platform logs, including:

The following log types are not supported as part of this integration:

  • Azure tenant logs
  • Logs from Azure compute services, such as Virtual Machines

In the Azure console, you can configure ingestion of Azure logs into either a new or existing Elastic Stack deployment:

  • When you create a new deployment, from the Logs & metrics tab in Azure you can specify the log type and a key/value tag pair. Any Azure resources that match on the tag value will automatically send log data to the Elastic Stack deployment, once it’s been created.
The Logs & Metrics tab on the Create Elastic Resource page
  • For existing deployments, you can configure Azure logs from the deployment overview page in the Azure console.

Note that following restrictions for logging:

  • Only logs from non-compute Azure services are ingested as part of the above configuration. Logs from compute services, such as Virtual Machines, into the Elastic Stack will be added in a future release.
  • The Azure services must be running in one of the following regions:

    • Eastus
    • Eastus2
    • Westus2
    • Centralus
    • Southcentralus
    • Uksouth
    • Francecentral
    • Japaneast
    • Southeastasia
    • Westeurope
    • Australiaeast
    • Northeurope

All regions will be supported in the future.

To see which of your Azure resources are currently being monitored, navigate to your Elasticsearch deployment and open the Monitored resources tab. Each resource shows one of the following status indicators:

  • Sending - Logs are currently being sent to the Elasticsearch cluster.
  • Logs not configured - Log collection is currently not configured for the resource. Click the Edit tags link to configure which logs are collected. For details about tagging resources, see Use tags to organize your Azure resources and management hierarchy in the Azure documentation.
  • N/A - Monitoring is not available for this resource type.
  • Limit reached - Azure resources can send diagnostic data to a maximum of five outputs. Data is not being sent to the Elasticsearch cluster because the output limit has already been reached.
  • Failed - Logs are configured but failed to ship to the Elasticsearch cluster. For help resolving this problem you can contact Support.
  • Region not supported - The Azure resource must be in one of the supported regions.
How do I ingest metrics from my Azure services?
Metrics are not supported as part of the current native integration. This will be implemented in a future phase. Metrics can still be collected from all Azure services using Metricbeat. For details, see Ingest other Azure metrics using the Metricbeat Azure module.
How can I monitor my Azure virtual machines in Elasticsearch?

You can monitor your Azure virtual machines by installing the Elastic Agent VM extension. Once enabled, the VM extension downloads the Elastic Agent, installs it, and enrols it to the Fleet Server. The Elastic Agent will then send system related logs and metrics to the Elastic Cloud cluster where you can find pre-built system dashboards showing the health and performance of your virtual machines.

A dashboard showing system metrics for the VM

Enabling and disabling VM extensions

To enable or disable a VM extension:

  1. In Azure, navigate to your Elasticsearch deployment.
  2. Click the Virtual machines tab
  3. Select one or more virtual machines
  4. Click Install Extension or Uninstall Extension.
The Virtual Machines page in Azure

While it’s possible to enable or disable a VM extension directly from the VM itself, we recommend always enabling or disabling your Elasticsearch VM extensions from within the context of your Elasticsearch deployment.

Managing the Elastic Agent VM extension

Once installed on the virtual machine, you can manage Elastic Agent either from Fleet or locally on the host where it’s installed. We recommend managing the VM extension through Fleet, because it makes handling and upgrading the agents considerably easier. For more information on the Elastic Agent, see Manage your Elastic Agents.

Operating system compatibility matrix

The Azure Elastic Agent VM extension is supported on the following operating systems:

Platform Version

Windows

2008r2+

CentOS

6.10+

Debian

9,10

Oracle

6.8+

RHEL

7+

Ubuntu

16+

Troubleshootingedit

This section describes some scenarios that you may experience onboarding to Elastic Cloud through the Azure console. If you’re running into issues you can always get support.

I receive an error message about not having the required authorization.

When trying to access Elastic Cloud resources, you may see an error message indicating that the user must have the required authorization.

Error message displayed in the Elastic Cloud console: To access the resource {resource-name}

Elastic is not currently integrated with Azure user management, so sharing deployment resources through the Cloud console with other Azure users is not possible. However, sharing direct access to these resources is possible. For details, check Is the Elasticsearch native integration connected with Azure user management?.

Getting supportedit

How do I get support?

Support is provided by Elastic. To open a support case:

  1. Navigate to the deployment overview page in the Azure console.
  2. Click the Support tab.
  3. Click on the link to launch the Elastic console where you can provide further details.

    A member of the Elastic Support team will respond based on the SLA response time of your subscription.

    The New Support Request page in Azure

In case your Elastic Cloud resource is not fully set up and you’re not able to access the Support page, you can always send an email to support@elastic.co.

How can I change my subscription level / support level?
Your Elastic subscription level includes the support level. See How can I change my Elastic Cloud subscription level? to make an update.