Loading

Deploy EDOT Cloud Forwarder for Azure

Follow these steps to deploy EDOT Cloud Forwarder for Azure in a resource group to send telemetry data to the Elastic Cloud Managed OTLP Endpoint endpoint.

Before you deploy EDOT Cloud Forwarder for Azure, consider these points:

  • The logs event hub supports Azure resource logs.

  • You can choose between two logs encoding modes:

    • OTel Native (default): encodes logs in the native OpenTelemetry format.
    • ECS: encodes logs into Elastic Common Schema format, compatible with Elastic integrations. Requires the Azure Logs integration.

    For details and examples, refer to Logs encoding modes.

You can deploy EDOT Cloud Forwarder for Azure automatically by clicking the Deploy to Azure button:

Deploy to Azure

This opens the Azure portal with the Bicep template pre-filled. You can then configure the parameters and deploy the template. In the Essentials step, the Encoding section lets you select the logs encoding mode. The template is also available at the following URL: https://ela.st/edot-cf-azure-template

  1. Create the resource group

    Create the resource group that hosts all the resources for EDOT Cloud Forwarder for Azure.

    az group create --name <resource_group_name> --location <azure_region>

  2. Download the Bicep template

    Download the Bicep template to deploy EDOT Cloud Forwarder for Azure at the following URL: https://ela.st/edot-cf-azure-template

  3. Deploy the Bicep template

    Deploy the Bicep template using the following command:

    az deployment group create \
    --resource-group <resource_group_name> \
    --template-file ecf.bicep \
    --parameters \
        otlpEndpoint=<otlp_endpoint> \
        elasticApiKey=<elastic_api_key> \
        logsEncodingExtension=<encoding> \
    1. The OTLP endpoint is the URL of the Elasticsearch OTLP endpoint. Refer to Prerequisites.
    2. The Elastic API key is the API key for authentication. Refer to Prerequisites.
    3. Optional. The logs encoding extension to use. Use azure_encoding (default) for OTel Native format, or beats_encoding for ECS format.

    For a complete list of parameters, refer to Configure EDOT Cloud Forwarder for Azure.

Follow these instructions to collect telemetry data depending on your use case.

Create a diagnostic setting to collect Activity logs from an Azure subscription.

  1. In the Azure portal, open Subscriptions from the search bar.
  2. Select the Azure subscription from which you want to collect Activity logs.
  3. Go to Activity log → Export Activity Logs → Add diagnostic setting.
    • In Diagnostic setting name, enter a name for the diagnostic setting.
    • Select all categories in Logs > Administrative.
    • In Destination details, select Stream to an event hub.
    • In Event hub namespace, select the namespace created by the EDOT Cloud Forwarder (ECF) for Azure.
    • In Event hub name (optional), select the logs event hub created by the EDOT Cloud Forwarder (ECF) for Azure.
  4. Select Save to create the diagnostic setting.
  5. After a few minutes, the diagnostic setting starts streaming the Activity logs to the logs event hub.
  6. Go to Discover in your Elastic Cloud deployment or Serverless project, and select the logs-* data view.
  7. Filter the docs by data_stream.dataset, to browse the Activity logs streamed to Elasticsearch.

Azure platform metrics can be streamed to the metrics event hub using either a Data Collection Rule or a Diagnostic Setting. For details on the differences between the two methods, refer to Metrics collection.

To select the best method for your case, follow this guidance:

  • Use Diagnostic Settings when you need the simplest setup and metric totals are enough.
  • Use a Data Collection Rule when you need dimensional metrics (for example, per-pod or per-namespace views), or when you need more control over metric collection.
  • If your Azure Monitor metric namespace is not available through Diagnostic Settings for a resource type, use a Data Collection Rule.

Create a Data Collection Rule to collect platform metrics from Azure resources.

  1. In the Azure portal, open Data collection rules from the search bar.
  2. Select Create.
  3. In Basics:
    • Turn on the platform metrics by selecting the section labeled To create a Data Collection Rule that collects platform metrics, click here.
    • Enter a Name for the Data Collection Rule.
    • Select or create a new Resource group. You can use the same resource group created by the EDOT Cloud Forwarder for Azure.
    • In Identity, turn on Enable Managed Identity and select System assigned. This is required to stream the metrics to the Event Hub.
  4. In Resources:
    • Select Add resources, then select the Azure resources from which you want to collect metrics.
  5. In Collect and deliver:
    • Select Add new dataflow.
    • From Data source type, select Platform metrics.
    • From Resource types, select the types of resources from which you want to collect metrics.
    • Select Destination.
    • From Destination type, select Event hub, and then select the metrics Event Hub created by the EDOT Cloud Forwarder for Azure.
    • Go to Review and create and select Create.
Note

The Data Collection Rule can take up to 30 minutes, or more, before starting to stream metrics to the Event Hub.

Create a diagnostic setting to collect platform metrics from Azure resources.

  1. In the Azure portal, navigate to the Azure resource from which you want to collect metrics.
  2. Go to Monitoring → Diagnostic settings → Add diagnostic setting.
    • In Diagnostic setting name, enter a name for the diagnostic setting.
    • In Metrics, select AllMetrics.
    • In Destination details, select Stream to an event hub.
    • In Event hub namespace, select the namespace created by the EDOT Cloud Forwarder for Azure.
    • In Event hub name (optional), select the metrics event hub created by the EDOT Cloud Forwarder for Azure.
  3. Select Save to create the diagnostic setting.
  4. After a few minutes, the diagnostic setting starts streaming the platform metrics to the metrics event hub.
Note

Diagnostic Settings metrics do not include dimensions. If you need dimensional data (for example, per-pod or per-namespace breakdowns), use a Data Collection Rule instead. Refer to Metrics collection for details.

After setting up either method:

  1. Go to Discover in your Elastic Cloud deployment or Serverless project, and select the metrics-* data view.
  2. Filter the docs by data_stream.dataset, to browse the platform metrics streamed to Elasticsearch.

To upgrade an existing EDOT Cloud Forwarder for Azure deployment to a newer version, refer to Upgrade EDOT Cloud Forwarder for Azure.

If you no longer need the resources and want to remove them, use the following command:

az group delete --name <resource_group_name>

This removes all the resources in the resource group.

Warning

If you remove the resource group, data that's still unprocessed will be lost.