Loading

Deploy EDOT Cloud Forwarder for Azure

Follow these steps to deploy EDOT Cloud Forwarder for Azure in a resource group to send telemetry data to the Elastic Cloud Managed OTLP Endpoint endpoint.

  1. Retrieve the OTLP endpoint and API key

    To find out the URL of the managed OTLP endpoint and the API key for authentication, follow these steps:

    Steps to retrieve the OTLP endpoint and API key

    To retrieve your Elastic Cloud Managed OTLP Endpoint endpoint address and API key, follow these steps:

    1. In Elastic Cloud, create an Observability project or open an existing one.
    2. Go to Add data, select Applications and then select OpenTelemetry.
    3. Copy the endpoint and authentication headers values.

    Alternatively, you can retrieve the endpoint from the Manage project page and create an API key manually from the API keys page.

    You need an Elastic Cloud Hosted deployment version 9.2 or later.

    1. In Elastic Cloud, create an Elastic Cloud Hosted deployment or open an existing one.
    2. Go to Add data, select Applications and then select OpenTelemetry.
    3. Copy the endpoint and authentication headers values.

    In the Bicep templates, the OTLP endpoint is set as elasticsearchOtlpEndpoint, and the API key is set as elasticsearchApiKey.

    Important

    Trim the API key from Authorization=ApiKey MYKEYVALUE... to just MYKEYVALUE... before using it as the argument to the elasticsearchApiKey parameter.

  2. Set the environment variables

    Define the following environment variables:

    export AZURE_LOCATION="<your_region>"
export RESOURCE_GROUP="<group_name>"
export ELASTICSEARCH_OTLP_ENDPOINT="<your_otlp_endpoint>"
export ELASTICSEARCH_API_KEY="<your_api_key>"

  3. Create the resource group

    Create the resource group that hosts all the resources for EDOT Cloud Forwarder for Azure.

    az group create --name $RESOURCE_GROUP --location $AZURE_LOCATION

    Example output:

    {
  "id": "/subscriptions/<subscription_id>/resourceGroups/<group_name>",
  "location": "<your_region>",
  "managedBy": null,
  "name": "<group_name>",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": "Microsoft.Resources/resourceGroups"
}

  4. Download the Bicep template

    Download the Bicep template to deploy EDOT Cloud Forwarder for Azure at the following URL: https://ela.st/edot-cf-azure-template

  5. Deploy the Bicep template

    Deploy the Bicep template using the following command:

    az deployment group create \
    --resource-group ${RESOURCE_GROUP} \
    --template-file ecf.bicep \
    --parameters \
        elasticsearchOtlpEndpoint=$ELASTICSEARCH_OTLP_ENDPOINT \
        elasticsearchApiKey=$ELASTICSEARCH_API_KEY \
        logsDecoder=ds \
        metricsDecoder=dcr \
        eventHubPartitionCount=8 \
        eventHubMessageRetentionInDays=1
    1. To collect Activity logs from a diagnostic setting, set logsDecoder to ds (diagnostic settings).
    2. To collect Storage Account metrics from a Data Collection Rule, set metricsDecoder to dcr (data collection rules).

    For a complete list of parameters, refer to Configure the template.

Follow these instructions to collect telemetry data depending on your use case.

Create a diagnostic setting to collect Activity logs from an Azure subscription.

  1. In the Azure portal, open Subscriptions from the search bar.
  2. Select the Azure subscription from which you want to collect Activity logs.
  3. Go to Activity log → Export Activity Logs → Add diagnostic setting.
    • In Diagnostic setting name, enter a name for the diagnostic setting.
    • Select all categories in Logs > Administrative.
    • In Destination details, select Stream to an event hub.
    • In Event hub namespace, select the namespace created by the EDOT Cloud Forwarder (ECF) for Azure.
    • In Event hub name (optional), select the logs event hub created by the EDOT Cloud Forwarder (ECF) for Azure.
  4. Select Save to create the diagnostic setting.
  5. After a few minutes, the diagnostic setting starts streaming the Activity logs to the logs event hub.
  6. Go to Discover in your Elastic Cloud deployment or Serverless project, and select the logs-* data view.
  7. Filter the docs by data_stream.dataset, to browse the Activity logs streamed to Elasticsearch.

If you no longer need the resources and want to remove them, use the following command:

az group delete --name $RESOURCE_GROUP

This removes all the resources in the resource group.

Warning

If you remove the resource group, data that's still unprocessed will be lost.