OpenTelemetry integrationedit

Elastic’s OpenTelemetry integration allows you to reuse your existing OpenTelemetry instrumentation to quickly analyze distributed traces and metrics with the Elastic Stack.

What is OpenTelemetry?edit

OpenTelemetry is a set of APIs, SDKs, tooling, and integrations that enable the creation and management of telemetry data. It formed through a merger of the OpenTracing and OpenCensus projects.

OpenTelemetry is an open-source project that provides the components necessary to observe your applications and services. If you’re unfamiliar with the project, see the spec for more information.

Elastic exporteredit

Elastic’s integration is designed to drop into your current OpenTelemetry setup. We’ve done this by extending the "contrib" OpenTelemetry collector and adding an Elastic exporter. This exporter translates the OpenTelemetry trace data collected from your services to Elastic’s protocol, before sending the data to the Elastic Stack. By extending the OpenTelemetry collector, no changes are needed in your instrumented services in order to begin using the Elastic Stack.

OpenTelemetry Elastic architecture diagram

How the OpenTelemetry Collector worksedit

The OpenTelemetry collector uses three different types of components to handle data: receivers, processors, and exporters.

  • receivers: Configure how data gets to the collector. At least one receiver must be configured.
  • processors: Defines optional transformations that occurs between receiving and exporting data.
  • exporters: Configures how data is sent to its destination—​in this case, the Elastic Stack.

Once a receiver, processor, and exporter is defined, pipelines can be configured in the services section of your configuration. Specifically, a traces pipeline will define the path of trace data through your collector, and bring all three of these components together.

More information is available in the OpenTelemetry pipeline docs

A final note: extensions can also be enabled for tasks like monitoring your collectors health. See the OpenTelemetry extension readme for a list of supported extensions.

Get startededit

This guide assumes you’ve already instrumented your services with the OpenTelemetry API and/or SDK. If you haven’t, see the Elastic APM install and run guide to get started with Elastic APM Agents instead.

Plan your deploymentedit

OpenTelemetry Collectors can be run as an Agent, or as standalone collectors. They can be deployed as often as necessary and scaled up or out.

Deployment planning resources are available in OpenTelemetry’s Getting Started documentation, and Collector Performance research.

Download the collectoredit

The Elastic exporter lives in the opentelemetry-collector-contrib repository, and the latest release can be downloaded from GitHub releases page.

Docker images are available on dockerhub:

docker pull otel/opentelemetry-collector-contrib-dev

You can also build the collector-contrib repository by cloning it and running:

make otelcontribcol

Configure the collectoredit

Create a yaml configuration file.

At a minimum, you must define the URL of the APM Server instance you are sending data to. For example:

exporters:
  elastic:
    apm_server_url: "https://elasticapm.example.com"

See the configuration reference for additional configuration options, like specifying an API key, secret token, or TLS settings.

The Elastic exporter must also be defined in service.pipelines.traces.exporters. For example:

service:
  pipelines:
    traces:
      exporters: [elastic]

If we put everything together, here’s an example configuration file that accepts input from an OpenTelemetry Agent, processes the data, and sends it to an Elasticsearch Service instance.

receivers:
  otlp:
    endpoint: localhost:55680
processors:
  batch:
    timeout: 1s
    send_batch_size: 1024
exporters:
  elastic:
    apm_server_url: "https://elasticapm.example.com"
    secret_token: "ESS_TOKEN"
service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [elastic]

For more information about getting started with an OpenTelemetry Collector, see the OpenTelemetry collector docs.

Elastic exporter configuration referenceedit

apm_server_urledit

Elastic APM Server URL. (required)

api_keyedit

Credential for API key authorization. Must also be enabled in Elastic APM Server. (optional)

secret_tokenedit

Credential for secret token authorization. Must also be enabled in Elastic APM Server. (optional)

ca_fileedit

Root Certificate Authority (CA) certificate, for verifying the server’s identity if TLS is enabled. (optional)

cert_fileedit

Client TLS certificate. (optional)

key_fileedit

Client TLS key. (optional)

insecureedit

Disable verification of the server’s identity if TLS is enabled. (optional)