Introducing Elastic Distribution for OpenTelemetry Python

Sun, 07 Jul 2024 00:00:00 GMT

We are delighted to announce the alpha release of the Elastic Distribution for OpenTelemetry Python. This project is a customized OpenTelemetry distribution that allows us to configure better defaults for using OpenTelemetry with the Elastic cloud offering.

Background

Elastic is standardizing on OpenTelemetry (OTel) for observability and security data collection. As part of that effort, we are providing distributions of the OpenTelemetry Language SDKs. We have recently released alpha distributions for Java, .NET and Node.js. Our Android and iOS SDKs have been OpenTelemetry-based from the start. The Elastic Distribution for OpenTelemetry Python is the latest addition.

Design choices

We have chosen to provide a lean distribution that does not install all the instrumentations by default but that instead provides tools to do so. We leverage the opentelemetry-bootstrap tool provided by OpenTelemetry Python project to scan the packages installed in your environment and recognizes libraries we are able to instrument. This tool can just report the instrumentations available and optionally is able to install them as well. This allows you to avoid installing packages you are not going to need or instrument libraries you are not interested in tracing.

Getting started

To get started with Elastic Distribution for OpenTelemetry Python you need to install the package elastic-opentelemetry in your project environment. We'll use pip in our examples but you are free to use any python package and environment manager of your choice.

pip install elastic-opentelemetry

Once you have installed our distro you'll have also the opentelemetry-bootstrap command available. Running it:

opentelemetry-bootstrap

will list all available packages for your instrumentation, e.g. you can expect something like the following:

opentelemetry-instrumentation-asyncio==0.46b0
opentelemetry-instrumentation-dbapi==0.46b0
opentelemetry-instrumentation-logging==0.46b0
opentelemetry-instrumentation-sqlite3==0.46b0
opentelemetry-instrumentation-threading==0.46b0
opentelemetry-instrumentation-urllib==0.46b0
opentelemetry-instrumentation-wsgi==0.46b0
opentelemetry-instrumentation-grpc==0.46b0
opentelemetry-instrumentation-requests==0.46b0
opentelemetry-instrumentation-system-metrics==0.46b0
opentelemetry-instrumentation-urllib3==0.46b0

It also provides a command option to install the packages automatically

opentelemetry-bootstrap --action=install

It is advised to run this command every time you release a new version of your application so that you can install or just revise any instrumentation packages for your code.

Some environment variables are needed to provide the needed configuration for instrumenting your services. These mostly concern the destination of your traces but also for easily identifying your service. A service name is required to have your service distinguishable from the others. Then you need to provide the authorization headers for authentication with Elastic Observability cloud and the Elastic cloud endpoint where the data is sent.

The API Key you get from your Elastic cloud serverless project must be URL-encoded, you can do that with the following Python snippet:

from urllib.parse import quote
quote("ApiKey )

Once you have all your configuration values you can export via environment variables as below:

export OTEL_RESOURCE_ATTRIBUTES=service.name=
export OTEL_EXPORTER_OTLP_HEADERS="Authorization="
export OTEL_EXPORTER_OTLP_ENDPOINT=

We are done with the configuration and the last piece of the puzzle is wrapping your service invocation with opentelemetry-instrument, the wrapper that provides zero-code instrumentation. Zero-code (or Automatic) instrumentation means that the distribution will set up the OpenTelemetry SDK and enable all the previously installed instrumentations for you. Unfortunately Zero-code instrumentation does not cover all libraries and some — web frameworks in particular — will require minimal manual configuration.

For a web service running with gunicorn it may look like:

opentelemetry-instrument gunicorn main:app

The result is an observable application using the industry-standard OpenTelemetry — offering high-quality instrumentation of many popular Python libraries, a portable API to avoid vendor lock-in and an active community.

Using Elastic Observability, some out-of-the-box benefits you can expect are: rich trace viewing, Service maps, integrated metrics and log analysis, and more.

What's next?

Elastic is committed to helping OpenTelemetry succeed and to helping our customers use OpenTelemetry effectively in their systems. Last year, we donated ECS and continue to work on integrating it with OpenTelemetry Semantic Conventions. More recently, we are working on donating our eBPF-based profiler to OpenTelemetry. We contribute to many of the language SDKs and other OpenTelemetry projects.

In the Python ecosystem we are active reviewers and contributors of both the opentelemetry-python and opentelemetry-python-contrib repositories.

The Elastic Distribution for OpenTelemetry Python is currently an alpha. Please try it out and let us know if it might work for you. Watch for the latest releases here. You can engage with us on the project issue tracker.

We are eager to know your use cases to help you succeed in your Observability journey.

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.

Elastic Observability Labs - Articles by Riccardo Magliocchetti

Introducing Elastic Distribution for OpenTelemetry Python

Background

Design choices

Getting started

What's next?

Resources