EDOT reference architecture

Serverless Observability Stack

The following sections outline the recommended architectures for Elastic Distributions of OpenTelemetry (EDOT) with different Elastic deployment options.

• Hosts and VMs
• Kubernetes

In the context of OpenTelemetry architecture, 'edge' refers to the environment where applications are running: individual hosts, virtual machines, or as daemon sets on Kubernetes nodes. Collectors deployed in these environments perform two primary functions:

1. They gather logs, infrastructure metrics, application traces, profiles, and other telemetry from the local environment.
2. They enrich application telemetry from OpenTelemetry SDKs running on the same host or node with resource metadata, such as host information and Kubernetes attributes.

You can use any OpenTelemetry Collector distribution at the edge, including:

• The contrib OpenTelemetry Collector.
• Custom-built Collector distributions.
• The Elastic Distribution of OpenTelemetry Collector.
• Any shipper capable of sending valid OTLP data.

The only requirement is that collectors deployed on edge environments send data using valid OpenTelemetry Protocol (OTLP) to Elastic Cloud (using managed OTLP endpoint) or to the OTLP receiver of a gateway EDOT collector for self-managed deployments.

While any OTLP-compatible collector works at the edge, using EDOT provides a more streamlined experience with:

• Preconfigured components optimized for Elastic Observability.
• Curated receivers and processors for common use cases.
• Simplified setup and configuration.

The Elastic Observability backend is the ingestion and storage layer where your telemetry data is processed, indexed, and made available for analysis. This includes:

• Elasticsearch: Stores and indexes your telemetry data.
• Kibana: Provides visualization and analysis interfaces.
• Managed OTLP Endpoint (for Elastic Cloud Serverless and Elastic Cloud Hosted): A managed ingestion service that receives OTLP data.
• Gateway Collector (for self-managed, ECE, and ECK): An EDOT Collector running in gateway mode that serves as the ingestion layer.

The need for an EDOT Collector in gateway mode as part of your Elastic backend depends on your deployment type:

Elastic Cloud Serverless and Elastic Cloud Hosted: Edge collectors can send OTLP data directly to the Managed OTLP Endpoint without requiring a self-managed Gateway Collector. The Managed OTLP Endpoint provides a fully managed ingestion layer as part of the Elastic backend. You can optionally deploy an EDOT Collector in gateway mode as part of your edge environment if you need additional processing before data reaches the Managed OTLP Endpoint.

Self-managed, ECE, and ECK deployments: An EDOT Collector in gateway mode is required as part of your Elastic Observability backend. This Gateway Collector exposes the OTLP endpoint that edge collectors send data to, and handles essential preprocessing, including:

• Metrics aggregation for traces and logs, which improves APM UI performance with lower latency.
• Format conversion for optimal storage in Elasticsearch.

For detailed information about Agent and Gateway modes and their specific requirements, refer to the deployment mode documentation.