Miguel Luna

Now GA: Managed OTLP Endpoint on Elastic Cloud Hosted

The Elastic Managed OTLP Endpoint is now generally available on Elastic Cloud Hosted, bringing managed Kafka-backed resilience and native OTLP ingestion to any OpenTelemetry shipper.

The Elastic Managed OTLP Endpoint (mOTLP) is now generally available on Elastic Cloud Hosted. Any OTLP-compliant source, whether it's an upstream OpenTelemetry SDK, any Collector distribution, EDOT, or a custom forwarder, can send traces, metrics, and logs to Elastic Cloud without deploying or managing ingestion infrastructure.

mOTLP was already GA on Elastic Cloud Serverless. With this release, Elastic Cloud Hosted deployments get the same managed ingestion path: an OpenTelemetry Collector-based architecture with Kafka-backed resilience that absorbs traffic spikes and protects against data loss during the moments that matter most.

You only need to set two environment variables

From the outside, sending OpenTelemetry data to Elastic using the Managed OTLP Endpoint is as simple as setting up these environment variables:

export OTEL_EXPORTER_OTLP_ENDPOINT="https://<your-motlp-endpoint>"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=ApiKey <your-api-key>"

Two environment variables. That's the entire integration surface. No Collector gateway to deploy, no ingestion pipelines to manage, no credentials to distribute across edge agents.

What makes such a simple design possible is a full OpenTelemetry Collector-based ingestion architecture designed to handle the worst moments in production, from the traffic spike during a deployment rollout to the burst of error traces during an incident. These are the moments when telemetry matters most, and they're also the moments when ingestion is most likely to be overwhelmed.

The managed endpoint receives OTLP data through an OpenTelemetry Collector layer that buffers to a managed Kafka cluster before indexing into Elasticsearch. Kafka absorbs bursts, decouples ingestion from indexing, and provides durability guarantees that in-memory queues can't. If Elasticsearch is temporarily under pressure, data sits in Kafka rather than getting dropped. When pressure subsides, the buffer drains and everything catches up. This is the same resilience pattern you'd build yourself with a Kafka exporter and Kafka receiver in a self-managed collector pipeline, except Elastic operates it for you.

This makes the entire path OpenTelemetry end-to-end:

  1. Your applications produce telemetry with OTel SDKs.
  2. Your collectors (or SDKs directly) export over OTLP.
  3. The managed endpoint receives that OTLP through an OTel Collector-based layer, buffers it through Kafka, and stores it natively in Elasticsearch using the OpenTelemetry data model.

No proprietary protocols, no schema translation, no format conversion at any stage of the pipeline. Just pure OTel.

Bring your OTLP data in, no matter the source

The endpoint accepts standard OTLP over HTTP and gRPC. Any tool that speaks OTLP can send data to it.

This means you can send data from:

  • OpenTelemetry SDKs (upstream, EDOT, or any distribution) exporting directly from your application.
  • OpenTelemetry Collectors (Contrib, EDOT, or custom builds) running as agents, gateways, or sidecars.
  • EDOT Cloud Forwarder forwarding cloud provider logs and metrics.
  • Any OTLP-compliant forwarder you've built or adopted.

This is worth pausing on, because it's not how most vendors work.

Many observability backends require vendor-specific components in your pipeline. Even when those components live in the OpenTelemetry Collector Contrib repository, they often reshape your data on the way out. A vendor-specific exporter might flatten resource attributes, drop the hierarchy between resources and scopes, or translate semantic conventions into a proprietary schema. By the time your telemetry reaches the backend, it's no longer standard OpenTelemetry data. It just started as OpenTelemetry data.

Elastic doesn't require any of that. The managed endpoint ingests standard OTLP, which means you use the standard otlphttp or otlp exporter that ships with every OpenTelemetry Collector. No Elastic-specific exporter, no vendor plugin, no translation layer. Your data arrives in Elasticsearch with the same resource hierarchy, the same semantic conventions, and the same attribute structure your instrumentation produced.

The practical result: teams adopt OpenTelemetry at different speeds and with different tools. Some start with the upstream SDK and a Contrib Collector. Others use EDOT for Elastic-specific optimizations. Many run a mix. The managed endpoint doesn't impose a choice, and it doesn't quietly reshape your data behind a vendor exporter.

For organizations already running OpenTelemetry in production, this means switching to Elastic or adding Elastic as a destination requires changing an exporter URL, not rearchitecting the pipeline or adding vendor-specific components.

What you no longer have to manage

Before the managed OTLP endpoint, sending OpenTelemetry data to Elastic Cloud Hosted required deploying your own OTLP-compatible ingestion layer, typically an EDOT Collector running as a gateway. That gateway needed to be sized, scaled, monitored, and kept available. If it went down, telemetry stopped flowing.

With mOTLP, that entire layer is Elastic's responsibility. Here's what moves off your plate:

  • The endpoint scales with your traffic. On Elastic Cloud Hosted, rate limits scale dynamically based on Elasticsearch backpressure. No pre-provisioning required.
  • The Collector-to-Kafka buffer handles burst absorption and backpressure. You don't need to operate your own Kafka cluster.
  • Your shippers authenticate directly with the endpoint using an API key. No intermediate gateway holding and distributing backend credentials.
  • The endpoint is managed and multi-tenant. You don't need to run redundant Collector replicas or configure health checks for your ingestion layer.

This doesn't mean you should remove all collectors from your architecture. Edge collectors (DaemonSet agents, sidecars, host agents) still serve a purpose: collecting infrastructure telemetry via pull-based receivers like filelog and hostmetrics, applying local transformations, and batching data before export. What changes is that the destination is now a managed endpoint rather than a self-operated gateway.

Dynamic rate scaling: the system adapts to your cluster

On Elastic Cloud Hosted, the managed endpoint doesn't have a fixed throughput ceiling. It uses dynamic rate scaling that adjusts based on your Elasticsearch cluster's capacity and current load.

This is a fundamentally different model from static rate limits. Instead of provisioning for peak and paying for idle capacity, the system continuously calibrates ingestion to what your cluster can actually handle. Sudden load spikes may still trigger temporary 429 responses while the system scales, but these resolve automatically.

If you are seeing consistent 429 errors, the signal is clear, your Elasticsearch cluster needs more capacity. Scaling the cluster reduces backpressure, which in turn increases the ingestion rate limit. The autoscaling capabilities in Elastic Cloud Hosted can help automate this, and AutoOps can assist by monitoring the deployment and providing recommendations to scale or adjust resources when capacity constraints are detected.

Native OTLP storage: OTel from first mile to last

The end-to-end OTel story doesn't stop at ingestion. Data that arrives through the managed endpoint is stored using the OpenTelemetry data model. Resource attributes, semantic conventions, and signal structure are preserved as-is. There's no translation to ECS or any other schema at any point in the pipeline.

This means the attribute names your SDK produces are the same you query in ES|QL and Discover: service.name, http.request.method, k8s.pod.name are stored exactly as the OpenTelemetry specification defines them. You're not debugging a mapping layer or wondering which schema translation dropped an attribute. What your instrumentation emits is what Elasticsearch stores and what you search.

If no specific dataset or namespace is configured, telemetry lands in default data streams: traces-generic.otel-default, metrics-generic.otel-default, and logs-generic.otel-default. You can route logs to dedicated datasets by setting the data_stream.dataset attribute, either in your collector configuration or via OTEL_RESOURCE_ATTRIBUTES:

processors:
  transform:
    log_statements:
      - set(log.attributes["data_stream.dataset"], "app.orders") where resource.attributes["service.name"] == "orders-service"

The failure store: a safety net for mapping conflicts

Even with careful schema design, mapping conflicts happen. A field that's a string in one service might be an integer in another. In traditional setups, these conflicts cause indexing failures and data loss.

The managed endpoint has the failure store enabled by default for all OTLP data streams. Documents that fail indexing due to mapping conflicts or ingest pipeline exceptions are stored in a separate index rather than dropped. You can inspect failed documents from the Data Set Quality page and fix the underlying issue without losing the data.

This is particularly valuable in OpenTelemetry environments where multiple teams instrument independently and attribute types can drift across services.

Getting started

The managed OTLP endpoint is available today on Elastic Cloud Hosted deployments (version 9.0+) in supported regions.

To find your endpoint:

  1. Log in to the Elastic Cloud Console.
  2. Select your deployment and go to Manage.
  3. In the Application endpoints section, select Managed OTLP and copy the public endpoint.

Then point any OTLP exporter at it:

export OTEL_EXPORTER_OTLP_ENDPOINT="https://<your-motlp-endpoint>"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=ApiKey <your-api-key>"

That's it. Traces, metrics, and logs will start flowing into your deployment within seconds.

For a detailed walkthrough, follow the Send data to the Elastic Cloud Managed OTLP Endpoint quickstart.

Learn more

Share this article