Following intensive collaboration between Elastic and the OpenTelemetry community, we are thrilled to announce that the OpenTelemetry Profiles signal has officially entered public Alpha. This milestone is a testament to the community's dedication and marks a significant step towards establishing profiling as the fourth key observability signal in OpenTelemetry, alongside logs, metrics and traces.
As a core contributor, Elastic is proud to have accelerated this effort by previously donating its Universal Profiling™ eBPF-based continuous profiling agent to OpenTelemetry. This production-grade agent enables whole-system visibility across all applications, covering a multitude of programming languages and runtimes including third-party libraries and kernel operations with minimal overhead. It allows SREs and developers to quickly identify performance bottlenecks, maximize resource utilization, and optimize cloud spend.
Additionally, over the last two years, Elastic has been heavily contributing to the OpenTelemetry Collector, Semantic Conventions and Profiling Special Interest Groups (SIGs) to lay the technical foundation for the promotion of Profiles to Alpha.
This Alpha milestone not only boosts the standardization of continuous profiling but also accelerates the practical adoption of profiling as the fourth key signal in observability. Customers now have a vendor-agnostic way of collecting profiling data and enabling correlation with existing signals, like logs, metrics and traces, unveiling new potential for observability insights and a more efficient troubleshooting experience.
What is continuous profiling?
Profiling is a technique used to understand the behavior of a software application by collecting information about its execution. This includes tracking the duration of function calls, memory usage, CPU usage, and other system resources.
However, traditional profiling solutions have significant drawbacks limiting adoption in production environments:
- Significant cost and performance overhead due to code instrumentation
- Disruptive service restarts
- Inability to get visibility into third-party libraries
Unlike traditional profiling, which is often done only in a specific development phase or under controlled test conditions, continuous profiling runs in the background with minimal overhead, eliminating the need for service restarts or manual intervention. This provides real-time, actionable insights without replicating issues in separate environments. SREs, DevOps, and developers can see how code affects performance and cost, making code and infrastructure improvements easier.
Elastic's contribution: Powering the Alpha
The Elastic-donated profiler now forms the reference eBPF-based profiler implementation within OpenTelemetry: opentelemetry-ebpf-profiler. With the Alpha release, the eBPF profiler operates as an OpenTelemetry Collector receiver and contains numerous improvements such as automatic Go symbolization and support for new language runtimes. Operating as an OpenTelemetry Collector receiver enables the profiler to seamlessly leverage existing OpenTelemetry processing and filtering pipelines.
For example, the k8sattributesprocessor can use the container.id resource attribute to automatically enrich every profile with its corresponding Kubernetes context.
This means you don't just see a raw stack trace; you see exactly which namespace, pod, and deployment produced it.
receivers:
# Profiling receiver
profiling: {}
processors:
k8sattributes:
passthrough: false
pod_association:
- sources:
- from: resource_attribute
name: container.id
extract:
metadata:
- "k8s.namespace.name"
- "k8s.deployment.name"
- "k8s.replicaset.name"
- "k8s.statefulset.name"
- "k8s.daemonset.name"
- "k8s.node.name"
- "k8s.pod.name"
- "k8s.pod.ip"
- "k8s.pod.uid"
Besides improvements to the eBPF profiler, Elastic has made significant contributions to:
- Correlating profiles with the information produced by OpenTelemetry eBPF instrumentation (OBI), a powerful auto-instrumentation tool that can enable distributed tracing.
- Process Context Sharing OTEP which is designed to bridge the gap between application SDKs and the profiler. This mechanism will allow OpenTelemetry SDKs to "publish" their resource attributes (like
service.name) into a small, standardized memory region. Because this data is stored in the process's own memory map, the eBPF Profiler can instantly discover and associate it with its corresponding Profile. - Semantic conventions and integration of OpenTelemetry Profiles with Google's pprof format (transparent conversion)
- OpenTelemetry Collector processing pipelines, allowing it to better integrate with the profiling receiver
Elastic's Next-Generation Profiling Development
Elastic remains deeply committed to OpenTelemetry's vision and is pushing the boundaries of what is possible with profiling data. We are dedicating a team of profiling domain experts to co-maintain and advance profiling capabilities within OpenTelemetry, while simultaneously working on groundbreaking features built on this new open standard.
Exciting areas of internal profiling-specific development include:
- OpenTelemetry Profiles derived Metrics: We are developing innovative ways to automatically generate actionable performance metrics directly from the raw OTel Profiles data, providing a new dimension for infrastructure modeling and alerting.
- Rapid Integration with the Elastic Stack: We are making swift progress on first-class support for OTLP Profiles within the Elastic Stack, ensuring seamless ingestion (the ebpf-profiler receiver is already integrated with the Elastic Distributions of OpenTelemetry (EDOT) collector), storage, and visualization of this new signal alongside your existing logs, metrics and traces.
- AI-Powered Workflows: We are leveraging the deep insights provided by continuous profiling data to power new AI-driven workflows, enabling automatic root-cause analysis, anomaly detection, and intelligent optimization suggestions for both code and infrastructure.
While the Alpha release marks a significant milestone, it is just the beginning. We encourage the community to start testing early preview versions of the OTel Profiles integration and contribute to the ongoing profiling work. To get started with an actual, local deployment, you can use the OpenTelemetry eBPF profiler in combination with a self-hosted Elastic Observability Stack or devfiler, a standalone desktop application that acts as an OpenTelemetry Profiles compliant backend aimed at experimentation and development.