SRE manager’s guide to OpenTelemetry

Management of your ever-evolving digital environment, stretched across multiple clouds and on-premises, is constantly expanding with data. Approximately 402.74 million terabytes of data are created each day. It's estimated that 90% of the world's data was generated in the last two years alone¹.

But that operational data often isn't consistent. Each component in your system can emit a telemetry format, resulting in mixed signals. Inconsistent data formats are a major challenge for SREs. The variety of data stream types and quality levels makes understanding what's working and what's not working in your systems harder than ever. Traditional monitoring tools and systems don't cut it anymore, and this is where a modern, unified observability platform comes in.

Observability answers the need to see your ecosystem holistically and correlate data across functional teams and through data silos to obtain meaningful insights that guide action. With an effective observability solution in place, developers can innovate faster while maintaining high reliability, and business leaders can make informed decisions that boost revenue. In this way, modern observability is key to operational resilience — and business success. However, effective observability is only as good as the data it relies on.

Consistent, high-quality telemetry data is key to a modern observability solution. However, generating and transmitting this type of telemetry consistently would require a massive and unsustainable engineering effort that goes beyond coding. Developers would have to align around protocols and semantic conventions for tags, attributes, and other metadata, looking down the barrel of intense labor for weeks on end. So, what's the alternative? Turnkey telemetry and open standards offer fast time-to-value, enable setting reasonable defaults that allow customization, pair with quality documentation, and most importantly, offer telemetry data that doesn't lock users in with a vendor. Why? Because efficient, future-proof observability requires that you own the data you generate and that you learn only a single set of APIs, data standards, and conventions.

Enter OTel, whose mission is to enable effective observability by making high-quality, portable telemetry the new standard.

As observability gained traction to meet the requirements of cloud computing, the need for a code instrumentation and telemetry data standard became apparent. Google's 2010 Dapper Paper chronicles its journey to creating Dapper, a predecessor of OTel that aimed to consolidate monitoring tools and simplify DevOps processes.

Google's developers weren't the only ones who saw a gap in the data ecosystem. The Cloud Native Computing Foundation (CNCF) projects, OpenTracing and OpenCensus, were trying to solve this problem. Unable to do so independently, they merged to form a single solution: OpenTelemetry.

Accepted by CNCF in early 2019, the OTel project has since matured into the incubation stage, following in the steps of projects like Kubernetes and Prometheus (with which OTel also collaborates — more on this later).

Since its inception, OTel has grown based on the principles that telemetry data should be:

1. Universal: Unified telemetry protocols and conventions across languages and signals — metrics, logs, traces — are key to effective modern observability.
2. Easy: The goal is to simplify telemetry for the end user with quick value outcomes and customizations that are supported by thorough documentation.
3. Vendor-neutral: Proprietary monitoring agents that lock users in, no more. As an open source project, OTel enables flexibility and innovation.
4. Loosely coupled: End users should be able to pick and choose which parts of the telemetry pipeline they need. This modularity is essential for flexibility, adaptability, and scalability.
5. Built-in: Telemetry doesn't have to be added after the fact — it should be built into the entire software stack.

These principles remain the foundation of OTel — and the source of its benefits.

At a glance, OpenTelemetry offers organizations improved observability practices through its standardized data collection and future-proof flexibility. The result? Increased efficiency and lower operational costs over the long term.

But what does that look like for your operations and your systems? OTel is the gateway to:

• Unified data collection: OTel offers a single API and SDK for collecting metrics, traces, and logs. This simplifies the integration process, reduces the need for multiple data collection tools, and minimizes data transformation work.
• Improved troubleshooting: By providing consistent telemetry data, OTel makes it easier to troubleshoot issues in distributed systems. Developers can quickly correlate metrics, traces, and logs to identify the root cause of problems more effectively.
• Future-proofed observability: OTel’s open standards protect organizations from being tied to specific monitoring vendors, allowing flexibility in choosing observability back ends. This ensures a future-proof solution that can adapt to changing requirements.
• Increased team efficiency: SRE teams benefit from streamlined instrumentation, reducing the complexity of managing multiple tools. This leads to improved response times for incidents, faster problem resolution, and greater overall productivity.

In a 2025 study by EMA, improved IT Operations and developer productivity, simplified metrics collection, and greater visibility into software and systems were cited as top observability benefits.

Current data trends indicate that the amount of data produced and processed by organizations will continue to grow. In fact, it's set to grow faster than ever before. Making sense of this data now is crucial to operations; making sense of the data in the future requires technological foresight. Open standards are key building blocks in observability architectures that will withstand the test of time — and the breakneck pace of innovation. OpenTelemetry's key tenet is in its name: open standards. In other words, no vendor lock-in.

Let's look back at the adoption curve, where OTel has secured its spot in the early adoption wave. This also means that early adopters are gaining a competitive advantage: By choosing a vendor-agnostic telemetry data framework that feeds into their observability solutions, early adopters are not only future-proofing their operations, they're also making strides to improve their operational resilience from the ground up. In an increasingly volatile world, achieving operational resilience, at least from a technology perspective, is paramount to business success. In other words, embracing OTel prepares organizations for the data demands of the future with no strings attached and without the limitations of their current technologies. It's a valuable leg up on the competition in the short and long term.

Since 2019, OTel has grown exponentially. It has added 9,160+ contributors, 55,640+ code commits, 67,250+ pull requests, 466,000+ contributions, and 1,100+ contributing companies — and it’s still just getting started.

Concerns about OTel’s maturity stem from the need to understand its reliability.

OTel develops signals like metrics, traces, logs, and baggage incrementally, building them on top of context propagation. Every signal consists of APIs, SDKs, OpenTelemetry Protocols (OTLPs), and a Collector, and every component cycles through draft, experimental, stable, and deprecated phases.

The OTel tracing specification is now completely stable, while metrics and logging are currently used in production with one component in development: SDK and event API, respectively.4 From the data standpoint (Semantic Conventions), the traces, metrics, and resources Semantic Conventions are defined as Mixed for status. Meanwhile, the Logs Semantic Conventions is trailing with a status of In Development. But not to worry, the Semantic Conventions are always evolving for new use cases while remaining highly stable for most standard observability scenarios.

Despite the evolving nature of OTel, industry sentiment is actually largely positive. According to the 2025 EMA survey, over 50% of respondents felt OpenTelemetry was mature enough to implement. However, an overwhelming 90% felt that OpenTelemetry was moderately important, very important, or critical. Like any developing industry standard, the direction of growth is key to understanding the underlying industry adoption trends.

Several tools within the CNCF ecosystem are gaining traction as part of the broader OpenTelemetry landscape:

1. Open Agent Management Protocol (OpAMP): OpAMP simplifies the management of telemetry agents across distributed environments at scale. It provides centralized control over agent configurations and facilitates monitoring and diagnostics, streamlining the way organizations handle their observability agents. This is particularly beneficial for large-scale deployments where managing thousands of agents manually can be daunting.
2. Profiling: A recent contribution by Elastic, profiling allows developers to understand the behavior of software applications by collecting information about its execution. This feature can be invaluable for diagnosing bottlenecks, improving application performance, maximizing resource utilization, reducing carbon footprint, and optimizing cloud spend.
3. Prometheus: While Prometheus was already a widely adopted tool for metrics collection, its integration with OpenTelemetry further strengthens OTel’s capabilities in metrics monitoring. Prometheus serves as a back end for OpenTelemetry metrics, providing deep insights into application and infrastructure performance.

Open standards ensure transparency and adaptability, and unified signals go one step further: They simplify data storage and processing, helping organizations improve security and meet regulatory requirements. Many compliance frameworks (PCI DSS, SOC 2, GDPR) require detailed observability and logging for future audits.

Data uniformity is an invaluable piece of security monitoring, a landscape that has progressively become more treacherous for organizations worldwide. Traces let security teams identify how a request moves through their systems after an incident, while logs and traces help reconstruct attack timelines and identify affected components. As a key part of a modern observability solution, OTel gives organizations visibility into their systems to handle security vulnerabilities quickly and effectively — or even prevent threats before they evolve. When OpenTelemetry is used for CI/CD pipelines and DevSecOps tools to monitor deployments, unauthorized changes can be detected, protecting code before it is even pushed to production.

Unified telemetry data can reduce MTTR and position security teams proactively with clearer visibility. As OTel evolves, it also ensures compliance for your whole system, updating to new industry standards over time.

Complex problems require examination from multiple vantage points. And complex problems are the norm in complex digital environments. The ability to collect and correlate data from multiple sources simultaneously — and efficiently — relies on the standardization of signals.

By standardizing tracing, metrics, and logs, OTel paves the way to eliminating data silos that hinder effective observability. OTel and standardization of observability data paves the way for easy correlation between signals through common metadata and increases the effectiveness of ML/AI to provide actionable insights in increasingly complex environments.

So, OTel not only enables faster anomaly detection and root cause analysis, it also serves as a foundation for proactive site reliability and risk management.

In fact, the EMA survey report found a clear correlation between observability maturity and OpenTelemetry adoption — 61% of respondents with a mature observability practice are already using OpenTelemetry, while only 1% are not.

In an increasingly volatile world, operational resilience is key to business continuity and OTel is the missing piece in the modern observability framework that safeguards operations and promotes innovation.

OTel deployment varies based on the needs of your organization, and three official deployment patterns exist: no collector, agent-based, and gateway.

No collector deployment

By understanding the state of your current observability solution, you can pinpoint which OTel components you need. Your current system and your goals determine which solution will be most impactful. When assessing your technology stack, you'll want to:

• Determine what monitoring and logging solutions are already in place, such as Prometheus, Grafana, Jaeger, or other APM tools.
• Identify areas where your current setup lacks visibility, such as missing traceability across services or inefficient log correlation.

A thorough assessment of your current observability setup will allow you to establish clear integration goals. As with any tech migration, starting small and specific is key.

Two primary paths exist on the OTel integration journey: code-based and zero-code instrumentation.

Instrumenting OTel begins with identifying your why. Do you develop software? Are you looking to get observability by writing code? Is your goal to have your dependencies automatically emit telemetry data? If yes, then code-based instrumentation is your way forward. (Start out with zero-code and then add custom code/manual instrumentation as needed.)

If instead, you’re looking to get telemetry data out of the applications you run without touching their code and send this telemetry data to your observability back end, then zero-code instrumentation is for you.

Now that you have a better understanding of your options, it's time to build a migration plan.

1. Select initial services for OTel integration: Begin with a pilot project focusing on your most critical services. This approach allows you to identify and resolve implementation issues before rolling out OpenTelemetry across your entire infrastructure.
2. Choose deployment method: Decide whether you will use OTel's auto-instrumentation capabilities, manual instrumentation, or a combination of both. Begin with basic instrumentation and gradually increase coverage based on actual needs rather than theoretical possibilities.
3. Map out data flow: Define how telemetry data will be collected, processed, and exported to observability back ends.
4. Address security and compliance: Ensure that the collection and transmission of telemetry data comply with security and regulatory requirements.
5. Establish success metrics: Define what success looks like by setting clear, measurable objectives for your OTel migration. Identify key performance indicators (KPIs) such as improved trace coverage, reduced mean time to detection (MTTD) and resolution (MTTR), lower observability costs, or increased system reliability.
6. Implement performance benchmarking: Before deploying OpenTelemetry agents, establish baseline performance metrics for your services. Monitor these metrics closely as you add instrumentation to identify and address any performance impact quickly. Some commonly seen areas where performance issues may arise would be database performance, excessive disk I/O operations and bottlenecks, along with high CPU loads.
7. Create service-specific instrumentation strategies: Different services may require different approaches to instrumentation. Develop clear guidelines for each service type, considering factors like traffic volume, criticality, and resource constraints. This targeted approach helps prevent over-instrumentation while ensuring adequate visibility where it matters most.

Leadership talking points

As with any major technological shift, strong top-down support is critical to success. Consider these arguments when pitching OpenTelemetry to the C-suite:

In the 2025 EMA survey report, over 60% of respondents reported IT operations and developer productivity increased by over 20%. OpenTelemetry helps immensely with the management of Kubernetes and cloud deployments due to benefits such as simplification of telemetry data collection, enhanced visibility across microservices, and improved troubleshooting and RCA.

OTel moves observability beyond logs. It is a low- to no-cost deployment, enables companies that are tracing to do more of it, increases correlation accessibility, and standardizes naming across the board. This results in a more structured observability framework and, as a vendor-agnostic solution, OTel not only reduces vendor lock-in but also maximizes ROI by delivering deeper insights with greater efficiency.

Successful adoption of OpenTelemetry requires that teams are adequately trained in its core concepts, tooling, and best practices. As part of your OTel migration efforts, consider:

1. Workshops: Conduct practical workshops to demonstrate instrumentation, data collection, and analysis.
2. Certifications: Explore certifications and learning resources offered by the OpenTelemetry community to empower team members.
3. Internal knowledge sharing: Establish a forum or group within your organization where engineers can discuss challenges, share learnings, and collaborate on OTel adoption.

Assigning specific roles in the migration is another valuable strategy for successful integration. Appoint observability champions to lead the OTel implementation and serve as internal experts. These observability champions should be DevOps team members, SREs, and application developers. Ensure that each observability champion has a clear role and stake in the migration effort.

One major challenge in implementing OpenTelemetry: maintaining context propagation across diverse services. In a polyglot environment, different services might be written in Java, Python, Go, or Node.js. This makes consistent trace context a complex puzzle.

Consider a typical enterprise transaction. A user request might begin at a modern Node.js front end, travel through a Java-based authentication service, hit an IBM MQ message queue, and finally reach a mainframe running CICS transactions. Each hop risks context loss, potentially leaving you with incomplete traces and blind spots in your observability pipeline.

However, many business-critical systems are notoriously difficult to instrument with OTel and modern observability tools. Message brokers using JMS 1.0 or IBM MQ, mainframe systems running z/OS, middleware solutions like Tibco EMS, and even certain older versions of enterprise databases can all present significant instrumentation challenges.

To bridge these gaps, organizations can take a few different routes:

• Manual instrumentation can provide precise control over what you monitor and how you monitor it.
• Developing custom OpenTelemetry plugins is another solution (and, if you're feeling generous, contributing them back to the open source community).
• OpenTelemetry-compatible instrumentations are available for particularly challenging systems like mainframes or legacy message queues to illuminate traditional blind spots.

When all else fails, remember the humble log file — your observability safety net. Even the most ancient of mainframes and the most obscure of legacy systems generate logs. While not as sophisticated as distributed tracing, logs can provide crucial visibility into otherwise opaque systems. By collecting and analyzing these logs, you can maintain end-to-end visibility across your entire technology stack. Consider enriching these logs with correlation IDs or trace identifiers where possible, creating bridges between your modern observability tools and legacy systems.

What's particularly exciting is how modern large language models (LLMs) are breathing new life into legacy logging data. These AI models can analyze unstructured log data, identifying patterns and anomalies that might be missed by traditional, manual parsing methods.

They can help correlate seemingly unrelated log entries across different systems, suggest root causes for incidents, and even predict potential issues before they become critical. Your legacy logs aren't just a last resort — they're an increasingly valuable data source that can provide deep insights when combined with modern AI analysis techniques. This hybrid approach ensures you're never completely in the dark, even when facing the most challenging instrumentation scenarios.

The promise of comprehensive observability is enticing. But don’t get too eager. Over-instrumentation can lead to unexpected performance issues. New OpenTelemetry agents running with default settings might collect more data than necessary, impacting system resources and lowering response times.

Beware the cardinality explosion! When instrumenting your services, it's tempting to add more and more attributes to spans and metrics for maximum visibility. However, this can quickly spiral into resource consumption issues and increased storage costs. High cardinality in span names and attributes can transform your observability solution from an asset into a long-term liability.

Your previous monitoring solution may have given you an easy button for reducing this cardinality. If so, these mitigations will need to be migrated over. Luckily there are a number of options here. The OpenTelemetry collector for example can preprocess the data so that it arrives in a low-cardinality format at your observability solution. When deploying new instrumentation, keep an eye out for this potential hurdle and be prepared to tune your OTel deployment as you deploy it.

Heterogeneous environments present unique challenges for OpenTelemetry implementation. Different services may require different instrumentation approaches, and not all components can be instrumented equally. This disparity can result in inconsistent observability coverage and complicated troubleshooting workflows.

Backend compatibility adds another layer of complexity. Not all observability back ends fully support OpenTelemetry's specifications, risking data loss or reduced functionality. It's crucial to verify compatibility between your chosen back end and OpenTelemetry's various signals (traces, metrics, and logs).

While the path to open standards and interoperability is of high value, there will be bumps and bruises along the way to removing vendor lock-in. For now, understanding these common pitfalls and having strategies to address them puts you ahead of the curve in building robust, observable systems.

Facing all this planning and these challenges might seem daunting, but the long-term health of your system will be well worth it. The key is to approach OpenTelemetry implementation with a clear strategy and realistic expectations. Perfect observability isn't achieved overnight — it's an iterative process that requires continuous refinement.

Focus on creating a sustainable observability practice that balances comprehensive monitoring with system performance. Regular audits of your instrumentation strategy, combined with performance tuning cycles, will help maintain this balance as your system expands and evolves.

The future of observability lies in finding the sweet spot between visibility and efficiency.
As OpenTelemetry continues to mature, many of these challenges will be addressed through improved tooling and best practices. Remember: The goal isn't to collect all possible data but to collect the right data that enables effective, continuously evolving systems. Start small, measure impact, and scale intelligently. Your future self (and your systems) will thank you for it.