Introducing Elastic's OpenTelemetry SDK for .NET
Adopting OpenTelemetry native solutions for observing .NET applications
Share on Twitter
Share on LinkedIn
Share on Facebook
Share by email
Print
We are thrilled to announce the alpha release of our new Elastic® distribution of the OpenTelemetry SDK for .NET. In this post, we cover a few reasonable questions you may have about this new distribution.
Download the NuGet package today if you want to try out this early access release. We welcome all feedback and suggestions to help us enhance the distribution before its stable release.
Check out our announcement blog post to learn more about OpenTelemetry and our decision to introduce OpenTelemetry distributions.
The Elastic .NET OpenTelemetry distribution
With the alpha release of the Elastic distribution of the .NET OpenTelemetry SDK, we are embracing OpenTelemetry as the preferred and recommended choice for instrumenting .NET applications.
In .NET, the runtime base class libraries (BCL) include types designed for native OpenTelemetry instrumentation, such as Activity and Meter, making adopting OpenTelemetry-native instrumentation even more convenient.
The current alpha release of our distribution is consciously feature-limited. Our goal is to assess the fitness of the API design and ease of use, laying a solid foundation going forward. We acknowledge that it is likely not suited to all application scenarios, so while we welcome developers installing it to try it out, we don’t currently advise using it for production.
In subsequent releases, we plan to add more features as we move toward feature parity with the existing Elastic APM agent for .NET. Based on user feedback, we will refine the API and move toward a stable release. Until then, we may need to make some breaking API changes to support additional use cases.
The current alpha release supports installation in typical modern workloads such as ASP.NET Core and worker services. It best supports modern .NET runtimes, .NET 6.0 and later. We’d love to hear about other scenarios you think we should focus on next.
The types we introduce in the distribution are to support an easy switch from the “vanilla” OpenTelemetry SDK with no (or minimal) code changes. We expect that for most circumstances, merely adding the NuGet package is all that is required to get started.
The initial alpha releases add very little on top of the “vanilla” SDK from OpenTelemetry, but by adopting it early, you can shape its direction. We will deliver valuable enhancements to developers in subsequent releases.
If you’d like to follow the development of the distribution, the code is fully open source and available on GitHub. We encourage you to raise issues for bugs or usability pain points you encounter.
How do I get started?
Getting started with the Elastic OpenTelemetry distribution is really easy. Simply add a reference to the Elastic OpenTelemetry NuGet package to your project. This can be achieved by adding a package reference to the project (csproj) file.
<PackageReference Include="Elastic.OpenTelemetry" Version="1.0.0-alpha.1" />After adding the package reference, you can use the Elastic OpenTelemetry distribution in your application. The distribution includes a transitive dependency on the OpenTelemetry SDK, so you do not need to add the OpenTelemetry SDK package to your project. Doing so will cause no harm and may be used to opt into newer SDK versions before the Elastic distribution references them.
The Elastic OpenTelemetry distribution is designed to be easy to use and integrate into your applications, including those that have previously used the OpenTelemetry SDK directly. When the OpenTelemetry SDK is already being used, the only required change is to add the Elastic.OpenTelemetry NuGet package to the project. Doing so will automatically switch to the opinionated configuration provided by the Elastic distribution.
ASP.NET Core example
A common requirement is to instrument ASP.NET Core applications based on Microsoft.Extensions.Hosting libraries, which provide dependency injection via an IServiceProvider.
The OpenTelemetry SDK and the Elastic distribution include extension methods to enable observability features in your application by adding a few lines of code.
This example focuses on adding instrumentation to an ASP.NET Core minimal API application using the Elastic OpenTelemetry distribution. Similar steps can also be applied to instrument other ASP.NET Core workloads and host-based applications such as Worker Services.
NOTE: This example assumes that we start with a new minimal API project created using project templates available with the .NET 8 SDK. It also uses top-level statements inside a single Program.cs file.
Add the Elastic.OpenTelemetry package reference to the project (csproj) file.
<PackageReference Include="Elastic.OpenTelemetry" Version="1.0.0-alpha.1" />To take advantage of the OpenTelemetry SDK instrumentation for ASP.NET Core, also add the OpenTelemetry.Instrumentation.AspNetCore NuGet package.
<PackageReference Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.7.1" />This package includes support to collect instrumentation (traces and metrics) for requests handled by ASP.NET Core endpoints.
Inside the Program.cs file of the ASP.NET Core application, add the following two using directives:
using OpenTelemetry;
using OpenTelemetry.Trace;
The OpenTelemetry SDK includes extension methods on the IServiceCollection to enable and configure the trace, metric, and log providers. The Elastic distribution overrides the default SDK registration, adding several opinionated defaults.
In the minimal API template, the WebApplicationBuilder exposes a Services property that can be used to register services with the dependency injection container. Ensure that the OpenTelemetry SDK is registered to enable tracing and metrics collection.
var builder = WebApplication.CreateBuilder(args);
builder.Services
.AddHttpClient() // <1>
.AddOpenTelemetry() // <2>
.WithTracing(t => t.AddAspNetCoreInstrumentation()); // <3>
<1> AddHttpClient registers the IHttpClientFactory service with the dependency injection container. This is not required to enable OpenTelemetry, but the example endpoint will use it to send an HTTP request.
<2> AddOpenTelemetry registers the OpenTelemetry SDK with the dependency injection container. When available, the Elastic distribution will override this to add opinionated defaults.
<3> Configures OpenTelemetry tracing to collect tracing and metric data produced by ASP.NET Core.
With these limited changes to the Program.cs file, the application is now configured to use the OpenTelemetry SDK and the Elastic distribution to collect traces and metrics, which are exported via OTLP.
To demonstrate the tracing capabilities, we will define a single endpoint for the API via the WebApplication.
var app = builder.Build();
app.UseHttpsRedirection();
app.MapGet("/", (IHttpClientFactory httpClientFactory) =>
Api.HandleRoot(httpClientFactory)); // <1>
app.Run();
<1> Maps an endpoint that handles requests to the application's root URL path. The handler will be supplied from a static class that we also need to add to the application. It accepts an IHttpClientFactory as a parameter, which will be injected from the dependency injection container at runtime and passed as an argument to the HandleRoot method.
namespace Example.Api
{
internal static class Api
{
public static async Task<IResult> HandleRoot(IHttpClientFactory httpClientFactory)
{
using var client = httpClientFactory.CreateClient();
await Task.Delay(100); // simulate work
var response = await client.GetAsync("http://elastic.co"); // <1>
await Task.Delay(50); // simulate work
return response.StatusCode == System.Net.HttpStatusCode.OK ? Results.Ok() : Results.StatusCode(500);
}
}
}
<1> This URL will require two redirects, allowing us to see multiple spans in the trace.
This static class includes a HandleRoot method that matches the signature for the endpoint handler delegate.
After creating a HttpClient from the factory, it sends a GET request to the elastic.co website. Either side of the request is a delay, which is used here to simulate some business logic being executed. The method returns a suitable status code based on the result of the external HTTP request.
If you’re following along, you will also need to include a using directive for the Example.Api namespace in your Program.cs file.
using Example.Api;That is all of the code we require for now. The Elastic distribution will automatically enable the exporting of telemetry signals via the OTLP exporter. The OTLP exporter requires that endpoint(s) be configured. A common mechanism for configuring endpoints is via environment variables.
This demo uses an Elastic Cloud deployment as the destination for our observability data. To retrieve the endpoint information from Kibana® running in Elastic Cloud, navigate to the observability setup guides. Select the OpenTelemetry option to view the configuration details that should be supplied to the application.
Configure environment variables for the application either in launchSettings.json or in the environment where the application is running. The authorization header bearer token should be stored securely, in user secrets or a suitable key vault system.
At a minimum, we must configure two environment variables:
OTEL_EXPORTER_OTLP_ENDPOINT
OTLP_EXPORTER_OTLP_HEADERS
It is also highly recommended to configure at least a descriptive service name for the application using the OTEL_RESOURCE_ATTRIBUTES environment variable otherwise a generic default will be applied. For example:
"OTEL_RESOURCE_ATTRIBUTES": "service.name=minimal-api-example"
Additional resource tags, such as version, can and should be added as appropriate. You can read more about the options for configuring resource attributes in the OpenTelemetry .NET SDK documentation.
Once configured, run the application and make an HTTP request to its root endpoint. A trace will be generated and exported to the configured OTLP endpoint.
To view the traces, you can use the Elastic APM Kibana UI. From the Kibana home page, visit the Observability area and from a trace under the APM > Traces page. After selecting a suitable time frame and choosing the trace named “GET /,” you will be able to explore one or more trace samples.
The above trace demonstrates the built-in instrumentation collection provided by the OpenTelemetry SDK and the optional OpenTelemetry.Instrumentation.AspNetCore package that we added.
It’s important to highlight that we would see a different trace above if we had used the “vanilla” SDK without the Elastic distribution. The HTTP spans that appear in blue in the screenshot would not be shown. By default, the OpenTelemetry SDK does not enable HTTP instrumentation, and it would require additional code to configure the instrumentation of outbound HTTP requests. The Elastic distribution takes the opinion that HTTP spans should be captured and enables this feature by default.
It is also possible to add application-specific instrumentation to this application. Typically, this would require calling vendor-specific APIs, for example, the tracer API in Elastic APM Agent. A significant benefit of choosing OpenTelemetry is the capability to use vendor-neutral APIs to instrument code with no vendor lock-in. We can see that in action by updating the API class in the sample.
internal static class Api
{
public static string ActivitySourceName = "CustomActivitySource";
private static readonly ActivitySource ActivitySource = new(ActivitySourceName);
public static async Task<IResult> HandleRoot(IHttpClientFactory httpClientFactory)
{
using var activity = ActivitySource.StartActivity("DoingStuff", ActivityKind.Internal);
activity?.SetTag("custom-tag", "TagValue");
using var client = httpClientFactory.CreateClient();
await Task.Delay(100);
var response = await client.GetAsync("http://elastic.co"); // using this URL will require 2 redirects
await Task.Delay(50);
if (response.StatusCode == System.Net.HttpStatusCode.OK)
{
activity?.SetStatus(ActivityStatusCode.Ok);
return Results.Ok();
}
activity?.SetStatus(ActivityStatusCode.Error);
return Results.StatusCode(500);
}
}
The preceding code snippet defines a private static ActivitySource field inside the Api class. Inside the HandleRoot method, an Activity is started using the ActivitySource, and several tags are set. The ActivitySource and Activity types are defined in the .NET BCL (base class library) and are defined in the System.Diagnostics namespace. A using directive is required to use them.
using System.Diagnostics;By using the Activity APIs to instrument the above code, we are not tied to any specific vendor APM solution. To learn more about using the .NET APIs to instrument code in an OpenTelemetry native way, visit the Microsoft Learn page covering distributed tracing instrumentation.
The last modification we must apply will instruct OpenTelemetry to observe spans from our application-specific ActivitySource. This is achieved by updating the registration of the OpenTelemetry components with the dependency injection framework.
builder.Services
.AddHttpClient()
.AddOpenTelemetry()
.WithTracing(t => t
.AddAspNetCoreInstrumentation()
.AddSource(Api.ActivitySourceName)); // <1>
<1> AddSource subscribes the OpenTelemetry SDK to spans (activities) produced by our application code.
A new trace will be collected and exported after making these changes, rerunning the application, and requesting the root endpoint. The latest trace can be viewed in the Kibana observability UI.
The trace waterfall now includes the internal “DoingStuff” span produced by the instrumentation that we added to our application code. The HTTP spans still appear and are now child spans of the “DoingStuff” span.
We’re working on writing more thorough documentation to be published on elastic.co. Until then, you can find more information in our repository readme and the docs folder.
As the distribution is designed to extend the capabilities of the OpenTelemetry SDK with limited impact on the code used to register the SDK, we recommend visiting the OpenTelemetry documentation for .NET to learn about the instrumenting code and provide a more advanced configuration of the SDK.
What are the next steps?
We are very excited to expand our support of the OpenTelemetry community and contribute to its future within the .NET ecosystem. This is the compelling next step toward greater collaboration between all observability vendors to provide a rich ecosystem supporting developers on their journey to improved application observability with zero vendor lock-in.
At this stage, we strongly appreciate any feedback the .NET community and our customers can provide to guide the direction of our OpenTelemetry distribution. Please try out our distribution and engage with us through our GitHub repository.
In the coming weeks and months, we will focus on stabilizing the distribution's API and porting Elastic APM Agent features into the distribution. In parallel, we expect to start donating and contributing features to the broader OpenTelemetry community via the OpenTelemetry GitHub repositories.
The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.
Share
Share on Twitter
Share on LinkedIn
Share on Facebook
Share by email
Print
