Troubleshootingedit

Use the information in this section to troubleshoot common problems and find answers for frequently asked questions. As a first step, ensure your stack is compatible with the Agent’s supported technologies.

Don’t worry if you can’t figure out what the problem is; we’re here to help. If you are an existing Elastic customer with a support contract, please create a ticket in the Elastic Support portal. If not, post in the APM discuss forum.

No data is sent to the APM Serveredit

If neither errors nor performance metrics are being sent to the APM Server, it’s a good idea to first check your logs and look for output just as the app starts.

If you don’t see anything suspicious in the agent logs (no warning or error), it’s recommended to turn the log level to Trace for further investigation.

Collecting agent logsedit

The way to collect logs depends on the setup of your application.

ASP.NET Coreedit

If you added the agent to your application as per the ASP.NET Core document with the UseAllElasticApm or UseElasticApm method, it will integrate with the ASP.NET Core logging infrastructure. This means the Agent will pick up the configured logging provider and log as any other component logs.

ASP.NET Classicedit

ASP.NET (classic) does not have a predefined logging system. By default, the agent is configured to emit log messages to a System.Diagnostics.TraceSource with the source name "Elastic.Apm". The TraceSource adheres to the log levels defined in the APM agent configuration.

TraceListeners can be configured to monitor log messages for the trace source, using the <system.diagnostics> section of web.config. For example, the following web.config section writes Elastic.Apm log messages to a file named my_log_file.log:

<configuration>
  <!-- other sections .... -->
<system.diagnostics>
  <sources>
    <source name="Elastic.Apm"> 
      <listeners>
        <add name="file"
          type="System.Diagnostics.TextWriterTraceListener"
          initializeData="my_log_file.log" />
      </listeners>
    </source>
  </sources>
</system.diagnostics>
</configuration>

Other logging systemsedit

If you have a logging system in place such as NLog, Serilog, or similar, you can direct the agent logs into your logging system by creating an adapter between the agent’s internal logger and your logging system.

First implement the IApmLogger interface from the Elastic.Apm.Logging namespace:

internal class ApmLoggerAdapter : IApmLogger
{
	private readonly Lazy<Logger> _logger;
	public bool IsEnabled(ApmLogLevel level)
	{
		// Check for log level here.
		// Typically you just compare the configured log level of your logger
		// to the input parameter of this method and return if it's the same/higher or not
	}

	public void Log<TState>(ApmLogLevel apmLogLevel, TState state, Exception e, Func<TState, Exception, string> formatter)
	{
		// You can log the given log into your logging system here.
	}
}

An example implementation for NLog can be seen in our GitHub repository.

Then tell the agent to use the ApmLoggerAdapter. For ASP.NET (classic), place the following code into the Application_Start method in the HttpApplication implementation of your app which is typically in the Global.asax.cs file:

using Elastic.Apm.AspNetFullFramework;

namespace MyApp
{
	public class MyApplication : HttpApplication
	{
		protected void Application_Start()
		{
			AgentDependencies.Logger = new ApmLoggerAdapter();

			// other application setup...
		}
	}
}

During initialization, the agent checks if an additional logger was configured-- the agent only does this once, so it’s important to set it as early in the process as possible, typically in the Application_Start method.

General .NET applicationsedit

If none of the above cases apply to your application, you can still use a logger adapter and redirect agent logs into a .NET logging system like NLog, Serilog, or similar.

For this you’ll need an IApmLogger implementation (see above) which you need to pass to the Setup method during agent setup:

Agent.Setup(new AgentComponents(logger: new ApmLoggerAdapter()));

Following error appears in logs: The singleton APM agent has already been instantiated and can no longer be configured.edit

See "An InstanceAlreadyCreatedException exception is thrown".

An InstanceAlreadyCreatedException exception is thrownedit

In the early stage of a monitored process, the Agent might throw an InstanceAlreadyCreatedException exception with the following message: "The singleton APM agent has already been instantiated and can no longer be configured.", or an error log appears with the same message. This happens when you attempt to initialize the Agent multiple times, which is prohibited. Allowing multiple Agent instances per process would open up problems, like capturing events and metrics multiple times for each instance, or having multiple background threads for event serialization and transfer to the APM Server.

As an example, this issue can happen if you call the Elastic.Apm.Agent.Setup method multiple times, or if you call another method on Elastic.Apm.Agent that implicitly initializes the agent, and then you call the Elastic.Apm.Agent.Setup method on the already initialized agent.

Another example might be when you use the Public Agent API in combination with the IIS module or the ASP.NET Core NuGet package, where you enable the agent with the UseElasticApm or UseAllElasticApm methods. Both the first call to the IIS module and the UseElasticApm/UseAllElasticApm methods internally call the Elastic.Apm.Agent.Setup method to initialize the agent.

You may use the Public Agent API with the Elastic.Apm.Agent class in code that can potentially execute before the IIS module initializes or the UseElasticApm/UseAllElasticApm calls execute. If that happens, those will fail, as the Agent has been implicitly initialized already.

To prevent the InstanceAlreadyCreatedException in these scenarios, first use the Elastic.Apm.Agent.IsConfigured method to check if the agent is already initialized. After the check, you can safely use other methods in the Public Agent API. This will prevent accidental implicit agent initialization.

Startup hooks failureedit

If the startup hook integration throws an exception, additional detail can be obtained by setting the ELASTIC_APM_STARTUP_HOOKS_LOGGING environment variable before starting the application

set ELASTIC_APM_STARTUP_HOOKS_LOGGING=1

and then running the application in a context where the environment variable will be visible. In setting this value, an ElasticApmAgentStartupHook.log file is written to the directory containing the startup hook assembly, in addition to writing to standard output.

The agent causes too much overheadedit

A good place to start is All options summary. There are multiple settings with the Performance keyword which can help you tweak the agent for your needs.

The most expensive operation in the agent is typically stack trace capturing. The agent, by default, only captures stack traces for spans with a duration of 5ms or more, and with a limit of 50 stack frames. If this is too much in your environment, consider disabling stack trace capturing either partially or entirely: