If you have something to add to this section, please consider creating a pull request with your proposed changes at https://github.com/elastic/kibana.
Also, check out the APM discussion forum.
No APM data foundedit
This section can help with any of the following:
- Data isn’t displaying in the APM app
- You see a message like "No Services Found",
- You see errors like "Fielddata is disabled on text fields by default…"
There are a number of factors that could be at play here. One important thing to double-check first is your index template.
An APM index template must exist for the APM app to work correctly.
By default, this index template is created by APM Server on startup.
However, this only happens if
You can create the index template manually by running
Take note that index templates cannot be applied retroactively — they are only applied at index creation time.
More information is available in Set up and configure.
You can check for the existence of an APM index template using the Get index template API. If you’re using the default index naming pattern, that request would be:
Using Logstash, Kafka, etc. If you’re not outputting data directly from APM Server to Elasticsearch (perhaps you’re using Logstash or Kafka), then the index template will not be set up automatically. Instead, you’ll need to load the template manually.
Using a custom index names
This problem can also occur if you’ve customized the index name that you write APM data to.
The default index name that APM writes events to can be found
If you change the default, you must also configure the
See Load the Elasticsearch index template.
If the Elasticsearch index template has already been successfully loaded to the index,
you can customize the indices that the APM app uses to display data.
Navigate to APM > Settings > Indices, and change all
apm_oss.*Pattern values to
include the new index pattern. For example:
Too many unique transaction namesedit
Transaction names are defined in each APM Agent; when an Agent supports a framework, it includes logic for naming the transactions that the framework creates. In some cases though, like when using an Agent’s API to create custom transactions, it is up to the user to define a pattern for transaction naming. When transactions are named incorrectly, each unique URL can be associated with a unique transaction group—causing an explosion in the number of transaction groups per service, and leading to inaccuracies in the APM app.
To fix a large number of unique transaction names, you need to change how you are using the Agent API to name your transactions. To do this, ensure you are not naming based on parameters that can change. For example, user ids, product ids, order numbers, query parameters, etc., should be stripped away, and commonality should be found between your unique URLs.
Let’s look at an example from the RUM Agent documentation. Here are a few URLs you might find on Elastic.co:
// Blog Posts https://www.elastic.co/blog/reflections-on-three-years-in-the-elastic-public-sector https://www.elastic.co/blog/say-heya-to-the-elastic-search-awards https://www.elastic.co/blog/and-the-winner-of-the-elasticon-2018-training-subscription-drawing-is // Documentation https://www.elastic.co/guide/en/elastic-stack/current/index.html https://www.elastic.co/guide/en/apm/get-started/current/index.html https://www.elastic.co/guide/en/infrastructure/guide/current/index.html
These URLs, like most, include unique names.
If we named transactions based on each unique URL, we’d end up with the problem described above—a
very large number of different transaction names.
Instead, we should strip away the unique information and group our transactions based on common information.
In this case, that means naming all blog transactions,
/blog, and all documentation transactions,
If you feel like you’d be losing valuable information by following this naming convention, don’t fret! You can always add additional metadata to your transactions using labels (indexed) or custom context (non-indexed).
After ensuring you’ve correctly named your transactions,
you might still see an error in the APM app related to too many transaction names.
If this is the case, you can increase the default number of transaction groups displayed in the APM app by configuring
While this can happen with any APM Agent, it typically occurs with the RUM Agent.
For more information on how to correctly set
transaction.name in the RUM Agent,
see custom initial page load transaction names.
The RUM Agent can also set the
transaction.name when observing for transaction events.
apm.observe() for more information.
If your problem is occurring in a different Agent, the tips above still apply. See the relevant Agent API documentation to adjust how you’re naming your transactions.
The transaction overview will only display helpful information when the transactions in your services are named correctly. If you’re seeing "GET unknown route" or "unknown route" in the APM app, it could be a sign that something isn’t working as it should.
Elastic APM Agents come with built-in support for popular frameworks out-of-the-box. This means, among other things, that the Agent will try to automatically name HTTP requests. As an example, the Node.js Agent uses the route that handled the request, while the Java Agent uses the Servlet name.
"Unknown route" indicates that the Agent can’t determine what to name the request, perhaps because the technology you’re using isn’t supported, the Agent has been installed incorrectly, or because something is happening to the request that the Agent doesn’t understand.
To resolve this, you’ll need to head over to the relevant Agent documentation. Specifically, view the Agent’s supported technologies page. You can also use the Agent’s public API to manually set a name for the transaction.
Fields are not searchableedit
In Elasticsearch, index templates are used to define settings and mappings that determine how fields should be analyzed. The recommended index template file for APM Server is installed by the APM Server packages. This template, by default, enables and disables indexing on certain fields.
As an example, some agents store cookie values in
http.request has disabled dynamic indexing, and
http.request.cookies is not declared in a custom mapping,
the values in
http.request.cookies are not indexed and thus not searchable.
Ensure an index pattern exists
As a first step, you should ensure the correct index pattern exists.
In Kibana, navigate to Management > Kibana > Index Patterns.
In the pattern list, you should see an apm index pattern; The default is
If you don’t, the index pattern doesn’t exist. See No APM data found for information on how to fix this problem.
apm-* index pattern shows a listing of every field defined in the pattern.
Ensure a field is searchable There are two things you can do to if you’d like to ensure a field is searchable:
setup.template.enabled: true setup.template.overwrite: true setup.template.append_fields: - name: http.request.cookies type: object dynamic: true
Service maps: no connection between client and serveredit
If the service map is not showing an expected connection between the client and server,
it’s likely because you haven’t configured
This setting is necessary, for example, for cross-origin requests.
If you have a basic web application that provides data via an API on
and serves HTML from
localhost:4001, you’d need to set
to ensure the origin is monitored as a part of distributed tracing.
In other words,
distributedTracingOrigins is consulted prior to the agent adding the
traceparent header to each request.