You are looking at preliminary documentation for a future release. Not what you want? See the current release documentation.
Elastic Docs › ›Elastic Enterprise Search server
« Storage API Read-only mode »

Monitoringedit

Enterprise Search and the Elastic Stack provide monitoring features to keep a pulse on the health and performance of your Enterprise Search deployment.

Monitor Enterprise Search deployments on Elastic Cloudedit

Elastic Cloud provides the easiest path to monitor your deployment. Follow these recommended steps:

  1. Enable logging and monitoring for your deployment.
  2. Open Kibana in your logging and monitoring deployment.
  3. From the sidebar, under Management, choose Stack Monitoring.
  4. Choose the Enterprise Search tile to view information about your deployment including HTTP traffic, JVM statistics, App Search engine counts, Workplace Search content source counts, and number of crawler workers.

Additional monitoring options:

It’s not necessary to enable the Enterprise Search self-monitoring feature for Elastic Cloud deployments. The integrated stack monitoring capabilities on Elastic Cloud collect the same information.

Monitor Enterprise Search self-managed deploymentsedit

Follow these steps to monitor your self-managed Enterprise Search deployment:

  1. Enable Enterprise Search self-monitoring.
  2. Import pre-built Kibana dashboards.

If you need more insights into the low-level details of Enterprise Search itself, you can enable APM for Enterprise Search.

You may also call Enterprise Search monitoring APIs to access internal Enterprise Search system metrics and more product usage details.

Kibana dashboards for Enterprise Search monitoringedit

To visualize the metrics ingested into Elasticsearch via Cloud monitoring, via the self-monitoring feature, or via a dedicated Enterprise Search Metricbeat module, import pre-built Kibana dashboards into your deployment:

  1. Download the enterprise-search-overview-8.0.ndjson dashboard file from the elastic/ent-search-monitoring GitHub repository.
  2. Open Kibana.
  3. From the sidebar, under Management, choose Stack Management.
  4. Under Kibana, choose Saved Objects.
  5. Click Import and import the enterprise-search-overview-8.0.ndjson dashboard file you downloaded in step 1.
  6. Open the [Metricbeat Enterprise Search] Overview object to view the dashboard.

APM for Enterprise Searchedit

Use Elastic APM to collect in-depth performance metrics and errors for Enterprise Search itself.

Using APM to monitor Enterprise Search is an advanced use case that you are unlikely to need unless you are diagnosing a specific issue in Enterprise Search.

Using APM to monitor Enterprise Search consumes additional resources. Use APM with discretion on production systems.

To enable APM for Enterprise Search monitoring, follow these steps in your Enterprise Search deployment or in a separate deployment dedicated to collecting APM data.

Find the APM secret_token and server_url on the deployment you are using to collect APM data:

  1. Open Kibana.
  2. From the sidebar, under Observability, open APM.
  3. Click Add Data.
  4. Scroll down to the APM Agents section.
  5. Choose the Ruby on Rails tab.
  6. Copy the secret_token and server_url values from the Configure the agent section.

Configure Enterprise Search settings on your main deployment to add the APM secret_token and server_url settings:

apm.enabled: true
apm.server_url: 'your-apm-server-url-here'
apm.secret_token: 'your-token-here'

An example apm.server_url looks like this on Elastic Cloud: https://your-apm-deployment-name.apm.europe-west1.gcp.cloud.es.io:443.

An example apm.secret_token looks like this: 2FHHxzMDyU7G3t6pXb.

After the changes are applied, view Enterprise Search metrics in APM:

  1. Open Kibana.
  2. From the sidebar, under Observability, open APM.
  3. Open Enterprise Search from the list of services.

Enterprise Search self-monitoringedit

Enable Enterprise Search self-monitoring on self-managed deployments to capture metrics including Enterprise Search network, JVM, product usage, and background worker metrics.

To enable self-monitoring, add the following setting to your Enterprise Search configuration and restart Enterprise Search:

monitoring.reporting_enabled: true

This feature is intended for self-managed deployments. It is not necessary to enable self-monitoring on Elastic Cloud deployments. See Monitor Enterprise Search deployments on Elastic Cloud.

The Enterprise Search self-monitoring feature only ships metrics to the same Elasticsearch deployment that Enterprise Search uses.

After self-monitoring is enabled, Enterprise Search will automatically provision a Metricbeat instance with the enterprisesearch module. The module will start reporting a detailed set of monitoring metrics to the underlying Elasticsearch cluster every 10 seconds to be stored in a set of indexes called .monitoring-ent-search-*.

Metricbeat indexes have ILM enabled on them. By default, data will be stored in Elasticsearch for up to 30 days. Indexes will be rotated every 100Mb.

You cannot control the ent-search-metricbeat ILM policy. Even if you use the Kibana UI or Elasticsearch API to directly update the policy, Enterprise Search will overwrite it.

If you need full control of the metricbeat ILM policies for your Enterprise Search monitoring data, please deploy a standalone Enterprise Search Metricbeat module.

See the monitoring section of the configuration guide for deployment configuration settings to control the frequency of monitoring data reporting, index names used and other aspects of the self-monitoring feature within Enterprise Search.

You can view self-monitoring metrics in Kibana. Use the Kibana dashboards for Enterprise Search monitoring or choose Stack Monitoring from the sidebar menu under Management.

Enterprise Search Metricbeat moduleedit

Unless you have unique Enterprise Search monitoring needs, you probably do not need to run the Enterprise Search Metricbeat module directly.

Elastic Metricbeat contains an official Enterprise Search monitoring module that can be used to ingest monitoring data from your Enterprise Search deployments into any Elasticsearch cluster. This module powers the Enterprise Search self-monitoring feature underneath. When using Elastic Cloud, enabling logging and monitoring for your deployment runs a built-in Enterprise Search Metricbeat module internally.

The data produced by the Metricbeat module can then be accessed in Kibana, for example by using the Kibana dashboards for Enterprise Search monitoring.

See the Enterprise Search Metricbeat module documentation for more details on how to use this module.

« Storage API Read-only mode »