Elastic APM is a free, open and powerful observability tool that provides intelligence into application performance for a myriad of production applications (e.g., throughput, error rates, latency, resource usage, transaction traces). You can now enable Elastic APM integration to gain deep insight into the performance behaviors of your Elastic Enterprise Search deployment!
This article provides step-by-step instructions for enabling the integration between Elastic's APM and Enterprise Search products and showcases how our Enterprise Search team leverages Elastic APM to capture additional intelligence to improve Enterprise Search performance during the development lifecycle.
Setting up the integration
Step 1: Deploy an Elastic APM Server
Elastic CloudThe easiest way to deploy an Elastic APM Server is on Elastic Cloud. APM Server is included in new Elastic Cloud deployments (APM & Fleet section) by default:
If you already have an Elastic Cloud deployment and had previously disabled APM & Fleet nodes, you can re-enable it from the Cloud console by clicking on APM & Fleet link > Edit deployment > Add Capacity from the APM & Fleet section.
You have the option to deploy APM in the same Elastic Cloud deployment where you are running Enterprise Search, but this is not mandatory. If desired, you can set up a new Elastic Cloud deployment to deploy APM and integrate with an existing Enterprise Search deployment.
If your Enterprise Search deployment is on-premises, you can either install a new Elastic APM Server in your infrastructure, use an existing Elastic APM Server, or create an Elastic Cloud deployment (see "Elastic Cloud" section in Step 1 above) with APM & Fleet enabled.
Enterprise Search's embedded APM Ruby agent is compatible with APM server >= version 6.5.
If you are already using the Elastic APM solution for performance monitoring of other applications and have an APM Server available, skip to Step 2.
If you are installing a new APM Server in your infrastructure, install the latest APM Server (follow only Step 1: Spin up the Elastic Stack in the Self-managed tab) and connect it to a new or existing Elasticsearch cluster. Install a Kibana instance to connect to the Elasticsearch cluster so you can visualize ("Self-managed" tab) the resulting APM data in Step 4.
Step 2: Copy the APM Server endpoint and secret token
Elastic CloudOnce APM & Fleet is enabled in your Elastic Cloud deployment, copy the APM endpoint and secret token from the Cloud user console. You will use these values in Step 3 in the Enterprise Search configuration.
If you are shipping Enterprise Search APM data collected for an on-premises Enterprise Search deployment to an Elastic Cloud APM & Fleet deployment, follow the "Elastic Cloud" instructions above to copy the endpoint and secret token for later use.
If you are using a new or existing APM Server in your infrastructure, start the APM Server and bring up the Kibana instance that is connected to the Elasticsearch cluster you have configured in Step 1 ("On-premises" section). From Kibana, click on Observability > APM > Services.Scroll down to the bottom of the page and click on Setup Instructions:
If you are using an existing APM Server setup in your infrastructure with existing services defined, simply click on Add data at the top right corner of the Services page (instead of "Setup instructions" above)
On the resulting page, click on Ruby on Rails tab:
This Ruby on Rails page provides a list of setup steps. Skip Step 1 because you do not have to install an APM agent since it is already embedded in Enterprise Search.
Keep the Default Standalone configuration policy and copy the value of secret_token and server_url. You will use these values in Step 3 in the Enterprise Search configuration.