Troubleshoot Enterprise Search setupedit

When navigating to Enterprise Search within Kibana, you may see "Unable to connect". To troubleshoot this, first enable and view logs for Enterprise Search and Kibana. Then review common setup issues to help identify and solve your problem.

Enable and view logsedit

Enterprise Search and Kibana each log information that may help identify and solve your problem. If necessary, first enable logging. Then view the logs for each service.

Refer to the following documentation for your deployment type:

Review common setup issuesedit

Review the following common issues to help identify and solve your problem:

Enterprise Search’s kibana.external_url setting is incorrect or unreachableedit

Enterprise Search will not start and logs the following:

[2021-09-28T17:57:33.438+00:00][8][2002][app-server][INFO]: Performing pre-flight checks for Kibana running on http://kibana1:5601...
Unexpected exception while running Enterprise Search:
Error: kibana1: Name or service not known at /usr/share/enterprise-search/lib/war/gems/gems/manticore-0.7.0-java/lib/faraday/adapter/manticore.rb:88:in `block in call'
        /usr/share/enterprise-search/lib/war/gems/gems/manticore-0.7.0-java/lib/manticore/response.rb:79:in `call'
        /usr/share/enterprise-search/lib/war/gems/gems/manticore-0.7.0-java/lib/faraday/adapter/manticore.rb:92:in `call'
        /usr/share/enterprise-search/lib/war/lib/middleware/request_logging_middleware.class:25:in `call'
        /usr/share/enterprise-search/lib/war/gems/gems/faraday-1.3.0/lib/faraday/rack_builder.rb:154:in `build_response'
        /usr/share/enterprise-search/lib/war/gems/gems/faraday-1.3.0/lib/faraday/connection.rb:492:in `run_request'
        /usr/share/enterprise-search/lib/war/gems/gems/faraday-1.3.0/lib/faraday/connection.rb:198:in `get'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/kibana_checks.class:89:in `check_kibana_connection_with_retries!'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/kibana_checks.class:36:in `check_kibana_connection!'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/kibana_checks.class:20:in `block in run!'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/tagged_logging.rb:69:in `block in tagged'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/tagged_logging.rb:26:in `tagged'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/tagged_logging.rb:69:in `tagged'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/kibana_checks.class:19:in `run!'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/kibana_checks.class:13:in `run!'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo.class:290:in `configure_kibana!'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo.class:248:in `configure!'
        /usr/share/enterprise-search/lib/war/config/application.class:19:in `<main>'
        org/jruby/RubyKernel.java:1016:in `load'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/dependencies.rb:286:in `block in load'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/dependencies.rb:258:in `load_dependency'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/dependencies.rb:286:in `load'
        /usr/share/enterprise-search/lib/war/config/application.rb:1:in `<main>'
        org/jruby/RubyKernel.java:974:in `require'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/dependencies.rb:292:in `block in require'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/dependencies.rb:258:in `load_dependency'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/dependencies.rb:292:in `require'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli/command.class:36:in `initialize'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli/command.class:10:in `run_and_exit'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli.class:143:in `run_supported_command'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli.class:125:in `run_command'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli.class:112:in `run!'
        bin/enterprise-search-internal:15:in `<main>'
Solutionedit

Set kibana.external_url to an accessible Kibana address.

Kibana’s enterpriseSearch.host setting is incorrect or unreachableedit

Kibana logs the following:

{"type":"log","@timestamp":"2021-09-28T18:05:43+00:00","tags":["error","plugins","enterpriseSearch"],"pid":1221,"message":"Could not perform access check to Enterprise Search: FetchError: request to https://enterprise-search1:3002/api/ent/v2/internal/client_config failed, reason: getaddrinfo ENOTFOUND enterprise-search1"}
Solutionedit

Set enterpriseSearch.host to an accessible Enterprise Search address. See Configure Kibana.

Kibana’s enterpriseSearch.host is set to http while Enterprise Search is running on httpsedit

Kibana logs the following:

{"type":"log","@timestamp":"2021-09-28T17:39:05+00:00","tags":["error","plugins","enterpriseSearch"],"pid":1219,"message":"Could not perform access check to Enterprise Search: FetchError: request to http://enterprise-search:3002/api/ent/v2/internal/client_config failed, reason: Parse Error: Expected HTTP/"}
Solutionedit

Set enterpriseSearch.host to the correct protocol within Kibana. See Configure Kibana.

Kibana’s connection to Enterprise Search is timing outedit

Kibana logs the following:

{"type":"log","@timestamp":"2021-09-28T17:52:31+00:00","tags":["warning","plugins","enterpriseSearch"],"pid":1219,"message":"Enterprise Search access check took over 300ms. Please ensure your Enterprise Search server is responding normally and not adversely impacting Kibana load speeds."}
{"type":"log","@timestamp":"2021-09-28T17:52:35+00:00","tags":["warning","plugins","enterpriseSearch"],"pid":1219,"message":"Exceeded 5000ms timeout while checking https://enterprise-search:3002. Please consider increasing your enterpriseSearch.accessCheckTimeout value so that users aren't prevented from accessing Enterprise Search plugins due to slow responses."}
Solutionedit

Troubleshoot performance issues in Enterprise Search. To get started, see Monitoring your deployments.

If necessary, contact Elastic Support.

Kibana cannot find or does not trust the SSL certificates for Enterprise Searchedit

Kibana logs the following:

{"type":"log","@timestamp":"2021-09-28T17:42:11+00:00","tags":["error","plugins","enterpriseSearch"],"pid":1220,"message":"Could not perform access check to Enterprise Search: FetchError: request to https://enterprise-search:3002/api/ent/v2/internal/client_config failed, reason: unable to verify the first certificate"}
Solutionedit

Configure enterpriseSearch.ssl.certificateAuthorities to reference the correct certificate or certificate chain. See Configure Kibana to Trust Your SSL Certificate Authority.

Enterprise Search has SSL enabled but hostname verification failsedit

When enterpriseSearch.ssl.verificationMode is configured to perform hostname verification but hostname verification fails.

Kibana logs the following:

{"type":"log","@timestamp":"2021-09-28T18:39:04+00:00","tags":["error","plugins","enterpriseSearch"],"pid":1220,"message":"Could not perform access check to Enterprise Search: FetchError: request to https://enterprise-search:3002/api/ent/v2/internal/client_config failed, reason: Hostname/IP does not match certificate's altnames: Host: enterprise-search. is not in the cert's altnames: DNS:localhost, IP Address:127.0.0.1, DNS:node1"}
Solutionedit

In production or other environments that must be secure, regenerate the certificates with the correct Subject Alternative Names matching enterpriseSearch.host in Kibana.

In development and some other environments, reducing security may be acceptable. In these cases, set enterpriseSearch.ssl.verificationMode to a lower setting. For example, change full to certificate or none. See Configure Kibana to Trust Your SSL Certificate Authority.

Reduce the security of your deployment only as a last resort. Ideally, hostname verification should be enabled in all environments.

Enterprise Search and Kibana versions are incompatibleedit

The Kibana UI displays "Incompatible version error".

Kibana logs the following:

{"type":"log","@timestamp":"2021-09-28T19:03:00+00:00","tags":["warning","plugins","enterpriseSearch"],"pid":1225,"message":"Your Kibana instance (v7.15.0) is not the same version as your Enterprise Search instance (vundefined), which may cause unexpected behavior. Use matching versions for the best experience."}
Solutionedit

Run Enterprise Search and Kibana services of the same major and minor versions.

Enterprise Search in Kibana is "forbidden" when traffic filters are set on Elastic Cloudedit

The Kibana UI displays "Forbidden".

Kibana logs the following:

ResponseError: security_exception: [security_exception] Reason: action [indices:data/read/search] is unauthorized for user [cloud-internal-enterprise_search-server] with roles [cloud-internal-enterprise_search-server] on indices [.kibana_7.14.0_001,.kibana], this action is granted by the index privileges [read,all]
    at onBody (/usr/share/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:337:23)
    at IncomingMessage.onEnd (/usr/share/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:264:11)
    at IncomingMessage.emit (events.js:387:35)
    at endReadableNT (internal/streams/readable.js:1317:12)
    at processTicksAndRejections (internal/process/task_queues.js:82:21) {
  meta: {
    body: { error: [Object], status: 403 },
    statusCode: 403,
    headers: {
      'content-length': '625',
      'content-type': 'application/json; charset=UTF-8',
      'x-cloud-request-id': 'O_v__Y_hRf-I528DechRKQ',
      'x-elastic-product': 'Elasticsearch',
      'x-found-handling-cluster': '<deployment_id>',
      'x-found-handling-instance': 'instance-0000000000',
      'x-opaque-id': 'd77d5906-8317-40c1-86e4-d78a408908b5',
      date: 'Mon, 30 Aug 2021 12:14:08 GMT'
    },
    meta: {
      context: null,
      request: [Object],
      name: 'elasticsearch-js',
      connection: [Object],
      attempts: 0,
      aborted: false
    }
  }
}
Solutionedit

Contact Elastic Support for the latest information about this issue.