8.0.0 release notesedit

Upgrading to Enterprise Search 8.0.0? See:

Breaking changesedit

See Upgrading from Enterprise Search 7.x for more details on these changes and for other considerations when upgrading to 8.0.0.

  • Enterprise Search no longer runs on Java 8. Use Java 11 when running Enterprise Search on your own hardware. Enterprise Search Docker images use Java 11 by default.
  • Security for the Elastic Stack is required. Enterprise Search will not start if you disable security.
  • The standalone Enterprise Search management interface is removed. Manage Enterprise Search in Kibana.
  • Workplace Search API keys replace admin access tokens. Existing admin access tokens are automatically migrated to API keys, but API keys have different behavior regarding document-level permissions. See Workplace Search API keys replace admin access tokens for details.
  • The Workplace Search whoami API will no longer accept admin access tokens (or API keys). Instead, use access tokens generated by the Workplace Search OAuth service.
  • Web crawler API endpoints change from v0 to v1. The v0 endpoints are deprecated, but they will continue to work in 8.0.0. They will be removed in a future release.
  • The web crawler event log (crawl-status events) and crawler status API now use millisecond resolution for all duration fields.
  • The updated_at field within Workplace Search content source documents is now populated as the last time when the document was indexed into Workplace Search.
  • Monitoring indices now use the pattern .monitoring-ent-search-*.
  • Log indices use data streams instead of rollover aliases.
  • The following configuration settings are removed:

    • enable_ecs_logs
    • ent_search.auth.* - Configure user authentication within Elasticsearch and Kibana.
    • ent_search.monitoring_enabled
    • ent_search.monitoring_port
    • ent_search.monitoring_ssl
    • telemetry.* - Configure telemetry within Kibana.
  • The following configuration settings are renamed:

    • workplace_search.custom_api_source.document_sizeworkplace_search.content_source.document_size.limit
    • workplace_search.custom_api_source.total_fieldsworkplace_search.content_source.total_fields.limit
  • The following configuration settings are required:

    • kibana.host - Default value changes to http://localhost:5601.
  • The following configuration settings are deprecated:

    • ilm.enabled

New featuresedit

  • Web crawler APIs are promoted from beta to generally available.
  • The web crawler now supports crawling with stateless auth by configuring each domain (requires an appropriate subscription level).
  • Web crawler URL validation checks have been improved to always check the validity of the provided URL.
  • The web crawler now supports the Brotli content compression algorithm. It is now possible to disable HTTP content compression in the web crawler.
  • Web crawler metrics have been added to the Enterprise Search Stats API.
  • The search API for App Search now supports geographical sorting.
  • Curations organic results displayed in the UI now take into account search relevance tunings.
  • App Search API users can now control the number of Elasticsearch shards that will be used for their engines by providing a index_create_settings_override parameter for the Engine Create API call. If App Search fails to create an Elasticsearch index for an engine, it rolls back the engine creation process to ensure the product remains in a consistent state.
  • Kibana adds a user interface to manage Workplace Search API keys. See Workplace Search API keys replace admin access tokens for more details on Workplace Search API keys.
  • Document-level permissions are now available for ServiceNow content sources in Workplace Search.
  • The Salesforce connector can index additional fields for predefined Salesforce objects using custom indexing rules.
  • The performance of the Salesforce connector is improved due to new caching logic.
  • Enterprise Search now validates encryption keys on startup. A warning message is logged when encryption keys cannot decrypt some of the stored secrets. To enforce validation of the encryption keys, use the configuration setting secret_management.enforce_valid_encryption_keys: true. When true, Enterprise Search will not start when encryption keys are incorrect.

Bug fixesedit

  • The web crawler now fails gracefully when detecting an unsupported content encoding method in an HTTP response.
  • During a purge crawl, the web crawler would previously fetch URLs from sitemaps. This is not needed and would potentially result in the crawl taking much longer to finish for configurations with a lot of sitemaps (both manually added sitemaps but also auto-discovered from robots.txt)
  • App Search now ensures boost factors are greater than or equal to zero. Using negative boost factors will result in a 400 error response.
  • App search was writing web crawler logs to Elasticsearch, even when web crawler logs were disabled in settings. This issue is fixed.
  • Background work queueing code (background jobs, crawler jobs, etc) is now more resilient. Fixed by introducing Elasticsearch transient error retries.
  • The web crawler crawl queue is more reliable. Fixed by retrying transient errors from Elasticsearch.

Known issuesedit

  • In rare cases, a Confluence Cloud content source cannot sync. This issue applies only when both of the following are true:

    • Document-level permissions are enabled for the content source.
    • Each Confluence space within the source contains only a few documents.
  • Enterprise Search does not support restoring snapshot data from one deployment’s snapshot in Elastic Cloud to another deployment.
  • Within the search application for Workplace Search, selecting the thumbnail of a result may cause a redirect to the Kibana management interface.
  • App Search users with dev and editor roles do not have access to the web crawler.