8.8.0 release notesedit

Upgrading to Enterprise Search 8.8.0? See Upgrading and migrating.

Breaking changesedit

  • Due to a breaking change in the behavioral analytics events data model, users seeking to upgrade their Behavioral Analytics deployment from 8.7 to 8.8.0 will need to manually migrate their data. Refer to this migration guide for more information.

New featuresedit

  • Use the new Elastic Late Sparse EncodeR (ELSER) machine learning model for enhanced ingestion and query expansion in Enterprise Search.

    • Elastic’s first proprietary sparse encoder model significantly boosts semantic search performance, without needing to fine tune on your domain data.
    • Easily deploy ELSER in a few clicks:

    • The fastest way to get up and running using ELSER text expansion in your queries is to use the Elastic Search Applications feature also released in 8.8.0.
  • We have delivered a number of user experience improvements in 8.8.0:

    • Global Kibana search for our connectors and the web crawler.
    • Easy preview of connector types before index creation.
    • Streamlined and simplified UI workflow for both native connectors and connector clients.
    • More intuitive and configurable fields to simplify connector configuration.
    • Consistent and simplified sync rule support for database type connectors.
  • Two Elastic native connectors for databases are now generally available:

  • Two connector clients are now promoted to native connectors:

  • Native connectors can now be run on 2GB deployments (ideal for trial users).
  • We’ve added new connector clients for three popular collaboration tools:

  • The Elastic Python open code connector framework is now generally available.

    • The MongoDB connector has been migrated to the Elastic connector framework from the Ruby framework.

      All future Elastic connector development will be made using the Python framework. The Ruby framework and Ruby connector will remain in beta, and will continue only as an unsupported reference implementation.

  • Use Search Applications (technical preview) to simplify adding search features to your application.

    • Formulate and expose simplified, user-friendly search requests, while still harnessing Elasticsearch’s advanced search capabilities. Access your search request using the Search Application Search endpoint in your web client or the Dev Console.
    • Easily create, manage and query search applications made up of one or more indices.
    • Quickly view schemas and identify field conflicts across associated indices.
    • Integrated as an Elasticsearch module, with Elasticsearch-native APIs.
    • Configure a search template that can be tuned for ELSER enriched queries and other _search parameters.
  • We’ve made a number of improvements to Behavioral Analytics (beta) including:

    • All new design, with dashboards and visualizations now available out of the box.
    • Increased focus on performance and storage:

      • Benchmarked with a range of use cases.
      • The events footprint is now significantly smaller.
      • You can now capture user geolocation and user-agent information.
    • This feature is now integrated as an Elasticsearch module, with Elasticsearch-native APIs for managing your collections.
    • We have added a number of tracker improvements, including:

      • Sampling options for larger clients.
      • Integration with Searchkit.

Bug fixesedit

  • Fixed a bug where connectors sync jobs could not update more than ~2GB of document volume. The limit is now ~2PB.
  • Fixed out of memory errors for Docker container deployments on 2 GB instances, including Elastic Cloud, ECE, and self-managed Docker images.
  • Fixed a bug where Workplace Search connector packages could not be connected if hosted at non-root URLs.
  • Fixed an issue in Workplace Search where SharePoint Online and OneDrive content sources had to be connected by an authenticated Azure Global Administrator, and could not use permissions granted by an administrator.
  • Fixed a bug where XML files could not be downloaded and processed by the Elastic web crawler, even if they were included in connector.crawler.content_extraction.mime_types.
  • Fixed a bug that caused editing field rules to always overwrite the first rule in the list when saved.
  • Fixed a bug where the connector scheduling UI displayed incorrect information when weekly intervals were selected.

Known issuesedit

  • The connector service will fail to sync when the connector tries to fetch more than 2,147,483,647 (2^31-1) documents from the data source. A workaround is to manually partition the data to be synced using multiple search indices.
  • The web crawler uses a non-deterministic method to determine thread pool size, which can lead to unexpected behavior. This can be worked around by overriding the crawler.workers.pool_size.limit value in the elasticsearch.yml file.
  • Python connectors that upgraded to version 8.8.0 from 8.7.1 will report document volumes in gigabytes (GB) instead of megabytes (MB), under-reporting true document volume by a factor of 1024.
  • Upgrading to 8.8 does not migrate MySQL sync rules.
  • Upgrading from a tech preview connector (8.7 or earlier) will cause the MySQL connector configuration to be invalid. Refer to the MySQL connector documentation for a workaround.
  • Users are not able to complete the configuration of the Postgres connector client because the UI throws a Connector has missing configuration fields: username error even when a username is specified. To workaround this issue, backup the .elastic-connectors index and execute the following request from Kibana’s Dev Tools:
POST .elastic-connectors/_update_by_query
{
    "script" : """
      def value = ctx._source.configuration.user;
      ctx._source.configuration.remove('user');
      ctx._source.configuration.username = value;
      """,
    "query" : {
        "term": { "service_type": "postgresql" }
    }
}

Refer to the Enterprise Search Known issues for a full list of known issues and workarounds.