Elasticsearch X-Pack version 6.2.0

New Features

Machine Learning
  • Added the ability to identify scheduled events and prevent anomaly detection during these periods. For more information, see Calendars and Scheduled Events.
Security
  • X-Pack security now supports user authentication using SAML Single Sign on. For more information, see SAML Authentication.
Watcher

Enhancements

Machine Learning
  • Increased tokenization flexibility for categorization. Now all Elasticsearch analyzer functionality is available, which opens up the possibility of sensibly categorizing non-English log messages. For more information, see Customizing the Categorization Analyzer.
  • Improved the sensitivity of the analysis to high variance data with lots of values near zero.
  • Improved the decay rate of the model memory by using a weighted moving average.
  • Machine learning indices created after upgrading to 6.2 have the auto_expand_replicas: 0-1 setting rather than a fixed setting of 1 replica. As a result, machine learning indices created after upgrading to 6.2 can have a green status on single node clusters. There is no impact in multi-node clusters.
  • Changed the credentials that are used by datafeeds. When X-Pack security is enabled, a datafeed stores the roles of the user who created or updated the datafeed at that time. This means that if those roles are updated, the datafeed subsequently runs with the new permissions that are associated with the roles. However, if the user’s roles are adjusted after creating or updating the datafeed then the datafeed continues to run with the permissions that are associated with the original roles. For more information, see Datafeeds.
  • Added a new scheduled forecast status, which indicates that the forecast has not started yet.
Monitoring
  • X-Pack monitoring indices (.monitoring) created after upgrading to 6.2 have the auto_expand_replicas: 0-1 setting rather than a fixed setting of 1 replica. As a result, monitoring indices created after upgrading to 6.2 can have a green status on single node clusters. There is no impact in multi-node clusters.
  • Added a cluster alert that triggers whenever a node is added, removed, or restarted.
Security
  • Added the ability to refresh tokens that were created by the token API. The API provides information about a refresh token, which you can use within 24 hours of its creation to extend the life of a token. For more information, see Token Management APIs.
  • Added principal and role information to access_granted, access_denied, run_as_granted, and run_as_denied audit events. For more information about these events, see Auditing Security Events.
  • Added audit event ignore policies, which are a way to tune the verbosity of an audit trail. These policies define rules for ignoring audit events that match specific attribute values. For more information, see Logfile Audit Events Ignore Policies.
  • Added a certificates API, which enables you to retrieve information about the X.509 certificates that are used to encrypt communications in your Elasticsearch cluster. For more information, see SSL Certificate API.
Watcher
  • Added the ability to set the index and doc_type dynamically in an index action. For more information, see Index Action.
  • Added a refresh index action attribute, which enables you to set the refresh policy of the write request. For more information, see Index Action.
  • Added support for actions in slack attachments, which enables you to add buttons that can be clicked in slack messages. For more information, see Slack Action.
  • Watcher indices (.watch* and triggered_watches) created after upgrading to 6.2 have the auto_expand_replicas: 0-1 setting rather than a fixed setting of 1 replica. As a result, Watcher indices created after upgrading to 6.2 can have a green status on single node clusters. There is no impact in multi-node clusters.

Bug Fixes

Machine Learning
  • Improved error reporting for crashes and resource problems on Linux.
  • Improved the detection of seasonal trends in bucket spans longer than 1 hour.
  • Updated the forecast API to wait for validation and return an error if the validation fails.
  • Set the actual bucket value to 0 in model plots for empty buckets for count and sum functions. The count and sum functions treat empty buckets as 0 rather than unknown for anomaly detection, so it was inconsistent not to do the same for model plots. This inconsistency resulted in problems plotting these buckets in Kibana.
Security
  • Updated the setup-passwords command to generate passwords with characters A-Z, a-z, and 0-9, so that they are safe to use in shell scripts. For more information about this command, see setup-passwords.
  • Improved the error messages that occur if the x-pack directory is missing when you run users commands.
  • Fixed the ordering of realms in a realm chain, which determines the order in which the realms are consulted. For more information, see How Authentication Works.
Watcher